# Async Generators and for await
# Goal: Async Generators and for await
Read the chapter Async iteration and generators (opens new window) of JavaScript.info reproducing the examples and exercises.
To make an object iterable asynchronously:
- Use
Symbol.asyncIterator
instead ofSymbol.iterator
. - The
next()
method should return a promise (to be fulfilled with thenext
value).- The
async
keyword handles it, we can simply makeasync next()
.
- The
- To iterate over such an object, we should use a
for await (let item of iterable)
loop.- Note the
await
word.
- Note the
# Exercise: Write the fetchCommits
async generator
Create a folder async-iteration-and-generators
. Use this folder for your solutions to the exercises in the chapter Async iteration and generators (opens new window) of the JavaScript.info book.
Add inside it a file paginated-data.js (opens new window) client program:
import _ from 'lodash';
import { fetchCommits } from './fetch-commits.js';
(async () => {
let someRepos = ['torvalds/linux',
'ULL-MII-SYTWS-2324/ULL-MII-SYTWS-2324.github.io',
'javascript-tutorial/en.javascript.info',
'ULL-MII-SYTWS-2324/generators-marcos-barrios-lorenzo-alu0101056944']
let count = 0;
let repoName = _.sample(someRepos);
console.log(`repoName = ${repoName}`);
for await (const commit of fetchCommits(repoName)) {
if (!commit.author) console.log(commit.commit.author.name);
else console.log(commit?.author?.login || "no login known");
if (++count == 100) { // let's stop at 100 commits
break;
}
}
})();
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
Write the fetchCommits
async generator that yields the commits. Put it in the module async-iteration-and-generators/fetch-commits.js
.
You can find a solution and more details about the exercise in the section Real-life example: paginated data (opens new window) of the chapter Async iteration and generators (opens new window) of the JavaScript.info book
# The Link header
The following script gets the commits of a repo using the GitHub API. It only shows the headers:
➜ async-iteration-and-generators git:(main) cat get-commits.sh
#!/bin/bash
# GitHub CLI api
# https://cli.github.com/manual/gh_api
OWNER=torvalds
REPO=linux
if [ -z "$1" ]; then
echo "No owner provided, using defaults owner: ${OWNER} repo: ${REPO}"
else
OWNER=$1
if [ -z "$2" ]; then
echo "No repo provided, using default: linux"
else
REPO=$2
fi
fi
# Option --verbose Includes full HTTP request and response in the output
# Option --silent does not print the response body
gh api \
-H "Accept: application/vnd.github+json" \
-H "X-GitHub-Api-Version: 2022-11-28" \
/repos/${OWNER}/${REPO}/commits \
--include --silent
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
The excution of the script gives the following output:
➜ async-iteration-and-generators git:(main) ./get-commits.sh
No owner provided, using defaults owner: torvalds repo: linux
HTTP/2.0 200 OK
Access-Control-Allow-Origin: *
Access-Control-Expose-Headers: ETag, Link, Location, Retry-After, X-GitHub-OTP, X-RateLimit-Limit, X-RateLimit-Remaining, X-RateLimit-Used, X-RateLimit-Resource, X-RateLimit-Reset, X-OAuth-Scopes, X-Accepted-OAuth-Scopes, X-Poll-Interval, X-GitHub-Media-Type, X-GitHub-SSO, X-GitHub-Request-Id, Deprecation, Sunset
Cache-Control: private, max-age=60, s-maxage=60
Content-Security-Policy: default-src 'none'
Content-Type: application/json; charset=utf-8
Date: Wed, 08 Nov 2023 11:08:41 GMT
Etag: W/"cf1b48d1ef7459a0ee74ff2e86a4205b5d1c16945f763566fcc45a48b6d33720"
Last-Modified: Wed, 08 Nov 2023 01:16:23 GMT
Link: <https://api.github.com/repositories/2325298/commits?page=2>; rel="next", <https://api.github.com/repositories/2325298/commits?page=41087>; rel="last"
Referrer-Policy: origin-when-cross-origin, strict-origin-when-cross-origin
Server: GitHub.com
Strict-Transport-Security: max-age=31536000; includeSubdomains; preload
Vary: Accept, Authorization, Cookie, X-GitHub-OTP, Accept-Encoding, Accept, X-Requested-With
X-Accepted-Oauth-Scopes:
X-Content-Type-Options: nosniff
X-Frame-Options: deny
X-Github-Api-Version-Selected: 2022-11-28
X-Github-Media-Type: github.v3; format=json
X-Github-Request-Id: 3B33:6EFB:13ED65F5:142E2662:654B6C38
X-Oauth-Scopes: admin:enterprise, admin:gpg_key, admin:org, admin:org_hook, admin:public_key, admin:repo_hook, codespace, delete:packages, delete_repo, gist, notifications, project, repo, user, workflow, write:discussion, write:packages
X-Ratelimit-Limit: 5000
X-Ratelimit-Remaining: 4993
X-Ratelimit-Reset: 1699444433
X-Ratelimit-Resource: core
X-Ratelimit-Used: 7
X-Xss-Protection: 0
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
# Exercise: for-await-of in a first come first served order
If you use for-await-of on an array of promises, you iterate over it in the specified order, doesn't matter if the next promise in the given array is resolved before the previous one:
const sleep = time => new Promise(resolve => setTimeout(resolve, time));
(async function () {
const arr = [
sleep(2000).then(() => 'a'),
'x',
sleep(1000).then(() => 'b'),
'y',
sleep(3000).then(() => 'c'),
'z',
];
for await (const item of arr) {
console.log(item);
}
}());
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
Output:
➜ firstcomefirstserved git:(main) node examples/for-await-simple.js
a
x
b
y
c
z
2
3
4
5
6
7
But sometimes you want to process the results as soon as the promises yield them.
Write a Node.JS module frstcmfrstsvd
that exports an async generator that can be used with for-await-of and provides the results in a first come first served order:
import firstComeFirstServed from 'frstcmfrstsvd';
// See https://stackoverflow.com/questions/40920179/should-i-refrain-from-handling-promise-rejection-asynchronously
process.on('rejectionHandled', () => { });
process.on('unhandledRejection', error => {
console.log('unhandledRejection');
});
const sleep = time => new Promise(resolve => setTimeout(resolve, time));
const arr = [
sleep(2000).then(() => 'a'),
'x',
sleep(1000).then(() => 'b'),
'y',
sleep(3000).then(() => 'c'),
'z',
];
console.log(firstComeFirstServed);
(async () => {
for await (let item of firstComeFirstServed(arr)) {
console.log("item = ",item);
}
})()
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
Output:
➜ firstcomefirstserved git:(main) node examples/hello-frstcmfrstsvd.mjs
[AsyncGeneratorFunction: frstcmfrstsvd]
item = { value: 'x', index: 1, status: 'fulfilled' }
item = { value: 'y', index: 3, status: 'fulfilled' }
item = { value: 'z', index: 5, status: 'fulfilled' }
item = { value: 'b', index: 2, status: 'fulfilled' }
item = { value: 'a', index: 0, status: 'fulfilled' }
item = { value: 'c', index: 4, status: 'fulfilled' }
2
3
4
5
6
7
8
# Error Management Example
Here is an example of how has to behave when there are rejections:
import frstcmfrstsvd from 'frstcmfrstsvd';
// See https://stackoverflow.com/questions/40920179/should-i-refrain-from-handling-promise-rejection-asynchronously
process.on('rejectionHandled', () => { });
process.on('unhandledRejection', error => {
console.log('unhandledRejection');
});
const sleep = time =>
new Promise(resolve => setTimeout(resolve, time));
const arr = [
sleep(2000).then(() => 'a'),
'x',
sleep(1000).then(() => 'b'),
'y',
sleep(3000).then(() => { throw `Ohhh:\n` }),
'z',
];
(async () => {
try {
for await (let item of frstcmfrstsvd(arr)) {
console.log("item = ",item);
}
} catch(e) {
console.log('Catched!:\n', e);
}
})()
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
Gives as output:
➜ firstcomefirstserved git:(main) ✗ node examples/reject-frstcmfrstsvd.mjs
item = { value: 'x', index: 1, status: 'fulfilled' }
item = { value: 'y', index: 3, status: 'fulfilled' }
item = { value: 'z', index: 5, status: 'fulfilled' }
item = { value: 'b', index: 2, status: 'fulfilled' }
item = { value: 'a', index: 0, status: 'fulfilled' }
item = { reason: 'Ohhh:\n', index: 4, status: 'rejected' }
2
3
4
5
6
7
# Compare the Performance of your solution with the performance of allSettled
Write a program to compare the performance of your solution with the performance of Promise.allSettled (opens new window).
You can find the full code in the npm package frstcmfrstsvd (opens new window).
At first view, performance of Promise.allSettled (opens new window) seems to be a bit better:
> node examples/performance-reject-frstcmfrstsvd.mjs
frstcmfrstsvd: 323.104ms
allsettled: 317.319ms
> node examples/performance-reject-frstcmfrstsvd.mjs
frstcmfrstsvd: 327.142ms
allsettled: 315.415ms
> node examples/performance-reject-frstcmfrstsvd.mjs
frstcmfrstsvd: 322.753ms
allsettled: 318.955ms
> node examples/performance-reject-frstcmfrstsvd.mjs
frstcmfrstsvd: 325.562ms
allsettled: 317.375ms
> node examples/performance-reject-frstcmfrstsvd.mjs
frstcmfrstsvd: 322.25ms
allsettled: 318.09ms
2
3
4
5
6
7
8
9
10
11
12
13
14
15
See file examples/performance-reject-frstcmfrstsvd.mjs (opens new window)
# See
- This a repo with the source for a npm package crguezl/frstcmfrstsvd (opens new window). Don't look at it unless you are blocked. The repo ULL-MII-SYTWS/for-await-solution (opens new window) is a private repo with the solution to this lab.
- Stackoverflow question: Performance of Promise.all and for-await-of (opens new window)
- ULL-MII-SYTWS-2022/learning-async-iteration-and-generators (opens new window) (Private repo)
- campus-virtual/2021/learning/asyncjs-learning/learning-async-iteration-and-generators
- Chapter Async iteration and generators (opens new window)
- Chapter Iterables (opens new window)
- Chapter Generators (opens new window) of JavaScript.info
# ES6 Modules in Node.JS
- See the Node.js doc Modules: Packages (opens new window) for more details on the use of ECMA 6 Modules in Node.js.
- How Can I use en es6 Import in Node.JS (opens new window) Stackoverflow