Iterators and Generators

Most code builds a whole array and then loops over it. That's fine until the list is a million rows, or comes from an API one page at a time, or never ends. Iterators let you hand out values one at a time, only when the loop asks for the next one. Generators are the easy way to write them.

What for...of actually does

for...of doesn't know about arrays. It asks the object for an iterator (the Symbol.iterator method), then calls next() until it gets done: true:

const sizes = ["S", "M", "L"];
const iterator = sizes[Symbol.iterator]();
console.log(iterator.next());
console.log(iterator.next());
console.log(iterator.next());
console.log(iterator.next());

{ value: 'S', done: false }
{ value: 'M', done: false }
{ value: 'L', done: false }
{ value: undefined, done: true }

Arrays, strings, Map, Set and many Node objects all have this method. Spread ([...x]), destructuring and Array.from use the same protocol.

Generators

Writing next() by hand is tedious. A generator function (function*) does it for you. Each yield hands out one value and pauses the function until the next value is asked for:

function* orderNumbers(start) {
  let n = start;
  while (true) {
    console.log("  making", n);
    yield `ORD-${n++}`;
  }
}

for (const number of orderNumbers(1040)) {
  console.log(number);
  if (number === "ORD-1042") break;
}

  making 1040
ORD-1040
  making 1041
ORD-1041
  making 1042
ORD-1042

while (true) looks like an infinite loop, and it would be in a normal function. Here it's fine, because the generator only runs when the loop asks for a value. When the loop breaks, the generator stops. Nothing extra is computed.

Lazy pipelines

Because nothing runs until asked, you can chain steps without building arrays in between:

function* take(items, count) {
  if (count <= 0) return;
  for (const item of items) {
    yield item;
    if (--count === 0) return;
  }
}

function* filter(items, test) {
  for (const item of items) if (test(item)) yield item;
}

function* naturalNumbers() {
  let n = 1;
  while (true) yield n++;
}

const firstFiveEven = take(filter(naturalNumbers(), n => n % 2 === 0), 5);
console.log([...firstFiveEven]);

[ 2, 4, 6, 8, 10 ]

Recent versions of Node (22 and later) and current browsers also have iterator helpers built in, so you can write naturalNumbers().filter(n => n % 2 === 0).take(5).toArray() without your own take and filter. Check your target runtime before you use them.

Async generators: paging through an API

APIs rarely return everything at once. They give you page 1 and a way to ask for page 2. An async function* can await each page and yield each record, and the caller loops with for await:

const pages = {
  1: { items: ["A", "B"], next: 2 },
  2: { items: ["C", "D"], next: 3 },
  3: { items: ["E"], next: null },
};

async function fetchPage(page) {
  console.log(`  fetching page ${page}`);
  await new Promise(resolve => setTimeout(resolve, 10));
  return pages[page];
}

async function* allItems() {
  let page = 1;
  while (page) {
    const data = await fetchPage(page);
    yield* data.items;
    page = data.next;
  }
}

for await (const item of allItems()) {
  console.log(item);
  if (item === "C") break;
}

  fetching page 1
A
B
  fetching page 2
C

The loop stopped at "C", so page 3 was never fetched. The calling code doesn't know pages exist. It just reads items. yield* hands out every item of an array (or any iterable) one by one.

When not to bother

If the data is small and already in memory, use an array. Arrays are easier to debug, you can look at them in one console.log, and you can loop over them twice. Generators earn their place when the data is big, slow to get, or endless.

Common mistakes

• Looping over the same generator twice and getting nothing the second time.
• Calling [...generator] on an endless generator. It never finishes.
• Using for...of instead of for await...of on an async generator. You get a TypeError, because it isn't sync iterable.
• Building a full array "to be safe" before passing it to code that only needed to loop once.

What happens if you skip this

You'll load every page of an API into one array before doing anything. It works in testing with 50 records. In production with 400,000, it runs out of memory or takes so long the request times out.

In the paging example, how many pages would be fetched if the loop broke at "A"? Why not zero, and why not all three?