For statements with range clause

…

6. For a function f, the iteration proceeds by calling f with a new, synthesized yield function as its argument.

Let’s talk about yield today.

If I’m honest, this was probably the scariest part of these new function iterators, before I started actually using them.

It looks an awful lot like a new keyword, with weird usage. I know I’m not alone in thinking this. 😊

But it’s not a new keyword. It’s actually a callback function. One that’s generated “synthetically” by the compiler.

The purpose of this special yield callback function is both for our custom iterator function to return the next value(s) to the range loop, and to inform the iterator function if it should stop processing, in case of early termination of the loop via break or return, for example.

The yield function has one of three possible signatures (closely related to the function signatures we discussed Friday):

• func() bool
• func(V) bool
• func(K, V) bool

Where K and V correspond to the key and value types in the for loop:

for k, v := range myFancyIterator {

If yield is called before f returns, the arguments to yield become the iteration values for executing the loop body once.

This would be the “normal” operation. The “happy-path”, if you will. You’ll typically expect to call yield before f returns. This is how you return values!

func myFancyIterator(yield func(k, v string) bool) {
	yield("key 1", "value 1")
}

Of course, the whole point here is to operate in a loop. We want this to keep happening potentially many times.

After each successive loop iteration, yield returns true and may be called again to continue the loop.

func myFancyIterator(yield func(k, v string) bool) {
	i := 1
	for {
		yield("key "+strconv.Itoa(i), "value " + strconv.Itoa(i))
		i++
	}
}

There we go. Now we have a proper loop!

Well, sort of.

We’re still ignoring the return value from yield. For a simple loop like this, the consequences of ignoring that return value would be minimal, if it weren’t for the fact that it will really upset the Go runtime, and cause a panic.

But in more complex situations, where you’re maybe iterating over results read from a file, a database connection, or an API, when yield returns false, it’s similar to calling Close() on another object. That is, it’s your signal to close files or connections, and generally clean up.

So let’s handle that:

func myFancyIterator(yield func(k, v string) bool) {
	i := 1
	for {
		if !yield("key "+strconv.Itoa(i), "value " + strconv.Itoa(i)) {
			return
		}
		i++
	}
}

Ah nice. No more panic!

But that’s an infinite iterator. It goes forever until the loop terminates via break or return. What about iterating over finite datasets?

It was subtle and quick, but we already read it: “… yield returns true and may be called again to continue the loop.”

In other words, it is the iterator’s calling of yield that continues the loop.

So if the iterator itself is ready to terminate the loop, all it has to do is not call yield.

func myFancyIterator(yield func(k, v string) bool) {
	i := 1
	for {
		if !yield("key "+strconv.Itoa(i), "value " + strconv.Itoa(i)) {
			return
		}
		i++
		if i > 5 {
			return
		}
	}
}

Now the loop will terminate on its own after i grows beyond 5.

As long as the loop body does not terminate, the “range” clause will continue to generate iteration values this way for each yield call until f returns. If the loop body terminates (such as by a break statement), yield returns false and must not be called again.

We covered those bits above already.

Is it making sense?

I know that these new iterators confuse many folks. Please let me know if you’re experiencing confusion!

Quotes from The Go Programming Language Specification Language version go1.23 (June 13, 2024)