For this discussion of iterators, let’s establish a baseline example. It’s made up, but realistic and common: A database method that returns all user orders. We’ll be experimenting with different function signatures, but in general, this is what we can imagine it will look like:

func (DB) Orders(ctx context.Context, userID string) ([]*Order, error)

And we would consume it with code something like this:

orders, err := db.Orders(ctx, userID)
if err != nil {
	return err
}
for _, order := range orders {
	/* Do something with each order */
}

If a user has only one past order, no problem. If they’re as adicted to Amazon as I am, they likely have hundreds or thousands of orders over the last decade, and returning them all could take a while, which is a great reason to prefer an interator over a slice of all orders since the dawn of time.

And one of Go’s “special” features is channels. So let’s look at improving this situation with channels. Here’s what the function might look like using a channel:

func (DB) Orders(ctx context.Context, userID string) (chan <- *Order, error)

This lets us fetch orders from the database, potentially as they are read from the database, one at a time, rather than loading them all into memory first. And if we decide we only need the first three results, we never even bother reading results 4 through… whatever. Win win!

Let’s see what it looks like to consume this function:

orders, err := db.Orders(ctx, userID)
if err != nil {
	return err
}
for order := range orders {
	/* Do something with each order */
}

This is pretty close to the slice version above. But there are a few subtle differences, from the consumer’s viewpoint (there are much bigger differences on the producer’s side, which we’ll get to later).

1. In the slice version, the entire operation is atomic, which means it either succeeds (and we get all orders), or it fails (and we get an error). With the channel version, we may encounter an error part way through the iteration, after having successfully read a number of results. This can be fine (and even ideal) in many situations. But it requires thought. Is it okay, in your situation, to process 3 *Order values, then encounter an error? Or should the first three results be discarded in that case?

2. Special consideration must be made to report such errors mid-iteration! There are a few approaches, and we’ll talk about them in more depth when discussing the producer’s side. But for now, a preview:

   • Ignore the error. Meh. Almost always a bad approach.

   • Include the error in the channel. e.g. replace chan <- *Order with chan <- struct{ Order; Err error }. Solves the problem, but it’s a bit ugly to use.

     results, err := db.Orders(ctx, userID)
     if err != nil {
     	return err
     }
     for _, result := range results {
     	if result.Err != nil {
     		return result.Err
     	}
     	order := result.Order
     	/* Do something with each order */
     }
     

   • Report the error after iteration. There are a few ways this could be done, but they all amount to the same thing: After you finish iterating, check if there was an error, and then handle it.

     orders, err := db.Orders(ctx, userID)
     if err != nil {
     	return err
     }
     for _, order := range orders {
     	/* Do something with each order */
     }
     if err := db.OrdersError(); err != nil {
     	return err
     }