I’ve been away for a while. Last week I spoke at GoWest 2025, and have been just generally busy. But now I’m ready to pick up on the topic I started nearly two weeks ago: Drawbacks of channel-based iterators!
In addition to the issue of error handling with a channel-based iterator, there’s the potentially stickier issue of how to abort iterating early.
To illustrate, let’s look at some code that consumes our grep iterator, but stops processing after the first result:
result, err := grep(file, pattern)
if err != nil {
return err
}
for match := range result.Matches() {
/* do something with first result */
break; // Stop iterating
}
What becomes of that channel returned by result.Matches()?
Well, if it’s an unbuffered channel (as in the implementation from last time), then the goroutine that processes results will get stuck. Forever!
go func() {
defer close(matches)
scanner := bufio.NewScanner(r)
var matches []string
for scanner.Scan() {
line := scanner.Text()
if re.MatchString(line) {
result.matches <- line // <--- This line waits for a read, which never occurs
}
}
if err := scanner.Err(); err != nil {
result.err = err
}
}()
One might think of using a buffered channel to solve this problem, but that largely defeats the purpose of using an iterator in the first place, as it means reading a bunch of results into memory that may never be used. And it isn’t guaranteed to solve the problem anyway, since you have no way ahead of time to know how big to make your buffer.
So where does that leave us?
Well, in short, it leaves us with the conclusion that channel-based iterators aren’t ideal. But if you really want to make them work, you could try to devise a way to clean up the iterator early, when needed. What options do we have for this? I won’t go into detail, because none of these are particularly good options. But for the sake of completeness:
- Similar to the
Err()method proposed for error handling, we could add aClose()method that closes the channel and does any clean up. This relies on the consumer of the iterator to remember to call this method, though. Something I frequently see overlooked in similar code. - Let the consumer close the channel. This means you have to expose the read-only channel (because only write channels can be closed).
In either of these cases, you need to arrange for writes to a closed channel to fail gracefully, which is a bit involved.
This leads me to the only solution I would realistically ever consider: Using context-based cancelation. You could arrange for your iterator to clean up whenever the context is canceled:
func grep(ctx context.Context, r io.Reader, pattern string) (*GrepResult, error) {
This still puts a heavier burden on the caller, as they must set up a context that can be canceled early, and then cancel it:
ctx, cancel := context.WithCancel(context.Background())
defer cancel()
result, err := grep(file, pattern)
if err != nil {
return err
}
for match := range result.Matches() {
/* do something with first result */
cancel() // instruct iterator to clean up
break; // Stop iterating
}
In short, none of these solutions is very graceful, and each one invites the consumer of the iterator to forget the cleanup operation.
Conclusion: channel-based iterators are not the best option in the majority of cases.