Limiting concurrency in Go

A buffered channel used as a token bucket caps how many run at once.

func fetchAll(urls []string, limit int) []string {
    tokens := make(chan struct{}, limit)
    results := make([]string, len(urls))
    var wg sync.WaitGroup

    for i, url := range urls {
        wg.Go(func() {
            tokens <- struct{}{}        // acquire
            defer func() { <-tokens }() // release
            results[i] = strings.ToUpper(url)
        })
    }

    wg.Wait()
    return results
}

How it works

1. Acquiring is a send; releasing is a receive.
2. The buffer size is the limit.
3. This is the standard Go idiom, with no extra dependency.

Keywords and builtins used here

• chan
• defer
• for
• func
• int
• len
• make
• range
• return
• string
• struct
• var

The run, in numbers

Lines

16

Characters to type

334

Tokens

101

Three-star pace

110 tpm

At the three-star pace of 110 tokens a minute, this run takes about 55 seconds.

Type this snippet

Step 2 of 5 in Patterns, step 20 of 25 in Concurrency.

← Previous Next →

Limiting concurrency in other languages

• JavaScript
• Python