Quick take
Goroutines are cheap. Debugging leaked goroutines isn’t. Every goroutine needs an exit path. Every channel needs a single closer. Bound your concurrency, propagate cancellation, and prefer boring control flow over clever tricks.
I write Go professionally and I contribute to the Go ecosystem. After building concurrent systems at Decloud, a large consumer platform, and a real-time messaging company, the patterns that survive production are remarkably consistent. The failures are consistent too: leaked goroutines, unbounded fan-out, data races on maps, and channels that block forever because someone forgot to close them.
This post is the concurrency playbook I wish I had when I started writing Go services. Code included.
Two Rules
Before any patterns, internalize these:
- Every goroutine has an exit path. If you can’t point to the line of code that causes a goroutine to return, you have a leak.
- Every channel has a single, obvious closer. The producer closes, never the consumer. If multiple producers share a channel, use a
sync.WaitGroupto close after the last one finishes.
Violate these and you’ll spend your Friday evening staring at pprof goroutine dumps.
Bounded Worker Pool
The most common pattern I use. A fixed number of workers pull jobs from a channel. Concurrency is capped. Backpressure is natural – if all workers are busy, the send on the jobs channel blocks.
func RunPool(ctx context.Context, jobs <-chan Job, maxWorkers int) <-chan Result {
results := make(chan Result)
var wg sync.WaitGroup
for i := 0; i < maxWorkers; i++ {
wg.Add(1)
go func() {
defer wg.Done()
for {
select {
case <-ctx.Done():
return
case job, ok := <-jobs:
if !ok {
return
}
results <- process(job)
}
}
}()
}
go func() {
wg.Wait()
close(results)
}()
return results
}
Key details: the pool owns the workers. The caller owns the jobs channel and closes it when done. The results channel is closed after all workers exit. Context cancellation gives you a clean shutdown. No goroutine leaks.
At a large consumer platform, we used this pattern for processing order events. 50 workers, bounded channel, context cancellation tied to the service shutdown signal. Boring and reliable.
Fan-Out / Fan-In
Distribute work across multiple goroutines, then merge results back into one stream. This is useful when individual items are independent and you want parallelism.
func FanOut(ctx context.Context, input <-chan Item, workers int) <-chan Result {
channels := make([]<-chan Result, workers)
for i := 0; i < workers; i++ {
channels[i] = worker(ctx, input)
}
return merge(ctx, channels...)
}
func worker(ctx context.Context, input <-chan Item) <-chan Result {
out := make(chan Result)
go func() {
defer close(out)
for item := range input {
select {
case <-ctx.Done():
return
case out <- transform(item):
}
}
}()
return out
}
func merge(ctx context.Context, channels ...<-chan Result) <-chan Result {
var wg sync.WaitGroup
merged := make(chan Result)
for _, ch := range channels {
wg.Add(1)
go func(c <-chan Result) {
defer wg.Done()
for val := range c {
select {
case <-ctx.Done():
return
case merged <- val:
}
}
}(ch)
}
go func() {
wg.Wait()
close(merged)
}()
return merged
}
The lifecycle is clear: each worker closes its output channel when the input is exhausted or the context is canceled. The merge function waits for all workers to finish, then closes the merged channel. One closer per channel. No ambiguity.
Results are unordered. If you need ordering, include a sequence number in the result and sort afterward. Don’t try to enforce ordering through the concurrency model – it defeats the purpose.
Pipeline
A pipeline chains stages together with channels. Each stage reads from its input, processes, and writes to its output. Backpressure propagates naturally: a slow stage slows everything upstream.
func pipeline(ctx context.Context, input <-chan RawData) <-chan Output {
validated := validate(ctx, input)
enriched := enrich(ctx, validated)
return format(ctx, enriched)
}
func validate(ctx context.Context, input <-chan RawData) <-chan ValidData {
out := make(chan ValidData)
go func() {
defer close(out)
for raw := range input {
if v, ok := raw.Validate(); ok {
select {
case <-ctx.Done():
return
case out <- v:
}
}
}
}()
return out
}
Each stage is small, testable, and has a clear exit condition. I used this at Decloud for our data ingestion pipeline – raw events came in, got validated, enriched with metadata, and written to storage. Each stage could be tested independently and swapped without touching the others.
Cancellation and Timeouts
Context propagation isn’t optional. Every function that starts a goroutine or makes a blocking call takes a context.Context as its first argument. Period.
func fetchWithTimeout(ctx context.Context, url string) ([]byte, error) {
ctx, cancel := context.WithTimeout(ctx, 3*time.Second)
defer cancel()
req, err := http.NewRequestWithContext(ctx, "GET", url, nil)
if err != nil {
return nil, err
}
resp, err := http.DefaultClient.Do(req)
if err != nil {
return nil, err
}
defer resp.Body.Close()
return io.ReadAll(resp.Body)
}
When the parent context is canceled – because the HTTP request was aborted, the service is shutting down, or the deadline expired – all child work stops. Fast. This is one of Go’s best features and I’m consistently surprised by how many codebases I’ve seen don’t use it properly.
Avoid hard-coded timeouts in library code. Accept them as parameters or derive them from the context. Hard-coded timeouts are the kind of thing that works fine in development and causes incidents in production.
Error Groups
The errgroup package from golang.org/x/sync is my go-to for concurrent work where any error should abort the group.
func fetchAll(ctx context.Context, urls []string) ([]Response, error) {
g, ctx := errgroup.WithContext(ctx)
responses := make([]Response, len(urls))
for i, url := range urls {
i, url := i, url
g.Go(func() error {
resp, err := fetchWithTimeout(ctx, url)
if err != nil {
return fmt.Errorf("fetch %s: %w", url, err)
}
responses[i] = resp
return nil
})
}
if err := g.Wait(); err != nil {
return nil, err
}
return responses, nil
}
First error cancels the derived context, which cancels all other goroutines in the group. Clean shutdown. No leaked work. The Wait call blocks until all goroutines finish and returns the first error.
For batch work where you want to collect all errors instead of aborting on the first, build a simple error collector with a mutex. But default to fail-fast – it’s usually what you want in request-serving code.
Semaphore Pattern
When you need a concurrency limit but a full worker pool is overkill, use a buffered channel as a semaphore:
func processWithLimit(ctx context.Context, items []Item, limit int) error {
sem := make(chan struct{}, limit)
g, ctx := errgroup.WithContext(ctx)
for _, item := range items {
item := item
g.Go(func() error {
select {
case sem <- struct{}{}:
case <-ctx.Done():
return ctx.Err()
}
defer func() { <-sem }()
return processItem(ctx, item)
})
}
return g.Wait()
}
Acquire a slot by sending to the buffered channel. Release by receiving. Simple, no external dependencies, and it composes well with errgroup.
Synchronization
Prefer ownership over sharing. Pass data through channels so only one goroutine owns it at a time. When shared state is unavoidable:
sync.Mutexfor protecting small critical sections. Keep the locked region tiny.sync.RWMutexwhen reads vastly outnumber writes.sync.Oncefor one-time initialization. Don’t reinvent this.sync.Mapalmost never. A regular map with a mutex is clearer and usually faster for most workloads.
Run go test -race on everything. The race detector is one of Go’s killer features. Use it in CI. No exceptions.
The Failure Checklist
Before shipping concurrent code, ask:
- Can every goroutine exit? What triggers the exit?
- Is every channel closed by exactly one goroutine?
- Are there any sends that could block forever?
- Is shared state protected? Did you run the race detector?
- Does cancellation propagate to all blocking operations?
If you can answer all five with confidence, the code is probably fine. If you can’t answer even one, fix it before it ships.
The Philosophy
Go concurrency is powerful because the primitives are simple. Goroutines, channels, select, context. The complexity comes from composition, and composition goes wrong when lifecycles are unclear.
Keep lifecycles explicit. Keep concurrency bounded. Prefer boring patterns over clever ones. The service that runs for months without a goroutine leak is worth more than the service with the most elegant pipeline design.