/blog

8 min read

A small Go worker pool that shuts down cleanly

Forty lines, a context and a WaitGroup. A worker pool that drains its jobs on SIGTERM instead of dropping them.

Most worker pools work fine until the deploy. Then the process gets SIGTERM, the jobs in flight vanish, and somebody reruns them by hand the next morning.

The whole thing

func Run(ctx context.Context, jobs <-chan Job, workers int) {
    var wg sync.WaitGroup
    for i := 0; i < workers; i++ {
        wg.Add(1)
        go func() {
            defer wg.Done()
            for job := range jobs {
                job.Do(context.WithoutCancel(ctx))
            }
        }()
    }
    <-ctx.Done()
    wg.Wait()
}

Why it holds up

  • Workers range over the channel, so they stop when the producer closes it, not when the context is cancelled.
  • Each job gets a context that isn't cancelled mid-flight, so a half-written row doesn't happen.
  • wg.Wait() is the drain. Give your orchestrator a grace period longer than your slowest job.

One more thing

Close the jobs channel from the producer side only. Closing it from a worker is a panic waiting for a busy day.

$ subscribe

Want the next one?

Leave your email and new posts from Stack & Heap come to your inbox.

$ subscribe --email

Get new posts by email

Leave your address and new posts from Stack & Heap come to your inbox.

Rather write back? hello@stackandheap.example