/blog

7 min read

Reading a flame graph without guessing

Width is time, height is depth, colour is usually noise. A short guide to finding the slow part on the first try.

Flame graphs look busy, and that makes people read them like a picture. Read them like a table instead: every box is a function, and its width is the share of samples it was on the stack.

Three rules

  1. Width is time. A wide box is where the samples were. Order left to right is alphabetical, not chronological.
  2. Look for plateaus. A wide box with nothing on top is time spent in that function itself.
  3. Ignore colour unless your tool says what it encodes.

Capture one that's worth reading

Profile the slow request under real load, for long enough to collect a few thousand samples:

perf record -F 99 -g -p $(pgrep -n api-server) -- sleep 30
perf script | stackcollapse-perf.pl | flamegraph.pl > api.svg

Then search the SVG for the function you suspect. If json.Marshal covers 18% of the width, that is your next commit, whatever the code review said.

$ subscribe

Want the next one?

Leave your email and new posts from Stack & Heap come to your inbox.

$ subscribe --email

Get new posts by email

Leave your address and new posts from Stack & Heap come to your inbox.

Rather write back? hello@stackandheap.example