Skip to content

Play

On call. Find what broke before the pager does it again.

Every failure here is something I fixed in production or wrote about on this site. The alerts are rebuilt from memory and my notes, the causes and the fixes are real. Same kind of evidence I had: the curl, the kubectl output, the terraform plan. Each shift deals 8 of the 12, so the next one is different.

pager · 02:44

You are on call. 8 alerts are coming.

  • 01Read the alert and the evidence.
  • 02Pick what actually broke. 25 seconds each.
  • 03Stuck? Dig deeper for one more clue, it costs 40 points.
  • 04Fast answers and streaks score more.

Keys: 1 to 4 to answer, H to dig deeper, Enter for the next alert.

Why a game

Reading about incidents does not stick.

Guessing wrong with the clock running does. Most of these took me longer than 25 seconds the first time, some took a whole night. If you run production, a few will feel familiar. If you hire for it, ask your candidates these.

  • 01Edge1 alert
  • 02Load balancer2 alerts
  • 03Cluster2 alerts
  • 04App3 alerts
  • 05Data2 alerts
  • 06Machine2 alerts