Day 87: a booking’s sweeps, joined by load

Date: 2026-10-19 · Week 14 · Phase 5 Benchmark and optimize

What I added today

Day 86 made --table take a booking’s worth of CSVs and print each table under its path. Nothing compared them. The question a reader has about five sweeps is one question, whether 2 rps moved between the first day and the last, and answering it meant scrolling between tables and lining the rows up by eye.

With more than one sweep, the last block joins them by load:

itl_p50_speedup, joined by load:
      load  sweep 1  sweep 2
----------------------------
     1 rps    1.25x        -
     2 rps    1.10x    1.50x
     4 rps        -    0.90x
sweep 1: d74.csv
sweep 2: d79.csv
  • Loads come in the order they first appear, walking the sweeps in the order given. Not sorted: load is a label, and "16 rps" sorts before "2 rps" as text.
  • A gap is -. Not 0.00x, which is a number, and not a missing line, which would shift every column under it.
  • A rerun or ? row prints its mark, not its number. Day 84 decided those numbers are not a comparison, and a comparison table is exactly where they would pass for one. lost prints its number: it is a result that says no.
  • The columns are sweep 1, sweep 2, …, with the paths listed under the block. A path is too wide for a heading, and two days both write sweep.csv.
  • One sweep prints no block. Day 83’s output for one file is unchanged.
  • A load twice in one sweep is refused before anything prints. The cell has room for one row, and keeping either hides the other.
  • --strict still decides last, after the block, so a failing booking shows its join too.

join_by_load is pure (the sweeps in, (load, [row or None, ...]) out) and print_join prints it. reprint_sweeps joins before printing any table, so the duplicate-load refusal goes through the same “nothing on stdout” path as Day 86’s.

tests/test_table_compare.py has 8 tests: the join’s order and gaps, the duplicate refusal (pure and through the CLI with empty stdout), the block’s exact lines, the mark in place of the number, no block for one sweep, the block coming after the last table, and --strict failing after the block. Day 86’s two-sweep test asserted the output was exactly 9 lines; it now asserts the join starts on line 10.

Suite 2520 green (34 GPU-gated skips), ruff clean.

Why it matters

The comparison is the deliverable of a booking. Each sweep on its own says whether graphs beat eager on that day. The reason to book the card five times is the trend across them, and that only exists once the rows share a line.

A join is where bad numbers spread. Inside its own table a rerun row carries its rerun mark next to it. Pulled out into a column of speedups, the mark would be gone and 1.25x would read like every other cell. Printing the mark in the cell keeps the Day 84 verdict attached to the number it was about.

What I learned

  1. Sorting by load was my first instinct, and it was wrong. A sorted join put 16 rps before 2 rps. Parsing the number back out of the label would fix it for rps and break on burst. First-seen order needs no parse and matches the order the sweep was run in, which is the order the reader thinks in.
  2. An exact-length assertion is a promise about the future. Day 86’s len(lines) == 9 was right for Day 86 and failed the moment output grew a block. That is the test doing its job, but the rewrite asserts where the new block starts rather than a new total, so the next block does not break it again for nothing.
  3. Validate a join before printing what it joins. A duplicate load is only discovered by joining, so the join runs before the first table: the same read-all-then-print rule as Day 86, applied to a check that is not a parse.

Diagram

table-join.png. Left: the two numbers that answer the question sitting a table apart, and the three rules for a cell. Right: the joined block with a gap, a lost row, and a rerun and ? printed as marks.

Tomorrow

The join shows one metric. The p99 is where graphs earn their keep under load, and a reader comparing days wants it next to the p50. Next: a --join option choosing the column (itl_p50_speedup default, itl_p99_speedup, split_itl_p50_speedup), refusing a column some sweep does not have, and the split column only where every sweep was a split boot.

The card run still comes first when hardware is booked.

Post angle

Day 87 of building an LLM inference engine from scratch. My benchmark printed five sweeps as five tables, and the one question I had (did 2 rps get faster between day 74 and day 79?) needed me to line rows up by eye. Now the last block joins them by load, one speedup column per sweep, - for a gap, and a rerun prints as rerun, not as a number. 2520 green.