Day 73: the split arm reads its own boot grades and holds them to what served
Date: 2026-10-02 · Week 14 · Phase 5 Benchmark and optimize
What I added today
Yesterday’s “Tomorrow” was a report. Day 71’s probe and Day 72’s timing both land in /health as split_probe and split_timing, and the acceptance harness read neither. run_arm lifted the arena off the payload (Day 68) and the read counters (Day 60) and left behind the two sections that say whether the read is right on this card and what it bought. Now the split arm carries them, prints them under its row, and is held to them.
graphbench.py. Three additions:
SplitBoot, the record: the probe and the timing as published, plain dicts, withgraded,backend,worst_error,predictedandmeasuredread off them. Arender()that prints one line (“graded on tlsim: read 3.0e-07 (bound 1e-04); predicted 4.00x, measured 0.52x (…), did not pay on the 4-row timed case”) and arow()with five CSV columns that has the same keys whether anything was graded or not.split_boot_from_health(payload), the lift. It copies, and returns an empty record for the other two reads, the same wayworkspace_from_healthdoes.check_arm_split_graded(arm), the gate. A split arm has to carry a probe inside its bound and a timing on the probe’s backend, and that backend has to be the one the read counters say served the crowd.
ArmReport gains split_boot, run_arm fills it from the first reading, and ArmReport.render() grows a second line when the arm was graded. Streamed and rectangle arms still print one line.
tests/test_split_report.py. 22 new tests. They cover the lift (including the real SplitProbe/SplitTiming dataclasses through as_dict and JSON, and a real split boot’s boot_info on the host), the render, the columns, and every refusal. The Day 68 three-arm socket test in tests/test_reads.py now also runs the new gate on its split arm and checks that the other two arms graded nothing. Suite 2310 green (34 GPU-gated skips), ruff clean.
Why it matters
The boot record never sees a decode step. build_app holds the probe against the reference, and check_boot_info holds the timing against the probe. Both look at one record written before the door opened. The read counters taken around a real crowd name the backend that actually answered, and the harness is the only party holding both. A grade on tlsim over a crowd served on triton graded a kernel that answered nobody, and the kernel that did answer was never graded. Today that’s a named refusal rather than a green run.
The number lands next to the throughput it bought. The first card run of the three arms will put “graded on triton, predicted 4x, measured Nx” on the line under the split arm’s ITL. Until now those lived in two places: the latency in the bench output and the timing in a JSON blob nobody read.
The verdict is printed, not gated. Same reason as Day 72: the timed case is four rows and a serving batch is not. The gate checks that the grade is about the read that served. It never checks whether the split paid.
What I learned
- An empty backend isn’t a mismatch. A window with no decode reads has
backend == "", and comparing it to the probe’s “tlsim” would fail every idle arm with a message about two processes. So an empty window is checked first and refused asMeasurementUnsound: the run’s fault, not the server’s, which is whatcheck_arm_splitalready calls it. - Keep the CSV keys fixed.
csv.DictWritertakes its header from the first row. If an emptyrow()returned{}, a sweep that writes a split arm after a streamed one would crash on the second row, or quietly drop the columns the other way round. Same keys every time,Nonewhere nothing was graded. - Test the fixture against the real producer. The test payload is hand-written, and a hand-written payload drifts. One test builds the real
SplitProbeandSplitTiming, sends them throughas_dictand JSON, and asserts they come out equal to the fixture. Another lifts a real host split boot’sboot_info. Ifas_dictrenames a key, the unit tests fail instead of passing against a shape the server no longer sends. - Under the row, not in it. The boot grades are fixed for the life of the process and a row is one offered load, so a sweep prints the same second line under every row. That repetition is the honest picture: one boot, many loads.
Diagram
split-arm-report.png. Top left: one /health reading, three sections, into ArmReport.split_boot and the gate. Top right: what the split arm’s row prints now. Bottom left: the gate’s refusals in order. Bottom right: which check can see which mismatch, and why only the harness sees the last one.
Tomorrow
graphbench.py the script still runs two arms, graphs and eager, and neither one splits. The three-arm run only exists inside tests/test_reads.py. The next step is a --split-read arm in the script: a third server with bucket_decode on, prompts past the merge floor, check_arm_split and check_arm_split_graded in its claims list, and the SplitBoot.row() columns in its table and CSV.
The hardware caveat hasn’t changed: graphbench.py --weights ./weights --device cuda --rates 1,2,4,8 with prompts past 514 tokens is still the first run to book.
Post angle
Day 73 of building an LLM inference engine from scratch. My split server grades its attention kernel at boot and times it. Today the benchmark harness reads both off /health and prints them under the split arm’s latency row. It also checks that the backend graded at boot is the backend that served the crowd, because the boot record never sees a decode step and only the harness holds both. vLLM and SGLang ship flash-decoding; I’m learning to make every number about the kernel that answered. 2310 green.