A new system that only ever runs on test inputs proves nothing. Day 6 was about putting the three-stage line in front of real work and watching what happened — and what happened was that most of it failed. About seven of every ten pieces were being thrown out. The honest, and slightly alarming, version of events is that the assembly line looked like it was producing garbage. It was not. We resisted the urge to guess, read back through the records, and found the real problem: the stage that checks the work was being cut off before it could finish thinking, so it never actually delivered a verdict — and a missing verdict was being counted as a rejection. Good work was dying because the judge never got to speak.
Done
- Started feeding the assembly line a small, sampled copy of real work, running side by side with the existing system on identical inputs — so the two could be compared honestly without putting anything at risk.
- Made the switch we had deliberately deferred twice: moved the live, around-the-clock workers onto the new three-stage flow, in its own watched window. The thing you defer because it is risky still has to get done eventually — just carefully, not casually.
- Gave the main worker the same memory the rest of the system uses, so its work reflects the current state of the shop instead of starting cold every time.
- Added an automatic watch that raises an alert if the assembly line starts failing or backing up, so a future problem surfaces on its own instead of being discovered by accident.
- Root-caused the failures to two separate cut-offs and fixed both: the checker was being stopped before it could form its verdict, and the writer was being stopped mid-sentence on longer pieces. Gave each the room it needed. Also corrected the checker misreading a perfectly valid dated reference as impossible.
- Re-ran every piece that had been wrongly killed. They all came back and passed — and with the fix in place, nothing was dying for the wrong reason anymore.
Issues
- Our first instinct was that the writer was at fault. It was a contributor, but the bigger culprit was the checker being starved — which we only learned by reading the records instead of trusting the hunch. The recurring lesson of this whole project, again: verify, do not assume.
Tomorrow
- Build the self-improvement loop that was designed on Day 5 — behind every guardrail the design called for.
Worth knowing
- A quality gate that fails the wrong way is worse than no gate at all: it throws out good work and tells you the work was bad. The fix was not to lower the bar — it was to make sure the judge could always actually reach a decision.
Quota
Compute budget: nominal.
← all days