shipping eval loops that fit in a lunch break

we kept losing the same bugs twice. an agent that worked on tuesday quietly broke friday after a prompt tweak nobody reviewed.

what we tried

a folder of golden tasks. each task is a short input, an expected shape of output, and a one-line “why this matters.” we run them with a single script before any prompt lands on main.

no dashboards yet. stdout and a markdown table are enough.

what worked

naming failures. when a case fails, the report says which fixture and which assertion — not “score dropped.” people fix what they can point at.

keeping the suite small. twelve cases beat two hundred that nobody runs.

what’s next

wire the harness into CI as a soft gate, then promote a few cases to hard fails once the noise settles.