Workshop, 04.3, THE WORKSHOP BENCH
darker
Turn the lights on only when needed
Can you believe it when it says it’s done?
A workshop experiment, not a product. A harness around AI coding agents that checks their work with real tests before any AI review, and hands control back to a person when it has to.
Status: Workshop
Overview
Why it is in the cavern
darker is not a Quiet Publish product, and we don’t want it mistaken for one. It is here because it is where we worked hardest on a question every tool in this world depends on: when software says the work is done, can you believe it?
The name comes from lights-out factories, which run in the dark and turn the lights on only when a person is needed. The goal was never maximum autonomy; it was the least wasted attention.
What it has taught us so far
The most useful moments were the ones where it told the truth. In June we gave it a real backlog ticket in Mardi Gras, our own terminal tool: put the orchestrator code behind a driver interface. darker found that the interface already existed on main, added after the ticket was filed; the ticket had simply never been closed. It made no changes.
About a week later we gave it a rounding bug in go-humanize, a widely used Go library: Bytes(999999) read “1000 kB” instead of “1.0 MB”. It wrote a failing test, then a fix, and corrected the library’s own tests, which had recorded the wrong output as right. When it ran out of turns before its checks were done, it stopped short of claiming a pass; a follow-up run finished them. The whole run cost $2.94, and the fix stayed on our machine: nothing was sent upstream.
It was also honest about itself. A re-grade in June showed that it had been checking whether its work was consistent and calling that correct, and much of the summer went into closing that gap.
Where it stands
The small, friendly jobs it was tested on went well. Long, sustained work across many steps has not yet run cleanly from end to end, and that is the test that matters. It is what we are working on now.
PRINCIPLES AT WORK
The problem
AI agents write code readily. They are much less reliable judges of their own work: a “pass” can mean the pieces compile while the whole is broken. And a person’s attention is the scarcest thing in the loop.
OUR APPROACH
Sort each task by how hard it is, so small fixes stay small. Run real type checks and tests before any AI review. Keep the agent that writes the code from grading it. Write every step down, and turn the lights on for a person only when one is needed.
What it does
- Sorts a task into a tier, from a short path for small fixes to a full loop of planning, building, checking and retrying. The tiers are named for light, from “pitch black” to “lights on”.
- Detects a project’s type checkers, linters and test runners, and runs them before any model looks at the work.
- Has a model write the critique while code computes the score, and penalizes hedging language.
- Keeps the judges read-only through tool permissions, not instructions, so the evaluator cannot edit code even when everything else is allowed.
- Offers a test-first lane, where one agent writes failing tests and another makes them pass.
- Leaves a file record of every step, read live by a terminal control room and a local web cockpit.
- Installs safety hooks that block destructive commands in the projects it works on.
ScreenshotFictional dataAttempt drill-down: gates, per-criterion scores and a blocking issue. 
ScreenshotFictional dataSprint list and detail: criteria, budget, attempts, delivery. 
ScreenshotFictional dataReplay viewer: a sprint's whole run as a scrubbable timeline.
How it’s built
- A Claude Code plugin of agents and skills
- Bash and Python for the harness
- JSON Schema for every record it writes
- Go for the terminal control room
- Next.js for the local web cockpit
- Claude models: Sonnet for most of its agents, and Opus for the one that checks whether a multi-sprint run still holds together
DECISIONS, AND WHY
The analysis from the model, the score from code
Letting a model grade its own findings pulled every score toward the middle. Scoring the structured findings in code made the grades sharper and repeatable.
Files are the handoff, not the conversation
Each phase reads the records the last one wrote, never the chat history. The harness, the control room and the cockpit all share the same written contract.
Scaffolding meant to be removed
Some guards only make up for what today’s models can’t do. A ledger tracks them, so each can be deleted when it is no longer needed.
Timeline
The first version. Within days, darker built part of its own control room.
Work orders, intake and headless runs. Its first run on code it had never seen: a security audit of Mardi Gras, turned into four work orders that each passed on the first attempt.
A re-grade finds darker measuring its own coherence and calling it correctness. The honesty work that follows, and the local web cockpit.
Real repositories. A stale Mardi Gras ticket it declined to redo, and a rounding bug in go-humanize.
More real work, and calibration.
Multi-sprint campaigns, none yet clean from end to end. Active development continues.
Status
Status: WorkshopWorkshop. Experiments, shown for what they teach us.
A workshop experiment in active development. Our own notes call it promising and unproven. Its repositories are coming to GitHub; there is no link until they are public, and it is not offered as a product.
Credits
- Designed and built by one of the studio’s founders, largely with the harness running on itself.