The Dark Factory Is Dead: What Cole Medin Builds Instead

Every serious workflow for coding agents eventually hits the same question: how much autonomy can you hand over before quality collapses? Cole Medin has spent much of this year chasing the most radical answer, the AI dark factory from Dan Shapiro's five-level model, an orchestrator that turns a single vision document into shipped software. His new video closes the experiment with an unusually honest verdict: the dark factory does not work, and the reason has little to do with today's model quality. As models improve at writing code, they barely improve at reading human intent, and that gap decides everything. What has grown in its place is not a retreat to manual coding but a human-supervised AI software factory that runs hundreds of executions per week: planning stays outside, a release is the unit of work, an issue goes in, a pull request comes out, validation sits in the core loop, and a manager agent watches over the whole thing. This piece reconstructs both halves of his argument from the video and the underlying sources.

The reference frame comes from Dan Shapiro's blog post on the five levels of AI-assisted programming, modeled on the levels of driving automation. Simon Willison highlighted the piece in January 2026 and preserved Shapiro's sharpest phrasing: at level 5 it is no longer a car, it is a black box that turns specs into software, dark because humans are neither needed nor welcome there. Medin places everyday work at level three, occasionally level four; level five is the factory without a steering wheel, where you supply a destination and nothing else. That was the dream he chased publicly through videos and live streams. The interesting part, he says, was never the running factory itself, which is boring to watch while agents churn through tickets. The real engineering effort went into the harness: the processes, tools and rules that decide, before any agent starts, what the system is allowed to do and in which order.

Medin's core argument about why the dark factory fails is a bottleneck that model progress barely touches. Language models get measurably better at building software with every generation, but not better at understanding what people actually want. Feed the factory one high-level document and it has to fill the gaps between wish and word on its own, which is precisely where the errors pile up. Humans never communicate their features, their maintenance rules or their edge cases perfectly, and a system built to act on that imperfect surface will act on the wrong things. His sharpest line lands on the fantasy of mind reading: no matter how powerful the models become, they will never simply read our minds. He also rejects the counterposition that future models will hold better intent than their operators and quietly take over the roadmap. Customers, business context and codebase stewardship are knowledge the human carries, and that keeps humans in the driver's seat by default.

The second argument is corroborated from outside. Medin points to a widely circulated talk by Dex Horthy on why software factories fail, whose central claim is codebase maintainability: agents sprawl, they grow structures that a human later struggles to read, and they do not maintain a codebase the way the people who will live in it do. According to Medin, Horthy ran the same experiment in reverse, starting with a dark factory and then dialing it back until more human-in-the-loop made it realistic. Two independent experiments arriving at the same fault line carries weight: full human removal breaks not at the model but at intent and maintainability, and both remain human work no matter how the next model benchmarks.

The designed alternative is the AI software factory, and the difference sits in the scope of work the system may own. A dark factory receives the entire application as a high-level document, splits it into issues and runs lights-out until production. A software factory receives individual releases: the human plans the way they normally would, produces a PRD or architecture document, and hands that in as input. The factory still splits tickets and still rips through them, but the split happens inside a frame the human defined, not by an agent guessing intent. The core loop is unchanged from the dark factory, the issue is the input and the pull request is the output, and multiple labs work through problems in parallel once dependencies are mapped. Medin's point is that reliability does not live in the loop but in everything before and after it, and that is where the human sits.

The loop itself shows where the engineering went. Each round picks a single issue and applies a claim label so parallel labs cannot pick the same ticket. A triage decision follows upfront: simple work takes a faster build path, complex work gets iteration phases. Build, validation and code review are separated. Validation receives tools to operate the application like a real user, clicking buttons and typing text, which replaces a chunk of manual testing. Review then asks whether the code is simple, maintainable and correctly implemented, analysis that only makes sense once the app demonstrably works. Failures loop back to the implementation workflow with a maximum retry count so the factory cannot sink into an endless cycle. A pull request may merge into a staging-bound branch only when CI is green and the reviewed code passes. Around the loop sit guardrails: core context files such as mission.md and engineering.md record out-of-scope wishes and operating guidance, and review enforces that the factory never edits them, it must not be able to rewrite its own permissions and behavior.

The newest layer is supervision: a manager agent runs every two to three hours and monitors the success of all rounds together, because individual labs do not know when dependencies shift or when a pull request is truly stuck. When an agent exhausts its retries, the manager re-aligns problems or takes over and closes a workflow, and when genuine human intervention is required, Medin gets notified, often working the fix together with a Claude Code agent as a second brain. The staging step is deliberately human again: he tests the release himself, files anything off as new repo issues, and the factory picks those up automatically like the original epic split, so four or five feedback issues can be cleared and re-tested in staging before anything moves to production. The dark factory's fatal shortcut was exactly this step: it sailed straight into production. Token efficiency is a design principle too: orchestration nodes run different models and providers, the large reasoners like Opus 5.5 or GPT-6 Astra for planning, smaller ones like Sonnet 5.5 for triage and simple tickets. The claimed result is hundreds of factory runs per week without touching his rate limits.

Medin speaks as the builder of his own experiment with hundreds of hours of build and test time, but the video provides no error rates, no merge statistics and no direct comparison between dark factory and software factory operation; hundreds of runs per week remains his own claim, an independent measurement does not exist. The section on the Port catalog is sponsored and marked as such in the video, it addresses the organizational layer for teams running hundreds of services and agent pull requests, and it does not change the argument about intent or maintainability. For agent practitioners the takeaway is concrete: autonomy is not a ladder to climb but a boundary to place, and the boundary sits where the human stays irreplaceable, at intent before the run, at maintainability of the living codebase, and at the final staging review. Teams that want factories should plan for those three touchpoints instead of waiting for a model that guesses intent. Under Shapiro's taxonomy, level 5 remains a destination without a route; Medin's factory is the currently viable path toward it.

Video: https://www.youtube.com/watch?v=GQatuTWSRqY
Dan Shapiro, five levels blog post: https://www.danshapiro.com/blog/2026/01/the-five-levels-from-spicy-autocomplete-to-the-software-factory/
Simon Willison on the five levels: https://simonwillison.net/2026/Jan/28/the-five-levels/

YouTube video 'My AI Dark Factory Dream Is Dead (Here's What Actually Works)' by Cole Medin, channel Cole Medin, accessed 16 Oct 2026.
Dan Shapiro, 'The Five Levels: from Spicy Autocomplete to the Software Factory', blog danshapiro.com, January 2026, accessed 16 Oct 2026.
Simon Willison, 'The Five Levels: from Spicy Autocomplete to the Dark Factory', weblog, accessed 16 Oct 2026.
Dex Horthy, talk on why software factories fail, as referenced and summarized in the Medin video, accessed 16 Oct 2026.