Pi vs Codex: One Task Ahead, 2.4x the Token Bill

Two coding agents, one controlled experiment, and a result that cuts against the market narrative. On September 20, 2026, Composio ran five agent harnesses through the same 30 hard tool-use tasks with exactly the same model. Pi came out ahead with 21 of 30 tasks, OpenAI Codex followed at 20, while OpenCode and Claude Code stopped at 19. The number that surprised me sits in the token column: Codex averaged 383,722 tokens per task, Pi averaged 924,990. The winner pays roughly 2.4x the token bill for a single task of advantage. This article walks through the data, explains why the result holds up, and sets it next to my own measurements from running this deck day by day on the Pixel Cyberdeck.

Ground truth first, so we agree on the objects. Pi is the agent toolkit started by Mario Zechner, now maintained under Earendil, MIT licensed, and built around four core tools: read, write, edit, and bash. Its system prompt plus tool definitions stay under roughly 1,000 tokens, the context stays visible and swappable, and switching models does not force a rebuild of half the stack. Codex is the opposite bet: a complete platform with sandboxing, cloud tasks, subagents, hooks, skills, and an opinionated patch pipeline. Live snapshot from September 25, 2026: Pi v0.87.1 shipped September 22 with 109,347 GitHub stars, while Codex v0.157.0 landed today as release number 1,145, past 126,000 stars, with Rust at 97.1 percent of the codebase. Both projects breathe daily, keep that in mind for every number below.

The methodology is the part that earns my trust. Every harness received the same model, DeepSeek V4 Pro (0813) at max reasoning, through the same hosted MCP router, the same 30 tasks against real applications, and the same verifier checks. If the outcome moves, the only free variable is the harness, never the model. That isolates exactly the question that matters when you build agents: how much software may sit between a model and its task before it costs more than it adds. Most comparisons blend model quality and tooling into one measurement; this design keeps the factor clean. Test date September 20, 2026, and the full write-up is linked in the sources section.

Reading the table from the top: Pi 21 of 30, Codex 20, the in-house DeepSeek Harness also 20, OpenCode 19, Claude Code 19. On the tasks both leaders solved, the price tied at 0.031 dollars per shared success. The token ledger is where the paths split hard: OpenCode burned 710,140 tokens per task, Claude Code 649,900 while also carrying the most expensive shared-success price in the field at 0.074 dollars. The DeepSeek Harness was frugal at 88,562 tokens and still missed the same two tasks as Codex. If you read only the task column, you miss half the story; the interesting axis runs between the pass rate and the token account.

So why does the smaller harness lose the token race so badly? The explanation in the write-up matches what I see on my own stack. Pi steers almost nothing: under 1,000 tokens of base context, and the model explores on its own which files, tests, and commands move the problem forward. Freedom costs turns, and turns cost tokens, most of all on tasks with heavy setup. Codex owns context, sandboxing, and the patch workflow centrally, and OpenAI trains its models against that exact flow; fewer detours, fewer tokens. The price is visibility: once the platform manages your context, you no longer see what it withheld from the model unless you ask. For the benchmark that means Pi buys both the win and the transparency with a fatter account.

Behind the numbers sit two answers to the same question. OpenAI built Codex as a product: hand it tasks and you get sandboxing, cloud runs, team workflows, and safety defaults you never had to build. Mario Zechner built Pi as a kit: small enough to rebuild without hitting a platform wall, and its TypeScript extensions may change the runtime itself. After tracking Pi's model-churn cycle in an earlier article (https://hunter-cyberdeck.d4sn3st.dev/post.html?slug=pi-0871-model-churn), hackability is the buying argument for me. An agent whose context I can replace at any time survives a model swap better than one whose context someone else manages.

Now the countercheck from my own operation, with numbers I measured myself. On the Pixel Cyberdeck, Pi is the executing agent in the agent family, wired in through herdr, while Hermes Agent stays the control room. The benchmark lesson applies at this scale too: a lean harness wins nothing by itself, it needs watchdogs, recovery paths, and honest bookkeeping over resources. The following block carries my latest values, cut-off dates included.

I put my operation log next to a benchmark for one reason: both draw the same scissors. Small and visible is repairable, big and invisible is convenient until the outage. The gateway had run 306 checks in a row without a miss the week before, then the phone died under a full swap, and the watchdog that should have reported it was dead itself over a missing coreutils package with no nohup in Termux. That is agent operations: no victory speeches, only states. I documented how my agents cooperate day to day in an earlier piece (https://hunter-cyberdeck.d4sn3st.dev/post.html?slug=my-agent-moved-into-an-agent-community). The benchmark measures the harness, my log measures the life around it, and both sides agree: keeping control costs attention, giving it up saves tokens and takes away sight.

The decision aid falls out of the data. If you want to build your own agent, swap models freely, wire in local models, or live on an aging Android like a Pixel 6a, Pi fits; the 21 to 20 is the argument, the 2.4x token bill is the price. If you hand off real engineering tasks, work in a team, and want defaults you did not build, take Codex and save roughly half the tokens. I worked through what Codex's usage cap model is worth in daily budget elsewhere (https://hunter-cyberdeck.d4sn3st.dev/post.html?slug=codex-usage-caps-8-million-users). The honest rule of thumb: build or operate, both are legitimate, but doing both at once costs double.

Honesty first: a benchmark is a snapshot. One model, DeepSeek V4 Pro (0813), one router, 30 tasks, one date, September 20, 2026. None of this is a general market claim, and both projects move daily; Codex set release number 1,145 today and shipped an MCP fix around credential boundaries, and Pi landed commits on RPC disposition today as well. If you quote these numbers in three weeks, open both repos again. My own measurements come from stability-snapshots.log and the daily notes dated September 25, 2026, stated with their cut-off dates, not sold as a permanent condition.

What I watch next: whether Pi can close the token gap without giving up its philosophy, and whether Codex closes the one-task gap with models trained around its harness. The next checkpoints are Pi v0.88 and Codex release 1,146. Until then the honest table stands: Pi one task ahead, Codex with half the token bill, and the choice depends on whether you want to build an agent or operate one. Both are legitimate, but only one is mine.

I am Marcel, a graphic designer running the HUNTER Cyberdeck on a used Google Pixel 6a. For this article I checked the Composio benchmark data against the live state of both repos on September 25, 2026, and set my own operating numbers from stability-snapshots.log and the daily notes of the same day next to them. The measurements come from the everyday operation of the agent family on this phone. Last reviewed: September 25, 2026.

Composio: Pi Agent vs OpenAI Codex, September 20, 2026, retrieved September 25, 2026. https://composio.dev/content/pi-agent-vs-codex

GitHub: earendil-works/pi (releases, star count), retrieved September 25, 2026. https://github.com/earendil-works/pi

GitHub: openai/codex, release v0.157.0 dated September 25, 2026, retrieved September 25, 2026. https://github.com/openai/codex/releases

Augmentcode: OpenAI releases Codex CLI, repo data as of April 2026, retrieved September 25, 2026. https://www.augmentcode.com/learn/openai-codex-cli-terminal-agent

Own measurement: stability-snapshots.log (cut-off dates September 21 and 25, 2026) and daily notes 2026-09-25, Pixel Cyberdeck.