Claude Haiku 5.5 vs GPT-6 Luna

Claude Haiku 5.5 ran in Claude Code 2.1.284 and GPT-6 Luna in Codex CLI 0.159.0, in the bench's Docker container with the account's apps, plugins and connectors off, at medium and at xhigh effort. Those are each tool's own settings: Claude Code's --effort and Codex's reasoning effort. Left alone, Claude Code 2.1.284 runs Haiku 5.5 at high and Codex runs GPT-6 Luna at medium, so I ran both at the same two levels. Every run is reported, failed ones included, and the plan was written down before the runs.

Claude Code 2.1.284 predates the first release that lists Haiku 5.5, 2.1.293: 2.1.284 accepts the model, but prices its runs at Opus 5.5's rates and runs it at high effort when none is set. So I reran Haiku 5.5's speed test on 2.1.293: 6.6 to 9.0 s at medium and 11.4 to 14.9 s at xhigh, against 6.4 to 7.9 s and 8.9 to 16.1 s on 2.1.284, with 503 and 1,955 thinking tokens in the answer by median against 500 and 2,087. The runs overlap and every counted run set its effort, so the 2.1.284 runs stand, though I had planned to rerun everything if the default differed, and it did: with no effort set, 2.1.293 ran Haiku 5.5 at medium, the same default Codex uses for GPT-6 Luna. Only speed was checked, so skills and real-work token counts on 2.1.293 could differ. These check runs ran while other tests were running.

The speed test is the explain task the GPT-6.1 Sol page uses: the tool reads a small Python file and explains how it adds up monthly totals and what happens with a bad date, without changing anything. The clock runs from starting the tool until it exits, one run at a time, with nothing else running, except the seventh GPT-6 Luna medium run, which replaced a failed one, and the Claude Code check below; both ran while other tests ran. A run passes when its answer contains both "month" and "ValueError".

TPS here is visible tokens a second: the answer reply's output tokens minus its hidden thinking, each vendor's own count, over the seconds from its first text event to its last, as the text reaches my machine. A rate counts only when the text arrived in at least 20 separate bursts, where text more than 5 ms after the previous text starts a new one. Text a service holds back and sends at once arrives in one or two bursts, and its rate would mean nothing. In the trial runs before the counted ones, Claude Sonnet 5.5's answers arrived in 1 and 2 bursts, which is why the GPT-6.1 Sol page has no Claude TPS, while Haiku 5.5's arrived in about a hundred. Anthropic and OpenAI count tokens differently, so a Haiku token and a Luna token are not the same amount of text.

The skills test is the Skills page's: four small tasks (explain, feature, bug fix and build), each run three times with no skill, with a placebo instruction file and with caveman, ponytail and karpathy-skills as instruction files, for both models at both efforts. The ASD-STE100 test adds Karpathy's one-line instruction to the explain task, three runs each. Costs use each vendor's list price per token for prompts up to 100k tokens. No skills run's prompt passed 100k; eight real-work runs did, and their costs show as a range there.

Changes after the runs started, each logged with its reason: one failed GPT-6 Luna medium speed run was replaced by a seventh, and speed comparisons use passed runs; GPT-6 Luna's costs became a range because Codex logs no cache writes; Haiku 5.5 runs whose prompts passed 100k tokens show a cost range because only the last request's size was kept; the X video shows the six passed GPT-6 Luna medium runs without noting the failed one, which this page and the table list; Haiku 5.5's cache writes use the one-hour rate of $0.20, the one Claude Code's requests pay; and a TPS median needs at least three runs that streamed. The plan and every change to it, as logged, are in haiku-luna-plan.txt.

The run records are in results.json, under h2h_timing_runs, h2h_runs, h2h_real_runs, h2h_cli_check and h2h_timing_smoke.