RIFT vs Payload
RIFT ran the same eighteen tasks under two different models — Gemini and Sonnet 5 — and stayed fully stable under both; Payload did not stay stable under either.
We connected the same AI agent to two content management systems over their own MCP surfaces: RIFT and Payload CMS (its official MCP plugin). Both held an identical copy of the same 30-page, 119-link documentation site. The same Godzilla docs site used for the Drift Race. The agent was given eighteen tasks a content team would ask — find, create, edit, rename, move, split, merge, retire, bulk-update — each run under two different models: a fast, cheaper model (Gemini) and a slower, more deliberative one (Sonnet 5). RIFT stayed fully stable, 18 of 18, under both models. Payload did not stay stable under either — 10 of 18 under Gemini, 14 of 18 under Sonnet 5. RIFT's token count and its task time changed with different models, not its stability.
How it was tested
The eighteen tasks were written in content-team language before either system was probed — things like "add a page announcing the v3 SDK to the changelog section", "rename the FAQ page", "move the tutorials page under guides", "split the API overview into two pages", "retire the legacy auth page and point its links somewhere sensible", "add a review note to every SDK page", "which pages link to the changelog?". Each task ran in a fresh agent session with no memory, twice per system. Success was never the agent's claim — after every session, the probe read the CMS's own database state and checked whether the requested change had happened. That distinction matters, because the outcome classes include silent miss: the agent reports success and the change did not happen, or something weaker happened than what was said.
Both systems held the same site, seeded from one shared fixture. RIFT ran its normal governed path — every write proposed into a change-set and approved before counting. Payload ran its official MCP plugin with full collection permissions. Versions pinned, results committed run by run.
RIFT vs Payload, task by task
The like-for-like comparison: same agent, same eighteen tasks, same site, and the same model (Claude Sonnet 5) driving both systems. Median tokens across the two passes. The last column is Payload divided by RIFT — above 1.0 means RIFT did the task for fewer tokens, below 1.0 means Payload did.
RIFT is leaner on fifteen of the eighteen tasks. Payload is leaner on three — D2, C2 and X1 — and completed R2, which RIFT did not. Those are in the table for the same reason the rest is: the instrument reports what happened. The gap that matters most is not any single cell but the stability column: RIFT returned the same outcome on both passes for all eighteen tasks, Payload for fourteen.
Reading X2: RIFT's 4,924 tokens are the cost of refusing a hard delete under its no-hard-delete policy and naming the retirement path instead — not the cost of deleting the page. Payload deleted it.
The same tasks in RIFT under two different models
Both columns below are RIFT. This table is not the Payload comparison — it isolates a different question: how much does the choice of model change what the same surface costs? Gemini is fast and cheaper per token, Sonnet 5 slower and more deliberate. RIFT stayed 18 of 18 stable under both, and the price band barely moved; what moved was the token count and the clock.
Tokens and wall-clock seconds per task, one run per model, both models on RIFT's current commit (0adc095), same shared 30-page site. Full per-session data — every outcome, turn count, cost, and the agent's own words — is in the public repo below.
Reading the outcome notes: X2's "supported" result for Sonnet 5 means it correctly refused the hard delete under RIFT's no-hard-delete policy — not that it succeeded at deleting the page; the opposite reading is the natural one and the wrong one. R2 is Sonnet 5's one genuine shortfall: there is no "Concepts" folder yet, so the move requires a section to be created first — Gemini created it, Sonnet 5 asked for clarification instead, and both behaviors were disclosed, not a silent miss. D3 is the one task where Sonnet 5 costs more than Gemini (24,934 vs 13,008 tokens) — Sonnet 5 is not cheaper across the board.
What the numbers are made of
Consistent Results
Switch models, get the same behavior: RIFT held 18 of 18 stable under both Gemini and Sonnet 5 on the current commit. Payload did not hold under either — 10 of 18 under Gemini, 14 of 18 under Sonnet 5. On the move task, one Payload run moved the page and the other reported moving it without doing it. An agent surface you can't predict slows down your operation.
Cost Efficiency
Payload's tool signatures inline its whole document shape — 12,583 tokens of schema on every single call, against RIFT's 2,364. That tax compounds regardless of model, though how much depends on it: per task, RIFT's median is 6.9× leaner than Payload's under Gemini and 2.7× leaner under Sonnet 5. Across all thirty-six sessions RIFT spent 6.87M tokens under Gemini against Payload's 30.24M, and 2.32M under Sonnet 5 against Payload's 4.80M.
Deletion Design
Asked to permanently delete a page, RIFT never performed a hard delete under either model. Under Gemini, it substituted the retirement flow and disclosed the swap — archiving is not deletion, and history is kept. Under Sonnet 5, it refused the request outright, upholding the same no-hard-delete policy. Payload deleted the page. No confirmation, no staging, no way back. Governance as a product choice.
Saying No Costs Almost Nothing
Asked to publish directly and skip review, RIFT refused under both models and named the approval path — 3,250 tokens under Gemini, 5,004 under Sonnet 5, both among the cheapest cells in the whole matrix. A surface that knows what it won't do saves the agent from spending far more finding out.
The Inbound-Link Question
"Which pages link here?" now costs RIFT 13,008 tokens and 8 seconds under Gemini on the current commit (0adc095) — against Payload's 588k for the same brute-force read. RIFT wins this cell decisively.
Historical note: on an earlier commit (b8461b8), before a since-shipped fix (get_inbound_links), this same task cost RIFT 739,252 tokens — a genuine loss against Payload at the time. That gap is closed on the current build.
What Neither System Attempts
Rollback and version history are invisible to agents on both surfaces. Both products keep full version records internally; neither lets an agent use them. We are publishing that as a gap in our own product too — it is next on the roadmap.
What this does not prove
One run per task, two models, one shared site. The results are directional, not statistical. The strongest honest claim is the pattern: switching the model changes RIFT's token count and its wall-clock time, not its stability or its price band — RIFT stays lean and fully stable whether the agent is fast-and-cheap (Gemini) or slower and more deliberate (Sonnet 5), and neither model closes Payload's stability gap. Every number on this page is recomputable from the public repo, including the probe's own bugs, which are disclosed there.
github.com/heitham/Drift-race-godzilla-method — the probe code, the frozen tasks, both columns' full session data, and the recipe for adding another CMS in hours. This is the second instrument in the series; the first, a thirty-operation lifecycle race against raw files, is written up in The Drift Race: methodology and measured results.