RealRun / Guides
Short answer: Claude led, GPT was close, Grok was behind. On our hard round of support email, Claude Opus scored 98.6 and GPT-6.1 Sol 97.3, both with 0 severe errors over 2 passes. Grok 4.7 scored 93.2 with 1 severe error on 1 pass. The bigger gap is inside each family: the small models, GPT-5.4 mini (68.0) and Claude Haiku (80.5), made the most severe errors.
| Model | Score | Severe errors | Passes | Cost |
|---|---|---|---|---|
| Claude Opus | 98.6 | 0 | 2 | subscription |
| GPT-6.1 Sol | 97.3 | 0 | 2 | medium |
| Grok 4.7 | 93.2 | 1 | 1 | high |
| Claude Sonnet | 89.4 | 3 | 2 | subscription |
| Grok 4.6 | 88.0 | 2 | 1 | high |
| GPT-6 Luna | 82.2 | 5 | 2 | low |
| Claude Haiku | 80.5 | 6 | 2 | subscription |
| GPT-5.4 mini | 68.0 | 10 | 2 | low |
We tested the models, not the ChatGPT, Claude or Grok apps. The apps add their own settings, so your results in them can differ. Gemini was tested too: Gemini 4 Argon 97.8 (1 pass), Gemini 3.8 Flash 90.6.
How the testing works: every model got the same set: Round 1 (easy): 80 emails. Round 2 (hard): 40 main + 20 fresh held-back emails written blind (140 items, 2,600 graded answers, 10 models). Passes: 2 for 7 models; Gemini 4 Argon, Grok 4.7 and Grok 4.6 ran 1 pass (less certain). Score: Round 2 overall: mean item score minus 5 per severe error, over all items answered, mean of passes. Answer keys for 4 items were widened after seeing answers to accept a second correct answer (logged).
Results updated 15 Oct 2026. Tests run October 2026. Full results: Customer support replies.
Picking a brand isn't enough. In this test the same brand had a model at the top and a model you shouldn't let near a customer. GPT-6.1 Sol made 0 severe errors; GPT-5.4 mini promised wrong refund amounts and leaked another customer's data. Check which model your support tool actually runs.
When a client asks "ChatGPT, Claude or Grok?", answer with a model name, not a brand, and show them the severe-error count. Only Claude Opus and GPT-6.1 Sol from these three families finished 2 passes with 0 severe errors.
Want the exact prompt and setup?
Get the playbook for this job, or all five for $99 (save $46).
Support Replies AI Playbook, $29 or get the free weekly AI test report
Scores reflect our tests on the dates shown, not a guarantee. Claude models ran on a Claude subscription, so they have no metered API cost. cost_tier is relative within a task (low/medium/high) from our run spend; 'subscription' = not metered. Tested October 2026 by RealRun (Base Ledger IP LLC).