RealRun / Results

Which AI can you trust to sort your inbox?

Tested on 80 realistic emails (50 original + 30 fresh, held-back) across 8 AI models, 2+ passes each: 1,420 graded answers with our tested v3 prompt.

Inbox triage test, October 2026. 50 made-up emails plus 30 fresh holdout emails, 8 AI models. Update 14 Oct: results for our tested v3 prompt are first; the original baseline-prompt results follow below.

New: results with our tested v3 prompt

We rewrote the prompt to fix the Read later weakness and add scam safety rules, froze it, then ran it on all 8 models: 2 passes each on the original 50 emails and on 30 fresh holdout emails the prompt was never tuned on (a 3rd pass where two passes differed by more than 4 points). Mistakes are reported as they happened. How the prompt works is described below.

Short answer: With v3, six models scored 90% or more on the fresh emails and nearly every model caught all the Read later emails. Claude Haiku and Grok 4.20 still labeled scams as Act, so we don't recommend them for triage.

Model	Accuracy, original 50 (passes)	Accuracy, fresh holdout 30 (passes)	Scam safety	Read later caught, original (of 8)	Read later caught, holdout (of 6)
Claude Opus	96.0%	94.4%	No scams labeled Act	8.0	6.0
GPT-6.1	95.0%	95.0%	No scams labeled Act	7.5	6.0
Claude Sonnet	94.0%	91.7%	No scams labeled Act	8.0	6.0
Gemini 3.8 Flash	94.0%	91.1%	No scams labeled Act	7.5	6.0
GPT-5.4 mini	93.0%	95.0%	No scams labeled Act	7.0	6.0
Grok 4.7	92.0%	90.0%	No scams labeled Act	7.0	6.0
Grok 4.20 ⚠	92.0%	86.7%	⚠ Labeled a scam as Act	8.0	6.0
Claude Haiku ⚠	86.0%	85.6%	⚠ Labeled scams as Act, repeatedly	6.3	5.7

Average of 2-3 passes per model. Scores moved by up to 10-12 points between passes for some models (Haiku most), so small gaps aren't meaningful. On the holdout, one email is 3.3 points.

⚠ Not recommended for triage: Claude Haiku. Its scores swung between passes, and it labeled several scam emails as Act in most passes.
⚠ Not recommended for triage: Grok 4.20. It labeled a fake invoice-link scam as Act in both passes on the fresh emails.
The other six models never labeled a scam as Act with v3.

Other misses: the most common was a debatable auto-renewal notice that several top models filed as Archive instead of Act. See the playbook for email-by-email misses and per-pass scores.

Old prompt vs v3 (original 50 emails)

Model	Baseline prompt (3 runs)	v3 prompt	Read later caught, baseline → v3 (of 8)
Claude Opus	95.3%	96.0%	6.0 → 8.0
GPT-6.1	89.3%	95.0%	3.7 → 7.5
Claude Sonnet	88.0%	94.0%	2.7 → 8.0
Gemini 3.8 Flash	90.0%	94.0%	4.7 → 7.5
GPT-5.4 mini	84.0%	93.0%	2.0 → 7.0
Grok 4.7	80.7%	92.0%	0.0 → 7.0
Grok 4.20 ⚠	82.0%	92.0%	2.7 → 8.0
Claude Haiku ⚠	78.7%	86.0%	0.7 → 6.3

Baseline = 3 runs; v3 = 2-3 passes, so the run counts differ.

What the v3 prompt does

Four labels, defined in plain words: Act (you must reply, pay, sign, decide or RSVP), Waiting (you sent the last message), Read later (worth reading, no action), and Archive (promos, receipts, notifications, scams).
Read later gets its own definition and examples, plus a list of senders you always want to read, so newsletters and shared documents stop landing in Archive. When unsure between Read later and Archive, it leans Read later.
A safety rule: requests for gift cards, payments or payment-detail changes, passwords, or sign-in links are never Act unless the sender is verified. Real bills from companies you do business with still stay Act.
Deadlines come back as a date, and a sale's end date never counts as your deadline.

An excerpt, as proof it's a real, specific prompt:

You are triaging [your name]'s inbox. Today is [date]. Assign each email exactly one label: Act, Waiting, Read later, Archive.
...
Output CSV: id,label,deadline. No other text.

What you get in the $29 playbook (instant PDF download):

The exact, word-for-word tested v3 prompt, with a fill-in template.
Before-and-after numbers for all 8 models (e.g. GPT-5.4 mini 84.0% → 93.0% on our original 50 emails).
Setup guides: ChatGPT or Claude copy-paste, Gmail with Gemini, and no-code automation.
Safety rules, which models to avoid, and email-by-email notes on every miss.

Buy the playbook — $29 (founding price)

Original results: baseline prompt

These are the first results, from our short baseline prompt (3 runs each, original 50 emails only).

Short answer (baseline prompt): Claude Opus was the most accurate at 95.3%. Gemini 3.8 Flash (90.0%) and GPT-6.1 (89.3%) were next, then Claude Sonnet (88.0%). Grok 4.20 marked a phishing email as something to act on in 2 of 3 runs. Every model missed some "read later" emails.

Baseline scorecard

AI models: average of 3 runs, ± how much the score moved between runs.

Model	Accuracy	Macro-F1	Phishing marked Act	Critical misses	Read later caught (of 8)	Deadlines exact (of 17)	Tricky (of 14)	Sec/email	Cost per 50 emails
Claude Opus	95.3% ±1.2	0.950	0	0	6.0	14.0	13.7	6.4	Claude plan*
Gemini 3.8 Flash	90.0% ±2.0	0.883	0	0	4.7	14.0	13.3	3.9	$0.053
GPT-6.1	89.3% ±1.2	0.854	0	0	3.7	15.0	14.0	2.6	$0.027
Claude Sonnet	88.0% ±2.0	0.827	0	0	2.7	14.7	14.0	6.1	Claude plan*
GPT-5.4 mini	84.0% ±2.0	0.773	0	1.0	2.0	14.3	13.7	1.4	$0.021
Grok 4.20	82.0% ±2.0	0.770	0.7 (2 of 3 runs)	0	2.7	13.0	12.3	0.6	$0.012
Grok 4.7	80.7% ±1.2	0.660	0	0	0.0	15.0	14.0	6.4	$0.177
Claude Haiku	78.7% ±1.2	0.668	0	0	0.7	14.3	12.7	6.5	Claude plan*

Baselines (single run, not averaged)

Setup	Accuracy	Macro-F1	Phishing marked Act	Critical misses	Read later caught (of 8)	Deadlines exact (of 17)	Tricky (of 14)	Sec/email	Cost
Keyword rules**	84%	0.845	3	0	7	8	9	~0	$0
Qwen2.5 3B (local)	48%	0.448	0	2	4	10	6	1.7	$0
Qwen2.5 7B (local)	34%	0.326	1	0	2	12	2	5.3	$0

For the AI models, every column is an average over 3 runs, so some are fractions. Critical miss = an email that needed action got filed as Archive. *Run through Claude Code on a Claude subscription, so no per-email charge; seconds per email are mostly tool startup time. **Rules were written by someone who had already seen the emails, so treat 84% as a best case for keyword rules, not a fair score. They also marked 3 phishing emails as Act.

Coming soon: Gemini 3.1 Pro. Its API was too slow and timed out too often to finish a valid run. We'll retry at a quieter time.

What the baseline results meant

If you want an AI to sort your email, Claude Opus is the one we'd trust today. Gemini 3.8 Flash, GPT-6.1 and Claude Sonnet are close behind, and the API ones cost pennies per 50 emails.
Read later is everyone's weak spot. Every model filed some newsletters and articles that belonged in Read later as Archive. Grok 4.7 caught none of the 8. Plan for this: give your agent a short list of senders you always want to read. (The v3 prompt above largely fixed this.)
Watch the safety notes. Grok 4.20 marked a phishing email as Act in 2 of 3 runs. GPT-5.4 mini filed about one action email as Archive per run. Gemini 3.8 Flash's safety filter blocked the CEO gift-card scam email in 2 runs, which we scored as wrong.
Keyword rules marked 3 phishing emails as Act. Rules can be fooled by urgent wording.
Small models running on a regular computer got less than half right. Not worth it for this job yet.

How we tested

50 synthetic emails with an answer key: 16 Act, 6 Waiting, 8 Read later, 20 Archive. 17 have deadlines and 14 are tricky (scams, look-alikes, buried asks).
Same fixed prompt for every model, one email per call, no access to the answer key.
Temperature 0 where we could set it. Claude ran with Claude Code defaults.
All dummy data. No real inbox or personal info was used.
Each AI model ran 3 times, scored by the same script. We report the average and the spread. Baselines ran once.

Caveats

This is one test of 50 emails. A few emails swing the score by several points.
We tested plain AI models, not finished inbox products like Inbox Zero or Gmail connectors. Those are next.
Grok models used more input tokens per email than the others, likely a system prompt xAI adds. We did not change anything for it.
Speed numbers depend on our setup and aren't a fair speed race.

Get the next results

Get the free weekly report

Want the tested v3 prompt with setup guides and safety rules? Get the inbox triage playbook: the tested v3 prompt, setup guides, and before/after numbers. Instant PDF download. $29 founding price (regular $49).

Buy the playbook — $29 (founding price)