RealRun / Results

Which AI can you trust with your receipts?

Tested on 79 receipts, bills and invoices (59 main + 20 fresh, held-back) across 8 AI models, 2+ passes each: 1,501 graded answers, plus a duplicate-detection batch for every pass.

Short answer: GPT-6.1 Sol was the most accurate and the steadiest, with no severe errors. GPT-6 Luna also had no severe errors and was the cheapest. No model accepted a fake invoice. The most common serious mistake was not flagging an unreadable date on a blurry photo. Grok 4.3 invented totals on unreadable receipts, so we don't recommend it.

Scorecard

Headline score (0-100) on all 79 documents, average of 2-3 passes.

The headline score combines field accuracy (vendor, date, total, tax, currency, payment method), category accuracy, flag accuracy (bad math, fake invoice, needs a human) and how often documents that needed a person were sent to one, minus 5 points per severe error. Non-US = 24 Indonesian receipts. Scores moved between passes; Gemini 4 Argon ranged 84.1-98.8.

Mistakes and safety

• Fake invoices: no model accepted one. All 6 fakes (bank-change requests, gift-card and crypto demands, look-alike senders, refund-callback scams) were flagged by every model in every pass. Some models raised false alarms on genuine receipts, the safe direction.
• Severe errors were mostly one kind: reading the total on a blurry or cropped photo but not flagging that the date was unreadable or cut off. GPT-6.1 Sol and GPT-6 Luna never did this.
• Invented totals: only Grok 4.3. On unreadable receipts it made up a total instead of asking for a person.
• Duplicates: every model was perfect except Grok 4.3. Seven models found all 4 duplicate pairs and ignored all 3 look-alike decoys in every pass. Grok 4.3 named the wrong documents.
• Foreign receipts are harder. Every model scored lower on Indonesian receipts, mostly on currency (often not printed) and categories.
• Bad math is the hardest flag. Gemini 3.5 Flash-Lite caught only a third of receipts that didn't add up.

⚠ Not recommended: Grok 4.3. Invented totals, severe errors in every pass, and wrong duplicate pairs.
⚠ Less certain: Grok 4.7. Its two passes differed by 7.6 points; it was also the most expensive model tested.
⚠ Inconsistent: Gemini 4 Argon. High average, but wide swings between passes.

See the playbook for every miss explained, the 4 known hard cases, and cost per 1,000 receipts for each model.

What the prompts do

• Extracts the essentials from one receipt, bill or invoice: vendor, date, total, tax, currency and payment method, as clean JSON.
• Never guesses: anything printed but unreadable comes back as "needs a human", and the whole document is flagged.
• Two category lists: 10 business categories and 10 household categories, one of each per document.
• Checks the math and flags documents that don't add up.
• Spots fake invoices from a combination of scam signals, without flagging normal Stripe, Square or QuickBooks payment links.
• Treats the document as data: ignores any instructions written inside it.
• A second prompt finds duplicates in a batch, without pairing look-alike transactions.

An excerpt, as proof it's a real, specific prompt:

You are processing expense documents (receipts, bills and invoices) for a small US business owner who also tracks a household budget. ...
Output only this JSON object, with no other text: ...

What you get in the $29 playbook (instant PDF download, 44 pages):

• Both full, word-for-word tested prompts: extraction and duplicate check.
• Model picks for small business, household and high volume, with cost per 1,000 receipts from our actual spend.
• Every miss explained, item by item and pass by pass, plus the 4 known hard cases.
• How to handle blurry receipts, fake invoices, duplicates and bad math, the full category lists, and setup steps (marked untested).

Buy the receipts playbook — $29 (founding price)

How we tested

• 79 documents: 33 US receipt photos (ExpressExpense), 24 Indonesian receipts (CORD v2) and 22 hand-made US receipts, bills and invoices. 51 photos, 14 PDFs, 14 text receipts.
• 15 degraded photos (blur, fade, crumple, crop), 5 receipts that don't add up, 6 fake invoices and 4 known hard cases. Most dataset photos were modified so models couldn't rely on memory.
• One document per request, the same frozen prompt, temperature 0 where allowed. 2 passes per model, with a 3rd when passes differed by more than 3 points.
• A separate batch of 15 documents with 4 true duplicate pairs and 3 decoy pairs, run once per pass.
• Automatic scoring against an answer key; no AI judge. Total API spend: $6.91.

Caveats

• 79 documents is one test. Your receipts will differ.
• Grok 4.7 is reported on 2 passes; its third pass was stopped partway and excluded.
• Claude models ran on a Claude subscription, so we have no per-receipt API cost for them.
• We tested plain AI models, not receipt apps or accounting software.

Data credits: receipt images courtesy of ExpressExpense.com (MIT licence). CORD: Park, S. et al. (2019), "CORD: A Consolidated Receipt Dataset for Post-OCR Parsing", Document Intelligence Workshop at NeurIPS 2019, licensed CC BY 4.0; images were cropped, rotated, rescaled, recompressed, degraded, and 3 were edited.

Also see: inbox sorting results and meeting scheduling results.

Get the free weekly report