About this project

This is a demo of one narrow job done carefully: turning a photograph or a PDF of a receipt into typed fields a person can check, correct and export. It has no chat box. You give it a receipt, it gives you data, and it shows its working at every step so you can tell a reading from a guess.

What it does to a receipt

Your browser prepares the file first. A PDF is rasterized at twice its page size and only the first page is kept, and a photograph too large for the request body is re-encoded and shrunk until it fits. The image you then see in the document pane is the image the model reads, which is why a highlight drawn over it can be trusted to sit where it claims.

One request goes to the model, with the field set as the schema it has to satisfy. The answer comes back already shaped, so nothing hunts for JSON inside prose, and Zod checks it again on the server before it is sent to your browser. A file that is not a receipt is a first-class answer: the app says so, repeats the reason the model gave, and stops.

The highlight is matched rather than generated. The app never asks the model for coordinates, because a model will happily answer with numbers that look precise and are wrong. Instead the browser reads the words off the document with their rectangles, then scores runs of adjacent words against the characters each field was read from. Nothing below the match score draws a mark at all.

The arithmetic then runs in decimal code written for this repo. Line items have to sum to the printed subtotal, and the subtotal plus the taxes plus the tip has to equal the total. A mismatch becomes a warning naming the difference on both fields involved. The app never edits a number to make a sum close, because a figure that disagrees with its neighbours is the most useful thing on the page.

Everything is editable where it sits, and every edit reruns both checks. A field you typed into keeps the characters the model originally read it from, so the highlight still works, and it carries no confidence at all, because the app will not claim a certainty on your behalf.

How well it reads a receipt

40 labelled receipts went through the same extraction this page describes on 2026-10-07, against gpt-6.1-sol, with nothing stubbed and nothing retried. The 40 fixtures are drawn and then damaged in code rather than photographed, so synthetic fading is not thermal paper and the set carries no handwriting and no language but English. The label of each fixture was written in the same run that drew its image, so no expectation was typed by hand, and fixtures/REPORT.md in the repository holds the per-field and per-fixture rows behind every number below.

98.7%

of fields read correctly

100.0%

of totals read correctly

85.0%

of dates read correctly

98.7% is 1,662 of 1,684 compared fields, and the 22 misses are worth more than the headline. The app read the printed total on all 40 fixtures. It read 34 of the 40 dates, which makes date the weakest field in the set: every one of those six misses prints a day of 12 or under, so 05/06 is a genuinely ambiguous string and the model answered with the other day-month reading. The eight fixtures printing an ISO date read 8 of 8. Five more misses are a payment method the paper prints as CASH and the model returned as Cash, which this site counts as wrong rather than widening the comparison to flatter the figure; a reader who discounts capitals gets 99.0%. The remaining eleven are a tax line the model invented on two fixtures, two short merchant addresses, a wrong digit in a third, and four single-character slips inside line items.

One thing this run does not show is the damage mattering. Every degradation scored between 98.2% and 98.8% against 99.0% for the clean renders, so drawn fading and drawn blur are not what a creased till roll in a hot car does to the page. The first run of this harness reported 76.6%, and the fault was the labels rather than the model: the generator disagreed with itself about whether a layout prints a quantity column, so 374 item cells were compared against a value the paper never printed. One predicate now answers that question for both the drawing and the label, and fixtures/REPORT.md keeps both runs.

The figures on this site

The three measured figures carry the run above. Every other one is read out of the constant the app enforces, so a cap that changes changes this table, the tiles on the front page and the README together.

FigureWhat it counts
98.7%

of fields read correctly

1,662 of 1,684 compared fields matched their label across the 40 fixtures, measured on 2026-10-07 against gpt-6.1-sol. The 40 fixtures are drawn and then damaged in code rather than photographed, so synthetic fading is not thermal paper and the set carries no handwriting and no language but English. Twenty-two fields came back wrong and fixtures/REPORT.md names every one of them.

100.0%

of totals read correctly

The app read the printed total on all 40 fixtures, 40 of 40. It is broken out beside the date because those are the two values a person checks first.

85.0%

of dates read correctly

34 of 40, the weakest field in the set. All six misses are one mistake: a printed date whose day is 12 or under read in the other day-month convention, so 05/06 can come back as either the fifth of June or the sixth of May. Nothing on the paper says which convention it used, and the app sends no locale with the image. The eight invoices printing an ISO date read 8 of 8, so check a slash-separated date before you trust it.

12

fields per receipt

The merchant and its address, the date and the time, the currency, the subtotal, the taxes, the tip, the total, the payment method, the card's last four digits and the line items. Each value arrives with the model's confidence in it and the characters it was read from.

4

values on every line item

A receipt's items come back as a list the app can sum, and each entry holds its description, quantity, unit price and amount. A weighed item keeps a quantity of 0.734, because a quantity is a decimal string like an amount.

4

file formats

JPEG, PNG, WebP and PDF, up to 8 MB. A PDF is rasterized in the browser and the first page is what the model reads.

40

scans a session

The allowance a browser tab holds. It rests on a value the browser asserts, so it is a courtesy rather than a control, and the app says so where a refused visitor reads it.

20

scans an hour per address

The control behind the courtesy, counted in Redis before the model is called. A refusal names the wall-clock hour the allowance returns, because the window is fixed rather than rolling.

0.72

match score a highlight needs

The app does not ask the model where a value sits on the page. It reads word boxes off the document and scores them against the characters the model says it read, and nothing below this score draws a mark.

0.8

confidence that asks for a check

A field the model is less sure of than this is flagged for review. The deployed app has not yet produced one, which the failure cases below say plainly.

1

model call per receipt

One request to gpt-6.1-sol with Structured Outputs in strict mode, whose schema is the field set. There is no JSON to parse out of prose and no retry loop around a malformed answer.

What it is built on

FrameworkNext.js 16 on the App Router, React 19, TypeScript
InterfaceMantine, with its PostCSS preset. No Tailwind sits beside it.
ModelThe OpenAI API, a vision model, Structured Outputs in strict mode
ValidationZod, the same schema on the server and in the browser
Word boxespdf.js for a PDF's text layer, tesseract.js in a web worker otherwise
MoneyDecimal strings compared over scaled integers, written in this repo
SessionIndexedDB keyed to the browser tab. No database, nothing on a server.
Rate limitingA fixed hourly window per address, counted in Upstash Redis
HostVercel, with production deploying from the main branch and Web Analytics counting its page views
TestsVitest over the schema, the arithmetic, the matcher and the exports

What it keeps

Your upload lives in memory for the length of one request and is never written to disk. The extracted receipts live in your own browser, keyed to this tab, so a reload keeps them and closing the tab ends them. Two things are kept on a server. A counter records how many scans an address has spent this hour. Vercel Web Analytics counts each page view on the live site with no cookie, recording the page, the referring site, the country and the kind of browser and device. Neither holds a receipt or an image, because no receipt ever reaches a page address.

The three sample receipts are invented. The merchants, the addresses and the card digits belong to no real business and no real card, and each image says so in its own footer as well as on the page beside it.

Where it fails

Every case below is one you can reproduce. They are listed because a demo that only shows its good path tells you nothing you can check.

A date whose day could be its month

This is the measured weakest field: the date read 34 of 40 on the fixture run, and all six misses are the same mistake. Each prints a day of 12 or under, so 05/06 is genuinely either the fifth of June or the sixth of May, and the model answered with the other reading. Nothing on the paper says which convention it used, the app sends no locale with the image, and it will not guess one on your behalf. The eight fixtures printing an ISO date read 8 of 8, so a slash-separated date is the one to check before you export.

Faded thermal paper

A till receipt that has spent a month in a hot car prints grey on grey. The model guesses at the digits, and the OCR pass that draws the highlight finds no words at all, so a wrong figure can arrive with no mark beside it to check it against. The fixture run does not measure this case: its fading is drawn in code and the faded fixtures scored 98.4%, close to the clean renders, so real thermal paper is worse than any figure on this site.

Handwriting

A total written in by hand, or a tip scrawled on a card slip, reads badly. Printed type is what the model and the OCR pass both handle well, and the word boxes OCR returns for cursive rarely match the value the model read.

Angled or crumpled photographs

A receipt shot from the side skews every printed line. The fields often still arrive; the highlight often does not, because the matcher scores runs of words inside one OCR text line and a curved line breaks into several.

A PDF past its first page

The app rasterizes page one and stops there. A two-page invoice loses everything on page two, and the document pane says which page of how many it read rather than hiding the gap.

Receipts not printed in English

The prompt is written in English. Amounts and dates usually survive, and the currency code often does, but a merchant name in another script can come back as an English guess at it. The fixture set says nothing either way, because every fixture in it prints English.

A receipt whose own arithmetic is wrong

The checks compare figures printed on the page, so a receipt that rounds its own tax line raises the same warning a misread digit raises. The app names the difference and leaves the judgment to the reader, because it cannot tell those two apart and will not rewrite a number to close the gap.

The review flag is barely exercised

gpt-6.1-sol has not yet reported a confidence under 0.8 on anything uploaded to the deployed app, and the fixture run confirms it: not one of the 1,684 compared fields came back under the line, the lowest confidence reported anywhere was 0.9, and every field the model read wrong reported 0.93 or above. On this set the model's own confidence predicts nothing about whether a value is right, so the flag is a mechanism waiting for a model that uses it rather than a signal to lean on.

A missing highlight is not a missing field

When no run of words scores above 0.72 against what the model says it read, the app marks nothing rather than marking the wrong region. A correct field can therefore sit on the page with no mark, and the pane says which of the two reasons applies.

A mark can land on the wrong words

The threshold refuses a poor match. It has no way to refuse a confident mistake. Over 1,251 measured samples the shipped 0.72 lost no correct mark and admitted 37 wrong ones, and 29 of those 37 are the currency: asked which characters it read the symbol from, the model often answers with the symbol and a whole address line, so the matcher finds exactly those words and marks the address. 25 of the wrong marks scored a perfect 1, which no threshold can refuse. The fix belongs in the prompt, and it is not in this change.

Who built it

Bob Dempsey, as the fourth AI project on bobdempsey83.com. The source, including the spec each slice was built against, is on GitHub.

  • No account, no key and no sign-up is needed to use the demo.
  • Nothing one visitor does changes what the next one sees.