Image to Text: How OCR Works and How to Get Clean Output

Input quality dominates every other factor. The long-standing scanning standard of 300 DPI exists because at that resolution, ordinary body text renders with enough pixels per character for an engine to distinguish similar shapes; the practical floor is a character height of roughly 20 to 30 pixels. Below that, an e and a c, or an 8 and a B, collapse into the same smudge and the engine starts guessing.

After resolution, the big three are contrast, skew, and noise. Engines want dark text on a light, even background; a shadow falling across half a page photographed on a desk creates two exposure zones and a band of errors along the boundary. Skew — the page rotated a few degrees — degrades the line-detection step that everything downstream depends on. Noise covers the rest: JPEG artifacts from heavy compression, paper texture, bleed-through from the reverse side of thin paper, and the moiré patterns that appear when you photograph a screen instead of taking a screenshot.

This is why screenshots are the best-case input for OCR and phone photos of paper are the worst. A screenshot is perfectly flat, perfectly lit, and pixel-aligned. A phone photo adds perspective distortion, uneven lighting, focus falloff at the edges, and lens curvature — every one of which the engine has to overcome before it reads a single letter.

An OCR pipeline starts with preprocessing that has nothing to do with letters. The image is converted to grayscale and then binarized — every pixel forced to pure black or pure white — using a threshold the software chooses from the image's brightness distribution, often adaptively so a shadowed corner gets a different threshold than a bright center. The page is then deskewed by detecting the dominant text angle and rotating it flat.

Next comes layout analysis and segmentation. The engine finds blocks of text and separates them from images and rules, splits blocks into lines, and lines into words. Older engines went one step further and cut words into individual characters, then classified each one by comparing its shape features — strokes, loops, endpoints — against trained patterns. That character-by-character approach is exactly where broken or touching glyphs caused cascading errors.

Modern engines instead feed whole line images to a neural network — typically a recurrent architecture trained on millions of text-line samples — that outputs a character sequence directly, without ever cutting the line into pieces. The network learns context along the way: having read the pixels of a slightly smeared word, it knows that t-h-e is overwhelmingly more likely than t-l-i-e. A language layer weighs alternatives the same way, which is how engines resolve visually identical cases — rn versus m, l versus 1 — that pure shape-matching cannot. Engines also report a confidence score per word, which is why good tools can flag their own shaky guesses.

That contextual strength has a flip side worth remembering: the language layer only helps where language exists. Serial numbers, phone numbers, prices, and reference codes give it nothing to lean on, so digit strings carry the highest error rates in any extraction — a fact that should shape how you proofread.

A minute of preparation routinely cuts the error count more than switching engines would. The principle behind every item below is the same: do part of the preprocessing yourself, with judgment the software does not have. You know which region of the page matters, which shadow is a shadow rather than ink, and whether the document was photographed at an angle — the engine has to infer all of it from pixels.

Work down this list before uploading:

The classic confusable glyphs remain the largest error class: 0 and O, 1 and l and I, 5 and S, 8 and B, rn and m. In prose, the language layer silently fixes most of them. In numbers, nothing fixes them — an invoice total, an IBAN, a part number, or a phone number can come through plausible-looking and wrong. The efficient proofreading strategy follows directly: skim the prose, but verify every digit string against the source, character by character, because those errors are both the most likely and the most costly.

Layout is the second weak spot. Multi-column pages can be read straight across the columns, interleaving two articles into nonsense; tables lose their cell structure and come out as word soup in roughly reading order; and text wrapped around images may be reassembled out of sequence. When a document is column-heavy, cropping and extracting one column at a time is often faster than untangling a merged result.

Handwriting is its own tier of difficulty. Neatly printed block letters extract passably; cursive remains unreliable for general engines because segmentation and letterforms vary per writer, per word. Stylized display fonts, dense watermarks behind text, and colored text on colored backgrounds all push error rates up for the same underlying reason — they widen the gap between what the engine was trained on and what it is being shown.

A practical habit that catches most of what slips through: paste the extracted text next to the source image and read the output while glancing at the original, rather than reading the output alone. Errors that survive OCR are by definition plausible-looking, so reading the extraction in isolation feels clean even when it is not. Side-by-side review takes marginally longer and catches the substitutions that matter — a wrong digit in a total, a swapped letter in a name — before they propagate into a spreadsheet or a contract.

The everyday case is rescuing text trapped in a screenshot: an error message posted as an image, a quote from a video slide, a paragraph in a PDF that disables copying. Ten seconds of upload-and-copy replaces retyping, with the bonus that the extracted text becomes searchable later. The break-even point is lower than most people assume — anything past a sentence or two is usually faster to extract and proofread than to type, and unlike typing, extraction does not introduce its own transposition errors into names and addresses.

Beyond that, a few workflows come up constantly: