Text extraction from images looks like a single action but involves half a dozen distinct stages. Knowing them tells you exactly why a given image fails.
Extracting text from an image is one of those tasks that feels like it should be simple. The words are right there. But to software, an image is only a grid of coloured values — there is no text in it at all until an algorithm constructs the interpretation. This article looks at how that construction works and what it implies for real use.
The fundamental gap
Human reading is so automatic that we forget how much processing it involves. We locate text within a scene, separate it from background, identify individual letterforms across enormous typographic variation, group them into words, and use context to resolve ambiguity — all effortlessly and in parallel.
Text extraction software must do all of this explicitly, in stages, from pixel values alone. Every stage can fail, and each failure mode looks different in the output.
Stage one: normalising the image
The first task is to reduce visual variability. Colour is usually discarded, since it rarely helps recognition and triples the data. Noise and speckles from scanning or sensor grain are filtered out. Contrast is normalised.
Then comes binarisation: every pixel is classified as either ink or background. Naively this uses a single global threshold, but that fails badly on unevenly lit photographs where one side of a page is brighter than the other. Adaptive thresholding computes a local threshold for each region instead, which is why modern engines handle phone photos far better than older ones.
Stage two: geometry correction
Text lines must be roughly horizontal for later stages to work. The engine estimates the dominant skew angle — often by projecting ink density across candidate angles and choosing the one producing the sharpest line peaks — and rotates the image to correct it.
Perspective distortion from photographing a page at an angle is harder, because letterforms themselves are warped rather than merely tilted. Some engines attempt to detect page edges and flatten the perspective; others simply perform worse. Capturing straight on avoids the problem entirely.
Stage three: layout analysis
The engine must decide which parts of the page are text and which are photographs, rules, tables, or decoration. It must then determine reading order — a genuinely hard problem for magazine layouts, newspapers, invoices with side-by-side blocks, and anything multi-column.
Layout errors are a common and confusing failure mode: the characters are recognised correctly but delivered in the wrong order, so the output is a jumble of correctly spelled fragments. Cropping tightly to a single text block sidesteps this entirely, which is why the Image Cropper is a genuinely useful preparation step.
Stage four: segmentation
Within each text region, horizontal bands of ink are identified as lines. Lines are split into words using gap analysis — larger gaps indicate word boundaries, smaller ones inter-letter spacing. Words may then be split into individual glyphs.
Segmentation is where touching letters, ligatures, italic slant, and tight kerning cause trouble. Two adjacent characters merged into one blob will be classified as a single unlikely character. This is why low-resolution and over-compressed images fail: the compression artefacts bridge the gaps between letters.
Stage five: recognition
Historically, recognition compared each isolated glyph against stored templates, or measured geometric features — counting loops, detecting stems, locating intersections — and matched the feature vector against known letters. This worked adequately for clean, consistent print and poorly for anything else.
Contemporary engines, including the Tesseract engine used in browser-based OCR, use recurrent neural networks trained on enormous quantities of rendered and scanned text. Crucially, they process an entire line as a sequence rather than classifying isolated characters. This means the network sees context, handles proportional spacing naturally, and copes far better with imperfect segmentation. The accuracy improvement from this architectural change was substantial.
Stage six: linguistic correction
Raw recognition output is refined using dictionaries and statistical language models. The engine knows which character sequences are plausible in the target language and adjusts uncertain classifications accordingly.
This is why common words are read reliably from poor scans, and equally why serial numbers, product codes, unusual surnames, and random strings are noticeably less accurate — there is no linguistic context to lean on. It is also why selecting the correct language matters so much: an English model applied to French or Cyrillic text actively degrades results by pulling recognition toward the wrong vocabulary.
What this means for accuracy in practice
- Character height is the dominant factor. Roughly 20 to 30 pixels per character is the practical minimum; 300 DPI scanning reliably achieves this for body text.
- Contrast matters more than resolution beyond that point. Faded photocopies and grey-on-grey design defeat binarisation.
- Blur is fatal. Out-of-focus or motion-blurred captures destroy the edges segmentation depends on.
- Typeface matters. Clean body faces perform excellently; condensed, decorative, and script faces perform poorly.
- Handwriting is a different technology. Intelligent character recognition handles it, and even the best systems struggle with unconstrained cursive.
Putting it to work
Our Image to Text (OCR) tool runs the full pipeline inside your browser, so the image never leaves your device — which matters when the source is an invoice, a payslip, or medical correspondence. If your source is a document rather than an image, render the pages first with PDF to JPG. If the capture is rotated, straighten it with Rotate/Flip Image before recognition.
Then proofread. Recognition errors cluster in specific places — 0 versus O, 1 versus l, 5 versus S, decimal points, and proper nouns. Numbers deserve particular scrutiny, because a misread digit produces a plausible-looking wrong answer rather than obvious nonsense. Used with that discipline, text extraction turns hours of retyping into minutes of review.