Run the same images through every installed OCR engine and the full filtering pipeline, then compare what each one extracted and caught.

OCR Engine Benchmark runs each uploaded image through every installed OCR engine and then through the full image-analysis pipeline, so you can see which engine reads your documents best before you commit an appliance to it.

What do I need?

Licence: Any
Role: admin_user

Prerequisites

The appliance runs with CID_ENABLE_LAB=true. On a licensed build this page does not exist.
The OCR service is running, with at least one engine installed.
Representative sample images, each 10 MB or smaller.
Time: every image is run once per engine, sequentially, and OCR is slow on CPU.

Danger

This is a lab surface. The whole Testing sidebar group is gated on CID_ENABLE_LAB and is hidden on every licensed build, so a customer appliance has no OCR Benchmark page.

Screenshot pendingOCR Engine Benchmark with the test-images tile listing selected files and engine tags, beside the timing chart and per-engine comparison/assets/screenshots/ocr-benchmark@0.4.png

Run a benchmark

Open the page

Select Testing → OCR Benchmark.

The Test images tile shows the upload zone; the results column shows No results yet.

Check which engines will be compared

Read the Engines to compare tags.

Each installed engine is listed. An engine the appliance knows about but does not have shows under Not installed, with its install command in a code block — that command is run on the host, not from the dashboard.

Add the images

Drop files on Click or drag to upload, or select it to browse. Add as many as you want.

The hint reads PNG, JPG, WEBP · max 10MB; the validator also accepts GIF, BMP, TIFF and ICO. A rejected file raises a toast naming the file and the reason.

Run it

Select Run benchmark.

Progress is reported as toasts, one per image and engine — {file}: Testing {engine}… — and the button reads Running benchmark… until every combination has finished. Each image is analysed once per engine, so the run takes engines × images passes.

Compare the engines

Read the timing chart and the per-engine comparison below it.

Each engine reports the text it extracted, the number of entities found, the pipeline's action and the processing time, with the redacted image beside it.

Keep the numbers

Select Export JSON.

A file named ocr-benchmark-<timestamp>.json downloads, carrying the summary, the engines compared and the full per-image result. There is no CSV or PDF export on this page.

Reading the comparison

The benchmark runs the full filtering pipeline, not OCR alone: one OCR pass, then regex and model detection, identity-layout extraction, word-level box matching and redaction. So a difference between two engines is a difference in what the gateway would actually have caught, not only in the text extracted.

What to compare	Why it matters
Extracted text	An engine that misreads a digit produces a national ID that no pattern matches
Entity count	The count the pipeline derived from that text — this is the number that decides masking
Action	The verdict the pipeline reached for that engine's reading of the image
Processing time	OCR dominates image-analysis latency; a slower engine costs on every uploaded image
Redacted image	Whether the boxes landed on the right words

Warning

An engine is marked as failed when OCR produced nothing, even though the API returns success with an allow action for that case. Read a green result with zero entities as "this engine found nothing", and check the extracted text before concluding the image is clean.

Fields reference

Control	Effect
Upload zone	Accepts multiple images. Each is validated for type and a 10 MB size cap in the browser
Clear	Removes every selected image and the current results
Engines to compare	The engines the OCR service reports as available. Not selectable — every available engine is run
Run benchmark	Runs every image against every available engine, sequentially
Export JSON	Downloads the full result set. Disabled until a run has finished

Verify

Every engine tag under Engines to compare appears as a column or row in the results — an engine that is missing from the results failed on that image.
An image containing an obvious entity — a printed email address — yields a non-zero entity count on at least one engine, and the redacted image has the value covered.
Export JSON produces a file whose enginesTested list matches the tags on the page.

Note

A benchmark run writes an image-analysis detection per image and engine, so runs surface in All detections under the Image analysis source and the event type IMAGE_ANALYSIS_DETECTION. Nothing on this page is stored as a benchmark record — leave the page and the comparison is gone unless you exported it.

If it fails

The page is missing from the sidebar, or its requests 404 — the build is not a lab build. CID_ENABLE_LAB is not true.
Engines to compare shows Loading engines… and never resolves — the gateway cannot reach the OCR service.
Everything is listed under Not installed — the OCR image was built without its models. On an air-gapped appliance this is the failure mode to look for first: an image can build cleanly and still hold no detection or recognition model.
{file}: Invalid file type: {type}. Supported: JPEG, PNG, GIF, WebP, BMP, TIFF, ICO — the file is not an image the pipeline accepts.
{file}: File too large: {n}MB. Max: 10MB — downscale the image.
One engine is far slower than the rest and times out — image analysis allows two minutes per call. A CPU-only appliance running a heavy engine can exceed it on a dense page.

Next steps

Redact a file — the same pipeline applied to a document you need to clear.
ML engine — check the text detector behind the OCR pass.
All detections — see the detections these runs produced.

Benchmark the OCR engines