Run the current recognizer
Capture its exact code, settings, OCR environment, and—if enabled by policy—model identity and request configuration.
Level 3 · Method
Last modified: (dates/navigation repair). Content current as of: (last retained content commit; content verification date unavailable). Original content last edited: . Current claims have not been freshly reverified.
Build the expected result first; test a frozen candidate later
Development uses two freezes for different reasons. The reference freeze prevents the expected answer from moving to fit the recognizer. The candidate freeze makes the final evaluation independent of further correction.
Phase 1 · Reference freeze
Define “what is on these pages?” independently of the recognizer
Original PDFplus an optional companion source
↓
Reference author inspects pagestext, logical structure, appearance, and source locations
↓
Reviewer or owner resolves ambiguityrecords accepted meaning and explicit unknowns
↓
REFERENCE FREEZEversioned expected result; recognizer has not supplied the answer
↓
Develop or correct recognizerPDF is its input
↓
Compare recognizer output with referenceclassify omissions, false interpretations, and unsupported certainty
A companion Word or source file can help recover text or author intent, but it is not automatically authoritative. Objects added during PDF production, visual emphasis, page furniture, and differences between versions must be judged against the PDF. Disagreement is recorded and resolved rather than silently copied.
Human responsibility. People define and approve the expected semantic result. Code may assist extraction and diffing; it must not use the recognizer under test as the oracle.
Phase 2 · Bounded correction
Failures become development evidence
Compare complete outputs
Check semantic values, source locations, order, relationships, assertion state, unresolved content, and rendering—not just whether some expected text appears.
Classify the cause
Separate capture failure, detector failure, resolver policy, serialization, rendering, reference error, and genuinely ambiguous source.
Correct within a fixed budget
Change general methods and tests. A failure used to guide a correction is now development evidence; it cannot later count as unseen evaluation.
The current plan bounds this work to two audited correction cycles before candidate freeze. The purpose is to prevent indefinite tuning to one small document set.
Phase 3 · Candidate freeze
Reserve documents for independent evaluation
CANDIDATE FREEZEcode, settings, OCR, and model policy fixed
↓
Independent evaluator runs reserved documentsno corrective feedback enters the candidate
↓
Full-output reportomissions, false interpretations, unresolved items, operational failures
↓
Release decisionaccept stated scope, narrow scope, or record failure
If reserved-document results are used to change the system, those documents move into development evidence for the next candidate. They are not still described as unseen. The release statement names the evaluated scope and remaining limitations.
Metric boundary. A format validation pass, OCR score, detector confidence, or model self-rating is not a source-fidelity measurement. Fidelity requires comparison with the frozen reference and inspection of the whole output.
Roles and rationale
Different actors control different decisions
ActorControlsWhy
Reference authorExpected resultSeparates human interpretation of the source from implementation behavior.
Reference reviewer or ownerAmbiguity and freezePrevents one author’s uncertain choice from becoming an invisible requirement.
DeveloperRecognizer candidateChanges general code and tests against development evidence.
ScriptsCapture and comparisonMake repeatable checks cheap and retain exact evidence.
Language modelBounded adviceMay answer a narrow, recorded ambiguity under policy; never defines the oracle by itself.
Independent evaluatorFrozen candidateMeasures performance on material not used to correct that candidate.
Release ownerAcceptanceAccepts the stated scope and residual risk, or records failure.
Current program status · September 2026. Reference auditing and recognizer correction are active. General model routing and an interactive review queue are planned, not deployed. This dated note is separate from the permanent method above.