🏆 ACoPDoc Leaderboard

About the benchmark

ACoPDoc tests OCR and layout systems on Armenian printed documents beyond newspapers: legal acts, government decisions, court rulings, dictionaries, textbooks, literature, religious texts, contracts and reports. It has 200 pages (10 pages from each of 20 source documents), mixing born-digital and scanned PDFs.

Settings

• Structure-aware — the model outputs whole text lines; its own line segmentation is scored as-is.
• Structure-independent — the model outputs words; words are grouped into lines by a fixed heuristic before scoring.
• With postprocessing — only bounding-box geometry is repaired before scoring; recognized text is never edited.
• Filters — exclude non-Armenian text, graphics & headers, or both, from scoring.
• Fair comparison — each model uses the configuration that performed best on ACoPDoc.
• ocarm-v0* / ocarm-v1* — OCArm, an OCR pipeline built for historical Armenian periodicals (see the dataset card).

DeepSeek-OCR, Qwen-VL, Chandra and Kimi were also evaluated but scored 50%+ CER and are not listed.

✉️ How to submit results

1. Evaluate your model with the kit in the dataset's evaluation_kit/ folder.
2. Open this Space's Files tab, click results.csv → Edit.
3. Add one row per setting: postprocessing (without / with), structure (aware / independent), document (Overall or a document ID), model, then the four CER values in percent.
4. Click Open as a pull request and include a link to your model and your evaluation output.

The maintainers review the numbers and merge; the leaderboard updates automatically.

postprocessing,structure,document,model,cer_no_filter,cer_filter_non_armenian,cer_filter_graphics_headers,cer_filter_all
without,independent,Overall,my-ocr-model,1.20,1.05,1.18,1.02
without,independent,Arlis,my-ocr-model,0.40,0.40,0.40,0.40