dots.ocr
Open Weights
RedNote
dots.ocr is RedNote hi lab's open-weight multilingual document parser, released 30 July 2025 under the MIT licence. A single vision-language model built on a 1.7B-parameter LLM handles layout detection and content recognition together, switching task by prompt instead of chaining separate detection and recognition stages. It returns one JSON object per page in which each element carries a bounding box, a category drawn from an eleven-class layout taxonomy, and its text - formulas as LaTeX, tables as HTML, everything else as Markdown - sorted into human reading order. Prompt modes also cover layout-detection-only, plain OCR and grounded OCR within a supplied bbox.
Strengths
- MIT licence, with weights, a Docker image and a parser CLI all published by the vendor
- A single vision-language model on a 1.7B-parameter LLM replaces a multi-model detect-then-recognise pipeline; task selection is by prompt alone
- Leads the OmniDocBench end-to-end table column at 88.6 TEDS (EN) and 89.0 (ZH), with overall edit distance 0.125 EN and 0.160 ZH
- The in-house dots.ocr-bench spans 1,493 PDF images across 100 languages, where it scores 0.177 overall edit distance against Gemini 2.5 Pro's 0.251
- Four prompt modes: parse-all, layout-detection-only, plain OCR, and grounded OCR restricted to a caller-supplied bounding box
- vLLM deployment instructions plus a vendor-hosted live demo at dotsocr.xiaohongshu.com
Caveats
- The vendor states the model is not yet perfect for high-complexity tables and formulas, and formula recognition is its weakest OmniDocBench row at 0.329 edit distance (EN) and 0.416 (ZH) - behind several pipeline tools on the same table
- Pictures embedded in documents are not parsed at all; the Picture category's text field is simply omitted
- The vendor describes it as not yet optimised for high-throughput processing of large PDF volumes despite the compact LLM
- Continuous special characters such as ellipses or underscores can send generation into an endless repetition loop, requiring a fallback prompt
- Requires trust_remote_code, and the weights must be saved to a directory name without periods (DotsOCR, not dots.ocr) as a temporary workaround pending transformers integration
- Vendor
- RedNote
- License
- MIT
- Release Date
- 2025-07-30
- Modalities
- textimage
| Benchmark | Score | Max |
|---|---|---|
| OmniDocBench Table TEDS (EN) | 88.6 | 100.0 |
| OmniDocBench Table TEDS (ZH) | 89.0 | 100.0 |
| dots.ocr-bench Table TEDS | 79.2 | 100.0 |
Capabilities
Vision
Audio
Video
Tool Use