granite-docling-258M
Open Weights
IBM
Granite Docling is IBM Research's 258M-parameter image-text-to-text model for document conversion, released 17 September 2025 under Apache 2.0. It builds on the Idefics3 architecture, replacing the vision encoder with siglip2-base-patch16-512 and the language model with a Granite 165M LLM, and emits DocTags that convert directly into DoclingDocuments. Its instruction set covers full-page conversion plus targeted chart-to-table, formula-to-LaTeX, code and OTSL table extraction, along with bbox-guided region inference and question answering about document structure. It is intended to be used through the Docling library.
Strengths
- Apache 2.0 licence, with an MLX build for Apple Silicon and an untied-weights branch for vLLM published alongside the main weights
- 258M parameters (siglip2-base-patch16-512 encoder plus a Granite 165M LLM) keeps it runnable on modest hardware
- Emits DocTags that convert into DoclingDocuments and onward to Markdown or HTML through docling-core
- Instruction set covers full-page conversion, chart-to-table, formula-to-LaTeX, code-to-text and OTSL table extraction
- Flexible inference modes including bbox-guided region inference and document-element QA about the presence and ordering of elements
- Improves on its smoldocling-256m-preview predecessor across every published row: layout MAP 0.27 vs 0.23, full-page OCR F1 0.84 vs 0.80, OCRBench 500 vs 338
- One-line use through the docling CLI, which downloads the model and converts files, URLs or whole directories
Caveats
- The card lists English as the only supported language; Japanese, Arabic and Chinese support is explicitly labelled experimental
- Absolute accuracy stays modest for a 258M model: OCRBench 500, MMStar 0.30, and full-page OCR edit distance still 0.45
- IBM recommends using it only as part of the Docling library and pairing it with Granite Guardian for safety, rather than treating it as a general vision model
- Current vLLM releases (including 0.10.2) break on the model's tied weights; you must serve the separate untied revision
- Older GPUs without bfloat16 support, such as a Colab T4, emit only exclamation marks unless dtype is forced to float32; IBM also flags that smaller models may be more susceptible to hallucination
- Vendor
- IBM
- License
- Apache-2.0
- Release Date
- 2025-09-17
- Modalities
- textimage
| Benchmark | Score | Max |
|---|---|---|
| OCRBench | 500.0 | 1000.0 |
| MMStar | 0.3 | 1.0 |
| Full Page OCR F1 | 0.8 | 1.0 |
| Layout MAP | 0.3 | 1.0 |
Capabilities
Vision
Audio
Video
Tool Use