DBRX
Open Weights
Databricks
Databricks' open MoE model (March 2024), a state-of-the-art open LLM at release with 132B total parameters and 36B active per input via a fine-grained mixture-of-experts. Pretrained on 12T tokens of text and code, DBRX serves a 32K context window under the Databricks Open Model License.
Strengths
- State-of-the-art open LLM at release
- Fine-grained MoE: 132B total with 36B active parameters
- Pretrained on 12T tokens of text and code
- Efficient inference relative to dense models of similar quality
- Open weights for self-hosting
Caveats
- 32K context smaller than newer open models
- Text-only (no vision or audio input)
- Superseded by newer open MoEs on many benchmarks
- Vendor
- Databricks
- Context Window
- 32,768 tokens
- License
- Databricks Open Model License
- Release Date
- 2024-03-27
- Modalities
- text
Capabilities
Vision
Audio
Video
Tool Use
Resources
Pricing
Open weights under the Databricks Open Model License - free to self-host; also available via the Databricks and Mosaic AI platforms.