MPT-30B
MosaicML
MosaicML's (now Databricks) efficient 30B parameter decoder-style transformer pretrained from scratch on 1T tokens of English text and code. Features 8K token context window with support for extrapolation via ALiBi. Trained initially on 1T tokens using 2K sequences then additional 50B tokens of 8K sequences. First LLM trained on NVIDIA H100 GPUs. Outperforms originally published GPT-3 using ~1/6th the parameters. Competitive with LLaMa-30B and Falcon-40B on open-source benchmarks. Surpasses purpose-built models like StarCoder on HumanEval. Designed for easy deployment on single GPU - 1xA100-80GB (16-bit) or 1xA100-40GB (8-bit). Features FlashAttention for efficient inference. Released under Apache 2.0. Available in base, instruct, and chat variants.
Strengths
- Outperforms original GPT-3 using only 1/6th the parameters (30B vs 175B)
- 8K context window extended via ALiBi positional embeddings
- First LLM trained on NVIDIA H100 GPUs - cutting-edge at release
- Designed for single-GPU deployment - fits 1xA100-80GB or 1xA100-40GB
- Strong coding performance - surpasses StarCoder on HumanEval
- FlashAttention enables efficient inference
- Apache 2.0 license - fully permissive commercial use
Caveats
- Released June 2023 - superseded by newer models (Llama 3.1, Qwen 2.5)
- 8K context smaller than modern models (128K-2M)
- Competitive but not state-of-the-art vs latest open models
- Training cutoff mid-2023 - missing recent developments
- Consider Llama 3.1 70B or Qwen 2.5 for current best open-source
- Vendor
- MosaicML
- Context Window
- 8,000 tokens
- Max Output
- 2,048 tokens
- License
- Apache-2.0
- Release Date
- 2023-06-22
- Modalities
- text
Capabilities
Resources
Pricing
Open source (Apache 2.0) - free. Fits on 1xA100-80GB (16-bit) or 1xA100-40GB (8-bit). API providers typically $0.40-$0.80 per 1M input tokens