BLOOM 176B
BigScience
Pioneering open-access multilingual model with 176B parameters designed and built through collaboration of 1,000+ researchers. Released July 2022 as first multilingual LLM trained in complete transparency. Decoder-only Transformer trained on ROOTS corpus covering 46 natural languages and 13 programming languages (59 total). Trained for 117 days (nearly 4 months) on 416 A100 80GB GPUs via Jean Zay supercomputer (French government). Training used 1.6TB multilingual dataset (350B tokens) achieving 150 TFLOPs throughput. Largest publicly available open multilingual model. Released under Responsible AI License. Represents unprecedented international research collaboration in AI.
Strengths
- Largest publicly available open multilingual model (59 languages total)
- First multilingual LLM trained in complete transparency
- Unprecedented collaboration - 1,000+ researchers globally
- Trained on Jean Zay supercomputer achieving 150 TFLOPs throughput
- 46 natural languages + 13 programming languages
- Responsible AI License with ethical considerations
- Represents international scientific collaboration milestone
Caveats
- Released 2022 - outdated vs modern models (2024-2025)
- 2K context window extremely small (GPT-4o: 128K, Gemini: 2M)
- 176B size requires massive GPU resources (similar to GPT-3)
- Lower performance than current state-of-the-art models
- Consider newer multilingual models (Llama 3.1, Qwen, etc.)
- Vendor
- BigScience
- Context Window
- 2,048 tokens
- Max Output
- 2,048 tokens
- License
- Responsible AI License (RAIL)
- Release Date
- 2022-07-06
- Modalities
- text
Capabilities
Resources
Pricing
Open access (RAIL) - free. Requires massive GPU resources. API providers typically $1.50-$3.00 per 1M input tokens