CodeLlama 34B
Meta
Meta's specialized code generation model with 34B parameters built on Llama 2 architecture. Released August 2023, trained on 500 billion tokens of code and code-related data from January-July 2023. Features extended 100,000 token context window for handling large codebases. Supports Python, C++, Java, PHP, TypeScript/JavaScript, C#, and Bash. Three variants: Base (code completion), Python (Python-specialized), Instruct (instruction-following). Achieves 53.7% HumanEval and 56.2% MBPP. Trained on Meta's Research Super Cluster requiring 400K GPU hours on A100-80GB hardware across all 9 CodeLlama models. Open weights enable customization and self-hosting.
Strengths
- Extended 100K context window handles entire large codebases
- Specialized for code across 7 major languages (Python, C++, Java, PHP, TS, C#, Bash)
- Three variants optimize for completion, Python, or instruction-following
- Trained on 500B tokens of code-specific data (Jan-July 2023)
- 53.7% HumanEval and 56.2% MBPP demonstrate strong code generation
- Llama 2 Community License enables research and commercial use
- Open weights enable customization and self-hosting
Caveats
- Superseded by newer code models (Qwen 2.5 Coder, GPT-4o, Claude)
- 34B size requires significant GPU memory
- Training data cutoff July 2023 - missing newer language features
- No multimodal or vision capabilities
- Consider newer models for state-of-the-art code generation
- Vendor
- Meta
- Context Window
- 100,000 tokens
- Max Output
- 4,096 tokens
- License
- Llama 2 Community License
- Release Date
- 2023-08-24
- Modalities
- text
Capabilities
Pricing
Open weights - free to download. Requires GPU. API providers typically $0.35-$0.70 per 1M input tokens