RedPajama
Open Source
Datasets
Open reproduction of LLaMA training data
About
RedPajama is Together AI's open dataset reproducing the LLaMA training mix, later expanded to RedPajama-V2 with over 30 trillion filtered tokens across five languages - a large, permissive corpus for open model pretraining.
Details
- Category
- Datasets
- License
- Apache-2.0 (code)