CogVideoX1.5-5B
Zhipu AI
Zhipu AI's open-weight text-to-video model, released 8 November 2024 as the newest generation of the CogVideoX family. It generates 1360x768 video at 16 frames per second in 5- or 10-second lengths, with frame counts of 16N+1 where N is at most 10 (default 81). Prompts must be English and are capped at 224 tokens. The vendor reports diffusers inference from 10 GB of VRAM in BF16 and from 7 GB with INT8, against 76 GB for the SAT BF16 path, and roughly 1000 seconds on a single A100 for a five-second clip. Image-to-video lives in a separate checkpoint, CogVideoX1.5-5B-I2V.
Strengths
- Open weights with a first-class Diffusers integration through CogVideoXPipeline
- 1360x768 output at 16 frames per second, up from 720x480 at 8 fps in CogVideoX-5B
- Generates 5- or 10-second clips using 16N+1 frames with N up to 10
- Runs from 10 GB VRAM in BF16 and from 7 GB with INT8 torchao quantisation via diffusers
- Supports BF16, FP16, FP32, FP8 and INT8 inference precisions
- Distributed on Hugging Face, ModelScope and WiseModel, with an SAT build for fine-tuning
Caveats
- The CogVideoX License is not open source: commercial use requires registering for a basic commercial licence, and that licence stops at 1 million visits per month
- English-only prompts capped at 224 tokens - the card says other languages "should be translated into English during prompt crafting with a larger model"
- Slow by the vendor's own figures: about 1000 seconds on a single A100 and 550 seconds on a single H100 for a five-second video, and 76 GB for SAT BF16
- Text-to-video only in this repository; image-to-video needs the separate CogVideoX1.5-5B-I2V checkpoint
- The card publishes no video-quality benchmark scores at all, only memory and inference-speed figures
- Vendor
- Zhipu AI
- Context Window
- 224 tokens
- License
- CogVideoX License
- Release Date
- 2024-11-08
- Modalities
- video
Capabilities
Pricing
Weights download free, but the CogVideoX License grants academic research use by default: "Users who wish to use the models for commercial purposes must register and obtain a basic commercial license". That basic commercial licence is free but capped, since "the number of service users (visits) for your commercial activities must not exceed 1 million visits per month"; above that a further commercial licence must be negotiated. Zhipu also sells larger hosted video models through QingYing and its API platform.