ShieldGemma 2
Open Weights
Google's image safety classifier: a 4 billion parameter model trained on Gemma 3's 4B instruction-tuned checkpoint. It takes an image plus a policy definition in the prompt and returns the probability of a 'Yes' or 'No' token, so a caller can set its own threshold. Google's card supplies three policies out of the box: sexually explicit content, dangerous content, and violence/gore. It is recommended as an input filter in front of vision-language models or as an output filter on image generation systems.
Strengths
- Classifies images against a safety policy supplied in the prompt, rather than against a fixed built-in label set
- Returns both 'Yes' and 'No' token probabilities, so callers can pick their own threshold and compute calibration metrics
- Ships three ready-made policies: sexually explicit content, dangerous content, and violence/gore
- Works at either end of a pipeline: input filter for vision-language models, output filter for image generation systems
- Open weights on a 4B Gemma 3 checkpoint, so it can be self-hosted and fine-tuned for a specific policy
- Leads Google's internal benchmark on all three harm types against LlavaGuard 7B, GPT-4o mini and the base Gemma-3-4B-IT
Caveats
- Violence and gore is its weakest category at 80.3 precision / 90.4 recall / 85.0 F1, well below the 93.7 F1 it reaches on dangerous content
- Training data covered only three harm types and was English only, so other languages and harms are out of scope
- Google's card warns the model is 'highly sensitive to the specific user-provided description of safety principles' and 'might perform unpredictably under conditions that require a good understanding of language ambiguity and nuance'
- The published numbers come from an internal synthetic benchmark of roughly 500 examples per policy; Google states 'the training and evaluation data might not be representative of real-world scenarios'
- Distributed under the Gemma Terms of Use and Google's prohibited use policies, not an OSI-approved open-source licence, and Hugging Face access is gated behind accepting that licence
- No release date, context window or output limit is stated on the model card
- Vendor
- License
- Gemma Terms of Use
- Modalities
- imagetext
Capabilities
Vision
Audio
Video
Tool Use
Resources
Pricing
Open weights, no vendor pricing published. The Hugging Face repository is publicly accessible but gated: you must review and agree to Google's usage license before downloading files.