Gemma 3 27B
Multimodal
Zero-eval
#1MMMU (val)
#1WMT24++
#1BIG-Bench Extra Hard
+5 more
by Google
About
Gemma 3 27B is a multimodal language model developed by Google. It achieves strong performance with an average score of 65.4% across 26 benchmarks. It excels particularly in GSM8k (95.9%), IFEval (90.4%), MATH (89.0%). The model shows particular specialization in math tasks with an average performance of 78.2%. It supports a 262K token context window for handling large documents. The model is available through 2 API providers. As a multimodal model, it can process and understand text, images, and other input formats seamlessly. It's licensed for commercial use, making it suitable for enterprise applications. Released in 2025, it represents Google's latest advancement in AI technology.
Pricing Range
Input (per 1M)$0.10 -$0.11
Output (per 1M)$0.20 -$0.20
Providers2
Timeline
AnnouncedMar 12, 2025
ReleasedMar 12, 2025
Specifications
Training Tokens14.0T
Capabilities
Multimodal
License & Family
License
Gemma
Benchmark Performance Overview
Performance metrics and category breakdown
Overall Performance
26 benchmarks
Average Score
65.4%
Best Score
95.9%
High Performers (80%+)
8Performance Metrics
Max Context Window
262.1KAvg Throughput
33.0 tok/sAvg Latency
0msTop Categories
math
78.2%
factuality
74.9%
code
73.4%
vision
71.6%
general
53.5%
Benchmark Performance
Top benchmark scores with normalized values (0-100%)
Ranking Across Benchmarks
Position relative to other models on each benchmark
GSM8k
Rank #7 of 46
#4Claude 3.5 Sonnet
96.4%
#5Claude 3.5 Sonnet
96.4%
#6Llama 3.1 405B Instruct
96.8%
#7Gemma 3 27B
95.9%
#8Qwen2.5 32B Instruct
95.9%
#9Qwen2.5 72B Instruct
95.8%
#10DeepSeek-V2.5
95.1%
IFEval
Rank #5 of 37
#2Llama 3.3 70B Instruct
92.1%
#3Nova Pro
92.1%
#4Claude 3.7 Sonnet
93.2%
#5Gemma 3 27B
90.4%
#6Gemma 3 4B
90.2%
#7Kimi K2 Instruct
89.8%
#8Nova Lite
89.7%
MATH
Rank #4 of 63
#1Gemini 2.0 Flash
89.7%
#2o1
96.4%
#3o3-mini
97.9%
#4Gemma 3 27B
89.0%
#5Gemini 2.0 Flash-Lite
86.8%
#6Gemini 1.5 Pro
86.5%
#7o1-preview
85.5%
HumanEval
Rank #24 of 62
#21GPT-4.5
88.0%
#22Claude 3.5 Haiku
88.1%
#23o1
88.1%
#24Gemma 3 27B
87.8%
#25GPT-4o mini
87.2%
#26GPT-4 Turbo
87.1%
#27Qwen2.5 72B Instruct
86.6%
BIG-Bench Hard
Rank #4 of 21
#1Gemini 1.5 Pro
89.2%
#2Claude 3.5 Sonnet
93.1%
#3Claude 3.5 Sonnet
93.1%
#4Gemma 3 27B
87.6%
#5Claude 3 Opus
86.8%
#6Gemma 3 12B
85.7%
#7Gemini 1.5 Flash
85.5%
All Benchmark Results for Gemma 3 27B
Complete list of benchmark scores with detailed information
GSM8k GSM8k benchmark | math | text | 0.96 | 95.9% | Self-reported |
IFEval IFEval benchmark | code | text | 0.90 | 90.4% | Self-reported |
MATH MATH benchmark | math | text | 0.89 | 89.0% | Self-reported |
HumanEval HumanEval benchmark | code | text | 0.88 | 87.8% | Self-reported |
BIG-Bench Hard BIG-Bench Hard benchmark | general | text | 0.88 | 87.6% | Self-reported |
DocVQA DocVQA benchmark | vision | multimodal | 0.87 | 86.6% | Self-reported |
AI2D AI2D benchmark | general | text | 0.84 | 84.5% | Self-reported |
Natural2Code Natural2Code benchmark | code | text | 0.84 | 84.5% | Self-reported |
ChartQA ChartQA benchmark | general | multimodal | 0.78 | 78.0% | Self-reported |
Global-MMLU-Lite Global-MMLU-Lite benchmark | general | text | 0.75 | 75.1% | Self-reported |
Showing 1 to 10 of 26 benchmarks
Resources