Gemini 1.0 Pro
Zero-eval
#1BIG-Bench
by Google
About
Gemini 1.0 Pro is a language model developed by Google. The model shows competitive results across 9 benchmarks. Notable strengths include BIG-Bench (75.0%), MMLU (71.8%), WMT23 (71.7%). The model is available through 1 API provider. Released in 2024, it represents Google's latest advancement in AI technology.
Pricing Range
Input (per 1M)$0.50 -$0.50
Output (per 1M)$1.50 -$1.50
Providers1
Timeline
AnnouncedFeb 15, 2024
ReleasedFeb 15, 2024
Knowledge CutoffFeb 1, 2024
Specifications
License & Family
License
Proprietary
Benchmark Performance Overview
Performance metrics and category breakdown
Overall Performance
9 benchmarks
Average Score
48.4%
Best Score
75.0%
High Performers (80%+)
0Performance Metrics
Max Context Window
41.0KAvg Throughput
120.0 tok/sAvg Latency
0msTop Categories
general
51.4%
vision
47.9%
math
39.6%
Benchmark Performance
Top benchmark scores with normalized values (0-100%)
Ranking Across Benchmarks
Position relative to other models on each benchmark
BIG-Bench
Rank #1 of 3
#1Gemini 1.0 Pro
75.0%
#2Gemma 2 27B
74.9%
#3Gemma 2 9B
68.2%
MMLU
Rank #58 of 78
#55Llama 3.2 11B Instruct
73.0%
#56Qwen2.5-Coder 32B Instruct
75.1%
#57Claude 3 Haiku
75.2%
#58Gemini 1.0 Pro
71.8%
#59Gemma 2 9B
71.3%
#60Qwen2 7B Instruct
70.5%
#61GPT-3.5 Turbo
69.8%
WMT23
Rank #4 of 4
#1Gemini 1.5 Flash 8B
72.6%
#2Gemini 1.5 Flash
74.1%
#3Gemini 1.5 Pro
75.1%
#4Gemini 1.0 Pro
71.7%
EgoSchema
Rank #9 of 9
#6Gemini 2.0 Flash-Lite
67.2%
#7Qwen2.5-Omni-7B
68.6%
#8Nova Lite
71.4%
#9Gemini 1.0 Pro
55.7%
MMMU
Rank #49 of 52
#46DeepSeek VL2 Small
48.0%
#47Llama 3.2 11B Instruct
50.7%
#48DeepSeek VL2
51.1%
#49Gemini 1.0 Pro
47.9%
#50Phi-3.5-vision-instruct
43.0%
#51DeepSeek VL2 Tiny
40.7%
#52GPT-3.5 Turbo
0.0%
All Benchmark Results for Gemini 1.0 Pro
Complete list of benchmark scores with detailed information
BIG-Bench BIG-Bench benchmark | general | text | 0.75 | 75.0% | Unverified |
MMLU MMLU benchmark | general | text | 0.72 | 71.8% | Self-reported |
WMT23 WMT23 benchmark | general | text | 0.72 | 71.7% | Unverified |
EgoSchema EgoSchema benchmark | general | text | 0.56 | 55.7% | Self-reported |
MMMU MMMU benchmark | vision | multimodal | 0.48 | 47.9% | Unverified |
MathVista MathVista benchmark | math | text | 0.47 | 46.6% | Unverified |
MATH MATH benchmark | math | text | 0.33 | 32.6% | Unverified |
GPQA GPQA benchmark | general | text | 0.28 | 27.9% | Unverified |
FLEURS FLEURS benchmark | general | text | 6.40 | 6.4% | Unverified |