Gemini 1.5 Flash
Multimodal
Zero-eval
#2XSTest
#2WMT23
#2PhysicsFinals
+5 more
by Google
About
Gemini 1.5 Flash is a multimodal language model developed by Google. It achieves strong performance with an average score of 66.8% across 22 benchmarks. It excels particularly in XSTest (97.0%), HellaSwag (86.5%), GSM8k (86.2%). With a 1.1M token context window, it can handle extensive documents and complex multi-turn conversations. The model is available through 1 API provider. As a multimodal model, it can process and understand text, images, and other input formats seamlessly. Released in 2024, it represents Google's latest advancement in AI technology.
Pricing Range
Input (per 1M)$0.15 -$0.15
Output (per 1M)$0.60 -$0.60
Providers1
Timeline
AnnouncedMay 1, 2024
ReleasedMay 1, 2024
Knowledge CutoffNov 1, 2023
Specifications
Capabilities
Multimodal
License & Family
License
Proprietary
Benchmark Performance Overview
Performance metrics and category breakdown
Overall Performance
22 benchmarks
Average Score
66.8%
Best Score
97.0%
High Performers (80%+)
5Performance Metrics
Max Context Window
1.1MAvg Throughput
150.0 tok/sAvg Latency
0msTop Categories
reasoning
86.5%
vision
69.2%
math
68.9%
code
67.7%
general
62.7%
Benchmark Performance
Top benchmark scores with normalized values (0-100%)
Ranking Across Benchmarks
Position relative to other models on each benchmark
XSTest
Rank #2 of 3
#1Gemini 1.5 Pro
98.8%
#2Gemini 1.5 Flash
97.0%
#3Gemini 1.5 Flash 8B
92.6%
HellaSwag
Rank #7 of 24
#4Qwen2 72B Instruct
87.6%
#5Command R+
88.6%
#6Claude 3 Sonnet
89.0%
#7Gemini 1.5 Flash
86.5%
#8Gemma 2 27B
86.4%
#9Claude 3 Haiku
85.9%
#10Llama 3.1 Nemotron 70B Instruct
85.6%
GSM8k
Rank #34 of 46
#31Phi-3.5-mini-instruct
86.2%
#32Jamba 1.5 Large
87.0%
#33Phi 4 Mini
88.6%
#34Gemini 1.5 Flash
86.2%
#35Qwen2.5-Coder 7B Instruct
83.9%
#36Qwen2 7B Instruct
82.3%
#37Granite 3.3 8B Instruct
80.9%
BIG-Bench Hard
Rank #7 of 21
#4Gemma 3 12B
85.7%
#5Claude 3 Opus
86.8%
#6Gemma 3 27B
87.6%
#7Gemini 1.5 Flash
85.5%
#8Claude 3 Sonnet
82.9%
#9Phi-3.5-MoE-instruct
79.1%
#10Claude 3 Haiku
73.7%
MGSM
Rank #18 of 31
#15Claude 3 Sonnet
83.5%
#16Qwen3 235B A22B
83.5%
#17Claude 3.5 Haiku
85.6%
#18Gemini 1.5 Flash
82.6%
#19Phi 4
80.6%
#20Claude 3 Haiku
75.1%
#21GPT-4
74.5%
All Benchmark Results for Gemini 1.5 Flash
Complete list of benchmark scores with detailed information
XSTest XSTest benchmark | general | text | 0.97 | 97.0% | Self-reported |
HellaSwag HellaSwag benchmark | reasoning | text | 0.86 | 86.5% | Self-reported |
GSM8k GSM8k benchmark | math | text | 0.86 | 86.2% | Self-reported |
BIG-Bench Hard BIG-Bench Hard benchmark | general | text | 0.85 | 85.5% | Self-reported |
MGSM MGSM benchmark | math | text | 0.83 | 82.6% | Self-reported |
Natural2Code Natural2Code benchmark | code | text | 0.80 | 79.8% | Self-reported |
MMLU MMLU benchmark | general | text | 0.79 | 78.9% | Self-reported |
MATH MATH benchmark | math | text | 0.78 | 77.9% | Self-reported |
Video-MME Video-MME benchmark | vision | video | 0.76 | 76.1% | Self-reported |
HumanEval HumanEval benchmark | code | text | 0.74 | 74.3% | Self-reported |
Showing 1 to 10 of 22 benchmarks