Gemini 1.5 Flash 8B
Multimodal
Zero-eval
#1FLEURS
#3XSTest
#3WMT23
by Google
About
Gemini 1.5 Flash 8B is a multimodal language model developed by Google. It achieves strong performance with an average score of 60.5% across 13 benchmarks. It excels particularly in XSTest (92.6%), FLEURS (86.4%), Natural2Code (75.5%). With a 1.1M token context window, it can handle extensive documents and complex multi-turn conversations. The model is available through 1 API provider. As a multimodal model, it can process and understand text, images, and other input formats seamlessly. Released in 2024, it represents Google's latest advancement in AI technology.
Pricing Range
Input (per 1M)$0.07 -$0.07
Output (per 1M)$0.30 -$0.30
Providers1
Timeline
AnnouncedMar 15, 2024
ReleasedMar 15, 2024
Knowledge CutoffOct 1, 2024
Specifications
Capabilities
Multimodal
License & Family
License
Proprietary
Benchmark Performance Overview
Performance metrics and category breakdown
Overall Performance
13 benchmarks
Average Score
60.5%
Best Score
92.6%
High Performers (80%+)
2Performance Metrics
Max Context Window
1.1MAvg Throughput
150.0 tok/sAvg Latency
0msTop Categories
general
67.2%
vision
60.0%
code
58.2%
math
48.7%
Benchmark Performance
Top benchmark scores with normalized values (0-100%)
Ranking Across Benchmarks
Position relative to other models on each benchmark
XSTest
Rank #3 of 3
#1Gemini 1.5 Flash
97.0%
#2Gemini 1.5 Pro
98.8%
#3Gemini 1.5 Flash 8B
92.6%
FLEURS
Rank #1 of 5
#1Gemini 1.5 Flash 8B
86.4%
#2Gemini 1.5 Flash
9.6%
#3Gemini 1.5 Pro
6.7%
#4Gemini 1.0 Pro
6.4%
Natural2Code
Rank #6 of 8
#3Gemini 1.5 Flash
79.8%
#4Gemma 3 12B
80.7%
#5Gemma 3 27B
84.5%
#6Gemini 1.5 Flash 8B
75.5%
#7Gemma 3 4B
70.3%
#8Gemma 3 1B
56.0%
WMT23
Rank #3 of 4
#1Gemini 1.5 Flash
74.1%
#2Gemini 1.5 Pro
75.1%
#3Gemini 1.5 Flash 8B
72.6%
#4Gemini 1.0 Pro
71.7%
Video-MME
Rank #4 of 5
#1Gemini 1.5 Flash
76.1%
#2Gemini 1.5 Pro
78.6%
#3Gemini 2.5 Pro
84.8%
#4Gemini 1.5 Flash 8B
66.2%
#5Phi-4-multimodal-instruct
55.0%
All Benchmark Results for Gemini 1.5 Flash 8B
Complete list of benchmark scores with detailed information
XSTest XSTest benchmark | general | text | 0.93 | 92.6% | Self-reported |
FLEURS FLEURS benchmark | general | text | 86.40 | 86.4% | Self-reported |
Natural2Code Natural2Code benchmark | code | text | 0.76 | 75.5% | Self-reported |
WMT23 WMT23 benchmark | general | text | 0.73 | 72.6% | Self-reported |
Video-MME Video-MME benchmark | vision | video | 0.66 | 66.2% | Self-reported |
MMLU-Pro MMLU-Pro benchmark | general | text | 0.59 | 58.7% | Self-reported |
MATH MATH benchmark | math | text | 0.59 | 58.7% | Self-reported |
MRCR MRCR benchmark | general | text | 0.55 | 54.7% | Self-reported |
MathVista MathVista benchmark | math | text | 0.55 | 54.7% | Self-reported |
MMMU MMMU benchmark | vision | multimodal | 0.54 | 53.7% | Self-reported |
Showing 1 to 10 of 13 benchmarks
Resources