Alama otomatiki za lm-eval-harness katika kazi za lugha nyingi. Juu zaidi ni bora — gusa safu au seli kuingia ndani.
Wastani wa alama katika modeli zote
Kiongozi kwa wastani wa alama (chini ya modeli 3)
Wastani wa chini zaidi miongoni mwa modeli zinazohitimu
Wastani wa juu zaidi miongoni mwa modeli zinazohitimu
| # | Modeli | Wast | Arabic | English | Spanish | French | Hausa | Igbo | Portuguese | Albanian | Swahili | Ukrainian | Urdu | Yoruba | Tazama mchanganuo |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| 1 | GPT-5 Nano 52/67 | 68.4 | 63.6 | 63.9 | 61.7 | 65.6 | 55.0 | -- | 78.0 | 85.1 | 76.1 | 80.1 | 59.5 | -- | |
| 2 | Functionary Swahili Large 29/67 | 67.7 | 69.4 | 83.7 | 62.8 | 76.8 | 46.1 | -- | 69.3 | 85.8 | 89.9 | 75.2 | 59.1 | -- | |
| 3 | Functionary Swahili Mini 22/67 | 62.2 | 43.4 | 23.6 | -- | 54.3 | 50.8 | -- | 75.8 | 85.4 | 85.9 | 91.8 | 67.2 | -- | |
| 4 | GPT-oss-120B | 59.1 | 38.9 | 66.8 | 43.6 | 58.7 | 50.2 | 48.0 | 77.4 | 87.3 | 74.1 | 81.6 | 47.1 | 46.7 | |
| 5 | GLM 5 12/67 | 57.0 | -- | 53.2 | 31.8 | 84.9 | -- | -- | -- | -- | 79.4 | -- | -- | -- |
Wastani wa alama katika kazi ambazo modeli zote mbili zilipiga
Tofauti ya alama — pau kulia = GPT-5 Nano mbele, pau kushoto = Functionary Swahili Large mbele
GPT-5 Nano dhidi ya wastani wa modeli zote zingine — jumla ya kazi 52
Functionary Swahili Large dhidi ya wastani wa modeli zote zingine — jumla ya kazi 29
