BenchmarksFunctionary Swahili MiniAll tasks

Functionary Swahili Mini

22 tasks

Each row below is a single benchmark task this model was evaluated on. The Score column averages every metric the task reports (accuracy, F1, exact-match, etc.). Click a row to browse the individual questions and the model's responses.

Average

62.2

Score	Language	Task	Metrics
95.9	Ukrainian (Ukraine)	ukrainian_polywrite ukrainian open generation open_quality_score: 95.9sample_len: 154.0	open_quality_score: 95.9sample_len: 154.0
88.7	French (France)	french_sib200 french classification f1_macro: 88.7sample_len: 204.0	f1_macro: 88.7sample_len: 204.0
87.8	Ukrainian (Ukraine)	ukrainian_sib200 ukrainian classification f1_macro: 87.8sample_len: 204.0	f1_macro: 87.8sample_len: 204.0
87.7	Portuguese (Portugal)	portuguese_hatebr portuguese classification f1_macro: 87.7sample_len: 1400.0	f1_macro: 87.7sample_len: 1400.0
86.8	Arabic (Saudi Arabia)	arabic_sib200 arabic classification f1_macro: 86.8sample_len: 204.0	f1_macro: 86.8sample_len: 204.0
85.9	Swahili (Tanzania)	swahili_sib200 swahili classification f1_macro: 85.9sample_len: 204.0	f1_macro: 85.9sample_len: 204.0
85.4	Albanian (Albania)	albanian_sib200 albanian classification f1_macro: 85.4sample_len: 204.0	f1_macro: 85.4sample_len: 204.0
83.8	Urdu (Pakistan)	urdu_facttool_qa urdu claim f1_macro: 83.8sample_len: 160.0	f1_macro: 83.8sample_len: 160.0
82.6	Hausa (Nigeria)	hausa_sib200 hausa classification f1_macro: 82.6sample_len: 204.0	f1_macro: 82.6sample_len: 204.0
75.0	Urdu (Pakistan)	urdu_factcheckbench urdu claim f1_macro: 75.0sample_len: 387.0	f1_macro: 75.0sample_len: 387.0
73.9	Urdu (Pakistan)	urdu_bingcheck urdu claim f1_macro: 73.9sample_len: 102.0	f1_macro: 73.9sample_len: 102.0
73.6	French (France)	french_mgsm french math exact_match: 73.6sample_len: 250.0	exact_match: 73.6sample_len: 250.0
72.8	Urdu (Pakistan)	urdu_fake_news urdu classification f1_macro: 72.8sample_len: 300.0	f1_macro: 72.8sample_len: 300.0
70.5	Portuguese (Portugal)	portuguese_tweetsentbr portuguese classification f1_macro: 70.5sample_len: 2010.0	f1_macro: 70.5sample_len: 2010.0
69.8	Hausa (Nigeria)	hausa_afrixnli hausa nli f1_macro: 69.8sample_len: 600.0	f1_macro: 69.8sample_len: 600.0
69.2	Portuguese (Portugal)	portuguese_hate_speech portuguese classification f1_macro: 69.2sample_len: 851.0	f1_macro: 69.2sample_len: 851.0
37.7	English (US)	english_mmlu_pro english mmlu pro exact_match: 37.7sample_len: 2100.0	exact_match: 37.7sample_len: 2100.0
30.4	Urdu (Pakistan)	urdu_emotion_class urdu classification f1_macro: 30.4sample_len: 200.0	f1_macro: 30.4sample_len: 200.0
9.6	English (US)	english_belebele english mcq f1_macro: 9.6sample_len: 900.0	f1_macro: 9.6sample_len: 900.0
0.5	French (France)	french_fquad french qa exact_match: 0.5f1: 0.5sample_len: 400.0	exact_match: 0.5f1: 0.5sample_len: 400.0
0.0	Arabic (Saudi Arabia)	arabic_tydiqa arabic qa exact_match: 0.0f1: 0.0sample_len: 921.0	exact_match: 0.0f1: 0.0sample_len: 921.0
0.0	Hausa (Nigeria)	hausa_afriqa hausa qa exact_match: 0.0f1: 0.0sample_len: 300.0	exact_match: 0.0f1: 0.0sample_len: 300.0