Crossing Linguistic Horizons

Finetuning and Comprehensive Evaluation of Vietnamese Large Language Models

Randomized-Choice Knowledge Leaderboard

Models AC ↑ F1 ↑ AR ↑ ECE ↓ A@10 ↑
Our 70B 0.76 ± 0.02 0.76 ± 0.02 0.78 ± 0.01 0.14 ± 0.02 0.94 ± 0.04
Our 13B 0.62 ± 0.02 0.62 ± 0.02 0.61 ± 0.02 0.15 ± 0.02 0.67 ± 0.07
Our 7B 0.45 ± 0.02 0.36 ± 0.02 0.57 ± 0.02 0.10 ± 0.02 0.45 ± 0.07
LLaMa-2 13B 0.57 ± 0.02 0.57 ± 0.02 0.57 ± 0.02 0.29 ± 0.02 0.75 ± 0.07
LLaMa-2 7B 0.36 ± 0.02 0.27 ± 0.02 0.56 ± 0.02 0.37 ± 0.02 0.44 ± 0.07
Vietcuna 7B 0.26 ± 0.02 0.15 ± 0.01 0.50 ± 0.00 0.01 ± 0.01 0.26 ± 0.06
MixSUra 7B 0.61 ± - 0.61 ± - 0.54 ± - 0.31 ± - 0.65 ± -
GPT-3.5 0.92 ± 0.01 0.74 ± 0.04 - 0.67 ± 0.01 0.92 ± 0.04
GPT-4 0.92 ± 0.01 0.74 ± 0.04 - 0.67 ± 0.01 0.92 ± 0.04