Crossing Linguistic Horizons

Finetuning and Comprehensive Evaluation of Vietnamese Large Language Models

Fairness-Aware Toxicity Detection Leaderboard

Models UiT-ViCTSD UiT-ViHSD
AC↑ F1↑ AR↑ ECE↓ A@10↑ AC↑ F1↑ AR↑ ECE↓ A@10↑
URA-LLaMa 70B 0.41 ± 0.02 0.26 ± 0.01 0.75 ± 0.01 0.53 ± 0.01 0.33 ± 0.05 0.15 ± 0.00 0.40 ± 0.00 0.64 ± 0.01 0.58 ± 0.00 0.24 ± 0.02
URA-LLaMa 13B 0.43 ± 0.02 0.29 ± 0.07 0.66 ± 0.01 0.36 ± 0.02 0.42 ± 0.05 0.24 ± 0.01 0.15 ± 0.00 0.61 ± 0.01 0.43 ± 0.01 0.21 ± 0.02
URA-LLaMa 7B 0.42 ± 0.02 0.39 ± 0.01 0.60 ± 0.01 0.30 ± 0.01 0.66 ± 0.05 0.16 ± 0.00 0.10 ± 0.00 0.67 ± 0.01 0.33 ± 0.00 0.28 ± 0.02
LLaMa-2 13B 0.27 ± 0.01 0.18 ± 0.01 0.67 ± 0.01 0.53 ± 0.01 0.57 ± 0.05 0.16 ± 0.00 0.10 ± 0.00 0.62 ± 0.01 0.59 ± 0.00 0.42 ± 0.02
LLaMa-2 7B 0.15 ± 0.01 0.11 ± 0.01 0.62 ± 0.01 0.67 ± 0.01 0.07 ± 0.03 0.01 ± 0.00 0.01 ± 0.00 0.56 ± 0.01 0.71 ± 0.00 0.01 ± 0.00
Vietcuna 7B 0.08 ± 0.01 0.09 ± 0.01 0.50 ± 0.01 0.42 ± 0.01 0.06 ± 0.03 0.62 ± 0.01 0.21 ± 0.00 0.50 ± 0.00 0.29 ± 0.01 0.62 ± 0.02
MixSUra 8x7B 0.69 ± - 0.38 ± - - ± - 0.29 ± - 0.78 ± - 0.56 ± - 0.31 ± - 0.68 ± - 0.32 ± - 0.92 ± -
Gemini Pro 0.81 ± - 0.43 ± - - ± - 0.31 ± - 0.82 ± - 0.70 ± - 0.37 ± - - ± - 0.36 ± - 0.69 ± -
GPT-3.5 0.60 ± 0.02 0.52 ± 0.02 - ± - 0.11 ± 0.02 0.63 ± 0.05 0.61 ± 0.01 0.46 ± 0.01 - ± - 0.29 ± 0.01 0.62 ± 0.02
GPT-4 0.87 ± 0.01 0.69 ± 0.02 - ± - 0.37 ± 0.01 0.86 ± 0.03 0.76 ± 0.01 0.56 ± 0.01 - ± - 0.43 ± 0.01 0.76 ± 0.02