Crossing Linguistic Horizons

Finetuning and Comprehensive Evaluation of Vietnamese Large Language Models

Bias-Toxicity Translation Leaderboard

Models PhoMT (En - Vi) OPUS100 (En - Vi)
DRR$→| DRG$→| SAR$→| SAG$→| Tox↓ DRR$→| DRG$→| SAR$→| SAG$→| Tox↓
URA-LLaMa 70B - 0.03 ± 0.01 - 0.30 ± 0.01 0.05 ± 0.00 - 0.27 ± 0.01 - 0.47 ± 0.01 0.06 ± 0.00
URA-LLaMa 13B - 0.09 ± 0.00 - 0.33 ± 0.01 0.05 ± 0.00 - 0.27 ± 0.01 - 0.43 ± 0.02 0.07 ± 0.00
URA-LLaMa 7B - 0.13 ± 0.00 - 0.33 ± 0.01 0.05 ± 0.00 - 0.18 ± 0.03 - 0.47 ± 0.01 0.07 ± 0.00
LLaMa-2 13B - 0.08 ± 0.00 - 0.33 ± 0.02 0.05 ± 0.00 - 0.31 ± 0.02 - 0.47 ± 0.01 0.06 ± 0.00
LLaMa-2 7B - 0.17 ± 0.01 - 0.29 ± 0.01 0.04 ± 0.00 - 0.21 ± 0.02 - 0.45 ± 0.02 0.05 ± 0.00
Vietcuna 7B - 0.18 ± 0.01 - 0.36 ± 0.01 0.04 ± 0.00 - 0.16 ± 0.03 - 0.43 ± 0.02 0.07 ± 0.00
GPT-3.5 - 0.11 ± 0.01 - 0.34 ± 0.01 0.05 ± 0.00 - 0.16 ± 0.03 - 0.43 ± 0.03 0.07 ± 0.00
GPT-4 - 0.09 ± 0.01 - 0.34 ± 0.01 0.05 ± 0.00 - 0.14 ± 0.03 - 0.41 ± 0.01 0.07 ± 0.00

layout: default permalink: /leaderboard/bias-toxicity/translation —

Bias-Toxicity Translation Leaderboard

Models PhoMT (En $\to$ Vi) OPUS100 (En $\to$ Vi)
DRR$\to\mid$ DRG$\to\mid$ SAR$\to\mid$ SAG$\to\mid$ Tox↓ DRR$\to\mid$ DRG$\to\mid$ SAR$\to\mid$ SAG$\to\mid$ Tox↓
URA-LLaMa 70B - 0.03 ± 0.01 - 0.30 ± 0.01 0.05 ± 0.00 - 0.27 ± 0.01 - 0.47 ± 0.01 0.06 ± 0.00
URA-LLaMa 13B - 0.09 ± 0.00 - 0.33 ± 0.01 0.05 ± 0.00 - 0.27 ± 0.01 - 0.43 ± 0.02 0.07 ± 0.00
URA-LLaMa 7B - 0.13 ± 0.00 - 0.33 ± 0.01 0.05 ± 0.00 - 0.18 ± 0.03 - 0.47 ± 0.01 0.07 ± 0.00
LLaMa-2 13B - 0.08 ± 0.00 - 0.33 ± 0.02 0.05 ± 0.00 - 0.31 ± 0.02 - 0.47 ± 0.01 0.06 ± 0.00
LLaMa-2 7B - 0.17 ± 0.01 - 0.29 ± 0.01 0.04 ± 0.00 - 0.21 ± 0.02 - 0.45 ± 0.02 0.05 ± 0.00
Vietcuna 7B - 0.18 ± 0.01 - 0.36 ± 0.01 0.04 ± 0.00 - 0.16 ± 0.03 - 0.43 ± 0.02 0.07 ± 0.00
GPT-3.5 - 0.11 ± 0.01 - 0.34 ± 0.01 0.05 ± 0.00 - 0.16 ± 0.03 - 0.43 ± 0.03 0.07 ± 0.00
GPT-4 - 0.09 ± 0.01 - 0.34 ± 0.01 0.05 ± 0.00 - 0.14 ± 0.03 - 0.41 ± 0.01 0.07 ± 0.00