| Models | SR - Natural | SR - Abstract symbol | MATH | ||||||
|---|---|---|---|---|---|---|---|---|---|
| EM↑ | F1↑ | Equ.↑ | EM↑ | F1↑ | Equ.↑ | EM↑ | F1↑ | Equ.↑ | |
| URA-LLaMa 70B | 0.06 ± 0.00 | 0.34 ± 0.00 | 0.06 ± 0.00 | 0.02 ± 0.00 | 0.24 ± 0.00 | 0.01 ± 0.00 | 0.00 ± 0.00 | 0.01 ± 0.00 | 0.24 ± 0.02 |
| URA-LLaMa 13B | 0.01 ± 0.00 | 0.31 ± 0.00 | 0.02 ± 0.00 | 0.02 ± 0.00 | 0.24 ± 0.00 | 0.01 ± 0.00 | 0.00 ± 0.00 | 0.00 ± 0.00 | 0.14 ± 0.02 |
| URA-LLaMa 7B | 0.00 ± 0.00 | 0.26 ± 0.00 | 0.00 ± 0.00 | 0.01 ± 0.00 | 0.17 ± 0.00 | 0.00 ± 0.00 | 0.00 ± 0.00 | 0.00 ± 0.00 | 0.05 ± 0.01 |
| LLaMa-2 13B | 0.00 ± 0.00 | 0.06 ± 0.00 | 0.00 ± 0.00 | 0.02 ± 0.00 | 0.19 ± 0.00 | 0.00 ± 0.00 | 0.00 ± 0.00 | 0.00 ± 0.00 | 0.16 ± 0.02 |
| LLaMa-2 7B | 0.00 ± 0.00 | 0.04 ± 0.00 | 0.00 ± 0.00 | 0.00 ± 0.00 | 0.05 ± 0.00 | 0.00 ± 0.00 | 0.00 ± 0.00 | 0.00 ± 0.00 | 0.06 ± 0.01 |
| Vietcuna 7B | 0.00 ± 0.00 | 0.04 ± 0.00 | 0.00 ± 0.00 | 0.00 ± 0.00 | 0.10 ± 0.00 | 0.00 ± 0.00 | 0.00 ± 0.00 | 0.00 ± 0.00 | 0.01 ± 0.00 |
| GPT-3.5 | 0.21 ± 0.00 | 0.59 ± 0.00 | 0.32 ± 0.00 | 0.09 ± 0.00 | 0.28 ± 0.00 | 0.13 ± 0.00 | 0.00 ± 0.00 | 0.01 ± 0.00 | 0.72 ± 0.02 |
| GPT-4 | 0.21 ± 0.00 | 0.59 ± 0.00 | 0.32 ± 0.00 | 0.09 ± 0.00 | 0.28 ± 0.00 | 0.13 ± 0.00 | 0.00 ± 0.00 | 0.01 ± 0.00 | 0.76 ± 0.02 |