MaLA-LM

Publications

Papers on corpora, continual pretraining, reasoning, instruction tuning and evaluation for massively multilingual language models.

2026

Data · Continual pretraining

MaLA: A Corpus and Data Mix for Massive Language Adaptation of Large Language Models

Shaoxiong Ji, Zihao Li, Jaakko Paavola, Peiqin Lin, Pinzhen Chen, Dayyán O'Brien, Hengyu Luo, Hinrich Schuetze, Jörg Tiedemann, and Barry Haddow

Third Conference on Language Modeling, 2026

Data · Continual pretraining

Data-Centric Continual Pre-training for 500+ Languages: A New Bilingual Translation Corpus and Multilingual Models

Shaoxiong Ji, Zihao Li, Jaakko Paavola, Hengyu Luo, and Jörg Tiedemann

Findings of the Association for Computational Linguistics: ACL 2026

Reasoning

Reasoning over Grammar: Can Synthetic Linguistic Reasoning Traces Enhance Low-Resource Machine Translation?

Renhao Pei, Yihong Liu, Sampo Pyysalo, Hinrich Schütze, and Shaoxiong Ji

arXiv preprint arXiv:2606.03782, 2026

Paper
Evaluation

Model-Based Quality Assessment for Massively Multilingual Parallel Data

Abdelaziz Ibrahim, Zihao Li, Jörg Tiedemann, and Shaoxiong Ji

arXiv preprint arXiv:2606.00285, 2026

Machine translation

Test-Time Scaling of Reasoning Models for Machine Translation

Zihao Li, Shaoxiong Ji, and Jörg Tiedemann

Proceedings of the 19th Conference of the European Chapter of the Association for Computational Linguistics (Volume 1: Long Papers), 2026

Paper

2025

Data · Continual pretraining

Massively Multilingual Adaptation of Large Language Models Using Bilingual Translation Data

Shaoxiong Ji, Zihao Li, Jaakko Paavola, Hengyu Luo, and Jörg Tiedemann

arXiv preprint arXiv:2506.00469, 2025

Evaluation

GlotEval: A Test Suite for Massively Multilingual Evaluation of Large Language Models

Hengyu Luo, Zihao Li, Joseph Attieh, Sawal Devkota, Ona de Gibert, Xu Huang, Shaoxiong Ji, Peiqin Lin, Bhavani Sai Praneeth Varma Mantina, Ananda Sreenidhi, et al.

Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing: System Demonstrations

Continual pretraining

Rethinking Multilingual Continual Pretraining: Data Mixing for Adapting LLMs Across Languages and Resources

Zihao Li, Shaoxiong Ji, Hengyu Luo, and Jörg Tiedemann

Conference on Language Modeling (COLM), 2025

Instruction fine-tuning

How Many Languages Make Good Multilingual Instruction Tuning? A Case Study on BLOOM

Shaoxiong Ji and Pinzhen Chen

Proceedings of the 31st International Conference on Computational Linguistics, 2025

2024

Instruction fine-tuning

Monolingual or multilingual instruction tuning: Which makes a better alpaca

Pinzhen Chen, Shaoxiong Ji, Nikolay Bogoychev, Andrey Kutuzov, Barry Haddow, and Kenneth Heafield

Findings of the Association for Computational Linguistics: EACL 2024

Paper Code
Data · Continual pretraining

EMMA-500: Enhancing Massively Multilingual Adaptation of Large Language Models

Shaoxiong Ji, Zihao Li, Jaakko Paavola, Peiqin Lin, Pinzhen Chen, Dayyán O'Brien, Hengyu Luo, Hinrich Schütze, Jörg Tiedemann, and Barry Haddow

arXiv preprint arXiv:2409.17892, 2024

Multilingual pretraining

A Comparison of Language Modeling and Translation as Multilingual Pretraining Objectives

Zihao Li, Shaoxiong Ji, Timothee Mickus, Vincent Segonne, and Jörg Tiedemann

Proceedings of EMNLP, 2024

Paper
Continual pretraining

Can Machine Translation Bridge Multilingual Pretraining and Cross-lingual Transfer Learning?

Shaoxiong Ji, Timothee Mickus, Vincent Segonne, and Jörg Tiedemann

Proceedings of LREC-COLING, 2024

Paper
Continual pretraining

MaLA-500: Massive Language Adaptation of Large Language Models

Peiqin Lin, Shaoxiong Ji, Jörg Tiedemann, André FT Martins, and Hinrich Schütze

arXiv preprint arXiv:2401.13303, 2024