FineOPUS: A Massively Multilingual Translation Corpus and Pipeline

Refining Parallel Texts in Many Languages

MaLA-LM Team, ELLIS Institute Finland, TurkuNLP, Helsinki-NLP

Nov 21, 2025

Original OPUS Code (Coming Soon) 🤗 Dataset (Coming Soon) Discord

The Mission: From Noise to Signal

Data is the fuel of modern AI, but in the realm of multilingual models, quality is quickly becoming more valuable than quantity.

For years, the OPUS collection has served as the backbone of open-source machine translation, offering a massive repository of parallel texts. However, as we push the boundaries of what Large Language Models (LLMs) and Machine Translation (MT) systems can do, the noise inherent in web-scraped and aggregated data has become a bottleneck.

Today, we announce FineOPUS: a project dedicated to transforming the vast OPUS collection into a foundational, high-quality parallel corpus for the global AI community. Our mission is simple yet ambitious: we are applying a rigorous, empirically-driven data curation philosophy to parallel data. Inspired by the success of the FineWeb project, we aren't just cleaning data; we are systematically engineering a resource to be reliable, equitable, and state-of-the-art.

Why FineOPUS?

Current open datasets often suffer from critical issues that hinder model performance:

  • Semantic Misalignment: Sentences that don't actually mean the same thing.
  • Language Contamination: The wrong language appearing in a dataset labeled for another.
  • Formatting Artifacts: HTML tags and broken encoding that confuse models.
  • Inequity: A massive gap in quality and volume between high-resource (e.g., English-French) and low-resource languages.

FineOPUS aims to mitigate these issues through a transparent, reproducible pipeline.

The Methodology: Radical Empiricism

We are not guessing which cleaning methods work best. We are proving it. The core of the FineOPUS strategy is validation through ablation. We will train dozens of models to empirically justify every single design choice in our pipeline. If a filtering step doesn't improve model performance, it doesn't make it into the final pipeline.

Our Multi-Stage Approach

  • Scalable Cleaning: A pipeline encompassing normalization, language re-identification, and neural parallelism filtering.
  • Principled Quality Estimation: Moving beyond simple heuristics to model-based quality controls.
  • Targeted Augmentation: For low-resource languages, we aren't just filtering (which reduces data); we are building. We will employ iterative back-translation and synthetic data generation to bridge the gap for under-represented languages.

What We Are Delivering

We believe in open science. Upon completion, FineOPUS will release three key assets to the community:

  1. The FineOPUS Dataset: A large-scale, open-licensed parallel corpus ready for training next-generation AI models.
  2. The FineOPUS Pipeline: The fully documented, reproducible open-source code used to create the dataset.
  3. The Technical Report: A comprehensive account of our ablation studies, providing transparency on why specific curation decisions were made.

Join Us on the Journey

We are building FineOPUS to empower researchers and developers to build more powerful, inclusive language technologies. By ensuring the world's linguistic diversity is represented in the future of AI, we hope to bridge the digital divide one sentence pair at a time.

Stay tuned for our FineOPUS release and technical report.

Discord: MaLA-LM