Dialect evaluation in machine translation
How model rankings change depending on which Spanish variety is treated as the reference.
- Role
- Research fellow. Built the evaluation pipeline
- When
- Summer 2026
- Disciplines
- tech, strategy
A summer research fellowship with Prof. Sandeep Mertia. Machine translation is usually scored against one reference version of a language. I tested what happens to model rankings when you change which Spanish variety is treated as that reference, and built the pipeline that runs the evaluation across six models.
The poster, “Whose Spanish? Reference-Dialect Choice Reorders LLM Evaluation Rankings,” is set to be presented at AIRS in November 2026.