LREC 2026: Diagnosing translated benchmarks
In May 2026, I presented our work “Diagnosing Translated Benchmarks: An Automated Quality Assurance Study of the EU20 Benchmark Suite” at LREC in Palma, Mallorca. What struck me most across the conference was a recurring theme: multilingual evaluation is becoming less about simply translating English benchmarks, and more about diagnosing whether our evaluation data is valid in the first place. This post collects the main idea of our paper, the most relevant work I saw at the conference, and a few personal takeaways for researchers working on multilingual LLM evaluation. ...