As neural metrics are a pillar for #MT, being extensively used for evaluation but also improving translation, we'd want them to be fair. 🚨 Our #ACL2025 paper shows they consistently, unduly favor masculine-inflected translations, or gendered forms, over neutral ones. arxiv.org/pdf/2410.10995