Public datasets of Chest X-Rays (CXRs) are a popular benchmark for ML computer vision models, which often report strong average-case performance. But does this aggregate performance reflect actual utility in the clinic?