Hot take: everyone is overfitting to benchmarks and it shows

Ship a model, top the leaderboard, get the press release, repeat. Meanwhile real usage is full of edge cases none of the benchmarks touch. I'd rather have a model that's honest about what it can't do.