0 score1 answer
Why can an agent's overall evaluation score improve while a critical workflow gets worse?
A candidate model improves the overall pass rate but fails more of the tool-authorization or escalation cases. Most tests are easy informational questions. Should the higher average justify shipping? This editorial example uses invented, explicitly synthetic pass counts to show aggregation risk. Th