The more research on this topic is created, the more knowledge how to game them ... | Hacker News

Hacker Newsnew | past | comments | ask | show | jobs | submit

		SoKamil 24 days ago \| parent \| context \| favorite \| on: Exploiting the most prominent AI agent benchmarks The more research on this topic is created, the more knowledge how to game them will be stored in future training data. And since it comes from university, it is ranked higher in data corpus. It sounds like a self fulfilling prophecy.

abirch 24 days ago [–]

Damned old Goodhart's Law: "When a measure becomes a target, it ceases to be a good measure".

https://en.wikipedia.org/wiki/Goodhart%27s_law

Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact