Hacker Newsnew | past | comments | ask | show | jobs | submitlogin

This is like saying coding is a known and solved problem and benchmarks for coding is pointless for a general model. And that using a model for coding is an abuse of the multiturn chats AIs are tuned for.

1. The user should be able to prompt the AI to act differently from its default behavior. A human assistant is capable of role playing without always sounding like an assistant.

2. If the user asks the AI to follow the script and not allow unrealistic things to happen it should push back. The user is not always absolutely correct.

 help



You have a point, this might help test the resistance to jailbreaks for example. There are probably better tests for that than OOC roleplaying, I think. At the same time, what you expect is likely not going to happen due to many reasons (e.g. can't optimize for two contradicting objectives), and harness design offers a practical workaround that is being used for years already and is reliable unless the user is actively trying to jailbreak the model+harness.



Consider applying for YC's Fall 2026 batch! Applications are open till July 27.

Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: