Jev, or 'Jevlikes', will go a long way towards trustable systems. There are demos of running every prompt through the first pass filter of Jev "Is this unsafe? y/N"
Seems to be that Jev is a "reflex" system for AI, where current LLMs are higher level thinking. Computers can now flinch!
Asking for proof of improvement is a very rational reaction to the last three to five years of constant AI hype cycling.
It's a simple question that seems to make AI cheerleaders really mad.
"How are you measuring improvement."
I use AI for coding every day and see it fail all the time, I'm a seasoned developer and early tech adopter just like everyone else on HN. I'm still very skeptical because of how often these systems just miss. It's gambler's ruin on a very large scale, we remember the hits and forget the misses.
What kind of metrics are you looking for? No one answers that question. It's like asking for clinical trial results about whether or not a parachute works and if you should use one when jumping out of a plane.
I don't expect AI to be perfect. Are people perfect? I leave room for corrections, but at this point they are usually minor and acceptable.
reply