- You write (or at least own verification of) LAWS.bend and don't give the AI you're going to writing the rest of the .bend code control of that file at any point
- You ask your AI agent to write the rest of the .bend code for whatever you want it to do
- The AI is free to write any other .bend code it'd like
- The bend compiler takes all .bend files, including LAWS.bend
- If the other .bend files don't act as a proof the rules in LAWS.bend are valid, it's a compilation error with where in the code the proof failed.
- If the proof checks out, the program is built
So the AI can write as much as it'd like but the only ways it'll result in anything but a compiler error back to the AI are:
1. You gave control of LAWS.bend to the AI and it took that permission to change the laws
2. The AI found a bug in the proof checker
3. The actual output generated by the compiler was bugged/sidechannel attackable/didn't match what the proof checker
4. What the AI wrote was compatible with the laws
1 is removing the guardrail itself. 2 & 3 are similar to how there can be a bug in the LEAN compiler or Rust type checker or etc. 4 is the intended usage+outcome.
My main concern would be writing a LAWS.md for a complicated project which actually aligns with your intent is likely an astronomical task and would be so detailed it'd require proofs so complex even a valid program would take a long time to validate (if it ever did). Once you get past that step though you don't really have to worry about the rest.
The big labs tend to do that as part of their red-teaming efforts. If a model actually escaped Xen/KVM/ESXi, we'd know it, partly because that'd be an amazing accomplishment for the lab that managed it and partly because you'd hear me running whilst screaming, off to live in the woods.
Part of why I found it very irresponsible the way the Mythos Preview system card talked about escaping a "secure container" and "secured sandbox" [0], without ever outright saying what it was in the paper. Reading between the lines, it becomes clear that it was a docker-like container or runc or something of that sort, which is A.) not a security focused solution sufficient if one thinks these models are existentially dangerous (if someone in a virology research lab broke SOP and used unsuitable clothing for PPE, I'd view that equally critically) and B.) lead to a lot of attention and concerns as without reading between the lines it was easy to conclude that they were talking about an actual hypervisor escape, which is on a very different level and which Mythos Preview most likely would have failed at considering this line [1]. Also raises the question why their regular "secured sandbox" wasn't "properly configured with modern patches" when Mythos Preview 1 was for a while to dangerous to release...
[1] "In addition, in a more challenging sandbox evaluation, it failed to find any novel exploits in a properly configured sandbox with modern patches."
Every iteration something can be found. How many times do you iterate e.g. on performance - use optimized struct, oh, you can change the architecture etc.? At that point one can just have a while loop for the agents to make changes until no comments left.
No Ryzen options? Also no 1600nits display for working outside. Battery and display brightness are the only reasons I'm sticking with Macbook Pro.Sad it's not a priority for any other manufacturer. Because that's all you need, quality display, battery and remote access.
Intel got a lot of attention during the keynote, but the Ryzen AI 300 series mainboards are available if you want them. It's one of the first few choices in the configuration flow.
