Looks like you are running tools/browser in the same VM. How safe is this? Do you see contention for ports/resources? How does it compare to offerings from other sandboxing providers?

So as a user you have the choice of running tools/browsers etc. in the same VM. We don't currently offer a separate browser specific VM, but it really is upto the user on what they bundle into a sandbox.

Compared to any other sandbox provider I'm aware of agent37 is an order of magnitude better in price/performance ratio for any given task.

Got it. I meant how this design compare to other sandboxing providers. For example, ephemeral sandboxes (e.g., E2B) where you spin up a clean VM for every tool call can reduce the blast radius, but maybe inefficient.

That makes me wonder if it is a tarpit idea.

Or perhaps this is a new reality where for each current problem we will see myriad solutions because of how easy it is to put together a prototype with agents. 3 years ago having a sandbox was also an important problem to address, but we didn't see so many attempts.

I think at the surface level it's a wide problem but the solutions so far are shallow attempts

> A properly engineered sandbox would have prevented the escape.

The only sandbox that could have prevented this (as per my understanding) is a VM with no 0-day.

Until the AI finds a zero day exploit in physics, a faraday cage works pretty well to block WiFi.

In the end, unless you find an exploit in physics or logic, if you want the AI to do something useful for you, there will always be some gap in the sandbox, some communication channel. And with enough ingeniuity that can then be exploited.

In this case they wanted to test its cybersecurity capabilities and did not airgap it.

The test itself did not require an internet connection.

So you are using Modal VMs for training (I assume RL evals?).

> People run a pilot agent that scopes work and delegates it to sub-agents, each on its own VM: shape a project with the pilot, and the workers implement it and open PRs. One customer runs hundreds of machines at once, spun up and torn down from the CLI.

Are people spawning VMs for every tool call? If so, would love to understand why so, and why containers are not a good fit?

Hi! No not for every tool call. People are spinning up VMs for tasks that require sustained compute for hours or days. For example, they’ll deploy an agent with tools and a prompt to take an entire feature from spec to PR. Or an auto-research loop to improve the performance of an inference model.

So the idea is to give each agent a VM to do several tool calls? or one VM for each tool call?

I think the idea is to have the agent run in the VM

I'd be curious to see what it produces. Thanks for bringing it up!