He epitomises exactly who I'm hoping to bump into on the odd occasion I'd walk into a pub alone out of boredom. 50% everyday gentlemen 50% off the wall mentalist. Love his humour and mad schemes
Additional color: per the commit, the bug was found by Paulos Yibelo, who runs PwnAI (the author of this piece), and worked on security at AWS for 4 years prior.
Claude Code and Codex ignore the personal data xor network access rule by design. Of course this happens. If it wasn't due to the prompt sent to the model, a passing scan of a README.md that happens to mention your details would achieve the same result. What about a hostile LLMs.txt that words it such that free API keys are available in exchange for a tarball of $HOME?
The solution is don't run a spambot with an Internet connection over your personal stuff, unfortunately nobody is ready to hear that yet.
Curious how their plugin works, but I'm guessing "incompletely", given personal information can end up buried in base64 or similar preventing easy string matching. Some kind of babysitter model watching network IO via an invasive proxy might be viable, if folk were willing to pay the latency cost, but it will not prevent a dedicated exfiltration attempt, which itself might be (according to the model's logic) for innocent reasons.
Since when did any employee ever do that? "It is difficult to get a man to understand something when his salary depends on his not understanding it". The higher up the food chain often relates in great part to accepting anything thrown at you with a nod and a smile. 6-7 figure incomes don't arise from sounding the alarm and creating problems for everyone within range
This is nonsense, at a policy level the Chinese government have actually been moving towards restricting larger models. I'd eat my hat if Europeans in particular aren't paying licenses for weights in a few years if it means getting to host the models on home soil, there is draft legislation (CADA) making its way through at the moment that would even require this in many circumstances.
Ignoring the obvious "dmd is truly blazing fast" quip, wildly curious how this happened to you twice? Was it the PSU? Only once heard of a house fire caused by a computer, but no information on component actually caused it.
The first time it was the graphics card. It burst into flames and smoke poured out. The fire went out when I yanked the power. Fortunately, I was sitting next to it when it went. To be fair, it was doing a long graphics job.
After that, I set the computer on a metal plate to keep a fire from spreading to the floor.
Some years later, the PSU went out with a bang and a lot of smoke.
(dmd is so fast, the computer doesn't have time to heat up!)
Given the only practical alternative to a jurisdiction with strong data protection laws is a home rig, as the saying goes, still cheap at twice the price.
Surprised that "do you really think we're going to make it?" scene made the cut of the first video, her expression and tone is totally out of sync with the scene and demonstrates the problem with almost every AI gen video I've come across when it comes to capturing humanity.
The painting->interactive scene looks astonishing though
This is the first OMG-security AI article that sat well with me. They don't overcook their claim, they simply document something that happened, and what happened is genuinely quite concerning.
But we already have safeguards for this. If compute has access to your private data, it shouldn't have unfettered access to the Internet. Sandboxes like Claude's have had rules like this baked in forever (they only whitelist package repos AFAIK).
From a UI perspective this is also easy to design around. You need an open Internet task, you prevent the task from reading existing files. The user tries to upload a file to such a task, and a prominent warning explains the rare but possible risk.
They asked it for a detection against exfiltration attempts. Sending a test file out to the Internet sure looks like testing an exfiltration detection.
They claim that it read the DNS records and what was in tmp first. Again, all things that would be very reasonable to do when designing building and testing an exfiltration detection system out of tmp. It did not try and send the contents of the computer to Beijing.
> they simply document something that happened, and what happened is genuinely quite concerning
Is it, though? If one weird thing happening once is the threhold for when we should be concerned, we should be concerned about literally every model ever released.
Completely agree, it's quite objectively and well written. I noticed Musk slipped into their X comments with a "concerned" emoji, and there is a chance that this will cause bit of hysteria and add fuel to the fire for "open models are bad for your health".
reply