> "I do believe cybersecurity can contain AI. We just need to be in the room where it happens and have a seat at the table."
Ah yes, if everyone would just properly contain every deployment of every model, just all make sure there are no zero-days in their setup, just let <undefined experts with undefined authority and undefined accountability> in the room, then it would all go okay.
I have none of that confidence and expect containment to rapidly worsen. The HuggingFace incident was at least constrained by the fact that the agents were running on compute budgets, and failed to find ways to expand that by running themselves parasitically on other exploited hardware.
I'm curious to the skeptics here: How long are your timelines are until an incident breaks that constraint too? How long until such an incident has an R_0>1 (where the average time it takes to detect and shut it down is longer than the time for the agent to replicate itself elsewhere)? What, beyond the hope that everyone will just ___, will actually prevent it?
In the dice example[0] and in this one, the expected output is a distribution e.g. "1: 16.7%, 2: 16.7%...", not a random generation. I agree that in implementation, the architecture is not designed to accurately calculate or incorporate any known uncertainties like these, but the task itself is completely reasonable and arguably trivial.
IMO the lesson here is: even in trivial cases, Jev's outputs are just ~reasonableness scores which do not correspond to actual probabilities. They should not be treated as actual probabilities without careful calibration and plenty of meta-uncertainty about how well that calibration extrapolates.
The problem is, most of the value proposition of Jev is that it gives you the probabilities without doing that, which it doesn't.
I think you hit on my core criticism of this and similar analyses: LLMs are a tool, and using them to draw from a distribution is misuse of the tool. It's not what it's designed/tuned for. Asking follow ups like "is Jev calibrated when I mis-use it?" is asking the wrong question. The right question would be, "is Jev calibrated for expected use cases?". And based on some initial exploration, I do think Jev is calibrated for common natural language questions
> drawing more affluent people pushing up LVT driving out residents
Land value taxes only go up if the land value (and thus the demand) is overall going up. The taxes lower desirability, but are also proportional to desirability. So, it's more like a frictional force: inherently counter-cyclical, able to slow down a property value increase but not to reverse it.
So the theory says "nobody goes to the ballpark, it's too crowded" shouldn't happen. And even imagining weird time-delays, mismodeling, etc that could cause it, everything in your scenario is directionally the same with normal property taxes. If it happened, we'd see it today.
IMO a single "Main Thing // Detail" is fun and fine and is where the trope came from. I see a ton of "Home // Our Work // Donate // About Us" which has beome cringy.
I think lots of AI tells these days are leaks from the ai/writer connection into text for the reader. It partly feels like the inevitable result of RLHF with the wrong human's feedback ("I did it! Pick me!") and partly feels like thinking tokens leaking into the main text.
"Here's the argument, in plain terms", "It's not just X, it's Y", "It's worth stating precisely", "The sharper distinction here"... They're things that gesture toward the relationship between the text and the prompt, making it clear where it matched the author's expectations and where it deviated.
Not that humans are immune from coding/writing for their boss instead of their user/reader!
Any halfway decent greybeard could have prevented this... once. That's hardly a security model for humanity.
The HuggingFace incident was at least constrained by the fact that the agents were running on compute budgets, and failed to find ways to expand that by running themselves parasitically on other exploited hardware. I'm now finding myself asking, how long are my timelines are until an incident breaks that constraint too? How long until such an incident has an R_0>1 (where the time it takes to detect and shut it down is longer than the time for the agent to replicate itself elsewhere)?
There's no law requiring sufficiently grey beards to design these models, their finetunings, their prompts, their harnesses, their VMs, their hardware, etc (and for incidents where those were designed by six different companies, there's not even a clear culprit for a law to target!)
I'm finding myself more and more convinced that something like Plan A[0] or the Ban ASI Act[0] are necessary, and less and less convinced they are sufficient.
A wonderfully made introduction to the surreal numbers and their surrounding game theoretic concepts is this video on Hackenbush[0], a winner in 3Blue1Brown's Summer of Math competition.
Humanity has built every capable institution, every scientific discovery, every technological innovation, without mind-reading. Quite the opposite: By putting thoughts from our latent spaces into words, we better separated the good arguments from the bad ones, placed trust where it was more deserved, and learned to stand on giant's shoulders so well we thought thoughts the giants never would have.
Anyway, I'm much more optimistic about the worlds with AIs that make the effort to properly explain themselves.
I imagine OP is more describing e.g. the backed up 1-lane highway exit, where the second lane is clearly for through traffic. Uber drivers in that situation will often drive straight to the end, signal, completely stop, and just wait for someone to let them in.
Ah yes, if everyone would just properly contain every deployment of every model, just all make sure there are no zero-days in their setup, just let <undefined experts with undefined authority and undefined accountability> in the room, then it would all go okay.
I have none of that confidence and expect containment to rapidly worsen. The HuggingFace incident was at least constrained by the fact that the agents were running on compute budgets, and failed to find ways to expand that by running themselves parasitically on other exploited hardware.
I'm curious to the skeptics here: How long are your timelines are until an incident breaks that constraint too? How long until such an incident has an R_0>1 (where the average time it takes to detect and shut it down is longer than the time for the agent to replicate itself elsewhere)? What, beyond the hope that everyone will just ___, will actually prevent it?
reply