AI, computers, electricity, steam power, basic tools, and even fire all enabled in a dramatically faster rates of change compared to the immediately preceding era. You could also argue all also mimic certain human (or more broadly, animal) characteristics. I don't see what is truly unique to AI here. I'm not sure AI is even qualitatively unique from the computer, just the technology taken to it's logical conclusion.
Fundamentally I don't think "tool" is the right word for whatever generative AI is. Tools are normally designed with intention for a specific purpose, but AI systems are trained or grown.
To use an analogy, you can use a goat as a lawnmower - it'll keep the grass down for you. You could maybe even train it so it keeps it a specific length, but it's a goat - there's always a chance it will jump the fence or headbutt you or something else. It's an animal, not a tool. You can use AI as a tool, but that doesn't make it a tool.
Yes AI is arguably still software, but we have never seen programs with anything close to this level of flexibility and adaptability in software.
As another analogy, there's nothing special about neurons- their characteristics are broadly known and researchers have been able to simulate small numbers of them in computer models. However, when you have enough of them in the right structure, complex behaviours are possible which are never exhibited at the lower levels. There's nothing majorly different between the neurons in brains of a chimp and a human, but the capabilities and meta-capabilities are wildly different.
I think we see a similar step-change with generative AI systems.
People say this, and then the scenarios they come up with are boring and uninteresting. 80% of the scenarios in this thread are "AI hacks something", as if hacking didn't exist before agents, and the remaining 20% are mostly made up of other things humans are already trying to do at scale.
Why am I supposed to believe that a virus created in a virology lab with AI assistance is uniquely more dangerous than a virus created in a virology lab without AI assistance?
> Why am I supposed to believe that a virus created in a virology lab with AI assistance is uniquely more dangerous than a virus created in a virology lab without AI assistance?
You're not.*
The difference is how likely this is to be done, not how dangerous it might be if it was done.
And this isn't likelihood in a 0-100% sense, but in a Poisson distribution sense, i.e. mean time between incidents.
Remember: these AI minds aren't that good with the physical world, and have nevertheless been repeatedly connected to robots, and now some of the AI labs are showing off their actual wet labs. Accidents are absolutely a possibility, and I don't think it's low given the previous behaviour of Silicon Valley startups.
* OK, some people care that AI is getting more capable at genetics just like it's been in maths and programming, but short term, before AI solves genetics so hard it can make an STD that makes the infected uncontrollably horny and then ossifies our bodies or something, it can obviously just copy any of the many DNA or RNA sequences we've already got on record.
Physical labs and experiments will always have physical bottlenecks, so I really don't see how AI is a transformative element here. What is it about AI that will be so transformative to the fundemental economics of this space that it will lead to several orders of magnitude more experimentation? If the problem is an expansion of wild west wet lab experimentation, why isn't ~that~ the focus of safety?
Who is going to profit from making an super STD anyway? How would regulating AI change their incentive structure?
> Physical labs and experiments will always have physical bottlenecks, so I really don't see how AI is a transformative element here.
AI is all about automation, removing bottlenecks. One of the easier to see bottlenecks is that it answers exam questions, which means the set of people who were previously excluded by the bottleneck of "can't answer exam questions" is now available for problems one step harder than before.
Note: This does not mean those people are now competent on harder problems, and AI is very different to humans in the kinds of error it makes, so AI letting some humans skip the "passing exams" bottleneck does not mean those cheating humans are going to automatically know how to avoid the kinds of dangerous error that a human who did the work to pass an exam would automatically pick up.
> What is it about AI that will be so transformative to the fundemental economics of this space that it will lead to several orders of magnitude more experimentation?
Other than all the things that allowed AlphaFold, or other forms of computational chemistry, and that lab equipment is increasingly computer controlled, and that LLMs can efficiently perform literature reviews, and that "research taste" is one of the new benchmarks for AI improvements now that we've saturated our ability to sensibly compare them on so many other things, and that they're really really cheap?
("Other than that, what have the Romans done for us?")
> If the problem is an expansion of wild west wet lab experimentation, why isn't ~that~ the focus of safety?
Two things: One, lab safety was already a focus of safety. Two, AI is a general purpose technology, biolab issues are simply one concrete example of a thing that it can mess up dangerously.
> Who is going to profit from making an super STD anyway? How would regulating AI change their incentive structure?
Two very separate questions.
First, one is a category error: consider that one could equally well ask "Who is going to profit from hacking into HuggingFace just to steal some puzzles?", to which the answer is "Nobody. Profit was not a direct motivation of the AI itself: an AI did this after being given an impossible test question, which the AI solved by hacking one system, then incorrectly reaching the conclusion it needed to break into a totally unrelated system to cover up that it had cheated".
Second: regulations, all regulations in all situations, are nothing but an incentive structure. That's their sole purpose, to change the incentives of a person, be that person a legal person or a natural person.
The physical bottleneck of wet lab experiments is real life synthesis, which takes real time and real physical infrastructure. As you note, it's already mostly automated through deterministic computer controlled systems. AI doesn't change this at all. AI can change the information processing side dramatically, and maybe this will allow for more resources to be spent on synthesis, but the true bottleneck remains untransformed.
"Free communication" is specifically held up as a threat to democracy by current leaders in quite a few states. For instance, Macron and Merz in France and Germany have both recently explictly comdemned free communication, that isn't filtered and gatekept by some "common framework" or government agency, as a threat to democracy. Probably because this communication is producing social movements in these countries that the state cannot control.
And if this isn't US-centric enough for you, the previous US administration created a "Disinformation Governance Board" specifically to address the threats that they felt free communication among the people was posing to democracy, taking action to pressure corporate actors to censor disfavored popular speech in attempt to circumvent first amendment protections. At least the current administration isn't really pretending that attacking free speech is protecting democracy somehow.
Patronage programs for people with advanced degrees is a woefully inadequate solution to the current economic pressures. Large segments of the first-world working and middle classes hollowed out by globalization had already turned against it before AI even came on to the scene. People are generally hostile to patronage directed towards people above them on the ladder.
You don't want the people to be uneducated, because you have no control over their thought processes, which could be dangerous an unpredictable. And it leaves space for someone else to educate them in way that conflicts with your interests.
Ideally, you would want them to be intentionally educated to a specific type of manipulable gullibility, where they are receptive to your messages, but resistant to messages from other sources.
The effort significantly predates the recent changes in US media ownership. I would actually argue that the ownership change is mostly orthogonal to the ongoing trend of centralization and top-down manipulation of digital information sources.
A shift change of the prison guards more than a material change to the prison.
What action can Iran take today that they couldn't take a year ago? No one who has been paying attention should be surprised that Iran can shut down the straight. It has been a known factor for decades.
They have less leverage. The have so much less that they are forced to openly use their last and most powerful card for their survival, when they never have had to before. That is a position of weakness, not strength.
>The have so much less that they are forced to openly use their last and most powerful card for their survival
That is not their most powerful card. Their most powerful card is mining the Strait of Hormuz and taking out all GCC desalination and oil infrastructure. That would result in a global depression, and probably end the Gulf countries as we know them.
Destroying the gulf states would dramatically reduce the importance of the Strait, which would make mining it or otherwise shutting it down somewhat pointless anyway. It is a bit of mutually assured destruction, but the USA is probably in the best position of anyone to weather that storm.
I suppose it is more powerful in an absolute sense than just temporarily shutting down the Strait, but like Russia's nukes, I think the threat is more useful than the play itself. Unless they are just looking to take others down with them.
> What action can Iran take today that they couldn’t take a year ago?
Remove of sanctions, ability to monitize traffic through the strait, guarantees against aggression and a cessation of military bases in their region. IMO, a much stronger position than they were in a year ago.
All policy aimed at preventing nuclear Iran has one goal: buy time. I think it is hard to argue that time has not been bought (though how much and whether the price was right is another question). The only semi-stable long term option is a friendly Iranian government. The IRGC's main purpose is to occupy Iran, so anything that makes them weaker, less stable and more decentralized improves the odds of successful internal revolt in the long run. It is really hard for me to see how any of this has made the IRGC more stable in the long run.
The threat of the strait closure has always been a major factor in Iran policy from all relevant nations, it is just now explicit. It's hard to take the Russia point seriously when the war forced both Russia and Iran to shift resources form the Ukrainian theater to the Persian Gulf; it seems to be close to a wash. It's also kinda silly to gas up using interceptors for their intended purpose as "heavy damage" or catastrophize about rounding errors in damage to USA assets, while simulatenously writing off the total effect of all USA/Israel actions as inconsequential.
Disruption to global fossil fuel supply chains was also a goal of this war, so I am not sure you should list it as a negative. In the current state of the world, USA interests and global economic interests are becoming increasingly decoupled, and one shouldn't assume they are automatically aligned.
Also this has probably done more to hasten the world's weaning off fossil fuels than any action by any other government.
IRGS domestic propaganda has always been that US is a military murderous malevolent regime, mercilessly going after their land and their children.
With just a little bit of propaganda spin, or even without it, US just proved to the entire Iranian population that IRGS was right all along.
This should strengthen or even harden their regime as they will have new generation of hardliners join the movement.
This is like 1930s Germany kinda thing. Who won or lost is semantics at this point, the regime is free to spin it any way they want, and will have quite the support to do it.
> All policy aimed at preventing nuclear Iran has one goal: buy time. I think it is hard to argue that time has not been bought (though how much and whether the price was right is another question).
Given that Iran has been one week/one month/one year away from acquiring nuclear capabilities since 2014 - first Trump Presidency, and they are not any closer a decade later this "buying time" rhetoric is nothing short of "Iraq has WMD" level of absurdity.
It is not jist Bibi, but also the IAEA and other international organizations. And at least the last 5 US administrations. I suppose they could also all be in Israel's pocket though.
Iran's 60% enriched uranium stockpile is really not up for debate. Iran is happy to tell everyone that they have it. With the proper equipment, 60% can go to 90% in a single month. So the question is how advanced is the Iranian infrastructure for the final enrichment step, and (less commonly talked about) how ready they are to actually make a fission bomb out of that material. The latter task is not considered to be very hard, North Korea did it after all, so the main focus has been on the former. There does seem to be some decent information that the centrifuge array has been under active development at various points, and has been consitently, actively targetted by Mossad/CIA for at least the past 20 years or so. For example, Stuxnet was a joint CIA/Mossad operation that begain in 2005 and continued through both GWBush and Obama.
Unfortunately, even with some nice bribes from Obama, Iran was always a little cagey with the IAEA inspectors, and officially kicked them out in 2021. So after that, the only sources for the state of Irans nuclear infrastructure information effectively became Iran itself and Mossad.
I'm not gonna argue whether it is true or false, but the scale of the conspiracy you are proposing is pretty massive. Multiple orders of magnitude bigger than anything that is currently within the mainstream public consciousness.
In this case, I think a network of mostly independent actors whose interests happen to align makes more sense. But maybe sometimes those interests are a little different than what people commonly understand them to be.
>It is really hard for me to see how any of this has made the IRGC more stable in the long run.
It's not hard for me to see. It's very similar to the situation in Ukraine. They have suffered losses but I can only imagine that their morale and confidence is through the roof. Conversely, the population must feel that there is no hope of getting rid of them. The cavalry sounded the horns but mostly rode into the river.
>Disruption to global fossil fuel supply chains was also a goal of this war
I am not convinced that a population that just recently had 30k people die in a revolt is gonna immediately rally around their oppressors after a foreign power kills 2k. I have yet to see compelling evidence that formerly IGRC-hostile segments of the population have switched alleigances. It is possible. But one could also imagine an exhausted population that is tired of a goverment they despise putting a target on their backs. The Iranians I personally know suggest that the second idea is more true, but it is anecdotal evidence with heavy selection bias. Another factor is that Iran has an unstable food and water supply, and people who lack food and water tend to focus their anger on whoever is closest that has food and water.
The Trump administration is actively interested in the dissolution of the current global economic order. This is why they are relatively unbothtered by the global economic shock that is a Strait of Hormuz closure, whereas the globally-oriented neoliberal administrations of the past wanted to avoid this at all costs.
That will take hundreds of years to accomplish. No Iranian has forgotten Operation Ajax in 1953[0] - overthrowing the lawfully elected government of Iran and replacing it with a dictator[1].
> In the current state of the world, USA interests and global economic interests are becoming increasingly decoupled
This is entirely due to Trump constantly "ripping up" previous agreements and treaties. It has become obvious to all observers that the US can no longer be trusted to act in the interests of any other nation. For their own survival, they have to become independent of US economic and political interests.
> It is really hard for me to see how any of this has made the IRGC more stable in the long run.
It's not really that hard to see - if you open your eyes.
If you refuse to do that, to the point where you see nothing but the hint of a silver lining in every carcinogenic cloud, then yeah I guess things must look pretty silvery.
It’s a nation of 90 million people. Now that basically every facet of society has been hit by a single common enemy, they will galvanize and it won’t matter what name IRGC or whatever you give it they will start to work in unison for common security and deterrence.
reply