The brazen disregard for safety is funny in this context, if a little frighteningly indicative about the toxic attitudes that permeate Silicon Valley at the moment.
Anyway, even if we froze training right now LLMs in their current state would have the ability to continue to provide tremendous utility. We’re not asking frontier model providers to delete their previous generations of LLMs (and harnesses), we’re trying to hold them accountable for the crimes they’re committing while training the next generation of LLMs (and harnesses).
All this whining about human safety just results in a complete lack of progress that could’ve probably saved hundreds of millions of lives. Instead all the short-term thinking and lack of appetite for risk is killing us. Everyone’s just thinking of the next 10 years of their life, there don’t seem to be old men that plant trees that they won’t sit in the shade of anymore.
This is some serious touch-grass territory you're treading in. Might I introduce you to the Bottomless Pit of Suffering[1] thought experiment of utilitarianism? Because it's incredibly relevant to the arguments you're presenting.
Disclaimer: I don't generally hold Silicon Valley rationalists in high regard, but this particular piece of writing has always stuck with me quite deeply, as someone who used to be superficially interested in utilitarianism.
Would disagree. It’s not Scott’s conclusion I’m referencing, it’s his thought experiment: “What if there was a bottomless pit of suffering that no matter how much you threw into it, there would be no meaningful progress into reducing that suffering?”
It’s a thought experiment that gets to the core of why fundamentalist utilitarianism (like you’ll often find in SV rationalist communities, especially AI fanatics) is an unbalanced approach to morals and ethics.
Utilitarians can declare that they believe that there is a proverbial universal panacea, and then work backwards to justify within their moral framework literally any action that they take that could be construed as work towards that panacea.
But what if no matter how much effort they put into achieving that panacea, they never make meaningful progress? And all the while they’ve massively increased the amount of suffering that people who they are surrounded by?
As an absurd example, imagine a utilitarian who’s convinced that if we sacrifice infants to the Gods that we’ll get to an AI singularity faster.
That’s an interesting thought experiment and the first time I have come across it. The only problem with it is there is no end to the suffering in that scenario. There is an end state to real utilitarianism and it’s achievable and makes your thought experiment not grounded in reality. There is a state of human development that the majority of people would agree approaches a utopia, which is a tangible and achievable goal.
> There is a state of human development that the majority of people would agree approaches a utopia, which is a tangible and achievable goal.
The Star Trek vs. The Expanse meme is relevant here. There’s no guarantee that there is a continued reduction of suffering in the future, especially due to technological progress. There may be technology that reduces the suffering only for a few select elites while the rest of humanity lives in destitution, unable to meaningfully revolt.
Anyway, even if we froze training right now LLMs in their current state would have the ability to continue to provide tremendous utility. We’re not asking frontier model providers to delete their previous generations of LLMs (and harnesses), we’re trying to hold them accountable for the crimes they’re committing while training the next generation of LLMs (and harnesses).