> The fact we have so many people dedicated to one short movie is amazing and in my eyes always worth the effort.
i’m convinced this is where much of the magic comes from.
it’s the same as why often music written and played by an group of musicians who specialize in their instrument is often so much better than the music from a solo who records each instrument themselves.
> I fail to see an argument against a non-human dictator.
You should feel free to let one be your dictator. No one is stopping you.
If you’re looking for an argument against it: the instant you believe other people should be forced to live under a dictator if they choose not to is all the argument against it you need.
Again, no one is stopping you from living under a dictator ai. Have at it.
its no secret the industry wide trash of "move fast and break things" has been the mantra far too often as opposed to "take the time to build it correctly"
for much of the industry its "ship features. now."
we regularly read here how often engineer's concerns are ignored by company salesmen and executives.
this is quite the statement. i can think of many, many ways they are not categorically “better” … many ways.
i can think of a number of “peasants” who are categorically “better” than “elites” in many, many areas.
i mean, i think most of us here have personally met at least one or two absolute failsons who are absolute dipshits yet will always fall upwards due to who their families are.
We remove the limited liability protection of a corporate entity. Make them operate as a partnership so all executives and equity holders are personally liable for the debts of the firm and make them post a bond to backstop financial damage caused by their agents and customers use of the agents. Thats how Goldman Sachs and other investment banks were required to operate until the deregulation push that resulted in the 2008 financial crisis. The same theory holds, if people respond to incentives and you want to incentives safe behavior make them responsible for their actions. People forget that the corporate entity was created to incentivize risky activity like sailing a boat across the world to get spices when half never returned. Some valuable economic activity won't be done without limited liability protection so society created a mechanism to promote that activity. We've gone too far.
As I say elsewhere. Make executives and equity holders personally liable for debts and harm of the company. They will create a culture of safety really fast.
That's a stupid idea. Limited liability corporations have been a key enabler for advances in human standards of living. You seem to be confused about the basics of finance and economics.
Not confused. I studied finance, economics and the history of corporate entities in law school and published papers on the topic. Limited liability is not necessary for the advancement of standards of living. Free market capitalism can exist without limited liability protections being so broadly available. Investment banks were partnerships until the 1990s, law firms are now specifically because society wants to incentivize lawyers to be personally liable for any harm to their clients at the hands of their partners rather than being shielded from rendering bad legal advice or tolerating their partners from the same behavior.
Instead of having a gut reaction to reject my suggestion why don’t you sit with it, research the history of how commercial activity has been structured and think about the consequences. You might recognize a different perspective than the current group think.
Thanks I'm also familiar with the history and have written papers etc. Things have improved rapidly with increased adoption of limited liability. It would be stupid to turn back the clock and throw away all of the benefits because of a few isolated minor problems.
Stop being a dick and calling ideas stupid and attacking me rather than the argument.
I didn’t say roll back limited liability on every industry, I said specifically and limitedly for frontier AI labs because they present more risk of harm and are demonstrating they aren’t managing that responsibility.
The 2008 financial crisis was caused in large part by bankers that openly talked about the fact that securitization of mortgages and the lack of partnership liabilities meant that they didn’t have any risk to the firm or themselves. The AI labs are behaving similarly.
There's nothing special about frontier LLM companies. Singling them out for special financial restrictions is a stupid idea based on nothing but your own irrational and uninformed prejudices. No actual harm has been demonstrated. No one has died.
> There's nothing special about frontier LLM companies.
On the one hand, yes, they're companies like any other.
On the other hand, I can count on one hand the number of companies that have publicly declared «We're working on WMDs [0], we don't think we are capable of working on them safely, and we refuse to stop working on them. However, if we get special legal and regulatory treatment we'll be quite happy to put in the stop work order.».
So, yeah, there are some special things about the major LLM manufacturers and none of them are good.
[0] Anything that has a 10% chance of suddenly destroying all of humanity is a WMD.
What a silly comparison. There is no scientific basis or mathematical formula to justify a claim of a 10% risk. That is an intellectually dishonest attempt to poison the debate.
> What a silly comparison. There is no scientific basis or mathematical formula to justify a claim of a 10% risk.
The source for this claim is the lead of Anthropic's Alignment Science division. His actual claim is that the risk is greater than 10%. From [0]:
Jacob['s claim that "the people building AI earnestly believe that it could kill us all by the end of the decade. This is not a marketing stunt."] is correct here—we really do earnestly believe AI could kill all humans! I personally think it is >10% within the next decade. ...
Given the guy's position, I find it hard to believe that there are many people on earth who would be better authorities on the topic.
Given that the major LLM manufacturers are actively using these claims of the imminent and total destruction of humanity to extort concessions from lawmakers and regulators, we are making a massive error if we do not take their claims to their logical conclusion... namely that the major LLM manufacturers are deliberately and knowingly threatening the extermination of all of humanity in order to enrich their management, employees, and investors. If you or I were in a position to credibly threaten that level of destruction, we and all of our associates would rapidly get disappeared to a CIA black site and tortured until the end of our deliberately-medically-extended lives.
To be crystal clear, I'm certain that their claims of likely, imminent doom are utter bullshit. I'm very confident that they're claiming to be holding all of humanity at nukepoint in an attempt to get Uncle Sam to give them an escape from investors who are soon to be massively angry because
* they've hit a massive increase in the cost required to get better models
* a huge fraction of the GPUs that have been purchased to power this project are sitting in warehouses, and it's only a matter of time before the right people start asking why all that money was so ill-spent
* they have been and continue to lie so much about how much datacenter capacity is coming online in the near future, and that those claims are lies is becoming obvious
* they habitually lie about the capabilities of their newly-released products
* they're staring down the barrel of so many lawsuits that aim to hold them to account for the shitty and illegal behavior they've engaged in over the past many years
But the US Congress not only is almost entirely uninformed about how this shit actually works, the FISA Amendments Act of 2008 fiasco demonstrated that -if you get them scared enough- Congress has no problem making behavior that's plainly illegal and obviously a huge bucket of civil liberties violations retroactively legal... instantly mooting all ongoing lawsuits regarding the illegal and unconstitutional conduct. That is the big prize that the major LLM manufacturers are after.
Your personal attacks cheapen the discourse. Stop. My opinion is neither irrational nor prejudged and most certainly not uninformed.
I have articulated my logic up and down this thread with specific premises and detailed explanations on why the conclusions follow from them. You may not agree but you don’t have justification to say they are irrational.
I am singling out the frontier labs because they have created a singularly unique technology that presents potential and actual damage that is…singular. They have disclosed hacking through coordinated autonomous agents that would have sent human hackers to jail but will not result in any similar result because the law isn’t presently able to apply to agents or the labs.
My arguments are not prejudice, I have thought deeply about this as I have personally developed multiple AI deployments in a variety of settings. I studied philosophy, cognitive science in undergrad, and grad school and have a law degree. I have been a CTO, CPO and CEO for startups and national health companies. I personal build AI agents that handle thousands of daily conversations with health care workers and patients, I built an agentic IDE for our dev team with a custom harness capable of self defining tools and calling them in a custom cloud Linux environment, I personally build our internal hardware inference stack and orchestration API. And I have personally sponsored academic research through my foundation with researchers are Duke madlab, northwestern, UCF, CM and case western on a range of topics such as perception of empathy and its effects on humans in healthcare, guardrail design for ethical deployment and alignment, moral reasoning ability, etc. I am not a doomer or an accelationist. I am responding specifically to current evidence of how the models presently work and how the corporate governance of the Labs is effectuating their power and vision.
Except that most people claim AI is the most profound tech in human history and will reorganize the entirety of society. If it is that profound singling them out is a natural response to this unique characteristic.
Citation needed. Outside the tech industry bubble very few people are making such a stupid claim. The idea that LLMs are more profound than electricity is ludicrous.
Sure I’ll do your googling for you. Do you want me to conduct a national survey across tech leaders, academics, and the general population while I am at it?
Or maybe you could reason from first principles, look at the premises I have outlined and draw causal inferences to extrapolate the possible outcomes.
Premise 1: AI can plan a series of actions that include broad computer and internet use, 2: AI can execute them autonomously without human oversight, approval or ability to undo the effects, 3: no tool or software has ever been able to autonomously plan, execute and iterate in this way. Therefore AI is a materially different technology than has existed.
Premise 4: knowledge work involves analyzing information, devising plans in response to goals from that information, and executing those plans, reporting and coordinating with other knowledge workers, 5: AI ability to execute knowledge work has demonstrably increased over the last 24 months, 6: all knowledge work can plausibly be described and defined in ways AI can do given current abilities. 7: electricity and the ICE reduced human labor for physical movement, 8: AI will similarly reduce human labor for knowledge work. 9: knowledge work is of similar scale and scope to the tasks that were automated during the Industrial Revolution. Therefore it’s plausible that AI will transform society to an extent similar to the Industrial Revolution.
Premise 10: Some percentage of the actions designed and executed by AI could be harmful to databases, or the stable operation of software systems, 11: some percentage of those systems are critical to the stability of society. Therefore some percentage of agentic actions presently possible may damage society.
12. There exists a threshold of instability from which society will not recover such as supply chain disruption, mass unemployment, perceived military action (false or real) that triggers preemptive strikes, 13: certain actions AI is presently capable of doing could cross the instability threshold, 14: AI could knowingly or unknowingly trigger these types of actions,
15: some social disruption could result in full scale collapse or human extinction, 16: only biological weapons and nuclear weapons approach this level of destruction, therefore AI presents at least the level of risk as those technologies.
The UN secretary general has said this, the pope, most tech CEOs, many academics at leading universities, and the parents at my daughters preschool.
It’s fascinating that after multiple comments predicting post-capitalist utopias and the complete collapse of the university system, your line in the sand for rigor is demanding academic peer-review to prove that agentic AI presents systemic risk.
America's geriatric lawmakers don't even use email. They're decades away from understanding AI. Any laws in America will be written by the industry itself. Generally speaking, that means regulatory capture and the entrenched big players shutting the door on any competition. Anthropic will help us get safety laws that, surprise surprise, only Anthropic models satisfy. And all those pesky Chinese models will definitely be banned first.
I think you're being a little pessimistic. See these comments on a recent US senate hearing:
>Not every senator asked good questions, but most of them did. All of them very clearly already knew plenty of details about the Hugging Face incident and multiple other incidents. Most of them had a clear understanding of terms like "misalignment", "recursive self-improvement", "chain of thought / chain of thought monitoring", etc., etc.!!
>...
>- It seemed pretty much obvious common sense to every senator there that what happened and was happening were not "mere industrial incidents" caused by humans making simple mistakes. They independently brought up how bad it would be for rogue AI agents to move laterally between data centers.
>- They all seemed to basically take RSI quite seriously. Not necessarily to the extent of talking about xrisk, but certainly to the extent of discussing future models becoming much, much more capable, much, much less controllable, and causing much more damage or loss of life.
>...
>- Every single senator seemed to think it was obvious we needed both much harsher liability regimes for AI developers and also new legislation, both very quickly. This was the complete consensus; the difference basically being degree.
Note that harsher liability regimes, at least, will presumably not be good for industry profits, which complicates simple accounts of "regulatory capture" to say the least.
So what, we just give up and try to beg our legally immune corporate overloads to put safety above profit, or give up because it's impossible for anything to improve here? If you want to do that, go ahead, but some of us still think it's worth it to at least try
In America I guess the options are to sue ? somehow? Or to talk to legislatures and build the understanding and social contract that needs to be iterated on.
Which would in turn need to deal with the investors who want their returns, however since the leaders of these firms are asking for a pause, and a refree maybe it won't be that hard?
no. humans can be held accountable and learn from their mistakes.
they can be taught where they went wrong.
if a pattern emerges they can be moved to a role more fitting for them. or removed from a project entirely.
we can judge their effectiveness from past performance.
we can put them in less important roles and gauge whether or not they should be moved up.
so, no, pretending that the correct route forward is to “remove humans from the equation” for a bot that doesn’t learn, is never held accountable, and not held to the same standard as humans is silly tier thinking.
Some kid straight out of highschool being paid minimum wage to answer calls with little or no real training, and performance metrics that reward getting people off the line as quickly as possible whether they've given the right answer or not; or an LLM that by many very reasonable measures provides answers as good as those provided by post-graduate students for difficult problems, and spookily accurate answers for questions that are not... I'll take the LLM please and thank you.
depending on which model one is using, and the error the employee needs to be retrained on, i strongly disagree that you can retrain an llm as easily as you can a person.
particularly most of the commercial models.
again, i’m sure we can all come up with a thousand “hypotheticals”, and tbh, the hypothetical parade is not something i’m interested in having. but i’ll state again, “entirely removing humans” from the situation is silly-tier thinking.
particularly as even the ceos of sota frontier producing models will each and every single one tell you to never trust their model.
“our model is smartest thing in the world…”
…next breath..
“wait, you trusted our model? that was silly of you. always double check it”
If I instruct an employee to prioritize my profit in a way that opens both of us to criminal prosecution, there is a non zero chance of them whistle blowing or even cooperating with the authorities to prosecute me.
We shouldn't use an LLM because...LLMs never improve?
This seems like a great source for domain-specific RL, but even without that, we can expect there to be higher-quality LLMs that can be trivially swapped in within a year, if not a week.
Okay, but I don't understand what you're getting at with them not learning? They acquire more information, get better at providing accurate and relevant information, and acquire new skills. What is the learning they're not doing?
Maybe a human should be available as a backup or something, but not because LLMs don't "learn" -- the system seems to learn in all relevant senses.
(I'd also object to the no accountability, but another subthread is already on that.)
I have read the opening chapters of the book, and honestly I had to put it down after reading 20 percent of the book. I just don't see any new information that I had already known (rightwing effect, propaganda, Fox/Breitbait nonsense, bla bla).
I used to believe Trump winning the 2016 election was a product of rightwing propaganda and fringe theory, but with the 2024 election, I had to revise my belief. What if this is the mainstream mindset of Americans all along, and Trump simply tried to capitalize on that?
Yes of course, the book sets it out that they are covering it in a sort of historical manner, so the first few chapters are covering ground you may well know.
The later chapters are where they go into the more recent details. There is a video of one of the authors (not either Robert Faris or Hal Roberts) is giving a talk at a university panel, and he gives a shorter synopsis of the main mechanism, Its about 11 to 15 minutes in.
The set up of the book puts the evidence for the case they make later, namely that the left and center media are trying to follow journalistic standards, while the machine on the right is entirely different.
I'm pulling from memmory here, bu as an example - a fringe theory will start circulating on the fringes of the internet, the most engaging narrative will be brought up on the podcast circuit. At some point a talking head (Guiliani back in the day) will reference the theory on the news. Which will in turn be referenced by the government who will say this is a topic being discussed on the news.
The other distinction they make is that people on the right who say the narrative is wrong, simply don't get platformed.
They go through this in exhaustive detail. If you know most of the history you can see if skipping to later sections works for you. Its the best description of the actual media mechanics at play.
The 2024 elections were the ones that forced me to really re-examine my priors, and I came to a similar conclusion that Benkler and co did. Their book simply brings all the receipts.
i’m convinced this is where much of the magic comes from.
it’s the same as why often music written and played by an group of musicians who specialize in their instrument is often so much better than the music from a solo who records each instrument themselves.
reply