They're clearly betting few will do this. And/or they'll make more money selling the storage elsewhere.

We'll see.

Not really. He said no one who knows anything about the brain thinks it's a binary system.

AFAIK, that is true.

Whether it's possible to replicate what happens in the brain on a binary system or not is a separate thing entirely. It's also not what LLMs are attempting to do.

You're reading him too narrowly. He's answering the question of whether the brain can be modeled (math or magic), and arguing against it. Or maybe he's arguing for a middle ground. Hard to say exactly, he's not very clear.

Except the brain can be modeled with arbitrary precision - so he's got an uphill battle in either case.

TLDR: Yes, he literally said "binary computer" but from the context it's clear he means more than that.

Yea.

I think a better argument for them not reasoning is that they are incapable -- seemingly, for now, that I know of -- of independently starting a reasoning task.

They do not ask whether or not they dream of electric sheep. Unless prompted to do so.

That's part of human reasoning, being prompted, but so is independent thought.

So at best they are partially reasoning, unless we philosophically argue these are two separate things. And frankly, shrug, I'm not a philosopher.

In the early reasoning models the chain of thought would often drift off into other topics, almost like a daydreaming. This behavior is undesirable so that's been RL'd away, but I do think this is something we could evoke from models. It's just difficult to contend with from an eval perspective.

This is a good point and we would have to understand whether this is noise in the machine or independent reasoning. I haven no idea how to measure that.

If somehow independent reasoning then we're getting very close to something alive that would need rights.

In my view.

And I have yet to have an experience with an LLM where I came away thinking -- this thing is alive and me using it this way is unethical. I've had that experience with animals, even people. Never -- yet -- an LLM.

> And I have yet to have an experience with an LLM...

I hope you agree that this question is too important to leave to gut feelings.

Humans have a poor track record in that regard.

No question. As we were discussing it needs measured somehow.

Right. So I'm wondering about the relevance of you citing your gut feelings. It seems like you are putting stock in them. No?

> they are incapable [...] of independently starting a reasoning task.

This is a simple and intentional design choice though. Biological lifeforms are always on and always receiving sensory input. LLMs don't functionally exist out side of when we decide to run them. An always on agent with a looping prompt of "if you aren't doing anything else, ruminate" overcomes this limitation.

Not really. Noone has to prompt you to ruminate.

In fact, if your brain were removed from your body, and you were locked in a room that was completely empty and precisely calibrated to your brain's ambient temperature, and you were then ordered on your life to not ruminate. Well -- I would posit that you wouldn't last very long.

There's something else going on with human cognition -- and reasoning by extension -- that LLMs are, to date, not replicating.

With the big caveat of AFAIK. I'm not in one of the labs close to this stuff.

> Not really. Noone has to prompt you to ruminate.

Why do the internal mechanics need to mirror how it works in humans?

I have agents that receive sensory input (video/audio) which run continuously processing it, receiving events as they occur. When something strange happens they notice and take action (i.e. there's an unrecognized person in the living room -> ask who they are.)

Presumably you've seen the agents that play video games - build things to progress in Minecraft, plan routes, avoid adversaries, etc.

Why doesn't that count?

I feel like you're failing to consider anything outside the chatbot UI.

EDIT: The first morning my agents had access to an outside video stream they commented on the sunrise. While I was sleeping through it.

You make a lot of out of context assumptions and feel.

I'm sorry? I'm answering your claim:

> they are incapable ... of independently starting a reasoning task.

I have provided several examples of them independently starting reasoning tasks.

... and I'm wondering why you seem to be entirely unaware of such examples. The most likely explanation being you are only aware of the chat interface.

... and I have absolutely no idea of what you mean by "... and feel".

I think you’re on the right path. My first thought was whether lack of constant sensory input is an issue. We’re always feeling something (save for sensory deprivation chambers, but even then…)

Or put it other way, lack of feedback loops between the “brain” and the environment.

Madison talked about this problem during the Virginia ratifying convention. There is no government structure that can survive bad actors. His solution was that informed voters would punish bad actors. -- We're in short supply of informed voters and accountability isn't a thing anymore.

Punishing bad actors depends on informed voters, but in a larger population democracy it also depends on the informed voter : uninformed voter ratio staying above some value.

The lower the ratio sinks, the less weight informed voters have, and the less intelligent the outcome gets.

The modern party duopoly exacerbated the issue (parties mobilize uninformed voters and often actively work against informing them), but voter enfranchisement also seriously impacted.

Universal suffrage is amazing from a human rights and ensuring everyone has political power standpoint, but troublesome when you have poor levels of universal education and ability for voters to inform themselves.

In modern times, with modern populations (US or larger), it feels like indirect democracy might be a better answer.

Allow voters to directly elect non-partisan voting representatives (many, so elections are hyper-local and candidates individually canvas), then pay those representatives a stipend for a few months (like jury duty) which they spend educating themselves on issues. Then have those representatives vote.

Sorry. This is utopian. If those, let's call them electors, "educated themselves" with sources from RAND or worse, Fox/Breitbart types. Or to be non-ideologically biased about the point, communistUSA. Then they too would act as bad actors when they voted.

At the end of the day, when all the BS gets boiled away and we are left with raw/brutal reality. -- Any group of people attempting to govern themselves will end up with the government their collective will and (i'll use the loaded term) virtue deserves. The only caveat is time.

This couldn't ever work. In a crisis like 2007 the US rapidly going into deficit spending is the only thing that kept the financial system running. Have a hard cap during the next crisis and the Great Depression will start having numbers beside it. Economics is a soft science and politics isn't a science at all.

Any economist on such a body would have seen that the post-Bear Stearns reaction in 2008 merited exceptions to fiscal policy.

Such a body presumably would also realize that deficit spending on an average non-crisis Tuesday should be much more tightly reigned in.

I chose the Fed as an example because, whether they're correct or not, they absolutely pivot around exceptions.

This is precisely my point. Who decides what is and is not a crisis. Germany tried a version of what you're advocating for and recently realized it was unsustainable, and declared a crisis so they could up defense spending.

Trump has declared crisis events no less than 20 times to justify his insane policy decisions.

We're just playing pretend games now.

The entire point of the Fed et al. is to remove the ability to decide what is and isn't exceptional from those who have political stakes in declaring everything exceptional.

You've probably noticed, people like this like to adjust the goalposts.

That won't solve anything. Congress already has this power they just abdicated it to the Executive. The only fix is better voters electing better people into Congress. -- good luck with both of those. The US is cooked.

No, it’s not good enough, you need a better system. Like in Switzerland

The Los Angeles metropolitan area has more people that Switzerland.

Population aside, there's no system that fixes what's broken in the US in the 21st century.

Trump, sadly, is the symptom of deeper structural ills that have been building in America for decades.

Congress moves a lot slower, though. Congress isn't going to change tariffs every 24 hours. It takes a lot longer to get anything through Congress.

I mean, you still have the "every 4 years", or worse, "every few months". It's not at the pace that Trump changes things, though.

There is absolutely 0 reason for tariffs to need to change every 24 hours. It is economically suicidal to do so. In fact.

That's not going to happen. These people are not competent.

Yeah, but I want the suckers who still defend them to put up or shut up.

Yes there are two interesting derivative questions from this, assuming I understand it.

First, is it reproducible consistently at ~50% of workers? If not, what is the rate.

Second, are there any lessons to be learned here to increase the rate of success by changing models/weights/training?

The news by itself isn't really good news. But it could lead to good news. Maybe.

In my experience with them, Trump voters have no shame.

We should bring honor and Shane back into our culture.