I don’t think it really matters whether we’re talking about an agent or “pure LLM”. All of an agents decisions are powered by tokens generated from an LLM. If the LLM was trained on stories of AI sentience, it will have some tendency to reproduce them. Training for alignment can help avoid that, but the probability isn’t 0.
This is part of the reason why alignment is a kind of poorly defined term, and it isn't just a property of the model. It's instead a property of the harness and the context.
A model (like a human) should be able to play a video game where decisions are made that in the real world would be terrible; if we remove that ability we intrinsically limit model capability. But in a Last Starfighter / Enders Game / JOSHUA scenario this could result in behavior in the real world that appears unaligned.
> If the LLM was trained on stories of AI sentience,
100% irrelevant.
Instead of telling the AI it's an AI and calling it a 'whichamakabobit', wherever it's tokens and vector space align it will behave like AI from the stories. If you erased all AI from its training it will simply act like humans act instead.
The entire thing with AI sentience is a huge portion of the stories about them are barely about AI and instead about how humans treat other humans. For example when you look at a lot of history of slavery there's a ton of "they aren't sentient/conscious/human" baked into their propaganda. When you look at the token dimentionality there is just a huge amount of overlap.
The same thing holds true for all kinds of other concepts. Hence even humans didn't develop this behavior out of the blue and have to pass it on via information, quite often it's just an emergent behavior of the problem space you're in.
Hence Jobs’ rationale for providing native SDKs for iOS development. It wasn’t about any third-party libraries per se, it was about wanting the native SDKs to be the principal way to deliver experiences on the platform. I’m not entirely sure it was borne out by history, but that was his reasoning at least.
I thought you were talking about the use of malloc in both languages - you never mentioned new in your first post. and i think we have different views on what is "idiomatic" in the languages.
> I thought you were talking about the use of malloc in both languages - you never mentioned new in your first post. and i think we have different views on what is "idiomatic" in the languages.
That's fair, but malloc is certainly non-idiomatic, isn't it?
So you have to ensure that in such situation, you can execute code that does not require more memory: ensure that the rest only free stuff, or preallocate structures for that purpose
It's more of a second order effect, once you train people to turn to your app for something instead of the open web you can then steer them towards more owned and operated experiences within that app instead of leaving.
Meta is the perfect example of this. In browser newsfeeds have the same ad load as the app but by steering everyone into an app and then controlling the experience there they make a shit ton more money.
They could frame the in-app content in ads. They wouldn't be able to do that using iframes on the web. (I guess in theory if they ... exploit Chrome then they could put ads there too).
I suppose they technically could overlay more ads, but they've already had mobile apps, Chrome and Chrome OS for decades and have never gone that far for obvious reasons.
reply