Arguably the customer gains the option of not paying, which has some value. Whether that option value is close to the extra they pay through the subscription is left as an exercise for the reader.

I can't justify this at all, but it feels like there's headroom in the currently-available hardware, which the current shortage will increasingly force us to optimise into. Then when the choke-point relaxes the upper bound of what's achievable will be even higher than it would have been.

I don't think there's much room that isn't being exploited by this SSD expert streaming stuff.

There may well be headroom within the software/architecture of LLMs but once you get down to streaming tokens, it is kind of running into the physics?

There's plenty of optimisations still possible if you optimise for specific GPU families. Eg see Hipfire[1] and q38rocm/HaloPFX[2], they're so much more faster than llama.cpp for consumer AMD GPUs.

[1] https://github.com/warpfront/hipfire

[2] https://github.com/julianmb/q38rocm

OK, yes, this I do agree with but I was mostly factoring these in (because I did that research a couple of weeks ago while speccing out a hypothetical R9700 box)

One of the architectural changes expected in the next series of Qwen models appears to be well suited to SSD streaming.

But also, there probably still is a lot of room for tuning and better utilizing the streaming capability, e.g. better speculative streaming to have the activated experts loaded in RAM by the time they're needed.

us-east-1 is all you need, surely.

Push-to-talk?

Yes I want push to talk in ness

But trying to implement it directly into ness hasn't worked that well. It seems like betterwispr/wisprflow/etc all seem to belong better as their own app, and we just need good integration points between the push-to-talk app and the text consuming apps (like the IDE)

It's more literal than that. The prospect of reward is very literally not motivating. It does not provide motive force. It does not make us move.

We consciously want the thing, but that isn't enough for executive function to kick in, so "I need to get off the sofa so I can make tasty dinner" results in not having moved two hours later.

We have people who want to fill those positions because there are enough people who find it rewarding enough. Take away reasons why they would find it rewarding and you will have fewer theories and results that will advance our civilisation.

And yes, fun counts. Nobody said this had to be only a hardship.

Money doesn't work that way though. There are plenty of jobs people would like to get paid to do, that doesn't mean someone needs to psy them to do it.

I'm well aware that if at some point AI is good enough to replace me as a software engineer then I won't have a job. I don't expect a company to continue to pay me simply because I enjoy it if there are cheaper options out there.

Math is no different.

As long as that company doesn't expect me to continue in their employment if I stop enjoying it, then we understand each other.

Total compensation includes fun.

Oh sure, you're well within your right to leave a job when you're sick of it. Nothing wring with that.

So what? Why should my life be ruled by money alone? Why should my life not have a mix of intrisic and extrinsic motivational drives? Why should I respect a company like OpenAI that disrespects my mix of extrinsic and extrinsic drives when I am the customer?

Or am I just a product to them?

This is really the critical thing: the fun is the incentive. (Or at least the dominant incentive in math, historically.) As economists like to say, the overarching lesson in economics is that incentives matter. Reduce the incentives and participation will decrease.

Perhaps that won't matter if we enter an era where AI participants are the main participants who matter for discovery-level mathematics. But it would likely be what economists would see as a market failure if only a small oligopoly of AI participants, closely held behind closed doors, is able to fill that intellectual role.

SBCL can produce very fast code. It's really slept on from that point of view.

A couple of years back there was a paper doing the rounds about software execution efficiency, which a lot of people in my neck of the woods got interested by because of the potential for environmental impact at the sort of scale we operate at. SBCL ranked disturbingly highly for something that is culturally never going to happen for us.

I got downvoted to oblivion on a reddit thread where someone was asking how they could handle the massive context that an agent needed to load to understand their codebase when I suggested that exactly the same techniques which let humans manage it would work on an LLM.

Models aren't good at this by default, so you end up with spaghetti mess. But if you tell them that you want strong interfaces, isolation, and types, they can work that way.

Yeah, I've got a 10 Pro, jumped up from an 8. Looked briefly at upgrading to the 11 but it's clearly not worth it.

There was a music editor I remember getting on a late Your Sinclair edition cover tape which displayed proper music staves and gave you 4-note chord output. Blew my mind a bit at the time.