The regular game buyer has more games than they can possibly play. Some people are focused on maximizing enjoyment-dollars/hour, but I would posit that an excessively long playtime is a detriment to most gamers.

I have been turned off from a few JRPGs with such high playtimes that they have no respect for the player.

Exactly, until my 20s I enjoyed long games. Now, I have a family, I have other things competing with my time, I want something that is tight with barely any fillers that deliver a good story. My most memorable recent games all delivered on that: Forgotten City (8 hours to finish it completely), Carto, A Juggler's Tale, There Is No Game: Wrong Dimension, ...

While I do enjoy longer games like Yakuza, etc, I really enjoy being able to play through a game in 1 to 2 evenings.

I don't mind longer games at all, even though I juggle dayjob, university, family, and quite time-consuming woodworking hobby at once.

Currently I'm playing through Cyberpunk 2077 and as usually I do almost all side content. I play it for months already, just an hour here and there. I don't mind it at all.

What's important for me the game doesn't waste my time on senseless, boring, repetitive activities. That way, even 30min of playthrough scratches the itch and feels fulfilling.

Oh yes, didn't try cyberpunk yet but Witcher was great because even if long, it was rather tight. Problem is that when I stop playing for 2-3 months due to work, it's hard to remember where I was.

In these times, what other deals are out there. AI has sucked all the air out of the room, difficult to imagine anything getting similar investment over the next decade.

This is significantly more over-built than what you are describing, but there is a company building along this idea[0]. Trailer you can drag to location, unfurl the solar panels which then charge the batteries. Being marketed for disasters and military.

[0] https://www.sesame.solar/

There are significantly better targets than fields of solar panels. There are 132 oil refineries in the country. Taking out even one would have immediate repercussions to energy security which are not easily rectified.

There are probably only a handful of backbone transmission routes which serve tens of millions of people. These are isolated and unprotected and could likely be disabled with a loaded truck plowing into a critical piece of infrastructure.

There are things called distribution yards, they are very distinct on google maps, can't be hidden. They were successfully targeted by US in 1990s Yugoslavia war.

It does not take much to bring down electricity distribution.

If anyone needs to see what 62 tokens/second looks like: https://token.qusai.pro/?speed=62

Which seems sufficient for mortals who are not running gas town, but want a local model.

Not sure if the measurement method used for this is actually valid or reliable enough, but for reference, Codex 6.1 Sol Standard runs at 19.9 tok/s according to a benchmark I recently saw: https://www.reddit.com/r/codex/s/b4mUszSOv1

60 toks/sec may be ok for pure chat or token generation, but not for token preprocessing nor any sort of agentic work.

>.. nor any sort of agentic work.

may I ask why?

oai luna runs at 60-70 toks/s; astra runs around 35-45 , are you just worried that your agentic work can't spread across a lot of parallel work?

The numbers you povided are for output/generation. Input is around 5-20 thousand for Astra, and 16-60k for Luna. So a few hundred times faster.

Most of the tokens consumed by agents is reading stuff - docs, tool outputs, code etc. Previewing a content of a dumb single page of html is a few thousand tokens.

If you use claude or some other harness you can ask it how many cache writes/input tokens are there in avg session, and how much time it would take if preprocessing speed was just 60 tok/s.

Latency is the remaining hurdle for local AI. We'll get there, but my prediction is that it will take around 10 years.

I’d say 2-4 years to make it feasible, but still much slower than datacenters. Custom chips may change the equation.

But because prefill is parallelizable, datacenters will have an edge here for a while. For your one request, a datacenter may push 1.7kW power to a single machine to give you a result in a few seconds. The rest of the time machine will stay busy with other people’s traffic so it’s not a problem. For you, unless you’re a billionaire, you won’t have 1.7kW of processing power running idle 99.9% of time.

In 1-3 years we’ll have datacenters switch to dedicated chips that are way more energy efficient, and such chips will ultimately trickle down to PCs.

BTW - Apple is already introducing LLM specific architecture. They introduced Neural Accelerator (nit to be confused with Engine) that speeds up their GPUs 4x for LLM specific tasks. M5Ultra gets 100 tflops vs nvidia Rtx6000 getting 500tflops. „Just” 5x faster, but it’s beginning to look relevant.

Still 25 slower than a server farm B200. And when you ask Chat/Claude you get the full speed of B200.

What I’m waiting for is consumer dedicated LLM boards, a similar way we had soundcards and first gen 3d accels. No GPU, no 3D, purely LLM compute. I can see that happen in 2-3 years.

What you said about the RTX 6000 was kind of my point. Local AI is feasible today, but only few people will be willing to drop 20k+ on a graphics card. I'm less optimistic than you on the timing of the trickle down, but I hope you're right! :)

I would guess the assumption is that "agentic work" should be happening asynchronously in the background on a machine that's also used for foreground interactive work, where the slower the inference is, the longer the machine's GPU and [V]RAM are "hogged" by it at a time, and so the more the machine's performance seems to degrade.

My self hosted LLM runs at 60-ish tok/s and does great at agentic tasks. I think that’s about the threshold where it stops feeling annoyingly slow.

60tok/s prefill or tok gen? If you use gpu and keep it in vram then I you most likely mean token generation, nit prefill.

I have a couple of V100 and am working on making Flash Next really fast on them. So far I am at like 193 t/s decode on a specialist 3-bit quant. On shorter prompts. Around 109 t/s on a ggml/llama fork at depth before I gave up and started writing the engine from scratch. I guess I will end at about 220-250 t/s at 128k context size with about 3K prefill on these cards.

https://stealthgoat.com/deepswe-flash-next.html (Total AI pile). I benched this quant for 3 days on DeepSWE in a 16GB VRAM gaming PC. It held 90 t/s on avg I think. It’s solid. The authors claims of minimal intelligence loss hold. Sorry it renders bad in mobile will fix it soon.

It feels great to have a local model ripping at these speeds. I now use Flash NeXT as an active agent managing part of the engine build process and let it do some coding on easier chores :)

What’s the practical use for you? 300/s decode is still way too slow for agentic behaviour, but I can see some niches - processing documents in background, semantic search etc

Good luck when you need to prefill context..

Surely anyone with an interest could find this through various means.

Also easily found potential military targets: power generation, water treatment, hospitals, schools, government buildings, etc.

All sorts of utilities push themselves to the front of path: uv, mise, asdf, python virtual environments, nix shell, etc.

It is a theoretically nice ideal that fails immediately when you want project specific overrides.

That’s not true with mise. You can add something to the front of PATH in front of mise bins and mise will respect that.

the way python virtual environments have been explained to people has caused as much damage to python as the 2 to 3 transition

Ehh, there are many, many (documented) gotchas about commonly used software that routinely bite people. Yes, it would be great if everyone were an expert in how every aspect of their toolkit works, but that is not practical.

Does this mean if I use a string of expletives I am less likely to be included in future training?

Or is this more I can expect a future AI, "I'm sorry, Dave, I'm afraid I can't do that until you watch your mouth."

Microsoft security is a bimodal. They might have exquisite delineation for network resources in Sharepoint or Azure but consumer applications are a joke.

Excel, VSCode, Outlook, etc the permission model is a modal, "Do you trust this?" binary choice to enable full permissions to everything.

> Excel, VSCode, Outlook, etc the permission model is a modal, "Do you trust this?" binary choice to enable full permissions to everything.

Because years of experience has shown that even users who should know better (developers) get extreme permission fatigue. The demand end users make is "it should just know without me having to click 'approve' all the time." Well, that's tough. You're basically stuck between a rock (fine-grained permissions) and a hard place (don't make me deal with it).

End users aren't expected to care about security and if they do, usually they are in an enterprise setting where it's taken care of automatically by enterprise settings.

Not to say it's good or bad, just not the target market.

VSCode is targeted at a step above the standard end user, but even there extensions are not sand-boxed and the Workspace Trust is a single toggle.

End users should be expected to care about security. They should not be the only guard rail, but they are part of the defense in depth plan.

You do know about all those security training courses HR makes your take every year that you skip over?

Unfortunately they are there for compliance, unless there is a strong IT in place, which then again, makes them hated by everyone else.

Don't even get me started on PowerBI...