Benchmarks are one thing, but I've yet to see anything substantial actually built with local models. Most of the posts on X are about benchmarks and tok/s. Has anyone actually built a quality production app or game with real users?
Well, Greg KH emphatically states that local LLMs are more than good enough for bug finding in the Linux Kernel. [0] Seems strong enough for the task to me. [1]
He goes on to recommend that one uses local LLMs exclusively for bug finding because he and his colleagues have seen multiple occasions where folks who used the LLMs provided by the Major LLM Manufacturers had the stuff they were working on get "mysteriously" leaked.
Even if that sort of thing were not known to happen, it's just smart to not process data you want to keep secret in systems that are known to straight up retain user data and/or "anonymize" user data and send it back to their HQ for use in production of newer versions of their products.
[1] And even if they're six months behind the latest publicly-available tools provided by the Major LLM Manufacturers, six months ago a notable subset of folks were on here screaming that their latest tools enabled fully-automated hands-off coding and had ushered in an era of Software Factories that were rapidly going to make all programmers obsolete. ;)
Tbh I think it's mostly a matter of time. I use SOTA at work, but only local at home, and the performance improvements out of local models in the last few months have been substantial, and more are on the way. To add to that, people are starting get better at harness engineering (which is itself improving with things like Harness2 etc) and also now we see people doing things like fine-tuning a Qwen 4B model to do 3D modelling with some success etc. The latent capability for local models to do what you're talking about is there, but the in-the-trenches reality is people are still working through unlocking the full potential.
edit: I should point out that it's also starting to accelerate now in that you can get your agents to help you with making them better at these tasks. So, it matters significantly less if you yourself don't have the skills to progress things, so long as you can figure out just enough to get the models to do it themselves. I suspect we'll see this dynamic ramp up quite a bit in 2027 with Qwen4.
I've built an RSS reader and I'm doing a half assed reverse engineer / web version of Tapped Out (a silly little Simpsons mobile game that EA killed a while back). I've had to go to Claude a couple times when it got stuck on something curly, but I think that was mostly just loading a fresh context and I could have solved it I'd I really tried.
I've been using Qwen 3.8 and more recently Next for the last two months almost exclusively on home projects (I have access to a system with an RTX 6000 for free). It does everything I want it to! I have no issues believing it would be fine for more than just pet projects.
I am having pretty good success with DeepSeek 0731 locally. I have vibe coded some small things like an orchestrator. For a production product with real users, I will let it do first passes on code which I then review and polish up. I'm not oneshotting, but it's still a workflow benefit.
Well in large corp this takes time, even if you have an army of Einsteins at your disposal (just like we saw with Elons Doge) because there will be 16 different spreedsheets/dbs talking abt the state of the system that all contradict each other. While the 6 people who maintain their own state to keep the lights on, will have job titles that have nothing to do with system state maintenance.
Small companies its hit or miss cuz they are overwhelmed by the pace of change. Basically no one wants to hear things will take time. Intelligence is just one system variable and others dont just auto sync.
The actual tokens might be non-deterministic, but you could look for proxy measures that are supposed to be invariant. Eg. correctness/performance on benchmarks, "thinking level" on complex problems, etc
My harness expands the prompt into a json representation that specifies layout much more rigorously, which is why you see such that amount of alignment between the two.
That internal json backing helps significantly when you want to maintain consistent design system components/patterns across multiple pages. The aligned layout is it working as intended.
Totally normal for modern models due to training on the same datasets supplied by third parties, dataset contamination, and mode collapse, especially for simple prompts that don't have enough semantic capacity. -isms are often very similar even without distillation, and tend to come and go in waves along with model generations.
Agreed. There's also a lot of bad tinging/yellow saturation that very much reminds me of early gpt-image outputs on a lot of the non-cherry picked stuff I've been seeing on Twitter/Reddit.
A lot of people were putting ZiT as a refiner downstream in early Qwen-Image 1.0 workflows, so I'm wondering if we're going to see something similar with 2.1.
Yep. I had posted a comment earlier detailing my usecases. But I too replaced that with my own system that does those same things.
It's way too bloaty, felt like operating windows start menu search.
But you might have missed so far some of the ideas they have. So it's useful to try it out, see what combination of features you use in particular and then just set those up for yourself with claude code or whatever as the LLM harness. Telegram integration is dead easy.
I ended up using QMD + custom skills to keep it up to date as new data comes in. Essentially a cross-referenced set of markdown files forming a knowledge base: https://github.com/tobi/qmd
You should open the issue with your IT department and see if they can optimize the performance. You can gather proof that GoGuardian is causing slow downs by looking at it's memory usage in the Task Manager (right click empty area in Chrome tab bar).
Thanks, completely agree. UX is probably the hardest part here. Prompting should not be a prerequisite for getting value. We have been thinking about making the system more proactive, for example surfacing relevant notes ahead of meetings or highlighting changes that need attention. Would love to hear how you think this should ideally work.
reply