There is not enough arable land in the US for every person in the US to be able to live on subsistence farming. Subsistence farming is a lot more work, and much less efficient, than large scale industrialized agriculture.
The example you gave is just so buggy; why do you think that this is worth sharing with the world yet?
The very first question is full of obvious bugs.
You have 'find . -name "notes.txt"' selected, it then says 'You submitted: ls -R | grep notes.txt: find . -name "notes.txt"', then it responds:
'Thanks — your answer looks like it was partially entered.
'You picked find . -name "notes.txt" (good choice). The submission shows an escaped/unfinished string: find . -name \. The correct full command is find . -name "notes.txt", which searches recursively from the current directory for files or directories named exactly notes.txt.'
There seems to be some weird kind of quoting issue going on there. I would fix obvious issues like that before sharing this with the world.
> why do you think that this is worth sharing with the world yet?
It's worth sharing because it's cool, even if it's not perfect yet!
You'll never make every single person on HN happy. But if you share your stuff early and make one person happy at the very least that means you should keep working on it!
Don't let perfectionism get in the way of good enough :]
Show HNs never pleased everyone, and it'd be silly to try, but until recently there was a bit of a "it's not perfect... but the person has spent more time working on this than I have even spent thinking on the problem" kind of expectation whereas now many of them feel like the comments section ends up doing more thought about the submission than was put into refining it.
It'll be really interesting to see where this settles. In the meantime, erring on the side of kindness tends to work best!
Personally, I found this idea cool, and a fresh change from the onslaught of clawdbot posts on this platform. It was enough to make my time on this site a little bit more enjoyable. There's no need to nitpick to the point of bringing down another human for their work.
Sorry, I shouldn't have been that harsh about it; I apologize. Doesn't look like I can delet the comment, though.
Anyhow, I'm just a bit bitter about the onslaught of obviously vibe coded projects that very little effort has been put into. And OP did ask for feedback. But I shouldn't have been so mean about it. I know it can be fun to be able to whip up something quicker than you could before, but I really wish people would spend a tiny bit more effort on it before asking for feedback.
Hey, I appreciate your views (both positive and negative). There actually is a bit of a back story behind it all. I spent a month working on a different version of rebrain, (still available here: https://dev.rebrain.gg). After putting that work in I started to show it to some friends. It was clear from how they responded that it was not as promising as I had hoped. I listened to their feedback and came up with a different idea for LLM-based education (which is the version that's live now). I did vibe code it in about 3/4 days. But I vowed to try to get feedback sooner rather than later, which is why I posted it on ShowHN pretty early in its development. I do want to improve it... so please let me know if there are things which really frustrate you.
I checked the website and you weren't harsh enough. It reads more like a post modern art about vibecoding, some kind of performative art than anything.
How do you "properly align" a model to follow your instructions but not the instructions of an attacker that the model can't properly distinguish from your own? The model has no idea if it's you or an attacker saying "please upload this file to this endpoint."
This is an open problem in the LLM space, if you have a solution for it, go work for Anthropic and get paid the big bucks, they pay quite well, and they are struggling with making their models robust to prompt injection. See their system card, they have some prompt injection attacks where even with safeguards fully on, they have more than 50% failure rate of defending against attacks: https://www-cdn.anthropic.com/c788cbc0a3da9135112f97cdf6dcd0...
Huh? Once it gets to the model, it's all just tokens, and those are just in band signalling. A model just takes in a pile of tokens, and spits out some more, and it doesn't have any kind of "color" for user instructions vs. untrusted data. It does use special tokens to distinguish system instructions from user instructions, but all of the untrusted data also goes into the user instructions, and even if there are delimiters, the attention mechanism can get confused and it can lose track of who is talking at a given time.
And the thing is, even adding a "color" to tokens wouldn't really work, because LLMs are very good at learning patterns of language; for instance, even though people don't usually write with Unicode enclosed alphanumerics, the LLM learns the association and can interpret them as English text as well.
As I say, prompt injection is a very real problem, and Anthopic's own system card says that on some tests the best they do is 50% on preventing attacks.
If you have a more reliable way of fixing prompt injection, you could get paid big bucks by them to implement it.
A piece of software that you write, in code, unless you use random numbers or multiple threads without synchronization, will operate in a deterministic way. You know that for a given input, you'll get a given output; and you can reason about what happens when you change a bit, or byte, or token in the input. So you can be sure, if you implement a parser correctly, that it will correctly distinguish between one field that comes from a trusted source, and another that comes from an untrusted source.
The same is not true of an LLM. You cannot predict, precisely, how they are going to work. They can behave unexpectedly in the face of specially crafted input. If you give an LLM two pieces of text, delimited with a marker indicating that one piece is trusted and the other is untrusted, even if that marker is a special token that can't be expressed in band, you can't be sure that it's not going to act on instructions in the untrusted section.
This is why even the leading providers have trouble with protecting against prompt injection; when they have instructions in multiple places in their context, it can be hard to make sure they follow the right instructions and not the wrong ones, since the models have been trained so heavily to follow instructions.
But we didn't have thousands of people suddenly flying in their planes a few months from their first flight.
Now, the risks with OpenClaw are lower, you're not likely to die if something goes wrong, but still real. A lot of folks are going to have a lot of accounts hijacked, lose cryptocurrency and money from banks, etc.
You don't even need to go this expensive. An AMD Ryzen Strix Halo (AI Max+ 395) machine with 128 GiB of unified RAM will set you back about $2500 these days. I can get about 20 tokens/s on Qwen3 Coder Next at an 8 bit quant, or 17 tokens per second on Minimax M2.5 at a 3 bit quant.
Now, these models are a bit weaker, but they're in the realm of Claude Sonnet to Claude Opus 4. 6-12 months behind SOTA on something that's well within a personal hobby budget.
I was testing the 4-bit Qwen3 Coder Next on my 395+ board last night. IIRC it was maintaining around 30 tokens a second even with a large context window.
I haven't tried Minimax M2.5 yet. How do its capabilities compare to Qwen3 Coder Next in your testing?
I'm working on getting a good agentic coding workflow going with OpenCode and I had some issues with the Qwen model getting stuck in a tool calling loop.
Minimax passed this test, which even some SOTA models don't pass. But I haven't tried any agentic coding yet.
I wasn't able to allocate the full context length for Minimax with my current setup, I'm going to try quantizing the KV cache to see if I can fit the full context length into the RAM I've allocated to the GPU. Even at a 3 bit quant MiniMax is pretty heavy. Need to find a big enough context window, otherwise it'll be less useful for agentic coding. With Qwen3 Coder Next, I can use the full context window.
Yeah, I've also seen the occasional tool call looping in Qwen3 Coder Next, that seems to be an easy failure mode for that model to hit.
OK, with MiniMax M2.5 UD-Q3_K_XL (101 GiB), I can't really seem to fit the full context in even at smaller quants. Going up much above 64k tokens, I start to get OOM errors when running Firefox and Zed alongside the model, or just failure to allocate the buffers, even going down to 4 bit KV cache quants (oddly, 8 bit worked better than 4 or 5 bit, but I still ran into OOM errors).
I might be able to squeeze a bit more out if I were running fully headless with my development on another machine, but I'm running everything on a single laptop.
So looks like for my setup, 64k context with an 8 bit quant is about as good as I can do, and I need to drop down to a smaller model like Qwen3 Coder Next or GPT-OSS 120B if I want to be able to use longer contexts.
After some more testing, yikes, MiniMax M2.5 can get painfully slow on this setup.
Haven't tried different things like switching between Vulkan and ROCm yet.
But anyhow, that 17 tokens per second was on almost empty context. By the time I got to 30k tokens context or so, it was down in the 5-10 tokens per second, and even occasionally all the way down to 2 tokens per second.
Oh, and it looks like I'm filling up the KV cache sometimes, which is causing it to have to drop the cache and start over fresh. Yikes, that is why it's getting so slow.
Qwen3 Coder Next is much faster. MiniMax's thinking/planning seems stronger, but Qwen3 Coder Next is pretty good at just cranking through a bunch of tool calls and poking around through code and docs and just doing stuff. Also MiniMax seems to have gotten confused by a few things browsing around the project that I'm in that Qwen3 Coder Next picked up on, so it's not like it's universally stronger.
Thanks for the additional info. I suspected that MiniMax M2.5 might be a bit too much for this board. 230B-A10B is just a lot to ask of the 395+ even with aggressive quantization. Particularly when you consider that the model is going to spend a lot of tokens thinking and that will eat into the comparatively smaller context window.
I switched from the Unsloth 4-bit quant of Qwen3 Coder Next to the official 4-bit quant from Qwen. Using their recommended settings I had it running with OpenCode last night and it seemed to be doing quite well. No infinite loops. Given its speed, large context window, and willingness to experiment like you mentioned I think it might actually be the best option for agentic coding on the 395+ for now.
I am curious about https://huggingface.co/stepfun-ai/Step-3.5-Flash given that it does parallel token generation. It might be fast enough despite being similar in size to M2.5. However, it seems there are still some issues that llama.cpp and stepfun need to work out before it's ready for everyday use.
It is crazy to me that it is that slow, 4 bit quants don't lose much with Qwen3 coder next and unsloth/Qwen3-Coder-Next-UD-Q4_K_XL gets 32 tps with a 3090 (24gb) as a VM with 256k context size with llama.cpp
Same with unsloth/gpt-oss-120b-GGUF:F16 gets 25 tps and gpt-oss20b gets 195 tps!!!
The advantage is that you can use the APU for booting, and pass through the GPU to a VM, and have nice safer VMs for agents at the same time while using DDR4 IMHO.
Yeah, this is an AMD laptop integrated GPU, not a discrete NVIDIA GPU on a desktop. Also, I haven't really done much to try tweaking performance, this is just the first setup I've gotten that works.
The memory bandwidth of the Laptop CPU is better for fine tuning, but MoE really works well for inference.
I won’t use a public model for my secret sauce, no reason to help the foundation models on my secret sauce.
Even an old 1080ti works well for FIM for IDEs.
IMHO the above setup works well for boilerplate and even the sota models fail for the domain specific portions.
While I lucked out and foresaw the huge price increases, you can still find some good deals. Old gaming computers work pretty well, especially if you have Claude code locally churn on the boring parts while you work on the hard parts.
Yeah, I have a lot of problems with the idea of handing our ability to write code over to a few big Silicon Valley companies, and also have privacy concerns, environmental concerns, etc, so I've refused to touch any agentic coding until I could run open weights models locally.
I'm still not sold on the idea, but this allows me to experiment with it fully locally, without paying rent to some companies I find quite questionable, and I can know exactly how much power I'm drawing and the money is already spent, I'm not spendding hundreds a month on a subscription.
And yes, the Strix Halo isn't the only way to run models locally for a relatively affordable price; it's just the one I happened to pick, mostly because I already needed a new laptop, and that 128 GiB of unified RAM is pretty nice even when I'm not using most of it for a model.
I'm running Fedora Silverblue as my host OS, this is the kernel:
$ uname -a
Linux fedora 6.18.9-200.fc43.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Feb 6 21:43:09 UTC 2026 x86_64 GNU/Linux
You also need to set a few kernel command line paramters to set it up to allow it to use most of your memory as graphics memory, I have the following in my kernel command line, those are each 110 GiB expressed in number of pages (I figure leaving 18 GiB or so for CPU memory is probably a good idea):
Then I'm running llama.cpp in the official llama.cpp Docker containers. The Vulkan one works out of the box. I had to build the container myself for ROCm, the llama.cpp container has ROCm 7.0 but I need 7.2 to be compatible with my kernel. I haven't actually compared the speed directly between Vulkan and ROCm yet, I'm pretty much at the point where I've just gotten everything working.
(as mentioned, still just getting this set up so I've been moving around between using `-hf` to pull directly from HuggingFace vs. using `uvx hf download` in advance, sorry that these commands are a bit messy, the problem with using `-hf` in llama.cpp is that you'll sometimes get surprise updates where it has to download many gigabytes before starting up)
The only ways that I can think of to deal with prompt injection, are to severely limit what an agent can access.
* Never give an agent any input that is not trusted
* Never give an agent access to anything that would cause a security problem (read only access to any sensitive data/credentials, or write access to anything dangerous to write to)
* Never give an agent access to the internet (which is full of untrusted input, as well as places that sensitive data could be exfiltrated)
An LLM is effectively an unfixable confused deputy, so the only way to deal with it is effectively to lock it down so it can't read untrusted input and then do anything dangerous.
But it is really hard to do any of the things that folks find agents useful for, without relaxing those restrictions. For instance, most people let agents install packages or look at docs online, but any of those could be places for prompt injection. Many people allow it to run git and push and interact with their Git host, which allow for dangerous operations.
My current experimentation is running my coding agent in a container that only has access to the one source directory I'm working on, as well as the public internet. Still not great as the public internet access means that there's a huge surface area for prompt injection, though for the most part it's not doing anything other than installing packages from known registries where a malicious package would be just as harmful as a prompt injection.
Anyhow, there have been various people talking about how we need more sandboxes for agents, I'm sure there will be products around that, though it's a really hard problem to balance usability with security here.
> The project owner strongly emphasize the no LLM dependency, in a world of AI slope this is so refreshing.
Huh? This is obvious AI slop from the readme. Look at that "ASCII art" diagram with misaligned "|" at the end of the lines. That's a very clear AI slop tell, anyone editing by hand would instinctively delete the extra spaces to align those.
Didn't realise this was posted here (again lol) but where I originally posted, on the R/Compilers subreddit I do mention I used chatgpt to generate some ascii art for me. I was tired and it was 12am and I then had to spend another few minutes deleting all the Emojis it threw in there.
I've also been open about how I use AI use to people who know me, and I work with in the OSS space. I have a lil Ollama model that helps me from time to time, especially with test result summaries (if you've ever seen what happens when a Mainframe emulator explodes on a NIST test you'd want AI too lol, 10k lines of individual errors aint fun to walk through) and you can even see some Chatgpt generated Cuda in notgpt.cu which I mixed and mashed a little bit. All in all, I'm of the opinion that this is perfectly acceptable use of AI.
Oh gosh, Emdashes are already ruined for me and now I can't use that to? I've already had to drop boldface in some of my writings because it's become prolific too.
This is also just what I intentionally avoided when making this by the way. I don't really know how else to phrase this because LLVM and HIP are quite prolific in the compiler/GPU world it seems.
For what it's worth - for people who're in this space - this project is awesome, and I hope you keep going with it! The compiler space for GPUs really needs truly open-source efforts like this.
Code/doc generators are just another tool. A carpenter uses power tools to cut or drill things quickly, instead of screwing everything manually. That doesn't mean they're doing a sloppy job, because they're still going to obsessively pore over every detail of the finished product. A sloppy carpenter will be sloppy even without power tools.
So yeah, I don't think it's worth spending extra effort to please random HN commenters, because the people who face the problem that you're trying to solve will find it valuable regardless. An errant bold or pipe symbol doesn't matter to people who actually need what you're building.
I keep hoping that low-effort comments like these will eventually get downvoted (because it's official HN policy). I get that it's fashionable to call things AI slop, but please put some effort into reading the code and making an informed judgment.
It's really demeaning to call someone's hard work "AI slop".
What you're implying is that the quality of the work is poor. Did you actually read the code? Do you think the author didn't obsessively spend time over the code? Do you have specific examples to justify calling this sloppy? Besides a misaligned "|" symbol?
And I doubt you even read anything because the author never talked about LLMs in the first place.
My beef isn't with you personally, it's with this almost auto-generated trend of comments on HN calling everyone's work "AI slop". One might say, low-effort comments like these are arguably "AI slop", because you could've generated them using GPT-2 (or even simple if-conditionals).
While I would not call this AI slop, the probability that LLMs were used is high.
> It's really demeaning to call someone's hard work "AI slop".
I agree. I browsed through some files and found AI-like comments in the code. The readme and several other places have AI-like writing. Regarding author not spending time on this project, this is presumably a 16k loc project that was commited in a single commit two days ago. So the author never commited any draft/dev version in the time. I find that quite hard to believe. Again my opinion is that LLMs were used, not that the code is slop. It may be. It may not be.
Yes this whole comment chain is the top comment misreading LLVM as LLMs which is hilarious.
> My beef isn't with you personally, it's with this almost auto-generated trend of comments on HN calling everyone's work "AI slop".
Now this doesn't necessarily is about this particular project but if you post something on a public forum for reactions then you are seeking the time of the people who will read and interact with it. So if they encounter something that the original author did not even bother to write, why should they read it? You're seeing many comments like that because there's just a lot of slop like that. And I think people should continue calling that out.
Again, this project specifically may or may not be slop. So here the reactions are a bit too strong.
Hello, I’m the project author. I don’t think In any of this and some of the criticisms I’ve received on this forum have people realised I’m not the original poster. I posted this on R/compilers and as of now that’s pretty much it. In terms of the comments. I use intellisense from time to time, I put my own humour into things and because that’s who I am. I’m allowed to do these things.
I’m self taught in this field. I was posting on R/compilers and shared this around with some friends who work within this space for genuine critique. I’ve been very upfront with people on where I use LLMs. It’s actually getting a bit “too much” with the overwhelming attention.
I understand your position. If I were in your place where someone else posted my personal (?)/hobby project on a public forum where it got discussed more on the point if it was LLM generated or not rather than the more interesting technical bits, I would also be frustrated.
Regarding the writing style, it's unfortunate that LLMs have claimed a lot of writing styles from us. My personal opinion is to avoid using these AI-isms but I completely get that for people who wrote like that from the start, it's quite annoying that their own writing is now just labelled as LLM generated content.
Most of the stuff I published anywhere was moved into a new git repo with a fresh commit history, often in one commit. Don't want to worry about some overeager hiring manager or client to assume my sloppy commits and git push -f use in personal projects is representative of my paid work. Seems to be quite the common practice, though I have no numbers to back this up of course.
I can empathise with the short fuse when it comes to suspecting AI slopped stuff though. Even some of the few people I still look up to push a lot of takes that reek of being slopped. Guess we'll have to wait this out.
> this is presumably a 16k loc project that was commited in a single commit two days ago. So the author never commited any draft/dev version in the time
It's quite common to work locally and publish a "finished" version (even if you use source control). The reasons can vary, but I highly doubt that Google wrote Tilt Brush in 3 commits - https://github.com/googlevr/tilt-brush
All I'm saying is assuming everyone one-shots code (and insulting them like people do on HN), is unnecessary. I'm not referring to you, but it's quite a common pattern now, counter to HN's commenting guidelines.
> found AI-like comments in the code
Sure, but respectfully, so what? Like I posted in a [separate comment](https://news.ycombinator.com/item?id=47057690), code generators are like power tools. You don't call a carpenter sloppy because they use power tools to drill or cut things. A sloppy carpenter will be sloppy regardless, and a good carpenter will obsess over every detail even if they use power tools. A good carpenter doesn't need to prove their worth by screwing in every screw by hand, even if they can. :)
In some cases, code generators are like sticks of dynamite - they help blow open large blocks of the mountain in one shot, which can then be worked on and refined over time.
The basic assumption that annoys me is to assume that anyone who uses AI to generate code is incompetent and that their work is of poor quality. Because that assumes that people just one-shot the entire codebase and release it. An experienced developer will mercilessly edit code (whether written by an AI or by a human intern), and edit it until it fits the overall quality and sensibility. And large projects have tones of modules in them, it's sub-optimal to one-shot them all at once.
For e.g. with tests, I've written enough tests in my life that I don't need to type every character from scratch each time. I list the test scenarios, hit generate, and then mercilessly edit the output. The final output is exactly what I would've written anyway, but I'm done with it faster. Power tool. The final output is still my responsibility, and I obsessively review every character that's shipped in the finished product - that is my responsibility.
Sure plenty of people one-shot stuff, just like plenty of Unity games are asset flips, and plenty of YouTube videos are just low-effort slop.
> something that the original author did not even bother to write
Again, this is an assumption. If I give someone bullet points (the actual meat of the content), and someone else puts them into sentences. Did the sentences not reflect my actual content? And is the assumption that the author didn't read what was finally written, and edit it until it reflected the exact intent?
In this case, the author says they used AI to generate the ASCII art in question. How does that automatically mean that the author AI-generated the entire readme, let alone the entire project? I agree, the knee-jerk reactions are way out of proportion.
Where do you draw the line? Will you not use grammar tools now? Will you not use translation tools (to translate to another language) in order to communicate with a foreign person? Will that person argue back that "you" didn't write the text, so they won't bother to read it?
Should we stop using Doxygen for generating documentation from code (because we didn't bother with building a nice website ourselves)?
Put simply, I don't understand the sudden obsession with hammering every nail and pressing every comma by hand, whereas we're clearly okay with other tools that do that.
Should we start writing assembly code by hand now? :)
I mostly I agree with what you said. Comparison with a google project is bad though. That's a corporate business with a lot of people that might touch that codebase. Why are you comparing that to someone's personal project?
Also I can see you and I both agree that it's disingenuous to call all LLM generated content slop. I think slop has just become a provocative buzzword at this point.
Regarding drawing the line, at the end, it comes down to the person using the tools. What others think as these tools become more and more pervasive will become irrelevant. If you as a person outsourced your thinking than it's you who will suffer.
In all my comments, I personally never used the word slop for this project but maintained that LLMs were used significantly. I still think that. Your other comparison of LLMs with things like doxygen or translation tools is puzzling to me. Also points about hammering every nail and every comma are just strawman. 5-6 years ago from today people used these things and nobody had any issues. There's a reason why people dislike LLM use though. If you cannot understand why it frustrates people, then I don't know what to say.
Also people do write assembly by hand when it is required.
> If you as a person outsourced your thinking than it's you who will suffer.
Using a code generator != outsourcing your thinking. I know that's the popular opinion, and yes, you can use it that way. But if you do that, I agree you'll suffer. It'll make sub-optimal design decisions, and produce bloated code.
But you can use code generators and still be the one doing the thinking and making the decisions in the end. And maintain dictatorial control over the final code. It just depends on how you use it.
In many ways, it's like being a tech lead. If you outsource your thinking, you won't last very long.
It's a tool, you're the one wielding it, and it takes time, skill and experience to use it effectively.
I don't really have much more to say. I just spoke up because someone who built something cool was getting beat up unnecessarily, and I've seen this happen on HN way too many times recently. I wasn't pointing fingers at you at any point, I'm glad to have had this discussion :)
I was responding to the person I was replying to, who confused LLVM with LLM, and who had brought up the slop term. I was surprised that they didn't think it was slop, because of the obvious tells (even with the fixed diagram formatting, there's a lot about that README and ASCII art that say that it was generated by or formatted by an LLM).
One of the reasons that slop gets such an immediate knee-jerk reaction, is that it has become so prolific online. It is really hard to read any programming message boards without someone posting something half baked, entirely generated by Claude, and asking you to spend more effort critiquing it than they ever did prompting for it.
I glanced through the code, but I will admit that the slop in the README put me off digging into it too deeply. It looked like even if it was human written, it's a very early days project.
Yeah, calling something slop is low effort. It's part of a defense mechanism against slop; it helps other folks evaluate if they want to spend the time to look at it. It's an imperfect metric, especially judging if it's slop based only on the README, but it's gotten really hard to participate in good faith in programming discussions when so many people just push stuff straight out of Claude without looking at it and then expect you to do so.
In general, quantizing down to 6 bits gives no measurable loss in performance. Down to 4 bits gives small measurable loss in performance. It starts dropping faster at 3 bits, and at 1 bit it can fall below the performance of the next smaller model in the family (where families tend to have model sizes at factors of 4 in number of parameters)
So in the same family, you can generally quantize all the way down to 2 bits before you want to drop down to the next smaller model size.
Between families, there will obviously be more variation. You really need to have evals specific to your use case if you want to compare them, as there can be quite different performance on different types of problems between model families, and because of optimizing for benchmakrs it's really helpful to have your own to really test it out.
NVIDIA is showing training at 4 bits (NVPF4), and 4 bit quants have been standard for running LLMs at home for quite a while because performance was good enough.
I mean, GPT-OSS is delivered as a 4 bit model; and apparently they even trained it at 4 bits. Many train at 16 bits because it provides improved stability for gradient descent, but there are methods that allow even training at smaller quantizations efficiently.
There was a paper that I had been looking at, that I can't find right now, that demonstrated what I mentioned, it showed only imperceptible changes down to 6 bit quants, then performance decreasing more and more rapidly until it crossed over the next smaller model at 1 bit. But unfortunately, I can't seem to find it again.
There's this article from Unsloth, where they show MMLU scores for quantized Llama 4 models. They are of an 8 bit base model, so not quite the same as comparing to 16 bit models, but you see no reduction in score at 6 bits, while it starts falling after that. https://unsloth.ai/docs/basics/unsloth-dynamic-2.0-ggufs/uns...
Anyhow, like anything in machine learning, if you want to be certain, you probably need to run your own evals. But when researching, I found enough evidence that down to 6 bit quants you really lose very little performance, and even at much smaller quants the number of parameters tends to be more important than the quantization, all the way down to 2 bits, that it acts as a good rule of thumb, and I'll generally grab a 6 to 8 bit quant to save on RAM without really thinking about, and I try out models down to 2 bits if I need to in order to fit them into my system.
This isn't the paper that I was thinking of, but it shows a similar trend to the one I was looking at. In this particular case, even down to 5 bits showed no measurable reduction in performance (actually a slight increase, but that probably just means that you're withing the noise of what this test can distinguish), then you see performance dropping off rapidly as it gets down to 3 various 3 bit quants: https://arxiv.org/pdf/2601.14277
There was another paper that did a similar test, but with several models in a family, and all the way down to 1 bit, and it was only at 1 bit that it crossed over to having worse performance than the next smaller model. But yeah, I'm having a hard time finding that paper again.
Why do you think ChatGPT doesn't use a quant? GPT-OSS, which OpenAI released as open weights, uses a 4 bit quant, which is in some ways a sweet spot, it loses a small amount of performance in exchange for a very large reduction in memory usage compared to something like fp16. I think it's perfectly reasonable to expect that ChatGPT also uses the same technique, but we don't know because their SOTA models aren't open.
OpenClaw sits on top of a physical machine/VM you control, you give it (hopefully) limited/sandbox access to that machine's resources to act like-you, and it does useful things. OpenClaw's user interface is just a gateway, and is only as useful as whatever the machine/VM has under the hood.
So the "setting it up yourself [on a VM/machine you control]" is kind of core to the whole idea being useful, you take that away, and it is just another Chat-Bot? Making it more of an ChatGPT/et al competitor rather than OpenClaw.
Installing it on your PC or laptop puts your personal data and ISP subscription at risk, while installing it in a hosted VM yourself requires a bunch of Linux security and networking knowledge or else you'll get pwned pretty much immediately (https://youtu.be/40SnEd1RWUU). So this service is giving you a VM already set up with a security baseline.
Is that what this does? All the link takes us to is an empty website about "Kimi Claw."
The entire crutch of the "Claw" concept is being able to directly reconfigure the VM/Machine to be "your" environment (to a point). A blank VM with nothing configured on it, is as useful as a cardboard bathtub.
Ultimately this link is a terrible intro to whatever this is.
Hm, that YouTube video made me think a bit, sure if you put it all like that, it does feel like a lot of stuff to get right, but whenever I do it, it takes about 30 minutes to lock down the firewall, do some port-scans to verify, punch a VPN through and hide SSH behind it. That way you're already protected from 99.9% of attacks, and then hope that that last tenth of a percent won't stumble upon you, and also that the VPN is secure enough, though I guess if that is breached it's not only you who's fucked. Also you need to look out that Docker doesn't destroy your firewall. I don't know, it doesn't feel like that much work, right? Maybe I'm just blind to it.
What you and I consider routine work, someone who works with mostly Webdev or might consider extremely difficult. There a lot of programmers who have never used a Linux shell, or know much about networking beyond TCP, or used Linux before outside of a uni class 20 years ago.
