I really like the approach with WASM and the browser sandbox. That alone makes this very usable. iam also working on an agent harnes in the browser sandbox called "hermit" using a local inference endpoint. It runs Python in WASM with a virtual file system.
I did a WebGPU-based HTML Single-File Chat Application even for larger models. It just depends on your machine. It is cool to show people what can be done on their machines without installing anything. It also works in air-gapped environments.
Thanks for sharing. I tried the online demo, which downloaded the model but, unfortunately, couldn't start it. (It says " Invalid magic number
Make sure your local server is running and CORS is enabled.")
I think you've fixed it! I just tested it, now it works. (In both Chrome on Windows with NVidia 1060 gpu w/ 6 GB vram, which is what it wasn't working on yesterday, and on Safari on 2026 Mac Mini M4 with 24 GB of RAM. The former is pretty slow, around 1 tok/sec, the latter is fast at 18 tok/sec.)
Yeah, I found a bug during model load occurring with Firefox and low VRAM machines. It is fixed now, thanks!
The 1060 is unusually slow. The Mac mini sounds about right. Have fun playing around and sharing. With the more VRAM of the mini you can also run Qwen 3.8 27b :)
I really like to showcase this to non-tech people to show what their home machines already can do WITHOUT INSTALLATION in an air-gapped scenario! And given all the recent fuss, we should not trust the big providers at all with important data.
This is very hard in Germany or not possible at all (beside maybe FAANG).
I have a PhD in Machine Learning (including Papers at SIGKDD / ECML / ACL etc.) as well as relevant working experiences and it is just not possible to get this numbers. I tried really hard to get something that pays this well (beside FAANG) that hire here because i can not move (family stuff).
I once talked with Google and Amazon Research (before Covid) and they told me no remote only (thats why i skipped them for now). Did this policy changed?
It would have been easier for you if you had papers in NeuroIPS, ICML, ICLR, or CVPR. The ones you mentioned don’t impress anyone.
A few months ago I made a 180k offer to a guy in England who wanted to be fully remote. He rejected it because he got a better offer. He had two papers (ICML and ICLR).
afaik yes the policy changed, there now is Germany-remote as a „location“ but this doesn’t come by default you put in a request and it needs Director or someone equally as high to approve it. Easier to do internally rather than as a new hire (when you’re not applying for a very experienced role). Sometimes teams have a „remote ok“ policy when specialists cannot be found locally but this is rare.
I am in similar position. I recently turned 40 and I’m a Development Manager but the role is moving me away from all the technical stuff I really like. I work in a classical industry position so no interesting FAANG Stuff only boring (but important) business software to keep up and running with my team.
The problem is that i originally did a PhD in Computer Science (Machine Learning / NLP) in 2010 and had papers at SIGKDD, ECML etc. So this business software thing is terrible boring for me but in Germany it was reallly hard to find interesting - well paying positions - at that time in this field.
And now i have no idea what to do next, because i fear that all my experience as a “Development Manager“ will not be accounted for if I apply somewhere in a more deep technical role like FAANG.
https://moooff.github.io/HermitUI/hermit-agent/
So this Tool comes in very handy and will try to integrate it.