Try 3.8 27B in MTPLX; I get about 30 tok/s with the same hardware as you. (Although it does use around 90-95W of power, compared to the ~60W that 3.6 35B-A3B uses to generate 55 tok/s. That’s about 3 J/tok instead of 1.)
I was interested in learning more about reverse image search, so I built audioguide.london. It’s a simple image search where you take a photo of an artwork in a gallery, tries to ID it and returns a pre-generated audio and a link to the museum website.
The audios were all generated locally, essentially looking at the contents of the website, running it through a LLM to generate a script and Kokoro for TTS.
I’ve built it as an app for myself almost a year ago, so I deployed it as a vibe coded website in here: https://audioguide.london/
URL Shortener is still one of the most popular System Design questions, building this project is a great way to have some experience / understanding of it, for example.
I agree. But a URL shortener with a twist isn’t just fun, it’s funny. The joke—as opposed to the usefulness—is what’s interesting about it. But when the same joke is overdone, it’s no longer funny.
> building this project is a great way to have some experience / understanding of it
Only actual update I do is to change my title / company when I change jobs, otherwise it's just a landing page to link to my GH, LinkedIn, etc. No blogs or anything exciting.
When AWS released the S3 Vectors last year, I wanted to try it since I had no experience with Vector DBs, so I built a simple audioguide app to play around with it.
Basically, just use it to identify artworks from a photo, then return a pre-generated AI-Audio for that artwork based on the data on their site. I put up a basic live version on my site for The National Gallery in London: https://victorsantiago.me/audioguide/national_gallery.html
Aside from having like 9 managers, 8 of whom are totally purposeless in your professional life, then yeah it’s not bad. The benefits are good.
I worked with some pretty talented and dedicated people at IBM. The “hop on a 2am call to put out a fire because they happened to check their email and they owed the person on pager duty a beer” kind of people.
That company was a red tape rats nest, but that’s management’s fault. And you get lazy people or shit departmental culture at various points in nearly every company, but painting a tens-of-thousands strong workforce with that brush is ridiculous.
It varies by model, but I’m getting 50-60 t/s with Qwen 3.6 35B and Qwen 3 coder 30B.
I’ve also used Qwen 3.8 27B but I get 10t/s on it.
It’s useable in some use cases, but I rely mostly on my $20 Claude subscription.
reply