Look up what some Chinese LLMs are doing, e.g. mixing some attention mechanism with linear attention layers. Mamba2 layers, Gated Delta Net (2), Kimi Delta Attention and recent Raon Delta Attention (Gated Delta Net 2 + improvement from Kimi Delta Attention)
You are likely to find kernels for Gated Delta Net 2 as Nvidia had a paper on that.
Very little in terms of the layers they use. Calling it now, they are using global layers everywhere, making the model basically unusable due to high KV Cache use.
When it released, it cost $595 (back then, not in today's money), considering the price dropped to just $229 the following year, that was not a good buy when it came out.
Not unusual in that era though, home computers were evolving ridiculously fast.
There was only 5 years between the C64 and the Amiga 500 which launched for $699 (adjusted for inflation, using https://www.usinflationcalculator.com/ $595 comes out to $700), it was a wild time to be into computers, I didn't start til 1987 (I was 7) and the pace of progress was crazy to grow up through.
It's even a bit crazier than that: the Commodore Amiga 1000 came out in 1985, a bit less than three years after the C64, in july 1985.
A friend of mine, whose father was an art teacher, had an Amiga 1000 at home. Mind blown (I'm super lucky for later on another friend had an early SGI Indy at home for his stepfather was president of SGI Benelux). So I got to see and salivate on those mythic machines early on.
Of course I was stuck on the C64 until the Amiga 500, way more affordable, came out.
Still have my C128 and it was still working last time I cleaned it/tried it (when I was locked inside during Covid) and a few floppy disks were still reading fine (so I took a NOS --new old stock-- pack of 5"1/4 floppies and made a copy of Fast Hack'em and a copy of the Commando game): pure time capsule.
Fetched two C64s too: I know the PSU is tricky on the C64 and I don't know if those work or not. I've got floppy disk readers too.
But reading that headline is bittersweet: I was nine in 1982, I'm now 53. Where have all those years gone? Don't get me wrong, I had a great life (so far) and I'd replay it just the same but... It's still bittersweet to think about my mates and I learning to code on those 8-bit machines. And the games. So many games...
Feel that way every time I see a YT video on a "retro" computer I would have been able to play while having a beer legally (18 in the UK so 1998), it's then a shock to realise that was nearly 30 years ago.
> Don't get me wrong, I had a great life (so far) and I'd replay it just the same.
So would I and I didn't have a great life but if I alter the past I don't meet my missus/have my family so as much as it did suck, I'm happy now and don't want to change anything.
> And the games. So many games...
Still a gamer, there are three constants in my life, programming, reading and video games (roughly in that order).
> So would I and I didn't have a great life but if I alter the past I don't meet my missus/have my family so as much as it did suck, I'm happy now and don't want to change anything.
That's the thing right? I love the missus (only met her at 38 y/o but it's been 15 years of love) and our daughter she's now 11 y/o. And it's perfect.
I just wish life would play in slow-mo. The same life, but in slow-mo.
> But reading that headline is bittersweet: I was nine in 1982, I'm now 53. Where have all those years gone? Don't get me wrong, I had a great life (so far) and I'd replay it just the same but... It's still bittersweet to think about my mates and I learning to code on those 8-bit machines. And the games. So many games...
It was an incredible time to live through as a young kid. There was a Cambrian explosion of computer designs - new systems coming out every month, each outdoing the previous ones. I remember rushing to the store each month to buy and devour the latest computer magazines. Felt like a time of endless possibilities. Only downside was I couldn't afford any of it except for a ZX81 (which was great in its own way).
My grandfather worked for Sperry and we had circuit boards from Univacs in our basement when I was young. Then I bought an Atari 400 and B&W TV for the display when I was 11. Looking back it was pretty neat getting to see what came before, down in my basement. I don't think I ever had a tape drive for the Atari. I met some kids that had C64s (and modems) and that lured me over to the C64. The documentation for the C64 was so much better than anything I had for the Atari. I had BASIC for the Atari, but that was it. The C64 manual I had included assembly language programming, and the full schematics for the computer, so I was really able to learn everything about the system and explore all of it. It was so much fun.
And as a teen- going from making a MASH game using graphics characters on a trash-80 to color characters on a vic and getting my own PC for the first time, C64, and discovering sprites..
What's crazy is that the much more advanced Amiga custom chips were manufactured at the same 5 micron feature size as the C64 chips. That's why the Amiga needed two custom chips for video (Agnus and Denise) instead of the one in C64 (VIC-II).
It was a good buy for folks who got in early to build software/games for it. Friend of family was building games in 1982/1983 and had some early clones of games out (pacman, centipede, frogger clones, etc) and... while there were a lot of early publishers, it was fairly cheap to do in the early days, and they made more than enough to justify the $595 price (multiple units, IIRC).
It's one of the more popular retro-computers, but cheap to get into, there are a lot of them around: it's still always cheaper than its original sticker price! I think I paid more for the replacement power supply (the original ones haven't aged well, and have been known to fry them).
We ordered one of the first batch in Canada. $$$. They gave us a Vic20 loaner until it arrived. We didn't time it well but the family had a history of that. In fact, my accountant dad bought a Wang mainframe for a small fortune right before everyone was switching to ibm clone desktops.
> The Planner, Reviewer and Tester agents were assigned the same configuration in every run: claude-opus-4.8 via claude-code, only the Builder configuration was dynamic in the team setup. So, the results read as the difference between builders. That's a SWE in a team.
Yet they show the total cost. Making the comparison irrelevant. Most of the cost is going to be in there with model that expensive.
Also, they show open weight models in `opencode` (I have reservations about it, but fine) and `pi`, in it's most basic config (who actually uses it that way with models this capable?).
Show me actual comparison, from my experience, the planner, reviewer and tester should be different models from different families.
I understand where you're coming from. The research focuses on Builder configuration being variable and whether it affects metrics like cycle time and cost, which it shows the diff in detail.
I'm interested to see the total inference cost of a ticket type, that's why total cost is compared. Even though most of the cost is there in other stages, there's still huge difference even with one variable configuration in the loop.
The cost of each builder is also shown in mission breakdown, but good feedback for extending some of the tables, thanks!
For example x86_64 has v1, v2, v3 and v4 baselines, this tells you which instructions they support (e.g. v4 has AVX-512, v3 has AVX2, etc.).
RISC-V RVA22 and RVA23 aren't too different in this regard. Each one prescribes which extensions must be supported by the processor. I saw RV64GC mentioned, this is just a shortening of RV64IMAFDC, so I for baseline instructions, M for multiplication and division, A for atomic, F for floating point, D for double precision floating point and C for compressed instructions.
You can have a baseline E profile instead of I (less registers, some other features stripped), but I don't think we will ever see manufactured RV64E core, trough RV32EC cores exist.
Another aspect of the x86_64 architectural swamp is that Intel deliberately makes their optimized math libraries fail if run on a chip that is not 'INTEL INSIDE'. That block totally ignores cpu feature bits.
I've always wondered if virtualization software vendors made everything claim to be INTEL INSIDE, just to avoid this problem.
I know the RPCS3 (PS3 emulator) encountered some differences in a floating point instruction where Intel and AMD had different precision, causing de-synchronizations in LittleBigPlanet multiplayer.
You are likely to find kernels for Gated Delta Net 2 as Nvidia had a paper on that.
reply