Do you have any evidence that Nvidia or Apple are using non-standard memory chips? Link? From what I can tell they are both using standard chips, but very wide memory interfaces. Apple's lowest end is 128 bit, but offer 256, 512, and 1024 bit wide memory interfaces for more bandwidth, which is mostly a benefit for the iGPU in all of Apple's m series CPUs. This is part of why Apple are pretty good at LLMs, especially those needing more ram than is in even the most expensive GPUs.
Sad that the vast majority of x86-64 laptops and desktops have the same bus width of decades ago, while the core counts are ever increasing.
> Apple's lowest end is 128 bit, but offer 256, 512, and 1024 bit wide memory interfaces for more bandwidth
Apple doesn't have any 1024-bit offerings. The Ultras are 2x 512-bit which is different from a true 1024-bit since it's a NUMA configuration (two different memory controllers).
> Sad that the vast majority of x86-64 laptops and desktops have the same bus width of decades ago, while the core counts are ever increasing.
The consistent trend is CPU performance on consumer workloads is still much more sensitive to memory latency than it is bandwidth. This is probably also why those 512-bit M1/2 Max's don't even bother to let the CPU make use of it, they top out at "just" 200GB/s memory bandwidth even though there's 400GB/s available to the SoC as a whole.
Being able to pair a lot of DRAM with the GPU of an M* Max/Ultra is definitely a currently unique perk, but the bandwidth numbers available to those GPUs are not actually all that special. Desktop GPUs passed 400GB/s mark way back in 2016, and are currently pushing 1TB/s in modern flagship offerings.
Agreed. I did see a limit of 220GB or so for the M2 max, but haven't seen an updated number for the M3 max. Certainly GPUs have great bandwidth, but get quite expensive if you need more than 24GB ram. I've been impressed how well the M3 max runs 70B or larger models, plenty for single user use anyways.
Nvidia and Micron came up with GDDR6X, which isn't a JEDEC standard. JEDEC did standardize GDDR5X before that, but only Micron ever made it and only Nvidia ever used it AFAIK.
Sad that the vast majority of x86-64 laptops and desktops have the same bus width of decades ago, while the core counts are ever increasing.