A completely unquantized fp16 model weight 7B LLM is about 15GB on a disk. You need closer to 24GB of memory for inference with a decently sized context.
Quantization is black magic of the software variety that seems to be able to significantly reduce that without a commensurate loss in quality, though the results are a little subjective. Some well reviewed quantizations of 7B models can get them below 9GB.
Most brains don't treat themselves to an abundance of training data, the ones who do seem pretty knowledgeable to me.
Witty comment aside, the human brain is pretty efficient in terms of energy use considering it's taking in a ton of data while it's conscious. Two each audio and video streams, olfactory, gustatory, touch, vestibular, and all the interoception. Inference and training in real time. All for the low price of 125 watts, a quarter that if you're just measuring the brain and not the whole body.
I have a suspicion that organoids will be to artificial intelligence what ornithopters are to flight. Technically possible, but reinventing it from scratch instead of following biology strictly will give us dramatic performance improvements.
All for the low price of 125 watts, but you need to sleep one fourth of the time, during which it doesn't do much, and on the other three thirds "Most brains don't treat themselves to an abundance of training data".
I'm not sure I'd call this machine efficient at using its computational power.
Also, that is the net energy expenditure of the body, but not the input required. We use a relatively antiquated form of the internal combustion engine to turn food into energy, which seems to be around 25% efficient, at best. It's also built on chains of other fairly inefficient biological entities, so it's hard to say what the overall efficiency is, but suffice to say that we're prevented from using the most efficient energy sources.
It's basically a cleaning cycle for the brain. If we don't sleep we die pretty soon. Meaning it's necessary daily scheduled maintenance downtime, which should count directly to efficiency calculations!
I think ROM sites were an artifact of the internet speeds we had access to early on. Where it wasn't practical to just download an entire library for an older console. Downloading an N64 ROM over dialup still took a little longer than an mp3, whole collections of them were out of the question. Even early broadband in many areas was limited to speeds where a single ROM was more practical to download.
Complete ROM collections were, and probably still are, available for most old systems as torrents and on usenet.
Downloading N64 roms as ZIP files was not a daunting task. PSX ISOs, OTOH... those were badly ripped.
But most of the time you would just get Tony Hawk 2/3, the Zelda's, Mario 64, Jet Force Gemini, FIFA's, Wave Race and a few more in less than a week and the fun lasted months if not years.
You don't need to do this. I bought a 1TB sd card because I heard that was enough space to store every retro game up to n64. I ended up copying over a small hand-picked selection of games rather than the full romset. We need some people to keep copies of the full romsets for the sake of preservation, but typical users only want certain, popular games.
Quantization is black magic of the software variety that seems to be able to significantly reduce that without a commensurate loss in quality, though the results are a little subjective. Some well reviewed quantizations of 7B models can get them below 9GB.