Hacker Newsnew | past | comments | ask | show | jobs | submit | Feorn's commentslogin

A completely unquantized fp16 model weight 7B LLM is about 15GB on a disk. You need closer to 24GB of memory for inference with a decently sized context.

Quantization is black magic of the software variety that seems to be able to significantly reduce that without a commensurate loss in quality, though the results are a little subjective. Some well reviewed quantizations of 7B models can get them below 9GB.


Most brains don't treat themselves to an abundance of training data, the ones who do seem pretty knowledgeable to me.

Witty comment aside, the human brain is pretty efficient in terms of energy use considering it's taking in a ton of data while it's conscious. Two each audio and video streams, olfactory, gustatory, touch, vestibular, and all the interoception. Inference and training in real time. All for the low price of 125 watts, a quarter that if you're just measuring the brain and not the whole body.

The paper was published last year. https://www.frontiersin.org/journals/science/articles/10.338...

I'm not convinced this field will outpace silicon or whatever succeeds it, considering how big the semiconductor industry is.


I have a suspicion that organoids will be to artificial intelligence what ornithopters are to flight. Technically possible, but reinventing it from scratch instead of following biology strictly will give us dramatic performance improvements.


All for the low price of 125 watts, but you need to sleep one fourth of the time, during which it doesn't do much, and on the other three thirds "Most brains don't treat themselves to an abundance of training data".

I'm not sure I'd call this machine efficient at using its computational power.


Also, that is the net energy expenditure of the body, but not the input required. We use a relatively antiquated form of the internal combustion engine to turn food into energy, which seems to be around 25% efficient, at best. It's also built on chains of other fairly inefficient biological entities, so it's hard to say what the overall efficiency is, but suffice to say that we're prevented from using the most efficient energy sources.


> during which it doesn't do much

I know this was in jest, but if sleep was not a critical biological process we would have evolved to not do it already


It's basically a cleaning cycle for the brain. If we don't sleep we die pretty soon. Meaning it's necessary daily scheduled maintenance downtime, which should count directly to efficiency calculations!


I think ROM sites were an artifact of the internet speeds we had access to early on. Where it wasn't practical to just download an entire library for an older console. Downloading an N64 ROM over dialup still took a little longer than an mp3, whole collections of them were out of the question. Even early broadband in many areas was limited to speeds where a single ROM was more practical to download.

Complete ROM collections were, and probably still are, available for most old systems as torrents and on usenet.


Downloading N64 roms as ZIP files was not a daunting task. PSX ISOs, OTOH... those were badly ripped.

But most of the time you would just get Tony Hawk 2/3, the Zelda's, Mario 64, Jet Force Gemini, FIFA's, Wave Race and a few more in less than a week and the fun lasted months if not years.


Downloading entire ROM collections quickly becomes too much data for local storage, starting with Saturn/PS1 and escalating from there.


You don't need to do this. I bought a 1TB sd card because I heard that was enough space to store every retro game up to n64. I ended up copying over a small hand-picked selection of games rather than the full romset. We need some people to keep copies of the full romsets for the sake of preservation, but typical users only want certain, popular games.


Nah, you just need to scale up your storage expectation; I think 8TB will cover most of a PS1 localization.


Sony Playstation NTSC-U dump is only 472GB.

Oh, and they're available on Archive.org :)

https://archive.org/search?query=Sony+PlayStation+-+%28NTSC-...


Maybe it was PS2 localization, but those quantities are well within the consumer reach to maintain archives.

It's just work, like anything else.


Plenty of large rom sites are still very actively used and trafficked today.


A reference 3090 is longer by 69mm, wider by 29mm, and thicker by a slot than a PCIe A100.

Though I think the comment you're replying to was talking about them both using the same Nvidia GPU architecture, Ampere.


Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: