Well, I don't have hardware to run a 141B parameters model, even if only 39B are... | Hacker News

Hacker Newsnew | past | comments | ask | show | jobs | submit

		tinyhouse on April 17, 2024 \| parent \| context \| favorite \| on: Mixtral 8x22B Well, I don't have hardware to run a 141B parameters model, even if only 39B are active during inference.

navbaker on April 17, 2024 [–]

It will be quantized in a matter of days and runnable on most laptops.

azinman2 on April 17, 2024 | [–]

8 bit is 149G. 4 bit is 80G.

I wouldn’t call this runnable on most laptops.

Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact