> There are no memory improvements, people were not measuring correct.
Using filebacked pages instead of anonymous memory is a real improvement because it doesn't have to get swapped out if there's memory pressure. And this program probably isn't the only thing running on the machine.
I was referring that you would not gain any memory, there was no magic compression so you could use a bigger model on the same hardware. There were some wild claims made but it was some people meassuring memory usage wrong, but you are correct there might be some small memory improvements and soem speed improvements.
Well, you can use a bigger model now, it will "just" be really slow. This is different from GPUs, which would just fail to load larger models than VRAM because they don't support paging (unless you build that yourself.)
Using filebacked pages instead of anonymous memory is a real improvement because it doesn't have to get swapped out if there's memory pressure. And this program probably isn't the only thing running on the machine.