You could keep multimodal projector (understanding of audio, images & PDFs) in s...

		srigi 17 days ago \| parent \| context \| favorite \| on: Accelerating Gemma 4: faster inference with multi-... You could keep multimodal projector (understanding of audio, images & PDFs) in system RAM with `--no-mmproj-offload` in llama.cpp. Of course, then it is not accelerated with GPU, but you save its VRAM.

Interesting, I might try that, thanks!