Ah,you were the one that gave the "Refining ZFS compression" talk, isn't it? Thanks for all your work! Works great and it's pretty interesting.
I bought a super cheap QAT card with the idea of doing hardware accelerated gzip compression on ZFS (just for playing with it), but seems like the drivers are kind of a mess, and not being a developer who could debug it, maybe I should stay with the easy and fast zstd with early abort :P :) Less headaches .
I thought Intel stopped selling QAT cards with their newer generations of it?
Personally I find that gzip is more or less strictly worse than zstd for any use case I can imagine, though if you can hardware offload it, then...it's free but still worse at its job. :)
Though zstd can apparently be hardware offloaded if you have like, Sapphire Rapids-era QAT and the right wiring...sometimes. (I believe the demo code says it doesn't work on the streaming mode and only works on levels 1-12.) [1]
Seems like they stop selling dedicated QAT cards, but their new E2000 DPUs (kind of a smart NIC I think your employer uses for accelerating cloud loads) have QAT instructions. But I'd say that's because those DPUs have a couple Xeons inside :)
I think QAT API used to be a mess in past generations, and IIRC it's one of the factors drivers for older hardware weren't included in TrueNAS Scale. I think it was this ticket [1] where they said that it was complicated to add support for previous hardware, but can't confirm because JIRA's been acting funny since yesterday.
Feels like QAT has stabilized since this Gen4 hardware and interesting things are appearing again. Not only the ZSTD plugin you linked, but I just noticed QATlib added support for lz4/lz4s past July [2]! That's interesting.
They are also publishing guides for accelerating SSL operations in SSL terminators / load balancers like HAProxy [3], so between this and cloud providers using DPUs to accelerate their load, doesn't feel like they're shutting this down soon.
I'd love if they make this a bit more accessible to non-developers.
I bought a super cheap QAT card with the idea of doing hardware accelerated gzip compression on ZFS (just for playing with it), but seems like the drivers are kind of a mess, and not being a developer who could debug it, maybe I should stay with the easy and fast zstd with early abort :P :) Less headaches .