I agree it is tautological in some sense. It is more precise to say compression requires prediction, and rather than sampling (as in a generative model), we use the probability distribution to efficiently encode observations. A better predictive model of the space you want to compress gives you a better compressor.
To your second point, it depends on what you mean by “futures you care for”. In my view, if you optimize your compressor for a set of particular inputs at the expense of all other inputs, you are explicitly making a judgement about what future data you expect to see (and in some sense what data you care about). If the space of data you want to compress well includes all truthful, valid English sentences, then in the limit you need a model of the world to make accurate predictions to compress well.
If you want further evidence of the predictive ability of compression, Marcus Hutter has a talk [0] where he presents a few papers that used a compressor (gzip) to approximate the Kolmogorov complexity of a string. Using this computable approximation you can construct the algorithmic information distance between two string by compressing the concatenation of two strings (pieces of text, DNA sequences, etc.) and measuring how well they compress together vs. their independent compressed sizes. The idea is that the Kolmogorov complexity of the concatenation of two similar strings is lower than the complexity of two dissimilar strings, since you can learn the structure of the data from a portion of the data, and use the learned predictive model to better compress other parts of the input data. Using this idea, the paper’s authors were able to perfectly reconstruct phylogenetic trees and linguistic family trees _simply by measuring the compressed size of concatenations_ of DNA sequences and a piece of text available translated to many languages.
To your second point, it depends on what you mean by “futures you care for”. In my view, if you optimize your compressor for a set of particular inputs at the expense of all other inputs, you are explicitly making a judgement about what future data you expect to see (and in some sense what data you care about). If the space of data you want to compress well includes all truthful, valid English sentences, then in the limit you need a model of the world to make accurate predictions to compress well.
If you want further evidence of the predictive ability of compression, Marcus Hutter has a talk [0] where he presents a few papers that used a compressor (gzip) to approximate the Kolmogorov complexity of a string. Using this computable approximation you can construct the algorithmic information distance between two string by compressing the concatenation of two strings (pieces of text, DNA sequences, etc.) and measuring how well they compress together vs. their independent compressed sizes. The idea is that the Kolmogorov complexity of the concatenation of two similar strings is lower than the complexity of two dissimilar strings, since you can learn the structure of the data from a portion of the data, and use the learned predictive model to better compress other parts of the input data. Using this idea, the paper’s authors were able to perfectly reconstruct phylogenetic trees and linguistic family trees _simply by measuring the compressed size of concatenations_ of DNA sequences and a piece of text available translated to many languages.
[0]: https://youtu.be/8Q2G2OGHm1c?t=2483