If you overlap a hundred different FFTs, then the result is likely fine copyright-wise.
These networks are not [supposed to] contain much of the original data. Like the trivia point that Stable Diffusion has less than two bytes per source image, on average.
Stitch them side by side. Yes, this is not how those DNNs work, but the example was more about highlighting that "a giant table of token frequencies" by itself is probably reversible back to original data, or at least something resembling it.
> Stable Diffusion has less than two bytes per source image, on average.
I'm not convinced by this trivia point, though. Stable Diffusion is, effectively, a lossy compression of the training data. Nothing says lossy compression algorithms can't exploit some higher-level conceptual structures in the inputs[0], and applying lossy compression to some work doesn't automatically erase the copyrights of the original input's author.
--
[0] - SD isn't compressing arbitrary byte sequences, it's compressing images - which is a small subset of all possible byte sequences as large as the largest image used in training. "Less than two bytes per source image, on average" doesn't sound to me like something implausible for a lossy compressor that is focused on such small subset of possible inputs, and gets to exploit high-level patterns in such data.
> the example was more about highlighting that "a giant table of token frequencies" by itself is probably reversible back to original data, or at least something resembling it
That depends entirely on how many frequencies you're keeping.
> high-level patterns in such data
High level patterns across thousands of images are generally not copyrightable.
I might even describe the purpose of stable diffusion as extracting just the patterns and zero specifics.
>"Less than two bytes per source image, on average" doesn't sound to me like something implausible for a lossy compressor that is focused on such small subset of possible inputs, and gets to exploit high-level patterns in such data.
Two bytes would only let you uniquely identify ~65k images though, which to me doesn't sound plausible for a lossy compressor.
If you overlap a hundred different FFTs, then the result is likely fine copyright-wise.
These networks are not [supposed to] contain much of the original data. Like the trivia point that Stable Diffusion has less than two bytes per source image, on average.