Good thoughts, and I don't particularly disagree. Regardless of any derivative you might obtain from the data, most folks want to retain high-fidelity historical data for two reasons:
(a) Future analysis techniques are unknown. Today you might be using method X, but tomorrow you might want to try method Y which calls for an entirely different massaging of the raw data.
(b) Backtesting and replay. Simulations and replays of previous trading events are valuable not only for testing a model but also testing the particulars of new "infrastructure" code.
Both require high-fidelity source data. This stuff is so hard to come by and costs so much money that most people want to be safe and save it forever.
To your point on analysis: it is rare that one would directly analyze the source data itself. You'll almost always want to transform the data into something more manageable for model development. The most crude form of this is idea of "bars". Instead of looking at tick data, traders tried to reduce the noise by looking at arbitrary aggregations of that data: 1 minute, 1 hour, 1 day, etc. Technical analysis using moving averages and other indicators are also examples.
Seems like hosting a large hifi dataset in the cloud (eg rolling last 5 years) and charging a small fee to crawl it might be a good opportunity. Or maybe there's no margin in it when people are prepared to pay $$$ as you describe even if they're reinventing the wheel in the process.
Not that I'm a mathematical or economic genius of any kind, but I continue to be surprised at how primitive financial analytics seem. When people do find something interesting (eg Li's Gaussian Copula) they almost invariably make a fetish out of it and hurl themselves off the nearest cliff shortly afterwards. Economics faculties are as much to blame as anyone, I feel.
(a) Future analysis techniques are unknown. Today you might be using method X, but tomorrow you might want to try method Y which calls for an entirely different massaging of the raw data.
(b) Backtesting and replay. Simulations and replays of previous trading events are valuable not only for testing a model but also testing the particulars of new "infrastructure" code.
Both require high-fidelity source data. This stuff is so hard to come by and costs so much money that most people want to be safe and save it forever.
To your point on analysis: it is rare that one would directly analyze the source data itself. You'll almost always want to transform the data into something more manageable for model development. The most crude form of this is idea of "bars". Instead of looking at tick data, traders tried to reduce the noise by looking at arbitrary aggregations of that data: 1 minute, 1 hour, 1 day, etc. Technical analysis using moving averages and other indicators are also examples.