AI-based profiling of anything is bound to filter off outliers. Censuses are made in the field for a reason. The goal is to gather data so that you can make statistical reasoning on it. Not the opposite!!!!
Here, they just gather some car data, and infer demographic data from it. And then what? We've just created a population which matches the car sample. We can't draw ANY conclusion from that data, beyond the type of cars that are found in such and such neighborhoods.
In my parts of the world (Eastern Europe) you can infer a lot more about the people inhabiting a certain area by looking at the cars they drive, I mean, compared to census and income/tax data. That's because there are lots of way of not telling the authorities how much you earn, but once it comes to purchasing a car you leave aside such niceties and you get the car that best fits your real economic status (with few exceptions).
Yes, of course there is obviously some correlation between demographics and car ownership. But this paper makes it seem that their method is good to replace a in-the-field census.
Basically, replacing all demographic data that the census brings with the single factor of car ownership. In the region where the actual correlation is less than 90%, the data generated is completely useless.
> But this paper makes it seem that their method is good to replace a in-the-field census.
Ah, I missed that, sorry! To be honest I only skimped through the description, for other lazy people like me here's the relevant line:
> Our results suggest that automated systems for monitoring demographic trends may effectively complement labor-intensive approaches, with the potential to detect trends with fine spatial resolution, in close to real time.
which I also find as a little bit crazy.
Otherwise I find the project quite interesting. I quite like to lose (too much time) on GStreetView, and at the same time I developed an interest in 20-30-40-year old cars, so I wrote a small Chrome extension that allows me to locally save images of old cars that I find on my country's roads while "taking a walk" on GStreetView, along with the relevant info (lat-lng, address, and the make and model of the cars which I input by hand using said extension). I had also noticed that there's a distinct correlation between a city's economic status and the cars you can find on its streets, and I was wondering how hard would it be to do the car "recognition" using some AI-thing and try to draw some conclusions from that. Glad that someone actually did it.
Must be why I've often met with astonishment and incredulous remarks when I meet new colleagues. "He's got to have a lot of money; why does he drive that crap car?"
I always answer: that's why I have a lot of money.
That's weird. Because I've met plenty of people who are rich because of entrepreneurship and drive a luxury vehicle. Why are you rich because of the car you drive?
Is this just trolling now? Because saving money by not spending on luxuries is generally understood to conserve money, making one wealthier. Saving on cars can obviously save 10's to 100's of thousands of dollars.
So, you don't trust surveys? You don't A/B test your marketing?
Estimates are useful. Hell, your brain is just estimating what's in front of you based on a sample of photons coming in through your eyes. If you trust your vision, you're trusting statistical sampling. I suppose it's possible the USS Enterprise is cloaked and somehow your eyes are deceiving you, but the photons are certainly correlated with what's in front of you.
Some of the best statistical innovations are simply discoveries of good proxies for something that's difficult to measure. Google found that what you search for is a good proxy for what you want to buy. Facebook found that what your friends click on is a good proxy for what you'll click on. Of course you won't like everything Netflix suggests, but its suggestions will (hopefully) be better than just picking movies randomly.
Of course that estimates are useful. But when you use proxy data such as cars to infer demographic data and present these as the result of a census, you can see why it's problematic. You're basically using the number of pick up trucks to estimate the degree of education in the neighborhood, then you use that same number of pick up trucks to evaluate the wealthiness of that neighborhood, and then you draw a conclusion on the correlation between wealth and education, which is the very input that's hidden in your AI model.
You may use census data to train a neural net to draw some conclusions, not to produce brand new census data.
It doesn't matter if it's only for complementing. Census data is supposed to be the source data, if you start to mingle and add some model in order to "enhance" your source data, you're tainting it with wrong samples. Then from there everything is biased.
My point is that there is a hype around AI that makes it look that it can do things that it can't. What they do in this paper is equivalent to using a correlation matrix, and a bunch of observations to generate a completely random population that matches what we input in there (ok, the neural net may find some non-linear relationships, but you get the idea). Yes, there is a significant correlation between car ownership and demographics, so the results do look good from far away, but the reality is that they will only map the car ownership factors on to other traits, and in terms of information, it is way poorer than actually doing a census.
If you had multiple points of data, say street view from around the time of the last census, and an updated street view within the last few months. Could you not infer changes in the demographics? With some level of significance?
I am curious, because at first glance this seems really useful. But trying to see where the limits actually are.
Well, for one thing, if you're worried that your census is misreporting, and this method gives you clues about areas to double-check, and they turn out to have been good suggestions, then it's useful.
AI-based profiling of anything is bound to filter off outliers. Censuses are made in the field for a reason. The goal is to gather data so that you can make statistical reasoning on it. Not the opposite!!!!
Here, they just gather some car data, and infer demographic data from it. And then what? We've just created a population which matches the car sample. We can't draw ANY conclusion from that data, beyond the type of cars that are found in such and such neighborhoods.