Edit up top: The identification is done on-device. The Verge article didn't mention this.
> And this year’s Pixel will take advantage of the phone’s always-on microphones to listen for music (not just the phrase “OK Google”) and display what you’re listening to on the screen, even if it’s something on the radio.
This sounds creepy. So now when excessive microphone data is seen to be going out to the cloud, they can just say "Oh, the phone thought there was music playing and was trying to identify it. Simple misunderstanding, nothing nefarious!".
Other articles did mention that this will be local.
I have a feeling they are using federated machine learning for this to have a lot of the processing done locally and not need to activate the radios for as much of the processing as possible. They have been making big strides in that area lately and this might be the start of some of the major applications of it (I think they are using it in their keyboard prediction as well from a bit ago)
so there's going to be a huge database of songs on my mobile!!! i don't think anybody asked for it... sounds like trickery... this is going to take up some storage space now...
anyone know how many gb or mb this occupies in my phone... can v jus clear this data...
The phone is just storing song fingerprints, probably no more than a few megs for every song ever recorded. It would be great to extract the data and release it for public use.
I doubt it, that would be a good chunk of data for what I see as a fairly small feature.
More likely (this is a guess, nobody outside google really knows at this point), they will use federated machine learning to figure out that something is "a song", then perhaps clean up and isolate the actual "song" part of it and send that over to a google server for processing.
But again it was just announced, so nobody really knows how this works, where the data is or goes, and what tradeoffs were made.
I don't do neural nets, but if I had to crudely estimate...
10,000 songs (the outputs) * 16 layers * 16 parameters per node * 4 for the bytes per float = 10MB + a DB of song/artist names.
I'm probably underestimating the parameters per node, and overestimating the size of the layers closer to the input. Further, it's more likely structured as an LSTM than a convolutional network, since sound is a streaming source.
Lots of devices already do this. My Echo occasionally misinterprets its wakeword and broadcasts up little 4-second clips of whatever is going on at the time it decides to do that. If you're worried about so-called "accidental" identification that allows them to activate listeners and receive the sound data from the room, that's already a pervasive threat.
Reminder that US intel exploited a bug (or a "bug") in Samsung Smart TVs that allowed them to surreptitiously activate the built-in microphones and stream the room's sound on-demand, obviously with no notification to the user. [0]
That gets me curious, did anyone try running that malware and see which servers it transmitted up to? Would be interesting to go through logs and see, retroactively, who this was used on in the wild. Would be even _more_ interesting if it proxied through a tunnel at a cooperative BigCo...
> And this year’s Pixel will take advantage of the phone’s always-on microphones to listen for music (not just the phrase “OK Google”) and display what you’re listening to on the screen, even if it’s something on the radio.
This sounds creepy. So now when excessive microphone data is seen to be going out to the cloud, they can just say "Oh, the phone thought there was music playing and was trying to identify it. Simple misunderstanding, nothing nefarious!".