I see a lot of talk in HN about how Apple running models on the device is such a great idea, for privacy reasons.
However I feel like outside of a few niches like HN, people don’t give a damn about privacy (for example, the company I work for has a website with millions of daily unique visitors; GDPR consent rates are usually around 98%).
So if other phones use models running on the cloud and they’re higher quality than the local running ones in iPhones (which will prob be smaller) that might be more important to the average user.
A big differentiator is that local models have access to more user data without sending anything over the wire (which ignoring privacy implications, isn’t free).
Local models can also more easily plug into existing macOS/iOS facilities like AppleScript, Automator, Services, Shortcuts, and NSUserActivity which means they’ll be a lot more functional, with the ability to wire together apps that haven’t explicitly added AI support.
In other words there’s a lot of potential here. If implemented well, LLM-powered Siri would have vastly more context and integration than anything else right out of the gate.
Apple is primarily a hardware company. Privacy claims aside, on-device gives Apple a reason to keep making, improving, and selling hardware.
I also don’t think the GDPR consent is a good measure. Those suck to opt out of manually and I doubt most people consider finding an extension. Apple sells its privacy as zero to low configuration. You “buy” privacy, you don’t configure it. Whether their claims are misleading or not. That’s extremely appealing.
A few other thoughts:
* Local models would be a requirement if Advanced Data Protection[0] were enabled
* Local models could potentially do some interesting work with hefty data (something media related maybe) that isn’t fit for cloud workflows
However I feel like outside of a few niches like HN, people don’t give a damn about privacy (for example, the company I work for has a website with millions of daily unique visitors; GDPR consent rates are usually around 98%).
So if other phones use models running on the cloud and they’re higher quality than the local running ones in iPhones (which will prob be smaller) that might be more important to the average user.