We've tried this for a while (companies using customers data) and I see a strong case for governments to start protecting users more strongly, because the current way this is being treated isn't working (medical data needs to be handled with greatest care everywhere else, why not here?).
The user already pays for the insurance (or hardware or service in other cases), so the data collected should only be used for those purposes, e.g., handling the claim and not building AI models without the necessary care. The customer should have the right to sell their own data to a company they see fit to take care of it appropriately and that compensates them for it.
The problem is with ownership I think. I'm not German so I'm really only speaking from what I've seen online (mostly here), but my understanding is that for Germans in particular (maybe EU writ large), you own data about you regardless of how it is collected or who has custody of it. In the US, it seems like the data about people is owned by whoever creates that data (in this case, an insurance company), so they are free to do with that data whatever they like provided it doesn't violate some other non-data-specific law.
The user already pays for the insurance (or hardware or service in other cases), so the data collected should only be used for those purposes, e.g., handling the claim and not building AI models without the necessary care. The customer should have the right to sell their own data to a company they see fit to take care of it appropriately and that compensates them for it.