You also send your number as part of the invite. Responsible VoIP providers limit this to numbers you own, but it's totally possible to send calls using a number you don't own. There are CNAM databases but not all carriers make use of them and they aren't a source of truth. How would your solution deal with numbers which go back into a carriers pool? How would your solution deal with newly claimed numbers? How do you verify the numbers on file for an organization?
I was thinking along these lines: It may be possible to send calls using a number I don't own. It should not be possible to send calls using a number that the VoIP provider doesn't own. If I'm sending a VoIP call to someone's cell phone, and I'm saying that I'm a cell phone in the same block of numbers as the destination phone, a responsible VoIP provider should block that, even if they don't restrict me only to my number.
But we both used the word "responsible". I think that's the problem - there are some irresponsible VoIP providers. Maybe even deliberately irresponsible. For obvious reasons, they attract the spammers and scammers. And that brings us back to this lawsuit as a reasonable response to that kind of irresponsible behavior.