Depends on what you mean by self driving cars. To me it is not even clear that driving assist tech will ever be advanced enough for cars to drive themselves reliably.* But that aside. Human in control + driving assists is already safer than fully manual driving. The question is whether the opposite is even safer. It is obvious to me that self driving with the human as backup is a joke. Humans cannot react fast enough in real emergencies. So if you want the car to mostly drive itself, it has to actually always drive itself. And to me it is not clear whether this is really going to be safer than the human+driving_assist scenario.

*Of course I am talking about tech based on the current DL approaches. If we had AGI, then the question does not even need to be asked.