How much of this delay is just perfectionism and excess risk aversion? In pretty much any pursuit, demanding perfection ensures that you'll never be finished. Based on what I'm seen, autonomous vehicles already work. Waymo's cars have driven millions of miles. Other companies have working systems too. The technology is already at the point where it's useful.
I understand wanting to keep people safe, but the benefits of this technology so are enormous that I'd rather companies launch now and iterate than spend decades wringing every last disengagement out of the system while the carnage on our highways continues.
I feel like we're in one of those situations where everyone is just afraid to be first. Meanwhile, people die every day.
You'd rather have car companies release beta software to millions of cars than stick with the status quo? This isn't the Windows 10 Insider Preview. Cars can kill people, and production vehicles should never contain beta software, which is beta because it's buggy/untested/unreliable.
No software is perfect --- and that goes for the wetware in our heads too. Why would you want people to keep dying on the roads just so we can avoid shipping imperfect-but-still-superhuman autonomous driving vehicles? We need a more nuanced outlook than "Disengagement! NO RELEASE!".
In any case, that degree of risk aversion is unachievable, unaffordable, and unproductive when discussing personal automobiles. Cars are killing people now. It is not reasonable to wait for some Utopian future where every car has the ability to pilot itself with NASA-grade software.
Human drivers are already "moving fast and breaking things," and they're not getting any better at it.
But we're not waiting. Cars today are absolutely crammed with automated safety features. Far more than the airbags and ABS on my 20 year old VW. Think automatic collision avoidance, blind zone detectors, traction control, lane departure warnings, back-up cameras, adaptive cruise control, and more that I've probably never heard of yet.[0]
To think that we're waiting until perfect level 5 automated driving before we release it to the public is ridiculous. As the article clearly states,
> Most OEMs are now more forthright about the fact that autonomy will be a succession of small, graduated steps.
We didn't go straight to landing on the moon. Neither will we go straight to autonomous driving, or flying cars. Flying cars will never happen, because there are no gradual steps between flying and not flying, but software can be gradually improved. That doesn't mean running untested beta software in cars though.
[0] Also, advanced crumple zones, numerous airbags, proper safety restraints, crash guards on transport trailers, high-tech road makings, restrictive driving tests, and strict laws on driving while intoxicated or distracted.
> I feel like we're in one of those situations where everyone is just afraid to be first.
Tesla is not afraid. They killed few people already with autopilot. Best part about it where they fixed bug that lead car into concrete barriers and then regressed few months later...
Waymo has already come out and stated that realizing AVs as they've been pitching them are much further away than they've previously claimed, and now they've moved to try to become a Tier 1 component supplier to try to monetize at least _something_ for all the money they've dumped into this endeavor.
Additionally, despite driving the exact same corridor for several years, there are still trivial and commonplace scenarios that their system can't handle reliably due to environmental ambiguity (i.e. multimodal human intent).
Though, not to gang up on Waymo here. Probably most programs are stuck/grappling with these problems. You just mentioned them directly, so I picked on them.
That you are willing to let other people die for the sake of increasing your convenience isn't very compelling.
And I think that's also unnecessary. From what I can tell about Waymo's approach, they are launching with something well below level 5, but where they cut over to a remote driver whenever the car is unsure. It's a smart approach, in that they can offer a level-5 service with level-3 technology. They've turned the technology cliff into a cost-optimization problem, which they can chip away at over time.
And they can't as they current are, and won't be able to for quite some time.
Also, what they need to beat is not the aggregate safety level, but the average reliability of human drivers. Not all deaths are due to driver error. And they need to beat it by a large enough margin that people will accept that they are more reliable.
Finally, as another poster pointed out, there is the issue of liability. We know how to assign liability if a human driver makes a mistake. How do we assign liability if an autonomous self-driving system makes a mistake? I can tell you that, if I'm the human who owns the car and I'm still going to be held liable if the car's autonomous self-driving system makes a mistake, I'm going to be very, very hesitant about letting that system control the car, even if the statistics show it's more reliable than I am. At the very least I'm going to want to closely monitor what the system is doing--which of course defeats the whole purpose of having it. And if I'm an automaker and am going to be held liable if my self-driving autonomous system makes a mistake, even if a human driver is in the car and I can't control how they operate the system, I'm going to very, very hesitant about selling cars with that system, even if the statistics show it's more reliable than human drivers.
> it has to be the automaker that stands behind its product with a financial safety guarantee.
And, as I said, if I were the automaker, I would be extremely hesitant to give such a guarantee with human drivers in the car whose operation of the systems I am unable to control. So cars with such guarantees would basically have to be entirely self-driving, i.e., human intervention not even possible by design. That is a huge increase in required reliability, not to mention a huge change in how cars are currently used. Which is not to say it won't eventually happen, just that such a liability regime would, I think, significantly increase the time it will take to get to ubiquitous use of self-driving cars.
More than 10k of those U.S. people die from alcohol related crashes. Would you be happy with your car being just marginally better than a drunk driver? It could save so many lives!
Okay maybe just drunk drivers are a bad comparison, let's look at all of those accidents. They aren't all drunk people. Turns out like 95% of them are due to similar human stupidity though. So unless you want your car to be better than a terrible driver, it doesn't have to just reduce accidents by a little, it has to reduce accidents by 95%. Otherwise you're advocating for the analogue of a lyft driver who is looking at their phone and only a little tipsy to replace you behind the wheel. Yeah maybe it'll decrease deaths, but I'm not buying one, nor am I advocating for legalizing the equivalent of a driver on their phone and kinda tipsy.
I understand the theory, but good luck selling a car on the basis of "probably as safe as the average driver!" And good luck convincing the parents of some kid run over by Google that it's just one of those things.
Note also that for many of those 37,000 deaths, a human is found culpable, often doing time. Who do you propose should do the time for autonomous cars? An engineer? Their manager? The CEO?
In practice, our corporate accountability is such that nobody will pay any penalty. Maybe that's fine when, as with the 2008 financial crisis, the losses are abstract and distributed. But when it's the life of somebody's kid, somebody's mom, somebody's sister? That at best will be an extremely volatile situation. Especially if Google or Uber captures 20% of the self-driving car market and is now responsible for 37,000 * 0.20 = 7400 deaths per year, or about 20 per day.
I wonder how many of those deaths anyone is punished for? My guess is under 10%. Mostly it's just considered an "accidental death" and life goes on, as I understand it.
But I (fortunately) don't have much experience with this.
The only possible solution is that the SDC and/or software manufacturer is legally responsible for any accidents the SDC causes. I think that coverage has to be part of the legal package for these things to ever be sold in the US. But that would be money, not jail time, of course.
Fortunately the car(s) should have video and data logs from the event, so the evidence situation should typically be very clear.
> The only possible solution is that the SDC and/or software manufacturer is legally responsible for any accidents the SDC causes. [...] But that would be money, not jail time, of course.
I don't think there's any "of course" there. As we see with the 2008 crash and the financial industry generally, corporate financial liability for the high-risk decisions of individual managers and execs doesn't do much to reduce systemic risk. For the managers, it's a "heads I win, tails you lose" play.
The difference between banks and robot cars is that a banking failure just means slightly higher prices for the customers and/or slightly lower dividends for investors. Whereas with robot cars, in comes in funerals. Especially given the endemic low quality in the software industry, there's little doubt in my mind that unless "disruptors" face actual jail time, we'll see what is in effect scaled-up negligent homicide.
> I mean, we have a century of settled tradition/law for how to handle deaths and injuries from badly designed cars.
It's settled for cars with human drivers, because the drivers bear most of the responsibility. For cars without drivers, who's to blame for bad driving? "Somebody's bank account" is not a compelling answer.
> I don't really see the analog with the banks.
What part is unclear for you? I'm saying that with banks for the last 10+ years, we see that financial penalties don't solve problems of risk or accountability.
> Software has endemic low quality where it doesn't much matter. In mission critical systems, it is usually reliably as solid as it needs to be.
I don't think either of those statements is correct, unless you mean the first one tautologically. But the latter is definitely untrue. Look at the way Toyota was building their software. Or the recent Boeing fiasco. Or Volkswagen's emissions scandal, which is responsible for circa dozens of deaths.
I would buy a car guaranteed to drive as safely as I do, except autonomously. I'd do it in a heartbeat, and millions of people would line up behind me. It's not lack of customer demand that's keeping autonomous vehicles out of the hands of the public. It's risk-averse bureaucracy.
What's being discussed is not driving as safely as you do. It's as safe as the average driver. Or, more accurately, a company claiming it's as safe as the average driver, which is why I said "probably".
So it seems pretty reasonable that any car that just kills the same number of people as the population average (which includes drunk drivers, tired drivers, distracted drivers, etc) will be seen as worse as worse driver than the person purchasing the car.
And I think it's a given that regulators, and the general public, who are the ones who will be hit by the self-driving cars, will see an automated car that only kills 37k people/year as a non-starter.
> That you are willing to let other people die for the sake of increasing your convenience isn't very compelling.
That's how we got the original automobile, and that's been a great benefit to the human experience.
Everything is costs and benefits. In exchange for reducing the amount of very dangerous manual driving that people do, we plug in autonomous technology that's very good, but not perfect. It still saves lives overall.
This idea that autonomous must be perfect or
you're a murderer --- it's simplistic and counterproductive, and following this kind of absolutist thinking guarantees that we'll never get autonomous technology and that people will continue to slaughter each other with manually-driven vehicles.
Taking an absolute stance against dangerous technology might seem good at first, but in the long run, it increases the sum of human misery.
It's perfectionism, not autonomous, that's killing people every day.
> The technology is already at the point where it's useful.
Useful for certain limited purposes, yes. But that's not where it needs to be to start taking the place of human drivers in everyday usage, in order to reduce harm in everyday usage (which I agree is a good thing to do). For that purpose it needs to be more reliable than the human drivers are, and it's not even close to that and won't be for quite some time.
Well I think they have in a way. The Tesla Autopilot is an example, and probably not a bad example of where the level of self driving cars is really at. Given Tesla's demonstrated in service capabilities, it seems that we can do a few things, but remove the steering wheel self driving is many many moons away.
I understand wanting to keep people safe, but the benefits of this technology so are enormous that I'd rather companies launch now and iterate than spend decades wringing every last disengagement out of the system while the carnage on our highways continues.
I feel like we're in one of those situations where everyone is just afraid to be first. Meanwhile, people die every day.