Cliff Kuang makes some interesting points in his essay The Secret UX Issues That Will Make (Or Break) Self-Driving Cars:

Recall that first principle that [Brian Lathrop at Volkswagen] laid out for designing autonomous cars—that the driver has to know whether the car is driving itself. That harks to probably the oldest dictate in interface design; mode confusion causes 90% of airplane crashes, and that insight helped invent the field of human-computer interaction. Think about all the times you’ve heard news reports about a pilot being confused about whether the flaps of the wings were down, or whether to auto-pilot was properly set. If you’ve ever failed to realize that your car was in park when you hit the accelerator, or you’ve ever tried typing into the wrong window on your computer screen, you’ve been a victim of mode confusion.

So, the scariest thing about self-driving cars is not whether or not the car can drive safely, but whether it can effectively communicate when it is driving and when it is not. It’s the age-old Visibility of system status UI heuristic in action.