Could Augmented Reality Windshields End Distracted Driving?

Countless drivers have gotten away with murder by telling a cop the magic words, "I just didn't see [that pedestrian/cyclist/wheelchair user] coming." But if a new augmented reality windshield technology works as designed, they might not have that excuse for much longer.

At last week's 2021 Consumer Electronics Show, technology giant Panasonic unveiled its new and improved "artificial intelligence-enhanced head-up display" that senses whether the driver is speeding, swerving, or approaching a "roadway hazard" like a human being on a bike. But perhaps more importantly, the sensors that power the display also sense whether the driver is aware that he or she is endangering others — and if not, they project an unmissable visual warning directly onto the windshield where the driver should be looking, accompanied by a vibration or auditory alert.

Panasonic says the system's ultra-advanced 3-D radar cameras take in more of the roadway than its competitors, and can even sense things that human drivers either can't see or frequently miss, such as a person walking across a road without streetlights, or a kid darting into one of an SUV's massive blind spots. And again,  those same cameras are also watching the driver herself, and dynamically adjusting the car's alerts based on her eye position.

Put another way: if a driver is checking himself out in the rear view mirror and a kid chases a soccer ball into the street in front his car, the alert system will bleat extra loud and flash a warning directly into the motorist's field of vision to give him an alert to stop. But if a driver is doing the right thing and giving the roadway his full and undivided attention, the system will give him a gentler warning — a shift that the designers of the system say could convince drivers to actually use such advanced features, as well as keeping them from getting quickly desensitized to life-saving safety alerts.

"My wife and I have a back-up camera on our car, and it screams at us pretty much every time we try to get out of the garage," said Andrew Poliak, chief technology officer for Panasonic Automotive. "A lot of drivers are so inundated with all these non-contextual warnings from their 'smart' vehicles — all these bells and chimes and beeps — that they don't always really hear them. ...We need to make sure we build systems that people do not turn off. "

The idea of overlaying a windshield with detailed safety information and noisy alerts isn't new: auto manufacturers have been experimenting with the idea since 1988, and superheroes like Tony Stark have been doing it far longer. But U.S. automakers have struggled with how to market technologies like head-up displays, which are usually sold as expensive, add-on safety features available only on the most expensive cars on the road. Poliak says costs of the systems are quickly coming down, but that doesn't address the tricky psychological question of how to compel drivers to keep their safety features activated, never mind staying alert to their warnings once all those beeps and bops starts to become familiar background noise.

Europe, of course, has taken the most simple route: automakers there will be required to include camera monitoring and some form of driver alert systems on all new cars by 2022 — along with automatic speed limiters that will prevent a driver from traveling above the legal limit to boot. But until that happens stateside, our own car giants have few incentives to push for the potentially life-saving technology.

Panasonic's approach to that challenge might seem a little counter-intuitive to sustainable transportation advocates at first. The company has essentially created a head-up display that's designed to minimize safety warnings to drivers, while offering them more straightforward luxury features like a turn-by-turn navigation system that can digitally draw a blue line on the windshield to show drivers exactly where to go without needing to rely on aural cues. But that same fancy navigation system could also help a driver better judge whether he or she is actually giving a cyclist at least five feet of passing space — and it could provide optimized safety alerts to which drivers will actually pay attention.

In a perfect world, of course, flawless automated driving technology with flawless roadway detection systems and speed governors on all cars would make head-up displays obsolete. But as it stands, even the most advanced AV tech commercially available not only isn't safe enough to trust with human lives, but may actually induce dangerous driving behaviors in the human beings that "smart" vehicles still rely upon when road conditions are unpredictable.

For all their many flaws, experts believe that human drivers are still generally better than an AV at making nuanced decisions about how to most safely pilot a vehicle in bad weather, in chaotic urban environments, or when something truly weird happens on the roadway that a computer can't possibly be programmed to predict. By essentially giving drivers the super-advanced sensing power of an automated car, a head-up machine could potentially make drivers a little safer — or at least definitively dispatch with the excuse that they didn't "see" a cyclist or pedestrian in time to prevent a tragedy.

And if such a technology can be sold to drivers as an attractive luxury item rather than a robotic scold that yells at them when they drive poorly, it might actually make its way into a lot of cars.

"Personally, I find the safety features of my head-up display really useful," said Poliak. "I love that it tells me my speed right now, that it gives me a little red alert when I’m going above the speed limit so I don't get a ticket. ... I think a lot of people are already finding that head-up displays can be a selling feature. Once you have one, you don’t really wanna go back to driving a car without one."

And for drivers who will simply never buy in to a heads up display, there are countless cheap and effective technologies available to us right now that can make walkers more visible to drivers without needing to augment their reality one bit — including the humble sidewalk, crosswalk, and protected bike lane.