Despite widespread deployment of advanced driver assistance systems in current vehicles, new test results from the American Automobile Association reveal weak spots in the technology in both ICE and EVs that could have dangerous consequences

Advanced driver assistance systems (ADAS) are bridging the gap between conventional driver-controlled vehicles of today and the driverless vehicles of the future.

But even with growing market demand for functions like automatic emergency braking, automated lane-keeping and steering, and adaptive cruise control, new data from the American Automobile Association (AAA) shows that even in the most advanced vehicles hitting the market there are still areas of concern in how the technology actually performs.

“Drivers tell us they expect their current driving assistance technology to perform safely all the time,” said Greg Brannon, director of AAA’s automotive engineering in a news release.

“Unfortunately, our testing demonstrates spotty performance is the norm, rather than the exception.”

Three vehicles tested

To evaluate the performance capabilities and reliabilities of some different ADAS technologies offered by OEMs, the AAA conducted an experiment using two combustion engine vehicles (a 2021 Subaru Forester with EyeSight and a 2021 Hyundai Santa Fe with Highway Driving Assist) and one electric vehicle (a 2020 Tesla Model 3 with Autopilot).

All three vehicles’ systems are classified as Level 2 autonomy based on the Society of Automotive Engineers’ six-level classification system for driving automation, in which fully manual is deemed Level 0 and fully autonomous (ie. driverless) is Level 5.

Given that Level 2 autonomy reflects current state-of-the-art in North American vehicles on the road today, the AAA’s findings have immediate relevance. But future implications are also significant.

Within the next five years, all major automakers are looking to offer Level 3 automation, which includes ADAS technology that can control the driving without direct human supervision but still requires a human’s ability to take control over the car at any moment, says Flavio Volpe, president of the Automotive Parts Manufacturers’ Association (APMA) in an interview with Electric Autonomy.

In this context, automobile advocacy groups say it is essential that drivers of all types of vehicles be aware of the limits of ADAS technology and remain vigilant behind the wheel.

“Safety is always a really important thing to keep in mind when it comes to advanced driver assistance systems. I stop on the word ‘assistance’ because that’s exactly what these technologies are: to assist a driver — never to replace one,” explains Kristine D’Arbelles, a senior director of public affairs with the Canadian Automobile Association (CAA) in an interview with Electric Autonomy Canada.

“It’s really, really important that these systems are there to assist you and that you completely understand how the system works.”

Four driving scenarios

In their latest tests, researchers at AAA simulated four different scenarios that included overtaking “dummy” cars and cyclists moving in the same direction as the test vehicle, confronting head-on collisions and avoiding bicyclists crossing the street.

According to the AAA news release, all three vehicles it tested were able to identify and avoid colliding with dummy cars and bicycles travelling in the same direction during 15 simulations.

However, during simulated head-on collision tests, the Hyundai Santa Fe and Subaru Forester did not detect or slow down in order to avoid colliding with the dummy car. The Tesla Model 3 — purported by the brand to be one of the most advanced driverless technology vehicles in the industry — recognized the oncoming vehicle and significantly reduced its speed, but still collided with it.

In five test runs, the Subaru Forester’s EyeSight also failed to identify a bicycle rider crossing its path. The ADAS systems in the Tesla Model 3 and Hyundai Santa Fe passed those tests.

“While it may be encouraging that these driving systems successfully spotted slow-moving cars and bicyclists in the same lane, the failure to spot a crossing bike rider or an oncoming vehicle is alarming,” said Brannon. “A head-on crash is the deadliest kind, and these systems should be optimized for the situations where they can help the most.”

While the AAA’s ADAS results reveal inconsistent performance is an ongoing issue with current driving assistance systems, Volpe explained it’s not exactly surprising.

Limitations known

“What they published is something that all of us in the industry already know, which is that these are the earliest commercially viable systems by several automakers,” says Volpe.

He adds that since the technology is still in its early stages of application, it means that there are limitations to its computing and processing powers.

Earlier this month, a Tesla in California crashed into a curb and collided with construction equipment. At the time, the driver was operating the car in Autopilot mode. The crash killed three people and left another three injured. The U.S. government’s road safety agency is now investigating the accident and is taking part in a larger inquiry on crashes involving ADAS.

Meanwhile, in Canada, Tesla recently deployed its Full Self-Driving (FSD) Beta program, first promised to Canadian drivers by the automaker’s CEO, Elon Musk, in January. Tesla’s FSD Beta is a more advanced level of ADAS capabilities than Autopilot and can assist drivers in auto parking, changing lanes on the highway, recognizing stop signs and traffic lights, and automatically slowing the car to a stop.

So far, no collisions or mishaps from Tesla’s FSD technology have been publicly disclosed in Canada.

Electric Autonomy reached out to Tesla and Subaru for comment on the results of the AAA test but did not receive a reply prior to publication.

In an email statement in response to questions from Electric Autonomy, Hyundai said the company “is reviewing the findings in AAA’s report as part of our ongoing commitment to customer safety.”

For its part, in a recent job posting in the Greater Toronto Area, Tesla is hiring an ADAS test operator.

“We are looking for a highly motivated individual to accelerate our vehicle-level testing for all current and future Autopilot features on the path to full self-driving,” reads the posting. “The ADAS Test Operator will be responsible to identify improvements and regressions across software iterations.”

Canadian ADAS standards coming

With numerous driver assistance systems with similar features hitting the market, the CAA’s D’Arbelles says the challenge will be making sure each one meets a universal regulation standard.

“I know that Transport Canada and the Canadian government are looking into putting out some sort of standards on the naming of the particular system or even the way that the system reacts, but at this point in time there are no national standards for advanced driving assistance systems,” says D’Arbelles.

Canadian drivers and OEMs are finding themselves in a grey zone that policy is struggling to catch up with and it appears there isn’t going to be an imminent resolution.

“In September 2020, Transport Canada held public pre-consultations regarding the creation of regulations for ADAS technologies and automatic emergency braking (AEB) systems,” says Transport Canada in an email statement to Electric Autonomy.

“The department is aiming to publish a draft regulation on the topic of ADAS technologies and AEB systems in the Canada Gazette Part I in 2023. This will be followed by a public consultation period to gather feedback on the proposed regulation.”

Although there is an absence of national regulations on ADAS, Canada’s Motor Vehicle Safety Act outlines safety rules with which all vehicles imported or manufactured in Canada must comply.

“At the core right now, the most important regulation is that you must have a steering wheel and a human being in the driver’s seat. And they must be able to override the assistant systems,” says Volpe.

“In the Motor Vehicle Safety Act for the 2023 model year, there is no allowance for Level 5 non-steering wheel or non-driver occupied vehicles. In spite of what popular culture says about Tesla’s autopilot system or General Motors Cruise, amongst others, you still have to be taking the wheel. This is just helping.”

D’Arbelles adds that, currently, it’s on the shoulders of the consumer to fully understand the operations and limitations of the ADAS technology of the car they are driving since each vehicle company will be offering features that are slightly different.

Until a real standard comes into place, the CAA has an autonomous vehicle advocacy webpage that is meant to help drivers learn about how the most popular ADAS work and also flags the different names OEMs might be using for the same features in their vehicles.