Despite widespread deployment of advanced driver assistance systems in current vehicles, new test results from the American Automobile Association reveal weak spots in the technology in both ICE and EVs that could have dangerous consequences
Advanced driver assistance systems (ADAS) are bridging the gap between conventional driver-controlled vehicles of today and the driverless vehicles of the future.
But even with growing market demand for functions like automatic emergency braking, automated lane-keeping and steering, and adaptive cruise control, new data from the American Automobile Association (AAA) shows that even in the most advanced vehicles hitting the market there are still areas of concern in how the technology actually performs.
“Drivers tell us they expect their current driving assistance technology to perform safely all the time,” said Greg Brannon, director of AAA’s automotive engineering in a news release.
“Unfortunately, our testing demonstrates spotty performance is the norm, rather than the exception.”
Three vehicles tested
To evaluate the performance capabilities and reliabilities of some different ADAS technologies offered by OEMs, the AAA conducted an experiment using two combustion engine vehicles (a 2021 Subaru Forester with EyeSight and a 2021 Hyundai Santa Fe with Highway Driving Assist) and one electric vehicle (a 2020 Tesla Model 3 with Autopilot).
All three vehicles’ systems are classified as Level 2 autonomy based on the Society of Automotive Engineers’ six-level classification system for driving automation, in which fully manual is deemed Level 0 and fully autonomous (ie. driverless) is Level 5.
Given that Level 2 autonomy reflects current state-of-the-art in North American vehicles on the road today, the AAA’s findings have immediate relevance. But future implications are also significant.
Within the next five years, all major automakers are looking to offer Level 3 automation, which includes ADAS technology that can control the driving without direct human supervision but still requires a human’s ability to take control over the car at any moment, says Flavio Volpe, president of the Automotive Parts Manufacturers’ Association (APMA) in an interview with Electric Autonomy.
In this context, automobile advocacy groups say it is essential that drivers of all types of vehicles be aware of the limits of ADAS technology and remain vigilant behind the wheel.
“Safety is always a really important thing to keep in mind when it comes to advanced driver assistance systems. I stop on the word ‘assistance’ because that’s exactly what these technologies are: to assist a driver — never to replace one,” explains Kristine D’Arbelles, a senior director of public affairs with the Canadian Automobile Association (CAA) in an interview with Electric Autonomy Canada.
“It’s really, really important that these systems are there to assist you and that you completely understand how the system works.”
Four driving scenarios
In their latest tests, researchers at AAA simulated four different scenarios that included overtaking “dummy” cars and cyclists moving in the same direction as the test vehicle, confronting head-on collisions and avoiding bicyclists crossing the street.
According to the AAA news release, all three vehicles it tested were able to identify and avoid colliding with dummy cars and bicycles travelling in the same direction during 15 simulations.
However, during simulated head-on collision tests, the Hyundai Santa Fe and Subaru Forester did not detect or slow down in order to avoid colliding with the dummy car. The Tesla Model 3 — purported by the brand to be one of the most advanced driverless technology vehicles in the industry — recognized the oncoming vehicle and significantly reduced its speed, but still collided with it.
In five test runs, the Subaru Forester’s EyeSight also failed to identify a bicycle rider crossing its path. The ADAS systems in the Tesla Model 3 and Hyundai Santa Fe passed those tests.
“While it may be encouraging that these driving systems successfully spotted slow-moving cars and bicyclists in the same lane, the failure to spot a crossing bike rider or an oncoming vehicle is alarming,” said Brannon. “A head-on crash is the deadliest kind, and these systems should be optimized for the situations where they can help the most.”
While the AAA’s ADAS results reveal inconsistent performance is an ongoing issue with current driving assistance systems, Volpe explained it’s not exactly surprising.
“What they published is something that all of us in the industry already know, which is that these are the earliest commercially viable systems by several automakers,” says Volpe.
He adds that since the technology is still in its early stages of application, it means that there are limitations to its computing and processing powers.
Earlier this month, a Tesla in California crashed into a curb and collided with construction equipment. At the time, the driver was operating the car in Autopilot mode. The crash killed three people and left another three injured. The U.S. government’s road safety agency is now investigating the accident and is taking part in a larger inquiry on crashes involving ADAS.
Meanwhile, in Canada, Tesla recently deployed its Full Self-Driving (FSD) Beta program, first promised to Canadian drivers by the automaker’s CEO, Elon Musk, in January. Tesla’s FSD Beta is a more advanced level of ADAS capabilities than Autopilot and can assist drivers in auto parking, changing lanes on the highway, recognizing stop signs and traffic lights, and automatically slowing the car to a stop.
So far, no collisions or mishaps from Tesla’s FSD technology have been publicly disclosed in Canada.
Electric Autonomy reached out to Tesla and Subaru for comment on the results of the AAA test but did not receive a reply prior to publication.
In an email statement in response to questions from Electric Autonomy, Hyundai said the company “is reviewing the findings in AAA’s report as part of our ongoing commitment to customer safety.”
For its part, in a recent job posting in the Greater Toronto Area, Tesla is hiring an ADAS test operator.
“We are looking for a highly motivated individual to accelerate our vehicle-level testing for all current and future Autopilot features on the path to full self-driving,” reads the posting. “The ADAS Test Operator will be responsible to identify improvements and regressions across software iterations.”
Canadian ADAS standards coming
With numerous driver assistance systems with similar features hitting the market, the CAA’s D’Arbelles says the challenge will be making sure each one meets a universal regulation standard.
“I know that Transport Canada and the Canadian government are looking into putting out some sort of standards on the naming of the particular system or even the way that the system reacts, but at this point in time there are no national standards for advanced driving assistance systems,” says D’Arbelles.
Canadian drivers and OEMs are finding themselves in a grey zone that policy is struggling to catch up with and it appears there isn’t going to be an imminent resolution.
“In September 2020, Transport Canada held public pre-consultations regarding the creation of regulations for ADAS technologies and automatic emergency braking (AEB) systems,” says Transport Canada in an email statement to Electric Autonomy.
“The department is aiming to publish a draft regulation on the topic of ADAS technologies and AEB systems in the Canada Gazette Part I in 2023. This will be followed by a public consultation period to gather feedback on the proposed regulation.”
Although there is an absence of national regulations on ADAS, Canada’s Motor Vehicle Safety Act outlines safety rules with which all vehicles imported or manufactured in Canada must comply.
“At the core right now, the most important regulation is that you must have a steering wheel and a human being in the driver’s seat. And they must be able to override the assistant systems,” says Volpe.
“In the Motor Vehicle Safety Act for the 2023 model year, there is no allowance for Level 5 non-steering wheel or non-driver occupied vehicles. In spite of what popular culture says about Tesla’s autopilot system or General Motors Cruise, amongst others, you still have to be taking the wheel. This is just helping.”
D’Arbelles adds that, currently, it’s on the shoulders of the consumer to fully understand the operations and limitations of the ADAS technology of the car they are driving since each vehicle company will be offering features that are slightly different.
Until a real standard comes into place, the CAA has an autonomous vehicle advocacy webpage that is meant to help drivers learn about how the most popular ADAS work and also flags the different names OEMs might be using for the same features in their vehicles.
Sure hope regulation doesn’t come soon. Tesla is putting out updates every few weeks, with improvements each time. I don’t doubt that with government in charge, each update would require approval. Since each update has proven to be a little better than the last, we don’t need slow!!!
To qualify for beta full self driving, you have to have achieved a high safety score (Tesla software monitors you) , agree to maintain control of the vehicle and be prepared to override at all times. With currently 100,000 users in the US, it seems there has been one bad accident with no deaths. That one bad accident would have been worse, but the car took enough evasive action to save the driver.
Autopilot, which only operates on major highways, has proven to be a significant safety aid, according to statistical data.
“… At the time, the driver was operating the car in Autopilot mode.”
I have been following this incident quite closely, along with most of the other collisions involving Teslas where autopilot may have been a factor (being an early TSLA investor, car owner and Musk fan) and in none of them can I find any confirmation that autopilot was actually being used in this particular tragic incident.
Even if it were, the simple fact is that the driver should have been sufficiently aware to take evasive action and avoid a collision – just as with *every other Autopilot-related collision*. Simply blaming Autopilot every time a Tesla crashes and then publishing that assumption as ‘fact’ is at best very sloppy writing and at worst just the usual clickbait, fake-news claptrap that seems to pervade modern journalism. A shame as the rest of the piece is quite well put together.
I would also take exception with the idea that ADAS drivers must “… completely understand how the system works” as I doubt there are more than a handful of people on the entire planet who are in this position. All users have to know is what Tesla has said from the very beginning of Autopilot, which is that the driver is responsible for the vehicle and they must be in a position to take control instantly should it be required. This does not seem a terribly complicated proposition to me.
Lastly, with the daily hullabaloo written in articles published both on paper and online, I have never seen *anyone* point out that which is, to me a very obvious *fact*, that autopilot being used correctly has beyond doubt already saved hundreds of lives. Just consider every single video you have seen of a driver of a Tesla, fast asleep behind the wheel, barrelling down the highway at 70mph or more. Without autopilot engaged, almost certainly every single one of these incidents would’ve resulted in a crash – quite possibly a fatal one. Obviously, for every one of these videos made there are probably thousands more where no-one was around to video it.
Consequently all this tends to support Tesla’s stated position that its cars are generally far safer than other vehicles specifically because of its ADAS features (without going into the specific statistics).
It will be interesting to see if the insurance industry reacts as it should and starts lowering premiums for Tesla-owning customers.
These often flawed ADAS systems have been out in the production vehicles for years …. And the authorities are just coming up slowly with standards??? Interesting obsession with speed kills and speed enforcement, yet zero or very low concern or urgency in regards to half baked at best ADAS systems???
Comments are closed.