Dynamic Trajectory and Collision Avoidance in action
Even four screens cannot compare to being in the vehicle. All autonomous driving functions will need to be utilized to maximize safety.

While there are no finalized public guidelines on how teleoperation should be treated, there are indications that this sub-system will form part of regulatory frameworks in key markets, says Amit Rosenzweig

The greatest contribution autonomous driving can bring to everyday lives is making the roads safer for all. In 2019, approximately 38,000 people lost their lives due to car accidents. Perhaps some of these were unavoidable by even the best of drivers. However, there is still a significant amount of human error and irresponsibility involved in driving to result in so much death.

And yet, we still do not have autonomous driving (AD) technology commercially available anywhere in the world.

The simple reason: autonomous vehicles (AVs) are just not yet ready to handle every situation, and that means manufacturers, regulators and the public are not willing to hand over the wheel. 

Part of the issue with AVs is that there are edge cases, situations where the artificial intelligence does not know how to behave. In such times a human is required to take over and solve the problem. To handle this, autonomy companies have a safety driver sitting in each AV ready to take over if needs be. But that defeats the purpose of a self-driving car. 

In comes teleoperation. Teleoperation is the technology that enables a human to remotely intervene when autonomous vehicles need assistance. Just like with AVs, teleoperation has its own challenges to overcome, especially when it comes to safety. 

Before delving into the details of addressing these problems, it would be helpful to put teleoperation safety in the broader context of autonomous driving safety. 

Current state of autonomous driving safety 

While a good start, today’s safety frameworks and requirements are not sufficient for autonomous driving safety. 

Autonomy is a major paradigm shift in human mobility, and therein lies the challenge in regulating its safety. It used to be that the industry only needed to ensure the products were reliable and the drivers would be held accountable for their operation. Now, they need to deliver a safe driving system as well. 

There are different layers of assurance needed to guarantee safety for the full system. Electronic components that enable better sensing, computation and connectivity are the foundation of today’s autonomous vehicles. The industry will need to meet safety requirements for these components individually, but also develop, collaborate on and finalize other novel requirements. 

Yet, to replace human driving decisions with a vehicle is a far greater change than better sensors or better chips. To do this safely requires the hardware to be tested for reliability. But even this is not enough. Minimizing failure in the hardware does not guarantee safe and reliable driving decisions. Functional safety will be a necessary, but insufficient framework for AD safety regulation.

Teleoperation will be within the scope of AD safety 

Ultimately, autonomous driving safety will include hardware reliability as well as software simulation and verification. And this will apply to teleoperation, along with all the other elements, equally. And while there are no finalized public guidelines on how this sub-system will be treated, there is a growing body of evidence that it will form part of regulatory frameworks in key markets. 

In the context of self-driving vehicles, high-level guidelines for teleoperation are being codified around the world at the national, state and local level.

As regulatory bodies assert their priorities, teleoperation will have its own requirements. These will be a result of both general safety frameworks that govern any autonomous driving-related system, and of challenges that are specific to teleoperation. 

Three takeaways for teleoperation safety 

1. Functional safety is a good starting point 

Functional safety verification of a teleoperation system is critical. Just like any other safety-related, electronic system, understanding risk levels and probabilities of failure are necessary. After all, like most other systems in an autonomous vehicle, the integrity of the system relies on the integrity of its hardware components. 

2. Teleoperation has unique limitations that drive specific requirements and solutions 

There are specific limitations inherent to remote human assistance and control, which need to be effectively addressed by a teleoperation system. These include:

  • Situational awareness. ​The sensory information available to a remote operator can approximate, but never really match, the experience of driving in a vehicle. 
  • Network latency. ​Data needs to be exchanged between a vehicle and a remote operator. However, that connection is made, it will be subject to some lag between what is observed remotely and reality. 
  • Network reliability. ​Any system or technology used to carry data can be subject to variable performance, whether it’s 4G, 5G or even 6G cellular networks. Any network has limitations, whether it be bandwidth, reliability or competing traffic. 
  • Human error. J​ust like the millions of in-vehicle drivers on roads today, that remote operator can experience fatigue, distraction or simply make mistakes. 

Together, these limitations have specific implications for safe teleoperation, and must be addressed in a teleoperation system to be considered adequately designed and developed. 

3. Teleoperation is safest when taking full advantage of the autonomous driving system’s capabilities 

One important principle in enabling safe teleoperation is to take advantage of the resources available to the entire autonomous driving system. In order to operate autonomously, a vehicle contains sensing and computing capabilities that are unprecedented in the automotive and robotics worlds. 

In the event that remote human assistance or intervention is needed, key elements of the autonomous driving software stack should be used, in service of safety. In other words, if the vehicle has a state-of-the-art sensing and computing stack, a teleoperation system should take advantage of them. 

Today’s autonomous driving stacks offer their developers a toolkit for enabling safe autonomy, and those same stacks can enable safe teleoperation. But these are still toolkits — ingredients and components, rather than recipes. They vary from company to company, and they vary from operating environment to environment. 

Many environments, many solutions, limited resources 

Within the world of autonomous driving, there are very different operating environments — often referred to as operational design domains (ODDs) — that will almost certainly result in different safety and regulatory needs. 

Autonomous trips that take place exclusively on federal highways will need to contend with one set of regulatory bodies, scenarios, speeds and behaviours. Autonomous trips performed by a yard tractor or forklift within a manufacturing plant will face entirely different regulations and requirements.

As companies mature working prototypes and minimum viable products into safe, commercial autonomy solutions, shrewd prioritization and smart partnering can help them achieve a safer, viable solution faster. 

Amit Rosenzweig

Amit Rosenzweig is CEO of Ottopia, a provider of teleoperation technology which enables vehicles to be controlled remotely and safely.