There is no doubt that the wide deployment of vehicle and vessel autonomy has a huge potential to increase efficiency and safety. However, leaping directly from human-operated vehicles and vessels to fully autonomous ones is proving to be challenging and time-taking. The ongoing pursuit of autonomous mobility has revealed many different paths to commercialization. And while many organizations are focused on developing advanced Level-4 and Level-5 autonomous vehicles, others turn to ways to implement vehicle autonomy features in a more incremental fashion.
A little history
In the summer of 1995, Dean Pomerleau and Todd Jochem of Carnegie Mellon University’s Robotics Institute took a journey from Pittsburgh to San Diego in an autonomous minivan. The famous trip called “No Hands Across America” was 2 850 miles long and it was driven in a fully autonomous mode for 2 800 miles (98.2% of the trajectory!)[1].
Nearly three decades have passed since that trip, and despite receiving a great deal of attention and strong financial impulses, the sector of autonomous mobility has not advanced at the initially expected rate. The main reason behind this is the need to guarantee safety in all possible edge and corner cases, which are traffic situations where one or more operating parameters are at extreme values or where one or more unusual operating circumstances take place.
From an engineering perspective, it is not only difficult but also expensive to identify, reproduce, test, and optimize such edge and corner cases before launching driverless Level-4 or Level-5 autonomous vehicles. This is why many organizations have been looking for ways to implement autonomous features in a more incremental fashion. Teleoperation is now believed to be the best solution for enabling incremental autonomous deployments to occur sooner.
What is teleoperation?
Teleoperation is a technology that enables operators to remotely control vehicles and vessels via an internet connection, allowing for an alternation between autonomous and human-controlled navigation. In general, two main types of teleoperation can be distinguished: indirect control teleoperation and direct control operation. In the first type, the operator performs the tactical-level maneuvers (pathway planning), but not the dynamic driving task (real-time braking, steering, acceleration, transmission shifting). It corresponds with what has been defined by SAE International as Remote Assistance in the latest update of their taxonomy and definitions for terms related to driving automation systems[2]. This means that in this case, the role of the remote operator is to provide information or advice (waypoints) to an automated vehicle when it encounters a situation that it cannot manage. Until now, this has been typically the role envisaged for teleoperation technology in the context of automated driving, being a backup solution for stranded Level-4 and Level-5 autonomous vehicles. This approach has been explored before in several EU-funded research projects, such as 5G-MOBIX and 5GCroCo.
5G-Blueprint approach
5G-Blueprint focuses on the second type of teleoperation: direct control teleoperation. In that paradigm, the operator performs the actual dynamic driving task. This corresponds with what SAE defined as Remote Driving. It is not a form of driving automation and therefore has not received that much attention in the context of bringing Level-4 or Level-5 autonomous vehicles to the roads. In general, the focus in the sector has been on making sure that the vehicle automation system can work everywhere (Level-5) or at least on the trajectories for which they have been approved (Level-4, where the Operational Design Domain or ODD covers the entire trajectory).
However, even if direct control teleoperation is not a form of automation, it does remove the physical coupling between the human driver/captain and the vehicle/vessel that they control. And because of this single characteristic, the 5G-Blueprint consortium believes this technology is the missing piece of the puzzle for deploying Level-4 autonomy in real life. Instead of ensuring that the automation function can cover 100% of the trajectory, direct control teleoperation allows to split up Level-4 trajectories into different segments with different ODDs, and assign each of them to either automated driving or remote driving, depending on how difficult the ODD is to automate. In other words, by looking at automation and direct control teleoperation as two complementary technologies instead of seeing one as the fallback of the other, the challenges that have been holding the sector back can be solved by doing the same as what Dean Pomerleau and Todd Jochem did in 1995: relying on human drivers to navigate those challenging 2% of the trajectory. The main difference is that today, unlike in 1995, a driver does not need to be physically present in the vehicle. Today, a remote operator can “jump in and out” of different vehicles or vessels to take over as needed. And this single delta between then and now makes this approach viable from a business perspective. In this new approach, a human driver is only tied to a vehicle for 2% instead of 100% of the trajectory, largely reducing the corresponding personnel cost.
What has changed since 1995?
One might ask: why has this approach not been adopted already in 1995? The answer is simple: because of the stringent connectivity requirements when performing direct control teleoperation. That connectivity does not only need sufficient bandwidth for uploading multiple parallel HD video streams from the vehicle or vessel to the operator station but it should also provide very low latency and very high reliability. And when used for international transport, the connectivity should seamlessly roam between network operators at the border crossing: a combination of characteristics that has not been possible to provide with mobile network technology – not until 5G came into play.
Compared to all previous generations of mobile network technology, 5G, for the very first time, is designed not as a horizontal infrastructure that supports all applications with the same type of performance, but as an infrastructure that can be tailored to meet the needs of specific verticals. But even then, the direct control teleoperation network requirements seem to combine elements of both enhanced Mobile Broadband (eMBB) and Ultra-Reliable Low-Latency Communication at the same time, while also putting the emphasis on uplink instead of downlink bandwidth consumption. Even for 5G technology, this specific combination might be challenging to realize, which means that it needs to be investigated and validated to verify if it can provide the connectivity needed for direct control teleoperation. This validation, both from technology but also from business and governance perspectives, is the main task and purpose of the 5G-Blueprint project.
[1] No Hands Across America Journal, trip journal of Dean Pomerleau and Todd Jochem written during their trip in 1995. http://www.cs.cmu.edu/~tjochem/nhaa/Journal.html . Accessed on July 25th 2022.
[2] SAE International, Taxonomy and Definitions for Terms Related to Driving Automation Systems for On-Road Motor Vehicles, Report J3016, April 2021, https://www.sae.org/standards/content/j3016_202104