I have always loved a challenge. In college, I enjoyed mathematical modeling competitions: racing against the clock—and against other teams—to solve difficult problems. Later, when I decided to pursue a PhD in Engineering, I chose to study in the US, partly because it is where technology leadership exists, and partly because it would take me out of my cultural comfort zone.

As a computer vision engineer working in the vehicle automation field, I have had plenty of opportunities to work on interesting and challenging problems. It’s one of the reasons I came to work on the perception system for AutoFollow, Peloton’s next-generation truck platooning system. 

My role is to design, implement, iterate, and deploy perception system modules for AutoFollow. I work on several different aspects of perception, from evaluating the newest camera technologies to implementing image-processing algorithms to training neural networks to detect lane markings and other road infrastructure. 

With AutoFollow, Peloton is approaching automated trucking with an exciting yet practical solution that will help fleets address problems caused by driver shortages and turnover. With our human-led automated platooning solution, Peloton’s goal is to shorten the lead time to commercial truck automation, providing much-needed help to an industry dealing with the challenges of an aging infrastructure, distracted motorists, driver shortages, and rapidly growing freight volumes. Peloton’s approach relies on the skills of the professional truck driver to provide a boost to the safety and efficiency of our automated driving system.  

Why perception is the most difficult design challenge in autonomous driving

As the level of vehicle automation increases from driver assistive (SAE Level 1) to fully autonomous (SAE Level 5), the safety and engineering challenges for system designers increase as well. Higher functioning automated systems must handle all of the steering, acceleration, and braking functions on their own, without driver intervention, under the full spectrum of operating conditions. 

The vehicle perception subsystem provides the situational awareness required for safe and effective operation of the autonomous vehicle. Perception systems continuously monitor the drivable space ahead of the vehicle, detecting lane lines, detecting and tracking vehicles and objects in the surrounding area as the vehicle travels along. 

Perception system designers use advanced sensors and artificial intelligence to identify and interpret objects in the environment, whether that object is a solid white line marking the road shoulder, an overpass, or an SUV coming into the lane ahead. 

But even the most advanced sensors and algorithms have their limitations, so false positives and false negatives still present a challenge when designing a perception system that supports reliable Level 4 driverless operation. The extent of this issue is heightened by the long-range perception and path planning required to automate heavy-duty truck operation for on-highway applications. 

Perception systems must be able to handle the same challenging driving scenarios that human drivers deal with on a regular basis, such as following directions from a law enforcement officer directing traffic, accommodating an emergency vehicle coming up on the shoulder, following temporary lane configurations around a construction zone, or navigating a stretch of road with degraded lane markings. 

In day-to-day operation, an automated vehicle could encounter any number of these scenarios—and more. 

Heavy-duty truck automation presents an even more complex perception challenge. Heavier vehicles require a longer stopping distance, so the perception system must provide superior long-range perception capabilities in order for a heavy-duty truck to respond quickly and safely to issues down the road.

Many developers working on standalone driverless truck systems are struggling to meet the requirements of long-range perception and path planning using available sensor technology.

Despite tremendous advances in perception sensors and AI, today’s technologies are unable to match the power of human perception when it comes to powering the driving tasks required for vehicle automation. 

Nevertheless, many in the autonomous vehicle industry are trying to take the leap from driver-assistive, Level 1 systems directly to Level 4 or 5, replacing the human driver with a standalone, driverless vehicle system. Level 4 autonomous vehicles require a powerful perception system that can see and comprehend what’s going on around the vehicle at all times.

Leveraging the human driver’s ‘superpower’

Peloton’s approach to designing for Level 4 truck automation leverages the power of human perception. In our automated following truck platooning system, a human driver pilots the lead truck while Peloton’s AutoFollow system pilots the follow truck. The paired trucks are communicating wirelessly over a secure V2V connection, and the human driver in the lead truck acts as an extremely capable system ‘sensor’ for AutoFollow. 

Human drivers have an innate ability to perceive and understand what’s happening in the driving environment. The human brain can quickly make sense of what the human eye sees, even if it’s something unusual and unexpected, like a bicyclist riding on a busy freeway or a car approaching from the wrong direction on a one-way street. This ‘superpower’ allows us to distinguish an endless variety of objects under a wide range of conditions. The ability to detect and understand changes in the surrounding environment is essential when it comes to making timely and appropriate driving decisions.  

The professional driver in the AutoFollow lead truck drives normally, monitoring weather and traffic conditions ahead and responding appropriately according to the rules of the road and fleet best practices, as well as their own driving experience and intuition.  We help this lead human driver be even more safe and effective by equipping the lead and follow trucks with industry-leading Advanced Driver Assistance and Active Safety Systems.

The situational awareness and cognitive abilities of the professional truck driver provide guidance to the automated system as it determines the best path forward for the follow truck. 

Having a human driver in the lead truck means AutoFollow can handle more challenging driving conditions compared to traditional, standalone Level 4 systems. The professional driver in the lead truck can easily handle real-world situations, pulling over and stopping to allow an ambulance through, being waved through a construction zone by a flagger, or navigating a stretch of road where the lane markings are obscured. The lead driver’s situational awareness and ability to make informed driving decisions help AutoFollow select the best path forward for the follow truck.

Human perception and V2V connectivity help bring L4 trucking within reach 

AutoFollow’s human-led approach simplifies the challenge of designing a perception system for L4 trucks on highway and speeds the way toward a deployable L4 system. 

Wireless connectivity over secure, reliable vehicle-to-vehicle (V2V) ensures that the lead and follow trucks are in constant communication as they travel along at a close following distance. 

This combination of V2V connectivity and close following creates a unique operating environment for AutoFollow and reduces the longitudinal range over which reliable perception must be supported. When the vehicles are connected and in close platooning formation,, perception support for the automated truck is largely limited to the space between the follow and lead trucks and to the space around the follow truck. 

In the AutoFollow platoon, long-range perception, a requirement for heavy-duty truck braking, is provided by sensors on the lead vehicle and by the actions of the human driver. The automated follow truck can reliably ‘see’ a greater distance ahead than a standalone driverless truck. If traffic ahead of the lead truck were to suddenly slow down, the follow truck would immediately know about it. 

Safety issues related to false positives and false negatives are mitigated by having a human driver in the connected lead truck. Humans can quickly and accurately distinguish between objects that might pose a safety hazard and ones that don’t. A plastic bag floating in front of a moving vehicle could trigger that vehicle’s active safety system and cause the vehicle to brake unnecessarily, but a driver would recognize that the bag wasn’t a threat and would continue driving with the flow of traffic. 

The follow truck is equipped with its own suite of sensors and perception capabilities that enable it to follow lane markings and to detect and track objects in the surrounding environment, including within the gap between lead and follow trucks. If the V2V wireless connection between trucks is interrupted, the AutoFollow system will continue to guide the follow truck, overseeing its safe operation until the truck can pull off the road and stop, out of harm’s way. Any object that intrudes into the space between the lead and follow truck, whether a motorcycle or a piece of debris kicked up by a passing semi, is detected by the perception system on the follow truck.

Leveraging production-ready technology from PlatoonPro

Fortunately, Peloton isn’t starting from scratch in designing an automated vehicle perception system. We’re building on a foundation of a production-ready perception system already in use by our Level 1 platooning system. 

When I first came to work at Peloton, I was impressed by the maturity of our automated vehicle technology, especially the system perception capabilities and connected vehicle safety systems. 

Riding along in the PlatoonPro follow truck for the first time, I’ll admit I was a little nervous when the driver engaged the platooning system and took his feet off the pedals. 

As the driver continued to steer the truck, PlatoonPro began to draw our vehicle toward the lead truck. My eyes were glued to the system display, where data from the on-board radar showed the distance counting down to the specified platooning gap. Video from a high-resolution, forward-looking camera on the lead truck was beamed to our display over the wireless V2V connection, providing the follow truck driver a view of the road conditions ahead. 

As we platooned down the road, the perception system on the follow truck detected the actions of the lead truck as the lead driver accelerated and braked with the flow of traffic. 

Later, when a car cut across the path of the lead truck in a last-ditch effort to exit the highway, it was immediately detected by the lead truck radar. The perception system recognized a safety hazard, and, even before the lead driver could react, the connected braking system began to slow the trucks. The platooning system separated the trucks to a safe manual following distance for follow driver takeover. As I witnessed the effectiveness of the perception system and of the mature, proven communication protocol (DSRC) that powers a reliable connection between paired Peloton vehicles, I was no longer nervous. I was excited.

AutoFollow – A practical and powerful approach to Level 4 automation 

When it comes to designing a Level 4 automated truck system, Peloton has the advantage of  our proven automated vehicle technology, which has been through a rigorous, multi-phase testing process and through tens of thousands of miles of commercial use, helping fleets deliver their loads with safety and efficiency. 

We’re building on the same proven hardware and sensor technology used for PlatoonPro, and we’re using the same robust vehicle-to-vehicle connectivity that enables system oversight of close-following platooning.  We’ve built in additional sensing capability on the follow truck to enable driverless automated following.

On top of this solid technology foundation, AutoFollow is using the world’s best sensor—the ‘superpower’ of human driver perception and intuition—to provide the best possible long-range perception capabilities for Level 4 automated trucking. 

With human-led automated platooning, Peloton is taking a practical yet powerful approach that will speed the deployment of Level 4 automated trucking — enabling drivers and fleets to become much more productive. This means that the safety and efficiency of automated trucking will be available sooner compared to standalone driverless systems.  

It’s an approach that’s adaptable and more widely scalable across a variety of commercial environments, from hub-to-hub highway hauling to more constrained environments such as factory yards, mining operations, and remote logging roads.

Each of these settings brings its own set of challenges for truck automation. Fortunately, Peloton doesn’t shy away from tough problems. We’re building products to solve the challenges customers are facing today as well as those they’ll face in the future. 

As an engineer, these are the kinds of challenges I live for.