Introduction

In recent years, developments in autonomous driving have appeared constantly across the media, attracting attention and expanding our imagination of future travel. The technology seems poised to move straight from laboratories to the market. Yet mature autonomous vehicles have not appeared on the market. Even technology that would merely let a driver play a round of a game has not been commercialized. Laws and regulations have also failed to keep up with this new technology: some things are technically possible but legally prohibited. Below, I briefly discuss the current state and prospects of autonomous driving through its research background, core challenges, and expected future developments.

Research Background

What Is an Autonomous Vehicle?

Before examining the technology, we should define an autonomous vehicle. Here is the definition from Wikipedia: an autonomous car, also called a driverless car, computer-driven car, or wheeled mobile robot, is a type of unmanned ground vehicle with the transport capabilities of a conventional car. As an automated vehicle, it can sense its surroundings and navigate without human operation. Fully autonomous cars have not yet been widely commercialized; most remain prototypes or demonstration systems, with only some reliable technologies reaching production vehicles. Nevertheless, their gradual emergence has already sparked many ethical discussions.

Advantages of Autonomous Vehicles

Autonomous vehicles in the full sense emerged in Carnegie Mellon University’s Navlab and ALV projects. Rapid development did not begin until the twenty-first century. Researchers pursue them because of their many advantages:

1. Fewer traffic accidents. Unlike human drivers, whose perception is limited, autonomous cars can use active and passive sensors such as lidar and radar to continuously scan a wide area, using visible light, infrared, and sound waves such as ultrasound. With a 360-degree field of view, they can respond safely to potential hazards, faster than human drivers.
2. Avoiding accidents caused by human factors such as following too closely, distracted driving, and dangerous driving.
3. Reducing required safety gaps and managing traffic flow better, increasing road capacity and reducing congestion.
4. Relieving occupants of driving and navigation tasks.
5. Allowing higher speed limits for autonomous cars.
6. Removing restrictions based on passengers’ circumstances: whether they are underage, elderly, unlicensed, blind, tired, intoxicated, and so on.
7. Easing parking shortages. A car could drop off its passengers, park somewhere with available space, and return when needed.
8. Increasing car sharing and reducing the total number of vehicles. After dropping someone off, an autonomous car could pick up another passenger elsewhere.
9. Reducing the physical space needed for parking.
10. Eliminating unnecessary occupants. An autonomous car could travel empty to where it is needed, for example to collect passengers or receive maintenance. This is particularly relevant to trucks, taxis, and car-sharing services.]
11. Reducing demand for traffic police and lowering car insurance costs.
12. Reducing physical road signs. Autonomous cars can receive the necessary information electronically, although human drivers would still need signs.
13. Providing a smoother ride.
14. Reducing vehicle theft through improved sensing capabilities.
15. Saving cabin space by removing the steering wheel and other driving interfaces, and allowing cabin designs that do not require anyone to sit in a driver’s seat.

Core Challenges

Levels of Driving Automation

Autonomous driving has considerable potential, but as a developing technology, it faces obstacles in many areas. Before discussing the core research challenges, it helps to understand how automation levels are classified.

In the United States, the National Highway Traffic Safety Administration (NHTSA) proposed an official five-level classification system for autonomous driving (2016 version):

- Level 0: no automation. The driver controls all mechanical and physical functions at all times. Vehicles equipped only with warnings and other functions unrelated to active driving also fall into this category.

- Level 1: the driver operates the vehicle, but individual systems can intervene at times. Electronic stability programs (ESP) or anti-lock braking systems (ABS), for example, help improve safety.

- Level 2: the driver remains primarily in control, but coordinated automation noticeably reduces the workload. For example, adaptive cruise control (ACC) combines automatic following with lane-departure warnings, while automatic emergency braking (AEB) combines some blind-spot detection and collision-avoidance technologies.

- Level 3: the driver must be ready to take control at any time. While automated assistance is in control—for example, while following another car—the driver may temporarily stop operating the controls. When the vehicle detects a situation requiring the driver, it immediately hands control back. The driver must handle situations beyond the system’s capabilities.

- Level 4: the driver can allow the vehicle to drive fully autonomously when conditions permit. Once activated, intervention is generally unnecessary. On roads meeting defined conditions—such as highways with smooth traffic flow, standardized signs, and clear markings—the vehicle can turn, change lanes, and accelerate on its own. Exceptions include severe weather, unclear roads, accidents, or the end of the autonomous-driving section. The system gives the driver “ample transition time.” The driver should monitor its operation, though supervised driverless parking may be included. (An autonomous car with a steering wheel.)

- Level 5: the driver does not need to be inside the vehicle and never controls it. Such a vehicle can activate its own driving systems and perform all essential safety-related functions without being limited throughout the journey to predefined road conditions, including when nobody is inside. It makes decisions independently of a driver’s wishes. (An autonomous car without a steering wheel.)

This was revised based on another classification published by SAE International, covering five degrees from driver assistance to full automation. Its guiding idea is to classify “who does what.”

This standard is widely used internationally. Most autonomous-driving technology currently sits between levels 2 and 3. The principal research obstacles fall into two areas: lidar-based hardware is too expensive, while solutions that depend heavily on computer-vision algorithms are not accurate enough.

Lidar-Based Approaches

Lidar works much like radar, using a laser as its signal source. Pulses emitted by the laser strike trees, roads, bridges, and buildings and scatter. Some of the light returns to the lidar receiver. Laser ranging calculations then determine the distance between the lidar and the target point. Repeated scanning collects data for all target points on an object, which can be processed into a precise three-dimensional image.

Compared with ordinary radar, lidar offers high resolution, low detectability, strong resistance to active interference, good low-altitude detection, small size, and low weight. But conventional lidar is difficult to mass-produce and therefore costly, prompting research into inexpensive solid-state lidar.

Researchers currently offer three main solid-state lidar approaches.

The first uses microelectromechanical systems (MEMS). A very small mirror directs a fixed laser beam in different directions. Because the mirror is tiny, its moment of inertia is low, allowing it to move fast enough to trace a 2D scan pattern in less than a second.

Luminar and Innoviz are two major companies developing MEMS lidar. Infineon, another lidar company, recently acquired Innoluce, a MEMS lidar startup. Last October, Chinese lidar maker RoboSense announced progress on MEMS and optical phased-array (OPA) solid-state lidar.

A major advantage of MEMS lidar is that its sensors can dynamically change their scan patterns to focus on particular objects, capturing and identifying details of smaller, more distant objects. Conventional mechanical lidar cannot do this.

The second approach uses a phased array. An array of emitters changes the direction of the laser beam by adjusting the relative phases of their signals. If the emitters fire in sync, the light travels in the same direction. If the left emitters lag behind those on the right, the beam points left, and vice versa. Although the principle is straightforward, most phased-array lidar systems remain in laboratories. Alex Lidow, CEO of Efficient Power Converter, says: “I have to say that phased-array lidar belongs to the future. We are still in the era of rotating or MEMS lidar, with the former dominating.”

Any discussion of phased-array lidar should mention Quanergy. This startup is a key technical adviser to Strobe, which General Motors acquired last October, so GM may also be working on this technology.

The third approach is Flash LiDAR, which operates more like a camera. The laser light spreads in all directions, illuminating an entire scene in a single flash. An array of tiny sensors then captures the light reflected from different directions.

One major advantage of Flash LiDAR is its ability to record a whole scene quickly, avoiding problems caused by movement of the target or lidar during scanning. But it has drawbacks too.

“The larger the pixels, the more signals you have to process. Packing huge numbers of pixels into a photodetector inevitably creates interference, which reduces accuracy,” explains Carnegie Mellon robotics specialist Sanjiv Singh.

Computer-Vision-Based Approaches

Computer-vision algorithms can be divided into supervised and unsupervised methods. The difference lies in how they learn.

- Supervised learning algorithms learn from training datasets and can continue learning until a specified confidence level is reached, minimizing the probability of error. They include regression, classification, anomaly detection, and dimensionality reduction.

- Unsupervised learning algorithms try to extract value from limited data. They establish relationships within available data to detect patterns or divide a dataset into subcategories based on similarity. Broadly, unsupervised algorithms can be divided into association-rule learning and clustering.

- Reinforcement learning is another form of machine learning, between supervised and unsupervised learning. Supervised learning has target labels for every training example, while unsupervised learning has no labels at all. Reinforcement learning has delayed, sparse labels: future rewards.

An agent learns appropriate behavior from these rewards. Understanding algorithms’ strengths and limitations and developing more efficient learning algorithms are goals of reinforcement learning. It can address many practical applications, from AI problems to control engineering and operations research, all relevant to autonomous vehicles. This can be divided into indirect and direct learning.

One major task for machine-learning algorithms in autonomous cars is to continuously represent the surrounding environment and predict possible changes. This can be divided into four subtasks: object detection, object recognition or classification, object localization, and motion prediction.

Machine-learning algorithms can be roughly divided into four categories: decision-matrix algorithms, clustering algorithms, pattern-recognition algorithms, and regression algorithms. A single category can handle more than two subtasks. Regression, for instance, can be used for object localization, recognition, or motion prediction.

Overall, today’s machine-vision algorithms struggle to match the accuracy of the human eye. They remain some way from practical application.

Future Expectations

Based on current developments, the following trends may emerge in autonomous driving:

Trend 1: If realized, autonomous driving will create enormous economic and customer value.

• Autonomous driving will improve personal safety and could reduce accidents by more than 90%.

• It could save drivers an average of 50 minutes a day.

• It could enable people unable to drive, such as people with disabilities and older adults, to travel independently.

• New mobility models enabled by autonomous driving could lower per-kilometer costs compared with renting or buying a car.

• Fewer accidents would also reduce congestion costs and medical expenses.

Trend 2: If autonomous driving takes hold in China, its prospects will be substantial.

• By 2030, autonomous driving will account for around 13% of passenger kilometers traveled (PKMT), rising to around 66% by 2040.

• The number of autonomous passenger vehicles will reach around 8 million by 2030 and around 13.5 million by 2040.

• Total autonomous-vehicle sales will reach around US$230 billion by 2030 and around US$360 billion by 2040.

• The value of autonomous mobility-service bookings will reach around US$260 billion by 2030 and around US$940 billion by 2040.

Trend 3: Vehicle digitalization will lower costs, making large-scale adoption of autonomous driving easier.

The turning point is predicted to fall between 2025 and 2027. Estimates of the cost curves for underlying technologies suggest this will be the point of economic parity between autonomous and human driving. In other words, the total per-kilometer cost of autonomous driving will roughly match that of a conventional car driven by a person. After this turning point, market demand for autonomous driving will grow steadily.

Trend 4: Autonomous-driving adoption will grow alongside mobility services.

China is rapidly expanding mobility services. Vehicles used for these services currently account for around 10% of total car sales in China. Private vehicles still dominate: they account for 90% of PKMT, with the rest coming from taxis, ride sharing, and other mobility services. McKinsey predicts that by 2030, autonomous vehicles deployed by mobility-service providers will account for 11% of PKMT, while privately owned autonomous vehicles will account for 2%. By 2040, providers’ share will reach as much as 55%, compared with only 11% for privately owned autonomous vehicles.

Trend 5: Chinese consumers show strong interest in autonomous driving.

A recent survey found that 49% of Chinese consumers consider full autonomy “very important,” while another 49% consider it “optional.” This contrasts sharply with Germany and the United States, where only 16% of consumers consider it “very important” and another 53% consider it “optional.” Chinese consumers are willing to pay a premium of up to US$4,600 for an autonomous vehicle, compared with US$3,900 in the United States and US$2,900 in Germany.