INDUSTRIAL

Image Sensor

Driving toward a world free of road accidents:
a story behind our original SPAD ToF Depth Sensor for Automotive LiDAR Applications

May 9, 2023

There are various initiatives underway in the mobility applications today with the aim of eliminating road accidents. Self-driving cars represent one of the key approaches to its realization. Autonomous driving is an amalgamation of a diversity of technologies offered by different companies, and Sony Semiconductor Solutions Corporation (SSS) contributes with what it is truly good at: sensing technology. We interviewed Yuqing Liu and Théophile Seijiro Mascia from Automotive Business Division of SSS. Liu works in the development of SPAD ToF Depth Sensor for Automotive LiDAR Applications, and Mascia is in the client business support team. They shed light on the current whereabouts of SSS’ sensing technology.

Liu Yuqing

Sony Semiconductor Solutions Corporation
Automotive Business Div.

Liu joined Sony Semiconductor Solutions Corporation in 2021. She undertook the development of analog models and system simulators for testing the circuit and signal process designs of SPAD ToF depth sensors. In 2022, she was in charge of designing and verifying an analog circuit for the sensor and also involved in the design of the bottom layer and peripheral circuits for SPAD pixels. Her responsibilities expanded in 2023 to the evaluation of sensor circuits.

Mascia Théophile Seijiro

Sony Semiconductor Solutions Corporation
Automotive Business Div.

Mascia joined Sony Semiconductor Solutions Corporation in 2021. He has been engaged in the business side of SPAD ToF Depth Sensor for Automotive LiDAR Applications.
Having been part of the effort to launch the US business operation for the SPAD depth sensor, he plays a role in negotiating with several customers and partners in liaison with SSS’s teams in Japan and the USA.
With the experience with the American market under his belt, he also handles the businesses in the Chinese and European markets.

Autonomous driving must be designed to safeguard all drivers and pedestrians from road accidents

In Japan, road accidents claimed 2,610 lives in 2022. The figure has come down by 26 from 2021 and is a record low in 6 consecutive years. However, there are evidently many issues remaining to be addressed before the government target of “below 2,000 deaths by 2025” is achieved.

The major cause of traffic accidents is attributed to human errors, such as mismaneuvering of a vehicle. This includes confusing the accelerator with the brake pedal, consequently running over a pedestrian; falling asleep on a wheel, leading to a head-on collision, and so on.

This is the background against which state-led initiatives are underway to advance autonomous driving and implement the technology at socially viable levels, where cars can drive themselves safely without the driver’s maneuvering. The technology can potentially eliminate road accidents from the world—some even expect it to be the “savior” for road users.

We are still in the early days for the diffusion of autonomous driving in society

What should we know when talking about autonomous driving? The US Society of Automotive Engineers (SAE) defines 6 levels of vehicle autonomy, ranging from 0 to 5, and Japan also adopts this framework:

Level 0: No driving automation
Level 1: Driver assistance
Level 2: Partial driving automation
Level 3: Conditional driving automation
Level 4: High driving automation
Level 5: Full driving automation

The public awareness of autonomous driving is gradually growing as we see it advertised by automobile manufacturers on the media, and some aspects of it, such as the automatic braking system and adaptive cruise control for safely following the vehicle in front, are already entering the social domain.

Nonetheless, Liu and Mascia agree with one another that, seen from their in-the-field viewpoint, the current state of the technology is still a long way away from the ideal.

(Liu)
“The state of the current technology is well at Level 3 of the vehicle autonomy scale, but there are still issues yet to be addressed, such as whether a vehicle can reliably detect obstacles in its vicinity under the rain, in the fog, snow, or other adverse conditions. There are many more challenges to be resolved before realizing Level 5 vehicles, which do not require the driver to drive at all.”

(Mascia)
“In the USA and China, self-driving taxis are implemented with advanced autonomous driving technology. I have tried some of them and been impressed by their technological sophistication. This being said, it is not advanced enough to be implemented in vehicles in the consumer market.”

One of the factors that impede the proliferation of self-driving cars is the high cost and power consumption associated with automotive LiDAR (light detection and ranging) sensors, a crucial component of autonomous driving technology. LiDAR is a mode of sensing technology for detecting and recognizing the vehicle’s surroundings: conditions on the road as well as the positions and shapes of the cars, pedestrians and other objects around the vehicle. To improve the accuracy of data requires the vehicle to have more sensors, but this would augment the cost and power consumption exponentially.

Addressing this challenge, SSS has developed the SPAD ToF Depth Sensor for Automotive LiDAR Applications.

What is original about SPAD ToF Depth Sensor for Automotive LiDAR Applications?

SPAD, or single photon avalanche diode, constitutes a pixel structure designed to amplify the electrons from a single incident photon, known as the “avalanche amplification.” It is characterized by the nano-order accuracy of detection even under faint light. Applying this to LiDAR as the photodiode of a dToF sensor enables long-distance ranging with high accuracy. The dToF sensor measures the distance based on the time elapsed as the light emitted from the light source is reflected on the object and detected by the sensor, hence “time of flight” or ToF.

SPAD ToF Depth Sensor for Automotive LiDAR Applications is realized by some of SSS’ proprietary technologies. One is the stacked structure.

(Liu)
“This sensor has our original stacked structure with Cu-Cu (Copper-Copper) connection technology which connects the two layers, one having a pixel chip and the other a logic circuit chip. This structure enables to realize a smaller sensor than other sensors. What is more, it allows the sensor to integrate more circuits without changing the size of its surface area, and the pixels can take up a larger proportion of it. As a result, it can improve the sensor’s sensitivity.”

Mascia adds from the business point of view.

(Mascia)
“The high sensitivity enhances the sensor’s performance so that, for example, the detectable range is increased and objects with low reflectance, such as rubber tires, can also be detected. This is a significant advantage in terms of LiDAR specifications which appeal to many clients. The stacked structure also contributes toward the simplification of LiDAR development and cost reduction.”

SPAD ToF depth sensor stacked configuration
(Top: SPAD pixels, bottom: distance measuring processing circuit)

Need to ensure the safety and security 20 years ahead

On top of the integration of proprietary technology, they cut no corners in their engineering efforts on the safety and security of the hardware.

(Liu)
“To design a chip for automotive implementation, we must make sure that it has a reliable service life so that it will remain effective in 10 or 20 years’ time. For example, it should perform normally after 10 or 20 years of constant operation in extreme conditions, such as high temperatures above 100 degrees Celsius, and similarly at several tens of degrees below zero. The sensor’s reliability must be guaranteed. It is also required that the sensor can withstand high-load use. It must be able to perform at its full capacity for 20 years.”

Liu says, with confidence, that these requirements are highly exceptional compared to the development of other sensors at SSS, such as the image sensors for mobile applications.

(Liu)
“A failure of an automotive sensor may result in collisions with other cars or people on the road and cost human lives. Therefore, the required levels of safety are incomparable to those for other sensors, and our designs are thoroughly considered to clear the most stringent criteria.”The strict quality verification is creating some secondary gains.

(Mascia)
“We have clients who show interest in using this SPAD ToF Depth Sensor for purposes other than automotive applications, such as in manufacturing plants. They seem to think that, if a sensor offers the automotive-use quality, it must be good to use under pretty much any conditions.”

In terms of safety and security, they also design the laser beam emitted by a sensor with great sensitivity.

(Liu)
“If a person steps out in front of a vehicle, it is well possible that the laser beam hits the person’s eyes. Our SPAD ToF Depth Sensor for Automotive LiDAR Applications is therefore designed with the laser intensity so low that it cannot harm people even if it directly hit their eyes.”

(Mascia)
“With its capability to accurately detect objects even with very weak signals, our SPAD may revolutionize the mobility. I believe that it will made a significant difference in the diffusion of LiDAR and the advancement of autonomous driving technology.”

He is confident in saying this.

Imaging example of LiDAR with SPAD ToF depth sensor
(Top: point cloud, bottom: distance intensity image)

Cutting power consumption—an eternal challenge

It is not only the sensor’s quality that they pursue in the development. A commercial launch cannot even be contemplated without addressing the cost. The actual development involves a series of trials and errors as well as making improvements day in day out. In particular, lower power consumption is, as mentioned earlier, a point that comes high in client requirements.

(Liu)
“I am designing the analog circuit, which uses far less electric power than the logic circuit does. And yet, I am required to reduce its power consumption further. It is trials and errors every day, thinking how we can minimize the circuit’s power consumption even when the transistor is dormant.”

It sounds akin to wild goose chase, but no wonder technology has so far been found to realize a substantial reduction of power consumption in a single leap. She says that, for the time being, every little improvement counts toward the attainment of the goal.

Creating the world’s first LiDAR standards through client engagement

The market demands something else, too. It is the speed to keep up with the high pace of progressive changes in requirement levels even in one year, as the automotive LiDAR is still very much young in terms of its market base. Liu emphasizes the necessity to be up to speed with those changes.

(Liu)
“We have to start deliberating the details of circuit design as soon as we learn of clients’ new requirements. It takes about one year to design, test and evaluate a chip, then if the chip does not perform to the client’s criteria (standards for evaluation), we will be in big trouble. This is why we absolutely need to keep an eye on where the world is heading and be agile in our development activities.”

This said, the market for automotive LiDAR is still small and in its early days, so clients do not necessarily have their criteria clearly established. The reality is that SSS and its clients are working together to create the standards through close communication in order to compensate for the lack of benchmarks.

(Mascia)
“The LiDAR market is in an early stage of growth. It is particularly true of SPAD. It is not infrequent that I have clients’ reactions like ‘What is this? What does it do?’ Knowing that this can be the case, we must be able to guide them so that they readily understand what solutions we can offer with the sensor.”

Specific features of LiDAR are not uniform among SSS and other semiconductor manufacturers, and there are no common benchmarks established yet. Mascia believes that, given the present state of the technology, various types of LiDAR will be released for a foreseeable future.

Semiconductor manufacturer’s unique contributions

We have so far seen that SSS is taking an approach to vehicle safety through sensing technology. People may think that mobility is an area predominantly for car manufacturers, but Liu points out that semiconductor manufacturers can make important contributions only they can offer.

(Liu)
“As car manufacturers build vehicles with components procured from semiconductor manufacturers, these parts in themselves are a “black box” to them. I have in-depth knowledge about what goes on inside this black box, for I am the one who engineers the design. The desire to improve its performance is always there as I work in this industry. For automobile companies, their users are limited to ones who drive their cars. For us, if we design a semiconductor with superior performance, various car manufacturers would want to use it, and consequently the number of users of our product will multiply.”

(Mascia)
“The automotive industry is undergoing a once-in-a-hundred-year major transformation. However, it is not for their solo performance. Autonomous driving, for example, cannot be realized without collaborating with various other players to make a complete, market-worthy vehicle. In this process, technological revolution in the true sense of these words takes place—if I can be candid—not on the side of car manufacturers, but on our side; semiconductor manufacturers are responsible for the vehicle control system, sensing technology, and other features, and we have a very important part to play there.”

SSS is on the right track towards the mobility of future society with a focus on autonomous driving. We are assured that, behind the scenes, there are young talents grappling with technological innovations that are yet to be realized.