Can parking-related concerns be solved with automotive sensing technology?! What is “Auto Parking Solution”? We asked employees from the Automotive Business Division.

Issues for Existing Parking Assistance Systems

reference image Zoom

Existing parking assistance systems have been commercialized by each OEM, but there is an issue in that only very few people use a parking assistance system regularly. This is due to reasons such as problems with the accuracy of parking space perception, the fact that their operation is complex and takes a long time, the unpredictability of the car movement, and the low accuracy of obstacle detection. In addition to solving these issues, we are aiming to eliminate the inconvenience of searching for a parking space by detecting a parking location in advance.

Auto Parking Solution Value and Component Technologies

reference image Zoom

The value offered by Auto Parking Solution is “Automatic parking assistance that can be used anywhere comfortably and without concern.” To achieve this, three component technologies that utilize the “Forward Parking Space Recognition” and “Parking Space Recognition” sensor technology possessed by SSS, are being developed. These are technologies to recognize empty spaces ahead in advance (Forward Parking Space Recognition), to recognize parking frames with high accuracy while assisting with automatic parking in any location (Parking Space Recognition), and to generate the optimized travel route to the parking location (estimation of own position, identification of areas where travel is possible, route generation). When the project was started, “technology to generate the optimized travel route to the parking location” was not an intended part of it, but as Kano reflects on the situation at the time, “The true value of automatic parking assistance that can be used anywhere comfortably and without concern can only be felt by actually experiencing it. For example, with just explanatory videos and documents, it is difficult to communicate the fact that there has been improvement from the inconvenience of pressing buttons multiple times to park, and from the frustration of the time spent waiting during automatic parking. We really wanted people to experience it for themselves in a vehicle, so we added technology to generate the optimized travel route to the parking location (estimation of own position, identification of areas where travel is possible, route generation) to the scope of development .”

Auto Parking Solution

reference image Zoom

Auto Parking Solution from SSS is a UX that realizes everything from the “search for an empty parking frame” to the “completion of parking” with just one action.

reference image Zoom

The figure above shows a comparison to existing vehicles equipped with parking assistance systems. For existing parking assistance systems, the number of manual operations is between six and nine (differs depending on the vehicle), and the stop time of the vehicle during operation also tends to be long (as shown by the grey lines). On the other hand, with SSS Auto Parking Solution, there are only two manual operations and the time the vehicle is stopped is greatly shortened. In experiments using vehicles with existing Parking Assistance Systems, it is anticipated that it will be possible to reduce the time required from the start of parking to its completion by half. In this way, “The sales points for SSS Auto Parking Solution are that smart parking is possible, the operation becomes easier, and the time is reduced.”

Parking Space Recognition (Technology to recognize the parking frame accurately)

reference image Zoom

With Parking Space Recognition technology, the range for the detection of empty parking spaces is a relatively close region within around 10 meters from the front end of the vehicle, and the goal is to achieve parking space detection that can recognize various types of parking areas. Nakagawa, who is responsible for its development, explains that “The use of Parking Space Recognition technology makes recognition possible in difficult situations, including the detection of various parking space types such as those with I-shaped or U-shaped white lines to mark spaces in car parks (see figure above), or hidden parking areas with only a small part visible (occlusion). These capabilities will be an advantage for us, even when compared with products from other companies.”

Forward Parking Space Recognition (Technology to recognize empty spaces ahead in advance)

reference image Zoom

reference image Zoom

With Parking Space Recognition technology, the targets for detection are empty spaces relatively close to the vehicle, but with Forward Parking Space Recognition technology, the targets are empty spaces roughly 10 meters or more from the front end of the vehicle. Furthermore, regardless of these distances, Forward Parking Space Recognition technology can handle peculiar spaces whose recognition is difficult with Parking Space Recognition technology, such as greened area parking, gravel car parks, and car parks that only have car stoppers, so it is anticipated that “Forward Parking Space Recognition technology will make up for the weak points of Parking Space Recognition technology.”

reference image Zoom

Nakagawa says, “Its strength is that parking spaces can be detected without relying solely on marking lines. For example, elements such as other vehicles, car stoppers, and walls can also be included.”

Further development will be conducted on future targets, including algorithms to recognize parking spaces in locations like the one below where there are no clear marking lines (the green outlines show the recognized spaces), and also the use of chronological information as context for detection to recognize the state of vehicles in motion.

reference image Zoom

Data Sets for Recognition Performance Improvement

Finally, we spoke about the data sets necessary to improve the recognition performance. In order to improve the accuracy of deep learning-based recognition devices, it is essential to have high quality training data. To obtain such data for the development of Auto Parking Solution, data is being collected from various car park locations and in various environments. Fujii’s team is responsible for the important role of collecting and annotating this data. They have travelled to 470 locations in the Kanto suburbs and collected data using a test vehicle equipped with the apparatus. He says that, “The data we collected for this project was from various types of parking locations during the daytime in clear weather, but data in rainy weather and during the night will also be necessary from now on. However, even just in clear weather, we collected data from as many as 470 locations, so it is very gradual, difficult work. ”

Future Issues and Initiatives

We asked about the issues currently being faced by Auto Parking Solution and its future initiatives.

Nakagawa: “Currently both Parking Space Recognition, which can recognize parking frames with high accuracy, and Forward Parking Space Recognition, which can recognize empty spaces ahead in advance, take images obtained from cameras and convert them into birds-eye view images. However, depending on the tilt of the vehicle, misalignment can occur between the actual parking space position and the birds-eye image. I want to advance development further to examine technical solutions to this issue, and also to perform recognition under a wide range of conditions such as during poor weather and at night.”

Fujii: “I believe that in the end, data is the most critical factor for the improvement of recognition accuracy. The most important points are how much data with a wide range of variations we can gather, and whether it can be used for learning. At present, we have collected data from 470 locations and I think our work is finally taking shape, so from here on I want to focus on how efficiently we can gather data under various conditions. When we looked at a live webcam in Hakone and noticed it was foggy, we even rushed over there to collect data.”

Kano: “We are latecomers to the automotive industry, so I feel that it will be difficult for us to be accepted by the market unless we reach and then exceed the level of existing mass-produced vehicles. To achieve this, I want to construct a robust parking assistance system capable of parking under any conditions, such as at night or in poor weather. As we look toward mass production, the recognition systems for our Parking Space Recognition technology and Forward Parking Space Recognition technology will be our selling points, so I want to conduct demonstrations to show that recognition is possible even in heavy rain, and reach the level where these systems can be selected and used by customers. We are currently proceeding with development mainly for use in Japan. To extend this technology to overseas use in the future, we will consider matters such as differences in the methods of parking according to country, to create systems suitable for each region.

In addition to being latecomers to automotive development, our department itself is also very young, so we will accumulate knowledge and experience from this point on. The future image and ideal state for this project is for our business to expand by having the systems we have developed be installed in mass-produced vehicles.