Accelerating AI and IoT applications
— Toward the world where edge AI is for everyone

Edge AI that extracts only essential information is a crucial element for handling growing data volume

── What challenges do businesses face in using IoT and AI?

People would associate IoT with the idea of gathering and analyzing data from sensors of various connected devices to centrally control them. It does not pose any serious problem where the data are simple values from sensors, such as temperatures of objects or ambient air, but as soon as the information includes image data, the whole system becomes rather complicated.

Visual data, whether of still or moving images, comprise arrays of information of the light intensity captured by millions of pixels. The data volume tends to become large and likely to require massive processing. If those massive data are uploaded directly to the cloud system via the internet, it will incur substantial cost both to the sender and receiver.

We can think of a case where security cameras or IP cameras are connected to a server to implement a large-scale AI processing. In this case, images transmitted from tens of thousands of cameras on streams must be processed, and the cost of computation will be significant.

There is another problem. IoT devices are usually employed to connect sensors in tens or hundreds of thousands, thereby the collected data are analyzed to enhance the efficiency of system management and operation. These small devices are typically not enough capable of handling large-size data such as images.

In order to integrate diverse information including images into IoT-like system and be implemented more deeply in society, we framed the challenge to have a solution that enables devices to handle complex data and, at the same time, be deployed in a large scale like IoT sensors are, where neither of the conventional approaches were not so far able to handle.

It is in this context that edge AI technology offers an answer. It extracts necessary information from a large volume of data and sends only this information to the cloud, thereby reducing the data volume for transmission to a fraction of a thousand or of a million. This will enable enhanced system efficiency.

── Can you describe the edge AI sensing platform AITRIOS?

We know everything about sensor specifications and image quality tuning.

Image sensors have been designed to produce images that are perceived beautiful by the human eye. However, the focus is shifting in those Edge AI cases from “beautiful” images to those which can be “accurately” judged by AI according to its purposes.

Take a backlit situation, for example. For standard cameras, we control the image to reduce blowout and blackout so that both the sky outside the window and the objects on the indoor table are appropriately rendered. Whereas, for the AI for detecting the number of people in a meeting room, the images should be optimized to capture the in-room situations more accurately than outside the window to obtain better results.

With AITRIOS, we are aiming to build a platform that allows people to make use of the expertise in taking images that we have developed over the years.

There is another major point, addressing privacy concerns.
AITRIOS is capable of processing data on the sensor so that only the extracted information is transmitted to the cloud. It can, for example, transmit abstracted information, such as the number of people, removing all identity-sensitive information such as facial features, genders and attire of the people included in the visual data. This allows a system design with addressing privacy concerns with flexibility.

Expanding the scope of use cases to logistics, smart city and more

── Can you give examples of actual use cases?

There is a customer in the logistics industry, with whom we are developing a solution to optimize their business resource management and improve their operations through a virtual space (digital twin) based on the on-site data.

One of the major challenges for the logistics industry is to organize a large volume of goods at temporary storage stations, which can be time-consuming and labor-intensive. We have introduced a system in which Intelligent Vision Sensor IMX500 with AI built into the sensor detect boxes to identify storage in use and space availability, enabling efficient arrangement of goods brought into the warehouses. This system gives the workers instructions as to where and by which route the goods should be stored. In this way, it reduces the work load and time for the workers.

AITRIOS is also implemented for the smart city trial in Rome, where they are trying to address their social issues such as inner-city traffic congestion. In the USA, also, we have some similar projects developing with several customers.

To monitor an entire city will require installing cameras on hundreds of thousands of traffic lights or poles in the city and managing the image data they generate on a cloud system.
Given this situation, it is necessary that data are processed to an extent on the camera side to monitor the traffic volumes and to detect dangerous situations of potential road accidents in real time. Otherwise, the cloud side cannot simply keep up.

I believe that AITRIOS can facilitate optimal edge AI processing for every single camera in situations such as these and offer totally new solutions that leverage the strengths of the Intelligent Vision Sensors connected through AITRIOS.

── Are these solutions proposed from the SSS side?

Initial ideas can come either from SSS or the customer side. In most cases, however, new ideas develop through interactive discussions with the customers, and I find it exciting.

For example, we explain to customers that AITRIOS can be used to monitor the number of people in a meeting room and, when the room becomes unoccupied, control the air conditioning system. The customers may then come up with some ideas in a new direction. We take it on board and, with the customers, develop a system that best suits the suggested usage.

Because AITRIOS is something that has never been known to the world, I believe that we can create various new solutions with it by working together with customers and partners.

── What difficulties did you face in developing AITRIOS?

I can only speak about it from the business perspective.
First of all, the great potential of AITRIOS was the most difficult part. Because we are proposing something not like a new model of an existing product range, we need to explain the product’s advantages from various perspectives. It is also necessary to offer multiple approaches to its applications.
Rather than just describing it as “an AI solution that incorporates visual data,” we convey the idea in such a way that facilitates interactive communication between us and the customers. Having them get excited with that it can connect a number of cameras to the cloud like IoT sensors, they may develop the idea further in relation to their own ideas or potential problems to be addressed, and through the dialog, a “solution” in its true sense can emerge.
We consider inter-personal interaction very important in our pursuit of a prosperous society. Such interaction enables to identify our technology with which customers’ issues can be solved, to combine our technology with their technology to address bigger challenges, to recognize, through discussions, greater issues which the customers have been unaware of, and so on.

Being a network-based platform, AITRIOS also has a strength in facilitating suitable matchmaking between partners in those contexts.

Taking on tough challenges to further personal competence

── How did you come to take on the secondment to the USA?

I played badminton at university, and it was always my impression that I improved my competence in games better by playing with someone who were better than me but I could win depending on my condition of the day rather than playing against the best players who would win a tournament.

This principle stayed with me when I joined SSS. I have always had my eyes on projects with challenges that stretched my competence one step further. So, I have mostly been working on something with new challenges or problems with no established solutions, like introduction of new features and, before the AITRIOS project, cloud system integration.

As for the secondment to the USA, it came to me recommended by the senior general manager of the current division, with whom I had worked on one of the previous projects. At the time, AITRIOS was not yet given a name, and it was still in an exploration phase. I guess that he saw the dauntless quality in me in facing new challenges. I decided to accept the invitation partly because I appreciated his expectations, and also partly because I saw this was an ideal opportunity to develop my competence.

── What has been the most memorable occasion since you came to Seattle?

I went out to see the Tacoma Narrows Bridge on the first weekend of my arrival. This bridge has a history of immediate collapse following its completion in 1940. The cause was unknown at the time, a phenomenon known today as self-oscillation.
The engineers finally identified this cause, then redesigned the bridge to complete the present bridge. Their findings were also referenced across the world to take preventive measures on the existing bridge structures.
I felt encouraged to face the difficult challenges to come when I saw the physical example in front of me of a failure leading to a global impact for the better by learning from it and applying necessary amendments.

── What are your ambitions through AITRIOS?

Our motivation in developing AITRIOS lies in making society more prosperous and convenient by proliferating the benefits of edge AI technology to everyone in society, young and old, like smartphones have made photography universally accessible. Just like the engineers who brought the Tacoma Narrows Bridge a success, we are determined to follow through with our project no matter what difficulties may lie ahead.

AITRIOS is a system and product totally new in the world. Watanabe came across as a hopeful talent with his undeterred attitude and abilities in research and application of latest technology to make AITRIOS find its footing in the US market.

*) AITRIOS and AITRIOS logos are the registered trademarks or trademarks of Sony Group Corporation or its affiliated companies.