Atsugi, Japan — Sony Semiconductor Solutions Corporation (“Sony”) announced today that its AITRIOS™ edge AI sensing platform will launch in Japan, the U.S. and Europe starting from late 2021. Sony is now widely recruiting corporate partners, inviting their involvement in the development and implementation of AI camera-driven and other sensing solutions that will make the most of this platform.
The AITRIOS platform makes it possible to streamline the development and implementation of AI camera-driven and other sensing solutions. The first step in the service entails providing a one-stop environment that offers various features that will make it easy for partnering solution providers such as AI developers, application developers, camera manufacturers/module integrators, and system integrators to build solutions spanning from edge to cloud.
Through this platform, Sony seeks to facilitate development of optimal systems, in which the edge and the cloud function in synergy, to support its partners in popularizing and expanding environmentally conscious sensing solutions using edge AI, and to deliver new value and help solve challenges faced by various industries.
Click here for the video with English text.
Operating environment and challenges
Advances in AI, the spread of IoT, and the accelerating digital transformation have resulted in an increasing need for sensing technology that extracts data from image sensors for recognition purposes, in order to provide new value and help solve business challenges. On the other hand, with the constantly increasing number of IoT devices acquiring and using images, such as smartphones, security cameras, home appliances, and automobiles, concerns have been raised regarding excessive reliance on cloud systems that serve all these multiple devices. The massive amount of data sent from IoT devices and increasing electric power consumption during the requisite system usage have become social issues. In addition, various other challenges also hinder the spread and expansion of sensing solutions, such as privacy concerns when personally identifiable data is stored in the cloud, latency in data transmission, data tampering and other security concerns, and the risk of service outage during emergencies.
Synergy of the edge and the cloud, and the Sony’s IMX500 intelligent vision sensor
Technologies which distribute the processing load across the cloud and IoT and other edge devices have begun to gain attention as a way to help solve these challenges. In contrast to cloud systems, which process data in a centralized fashion, carrying out the necessary processes on the edge reduces the volume of data sent to and processed within the cloud, thereby reducing electric power consumption and delay in data transmission. This also delivers a variety of other advantages, including positive implications for privacy, improved security by not connecting to the cloud and its many interfaces, and service continuity thanks to the capability to complete processes on the edge.
In May 2020, Sony announced the release of a new model of intelligent vision sensor, the IMX500, the world’s first*1 image sensors to be equipped with AI processing functionality. This sensor uses a stacked configuration consisting of a pixel chip and a logic chip, and is equipped with Sony’s original digital signal processor (DSP) mounted on the logic chip. This enables AI processing on the image sensor, the edge-most device, which serves as the entry point for image data. This design allows for high-speed AI processing and metadata output on the sensor itself, and has fueled sensor AI-driven sensing solutions in a variety of industries.
*1: Among image sensors. According to Sony research (as of announcement on May 14, 2020).
AITRIOS edge AI sensing platform features
Adding to the IMX500 intelligent vision sensor, AITRIOS offers partners a variety of features that help them craft solutions. Platform partners can include developers building AI that runs on cameras, developers building applications for AI-driven sensing applications, camera manufacturers/module integrators who produce AI cameras, and system integrators who build systems that integrate these AI cameras and sensing applications. Compared to other IoT devices, image sensors acquire a relatively large amount of information, and the difficulty in handling such a large volume of data has been an entry barrier for partners. Building on the technological advantages and expertise it has cumulated in the area of image sensor development, Sony enables data output optimized for AI. Moreover, Sony provides a one-stop platform that makes it easy to build sensing solutions from edge to cloud, which allows partners to streamline development and implementation of high-performance applications and solutions that meet their diverse needs. Future plans to expand service content are also in the works. Sony also plans to support other sensing image sensors on AITRIOS, in addition to the IMX500.
The platform supports partners in efforts to build optimal systems in which the edge and the cloud function in synergy and address global environmental issues, thereby helping solve issues with cloud systems. Furthermore, it will accelerate the popularization and expansion of edge AI-driven sensing solutions in various industries and provide new value and solutions to customers and the broader society.
In addition, Sony will work with partners to develop and deliver reliable AI that addresses AI ethics concerns regarding situations when AI that uses statistical methods outputs or makes decisions that were not anticipated at the time of system design.
The name “AITRIOS” consists of the platform keyword “AI” and “Trio S,” meaning, “three S’s.” Through AITRIOS, Sony aims to deliver the three S’s of “Solution,” “Social Value,” and “Sustainability” to the world.
AITRIOS service overview
- Launch date
Late 2021 (initial release) - Target audience
AI developers, application developers, camera manufacturers/module integrators, system integrators - Target regions
Service will first launch in Japan, the U.S., and Europe*2
*2: With plans to roll out the service in other countries and regions in the future. - Platform features
Features: Development environment
- AI and application development environment*3
The SDK and tools support development of compact AI models that run on the IMX500, and the development of sensing applications that utilize AI.
(For AI developers and application developers) - Cloud application development environment*3
The SDK and tools support the development of applications that link with sensing applications on the edge and optimize operations in the cloud.
(For AI developers and application developers) - AI camera development environment
A software development kit (SDK), tools,*3 and reference designs for supporting development of AI cameras equipped with the IMX500.
(For camera manufacturers/module integrators)
-AI cameras equipped with intelligent vision sensors are scheduled to be sold by partners.
*3: Not available upon initial service launch in late 2021. Scheduled for release at a later date.
Features: Market Place*3
- Download portal for AI models and sensing applications, includes a payment feature for purchases and sales. Developed AI models and applications can also be registered on the market place.
(For all partners)
Features: Cloud services
- Provisioning (service provision framework)
Easy AITRIOS setup: Simply scan the QR code to start.
(For system integrators) - Deployment (AI converter)
Install sensing applications purchased on the marketplace or designed on your own on an AI camera equipped with the IMX500 via the Sony’s original AI converter.
(For system integrators, AI developers, and application developers) - AI retraining
Retrain AI models to adapt to changing usage environments and conditions.
(For system integrators, AI developers, and application developers) - Device management
Manage the software versions and monitor the operating status and error logs of AI cameras equipped with the IMX500.
(For system integrators) - Connection with applications on external cloud services and applications
Connect with applications on external cloud services via the application programming interface (API) that AITRIOS provides, making it possible to add features as needed.
(For system integrators)
■ Related website: https://www.aitrios.sony-semicon.co.jp/en
AITRIOS usage examples
- At a U.S. AI developer
Using the AI development environment provided via AITRIOS, the company used an AI model of their facial detection and recognition as a base to develop a new, compact AI model that runs on the IMX500 DSP and achieves both functions simultaneously. They have also added remote update functionality by using the deployment (AI converter) feature. Furthermore, the company uses the AI development environment provided by AITRIOS to enable evaluation under conditions where image recognition is generally difficult, such as in dark locations, streamlining the development of AI that boasts high-precision results, even in varying environments and conditions. - At a U.S. application developer
The company’s electric power consumption control solution, which was developed for smart air conditioning systems inside buildings, was supported at the Co-Innovation Lab launched in a partnership between Sony and Microsoft. During the process, an SDK and tools provided by AITRIOS for sensing application development were used. By integrating the company’s original technologies with IMX500 and Microsoft Azure Cognitive Services, the solution detects the occupancy rate of conference rooms and shared spaces in office buildings and commercial complexes, enabling heating, cooling, and ventilating rooms as necessary instead of continuously running the air conditioning system regardless of the presence or absence of people, as was done in the past. - At a Taiwanese module integrator
The company’s AI camera prototypes, equipped with the IMX500 utilize the reference design provided by AITRIOS. This makes it possible to streamline the development process and contributes to a shortened development period. - With Japanese system integrators
At the Co-Innovation Lab launched in collaboration between Sony and Microsoft Japan, various partners worked to combine the IMX500 and Microsoft’s Azure Cognitive Services to develop a number of business solutions for the retail and manufacturing industries and others. These solutions utilize deployment and other features provided by AITRIOS.
※ AITRIOS and AITRIOS logos are the registered trademarks or trademarks of Sony Group Corporation or its affiliated companies.