Challenging the global environmental issues
Make clear the seagrass and seaweed beds in Akkeshi Bay, Hokkaido with 3D sensing technology and an underwater drone

“Blue carbon” gathering momentum around the world

A survey ship with about seven people aboard sailed at full speed in Akkeshi Bay in the eastern part of Hokkaido and went out to the open sea, where it stopped. Crew members lifted equipment with both hands and submerged it into the sea. After a while, another man on board who was looking at the PC screen said, “I see seaweed.” After that, they continued to move from one area to another in the bay to repeat similar processes.

SSS, Hokkaido University and FullDepth, an underwater drone (ROV: Remotely Operated Vehicle) manufacturer, have regularly conducted seagrass and seaweed bed surveys like this since spring of 2022.

―― What kind of equipment did you put into the sea?

（Matsui, SSS）
“It’s original equipment made by mounting a sensing camera of SSS on a commercially-available underwater drone of FullDepth. This camera is equipped with “3D environment sensing technology,” which analyzes photos taken to recreate a realistic 3D model of the surroundings. The equipment is upgraded every time a demonstration experiment is conducted. Today (end of April, 2023), it is equipped with a new sonar that utilizes acoustic sensing to determine the distance to the seabed”

The reason why we are surveying seagrass and seaweed beds in the first place is that we intend to accelerate the research on blue carbon, which has long been conducted by the Field Science Center for Northern Biosphere, Hokkaido University. Blue carbon refers to carbon stored in marine ecosystems such as seagrass and seaweed beds and shoals, which was named by the United Nations Environment Programme (UNEP) in 2009. It is attracting global attention as a new source of carbon dioxide (CO2) absorption in measures against global warming. “Blue carbon credit” programs for trading the amounts of CO2 absorption have already begun.

With the increase of these needs, correct determination of the amount of blue carbon has become an absolute must. Professor Masahiro Nakaoka, Director of the Akkeshi Marine Station, Field Science Center for Northern Biosphere, Hokkaido University, is an expert in this field who has been engaged in the research on marine ecosystems for over 30 years. While he welcomed the upsurge of discussions in the world, he felt limitations to carrying out surveys on blue carbon on their own.

（Mr. Nakaoka, Hokkaido University）
“We have conducted research on changes in the distribution of seagrass and seaweed beds up to now but, for utilizing blue carbon as a measure against climate change, extensive and detailed surveys at a national level are required. When a university carries out a survey independently, it is generally either broad and rough or narrow and detailed. For conducting a broad and detailed survey, it was necessary to dramatically advance the research by introducing cutting-edge observation technologies”
Professor Nakaoka emphasizes that, in that respect, the joint research opportunity with SSS is quite significant.

Satellite photos and aerial drones alone are insufficient

── How has Hokkaido University surveyed seagrass and seaweed beds up to now?

（Mr. Nakaoka, Hokkaido University）
“Conventionally, use of satellite photos or aerial drones and undersea surveys by divers were the mainstream. However, there was a problem with aerial photography of the inability to capture deepwater areas and an issue of survey areas. For example, what looked like seaweed beds often turned out to be a brown rocky tract. Areas of interest must be surveyed by diving but working undersea requires skills and physical strength and we don’t have sufficient time or manpower available for surveying many areas”

Of course, we were aware that use of an underwater drone would improve the situation but, in reality, purchasing expensive equipment was difficult for a university and we were obliged to give up. Professor Nakaoka says that the partnership with SSS and FullDepth in such a circumstance raised expectations.

── How did FullDepth come to participate in the project?

（Mr. Ohashi, FullDepth）
“Originally, we also wanted to implement the underwater camera function for inspecting underwater infrastructure facilities. We happened to find out that SSS was developing the camera and decided to monitor undersea areas by combining it with our underwater drone. ‘Informatization under the water’ is FullDepth’s corporate concept and we also cooperate for seagrass and seaweed bed surveys as part of it”

The Sony Group itself has also launched “Earth MIMAMORI platform,” an internal project for sensing all locations on the planet, and Matsui is a member of the team.

（Matsui, SSS）
“Up to now, technology development of the Sony Group has mainly been intended for amusing people. However, we have a sense of crisis in that, from now on, for human beings to spend their life with amusement, sustainable social environment is required. Accordingly, we started working on the sensing of the earth itself to predict and prevent risks”

── How come did you join hands with Hokkaido University in that situation?

（Matsui, SSS）
“We wanted to sense the earth but it was not possible with our capabilities alone. We thought that we should cooperate with appropriate partners. In addition, if we wanted to put sensors into the sea, there was no location available for a demonstration experiment. When we looked out across Japan, we found out that Hokkaido University with a rich natural environment and experts in marine ecology was the most appropriate as a partner”

It may be possible to realize “visualization of the sea,” which was difficult up to now, by the three parties making use of their respective fields of strengths. In this way, a demonstration experiment started in Akkeshi Bay in April 2022.

Akkeshi Bay with rich seagrass and seaweed beds posing difficulty in demonstration experiments

At present, a quarter of Japan’s blue carbon ecosystems exist in Hokkaido. Of those, Akkeshi Bay provides a major area of extensive seagrass and seaweed beds. The rate of growth of eelgrass that constitutes seagrass beds is among the highest in the world and leaves are said to possibly grow by a few to few ten centimeters in one day. “It is like bamboos” is the expression used by Professor Nakaoka.

On the other hand, luxuriant growth of eelgrass and seaweed may pose an obstacle to surveying seagrasss and seaweed beds.

（Mr. Nakaoka, Hokkaido University）
“The biggest problem is that the underwater drone gets caught in eelgrass and seaweed. With fishing boats, the propeller gets entangled in eelgrass, which is annoying fishermen. It’s a simple problem but nobody has found a solution. Movement of seagrass and seaweed is another headache. For example, coral reefs do not move but seagrass and seaweed sway. With an ordinary camera, how they look may significantly differ depending on the timing when they are shot”

Furthermore, it is not possible to go out to the sea in Hokkaido with a harsh winter and the period that allows undersea surveys is limited to roughly between April and October. Those severe conditions indicate the difficulty of this demonstration experiment.

When the actual demonstration experiment started, it met with the first setback of the density of seagrass and seaweed beds. Issues soon emerged such as the underwater drone unable to find its way and the object getting too close to the camera for sensing. However, if the position of the underwater drone is too shallow, the camera cannot sufficiently capture the object in dark undersea areas. As a result of repeated trial and error including providing the equipment with a light, it was found out that operating the underwater drone at a depth of about 5 meters from the surface of the sea was the best.

In addition, there are issues peculiar to underwater drones.

（Mr. Ohashi, FullDepth）
“An aerial drone is equipped with a GPS receiver, which indicates the flight location, but the GPS does not function under water in the first place, making it difficult to grasp the position of the underwater drone. To address this problem, a ship or buoy can be equipped with a GPS receiver and an acoustic device under it to communicate with the underwater drone, thereby acquiring the position information. However, the accuracy is still low and the error can be as large as one to two digits under water where it is only in the order of a few centimeters on land”

Measuring the 3D shape of seagrass and seaweed beds in the order of millimeters

While we are still in the process of trial and error, we have obtained several results in the past one year and a few months. One is that we can now observe detailed conditions of seagrass and seaweed beds from aboard the ship.

（Mr. Nakaoka, Hokkaido University）
“Seagrass and seaweed, which are distributed in deep areas, cannot be seen from the air or the surface of the sea and does not allow observation without actually diving. In addition, the species and amount of seagrass and seaweed that grow cannot be determined simply by suspending commercially-available underwater cameras. Use of the underwater drone has shown us the way to some extent. I have observed this sea for a long time but even I was surprised to find seagrass and seaweed beds in unexpected areas”

Another is that the 3D environment sensing technology has allowed the 3D shape of seagrass and seaweed to be determined with an error of less than 1 millimeter.

（Matsui, SSS）
“The width of a seagrass and seaweed and the thickness and height of leaves of seagrass and seaweed can also be measured by using the underwater drone. Up to now, divers dived into the sea to measure seagrasses and seaweeds using a ruler or cut seagrass and seaweed to measure them on the ground. We can now acquire data undersea in real time and without affecting the natural environment, which I think is epoch-making”

── What technological capabilities of the Sony Group supported this innovation?

（Matsui, SSS）
“The underwater drone is equipped with the 4th-generation global shutter image sensor, which is the latest model, and its high sensitivity allows clear, blurless images to be shot. In addition, I feel that the capability of real-time 3D sensing in the order of millimeters even with a vehicle moving at a high speed is realized by our unique technology”

── What are the advantages of these results to Hokkaido University?

（Mr. Nakaoka, Hokkaido University）
“Until now, the only way was to purchase a product already developed to use for observation. It is a great advantage of the joint research with SSS to the university to have witnessed the scene of technology development where feedback of a demonstration experiment was rapidly reflected. Naturally, the research on blue carbon is making progress because seagrass and seaweed beds that we could not see are now visible. We have a real feeling of synergy of the partnership”

This joint research is apparently inspiring to students as well.

（Mr. Nakaoka, Hokkaido University）
“While it is important to learn fundamental science such as the origin and mechanism of life and marine organisms but students are also interested in application, or how to utilize what they have learned. With technological progress as in the present project, what was unclear in the past can be made clear. I have realized anew that academic-industrial cooperation goes a long way in that respect as well and is beneficial to education of young people of the next generation”

Want to contribute to the planet and business as well

The aim of this project is to informatize undersea areas and chart a map of seagrass and seaweed beds. As the system of blue carbon credit is taking shape, it will doubtless become a necessity for reporting how much seagrass and seaweed beds are in which areas.

However, there are issues to be resolved before the map can be charted. The biggest of all is the reduction of human resources required. To that end, “automation” of surveys using underwater drones is required. It will improve versatility and lead to wider use in more places in Japan and overseas.

In addition, as a corporate activity, the perspective of “whether it can be sublimated as a business while assuming a social contribution activity” is also essential.

（Matsui, SSS）
“We intend to apply it to businesses that contribute to the company, such as carbon credit and an infrastructure inspection service using an underwater drone. Meanwhile, in order to improve at any rate the critical situation of the sea, we would like to continue to commit to global environmental issues as a corporation”

Despite the different positions of corporations and a university, we are all members who share the same objective of protecting the sea. With a major sense of mission in mind, we will continue to work on seagrass and seaweed bed surveys in Akkeshi Bay.