Fish recognition algorithms can stop invasive species

The meeting between energy production and nature is rarely as clear as in the case of hydro power plants and migrating fish. On the one hand, you have the force of enormous amounts of running water that create energy for the whole of Sweden, and on the other, salmon and other fish that want to migrate both up and down rivers and need routes past the power plants.

Stornorrfors is one of the largest of the 2,000 plants in the country and is located in Umeälven River, just west of Umeå in northeast Sweden. In 2010, Vattenfall installed a new fish ladder here, which would help spawning fish bypass the power plant, and a couple of years ago they took the first steps toward a fish recognition system that would make it easier for Vattenfall to monitor, categorise and analyse the fish and their movements.

“We worked out a way to determine the characteristics of salmon,” says Patrik Andreasson, Specialist for Research & Development at Vattenfall.

"And today we have working algorithms, which are really good at recognising size and the presence of fungal attacks, and at distinguishing whether the fish is wild or farmed, as seen by the clipped andipose fin on the salmon's back."

Difficult to track females

The purpose of the technology is to more clearly control which fish travel up and down the fish ladders that bypass the hydro power plants, and to monitor the evolution of species over a longer period of time. For example, when farmed salmon is released into Umeälven River, it is possible to use the artificial technology to see how many of them then return and also follow the wild salmon’s way to adjacent Vindelälven river.

In recent years, fish recognition has developed rapidly and there are several providers of the technology. Still, there is room for improvement and changes, not least in terms of what characteristics the algorithms recognise in the salmon.

Gender is a crucial, and one of the most difficult, characteristics to recognise in spawning salmon. It is especially important to track females. The more females, the more roe and fry and, in the long run, the more adult fish. Because it is extremely difficult to distinguish between females and males, especially early in the season, gender recognition at Stornorrforsen is currently done manually and visually by highly skilled staff. 

“If we’re going to use algorithms instead of people, the algorithms have to be just as good,” says Andreasson. "This is very difficult now, especially at the beginning of the season. We’ve made a first attempt to develop an AI that also recognises gender. It works, but not quite well enough yet. So we’ll continue developing it and give it a test run at the end of this season or at the beginning of next. This is an issue we will be working on throughout this year."

Future goal: to stop invasive species

Not only can the fish recognition technology be used to detect the right type of fish, but it also has great potential in discovering invasive species.

“We’ll be opening some Swedish watercourses, probably the Dalälven River first, to release wandering fish,” says Andreasson. "However, we may also want to prevent invasive species such as black gobies from migrating up the watercourses. Though we don't have a tangible project for this, we do have plans to use smart algorithms to release fish that are OK, but send back those that are not supposed to be there."

The problem with e.g., black gobies and many other invasive species, is that they reproduce rapidly, use up food resources at the expense of other species and eat the roe of other species, thereby reducing the populations of more important species. Gobies are an example of a species that is very adaptable and can survive in both fresh and salt water.

Sharing information is a matter of course

Regardless of how this AI technology is used in the future, the goal is to make it available to anyone who needs it.

The Vattenfall laboratory that works with fish recognition is unique in the world in purely technical terms, but maybe even more so in the transparency of its work. Patrik Andreasson reports that only a few weeks ago they were visited by the French power company EDF and received a great deal of interest in learning about their data and experiences from industry colleagues in Norway, Canada, the USA and Germany.

"It is a conscious choice on our part to be so open and accessible, a choice that everyone supports. As long as there is no competitive advantage to holding on to information, there is no reason to keep things secret, on the contrary. In the area of fish migration and large-scale hydro power, there is such sparse knowledge and so many opinions that we want new knowledge we learn to be passed on and increases the facts-based foundation."

Vattenfall often collaborates with universities. Not only are they good at analysing material but also at disseminating results. In addition, the website has more than 20,000 annotated images that anyone can use for creating new algorithms. 

"Such images are hard currency for anyone working with fish recognition. We have no intention of making this into a product. Instead, we are keen to make the material available to everyone."