Inside a massive AI project that has uncovered 7,000 whale deaths over 9 years

There are about 26,000 humpback whales in the North Pacific, and Ted Cheeseman has a picture of almost every single one of them. Or, to be more specific, he has pictures of their tails.



“Humpback whale tails are fantastic,” says Cheeseman, a life-long naturalist who is pursuing his PhD on humpback whales at Southern Cross University in Australia. “It’s this five-meter banner, saying: This is my identity!”



He means that quite literally. Each humpback’s tail is unique, like a fingerprint. Every speckle, spot, and scar offers a clue into the animal’s past, and they often raise their tails above the water before diving to lower depths, offering a prime photo opportunity for eager onlookers.



This is Frosty , a humpback whale known from the West Coast of the United States and Mexico, named for the snowman-shaped barnacle scars on the tail. Humpback whale tails display unique pattern and shape, allowing researchers to use AI-image recognition to track thousands of whales. [Photo: Ted Cheeseman]



In 2015, Cheeseman cofounded Happywhale.com, a website where scientists and civilians can submit photos of the whale tails they’ve spotted so they can be logged. An image-recognition tool powered by artificial intelligence scans each picture and compares it to the database to find a match. If a match is found, it means the whale has been seen before, and its location is updated. If not, the whale gets assigned a number and is added to the database. With more than one million photos submitted spanning 20 years, Happywhale has identified 30,000 different whales. It’s a powerful tool for researchers seeking to understand how climate change is affecting the humpbacks and the ecosystem on which they rely.



As part of a recent study published in the journal Royal Society Open Science, Cheeseman worked with 75 other researchers from 10 different countries to analyze the data from Happywhale and came to a startling conclusion: The humpback whale population in the North Pacific dropped by 7,000, or about 20%, from 2012 to 2021, after a strong marine heat wave killed off a lot of the whales’ prey, leaving them to starve. The figure was so shocking that the researchers were sure they’d made a mistake. They collected more data and ran the numbers again, but got the same results.



Health comparison of humpback whales: in poor body condition due to starvation (left) and healthy [Photo: Martin Van Aswegen/Marine Mammal Research Program/Pacific Whale Foundation.  NMFS Permit No. 21476/21321]



“The numbers that came out of that suggest it was very unhealthy for the whales,” Cheeseman says. “That was clearly a stressful event.”



But he’s quick to note that today’s humpback whale population is still much stronger than it was 50 years ago, before commercial whaling was banned and the whales were nearly hunted to extinction. “I’m not worried about species extinction or even a regional extinction,” he says. “What I am concerned about is the trajectory. The oceans are heating faster than our worst, most pessimistic models. So does today’s extreme event become the norm of 20 years from now? Then we’re really talking about a wholly different ocean ecosystem that supports less wildlife.”



Healthy mother and calf humpback whales, Maui, Hawaii [Photo: Martin Van Aswegen/Marine Mammal Research Program/Pacific Whale Foundation. NMFS Permit No. 21476/21321]



To that end, Cheeseman hopes the data from Happywhale can help researchers identify changes in the whales’ health more quickly. New photos are being submitted all the time, and the AI tool can identify a match with 97% to 99% accuracy almost instantaneously. It’s being trained to identify whales based on other body parts, like a dorsal fin, which is important because not all whales raise their tails, especially if they’ve been injured or caught in a fishing net.



Eventually, the researchers want to use the tool on other marine mammals facing their own population challenges. “Computers are getting faster, image recognition is getting better,” Cheeseman says. “The reality is this will only improve.”