Image tool to help AI scour wildlife shots for climate change clues

The advance could help scientists create new AI-powered algorithms to perform rapid, in-depth analysis of the millions of wildlife images uploaded to the internet by members of the public each year.

These could help reveal key insights into the impacts of climate change, pollution, habitat loss and other pressures on tens of thousands of animal and plant species, researchers say.

Citizen science websites are a potentially rich source of information on how animals and plants are responding to climate change. However, while existing AI algorithms can automatically identify species in uploaded images, it was unclear if they could reveal other information too.

Now, an international team of scientists has created a new tool to test how well AI algorithms can mine image banks for other information. This could include details such as what species are eating, how healthy they are, and with which other species they are interacting.

The tool -- called INQUIRE -- measures AI's ability to drawn conclusions from an image bank of five million wildlife photos uploaded to the iNaturalist citizen science website.

The team found that current AI algorithms are capable of answering some of these types of questions, but they fail on the more complex ones. These included those that require reasoning about small features within images and ones that contain detailed scientific terminology.

The findings highlight opportunities to develop new AI algorithms that can better help scientists efficiently explore vast image collections, the team says.

The peer-reviewed findings will be presented at the NeurIPS conference, one of the leading conferences on machine learning.

The team included researchers from the University of Edinburgh, University College London, UMass Amherst, iNaturalist and the Massachusetts Institute of Technology (MIT). The work was in part supported by the University of Edinburgh's Generative AI Laboratory.

Dr Oisin Mac Aodha, of the University of Edinburgh's School of Informatics, said: "The thousands of wildlife photos uploaded to the internet each day provide scientists with valuable insights into where different species can be found on Earth. However, knowing what species is in a photo is just the tip of the iceberg.

"These images are potentially a hugely rich resource that remains largely untapped. Being able to quickly and accurately comb through the wealth of information they contain could offer vital clues about how species are responding to multi-faceted challenges like climate change."

Dr Sarah Beery, Assistant Professor at MIT, said: "This careful curation of data, with a focus on capturing real examples of scientific inquiries across research areas in ecology and environmental science, has proven vital to expanding our understanding of the current capabilities of current AI methods in these potentially impactful scientific settings.

"It has also outlined gaps in current research that we can now work to address, particularly for complex compositional queries, technical terminology, and the fine-grained, subtle differences that delineate categories of interest for our collaborators."