Automatic, non-invasive identification of Holstein cows by their coat pattern

Azzopardi, assistant professor in the Information Systems research group at the Bernoulli Institute for Mathematics, Computer Science and Artificial Intelligence at the University of Groningen, The Netherlands, has worked for several years on a project to improve livestock management by developing novel intelligent system tools. One idea that stemmed from discussions during the project was the need to develop a method that could identify individual animals based on their visual appearance instead of through the use of wearable sensors.

Temperature

'The current systems rely on RFID collars or tags to identify cows in specific stations within a farm,' explains Azzopardi. 'The management of such RFID collars and their limited read-out range present a challenge. We wanted to investigate the potential of an approach that is non-invasive and relies entirely on visual information captured by a camera.' This idea has been realized in collaboration with the Dutch Dairy Campus in Leeuwarden, which gave the scientists access to their farm with 383 Holstein cows. This made it possible to acquire the necessary data in a systematic way. To collect images, a thermal camera was used that could take a pair of images in one snapshot; an infrared image that indicates the temperature at the scene and a regular colour image. The two images were then processed using an artificial intelligence-based pipeline.

The first step in the analysis was to distinguish the cow from its surroundings. 'This is achieved using the thermal image, coupled with basic image processing techniques, which provides the contour of the cow,' explains Azzopardi. The second step was to delineate the contour of the coat pattern on the side of the cow. For this step, the scientists adapted the brain-inspired CORF algorithm with a push-pull inhibition component that Azzopardi had developed with other collaborators in the previous years. The CORF algorithm is very robust in detecting contours in noisy regions. 'This algorithm turned out to be very important in achieving very high recognition rates.'

Dataset

Azzopardi: 'I am grateful to my three co-authors for their contributions. I am particularly grateful to Amey Bhole, a former Master's student under my supervision, who collected data, implemented our approach, and executed the experiments.' At the time of the study, existing benchmark datasets were very small and limited. Therefore, arrangements were made with the Dutch Dairy Campus in Leeuwarden to collect a data set over the course of nine working days. 'Amey first identified a suitable location to attach the camera with a good view of the cows when they were exiting a milking station and then spent nine days collecting image data from the same fixed location. In total, he acquired 3,694 paired thermal/colour pictures of 383 different Holstein cows.' The milking station was selected because it provided a realistic scenario where cow identification is very important in order to assign the correct cow IDs to the data collected from the milking robot (for example milk volume, proteins, or fats).

Promising

'The full pipeline includes a pre-processing step that removes any occluding rods and a convolutional neural network as a classification model.' By means of systematic experiments, they demonstrated that their proposed approach achieved a recognition rate of 99.7%. 'This shows that our approach is very promising,' says Azzopardi. The only cows that were not recognized were Holstein cows that had a uniform black or white coat. In their work, the scientists speculate that such a challenge could be addressed by including other cameras that can capture different views of the cow (e.g. face, top, and rear) and then developing a fusion approach that takes input from all the views. A system with such a multicamera setup could also be effective in recognizing the difficult cases. The entire implementation and data set have been made available in a public repository.

Azzopardi explains that there are several advantages in using imaging instead of RFID tags or collars to identify cows. 'First, it is non-invasive and, thus, better for the welfare of the animals. Second, it is cheaper, as farmers don't have to buy and manage collars or tags. And third, it is functionally and financially scalable: if the number of cattle on a farm increases, there is no need for extra collars. You would only need a few images of the new cows and use them to update the parameters of the respective classification model.'