Species Classification in Thermal Imaging Videos

A component of COMPSCI760: Datamining and Machine Learning

This semester I proposed the topic of Species Classification in Thermal Imaging Videos, and this was selected to be one of the group projects. The objective of this project was to improve the machine learning models used to identify species in thermal imaging videos, which can be used to automate monitoring of remote sites, and perhaps ultimately contribute to the development of smart traps which can improve pest and predator control.

We were fortunate to have The Cacophony Project provide us with a data set of videos they have collected from several locations in New Zealand.

Significant credit must be given to Hamish Duncanson. Between us, we contributed the vast majority of the research and writing for this paper.

I have embedded our final report in the page below. Alternatively, you can download a pdf copy of this paper here.

Abstract

We examine a labelled dataset of thermal imaging recordings of animals in New Zealand bush, for the purpose of developing models which can automate the labelling of species. After approaching the problem with generic video classification models, we propose two novel contributions:

This dataset differs from conventional video classification benchmarks, in that the subject of the recording (the animal) generally only occupies a small portion of the field of view. We address this by cropping each individual frame around the animal, and extracting the movement of the cropped region to retain information that would otherwise be lost in the cropping procedure. We show that this cropping can be implemented with a simple temperature threshold applied to the thermal images, requiring no additional expert information.
We adapt well known image augmentation techniques so that they are suited for the domain of video classification, and investigate which combination of transformations is most performant.

Experiments on the provided dataset support the efficacy of these contributions. We present a Inflated 3-D Convolutional Network (I3D), which combined with these techniques, can correctly label the species of over 90% of the thermal imaging recordings.

9^th of December, 2020.