Can Transfer Learning Overcome the Challenge of Identifying Lemming Species in Images Taken in the near Infrared Spectrum? ^†

Abstract

Using a camera system developed earlier for monitoring the behavior of lemmings under the snow, we are now able to record a large number of short image sequences from this rodent which plays a central role in the Arctic food web. Identifying lemming species in these images manually is wearisome and time-consuming. To perform this task, we present a deep neural network which has several million parameters to configure. Training a network of such an immense size with conventional methods requires a huge amount of data but a sufficiently large labeled dataset of lemming images is currently lacking. Another challenge is that images are obtained in darkness in the near infrared spectrum, causing the loss of some image texture information. We investigate whether these challenges can be tackled by a transfer learning approach in which a network is pretrained on a dataset of visible spectrum images that does not include lemmings. We believe this work provides a basis for moving toward developing intelligent software programs that can facilitate the analysis of videos by biologists.

1. Introduction

Lemmings are small mammals native to the arctic and subarctic zones, where they remain active throughout the long winter under the snow [1]. Their dramatic population fluctuations [2] affect the tundra food web, as several predators are dependent on them for survival [3]. Despite the crucial role of winter reproduction in these population dynamics, our understanding of subnivean animals during this long season remains very limited due to the difficulty of studying them in harsh and remote winter conditions of the Arctic. These animals are often studied using methods such as live trapping and post-winter nest surveys in summer due to significant logistical and financial constraints of direct winter studies. Camera trapping offers a promising alternative for continuous winter monitoring, yet its application in the Arctic, particularly for subnivean animals, is rare [4,5,6].

In earlier studies [5,7], we developed an autonomous camera system to monitor lemming activity under the snow. This system has captured thousands of short videos of lemming activity. However, manually analyzing this growing dataset is labor-intensive and time-consuming. One valuable application of this system is to estimate relative lemming abundance from recorded videos, which requires identifying species in the images.

Over the past decade, Convolutional Neural Networks (CNNs), has emerged as a powerful alternative to traditional image classification methods that rely heavily on manual feature engineering (e.g., [8]). In contrast, CNNs can automatically extract features directly from raw data, often achieving higher accuracy and better generalization (e.g., see [9]).

Various CNN architectures, such as AlexNet, VGGNet, and InceptionNet, have been developed for image classification (see [10] for a review). Despite outperforming traditional methods, CNNs typically require substantial computational resources and large labeled datasets, limiting their applicability to real-world problems like species identification in our dataset. Transfer learning offers a practical solution to this challenge. In this approach, a CNN pretrained on a large dataset like ImageNet is used as a base model. All layers are frozen, except the last few, depending on the size of the custom dataset, and only the unfrozen layers are retrained. The core idea is that low-level visual features (e.g., edges, textures) are generally transferable from one image classification task to another.

To the best of our knowledge, no existing model addresses lemming species identification, and most prior studies have focused on classifying large animals using high-quality images captured under favorable lighting conditions. In contrast, this study investigates whether a transfer learning approach can effectively tackle the challenge of identifying lemming species from low-quality images taken in darkness using near-infrared imaging. Among the top-performing architectures for image classification, ResNet has gained widespread popularity due to its robust performance [11]. Accordingly, we adopt a pretrained ResNet model and adapt it for our dataset.

2. Data and Method

We built a dataset from videos recorded by several units of a custom camera system [5] deployed on Bylot Island in the Canadian Arctic to monitor subnivean animal behavior between August 2018 to June 2020. Non-informative and low-quality images (previously extracted from videos featuring animals) were removed. Two lemming species were identified: brown lemming and collared lemming. Since collared lemmings turn white in winter, they were labeled separately as gray collared lemming and white collared lemming. The final dataset included 1909 images of brown lemming, 2466 images of gray collared lemming, and 387 images of white collared lemming (Figure 1). Each animal was also annotated with a bounding box using our custom software to define the region of interest for cropping. All programming was performed in Python, 3.10.13, using libraries such as TensorFlow, and run on a computer equipped with an NVIDIA GeForce RTX 3080 GPU (NVIDIA, Santa Clara, CA, USA).

To build a lemming species classifier, we adapted the ResNet50 architecture by replacing its final layer with a three-unit dense layer to match the number of classes. Instead of training the entire network, we applied transfer learning by fine-tuning a ResNet50 model pretrained on ImageNet while freezing all layers except the last (Figure 2). It was trained for 10 epochs using the Adam optimizer (learning rate = 0.001) and cross-entropy loss.

Model performance was evaluated using two variations of k-fold cross-validation: Stratified K-Fold (SKFCV) and Group K-Fold (GKFCV). While SKFCV preserves class balance across folds, SGKFCV ensures that groups do not overlap. Training was conducted on both the original and cropped datasets (DS1 and DS2), resulting in four model variants.

3. Results

The average accuracy across folds for the four variations of the model (M1 to M4) was 99%, 99%, 85%, 94%, respectively. The model performance in identifying each species is presented in

Table 1. Confusion matrices for four model variants (M1, M2, M3, and M4) trained on datasets DS1 and DS2 using Stratified K-Fold Cross-Validation (SKFCV) and Group K-Fold Cross-Validation (GKFCV). Each matrix shows predicted classes (columns) vs. true classes (rows). C1, C2, and C3: brown lemming, gray collared lemming, and white collared lemming, respectively.

		Predicted Class
		M1 (DS1 & SKFCV)			M2 (DS2 & GKFCV)			M3 (DS1 & SKFCV)			M4 (DS2 & GKFCV)
		C1	C2	C3	C1	C2	C3	C1	C2	C3	C1	C2	C3
True Class	C1	0.998	0.001	0.001	0.991	0.007	0.002	0.851	0.147	0.002	0.952	0.045	0.003
	C2	0.016	0.983	0.001	0.009	0.988	0.003	0.179	0.816	0.005	0.047	0.948	0.005
	C3	0	0	1	0	0	1	0.007	0.003	0.990	0	0.008	0.992

. Models M1 and M2 performed unexpectedly well. We suspect that data leakage between the training and validation sets may have artificially inflated the results. The performance decline observed for models M3 and M4, where GKFCV was used to ensure that images from the same sequence did not appear in both training and validation sets, supports this hypothesis. A comparison between results of M3 and M4 indicates that training on cropped images can significantly enhance model performance, likely by minimizing the influence of features from irrelevant objects such as background vegetation, and promoting focus on features related to the animals themselves.

4. Discussion and Future Work

Our results support the suitability of transfer learning for our image classification task. We demonstrated that features extracted by a ResNet architecture pretrained on a large dataset of generic object categories in the visible spectrum can be effectively repurposed. Specifically, these features proved useful for classifying new categories that were not part of the original training data and were captured in the near-infrared spectrum, where color and some texture information are lost. Several directions can be explored in future work: (1) comparing other architectures (e.g., VGGNet, InceptionNet) on our dataset; (2) expanding the dataset using recordings from our new system [6]; and (3) evaluating transfer learning on tasks like animal localization.