BugTracker: Software for Tracking and Measuring Arthropod Activity

Abstract

The automated video tracking of the activity/movement of an experimental organism is essential for reliable, repeatable quantitative analyses in behavioral ecology and also in other disciplines. There are only some open-access, open-source automated tracking software applications that can track unmarked organisms. Moreover, several of these software applications are substantially affected by brightness and differences in the lighting conditions of the video recording. Our Python-based software, called BugTracker, uses the latest innovations in computer vision technologies to solve these problems. By analyzing videos with considerably different lighting conditions with BugTracker and other available software, we demonstrate that our software could reliably track the studied organisms of any size and speed. Additionally, the results provide accurate measures of the organism’s movements. BugTracker is the most reliable currently available, easy-to-use, and automated tracking software compatible with the Windows, Linux, and MacOS operating systems.

1. Introduction

Global anthropogenic activities, such as agriculture, forestry, and urbanization, are leading to significant changes in environments and creating patchworks of modified land types [1]. Alterations in environmental parameters trigger pressures on organisms living there, modifying their life history (phenology, body condition, and productivity) and behavioral parameters (activity, exploratory, and boldness) [2,3]. Changes in behavior are possibly the fastest way to react to unfavorable changes in environmental conditions; thus, behavioral studies are nowadays widespread [4]. In many disciplines, including behavioral ecology, ethology, and evolutionary ecology, a key question concerns how to quantify the activity/movement of the studied organisms. However, movement is also a crucial parameter, and a proxy is frequently used to evaluate the effect of human-associated stressors, chemicals, and pharmaceuticals [5]. Therefore, precise and accurate methods for measuring movements using noninvasive techniques are highly requested [6]. One common way to study the movement of arthropods is with artificial laboratory test arenas using video recordings over short periods (minutes) [7].

The availability of video-making products and powerful personal computers that are capable of complex, frame-by-frame analyses is making it increasingly feasible to record and measure animal movements [8]. Thus, even with a small investment or by using a cell phone, we can make a video recording of their movements. The fast transfer of data enables video recordings to be sent directly to a computer. Thus, we can even process these data in real time, or we can automate the processes and perform multiple analyses simultaneously with high precision and accuracy.

Automated monitoring systems provide a solution to either detect behaviors that occur only for a short period and then blend into each other or involve long periods of inactivity or possibly require continuous attention [9]. Reducing human resources can minimalize calculation and processing errors and promote task parallelization. Today, several software applications provide solutions for modeling and processing ecological problems and tasks. Although most of these pieces of software are parts of a prewritten standard process, there are always experiment-specific steps that pose a challenge to existing ones, possibly requiring the modification of the existing code in ways that are not supported by the code that has already been written and would use incompatible algorithms. Most of them are well suited to particular arenas and animals but are not capable of tracking arthropods of any species, shape, or size, regardless of the arena shape. Also, some software is proprietary, which means its creator prohibits users from freely sharing or modifying it, and its usage usually costs money. Based on these aforementioned issues, the prospect is given to open-source programs created with modern coding tools that enable the easy and quick development of solutions for a given task.

There are software applications that are capable of tracking one or multiple animals [10,11,12,13]. However, they use different methods for tracking than what we provide in this study. They detect an object based on contrast, basically segmenting focal objects from the background. Because of this, the examined area should be substantially brighter than the trackable animal. So, these programs are substantially affected by brightness and differences in lighting conditions in the area. Here, we propose an open-source tracking software, BugTracker, which uses a more robust and complex algorithm for tracking than the threshold- or contrast-based alternatives. Below, we summarize the functions of the software, offer examples of its applications, compare its characteristics and performance to other tracking software, and discuss its strengths and limitations.

2. Materials and Methods

2.1. Software

BugTracker was created to use a newer algorithm for tracking than existing solutions and to be optimized for testing a specified behavioral parameter, which can be used to facilitate the measurement and evaluation process and increase repeatability. It uses the CSRT (discriminative correlation filter with channel and spatial reliability) algorithm; thus, it is less affected by brightness and differences in the lighting conditions in the area.

BugTracker is a Python-based program that uses the open-source OpenCV software library (https://opencv.org, accessed on 11 April 2020) for computer vision algorithms to identify shapes and track objects. Python is a simple and easy-to-understand programming language for nontechnical users, and it is very popular in science [14,15]. Because of the increasing popularity and usage of the R programming language among ecologists, we also intend to create an R version of the software.

The goal of the OpenCV software library is to provide tools for solving computer vision problems. This includes a large variety of algorithms both for low-level image processing, like object recognition and tracking functions, and high-level algorithms, such as face recognition, pedestrian detection, feature matching, and tracking [16].

The software is designed to run multiple instances at once, but there is no GUI (graphical user interface) available yet for the configuration. It is easy to install and configure, even for nontechnical users. It does not require any programming knowledge to use it. BugTracker is available under GNU General Public License v3.0, with an installation and usage guide accessible on GitHub: https://github.com/Roffagalaxis/Bugtracker (accessed on 29 March 2023). We chose GitHub because of its popularity within and outside of academia, which increases the probability of it being discovered by people with programming knowledge. Making your research publicly accessible in this way leads to higher citation counts and a larger impact [17]. If the complete research project is hosted on a free-to-use site like GitHub, it becomes more easily reproducible and transparent, as how the code and data changed during the journey from idea to publication can be reviewed. With a public repository, anyone can retrieve the previous status of the research at any time [18].

2.2. Defining the Testing Area

The first step in the usage of the BugTracker software is to identify the analyzed area. We can define a square area manually, or we can use automatic detection, which works with any shape if the area has a visible barrier.

Manual setting uses the following codes:

• initarea = cv2.selectROI(framename, image, fromCenter = False, showCrosshair = True)
• for i in range (0, area [2], step_ver):
• cv2.line(image, (area [0] + i, area [1]), (area [0] + i, area [3] + area [1]), (0, 255, 0), 1)
• for i in range (0, area [3], step_hor):
• cv2.line(image, (area [0], area [1] + i), (area [2] + area [0], area [1] + i), (0, 255, 0), 1)

Algorithm for manual setting:

1.

Stop the video and evoke the mouse control for the user to select the area.

2.

Draw the vertical lines of the squares.

3.

Draw the horizontal lines of the squares.

4.

The user can later adjust these lines with the W, A, S, and D keys on the keyboard at any time while the video is running.

The automatic mode searches for object contours based on border following algorithms [19]. A border following algorithm is a core technique for processing digital images. It derives a series of coordinates from the boundary between an I-pixel and a connected component of an O-pixel (i.e., background). These techniques have been thoroughly studied because of their many uses in computer vision [20,21,22,23,24,25,26,27].

The automatic area detector method uses the following codes for detecting the test arena and squares:

• gray = cv2.cvtColor (image, cv2.COLOR_BGR2GRAY)
• blur_gray = cv2.GaussianBlur (gray, (5, 5), 0)
• edges = cv2.Canny (blur_gray, 50, 150)
• contours, hierarchy = cv2.findContours (edges, cv2.RETR_EXTERNAL, cv2.CHAIN_A-PROX_NONE)

Algorithm for automatic area detection:

1.

Convert image to grayscale format.

2.

Apply binary thresholding.

3.

Find contours based on the border following algorithms.

Using various videos, we examined the effect of light conditions on the automatic area detection efficiency for open field tests. We concluded that the ideal conditions for recording video are cold light from a vertical 90° angle or using a lightbox in order to create as little shadow as possible around the studied animal (Figure 1).

2.3. Tracking the Defined Test Organism

After defining the test area, we need to focus on short-term and model-free object tracking, which is an open and actively studied problem [28,29,30]. Localizing a target in a continuously moving video requires significant computer resources. The results also depend on various factors like distinguishability from the background, lighting conditions, presence of shadows, fast object movement, disappearances, or deformations [31,32,33,34].

Currently available software solutions mostly use two types of detection techniques: static and dynamic background subtraction. The static one uses a reference frame or video, where only the background is visible without any animals. The dynamic technique works with moving or decaying average models, where detection counts on illumination or other gradual changes to the frame [35].

OpenCV includes the following built-in tracking algorithms:

• cv2.TrackerCSRT_create;
• cv2.TrackerKCF_create;
• cv2.TrackerMIL_create;
• cv2.legacy.TrackerBoosting_create;
• cv2.legacy.TrackerTLD_create;
• cv2.legacy.TrackerMedianFlow_create;
• cv2.legacy.TrackerMOSSE_create.

Based on a preliminary test, the built-in CSRT (discriminative correlation filter with channel and spatial reliability) tracker works with the best accuracy. The CSRT algorithm uses a spatial reliability map to find the examined object that is most suitable for tracking. The spatial reliability map is estimated using a graph labeling algorithm. This grants a significantly wider search region, the possibility to use complex backgrounds, and improved nonrectangular object tracking [36,37].

It was also chosen because it handles special cases like object crosses or disappearances, and there is no limit to the size of the examined individual, so it is also suitable for tracking the movements of both small and large animals.

The main functions of any of the above trackers are:

• initTracker = cv2.selectROI (showedframe, image, fromCenter = False, showCrosshair = True)
• tracker.init (sourceframe, initTracker)
• (success, box) = tracker.update (sourceframe)

Algorithm to call tracking:

1.

Stop the video and evoke the mouse control for the user to select the outline of the object they intend to track.

2.

Start the tracking based on these boundaries.

3.

Each frame looks for the object, and it reports back whether it finds it and the outline where it was found.

The user sets the outline of an animal to track, and the program will continue the tracking until the input video is finished.

2.4. Examples of the Software’s Applications

BugTracker was optimized on videos created by Magura and his colleagues. They collected rove beetles along a rural–urban gradient, in and near Debrecen city (Eastern Hungary). Eight forest stands were selected along the gradient: four rural sites, and four in urban areas. The rural sites were located in a large forest of 1082 hectares in old (>130 years) forest stands, where English oak (Quercus robur) was the dominant species. All urban sites were isolated fragments of a once-continuous old forest (>130 years) dominated by English oak (Quercus robur). The rove beetles were sampled during their main activity periods using live pitfall traps. The traps were checked twice a week, from early April to late June. The collected beetles were transported to the laboratory, where standard conditions (24 °C, 40% relative humidity, and natural L:D cycle) were provided. In the laboratory, the beetles were left to rest with access to water but no food [38]. After the resting period, the activity of the beetles was individually tested in a novel environment, also called an open field test [39,40]. In the test arena, each beetle was covered with a Petri dish, and when it stopped moving, the Petri dish was lifted, and the movement of the beetle was recorded for the next 90 s with a GoPro HERO6 camera (CHDHX-601-FW) [38].

The current version of the BugTracker software was designed to automate the processing of this test in the best possible way, as this test is often used to evaluate exploratory behavior [41,42,43]. In the basic videos, the software worked with a preset test environment, which consisted of an open box (364 × 230 mm) whose bottom was divided into 35 equal-sized squares [38]. The software was created in such a way that the grid lines could be added to the grids already placed at the bottom of the box, but this part can be omitted using the software, as it can create its grid structure during the session.

2.5. Software Output

The current version of the software provides basic information for the user, including a graph of the trajectory of the bug’s movement and the number of visited cells of the grid. After the video evaluation, we can print both the graph of the trajectory and the number of visited cells (see Figure 1). The number of visited cells is a vital parameter during an ecological study [38].

2.6. Comparing the Properties of BugTracker with Other Similar Software

We compared the properties of our software to the other, currently available alternatives to highlight the differences. The main properties of the programs are summarized in

Table 1. Main properties of the programs used to compare their performances.

Software	Environment	Interface	Open Source	Method	Visualization Process	Object Crosses Handling
BugTracker	Python, OpenCV	Terminal-based/Command-based	Yes	CSRT	Yes	Yes, by tracker
Tracktor [10]	Python, OpenCV	Command-based	Yes	Dynamic background subtraction	Yes	No
ToxTrac [11]	C++, OpenCV	GUI	No	Dynamic background subtraction	No	Yes, by matching similarities of the trajectory
Pathtrackr [13]	R	Command-based	Yes	Static background subtraction	Yes	No

Environment—programming language and library used; Interface—type of interactive user interface; Methods—computer vision technology used in the software; Visualization process—whether the user can follow the tracking process with the software; Object crosses handling—whether the software can solve crosses of movement paths or disappearances.

.

2.7. Video Quality

For the comparison, we used video recordings with the following lighting and camera stability parameters: (1) video recording with ideal lighting conditions (next to the ceiling lamps, an additional external lighting source directed towards the test arena) and a fixed camera (using a tripod); (2) video recording with moderate lighting conditions (using only ceiling lamps without an additional external lighting source) and a fixed camera (using a tripod); and (3) video recording with moderate lighting conditions (using only ceiling lamps without an additional external lighting source) and a nonfixed, hand-held camera.

The outline of the insect was set after the Petri dish was lifted during the set-up of ToxTrax and Bugtracker for the best possible results. There is no similar setting in Tracktor and Pathtrackr.

3. Results

We compared the performance of our software to the currently available alternatives using the same videos with the various lighting and camera stability conditions. Based on our testing, none of the example video recordings could be calibrated well using Tracktor, so the software did not produce accurate tracking results for the videos. After proper calibration, ToxTrac worked precisely on the video recordings with the ideal and moderate lighting conditions and a fixed camera, but it failed on the hand-held video recording. Pathtracker was able to track the animal with the ideal lighting conditions with a fixed camera, but it could not continuously track the animal’s movements under moderate lighting conditions with a fixed camera. In addition, the video recording with a hand-held camera under moderate lighting conditions could not be processed by this program at all. Our software, BugTracker, was able to precisely track the target animals during the whole video recordings with various lighting and camera stability conditions (Figure 2). The number of visited squares, a recognized measure of activity and exploration in arthropods [38], counted by the four animal-tracking software applications were also markedly different for the video recordings under suboptimal parameters (i.e., moderate lighting conditions and/or a nonfixed, hand-held camera;

Table 2. The number of visited squares counted manually and with the four animal-tracking software applications using video recordings with different lighting and camera stability conditions. The values highlighted in red indicate that the application was unable to track the arthropod’s movement. The values marked in orange indicate incorrect results, while the values highlighted in green indicate correct results for the visited square numbers.

Software	Video with Ideal Lighting Conditions and a Fixed Camera Recording the Movements of Pardosa alacris (Lycosidae)	Video without an External Light Source and a Fixed Camera Recording the Movements of Armadillidium vulgare (Armadillidiidae)	Video without an External Light Source and a Nonfixed, Moving, Hand-Held Camera Recording the Movements of Pardosa alacris (Lycosidae)
Manual counting	21	16	12
Tracktor [10]	0	0	0
ToxTrac [11]	21	17	7
Pathtrackr [13]	21	14	0
BugTracker	21	16	12

).

4. Discussion

Future Software Developments

An open-source software allows users and developers to use the knowledge of the software for their personalized experiments. The most important duty is to develop a proper GUI (graphical user interface) and an R version of the software to help the adaptation of the software among entomologists and other researchers.

The length of the trajectory, average acceleration, and speed are also useful information for researchers analyzing animal movements. We would like to include the measurement of these parameters in the software. Tracking multiple individuals at the same time in the same arena is also planned.

An improved version of our software can even be suitable for processing UAS camera recordings and for tracking location changes and behavioral patterns recorded on aerial photographs. Nowadays, the use of UAS in scientific research is increasing rapidly, as it can provide researchers with a completely new perspective.

Since the CSRT tracker does not need a constant background, it is not only suitable for tracking the movement of living beings but could be used for tracking vehicles. An advanced version of our software could also be suitable for following a live recording. Implementing this feature also means that we could track the movement of a species like Silpha tristis (Silphidae) for weeks in a closed terrarium. Even if the beetle hid somewhere, the tracker could continue tracking it when it came out of hiding. From these data, we could determine when and how much distance it moves during a day or week. We could examine whether it moves less when it becomes older.

5. Conclusions

With the use of the BugTracker software, human resources can be optimized during open field experiments. It can reduce calculation and processing errors caused by human factors and promote task parallelization. During studies, arthropods are not limited by species, shape, or size either; the movements of Acinophus ammophilus (Carabidae), which is a large beetle (20–28 mm), or even small ones, such as Armadillidium vulgare (Armadillidiidae) with a 0.7–18 mm length, can be monitored effectively. We armed the software with the recent tracking algorithms, which provide better results under unfavorable conditions than the existing solutions and are optimized for testing a specified behavioral parameter, which can facilitate the evaluation. Since the code for this software is open and available on GitHub, it is easily accessible and can be used by anyone in the Python environment. Its development is quite broad because of the use of the Python programming language, as it is simple and understandable, even for novice programmers, and later it can also be properly integrated into R, thus making it even more widely available to users.