Towards Automatic Bird Detection: An Annotated and Segmented Acoustic Dataset of Seven Picidae Species

Abstract

Analysing behavioural patterns of bird species in a certain region enables researchers to recognize forthcoming changes in environment, ecology, and population. Ornithologists spend many hours observing and recording birds in their natural habitat to compare different audio samples and extract valuable insights. This manual process is typically undertaken by highly-experienced birders that identify every species and its associated type of sound. In recent years, some public repositories hosting labelled acoustic samples from different bird species have emerged, which has resulted in appealing datasets that computer scientists can use to test the accuracy of their machine learning algorithms and assist ornithologists in the time-consuming process of analyzing audio data. Current limitations in the performance of these algorithms come from the fact that the acoustic samples of these datasets combine fragments with only environmental noise and fragments with the bird sound (i.e., the computer confuses environmental sound with the bird sound). Therefore, the purpose of this paper is to release a dataset lasting more than 4984 s that contains differentiated samples of (1) bird sounds and (2) environmental sounds. This data descriptor releases the processed audio samples—originally obtained from the Xeno-Canto repository—from the known seven families of the Picidae species inhabiting the Iberian Peninsula that are good indicators of the habitat quality and have significant value from the environment conservation point of view.

Data Set: https://doi.org/10.5281/zenodo.574438

Data Set License: CC BY-SA 4.0

1. Introduction

Coordination and organization in any kingdom of life require some kind of communication. The most widespread ways to communicate between animal species and their related species rely on visible (i.e., changing the body position) and/or vocal (i.e., sequences of sounds) expression patterns [1]. Ranging from insects to mammals, including amphibians and songbirds, they all generate complex acoustic sequences [2] associated to different life events (e.g., mate attraction and selection, predator alerts, or food availability). In recent years, animal communication [1] has become a hot research topic since it enables researchers to obtain valuable information about the species population, their evolution, and the environments in which they typically inhabit [3]. This is especially interesting in the domain of bird species since they use a complex acoustic communication—they distinguish songs from calls and include non-vocal sounds such as the drumming of woodpeckers and the “winnowing” of snipes’ wings in display flight [4]—which is known to be rich and meaningful [3,5]. Therefore, this work revolves around the analysis of this kind of acoustic sounds produced by birds.

One of the most time-consuming tasks that ornithologists and birders typically conduct is to record, annotate, and differentiate species in their natural habitat. Usually, this process requires the intervention of a human expert. That is, a specialist from the specific field of the species to be analyzed is asked to identify and accurately classify any individuals that live in a bio-diverse environment. Alternatively, some researchers from other areas have attempted to address this classification challenge by using signal processing techniques [6] or taking advantage of the latest advances in computer science and artificial intelligence to build learning classifier systems able to automatically identify each animal [7,8] with reasonable success. The accuracy of these automatic systems typically relies on (1) the acoustic properties of the animal sound; (2) the recording quality of the dataset; (3) the features selection extracted from each audio fragment; (4) the number of collected samples; and (5) the classification algorithm. Overall, the success of these computer-assisted strategies relies on the dataset used to train and test the classifier. For instance, samples recorded with background noise (Figure 1a), may negatively affect the accuracy of automatic classifiers.

Figure 1 emphasises this fact by depicting two t-Distributed Stochastic Neighbor Embedding (t-SNE) [10] plots (i.e., a space transformation that projects points from different classes in a bi-dimensional plane) of the same dataset. One with background noise (Figure 1a) and the other with the background noise removed (Figure 1b), which gives a visual perception of the class separability and the complexity of the classification problem. It can be seen that classes on the left plot are very overlapped while this effect is mitigated on the right plot. This means that if the dataset corresponding to the left plot were delivered to a learning classifier system, its performance would probably be very limited due to the fact that the system may confuse environmental noise with the actual bird sound. This stresses the need for a clean and reliable bird sound repository that researchers can safely use to train and tune machine learning systems.

So far, there exist several excellent databases containing general purpose recordings of birds and animal sounds. On the one hand, there are projects such as Xeno-Canto [11,12], Cornell’s Macaulay Library [13], the Western Soundscape Archive [14] and Avibase [15], which usually share general sound recordings of animals and soundscapes of their habitat. On the other hand, there are repositories such as the British Library [16], the Berlin Museum fur Naturkunde [17] or the Australian National Collection Sound Archive [18] that include metadata about the recordings (location, ecology, etc). However, these generic purpose databases suffer from two issues when used for automatic identification and classification of a closed set of bird species in their natural habitat: (1) label reliability and (2) environmental noise (also referred to as silence). First, these audio samples usually lack accurate and reliable annotations about the song types, which requires an expert to supervise the labelling process. Second, almost none of these repositories separates the silences (i.e., periods of time where the bird under interest is not emitting any sound but there might be environmental noise such as waterfalls or other species) between periods of bird sound, which also requires a further annotation in order to boost the classifier performance (see Figure 1). Additionally, the accuracy of automatic birdsong recognition and classification systems could be further improved if applied to taxonomic bird groups with songs of limited complexity, especially when these groups are good indicators of the habitat quality or its conservation status. For instance, this would be the case of woodpeckers (belonging to the Picidae family), which have simpler sound emissions and have significant value from the conservation point of view [19]—although some of their species are included in threat categories [20], they are still especially relevant for their role as indicators of quality and maturity of forest environments [19,21].

Therefore, the purpose of this paper is to present an annotated dataset of Picidae bird sounds lasting 4984 s in which bird sounds have been separated from the environmental noise. This dataset includes a collection of 1669 audio samples—derived from 161 audio files collected from the Xeno-Canto repository [11]—from the known seven families of the Picidae species inhabiting the Iberian Peninsula emitting up to three different sounds: call, drumming or song. Although the number of families included in this dataset is considerably smaller than other general purpose collections [22], the species herein incorporated have been specifically selected and neatly chosen according to their relevance for the habitat conservation and biodiversity quality domain [19,21]. Hence, researchers using this dataset might be able to train an automatic Learning Classifier System to (1) detect whether there is any Picidae bird in a given sound sample collected from a natural environment; (2) identify the type of Picidae bird; and (3) recognize the type of song. Automatic bird classification has emerged as a hot research topic in recent years. For instance, modern classification techniques based on deep neural networks and data augmentation have enabled researchers to reach an interesting accuracy (close to 60%) from a dataset of 33,203 audio recordings of 999 species collected in South America at the LifeCLEF Bird Task [22]. In comparison to the LifeCLEF Bird Task dataset (it has an average of 24 samples for each species) that covers a plethora of bird species, our proposal can be best seen as a reduced approach (it has an average of 96 samples for each species) targeted at analyzing a specific set of bird species with great biological interest [19,20,21].

The remainder of this paper is organized as follows. Section 2 describes the proposed dataset in terms of geographical distribution, number of files, duration, and spectral evolution over time. Section 3 details the segmentation and annotation process followed to obtain the proposed dataset. Section 4 specifies the files structure of the dataset and the instructions to download it. In Section 5, authors outline the conclusions of this work and present future research directions. Additionally, all the source audio files used from the Xeno-Canto project together with the names of the recorders are provided in the Acknowledgements section.

2. Dataset Description

The proposed Picidae annotated database has been built from 161 recordings obtained from the Xeno-Canto Project [11,12] (the detail and the names of the recorders for each original recording are provided in the Acknowledgements section of this paper). The geographic distribution of the original recordings, the composition of the resulting dataset, and the spectral analysis of the birdsongs are detailed in what follows.

Figure 2 depicts a map containing the location of every original audio from the Xeno-Canto [11] repository where each bird color represents a different Picidae species and a different type of sound (call, drumming or song). In those situations where different types of sound for the same species are available (e.g., Dendrocopos leucotos-call, Dendrocopos leucotos-drumming), each sound has been drawn with a different color to further improve the detail of the obtained map. As a result, this map provides a general overview of the geographical distribution of the collected samples that compose the dataset presented in this paper. It can be seen that although most of the recordings were taken in central Europe, some of them were registered in Africa and Asia.

After the acoustic processing herein presented, these 161 recordings resulted in 1669 labeled files. As a result, the corpus contains three different sound types (call, drumming, and song) from the following Picidae species: Jynx Torquilla, Picus viridis, Dryocopus martius, Dendrocopos major, Dendrocopos medius, Dendrocopos leucotos and Dendrocopos minor. The list of the species (i.e., species name), the number of acoustic samples (i.e., file count), the overall length of all samples (i.e., total length), and the duration distribution of all the files that compose the proposed dataset are shown in

Table 1. Files obtained after the segmentation and annotation processes. Proposed dataset composition.

. Note that the selected colors are consistent with the ones used in Figure 2. It can be seen that the longest individual sound is made by Dendrocopos medius, and corresponds to its song. Drumming in all species is usually a short and uniform sound, with a length ranging from 1 to 3 s. The songs of the species are also uniform in terms of duration, ranging from 1 to 8 s with median values around 3 s. Also, call signals are very variable in terms of length: on the one hand, Dendrocopos minor and Dendrocopos medius show high variability, with long samples, despite the median being around 2 s. On the other hand, Dendrocopos leucotos, Dryocopus martius and Dendrocopos major show calls from maximum 3 s.

Additionally, a 1 s length sample spectrogram for every different type of sound contained in the dataset is shown in Figure 3. The first row shows the spectrograms of the song from three species: the Picus viridis, the Jynx torquilla and the Dendrocopos medius. Although the shape of the spectro-temporal representation of the signal is similar in the three cases with harmonics, the Picus viridis has five repetitions, the Jynx torquilla has four and the Dendrocopos medius has only two repetitions. The spectro-temporal representation of the call from five different species can be observed in Figure 3d–h. The longest call is shown in Figure 3h from the Dryocopus martius while the other four are considerably shorter. Figure 3e depicts a single short call (Dendrocopos leucotos) while Figure 3d (Dendrocopos medius), Figure 3f (Dendrocopos minor) and Figure 3g (Dendrocopos major) show three to four repetitions of a smaller period call. Finally, the spectro-temporal representation of the drumming—despite it being non-vocal, it is also considered a birdsong—from four species is shown in Figure 3i–l. The drumming shows several wide-spectrum temporal repetitions for all the species. The following section elaborates on the process to segment and annotate the proposed dataset.

3. Segmentation and Annotation Process

Figure 4 illustrates the segmentation and annotation process that has been conducted to separate the background noise from the sound of interest (i.e., bird sounds) for each of the 161 audio samples downloaded from the Xeno-Canto repository. Segmenting and annotating a dataset resulted to be successful in similar works [22]. For the sake of this paper, this process has been decomposed into six steps detailed in the following.

Step I. 

Original audio sample acquisition. It consists of downloading the audio samples from the Xeno-Canto repository [11,12] and running an initial manual pre-filtering. That is, listening to the downloaded samples and ensuring that they correspond to the species to which they are labeled. Low quality and too noisy samples are discarded.

Step II. 

Energy threshold calculation. It consists of establishing an energy threshold to distinguish between a sample containing sound and a sample containing silence. It is computed by squaring every sample value of the whole original audio and averaging the result (i.e., average power of the audio recording).

Step III. 

Sample windowing. It consists of applying a 500 ms. sliding window with no overlap to the original audio sample (i.e., splitting the file in 500 ms. segments). We have found that 500 ms. is a convenient size because it captures enough acoustic energy of the bird sound to discern it from the background noise of the acoustic sample. A larger window size may increase the chance of capturing different bird songs in the same sample. Accordingly, a smaller window size may prevent the system from distinguishing background noise from the bird song.

Step IV. 

Local energy calculation. It consists of calculating the average power of the 500 ms. audio segment.

Step V. 

Energy comparison. It consists of deciding whether the 500 ms. audio segment is either sound or silence. If the result from Step IV is higher than the threshold established in Step III, the audio fragment is pre-labeled as a sample of interest (the bird is making some sound); otherwise, the fragment is labeled as background noise (i.e., silence).

Step VI. 

Pre-labeling. It consists of comparing the label obtained at Step V with the previous labels. If the label is the same (e.g., all the 500 ms. fragments so far have been labeled as silence), the system goes to Step III again with the following segment. If the label is different (e.g., the previous window was labeled as silence and the actual window was labeled as sound of interest), a new audio with all the previous same-label windows is generated and stored before going again to Step III with the next segment of the original audio sample. In the latter case, the segment will be included in the next sample collection. It is worth mentioning that acoustic samples—even from different 500-ms adjacent windows—that still contain subsequent repetitions from the same sound will be tied together as a single audio file at this step. This is done to capture the information of sound repetitions, which might be of great interest when identifying the bird species and its type of sound.

Step VII. 

Fine tuning and final annotation. It consists of conducting a manual review (i.e., removing false positives) and adjustment (i.e., moving background noises from actual sound samples to the Silence class) on the pre-labeled samples. For instance, there might be some audio samples that might introduce ambiguous information to the dataset and, thus, need to be deleted (e.g., a Jynx Torquilla and a Picus viridis singing at the same time and labeled as a Jynx Torquilla only). It is worth noting that the background noise (i.e., silence) segments might contain birdsongs from non-interesting bird species (i.e., birds that do not belong to any of the Picidae species that are being studied in this paper); the rationale behind this decision is to make sure that an automatic classifier will consider all the noises (even if they are birdsongs) that are not generated by the bird of interest as silence.

Overall, Figure 5 shows the spectrogram and temporal waveform from an original audio sample. The audio belongs to a sample where an exemplar of the species Dryocopus martius drums. The parts where the bird is drumming have been highlighted with a red bounding box. After running the proposed algorithm with this sample, three sub audios of sound of interest labeled as Dryocopus martius drumming and four sub audios labeled as silence would be generated.

4. Materials

All the materials are contained in a zip file that can be downloaded from https://doi.org/10.5281/zenodo.574438. Audio data are organized in thirteen folders: twelve containing the labeled audios of each birdsong type from every Picidae species listed in Table 1 and one folder containing the background sound samples. Every folder is named according to the following pattern: “Picidae species name—the type of sound” (e.g., JynxTorquilla-song). Analogously, all the audio files inside every folder have been named using the same strategy. More concretely, the structure of the audio file name is: Number of register of the audio at the Xeno-Canto repository—Name of the Picidae species—Name of the type of sound—Number of sub audio referenced to the original Xeno-Canto file (e.g., XC233286-JynxTorquilla-song-6.wav). For instance, if four sounds of interest were extracted from the original Xeno-Canto audio sample XCaaa, which was a recording from a bbb Picidae species doing a ccc type of sound, the generated audios will be the following: XCaaa-bbb-ccc-1.wav, XCaaa-bbb-ccc-2.wav, XCaaa-bbb-ccc-3.wav, and XCaaa-bbb-ccc-4.wav. Please, note that all the audio files have been converted to .wav. Finally, the zip file contains a Read Me document detailing the organization of the main folder.

5. Conclusions

The research conducted nowadays in automatic bird classification requires an extensive number of precisely labelled files of birdsongs recorded from real-life environments. Additionally, information about where the recordings were made, the type of sound and the animal species is very valuable. Unfortunately, annotating the pseudo-periodic songs, drumming or calls from each bird species, together with segmenting the silence recorded between samples is a delicate and challenging task. In this regard, we have constructed a segmented and annotated database of the seven Picidae bird species known to inhabit the Iberian Peninsula that enables such recordings to be shared for the community. We have also included the data structure to help researchers with the management and understanding of the dataset. Future work will be focused on increasing the information about all the files of the dataset, (e.g., the spectrogram for each file or embed the localization visualization).