Next Article in Journal
CellNet: A Lightweight Model towards Accurate LOC-Based High-Speed Cell Detection
Next Article in Special Issue
Quality Enhancement of MPEG-H 3DA Binaural Rendering Using a Spectral Compensation Technique
Previous Article in Journal
Enhanced Control of Three-Phase Grid-Connected Renewables with Fault Ride-Through Capability under Voltage Sags
Previous Article in Special Issue
Estimation of Azimuth and Elevation for Multiple Acoustic Sources Using Tetrahedral Microphone Arrays and Convolutional Neural Networks
 
 
Font Type:
Arial Georgia Verdana
Font Size:
Aa Aa Aa
Line Spacing:
Column Width:
Background:
Article

Acoustic Descriptors for Characterization of Musical Timbre Using the Fast Fourier Transform

Center of Mathematics, Computer Science and Cognition at Federal University of ABC, Av. Dos Estados 5001, Santo André 09210-580, SP, Brazil
*
Author to whom correspondence should be addressed.
Electronics 2022, 11(9), 1405; https://doi.org/10.3390/electronics11091405
Submission received: 4 April 2022 / Revised: 23 April 2022 / Accepted: 26 April 2022 / Published: 27 April 2022
(This article belongs to the Special Issue Applications of Audio and Acoustic Signal)

Abstract

:
The quantitative assessment of the musical timbre in an audio record is still an open-ended issue. Evaluating the musical timbre allows not only to establish precise musical parameters but also the recognition, classification of musical instruments, and assessment of the musical quality of a sound record. In this paper, we present a minimum set of dimensionless descriptors, motivated by musical acoustics, using the spectra obtained by the Fast Fourier Transform (FFT), which allows describing the timbre of wooden aerophones (Bassoon, Clarinet, Transverse Flute, and Oboe) using individual sound recordings of the musical tempered scale. We postulate that the proposed descriptors are sufficient to describe the timbral characteristics in the aerophones studied, allowing their recognition using the acoustic spectral signature. We believe that this approach can be further extended to use multidimensional unsupervised machine learning techniques, such as clustering, to obtain new insights into timbre characterization.

1. Introduction

The implementation of computational methods for the digitization of sound, through the Fast Fourier transform (FFT), has allowed important advances in Music Information Retrieval Systems [1,2], in the recognition and identification of Musical Instruments [3,4] and the characterization of musical audio records.
To characterize sound in general, and musical sound in particular, it is necessary to know the attributes of pitch, intensity, duration, and timbre. The first three characteristics of sound correspond respectively to the acoustic magnitudes of frequency, intensity, and time, and they are directly measurable quantities. Timbre, however, is a multi-dimensional and shallowly defined attribute that allows one to distinguish between different sounds even when they have the same intensity, duration, and tone; that is, it allows discriminating sounds of different musical instruments even when it is the same musical note, with the same duration and the same intensity. The timbre of a specific musical sound (with a fundamental frequency in the tempered musical scale) is mainly related to the attack, sustain, and decay lapses of that frequency, the tonal variation, and the harmonics present [5,6].
The problem of univocal characterization of musical timbre raises the need to elaborate descriptors (derived magnitudes, coefficients, or functional) that evaluate timbre from digital audio records [7]. This is important for problems such as Automatic Music Transcription (AMT) [8]. It has been identified that, for audio recordings, it is necessary to have efficient systems that identify the different musical timbres with high precision and quantitatively [9].
Since musical timbre is a phenomenon of auditory perception, many of the investigations are developed in the line of psychoacoustics, aiming to evaluate verbal descriptors that reveal measurable attributes of musical timbre [10,11,12,13], timbre attributes of musical sounds, and visual colors through subjective perception evaluation experiments [14]. Other, more recent studies focus on the analysis of timbre similarity between auditory images of various types of musical instruments and the linguistic-cognitive dimensions of timbres through a model where the timbre representation is contained in 20 dimensions [15,16].
Although the psychoacoustic perception of musical timbre cannot be ignored, it must be recognized that the main timbral characteristics must be somehow inscribed within the FFT that enables the recording and subsequent reproduction of musical sound. For the sake of argument, suppose any significant timbral characteristics that are not contained in the FFT performed on a musical audio record. Then, the deconvolved audio (inverse convolution) of the reproduced digital record could not be distinguished timbrally. However, this fact does not happen in musical digitization as we could distinguish timbral aspects from the deconvolved audio. Therefore, the FFT contains all the significant timbral characteristics.
Other research focuses on the presentation of an exhaustive collection of acoustic descriptors in the form of coefficients or functional (Timbre Toolbox, Librosa, etc.) that can be computationally extracted from the statistical analysis in the digitization of the spectrum (FFT). This focuses mainly on the statistical and mathematical characterization of the maximums in the FFT, such as the mean value in frequency (centroid) and amplitude, standard deviation, kurtosis, roots or poles of the distribution, arithmetic and geometric sequences in frequency, mean values and mean-squares of the amplitudes, among others [1,9,16,17,18,19,20,21]. Currently, there is no consensus on which and how many acoustic descriptors are to characterize musical timbre. However, it is recognized that many of them are derivatives or combinations of others and that in general they are correlated with each other. We wonder, then, what would be the minimum timbral descriptors that allow such a characterization of the musical timbre.
Peeters and collaborators [22] have studied the 34 most common statistical and mathematical descriptors and have found that only 9 descriptors are relatively independent with a correlation of 60%, and only 7 are so with a 50% correlation, which suggests that future studies on musical timbre should consider as a minimum: (1) a measure of the central tendency of spectral descriptors that vary in time; (2) a measure of the temporal spread of time-varying spectral descriptors; (3) a descriptor of the energy content of the sound signals and of the time envelope of the energy; (4) a periodicity descriptor. Peeters and collaborators [22] add: “It should be noted that this minimum requirement characterizes few or none of the previous studies on the perception of musical timbre”. Since we are studying only musical monophonic sounds of aerophones, we have been left with a minimum set of timbre descriptors from a single FFT analysis, excluding psychoacoustic and perceptual characteristics for quantification.
Despite recent advances using verbal (psychoacoustic) descriptors as statistical or mathematical descriptive analysis of musical timbre, this is still an open-ended problem. Taffeta et al. [23] state that most acoustic descriptors fail to examine all sound parameters that could be important when attempting to analyze timbral dimensionality in musical sounds and it constitutes a challenge for the automated identification of musical instruments and the quantitative evaluation of the “quality” of the sound in melodics, simple or compound fragments (polyphonic music).
An alternative approach is to construct the timbral descriptors from musical acoustics, using by analogy the analysis and digital processing of spectra in other areas of knowledge. FFT provides a representation in the two-dimensional plane of the frequencies and intensities (alternatively energy-frequency) present in a signal. Spectral analysis techniques consist of describing and interpreting the statistical distribution of the amplitudes for different frequencies. In that sense, it does not depend on the wavelength or frequency analyzed. Therefore, an infrared spectrum (Chemistry), visible (Astronomy), signals from ultrasound, Electroencephalograms and electrocardiograms (Medicine), seismic (Geology) and sonic (Ultrasound for failure analysis, or mechanical vibrations in acoustics and Engineering) are equivalent. From the physical point of view, and also from the mathematical description, an audio FFT spectrum does not differ from the spectra obtained in Mechanical Engineering for the analysis of vibrations, or from the infrared (IR) spectra in Chemistry for the rotational and vibrational oscillations of molecules.
The objective of this work is to propose a minimum set of acoustically motivated timbral descriptors (functional mathematical or dimensionless coefficients) that allow the computational extraction of timbral information from musical records using the FFT. To evaluate the effectiveness of these timbral descriptors, an empirical study of the timbral characteristics in digital musical sounds of a sample of wooden aerophones is presented: Clarinet, Bassoon, Transverse Flute, and Oboe.
This paper is organized as follows: the next section presents the methodology for obtaining the spectra and their quantification of the audio records. Then, the Timbral coefficients section shows the coefficients or descriptors and their evaluation for the selected aerophones. In the Spectral Signatures section, the chromatogram for the fourth octave of the tempered musical scale and the spectral signature (“fingerprint”) that characterizes each aerophone studied are evaluated. Finally, conclusions and future work directions are presented in the last section.

2. Methodology

The aerophone audio records were obtained from the TinySOL open-source sound library [24] which contains recordings of individual sounds, played ordinarily, at a level of dynamic/intensity mezzo-forte, in a WAV audio format that minimizes information losses, sampled at 44.1 kHz on a single channel (mono) at 16-bit depth. From this library, we have restricted our analysis to only some instruments of the woodwind family of a typical western symphonic music orchestra, these are the Transverse Flute (Fl), Oboe (Ob), Clarinet (Cl), and Bassoon (Bn).
The selection of the family of wooden aerophones is justified by having the greatest timbral diversity and more melodic possibilities for functioning as a solo instrument within a symphony orchestra but also presenting well-differentiated timbral characteristics when they function as a set of instruments [25].
Although Pons, Jordi, et al. [26] make the timbre analysis based on spectrograms, we propose in this work a simpler alternative way studying the FFTs since these must contain the same relevant information of the musical timbre, applied to monophonic sounds and considering the proposed set of instruments. This simpler alternative considers coefficients extracted from the full FFT spectra, instead of the spectrogram obtained using the short-time FFT.
To calculate the FFT and obtain the frequency spectra, the SciPy library module in Python [27] is used. Then, for the identification of the local maxima, the find_peaks function in Scipy.signal was used. Next, the tables of the maximum frequencies, expressed in Hz, with relative amplitudes, normalized considering the maximum amplitude value achieved in each spectrum of the FFT, were obtained to compare the various audio spectra with different power levels (Watt) or relative power (dB). Due to the FFT used, which operates on the entire function, information on size, overlap, etc. is not considered in this study.
The calculation of the coefficients was performed on the frequency spectrum of the common range of the instruments studied, corresponding to musical sounds between B3 to D#5 (~246 Hz to ~623 Hz).
The study of acoustic phenomena affirms that all musical instruments consist of a resonator (string, membrane, or resonant tube) whose geometry and materials will determine, for a given impulse, the natural modes of vibration of the generated stationary wave. Thus, the nature of the component materials of the musical instrument, and its geometric shape determine the secondary frequencies and harmonics possible of a mode of vibration or impulse associated with a certain musical note. This is what allows the reproducibility of the sounds for each musical note and consequently of the melodies.
The values of the amplitude of each vibration mode, both for the fundamental frequency and its harmonics and other component partial frequencies, will depend on several factors such as the energy associated with the initial impulse, the losses due to transmission and absorption, and the combination of standing waves (resonances, constructive and destructive interferences due to multiple reflections, reverberation, etc.). However, the distribution of the set of harmonics and the values of the fundamental frequencies and the partial frequencies do not change, even when the amplitudes of the waves are attenuated or intensified in the propagation of the waves inside the resonant cavity (aerophones), along the vibrating string (rubbed string instruments and plucked strings) or on the struck surface (flexible membrane or vibrating surface).
Then, the sound of a musical instrument will consist essentially of a succession of waves, which are characterized by the magnitudes of amplitude and phase. At each instant, the sound is then described by amplitudes (magnitude measured on any scale: intensities, decibels, energy, etc.) and frequencies. Consequently, a Fourier spectrum of a monophonic audio signal is essentially a finite, paired collection of numbers associated, respectively, with the component amplitudes and frequencies.
The discrete distribution of such points in a configuration space can be described in terms of (i) the maximum of the acoustically associated distribution with the sound of the corresponding musical note (fundamental frequency and fundamental amplitude); (ii) the general form of homogeneity and range of variability of the amplitudes of the partial frequencies that accompany the fundamental frequency of the musical sound considered; and (iii) the statistical measurements of the mean amplitudes and middle frequencies.
Quantitatively assessing the musical timbre then involves weighing these three aspects: the fundamental frequency and amplitude, a measure of the secondary or harmonic frequencies present in number and relative importance (assessable through their intensity relative to the fundamental frequency), and a measurement of the grouping of the frequencies present (cadence, monotony, sequence of harmonics, etc.). Two magnitudes (amplitude and frequency) are used for these three aspects (musical sound, shape of the distribution, mean value of the partial sounds). A minimum of six functionals are then required to describe the monophonic musical timbre of individual sounds, in terms of the normalized amplitudes and frequencies present.

3. Results: Timbral Coefficients

It should be considered that, unlike the timbral study of speech and environmental sounds, musical frequencies make up a finite, countable, and discrete set of only 12 different values in each musical octave, for a total of 96 possible fundamental frequencies and their integer multiples that are in the audible range: from 20 Hz to 20 kHz. Therefore, the musical timbre can be characterized by a limited set of timbral coefficients, which are dimensionless quantities related to the frequencies and amplitudes in the Fourier spectrum of the audio records. Motivated by musical acoustics, these coefficients are tonal descriptors and, in essence, functionally describe the discrete distribution of normalized frequencies and amplitudes. As the amplitudes of the spectra of the FFTs are normalized (using the quotient of the amplitude of each partial frequency between the amplitude greater measured in each spectrum) it is possible to compare the relative amplitudes among them. They can be grouped into descriptors of the fundamental frequency (musical scale, 96 possible frequencies) and descriptors of the rest of the partial frequencies that arise when performing the FFT of the audio under analysis (descriptors of the shape of the distribution and statistics frequency distribution), these proposed descriptors are dimensionless coefficients.
The FFT values are essentially a discrete collection of pairs of different amplitudes and frequencies; therefore, they can be summarized by the following dimensionless parameters:
  • Measurement of the fundamental frequency in relation to the average frequency, (Affinity A).
  • Measurement of the frequency distribution with respect to the average frequency (Mean Affinity MA).
  • Quantification of the amplitude of f 0 with respect to the collection of amplitudes (Sharpness S).
  • Quantification of the average amplitude of the pulse collection (Mean Contrast MC).
  • Descriptor of how close the secondary pulses are to being integer multiples of the fundamental frequency (Harmonicity H).
  • Descriptor of the envelope through the average slope in the collection of pulses (Monotony M).
The details of these dimensionless timbral coefficients are described next and exemplified for the FFTs of monophonic woodwind sounds.

3.1. Fundamental Frequency Descriptors

In the analysis of acoustic signals, the centroid is usually used to describe the properties of the sound; however, in the timbral characterization, its use entails the difficulty of its acoustic interpretation, since given an audio record, the value of the average frequency (centroid) does not correspond to the envelope, nor to any of the fundamental and harmonic frequencies of the sounds studied. Its interpretation in musical acoustics is difficult because it cannot be correlated with the set of discrete frequencies of musical sounds in the Tempered Scale used in Western music (96 characteristic frequencies). The centroid is defined as the average frequency.
f ¯ = i = 1 N a i f i i = 1 N a i  
where a i are the normalized amplitudes of the frequencies f i of the Fourier spectrum. The normalized amplitudes are obtained by dividing the amplitudes values by the amplitude of the fundamental frequency a 0 .
Figure 1 shows the Fourier Transform Spectra of the (4) wooden aerophones studied, particularly for the audio records of the musical sound E4 that corresponds to the nominal frequency of 329.6 Hz. The centroid or mean frequencies of the Fourier spectrum do not correspond to any of the harmonics of the analyzed sound.
Figure 2 shows the centroid values concerning the musical sounds of the common tessitura of the selected aerophones. It can be seen that the centroid values are almost constant for the frequency range considered.
The centroid, in any case, indicates the presence of frequencies other than the fundamental one, with a distribution of intensities such that the average frequency (centroid) has a magnitude much greater than the frequency prescribed for each musical sound. However, it does not report the harmonicity of the sounds and does not correlate with the “natural” sequence of the musical scale in any of the instruments in the sample.
The fundamental frequency f 0 only coincides with the Centroid f ¯ of the distribution if there is no harmonic or partial frequency in the sound. However, this is impossible in musical instruments whose resonators are exposed to multiple deflections, superpositions, and beats given by the geometry of the resonant tubes and openings, which characterize aerophones.
The separation between the fundamental frequency and the mean value of the frequencies would be important from the point of view of musical acoustics. This quantity can be evaluated through a dimensionless descriptor (Timbral Coefficient) that we call Affinity (A).
The Affinity coefficient describes how far the spectrum is from the ideal case, that is, how far the maximum Principal f 0 is from the mean value in frequency or centroid ( f ¯ ):
A f ¯ f 0
If f 0 and f ¯ are close there is more “Affinity” with the centroid and A tends to be one. In Figure 3, two Fourier spectra with very different coefficients of A are compared. Moreover, in the upper panel, it is noted that different sounds in different instruments may have the same centroid values. However, their affinity, defined by Equation (2), will be different.
The assessments of Affinity A for the audio records of the common tessitura (sounds B3 to D#5) for the aerophones studied are shown in Figure 4.
In Figure 4, it is observed that the Oboe has less Affinity with the fundamental frequency of the expected musical sounds (high values of A) and, at the other extreme, the Bassoon has a mean frequency closer to the expected musical sound (values of A closer to the unit).
If A 2 then the median or centroid frequency is twice the fundamental, as in the case of the Bassoon. For the Transverse Flute, the average frequency is of the order of 5/2 f 0 for the musical notes between B3 and F4, and of the order of 2 f 0 for the notes A#4 to D#5. Similarly, the Oboe behaves with more extreme values in both cases.
On the other hand, it can be observed in Figure 1 that various musical instruments have different amplitudes in their fundamental frequency for normalized Fourier spectra (that is, regardless of the intensity of the recording sound). Furthermore, for the same instrument, the amplitude of the fundamental frequency can, on occasions, remain constant between different musical sounds, or vary in amplitude for different frequencies.
One way to evaluate this variation of the amplitude of the fundamental frequency, concerning the amplitudes of the other frequencies of the spectrum, is by utilizing another dimensionless descriptor or Sharpness coefficient (S). Defined by:
S a 0 i = 1 N a i
Thus, the Sharpness (S) is a measure of the “height”, amplitude, or relative intensity of the fundamental frequency f 0 concerning the distribution, and its acoustic interpretation is immediate: the greater the Sharpness of a musical sound, the easier it will be due to its relative intensity to perceive the fundamental or nominal note that you want to play with the instrument. Ideally, a “pure” sound would be one that would have a single maximum without secondary frequencies S = 1. By construction S ≤ 1. Figure 5 shows how spectra of different musical instruments, and different fundamental frequencies, can have the same Sharpness. In addition, that the same musical instrument can, depending on the frequency, present very different Sharpness.
Figure 6 shows the variation in the Sharpness of aerophones. It is observed that, in general, (except for the F#4), the Clarinet is sharper than the Transverse Flute and this is sharper than the Bassoon; in any case, the Oboe is the least sharp of all. That is to say, the musical sounds are better defined in the Clarinet and poorly defined in the Oboe.

3.2. Descriptors of Frequency Distribution

It is also necessary to characterize the distribution of partial frequencies that appear associated with the fundamental frequency f 0 in the spectra of the FFTs of the audio records. Many authors use statistical descriptors of partial or secondary maximums, evaluating the kurtosis, the regular sequences when they occur, and statistical parameters.
However, it is possible to construct descriptors with a more direct acoustic meaning. In a given spectrum, there are always partial or secondary frequencies to f 0 and can or cannot be harmonics. Harmonics can be counted; however, it can also happen that a secondary frequency is not strictly an integer multiple of the fundamental frequency, but whose frequency is very close to an integer multiple of f 0 , and in a way, the sound would be more “harmonic” than if its frequency value were very different from the multiplicity of the fundamental frequency. To describe this property, the timbral coefficient of Harmonicity (H) is proposed and defined as:
H j = 1 N f j f 0 f j f 0
where the symbol [ ] denotes the integer part. This timbral function evaluates how harmonic the partial or secondary frequencies ( f 1 , f 2 , f 3 f j ) of the FFT spectrum are. The idea is that any frequency f j is a harmonic of f 0 if the quotient between them is an integer. Figure 7 shows this variation in examples of FFT spectra with different Harmonicity.
The Fourier spectra of the upper panel of Figure 7 show that the C4 sound of the Flute is much more harmonic than the Oboe, even though the latter has many more secondary frequencies, some of which may be harmonics of f 0 , as is the case of second, third and fourth maximum. If the secondary frequencies are all harmonics of f 0 then H = 0. Every time there are one or more frequencies that are not harmonics of f 0 , the j-th term, in the sum, will be non-zero and H increases. The maximums of the spectrum distribution of the Transverse Flute in Figure 7 are all integer multiples of the fundamental and, therefore, it is highly harmonic (H = 0).
The same instrument can present very different values of harmonicity, depending on the musical sound considered, as observed for the Oboe on the right side of Figure 7, in sounds C4 (H ≅ 17) and D5 (H ≅ 1). Even the same instrument with a qualitatively similar Fourier spectrum can present a very different harmonicity coefficient, as shown for the Transverse Flute in the left panel of Figure 7; sounds C4 (very harmonic: H ≅ 0) and B3 (very little harmonic: H ≅ 10).
For common ranges, the results of the harmonicity assessment are shown in Figure 8.
The Transverse Flute has the highest harmonicity (H close to zero), that is, its secondary frequencies are usually integer multiples of the corresponding fundamental sound, while the Oboe is the most anharmonic (large H). We note that the quality of harmonicity varies with the musical note and with the instrument, so for the characteristic sound of reference A = 440 Hz the Bassoon and Clarinet are completely harmonic, and the Transverse Flute is so for musical sounds C4, D4, E4, and A#4.
Another outstanding aspect of the distribution of maximums in the FFTs of the audio records is the variability of the amplitudes as a function of frequency. After the fundamental frequency f 0 , the following and successive maxima may have amplitudes that increase or decrease, that is, they could decrease in amplitude (decreasing Monotony) or increase (increasing monotony), or vary in growth with some tendency to increase or decrease.
Figure 9 shows records with different behaviors in the monotony of the secondary or partial frequencies (whether they are harmonic or not).
The timbral coefficient of Monotony (M) quantifies the regularity of the distribution of secondary frequencies (harmonic or not). It is defined as the average of the variations between two successive maxima:
M f 0 N j = 1 N a j + 1 a j f j + 1 f j
Essentially, monotony informs whether the harmonics appear in increasing (positive M) or decreasing (negative M) succession after the fundamental frequency. It can be said that this coefficient approximates and discretely quantifies the first derivative of the frequency distribution.
Essentially monotony informs whether the harmonics appear in mostly increasing (positive M) or decreasing (negative M) succession after the fundamental frequency. It can be said that this coefficient approximately and discretely quantifies the first derivative of the frequency distribution.
Figure 10 shows the assessment of the monotony of the common tessitura for the musical instruments studied. The Clarinet has negative values very close to zero and, on average, its frequencies decay in amplitude, which is to say that the sound attenuates very slightly after the fundamental frequency. The same homogeneity is appreciated in the sound of the Transverse Flute. On the contrary, the Bassoon shows high positive and negative values in several musical notes, that is, the bassoon has more sound alternation after the fundamental frequency. The Oboe has a certain alternation with a decreasing trend and the very negative M value of it reports a large alternation with a trend or decreasing envelope for several of the notes.

3.3. Distribution Statistics

Another aspect that is perceived in the FFTs of the audio records for different aerophones is that the secondary or partial frequencies occupy a different frequency range, that is, the partial frequencies (harmonic or not) are sometimes very close together or are grouped, and, in others, they are more spatially distributed or more separated. If the secondary frequencies are very close to each other or close to the fundamental, the sounds may appear more “compact or thick”, while the spatial separation of the frequencies gives more diaphanous or transparency to the fundamental frequency. Transparency is the timbral coefficient defined as the Mean Affinity (MA). It quantifies the frequency extent of the distribution and evaluates the separation of the partial frequencies concerning the centroid.
M A 1 N f 0 i = 1 N f i f ¯
The MA coefficient attempts to evaluate how compact the frequency distribution is concerning that mean value (including the fundamental frequency f 0 ). Note that the subscript i has been used instead of j to denote that the sum also includes f 0 In other coefficients the sum over j does not include f 0 because it refers to the secondary maxima, whether harmonic or not. Figure 11 shows how Mean Affinity (MA) varies in various FFT spectra. Figure 11 shows that the same sound can have different MA values in different instruments. For example, in the upper panel the C4 sound of the Clarinet (MA = 2.51) and the Transverse Flute (MA = 2.98) both have very similar affinity coefficients, as well as, in the lower panel, the D5 sound in Oboe (MA = 2.44) and Bassoon (MA = 1.56). This holds even for sounds and instruments with the same Affinity (A = 2.47) such as the C4 Clarinet (MA = 2.51) and the D5 Oboe (MA = 2.44). Two sounds with equal affinity values are observed in Figure 11 wherein both registers have the same Affinity A≅2.4 but their average Affinity are different (MA = 2.51 Clarinet and MA = 2.98 Transverse Flute).
Lower MA values refer to compact distributions of secondary sounds close to the centroid. Thus, the Oboe has a lower average Affinity (high A value) for the range of sounds studied, which is equivalent to saying that the secondary frequencies (different from the fundamental one) cover a greater range of frequencies. The opposite is the case with the Bassoon, whose secondary sounds are more compact for each of the nominal frequencies of the scale, as shown in Figure 12. It is important to note that, by its definition, the MA value multiplied by f 0 gives us the effective mean frequency.
Finally, it should be noted that two FFT spectra can have the same number of secondary or partial frequencies, even if in both cases they are harmonic, but if one of these partial spectra has higher amplitude compared to the other spectrum, they will appear to us with greater variability in frequencies (Figure 13).
The Mean Contrast (MC) is the coefficient that measures the amplitude of the partial frequencies (tones or “colors”) concerning the fundamental frequency.
M C 1 N j = 1 N a 0 a j  
MC increases as the intensity of partial frequencies. The MC values for the common tessitura of the aerophones studied are shown in Figure 14.
The Bassoon has a greater diversity of partial frequencies for the first half of the musical scale considered because the variations in amplitudes concerning amplitude a 0 of the fundamental frequency are greater when compared to other woodwind instruments. The same can be said of the Oboe for the other upper half of the musical scale concerning the other woodwind instruments. The Transverse Flute and the Clarinet exhibit approximately the same contrast of the amplitudes of their secondary frequencies related to the amplitude of the fundamental frequency.
In the appendix, the tables of the coefficients for all the tessitura of the four aerophones studied are presented. In these tables, the first column corresponds to the nominal frequency of the musical sound, the second column the Anglo-Saxon denomination of the musical sound, followed by the fundamental frequency and the centroid obtained utilizing the FFT of the audio records. Then, in the last columns, the values correspond to the coefficients of Affinity (A), Brightness (S), Harmonicity (H), Monotony (M), Mean Affinity (MA), and Mean Contrast (CM).

4. Discussion: Spectral Signatures

The identification of musical instruments, and in general of the wave emitting source, can be done through the analysis of their spectrum. In the case of electromagnetic radiation, the specific profile of intensities versus wavelength (absorption or emission) allows the identification of the emitter (be it a gas, a reflective surface, or an absorbing medium) and is called Spectral Signature. By analogy, we understand it as an “acoustic spectral signature” as the distribution of frequencies in the Fourier Transform Spectrum of musical sounds (tempered scale) that allow their univocal identification. If we restrict ourselves to the studied aerophones, which are only a small sample of all common musical instruments, a discriminatory identification between them can be made from the presented FFT spectra for common records in the range of B3 sounds to D#5, as shown in Figure 15.
The harmonic frequencies ( f n ) are those that are related to the fundamental frequency f 0 through the integer multiplicity (n = 2, 3, 4, …):
f n = n f
It is the case that not all the harmonics of a certain musical sound are always present. This allows, in principle, to discriminate the sounds of one of the aerophones from the others (Table 1). For a musical sound like B-3, the Oboe is distinguished from the other aerophones by presenting harmonics 2 to 13 inclusive. While the Bassoon lacks harmonic n = 6, the Flute contains harmonic n = 6 but lacks harmonic n = 13 that appears in the Oboe and the Clarinet, and the Clarinet lacks the harmonics n = 11 and n = 12 which distinguishes it from the Oboe and the Flute, respectively. From the collection of harmonics listed in Table 1 it is observed that one aerophone can be distinguished from another in most of the musical sounds of the common tessitura (in the range of sounds B3–D#5), so for example for C#4 the Bassoon only has up to the harmonic n = 6, the clarinet up to n = 7, the Flute reaches up to n = 9, and the Oboe even has harmonics of higher order.
It should be noted that the amplitudes of the obtained and normalized FFTs may present frequencies of negligible values compared to the frequency of greater amplitude. In any case, the harmonics counted in Table 1 and the calculation of coefficients are only those partial frequencies whose amplitude is greater than or equal to 1% of the maximum amplitude.
This discrimination, in terms of which harmonics are present and which are missing, is not univocal. It presents degeneration, in the sense that for some musical sounds two different aerophones present the same harmonics (highlighted with the symbol * in Table 1). However, as the FFTs are not the same between any two aerophones, the different relative intensities, through the Affinity and Sharpness coefficients, would allow the completion of the spectral signature and univocal identification of which instrument it belongs (Figure 16).
The degeneracy of the harmonic frequencies of the C4 sound between the Clarinet and the Bassoon is not resolved with the centroid value (Figure 2) which is the same for both, but with the coefficient of brilliants S, for example (Figure 6). Similarly, all other degenerations can be solved using one or more of the six proposed timbral coefficients.
Thus, for example, the triple apparent degeneration in sound A#4 is solved by using the coefficient of Mean Affinity AM (Figure 14) or Mean Contrast (Figure 16) which allows the musical instrument to be uniquely identified. On the other hand, it should be noted that the number of harmonic frequencies present does not necessarily mean greater harmonicity in the musical sound. Note, for example, that the C#4 sound of the Oboe has 12 harmonics (Table 1) and yet the value of H = 14 (Figure 8) shows great anharmonicity, while the Bassoon with only 4 harmonics exhibits great harmonicity (H = 0) in the same musical sound. All this illustrates that counting the harmonics present is not enough to determine the harmony of musical sounds.
From the preceding analysis, it is inferred that the timbral characteristics of monophonic musical sounds can be represented by a set of six dimensioned coefficients. That is to say that each sound of the tempered scale of a musical instrument corresponds to one and only one point in an abstract space of six dimensions, with coordinates (A, S, H, M, MA, MC). For certain musical instruments, the collection of points in that timbral space represents a bounded region, which characterizes it. Figure 16, Figure 17 and Figure 18 illustrate this grouping using the two-dimensional projections of the planes AS, HM, MAMC respectively of the sounds of the complete tessitura (Table A1, Table A2, Table A3 and Table A4 of the Appendix A) in the aerophones studied. Any monophonic musical sound can be located in said space through the timbral coefficients already defined from the FFT of the audio register and its Euclidean distance from the delimited regions. Therefore, the proposed coefficients would allow, in principle, the evaluation of the timbral characteristics using that can be applied to large dimensional spaces. In future work, we plan to exploit unsupervised machine learning techniques, such as clustering algorithms, to identify patterns from the timbral characteristics.

5. Conclusions

Given the discrete nature of the frequency spectrum of musical sounds, it is possible to construct dimensionless statistical descriptors, which relate the timbral characteristics of the fundamental frequency and the harmonic distribution in the FFTs of simple monophonic musical registers for wooden aerophones. Thus, the proposed timbral descriptors, Affinity and Sharpness, allow distinguishing the timbre of Clarinet, Bassoon, Transverse Flute, and Oboe.
The coefficients of Harmonicity, Monotony, Mean Affinity, and Mean Contrast are sufficient to describe the distribution of harmonics in the FFTs of the aerophones studied, observing distinctions and variabilities between the different sounds of the musical scale for the same instrument.
The Spectral Signature of the wooden aerophones, which allows the identification of the timbre for the instruments studied, can be obtained through the FFT by the distribution of harmonics present outside the loudness range of the common tessitura (B3–D#5). The number of harmonics present and their succession does not always characterize the timbre between different aerophones, and there may be degeneration (two or more instruments with equal harmonics); however, with the proposed coefficients, the musical timbre, by monophonic musical sound, is uniquely identified in the studied aerophones.
The FFT coefficients proposed in this work allow analyzing digital audio records of musical sounds, generating a unique spectrum that characterizes them. This spectrum is a finite, bounded, and countable collection of discrete frequencies with different amplitudes. We verify that, with the six proposed coefficients, we are able to discriminate between timbre differences and similarities for the set of test audios.
Their visualization in two-dimensional arrangements reveals some grouping patterns for specific instruments. We believe that this approach can be further extended to use multidimensional unsupervised that can be analyzed using Machine Learning techniques, such as clustering. Therefore, the characterization of the musical timbre is susceptible to being extended and automated.
The proposed timbral coefficients allow us to reduce the problem of the timbral characterization of musical instruments to a grouping problem in an abstract space of six dimensions for monophonic musical sounds. Therefore, in future work, we plan to perform a correlation analysis with the proposed coefficients, in addition to using unsupervised machine learning techniques.
To sum up, it is concluded that the proposed descriptors sufficiently describe the timbral characteristics in the aerophones studied and differentiate them, allowing their recognition and classification from the FFTs of the audio records. Finally, it is important to highlight that the proposed timbral coefficients can be extended to any of the melodic musical instruments, since the used methods are indifferent to the instruments studied.

Author Contributions

Conceptualization, Y.G. and R.C.P.; methodology, Y.G. and R.C.P.; software, Y.G. and R.C.P.; validation, Y.G. and R.C.P.; formal analysis, Y.G. and R.C.P.; investigation, Y.G. and R.C.P.; resources, Y.G. and R.C.P.; data curation, Y.G. and R.C.P.; writing—original draft preparation, Y.G.; writing—review and editing, Y.G. and R.C.P.; visualization, Y.G.; supervision, R.C.P.; project administration, Y.G. and R.C.P.; funding acquisition, Y.G. and R.C.P.; All authors have read and agreed to the published version of the manuscript.

Funding

This study was financed in part by the Coordenação de Aperfeiçoamento de Pessoal de Nível Superior—Brasil (CAPES)—Finance Code 001.

Institutional Review Board Statement

Not applicable.

Informed Consent Statement

Not applicable.

Data Availability Statement

The sounds used in this work are available at the following link: https://zenodo.org/record/3685367#.XnFp5i2h1IU%22, accessed date 6 June 2021).

Conflicts of Interest

The authors declare no conflict of interest.

Appendix A

Timbral coefficients for the tessitura of the Aerophones.
Table A1. Bassoon (Bn).
Table A1. Bassoon (Bn).
Sound (Hz)Soundf0 (Hz)Centroid (Hz)ASHMMAMC
58.27A#158.24504.018.650.050.30−0.01810.910.318
61.74B161.81546.078.830.0231.29−0.01210.730.214
65.41C265.37514.667.870.040.41−0.2229.750.250
69.30C#269.25535.297.730.040.48−0.4277.220.245
73.42D273.44538.177.330.0626.90−0.0118.830.166
77.78D#277.90521.516.690.0425.65−0.0138.710.196
82.41E282.34497.306.040.030.23−0.3304.410.199
87.31F287.40542.596.210.0518.80−0.0206.210.195
92.50F#292.46606.376.560.040.10−0.0026.860.153
98.00G298.02558.485.700.0414.96−0.0055.360.192
103.83G#2103.81582.735.610.031.010.0013.940.126
110.00A2110.02605.655.500.049.98−0.0014.430.135
116.54A#2116.60606.795.200.0316.91−0.0045.520.165
123.47B2123.45692.235.610.023.02−0.0034.470.184
130.81C3130.81663.115.070.020−0.0034.640.149
138.59C#3138.68611.694.410.029.96−0.0113.160.160
146.83D3146.88625.994.260.039.98−0.0123.200.205
155.56D#3155.53617.103.970.040.951.2003.180.205
164.81E3164.90519.323.150.0710.911.5453.200.184
174.61F3174.63571.593.270.109.931.3992.750.258
185.00F#3184.97631.073.410.030.01−0.0283.360.150
196.00G3196.00599.453.060.059.00−0.0763.080.210
207.65G#3207.71669.873.230.049.941.9492.760.190
220.00A3220.08604.872.750.074.04−3.0062.190.218
233.08A#3233.16598.012.560.066.99−0.1402.590.173
246.94B3246.89563.732.280.070−0.1911.910.211
261.63C4261.64633.212.420.184.963.1412.310.302
277.18C#4277.12560.822.020.120−0.1262.000.307
293.66D4293.66636.992.170.133.00−0.1652.210.254
311.13D#4311.19745.332.400.105.964.7121.660.215
329.63E4329.62703.922.140.110.02−6.9871.550.236
349.23F4349.23788.652.260.140.976.0841.540.276
369.99F#4369.96629.771.700.400.989.7110.970.606
392.00G4391.96876.692.240.4013.73−0.0911.570.937
415.30G#4415.34879.212.120.486.630.0311.340.890
440.00A4439.89742.191.690.450−0.1972.040.743
466.16A#4466.06774.811.660.410−0.2471.600.598
493.88B4493.93704.561.430.653.02−5.5681.150.868
523.25C5523.32767.901.470.582.02−8.4281.120.822
554.37C#5554.29708.981.280.722.020.0020.750.872
587.33D5587.28816.111.390.683.00−0.1201.560.843
622.25D#5622.19855.161.370.830.010.0022.120.965
Table A2. Clarinet (ClBb).
Table A2. Clarinet (ClBb).
Sound (Hz)Soundf0 (Hz)Centroid (Hz)ASHMMAMC
146.83D3146.90667.224.540.1511.96−0.0093.710.371
155.56D#3155.52807.575.190.190.06−0.0076.410.590
164.81E3164.79855.385.190.337.01−0.0026.400.895
174.61F3174.91872.034.990.3415.97−0.0046.060.890
185.00F#3184.96723.793.910.450.02−0.0025.310.918
196.00G3196.01606.963.100.536.00−0.0045.100.925
207.65G#3207.64708.663.410.482.0005.950.923
220.00A3220.06736.503.350.3611.9704.350.851
233.08A#3233.01943.534.050.360.030.0014.340.865
246.94B3246.96761.783.080.507.99−0.0383.730.876
261.63C4261.63647.422.470.547.000.0012.510.876
277.18C#4277.25646.102.330.546.000.0062.140.860
293.66D4293.62824.392.810.450.01−0.0033.670.877
311.13D#4311.15961.713.090.417.99−0.0043.500.856
329.63E4329.67950.002.880.416.99−0.0012.840.822
349.23F4349.19604.041.730.680−0.0022.010.905
369.99F#4369.98964.482.610.365.000.0012.030.646
392.00G4391.981013.862.590.351.985.6552.230.728
415.30G#4415.35736.151.770.686.00−0.0022.450.920
440.00A4439.97654.321.490.680−0.0761.260.843
466.16A#4466.15697.091.500.770−0.0071.900.924
493.88B4494.001015.802.060.534.99−0.0211.810.823
523.25C5523.20971.921.860.673.000.0021.930.901
554.37C#5554.271200.242.170.530.02−1.6622.170.875
587.33D5587.171625.552.770.220.03−7.3262.540.566
622.25D#5622.211321.242.120.410−0.0891.790.709
659.26E5659.32943.241.430.733.00−0.0511.740.909
698.46F5698.49944.361.350.784.00−0.0362.270.943
739.99F#5739.971103.771.490.701.00−0.0561.710.892
783.99G5784.011116.561.420.682.00−0.1021.290.845
830.61G#5830.601218.211.470.561.01−20.9091.120.801
880.00A5880.041504.441.710.503.02−13.2010.930.799
932.33A#5932.271336.471.430.630−0.1661.280.802
987.77B5980.691427.351.460.010.140.1481.290.210
1046.50C61046.511615.131.540.521.001.0610.970.819
1108.73C#61108.681659.611.500.620−0.1351.250.800
1174.66D61174.641386.541.180.870−0.0220.940.924
1244.51D#61236.231464.121.180.010.07−38.5710.670.232
1318.51E61318.411587.831.200.830−0.0530.930.900
1396.91F61389.251533.031.100.010.05−45.1570.820.262
1479.98F#61479.891864.281.260.780−0.0950.910.857
1567.98G61559.801633.741.050.020.03−60.7150.270.332
Table A3. Oboe (Ob).
Table A3. Oboe (Ob).
Sound (Hz)Soundf0 (Hz)Centroid (Hz)ASHMMAMC
233.08A#3233.081235.405.300.073.03−2.0854.470.288
246.94B3246.911187.764.810.100.982.3743.440.317
261.63C4261.691428.425.460.0716.902.9644.230.242
277.18C#4277.281249.264.510.0814.00−1.0183.440.380
293.66D4293.721319.324.490.0911.98−0.0184.190.213
311.13D#4311.111308.454.210.165.00−0.0153.440.383
329.63E4329.601275.103.870.102.99−0.0493.510.413
349.23F4349.161444.034.140.060.01−0.0273.620.205
369.99F#4369.971451.343.920.083.05−3.6532.640.330
392.00G4391.961459.303.720.081.03−2.3542.690.258
415.30G#4415.321735.334.180.0810.982.6162.860.230
440.00A4440.061470.573.340.098.04−8.1142.550.309
466.16A#4466.151412.773.030.130.04−9.9822.660.374
493.88B4493.891391.552.820.165.000.3221.910.359
523.25C5523.291543.432.950.128.00−0.1232.690.290
554.37C#5554.291421.102.560.221.00−0.1182.460.513
587.33D5587.281452.772.470.351.02−2.2252.440.834
622.25D#5622.191195.441.920.330.01−14.8831.220.529
659.26E5659.251278.221.940.302.001.8141.050.553
698.46F5698.501409.192.020.387.000.2661.890.768
739.99F#5740.021494.082.020.317.000.1531.780.524
783.99G5783.991540.351.960.234.00−0.1451.400.371
830.61G#5830.631641.411.980.377.01−4.4761.720.766
880.00A5871.651388.281.590.010.28−8.2621.480.187
932.33A#5932.271494.201.600.501.01−8.3020.770.799
987.77B5977.481582.061.620.010.151.1460.900.336
1046.50C61046.551713.491.640.545.01−1.9741.400.876
1108.73C#61108.781894.031.710.364.014.8641.120.581
1174.66D61174.601862.601.590.551.02−1.4881.230.896
1244.51D#61235.061741.561.410.010.09−30.1281.010.255
1318.51E61308.771936.831.480.030.16−13.9471.060.142
1396.91F61396.942425.151.740.352.994.9480.900.554
1479.98F#61471.432780.541.890.020.26−12.3101.030.129
1567.98G61552.422625.201.690.010.23−1.9070.950.147
1661.22G#61651.262645.091.600.010.11−23.8450.810.224
Table A4. Transverse Flute (Fl).
Table A4. Transverse Flute (Fl).
Sound (Hz)Soundf0 (Hz)Centroid (Hz)ASHMMAMC
246.94B3247.00656.412.660.309.98−0.0983.850.614
261.63C4261.75638.712.440.410.01−0.0952.980.819
277.18C#4277.31855.273.080.208.09−4.4232.470.439
293.66D4293.62814.332.770.270.01−0.1232.790.558
311.13D#4311.17862.682.770.267.04−5.6542.390.542
329.63E4329.87918.062.780.324.020.0132.620.874
349.23F4349.22919.002.630.292.03−0.6652.220.696
369.99F#4370.02996.032.690.215.04−6.8812.500.449
392.00G4391.931036.912.650.421.03−3.3772.230.860
415.30G#4415.331012.082.440.513.04−0.5602.180.903
440.00A4440.01858.621.950.544.970.5452.410.880
466.16A#4456.90760.751.670.010.351.7321.470.253
493.88B4493.95858.771.740.654.00−0.0342.010.892
523.25C5523.24918.831.760.620.993.7651.490.879
554.37C#5546.30962.231.760.010.25−10.2201.340.265
587.33D5587.32954.201.620.660−0.0251.190.831
622.25D#5622.29845.941.360.792.00−0.0101.570.911
659.26E5651.191006.101.550.010.19−0.1951.510.269
698.46F5698.531019.651.460.711.993.2261.320.896
739.99F#5731.941238.051.690.021.35−7.1991.430.129
783.99G5781.351303.961.670.031.22−1.4791.140.143
830.61G#5821.021039.681.270.020.16−12.3391.120.180
880.00A5871.601120.361.290.020.16−17.1411.400.190
932.33A#5924.891064.711.150.010.05−0.3160.760.361
987.77B5987.831112.081.130.900.010.0110.750.965
1046.50C61038.961192.671.150.020.06−30.5810.630.270
1108.73C#61100.581229.781.120.020.06−34.6710.630.259
1174.66D61174.681303.521.110.911.010.0080.750.968
1244.51D#61244.572150.821.790.013.49−0.9140.980.123
1318.51E61307.601377.561.110.902.010.0040.750.965
1396.91F61388.371406.131.080.060.05−0.3230.760.340
1479.98F#61479.981615.881.160.020.08−36.0791.170.206
1567.98G61560.071501.701.010.970.010.0010.340.986
1661.22G#61652.561675.731.070.010.04−48.6020.620.254
1760.00A61752.181784.121.080.020.04−52.2710.440.258
1864.66A#61856.441921.761.100.010.04−56.2040.620.264
1975.53B61975.522392.191.290.010.37−11.4011.120.100

References

  1. Lartillot, O.; Toiviainen, P.; Eerola, T. A Matlab Toolbox for music information retrieval. In Data Analysis, Machine Learning and Applications; Springer: Berlin/Heidelberg, Germany, 2008; pp. 261–268. [Google Scholar]
  2. Li, H.; You, H.; Fei, X.; Yang, M.; Chao, K.M.; He, C. Automatic Note Recognition and Generation of MDL and MML using FFT. In 2018 IEEE 15th International Conference on e-Business Engineering (ICEBE); IEEE: Piscataway, NJ, USA, 2018; pp. 195–200. [Google Scholar]
  3. Nagawade, M.S.; Ratnaparkhe, V.R. Musical instrument identification using MFCC. In 2017 2nd IEEE International Conference on Recent Trends in Electronics, Information & Communication Technology (RTEICT); IEEE: Piscataway, NJ, USA, 2017; pp. 2198–2202. [Google Scholar]
  4. Chakraborty, S.S.; Parekh, R. Improved musical instrument classification using cepstral coefficients and neural networks. In Methodologies and Application Issues of Contemporary Computing Framework; Springer: Singapore, 2018; pp. 123–138. [Google Scholar]
  5. Łętowski, T. Timbre, tone color, and sound quality: Concepts and definitions. Arch. Acoust. 2014, 17, 17–30. [Google Scholar]
  6. Adeli, M.; Rouat, J.; Wood, S.; Molotchnikoff, S.; Plourde, E. A Flexible Bio-Inspired Hierarchical Model for Analyzing Musical Timbre. IEEE/ACM Trans. Audio Speech Lang. Process. 2016, 24, 875–889. [Google Scholar] [CrossRef]
  7. Alías, F.; Socoró, J.C.; Sevillano, X. A Review of Physical and Perceptual Feature Extraction Techniques for Speech, Music and Environmental Sounds. Appl. Sci. 2016, 6, 143. [Google Scholar] [CrossRef] [Green Version]
  8. Benetos, E.; Dixon, S.; Duan, Z.; Ewert, S. Automatic Music Transcription: An Overview. IEEE Signal Process. Mag. 2018, 36, 20–30. [Google Scholar] [CrossRef]
  9. Hernandez-Olivan, C.; Pinilla, I.Z.; Hernandez-Lopez, C.; Beltran, J. A Comparison of Deep Learning Methods for Timbre Analysis in Polyphonic Automatic Music Transcription. Electronics 2021, 10, 810. [Google Scholar] [CrossRef]
  10. Jiang, W.; Liu, J.; Zhang, X.; Wang, S.; Jiang, Y. Analysis and Modeling of Timbre Perception Features in Musical Sounds. Appl. Sci. 2020, 10, 789. [Google Scholar] [CrossRef] [Green Version]
  11. Guven, E.; Ozbayoglu, A.M. Note and Timbre Classification by Local Features of Spectrogram. Procedia Comput. Sci. 2012, 12, 182–187. [Google Scholar] [CrossRef] [Green Version]
  12. Fourer, D.; Rouas, J.L.; Hanna, P.; Robine, M. Automatic timbre classification of ethnomusicological audio recordings. In Proceedings of the International Society for Music Information Retrieval Conference (ISMIR 2014), Taipei, Taiwan, 27–31 October 2013. [Google Scholar]
  13. McAdams, S. The perceptual representation of timbre. In Timbre: Acoustics, Perception, and Cognition; Springer: Cham, Switzerland, 2019; pp. 23–57. [Google Scholar]
  14. Liu, J.; Zhao, A.; Wang, S.; Li, Y.; Ren, H. Research on the Correlation Between the Timbre Attributes of Musical Sound and Visual Color. IEEE Access 2021, 9, 97855–97877. [Google Scholar] [CrossRef]
  15. Reymore, L.; Huron, D. Using auditory imagery tasks to map the cognitive linguistic dimensions of musical instrument timbre qualia. Psychomusicol. Music Mind Brain 2020, 30, 124–144. [Google Scholar] [CrossRef]
  16. Reymore, L. Characterizing prototypical musical instrument timbres with Timbre Trait Profiles. Music. Sci. 2021. [Google Scholar] [CrossRef]
  17. Barbedo, J.G.A.; Tzanetakis, G. Musical Instrument Classification Using Individual Partials. IEEE Trans. Audio Speech Lang. Process. 2010, 19, 111–122. [Google Scholar] [CrossRef]
  18. Joshi, S.; Chitre, A. Identification of Indian musical instruments by feature analysis with different classifiers. In Proceedings of the Sixth International Conference on Computer and Communication Technology; IEEE: Piscataway, NJ, USA, 2015; pp. 110–114. [Google Scholar]
  19. Ezzaidi, H.; Bahoura, M.; Hall, G.E. Towards a Characterization of Musical Timbre Based on Chroma Contours. Robotics 2012, 322, 162–171. [Google Scholar] [CrossRef]
  20. Böck, S.; Korzeniowski, F.; Schlüter, J.; Krebs, F.; Widmer, G. Madmom: A new python audio and music signal processing library. In Proceedings of the 24th ACM International Conference on Multimedia; IEEE: Piscataway, NJ, USA, 2016; pp. 1174–1178. [Google Scholar]
  21. McFee, B.; Raffel, C.; Liang, D.; Ellis, D.P.; McVicar, M.; Battenberg, E.; Nieto, O. Librosa: Audio and music signal analysis in python. In Proceedings of the 14th Python in Science Conference, Piscataway, NJ, USA; 2015. [Google Scholar]
  22. Peeters, G.; Giordano, B.L.; Susini, P.; Misdariis, N.; McAdams, S. The Timbre Toolbox: Extracting audio descriptors from musical signals. J. Acoust. Soc. Am. 2011, 130, 2902–2916. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  23. Elliott, T.M.; Hamilton, L.S.; Theunissen, F.E. Acoustic structure of the five perceptual dimensions of timbre in orchestral instrument tones. J. Acoust. Soc. Am. 2013, 133, 389–404. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  24. Cella, C.E.; Ghisi, D.; Lostanlen, V.; Lévy, F.; Fineberg, J.; Maresz, Y. 2020. OrchideaSOL: A Dataset of Extended Instrumental Techniques for Computer-Aided Orchestration. arXiv 2020, arXiv:2007.00763. [Google Scholar] [CrossRef]
  25. Adler, S.; Hesterman, P. The Study of Orchestration; WW Norton: New York, NY, USA, 1989. [Google Scholar]
  26. Pons, J.; Slizovskaia, O.; Gong, R.; Gómez, E.; Serra, X. Timbre analysis of music audio signals with convolutional neural networks. In Proceedings of the 2017 25th European Signal Processing Conference (EUSIPCO), Kos, Greece, 28 August–2 September 2017; pp. 2744–2748. [Google Scholar] [CrossRef] [Green Version]
  27. Virtanen, P.; Gommers, R.; Oliphant, T.E.; Haberland, M.; Reddy, T.; Cournapeau, D.; Burovski, E.; Peterson, P.; Weckesser, W.; Bright, J.; et al. SciPy 1.0 Contributors. SciPy 1.0 Fundamental Algorithms for Scientific Computing in Python. Nat. Methods 2020, 17, 261–272. [Google Scholar] [CrossRef] [PubMed] [Green Version]
Figure 1. FFT spectra for the aerophones studied. Note that in each instrument the centroids (dotted line) do not coincide with any of the frequencies in the distribution and are not directly linked to the musical sound E4 of nominal frequency 329.6 Hz.
Figure 1. FFT spectra for the aerophones studied. Note that in each instrument the centroids (dotted line) do not coincide with any of the frequencies in the distribution and are not directly linked to the musical sound E4 of nominal frequency 329.6 Hz.
Electronics 11 01405 g001
Figure 2. Variation of the centroid concerning the common tessitura of the wooden aerophones studied: Bassoon (Bn), Oboe (Ob), Transverse Flute (Fl), Clarinet in Bb (ClBb). For comparison purposes, the location of the musical notes of the scale below the abscissa axis is also included.
Figure 2. Variation of the centroid concerning the common tessitura of the wooden aerophones studied: Bassoon (Bn), Oboe (Ob), Transverse Flute (Fl), Clarinet in Bb (ClBb). For comparison purposes, the location of the musical notes of the scale below the abscissa axis is also included.
Electronics 11 01405 g002
Figure 3. Examples of Fourier spectra for sounds with different Affinity coefficients and similar centroids (top panel) and the same sound in different aerophones. The dashed line indicates the relative location of the centroid.
Figure 3. Examples of Fourier spectra for sounds with different Affinity coefficients and similar centroids (top panel) and the same sound in different aerophones. The dashed line indicates the relative location of the centroid.
Electronics 11 01405 g003
Figure 4. Variation of Affinity A in the selected sample of aerophones.
Figure 4. Variation of Affinity A in the selected sample of aerophones.
Electronics 11 01405 g004
Figure 5. Example of Timbral Coefficient S. Above: The spectra of the Clarinet for C#4 and of the Transverse Flute for A4 show the same Sharpness S = 0.54. Bottom: Bassoon spectra in B4 and E4, with coefficients of S = 0.65 and S = 0.11 respectively.
Figure 5. Example of Timbral Coefficient S. Above: The spectra of the Clarinet for C#4 and of the Transverse Flute for A4 show the same Sharpness S = 0.54. Bottom: Bassoon spectra in B4 and E4, with coefficients of S = 0.65 and S = 0.11 respectively.
Electronics 11 01405 g005
Figure 6. Variation of Sharpness (S) concerning common ranges.
Figure 6. Variation of Sharpness (S) concerning common ranges.
Electronics 11 01405 g006
Figure 7. Comparison of Harmonicity H for different sounds and instruments. Top panel: C4 Flute and Oboe sound comparison. Lower panel comparison of different sounds for the same instrument (Left: Transverse Flute, right Oboe).
Figure 7. Comparison of Harmonicity H for different sounds and instruments. Top panel: C4 Flute and Oboe sound comparison. Lower panel comparison of different sounds for the same instrument (Left: Transverse Flute, right Oboe).
Electronics 11 01405 g007
Figure 8. Variation of the Harmonicity H of aerophones in the range of sounds B3–D#5.
Figure 8. Variation of the Harmonicity H of aerophones in the range of sounds B3–D#5.
Electronics 11 01405 g008
Figure 9. Effect of monotony on FFT spectra. Decreasing sound A4 Clarinet (left), increasing sound B3 Oboe and alternates, Clarinet D#5, and F#4 Oboe.
Figure 9. Effect of monotony on FFT spectra. Decreasing sound A4 Clarinet (left), increasing sound B3 Oboe and alternates, Clarinet D#5, and F#4 Oboe.
Electronics 11 01405 g009
Figure 10. Monotony (M) of aerophones for common tessitura.
Figure 10. Monotony (M) of aerophones for common tessitura.
Electronics 11 01405 g010
Figure 11. Variation of the mean amplitude of Fourier spectra. Sound C4 top panel: Clarinet (MA = 2.80) and Transverse Flute (MA = 1.30). Lower panel sound D5 Oboe (MA = 2.44) and Bassoon (MA = 1.56).
Figure 11. Variation of the mean amplitude of Fourier spectra. Sound C4 top panel: Clarinet (MA = 2.80) and Transverse Flute (MA = 1.30). Lower panel sound D5 Oboe (MA = 2.44) and Bassoon (MA = 1.56).
Electronics 11 01405 g011
Figure 12. Variation of Mean Affinity MA in the range of sounds B3–D#5.
Figure 12. Variation of Mean Affinity MA in the range of sounds B3–D#5.
Electronics 11 01405 g012
Figure 13. Average contrast in the FFT spectra of the audio records. Top panel: E4 sound for Bassoon (CM = 0.236) and Oboe (CM = 0.413), both with similar brilliance (S ≈ 0.1). Bottom panel: G4 sound for Bassoon and Flute both with similar medium contrast (CM ≈ 0.9) and brightness (S ≈ 0.4).
Figure 13. Average contrast in the FFT spectra of the audio records. Top panel: E4 sound for Bassoon (CM = 0.236) and Oboe (CM = 0.413), both with similar brilliance (S ≈ 0.1). Bottom panel: G4 sound for Bassoon and Flute both with similar medium contrast (CM ≈ 0.9) and brightness (S ≈ 0.4).
Electronics 11 01405 g013
Figure 14. Mean contrast (MC) of the instruments studied for common tessituras.
Figure 14. Mean contrast (MC) of the instruments studied for common tessituras.
Electronics 11 01405 g014
Figure 15. Spectral Signatures for Aerophones: (a): Clarinet and Oboe, (b): Bassoon and Transverse Flute.
Figure 15. Spectral Signatures for Aerophones: (a): Clarinet and Oboe, (b): Bassoon and Transverse Flute.
Electronics 11 01405 g015
Figure 16. Two-dimensional representation in the AS plane of the timbral space of the coefficients for the selected aerophones.
Figure 16. Two-dimensional representation in the AS plane of the timbral space of the coefficients for the selected aerophones.
Electronics 11 01405 g016
Figure 17. Two-dimensional representation in the HM plane of the timbral space of the coefficients for the selected aerophones.
Figure 17. Two-dimensional representation in the HM plane of the timbral space of the coefficients for the selected aerophones.
Electronics 11 01405 g017
Figure 18. Two-dimensional representation in the MAMC plane of the timbral space of coefficients for the selected aerophones.
Figure 18. Two-dimensional representation in the MAMC plane of the timbral space of coefficients for the selected aerophones.
Electronics 11 01405 g018
Table 1. Set of harmonics depending on the musical sound and the musical instrument.
Table 1. Set of harmonics depending on the musical sound and the musical instrument.
SoundNumber of Harmonics (n)
Notef0 (Hz)ClBbObBnFl
B3246.94{3,…,8}{10}{13}{2,…,13}{2,…,5}{7}{2,…,10}{12}
C4261.6{3,…,8} *{2,…,14}{16,17}{2,…,8} *{2,…,9}
C#4277.2{3,…,7}{2,…,9}{11,12}{14}{16}{2}{4,…,6}{2,…,9}
D4293.7{2,…,11}{2,…,12}{15}{2,…,7}{2,…,9}
D#4311.1{2,…,11}{2,…,13}{2,…,5}{7}{2,…,6}{8,9}
E4329.6{2,…,8}{10}{2,…,14}{2,…,6}{2,…,10}
F4329.67{2,…,6} *{2,…,10}{12,13}{2,…,6} *{2,…,8}
F#4370{2,…,7}{2,…,12}{2,…,4}{2,…,9}
G4494{2,…,8} *{2,…,11}{2,…,6}{8}{1}{2,…,8} *
G#4415.3{2,…,7}{2,…,12}{2,…,5}{8}{2,…,8}
A4440{2,…,4}{2,…,10}{2,…,6}{2,…,6}{8}
A#4466.2{2,…,5} *{2,…,10}{2,…,5} *{2,…,5} *
B4466.2{2,…,6} *{2,…,9}{2,…,4}{2,…,6} *
C5523.25{2,…,6}{2,…,9}{2,…,4}{2,…,5}
C#5554.37{2,…,7}{2,…,9}{2,3}{2,…,5}
D5587.33{2,…,9} *{2,…,9} *{2,3}{5}{2,…,4}
D#5622.25{2,…,6}{2,…,5}{2,…,5}{7}{2,3}{5}
* Denotes degeneracy: two different instruments with the same set of harmonics.
Publisher’s Note: MDPI stays neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Share and Cite

MDPI and ACS Style

Gonzalez, Y.; Prati, R.C. Acoustic Descriptors for Characterization of Musical Timbre Using the Fast Fourier Transform. Electronics 2022, 11, 1405. https://doi.org/10.3390/electronics11091405

AMA Style

Gonzalez Y, Prati RC. Acoustic Descriptors for Characterization of Musical Timbre Using the Fast Fourier Transform. Electronics. 2022; 11(9):1405. https://doi.org/10.3390/electronics11091405

Chicago/Turabian Style

Gonzalez, Yubiry, and Ronaldo C. Prati. 2022. "Acoustic Descriptors for Characterization of Musical Timbre Using the Fast Fourier Transform" Electronics 11, no. 9: 1405. https://doi.org/10.3390/electronics11091405

Note that from the first issue of 2016, this journal uses article numbers instead of page numbers. See further details here.

Article Metrics

Back to TopTop