HOTGAME: A Corpus of Early House and Techno Music from Germany and America

Abstract

Many publications on early house and techno music have the character of documentation and include (auto-)biographical statements from contemporaries of the scene. This literature has led to many statements, hypotheses, and conclusions. The weaknesses of such sources are their selective and subjective nature, and the danger of unclear memories, romanticization, and constructive memory. Consequently, a validation through content-based, quantitative music analyses is desirable. For this purpose, the HOuse and Techno music from Germany and AMErica (HOTGAME) corpus was built. Metrics from the field of data quality control show that the corpus is representative and explanatory for house and techno music from Germany and the United States of America between 1984 and 1994. HOTGAME can serve as a reliable source for the analysis of early house and techno music using big data methods, like inferential statistics and machine learning.

1. Introduction

An analysis of early house and techno music may be of interest for researchers from the field of history, musicology, digital humanities, cultural studies, media studies, ethnology, economics, marketing, and gender studies. For researchers, sources like house and techno documentaries (e.g., [1,2,3]) and interviews (e.g., [4,5]), as well as (auto-)biographies and reports, like [6,7,8,9,10,11,12,13,14,15,16,17], are appreciated. But it has been claimed that such sources are not scientifically sound. They aim at constructing a certain, pithy narrative; do not take a critical or multiperspectival point of view; focus on selective aspects; ignore statements that do not match the desired narrative; focus on feelings rather than facts; and use montages and timelines that imply that things that have not been said explicitly [18], [19] p. 2, [20,21]. Moreover, people tend to unconsciously create false memories when recounting their past or when they give interviews repeatedly, a phenomenon known as constructive memory [22]. Then, redundancy and repetition can bias collective memory [23]. Clearly, researchers must critically reflect on the narratives that often originate from a small group of expert contemporary witnesses [19] p. 102, [24,25]. And it is important to put music analysis in relation to its social space [26] p. 81 and scene [27] p. 16.

The aim of the HOTGAME corpus is to serve as a dataset for the purpose of analyzing early house and techno music based on the music itself, providing additional evidence from a new perspective. This requires a high-quality dataset. The aim of this paper is to test whether the HOTGAME corpus has the necessary quality to allow generalizable conclusions on early house and techno music from Germany and the United States of America.

In the field of data science, the importance of data quality control for scientific research and decision-making has been highlighted [28,29,30]. That is, the generalization of observations from a sample to the general population is only valid if the dataset fulfills certain quality requirements:

• Valid Data
• Few Data Duplicates
• Large Sample Size
• Good Data Balance
• Few Missing Data
• No Gender Bias
• Explanatory Features
• Low Feature Multicollinearity

Following the guidance of [30], the HOTGAME corpus was built according to the three phases collect, assess, improve in the context of early house and techno music. In this paper, the quality of the corpus is evaluated regarding the above-listed requirements. The remainder of this paper is structured as follows: In the Section 2, the analyzed music and the extracted features are described. In the Section 3, it is explained how the fulfillment of the requirements was evaluated. The Section 4 summarizes the results and discusses their implications, which culminate in the Section 5.

2. Materials

HOTGAME, the HOuse and Techno from Germany and AMErica corpus, contains information on tracks, which is the recognized term for music pieces of these genres [26], [31] p. 2, [32] p. 114, and [33] p. 124. The purpose of the HOTGAME corpus is to enable the validation of statements on the music scene from the music itself, which requires high-quality data. Here, quality is a matter of the corpus itself and of features extracted from the music. Statements, the corpus, and the features are explained in this section.

2.1. Statements

Based on documentaries and interviews, many conclusions have been drawn concerning the relevance of German reunification [10] pp. 72–73, [34] p. 62, [35] p. 21, [36] p. 216, [27] p. 56, [11] p. 56, and [37] p. 26; the exchange between different German regions [34] p. 109, [27] p. 57 and p. 115, [38] p. 472, [39] p. 43, and [37] p. 25; the exchange between American metropoles [38] p. 472; the exchange between Detroit and Berlin [38] p. 472, [37] pp. 25–26, and [40] p. 107; the rivalry between Detroit and Chicago [41] p. 353; the rivalry between Berlin and Frankfurt [6] p. 9, [32] p. 153; and the rivalry between mainstream and underground techno in Germany [40] p. 107. With an appropriate dataset, these conclusions and other narratives can be validated from the music itself.

The least common denominator is that house and techno music, their emergence, and their development were distinct between America and Germany [19] p. 16, [38] p. 472 and p. 489, [41] p. 346, [42] p. 154, [43] p. 243, and [44]. This statement serves as a ground truth: explanatory features should differ significantly between nations, years, and nation*years.

2.2. Corpus

For the HOTGAME corpus, music was taken from private collections and university collections and acquired through second-hand online record and CD stores. This ensured that the data was valid and legal. In [15,16,17,19,27,32,37,40,42,45], German and US-American artists and record labels can be found. The labels served as a starting point to search for the respective artists and other licensed artists; their co-producers; features; and remixers. From these, all licensing record labels were identified. In Germany, 101 dance record labels were identified that were active between 1985 and 1994. In the United States, 91 labels were found that were active between 1984 and 1994. A list of record labels is provided in the repository. Artist names from these labels were collected. House and techno music from these artists were also considered when the music was released on labels from other countries or labels with a wider range of music genres. Based on this data, a representative music corpus was built.

Only house and techno music and their descendants were incorporated. Music genres considered ancestors of house, such as (Italy) disco, industrial, and Krautrock, were not included. Music clearly assigned to parallel developments, like Electronic Body Music (EBM), Miami bass, electro funk, synth-pop, high-NRG, Neue Deutsche Welle, and new wave, was also ignored. The selection was necessary to keep the corpus narrowly focused and in accordance with the house and techno music literature [27,32,35,46]. Typically, this selection excluded handmade music (not quantized, played by bands, and/or performed on concerts rather than in night clubs, like Devo and Die Krupps but unlike Barbara Tucker and Rob Acid) and music by producers only associated with the hip hop scene but not with the house or techno scene (like Egyptian Lover but unlike Afrika Bambaataa). Only music until 1994 was included for two reasons: (1) There is a consensus that the mainstream breakthrough of house and techno can be dated back to 1994 [37] p. 45, [12] p. 258, and [38] p. 193, which is why (2) the sheer number of record labels and releases grew too fast to collect a representative sample from 1995 onward cf. [47] p. 237, [38] p. 193, [37] p. 45, and [12] p. 258.

The HOTGAME corpus consists of CSV-files in two folders: germany and usa. A track was assigned to the nation where the producer(s) grew up. Their place of birth and place of residence were not considered. The nationality of the performers was not considered. When a track was remixed, the track was assigned to the nation where the remixer grew up. The filenames consist of artist-title (version) (year).format.csv. The CSV files contain feature vectors that are discussed in the next section.

2.3. Features

In HOTGAME, tracks were stored as lossily compressed audio files with a sample rate of 44,100 Hz. Music from audio-CDs was normalized to an absolute peak value of 1. Music recorded from vinyl was also normalized. But when the peak value was a short pulse (vinyl crackle artefact), the level was raised to normalize to the true peak, accepting the clipping of the short pulse. The decision to analyze lossy music was (a) pragmatic, in order to save storage space, and (b) sensible, so that the results can be compared with today’s typical music streams. Note that both the medium and the compression slightly affect the features. The music was sliced into time frames of 2048 samples, i.e., around 50 ms, with no overlap. The first and last 4 s were cropped to remove silence and pure vinyl crackle. From each frame, several audio features were extracted.

There is a consensus that the production process plays a central role in house and techno music [19] p. 6, [31,48], and [26] p. 84, particularly regarding sound design [40] p. 108, [49] p. 210 in terms of intensity [19]; stereo position [38] p. 142; dynamics [38] p. 142; tempo [19,40], [38] p. 102, [12] p. 256, and [49] p. 190; and sound color [19], [38] (Section 3.8). Analyzing house and techno music using the same tools that music producers use in the recording studio to tune their sound promises to deliver causal and interpretable results. The recording studio analyzers are briefly described in the following sections. A more detailed explanation can be found in [50,51,52]. As the features relate to audio engineering parameters and aspects of sound perception, the sound terminology of [53] is utilized, and care is taken to distinguish between acoustical and perceptual terms.

2.3.1. Phase Space

Phase space diagrams are point clouds in which the left channel is plotted over the right channel. They serve to monitor the volume and stereo position of a mix. Perceptually, these audio aspects relate to loudness, and auditory event angle [54].

2.3.2. Channel Correlation

Channel correlation meters display the Pearson correlation coefficient (P) between the two stereo channels over each time frame. Perceptually, the absolute value of the channel correlation relates to the apparent source width [54]. Phase scope and channel correlation meter are often combined in audio analysis tools [55]. In [52], the combination of phase scope and channel correlation meter served to tell various hip hop producers apart. In [51], phase scope and channel correlation meter were shown to be orthogonal and explanatory to distinguish between different music genres.

2.3.3. Peak

Peak meters display the largest absolute value of a time frame. Music producers consult this value to estimate aspects of loudness, understand the headroom, and avoid heavy clipping [56], Chapter 7 [50].

2.3.4. Root Mean Square

Root Mean Square (RMS) meters display the root mean square of a given time frame. In practice, this value is always smaller than the peak. RMS meters are consulted to estimate the dynamics of a music piece, as the RMS roughly approximates loudness perception [48,50].

2.3.5. Crest Factor

The crest factor is a logarithmic value that describes the relationship between peak and RMS. Sometimes, it is referred to as the dynamic range. Acoustically, it describes how impulsive the sound is, which relates to the perception of percussiveness [50]. Music producers consult crest factor meters, for example, to monitor how much drum sounds stand out of the rest of the mix. Peak, RMS, and crest factor meters are utilized in [48,50,57] to analyze aspects of dynamics in mainstream music, electronic dance music, and music mixing in general.

All the above-mentioned features are utilized in the recording studio to analyze aspects of the music mix. In [50,52], they were also extracted from three frequency bands, as the music producer usually treat bass (approx. 20–150 Hz), mids (approx. 150–2000 Hz), and highs (approx. 2000–10,000 Hz) differently [58] p. 25. Their balance contributes to the sound color, which is important in house and techno music [19], [38] (Section 3.8).

2.3.6. Beats per Minute

When producing house and techno tracks, the Beats Per Minute (BPM) are set (not monitored) to define the tempo of a song. The BPM are concordantly considered important for the definition of and distinction between house and techno music and their various styles [40], [19] p. 4, [38] p. 102, [12] p. 256, and [49] p. 190. BPM are extracted from each track, not from each time frame, and not from each frequency region.

This yields 21 features, namely, phase, scope, channel correlation, peak, RMS, and crest factor for the broadband signal and three different frequency regions, and the BPM.

3. Method

The HOTGAME corpus shall serve as a dataset to study the music and validate documented statements, narratives, and conclusions through automated content-based audio analysis. Therefore, the corpus has to be representative for the nations and the development over the years. Moreover, the extracted features have to be meaningful descriptors of the considered music. This means that they exhibit significant differences over nation and year. Ideally, the features are mutually independent.

3.1. Corpus

In data quality control, validity and few data duplicates are mostly relevant for material that has been gathered by third parties, like citizen science. As HOTGAME is a first-hand dataset, the validity and few data duplicates are secured but can be retraced with a look at the dataset and the feature extraction algorithms provided in the repository. To evaluate the appropriateness of the sample size, HOTGAME is compared to other music analysis datasets and to the size of the house and techno market of the time.

To test for a good data balance and few missing data, the number of tracks in HOTGAME is compared to the number of dance music labels in Germany and the United States.

3.2. Features

Second, the features are evaluated. Here, a test for feature multicollinearity is applied, using confusion matrices of the features’ mean, median, standard deviation, and skewness values. To test whether the features are explanatory features of the house and techno music of the considered era, a two-way MANOVA is carried out to test whether the features differ significantly over nation, year, and nation*year.

4. Results and Discussion

4.1. Corpus

The corpus contains music from 97 German and 85 US-American labels to avoid a bias towards certain labels. This is necessary, as each techno label has its unique sound [40] p. 108. As various sources agree that in 1995/1996, 1500 techno labels existed worldwide [47] p. 237, [34] p. 159, the number of labels included in HOTGAME is certainly closer to an all-of than a best-of. As both nations had a similar number of record labels, the HOTGAME corpus also contains an even distribution of 4667 German and 4362 US-American tracks.

Figure 1 shows the number of active record labels and the number of tracks in the HOTGAME corpus for each year. In Germany ($p=0.96$, $p<0.00001$) and in the United States ($p=0.95$, $p<0.00001$), a significant correlation between the number of record labels and tracks can be found, indicating that the HOTGAME corpus represents the development of house and techno releases well. The exponential growth over time is in accordance with the growing market [47] p. 237, [34] p. 159.

With almost all labels included, a balance between the two nations, and a correlation between the number of tracks and active record labels, the HOTGAME corpus avoids the dangers of missing data and data imbalance. The small number of female artists in the corpus is in accordance with the low number of releases from female artists on the considered labels, and the low number of female DJs at that time [24] p. 426, [59]. So, the gender bias in the dataset is representative for the actual gender bias at that time. Consequently, the corpus is more suitable for comparing nations, styles, and record labels with each other rather than comparing male with female producers or quantifying the impact of women on house and techno music. However, the HOTGAME corpus and the cited literature may serve as a starting point for gender studies in house and techno music, as the relevance of the gay community [8], [60] chapter 1, [26] p. 83, [5] min. 45, [4] min. 22, [6] p. 19, [14] p. 29, [61] p. 3, [62] min. 63, and [63] and of women [1,2,3] is undeniable.

With 9029 tracks over the course of 11 years, the early house and techno corpus is comparably large. For example, the corpus in [48] contained 4500 tracks over the course of 45 years to analyze the loudness war in mainstream music. In [50], 1841 tracks over the course of 5 years have been analyzed to predict which DJ would play which track. Ref. [52] analyzed about 350 songs over the course of 20 years to find out whether producers or rappers have a greater influence on the sound in hip hop music. Ref. [64] analyzed 55 multi-track recordings to explore the mixing behavior of audio engineers in rock music. One exceptional dataset is the million song dataset, which contains information about one million “western commercial” songs over a period of 90 years [65]. Although the corpus is huge, it contains very few early house and techno tracks. Overall, the HOTGAME corpus is large compared to the number of releases during the considered time period.

Overall, the high number of labels, the balance between nations, and the distribution over the years indicate a sufficiently large sample size and a good data balance.

4.2. Features

Figure 2 shows a correlation matrix of all features’ median values. Clearly, some features’ median values do not exhibit linear relationships with the others, like CrestFactorHigh and bpm. In contrast, PeakMeter and RMS exhibit a high correlation. This is true for the broadband signal and each frequency region. Both also exhibit a medium to high correlation with the PhaseSpace.

In contrast, the correlation matrix of the mean values, as illustrated in Figure 3, shows almost no high correlations. The only exception is a high correlation between RMShigh and CrestFactorHigh.

A similar picture is provided by the correlation matrix of the standard deviations, as illustrated in Figure 4. Here, almost none of the distributions correlate highly with each other. The only exception is the correlation between PeakMeter and PeakMeterMid.

Lastly, the skewness is compared between all features. The result is illustrated in Figure 5. Here, no high correlations can be found at all. This means that there is not a single pair of features that exhibits the same distribution skewness amongst the music corpus.

Some high correlations between the median values imply the risk multicollinearity, i.e., that the features are not orthogonal to each other. In big data analyses, redundancy reduction in terms of feature selection or dimensionality reduction is advisable. However, almost no high correlations are found between the features’ mean values, standard deviations, and skewness values, indicating the fair independence of the features and variation among the distributions.

The recording studio features were already selected because the literature indicates that the sound parameters that they quantify are highly relevant for house and techno music [19,26,31,38,40,48,49]. Two-way MANOVA reveals if the features are indeed explanatory in terms of capturing the difference between nation, year, and nation*year. This is expected, as the literature suggests that house and techno music evolved differently in Germany and the United States of America. The result is that the extracted features differ significantly between nation ($F(21,9005)=71.3565$, Wilk’s $\lambda =0.8573$, $p<0.00001$), year ($F(21,9005)=136.2374$, Wilk’s $\lambda =0.7589$, $p<0.001$), and nation*year ($F(21,9005)=71.55944$, Wilk’s $\lambda =0.8570$, $p<0.001$). This means that the extracted features seem explanatory for early house and techno music and can be utilized for further exploration and analyses of the HOTGAME corpus.

4.3. Limitations

Naturally, the HOTGAME corpus has some limitations. It has to be kept in mind that many house and techno tracks have never been released ([49] p. 185). They are missing in the corpus, just as bootlegs and mashups, which may have been influential, too. And clearly, the music corpus does not reveal which tracks have been played a lot in night clubs and which had no impact. Naturally, US-American music has been sold and played in Germany and vice versa. And DJs from both nations may have been playing in various countries and became part of a scene in a foreign country. And of course, music from other nations has been played in both countries, too. The two nations are not rigid classes, nor isolated scenes. Last but not least, music is more than just tracks. Researchers should analyze the HOTGAME corpus in relation to aspects of biographies, (sub-)culture, scene, fashion, zeitgeist, economics, politics, infrastructure, etc.

5. Conclusions

This paper introduced HOTGAME, a corpus of early HOuse and Techno music from Germany and AMErica. Placing the dataset in relation to the existing literature, the market, and other datasets indicated that the source material and the extracted features are representative and explanatory for the house and techno music of the considered time and nations. The gender bias is in accordance with the general house and techno market. The features are not perfectly independent and may require redundancy reduction. Then, the path is clear for the content-based exploration and examination of house and techno music. Researchers can add even more tracks to the dataset. The feature extraction algorithm is provided in the repository, and the presented quality control methods provide metrics for the corpus validity.