Exploring the Effects of Pitch Layout on Learning a New Musical Instrument ^†

Featured Application

The results obtained in this paper are applicable to the design of new musical instruments intended to facilitate the learning and playing of music.

Abstract

Although isomorphic pitch layouts are proposed to afford various advantages for musicians playing new musical instruments, this paper details the first substantive set of empirical tests on how two fundamental aspects of isomorphic pitch layouts affect motor learning: shear, which makes the pitch axis vertical, and the adjacency (or nonadjacency) of pitches a major second apart. After receiving audio-visual training tasks for a scale and arpeggios, performance accuracies of 24 experienced musicians were assessed in immediate retention tasks (same as the training tasks, but without the audio-visual guidance) and in a transfer task (performance of a previously untrained nursery rhyme). Each participant performed the same tasks with three different pitch layouts and, in total, four different layouts were tested. Results show that, so long as the performance ceiling has not already been reached (due to ease of the task or repeated practice), adjacency strongly improves performance accuracy in the training and retention tasks. They also show that shearing the layout, to make the pitch axis vertical, worsens performance accuracy for the training tasks but, crucially, it strongly improves performance accuracy in the transfer task when the participant needs to perform a new, but related, task. These results can inform the design of pitch layouts in new musical instruments.

1. Introduction

Designers of new musical instruments can often be concerned with ensuring accessibility for users either with no previous musical experience, or for those who already have training in another instrument, so that they can easily alter or learn new techniques. Several claims regarding the optimal pitch layout of new musical instruments or interfaces have been made, but as yet there is little empirical investigation of the factors that may enhance or disturb learning and performance on these devices. Our previous conference paper detailed the impact of adjacency and shear on pitch accuracy for the transfer task [1]; in this paper, we take a more comprehensive approach by also considering timing accuracy, and the training and retention tasks.

1.1. Isomorphic Layout Properties

Since the nineteenth century, numerous music theorists and instrument builders have conjectured that isomorphic pitch layouts provide important advantages over the conventional pitch layouts of traditional musical instruments [2,3,4,5]. Indeed, a number of new musical interfaces have used isomorphic layouts (e.g., Array Mbira [6], Thummer [7], AXiS-49 [8], Musix Pro [9], LinnStrument [10], Lightpad Block [11], Terpstra [12]).

An isomorphic layout is one where the spatial arrangement of any set of pitches (a chord, a scale, a melody, or a complete piece) is invariant with respect to musical transposition. This contrasts with conventional pitch layouts on traditional musical instruments; for example, on the piano keyboard, playing a given chord or melody in a different transposition (e.g., in a different key) typically requires changing fingering to negotiate the differing combinations of vertically offset black and white keys.

Isomorphic layouts also have elegant properties for microtonal scales, which contain pitches and intervals “between the cracks” of the piano keyboard [13]. Although strict twelve-tone equal temperament (12-TET) is almost ubiquitous in contemporary Western music, different tunings are found in historical Western and in non-Western traditions. Isomorphic layouts may, therefore, facilitate the performance of music both within and beyond conventional contemporary Western traditions.

One elegant property relevant to non-standard tunings is that, unlike the piano keyboard, isomorphic layouts do not have an immutable periodicity in their spatial structure. On the piano keyboard, only scale systems that repeat every twelve pitches can be intuitively mapped to its keys. Conversely, isomorphic layouts provide consistent spatial representations of scales regardless of their periodicity. This matters because there are many useful tuning systems that do not repeat every 12 chromatic pitches, such as meantone tunings in 19-TET or 31-TET, which are suitable for conventional Western music but provide better approximations to just intonation than the standard 12-TET; Bohlen-Pierce scales, which repeat every 13 equal divisions of the 3/1 tritave (instead of the standard 2/1 octave); the Javanese Pelog system, which is often approximated by a 7-pitch scale in 9-TET; and numerous other scale systems [14].

In this paper, we do not compare isomorphic and non-isomorphic layouts. Instead, we focus on how different isomorphic layouts impact on learning. This is because there are an infinite number of unique isomorphic layouts (and a large number that are practicable for conventionally tuned diatonic-chromatic music): they all share the property of transpositional invariance (by definition) but they differ in a number of other ways that may plausibly impact their usability. For example, successive scale pitches, such as C, D, and E, are spatially adjacent in some isomorphic layouts while in others they are not; additionally, in some isomorphic layouts, pitches are perfectly correlated to a horizontal or vertical axis while in others they are not [15]. With respect to the instrumentalist, the “horizontal” axis runs from left to right, the “vertical” axis from bottom to top or from near to far. In some layouts, octaves may be vertically or horizontally aligned; in others, they are slanted. Properties such as a vertical pitch axis or a vertical octave axis, or adjacent major seconds, and so forth, may be conjectured as desirable (or undesirable): either way, they are typically non-independent because changing one (e.g., pitch axis orientation) may change another (e.g., octave axis orientation). Choosing an optimal layout thus becomes a non-trivial task that requires knowledge of the relative importance of the different properties. However, due to their non-independence, it is challenging to investigate the relative importance of these features experimentally.

To address this, the experiment presented in this paper explores how two independent spatial transformations of isomorphic layouts—shear and adjacency—impact on learning in a set of melody retention and transfer tasks. The shear is used to manipulate the angles of the pitch axis and major second axis, while keeping the octave axis constant; the adjacency manipulation determines whether or not major seconds are spatially adjacent. These two transformations enable us to test our hypotheses that adjacent major seconds and a vertical pitch axis facilitate the learning and playing of melodies.

The four layouts that result from these transformations are illustrated in Figure 1a–d. Each figure shows how pitches are positioned, and the orientation of three axes that we hypothesize will impact on the layout’s usability. Each label indicates whether the layout has adjacent major seconds or not (A and $A^{\prime }$, respectively) and whether it is sheared or not (S and $S^{\prime }$, respectively). The three axes are the pitch axis, the octave axis, and the major second axis, as now defined (the implications of these three axes, and why they may be important, are detailed in Section 1.1.2).

• The pitch axis is any axis onto which the orthogonal (perpendicular) projections of all button centres are proportional to their pitch; for any given isomorphic layout, all such axes are parallel [16] (see the caption for Figure 1 for a practical demonstration of how this works).
• The octave axis is here defined as any axis that passes through the closest button centres that are an octave apart.
• The major second axis (M2 axis, for short) is here defined as any axis that passes through the closest button centres that are a major second apart.

When considering tunings different to 12-TET (e.g., meantone or Pythagorean), alternative—but more complex—definitions for the octave and M2 axes become useful.

1.1.1. Adjacent (A) or NonAdjacent ($A^{\prime }$) Seconds

Scale steps (i.e., major and minor seconds) are, across cultures, the commonest intervals in melodies [17]. It makes sense for such musically privileged intervals also to be spatially privileged. An obvious way of spatially privileging intervals is to make their pitches adjacent: this makes transitioning between them physically easy, and makes them visually salient. However, when considering bass or harmony parts, scale steps may play a less important role. This suggests that differing layouts might be optimal for differing musical uses.

The focus of this experiment is on melody so, for any given layout, we tested one version where all major seconds are adjacent and an adapted version where they are nonadjacent (minor seconds were nonadjacent in both versions). Both types of layouts have been used in new musical interfaces; for example, the Thummer (which used the Wicki layout (Figure 1b) had adjacent major seconds, while the AXiS-49 (which uses a Tonnetz-like layout [18]) has nonadjacent seconds but adjacent thirds and fifths.

1.1.2. Sheared (S) or Unsheared ($S^{\prime }$)

We conjecture that having any of the above-mentioned axes (pitch, octave, and M2) perfectly horizontal or perfectly vertical makes the layout more comprehensible: if the pitch axis is vertical or horizontal (rather than slanted), it allows for the pitch of buttons to be more easily estimated by sight, thereby enhancing processing fluency. Similar advantages hold for the octave and M2 axes: scales typically repeat at the octave, while the major second is the commonest scale-step in both the diatonic and pentatonic scales that form the backbone of most Western music.

However, changing the angle of one of these axes requires changing the angle of one or both of the others, so their independent effects can be hard to disambiguate. A way to gain partial independence of axis angles is to shear the layout parallel with one of the axes—the angle of the parallel-to-shear axis will not change while the angles of the other two will. A shear is a spatial transformation in which points are shifted parallel to an axis by a distance proportional to their distance from that axis. (For example, shearing a rectangle parallel to an axis running straight down its middle produces a parallelogram; the sides that are parallel to the shear axis remain parallel to it, while the other two sides rotate). As shown by comparing Figure 1a with Figure 1c, or by comparing Figure 1b with Figure 1d, we used a shear parallel with the octave axis to create two versions of the nonadjacent layout and two versions of the adjacent layout: each unsheared version ($A^{\prime }{S}^{\prime }$ or $A{S}^{\prime }$) has a perfectly horizontal M2 axis but a slanted (non-vertical) pitch axis; each sheared version version ($A^{\prime }S$ or $AS$) has a slanted (non-horizontal) M2 axis but a vertical pitch axis. In both cases the octave axis was vertical.

In this investigation, therefore, we remove any possible impact of the octave axis orientation; we cannot, however, quantitatively disambiguate between the effects of the pitch axis and the M2 axis.

Unsheared layouts are common in new musical interfaces because these typically use buttons arranged in a perfectly square or hexagonal array; we are not aware of a hardware interface that makes use of shear to make the pitch axis vertical or horizontal (although this is a design feature of the software MIDI sequencer Hex [15]).

1.2. Motor Skill Learning in Music Performance

Learning a new musical instrument requires a number gross and fine motor skills in order to physically play a note. This is often carried out in tandem with sensory processing of feedback from the body and of auditory features (e.g., melody, rhythm, timbre) in order to learn how to play specific sequences [19]. For the purposes of our experiment, by using musically-trained participants and sequences familiar to those musicians such as scales and arpeggios, we reduce this to a motor learning problem. How best can musicians learn to play on a new pitch layout?

In learning a motor skill there are three general stages [20]:

• A cognitive stage, encompassing the processing of information and detecting patterns. Here, various motor solutions are tried out, and the performer finds which solutions are most effective.
• A fixation stage, when the general motor solution has been selected, and a period commences where the patterns of movement are perfected. This stage can last months, or even years.
• An autonomous stage, where the movement patterns do not require as much conscious attention on the part of the performer.

Essentially, learning the motor-pitch associations of a new instrument requires the performer to perceive and remember pitch patterns. Once these pitch patterns are learned, the performer becomes more focused on eliminating various sources of motor error. Because achieving motor autonomy is a lengthy process—one that can seldom be captured by short-term experiments—our current study focuses on only the first two elements of motor learning.

Learning a pattern of actions and their associated responses can be affected by pre-existing action-response representations: essentially, the anticipated effects of an action have an influence on the performance of that action; for example, reaction time is faster when participants are instructed to press a button forcefully and this elicits a loud tone, rather than when the effect is not compatible with the action (e.g., a soft tone) [21]. Therefore, it may also hold that pre-existing expectations of the pitch effects of a sequence of actions may have an influence on the performance of that sequence.

Research into the Spatial-Musical Association of Response Codes (or SMARC effect) demonstrates not only a vertical alignment (increasing pitch height is mapped vertically from low to high), but also a horizontal alignment (increasing pitch height is mapped horizontally from left to right) in musically trained participants [22]. This horizontal effect is far more subtle in non-musicians [23] and in some cases non-existent [24,25], suggesting that musical training enhances this particular spatial dimension. It is posited that this may be a learned-association effect [26]. These pitch representations have been shown to influence motor planning and action. Keller and colleagues found that, for a sequence of three consecutive keypresses, timing was more accurate when the produced tones were compatible with the pre-existing associations that increasing vertical movement results in an increase in pitch height [27,28]. This appears to be evident across different levels of expertise (non-musicians and trained musicians), although, as expected, training enhances the strength of this existing representation [29]. We investigate only 2-dimensional pitch layouts, so do not consider the implications of Shepard’s helical model of pitch perception [30], which requires a cylindrical—hence 3-dimensional—form [31].

The tendency in the pitch-motor representation literature has been to reverse or scramble pitches from the traditional down-to-up or left-to-right assignment. Although many new pitch layouts may not violate this basic learned pitch-motor association, adjustments to the learned general motor pattern may still be required depending on the spacing of intervals, and the precise orientation of the pitch axis. Stewart and colleagues [26] demonstrated an effect on reaction time in a task using “normal” versus “stretched” representations of pitch along a horizontal axis (sequences which did or did correspond to a learned pattern of movement that could be played with the fingers of a single hand). This suggests that, despite their similarity to other layouts (both “normal” and “stretched” satisfied the left-right horizontal sequence), the patterns of notes may have fundamentally changed for the performer, and so require a certain amount of motor learning in this new (but clearly related) task.

It seems plausible then that certain aspects of a new layout, within the realm of satisfying the vertical and horizontal SMARC effects, will facilitate such learning, while others may hinder it. These aspects may be related to (a) previously learned pitch-motor mappings; (b) ergonomic issues, such as the physical ease of making the motions required to play the target pitches, and also from (c) processing fluency, such as how easy it is to see or sense, by proprioception, musical features that are relevant to the task. As detailed in Section 1.1.1 and Section 1.1.2, in this experiment, we focus on the last of these and, in particular, on two musical attributes that are important for melodies and two spatial attributes that have a plausible impact on processing fluency. The musical attributes are major seconds (important because of their prevalence in melodies and musical scales) and pitch height. The spatial attributes are verticality (we hypothesize that perfectly vertical, or horizontal, lines are easier to imagine than are slanted lines) and adjacency (we hypothesize that it is generally easier to find a spatially adjacent pitch than one that is separated). The experimental manipulation, therefore, involves participants learning and playing pitch layouts with vertical versus slanted pitch axes, and adjacent versus nonadjacent major seconds.

To test how well the participants have learned the new layouts and perfected their motor pattern, we are particularly interested in the transfer of learning from one task to another. For instance, a piano player will practice scales not only to achieve good performance of scales, but also to fluently play scale-like passages in other musical pieces. In our study, we designed a training and testing paradigm for the different pitch layouts such that the transfer task involved a previously unpracticed, but familiar (in pitch) melody.

1.3. Study Design

For this experiment, we were interested in examining how features of a pitch layout affected performance accuracy in the learning of a new motor pattern, how this skill was retained at test immediately after training, and performance accuracy in transfer of this skill to a new, untrained task. Musically experienced participants played three out of the four layouts under consideration (see Figure 1): all 24 participants played both $A{S}^{\prime }$ and $AS$, with 12 participants each playing either $A^{\prime }S$ or $A^{\prime }{S}^{\prime }$.

The independent variables were

• Adjacency $\in \{0,1\}$, where 0 is the code for a layout with non-adjacent major seconds ($A^{\prime }{S}^{\prime }$ or $A^{\prime }S$), and 1 is the code for a layout with adjacent major seconds ($A{S}^{\prime }$ or $AS$).
• Shear $\in \{0,1\}$, where 0 is the code for an unsheared layout ($A^{\prime }{S}^{\prime }$ or $A{S}^{\prime }$), and 1 is the code for a sheared layout ($A^{\prime }S$ or $AS$).
• LayoutNo $\in \{0,1,2\}$, where 0 is the code for the first layout played by a participant, 1 is the code for the second layout they played, and 2 is the code for the third and final layout they played.
• PerfNo $\in \{0,1,2,3\}$, where 0 is the code for their first performance of a given layout, 1 is the code for their second performance of a given layout, 2 is the code for their third performance of a given layout, 3 is the code for their fourth performance of a given layout. Note that participants gave three performances for the training, two performances for the immediate retention tasks, and four performances for the transfer task.

Each participant played the layouts in one of four different sequences, and each such sequence was played by 6 participants:

• $A{S}^{\prime }$ then $A^{\prime }{S}^{\prime }$ then $AS$
• $A{S}^{\prime }$ then $A^{\prime }S$ then $AS$
• $AS$ then $A^{\prime }{S}^{\prime }$ then $A{S}^{\prime }$
• $AS$ then $A^{\prime }S$ then $A{S}^{\prime }$.

This means that the nonadjacent seconds layouts ($A^{\prime }{S}^{\prime }$ and $A^{\prime }S$) were always presented second, and that participants who started with the unsheared adjacent layout ($A{S}^{\prime }$) finished with the sheared adjacent layout ($AS$), and vice versa.

In each such layout, participants received an equivalent training and testing program: first for the C major scale, then for arpeggios of all triads in C major. The scale task was used to support the learning of the spatial patterns of seconds in the diatonic scale; the arpeggios to support the learning of the spatial patterns of larger intervals such as thirds and fourths in the diatonic scale. Immediate retention (performance without any audiovisual training) was tested after each task. The transfer task required participants to perform a well-known melody (Frère Jacques) for which they had received no prior training. This melody contains numerous major and minor seconds but also larger intervals. Participants were given 20 s to practice before their performances were recorded. These procedures are further detailed in Section 2.2, Section 2.3 and Section 2.4.

Participants’ preferences were elicited in a semi-structured interview, a detailed analysis of which is available in [1]. The current paper will fully describe the results of the performances of training and testing materials (both retention and transfer tasks), assessed for their inaccuracy in terms of number of incorrect notes as well as the timing of the performed notes in comparison to either the audiovisual sequence (training) or the metronome beat (retention and transfer).

2. Results

2.1. Modelling Approach

In order to determine effect sizes and significances of the independent variables (Adjacency, Shear, LayoutNo, PerfNo), these variables were regressed on the dependent variable Inaccuracy. The method used to calculate Inaccuracy (detailed in Section 4.4) takes account of both pitch errors and timing errors in participants’ performances. Due to fundamental differences in the training tasks compared with the test tasks (retention and transfer), the method for calculating their respective Inaccuracy values differs (see Section 4.4). In both cases, however, Inaccuracy takes only positive values, because a value of 0 implies a perfect performance.

Initially, linear mixed effects models were fitted to the Inaccuracy data. (Mixed effects models are regressions where participants’ coefficients are treated as samples from a single, and best-fitting, multivariate normal distribution. Given this distribution, the means of the coefficients are termed fixed effects, their (co)variances are termed random effects. For within subject designs, mixed effects models are widely recommended to avoid Type I errors [32]).

However, the residuals were skewed and heteroscedastic. For this reason, a generalized linear mixed effects model with a gamma distribution and a log link was used—this combination of distribution and link function providing the best fit to the data. (Generalized linear models are designed to cope with data that do not meet the assumptions required by linear regression models; notably, that the residuals are homoscedastic and symmetric. These violations typically occur when the data can take only a subset of real values, such as Inaccuracy, which can take only positive values. For positive-valued data, the gamma distribution with a log link is commonly used. The link function transforms the mean of the chosen distribution so that it is linearly related to the independent variables [33].)

The log link means that the exponential of each predictor’s fixed effect value represents the multiplicative factor by which Inaccuracy changes for each unit increase of that predictor (all else being equal). For example, a fixed effect of $0.5$ means that a unit increase in that predictor multiplies Inaccuracy by $exp\left(0.5\right)=1.649$; put differently, Inaccuracy increases by 64.9%. In the subsequent tables, this exponential value is shown in the “Factor” column.

In each model, the intercept was included as a random effect grouped by participant to allow it to take account of participants’ differing abilities. Generally, maximal random effects structures [32] (with all fixed effects and their interactions included as random effects) were attempted, but such models failed to converge, even after removing interactions. The resulting models, therefore, assume that the independent variables are invariant across participants in the population, but that particpants’ overall ability does vary.

The experimental design was unbalanced because nonadjacent layouts occurred only during the second layout (i.e., $\mathit{Adjacency}=0\iff \mathit{LayoutNo}=1$). For this reason, the predictor Adjacency was not included in any interactions. Including Adjacency in interactions results in rank-deficient design matrices or in conditional effects for lower-order terms that are hard to interpret; for example, if the interactions Shear:Adjacency and Shear:LayoutNo are included, then the conditional effect for Shear refers to the effect of Shear when $\mathit{LayoutNo}=0$ and $\mathit{Adjacency}=0$, which never actually occurs in the experiment. All other possible interactions were included in the initial model. To simplify interpretation of lower-order terms, any nonsignificant 3-way interactions were removed from the model, and the model was refitted. For the same reason, any nonsignificant 2-way interactions not part of a 3-way interaction were then removed, and the model refitted.

Classical t-tests of significance in mixed effects models are considered to be anti-conservative (prone to incorrectly low p-values); for this reason, each predictor’s p-value (as shown in the subsequent summary tables) was estimated through theoretical (${\chi }^2\left(1\right)$) tests of the likelihood ratio between the model with that predictor and the model without that predictor. All models were fitted with the lme-4 package in R, and p-values estimated by the drop1 function using the Chisq option, or the anova function. The $r^2$ value shown in each table is the squared correlation between the model’s predictions and the observed data, hence loosely analogous to $R^2$ in linear regression.

2.2. Training Performances

Participants performed two training tasks: (1) Scales, and (2) Arpeggios. For each layout, participants completed three training performances of the scale, and three of the arpeggios. The accuracy of performances in the two training tasks, averaged across participants, are summarized in Figure 2 and Figure 3 as a function of the independent variables detailed in Section 1.3. The 95% confidence intervals were obtained with 100,000 bootstrap samples, calculated and plotted in MATLAB. The generalized linear mixed effects models used to estimate the effects’ sizes and significances of Adjacency, Shear, LayoutNo, and PerfNo are summarized in

Table 1. Generalized linear mixed effects model for the scale training task.

Fixed Effect	Estimate	Factor	p-Value
(Intercept)	3.48	32.61	<0.001 ***
Adjacency	−0.06	0.94	0.263
Shear	0.13	1.14	0.026 *
LayoutNo	−0.37	0.69	0.001 ***
PerfNo	−0.27	0.77	<0.001 ***
LayoutNo:PerfNo	0.12	1.13	0.003 **
Log Likelihood	−727.67
Num. obs.	208
Num. groups: ID	24
Var: ID (Intercept)	0.04
Var: Residual	0.17
$r^2$	0.44

*** p < 0.001, ** p < 0.01, * p < 0.05.

and

Table 2. Generalized linear mixed effects model for the arpeggio training task.

Fixed Effect	Estimate	Factor	p-Value
(Intercept)	3.92	50.36	<0.001 ***
Adjacency	−0.43	0.65	<0.001 ***
Shear	0.08	1.08	0.037 *
LayoutNo	−0.22	0.80	<0.001 ***
PerfNo	−0.09	0.91	<0.001 ***
Log Likelihood	−738.65
Num. obs.	209
Num. groups: ID	24
Var: ID (Intercept)	0.05
Var: Residual	0.08
$r^2$	0.78

*** p < 0.001, ** p < 0.01, * p < 0.05.

.

The significant coefficient for Shear ($1.14$) indicates that sheared layouts worsen performance accuracy in the scale training task: they multiply inaccuracy by 1.14 (increase it by 14%). Looking at Figure 2, it is apparent that this effect is arising mainly in the non-adjacent layouts—the accuracies for $A^{\prime }S$ (dashed orange) are lower than the accuracies for $A^{\prime }{S}^{\prime }$ (dashed blue). Note that the interaction between Adjacency and Shear was not tested for the reasons explained in Section 2.1.

The significant coefficients for LayoutNo ($0.69$), PerfNo ($0.77$), and their interaction LayoutNo:PerfNo ($1.13$), indicate that accuracy in the scale training task improves over successive layouts and performances, but less so when either is high. In Figure 2, the positive gradient of the lines in the first layout block (0/0, 1/0, 2/0) appear greater than those of the second (0/1, 1/1, 2/1) and third (0/2, 1/2, 2/2) blocks; the inaccuracy measured at the beginning of each layout block (performance 0) also appears to increase across layout blocks 0–2, but less so with performances 1 and 2 (the middles and ends of each layout block).

The significant coefficient for Adjacency ($0.65$) indicates that adjacent layouts strongly improve performance accuracy in the arpeggio training task: they multiply inaccuracy by $0.65$ (decrease it by 35%). This is visible in Figure 3 where the dashed lines in the second layout block are lower than an imaginary line connecting the first and third blocks.

The significant coefficient for Shear ($1.08$) indicates that sheared layouts slightly worsen accuracy in the arpeggio training task: they multiply inaccuracy by $1.08$ (increase it by 8%). Looking at Figure 3, it is apparent that this effect is arising mainly in the non-adjacent layouts—the accuracies for $A^{\prime }S$ (dashed orange) are lower than the accuracies for $A^{\prime }{S}^{\prime }$ (dashed blue). Note that the interaction between Adjacency and Shear was not tested for the reasons explained in Section 2.1.

The significant coefficients for LayoutNo ($0.80$) and PerfNo ($0.91$) indicate that accuracy in the scale training task improves over successive layouts and performances (inaccuracy is reduced by 20% and 9%, respectively). This is shown, in Figure 3, by the positive gradient across the three performances in the majority of the layouts presented, and the general increase in accuracy across the three layout blocks (bearing in mind the drop in the second block resulting from the effects of non-adjacency).

2.3. Immediate Retention

Performances to evaluate immediate retention were recorded for the two trained tasks (1) Scales; (2) Arpeggios. Test performance results are reported here only for item 1 because most of the test performances of item 2 had too many errors in pitch and timing to be reliably tracked with respect to the target pitches. The distinctly lower performance accuracy here may reflect the greater musical complexity of the arpeggios compared to the scales and therefore increased difficulty in memorising the sequence (compare Figures 7 and 8 in Section 4.2.2). It may also be because the tested pitch layouts are not as well suited to the arpeggios as they are to the scales. The results for the arpeggio training task—detailed in Section 2.2—are, however, still useful in the comparison with the scale training task.

For each layout, participants completed two retention performances of the scale without audiovisual support. The accuracy of these performances, averaged across participants, are summarized in Figure 4. The generalized linear mixed effects model used to estimate the effects’ sizes and significances of Adjacency, Shear, LayoutNo, and PerfNo is summarized in

Table 3. Generalized linear mixed effects model for immediate retention of the scale task.

Fixed Effect	Estimate	Factor	p-Value
(Intercept)	2.85	17.32	<0.001 ***
Adjacency	−0.54	0.58	<0.001 ***
Shear	−0.08	0.92	0.305
LayoutNo	−0.04	0.96	0.412
PerfNo	0.04	1.04	0.581
Log Likelihood	−437.37
Num. obs.	142
Num. groups: ID	24
Var: ID (Intercept)	0.06
Var: Residual	0.26
$r^2$	0.38

*** p < 0.001, ** p < 0.01, * p < 0.05.

.

The significant coefficient for Adjacency ($0.58$) indicates that adjacent layouts strongly improve performance accuracy in the scale retention task: they multiply inaccuracy by $0.58$ (decrease it by 42%). In Figure 4, note how the accuracies for the non-adjacent layouts $A^{\prime }{S}^{\prime }$ and $A^{\prime }S$ (dashed lines) are lower than the accuracies in the adjacent layouts.

2.4. Transfer

Transfer of learning was evaluated by performances of a separate test melody that participants had not received training for: Frère Jacques. For each layout, participants performed the transfer task for four consecutive performances.

The accuracy of performances for the transfer task, averaged across participants, are summarized in Figure 5. The generalized linear mixed effects model used to estimate the effects’ sizes and significances of Adjacency, Shear, LayoutNo, and PerfNo is summarized in

Table 4. Generalized linear mixed effects model for the transfer task (Frère Jacques).

Fixed Effect	Estimate	Factor	p-Value
(Intercept)	3.43	30.85	<0.001 ***
Adjacency	−0.05	0.96	0.408
Shear	−0.50	0.61	<0.001 ***
LayoutNo	−0.38	0.68	<0.001 ***
PerfNo	−0.13	0.88	<0.001 ***
LayoutNo:Shear	0.33	1.39	0.002 **
LayoutNo:PerfNo	0.07	1.07	0.021 *
Log Likelihood	−996.89
Num. obs.	288
Num. groups: ID	24
Var: ID (Intercept)	0.11
Var: Residual	0.26
$r^2$	0.66

*** p < 0.001, ** p < 0.01, * p < 0.05

.

The significant coefficient for Shear ($0.61$) indicates that, for the first layout, which is always adjacent, sheared layouts strongly improve accuracy in the transfer task: they multiply inaccuracy by $0.61$ (decrease it by 39%). The significant coefficient for the interaction LayoutNo:Shear (1.39) indicates that the positive effect of shear vanishes in the second and third layouts. In Figure 5, note how, in the first layout block (0/0, 1/0, 2/0, 3/0), the accuracies for the sheared layout $AS$ (orange) are higher than the unsheared layout $A{S}^{\prime }$ (blue) but, in the second or third layout blocks, the accuracies for the sheared layouts ($AS$, $A^{\prime }S$—orange) are, if anything, slightly lower than those for the unsheared layouts ($A{S}^{\prime }$, $A^{\prime }{S}^{\prime }$—blue).

The significant coefficients for LayoutNo ($0.68$), PerfNo ($0.88$), and their interaction LayoutNo:PerfNo ($1.07$) indicate that accuracy in the transfer task improves over successive layouts and performances, but less so when either is high. In Figure 5, the positive gradient of the slopes in the first layout block (0/0, 1/0, 2/0, 3/0) appear greater than those of the second (0/1, 1/1, 2/1, 3/1) and third (0/2, 1/2, 2/2, 3/2) blocks; the inaccuracy measured at the beginning of each layout block (performance 0) also appears to increase across layout blocks 0–2, but less so with later performances.

3. Discussion

3.1. PerfNo and LayoutNo

Three out of the four tasks (the two training tasks and the transfer task) show a strong positive effect of PerfNo and LayoutNo. Both effects are indicative of learning. The former indicates that the more a participant plays on a given layout the better they get (until the ceiling is reached). The latter indicates that learning in one layout extends to different layouts. This is unsurprising given the similarity of the four different layouts: they all exhibit a “3 + 4” scale pattern where, to play a major scale, a row of three buttons is played from left to right, then there is a “carriage return” to the next row above where a row of four buttons is played followed by a “carriage return” to the next row above, and the pattern starts again (in the next higher octave). The immediate retention of the scale task does not show any impact of PerfNo or LayoutNo. This is clearly due to the simplicity of the task—participants are close to their ceiling from the very start.

3.2. Adjacency

In two out of the four tasks (the arpeggio training task and the transfer task), Adjacency has a strong positive effect (35% and 42% decreases in inaccuracy). The statistical results from the other two tasks (see also Figure 2 and Figure 5) also hint at this positive effect but the values are not significant. It seems that any impact of adjacency in the transfer task is swamped by the learning that has already occurred by the time the adjacent layout is played (remember that an adjacent layout always came second for all participants and never first or third). A future experiment that balances the design, by also having nonadjacent layouts first and third, will have greater statistical power to detect the impact of adjacency in this context. Judging the results in total, we have strong evidence that adjacent major seconds improve playing accuracy in a variety of one-handed melodic tasks. Although this could be a result of the high number of major seconds present in both the scales and the nursery rhyme chosen for the transfer task, the positive effect of adjacency effect is also seen in the arpeggio task, which has a large emphasis on other intervals—notably major and minor thirds, and perfect fourths. Here, we might have expected the adjacent layout to be no more useful than the nonadjacent layout. However, the findings suggest that successfully learning major seconds may also provide a useful foundation for learning and playing other intervals. Hence, adjacent major seconds may be a broadly useful property.

3.3. Shear

The impact of Shear is a little more complicated. In both of the training tasks, it has a significant negative effect on playing accuracy (14% and 8% increases in inaccuracy). It has no significant effect in the immediate retention of the scale task. However, for the principal data collected in this experiment—the Frère Jacques transfer task—it has a strong positive effect for the first (and adjacent) layout presented. The reason this effect does not carry through to the second and third layouts presented may be due to performances hitting ceiling (as appears to be the case by looking at Figure 5); they may also be due to shear having a positive effect with adjacent layouts but not with nonadjacent layouts (as explained in Section 2.1, due to the experimental design, interactions with Adjacency were not included and, hence, not directly tested).

It remains to account for why shear may worsen performances in simple training tasks, but improve them when playing a new melody. We hypothesize that the slanted runs of major seconds in the sheared layouts provide an impediment to accuracy due to the slant making movement physically more difficult or confusing due to its unfamiliarity compared with the piano keyboard. However, in a task like playing a new melody, which requires finding correct pitches without having previously learned a sequential pattern, and has a more complex and variable sequence of movements, the additional clarity of the vertical pitch axis in the sheared layouts becomes more important and trounces the previously mentioned disadvantages. This hypothesis matches the outcomes we see, but would require further testing using differing playing tasks with differing requirements for quickly accessing and recognizing relative pitch heights.

3.4. Limitations

There are some limitations of the current experiment. First, the nonadjacent layouts ($A^{\prime }S$ and $A^{\prime }S$) occurred only second, which means that in the transfer task the impact of adjacency was overwhelmed by learning effects; it also means that the full range of interactions between the variables could not be tested. Secondly (as detailed in Section 1.1.2), by shearing parallel with the octave axis, the angles of the pitch axis and the angle of the major second axis covaried, so these two effects cannot be disambiguated; furthermore, the angle of the octave axis was invariant (it was always vertical) so not tested. Thirdly, only melodies were tested—we would expect different results for bass lines (where perfect fifths and fourths are prevalent) or for harmony, where thirds are also prevalent. Fourthly, all of the pitch layouts tested were rather similar in their overall form—this similarity is a natural consequence of good experimental design, which requires changing only the variables of interest across conditions, but it may limit generalizability. Finally, other covarying aspects of the layouts were not independently tested (e.g., adjacencies or angles of other intervals).

Most of these limitations can be overcome by conducting further experiments, although it would not be feasible to address all of them at once. For instance, the nonadjacent layouts could be presented first and third; the shears could be made parallel with the pitch axis or with the major second axis—doing both, in conjunction with this experiment, would enable the effects of the pitch, octave, and M2 axes’ angles to be fully disambiguated; differing musical contexts, such as chord progressions or bass lines, could be trained and tested; a wider range of layouts, with configurations distinctly different from those considered here, could be used.

With respect to the results, limitations can be seen in the ceiling effect achieved quickly for some of the tasks, particularly with the second and third layouts played. The experiment allowed for training with audiovisual performances for 4 separate instances (one watching the audiovisual sequence, and the remaining three playing along with the sequence). For the scale task, this was sufficient for participants to excel in the immediate retention test; conversely, for the arpeggio task, this may not have been enough training because performances in the immediate retention were too poor to analyse. For the transfer task, 20 s of exploration was allowed prior to the first performance on each layout. This is not a standard figure, but was used to avoid a floor performance for this untrained task. However, for the second and third layouts, this may have helped participants too much because ceiling performance was reached quickly. Selecting training period lengths and tasks with the appropriate difficulty are important in the design of learning experiments. A task of appropriate difficulty can ensure that a learning effect can be measured.

3.5. Summary

In broad terms, the results demonstrate that the precise form of a pitch layout has a crucial impact on its effectiveness, hence the importance of uncovering the underlying properties—such as interval adjacency and axis angles—that account for these effects on motor learning. With regard to these two properties, the data support our hypotheses that spatially privileging major seconds by making them adjacent, and making the pitch axis salient by giving it a vertical orientation, facilitates motor learning for melodies played on novel pitch layouts.

The two historically established pitch layouts that have adjacent major seconds and, for 12-TET, also have a pitch axis that is close to vertical or horizontal (and so require only a small shear to perfectly align them) are the Wicki layout [4] and the Bosanquet layout [2,3]. The results obtained here, therefore, suggest that appropriately sheared versions of these two layouts (e.g., the $AS$ layout, illustrated in Figure 1d, which is a sheared Wicki layout) are optimal for playing melodies.

4. Methods

The methods and materials are, for the most part, the same as in [1]. Previously, only the number of correct notes was reported for the transfer task. In this paper, we have developed a new method (detailed in Section 4.4) for measuring the inacuracies of participants’ performances during the training, immediate retention, and transfer tasks.

4.1. Participants

Twenty-four participants were recruited (mean age = 26, age range: 18–44) with at least 5 years of musical experience on at least one instrument (excluding the voice). Ethical approval for this experiment was obtained via the Western Sydney University Human Research Ethics Committee (approval number: H10487).

4.2. Materials

4.2.1. Hardware and Software

The software sequencer Hex [15] was modified to function as a multitouch MIDI controller, and presented on an Acer touchscreen notebook, as shown in Figure 6. Note names were not shown on the interface, but middle D was indicated with a subtly brighter button to serve as a global reference. The position of middle C was indicated to participants, this being the starting pitch of every scale, arpeggio, or melody they played.

In order to present training sequences effectively, both aurally and visually, Hex’s virtual buttons were highlighted in time with a MIDI sequence. All training sequences were at 90 bpm and introduced by a two-bar metronome count.

4.2.2. Musical Tasks

Melodies for musical tasks were chosen to be single-line sequences to be performed solely with the right hand. The training melodies consisted of a set of C major scales (Figure 7) and a set of arpeggios (again only using the notes of the C major scale—Figure 8) spanning two octaves, and all starting and ending on middle C. The well-known nursery rhyme Frère Jacques was used as the transfer task melody (Figure 9); it too was played in C major and began and ended on middle C.

4.3. Procedure

The layouts were presented in four different sequences, with each sequence played by 6 participants: $A{S}^{\prime }$ then $A^{\prime }{S}^{\prime }$ then $AS$; or $A{S}^{\prime }$ then $A^{\prime }S$ then $AS$; or $AS$ then $A^{\prime }{S}^{\prime }$ then $A{S}^{\prime }$; or $AS$ then $A^{\prime }S$ then $A{S}^{\prime }$.

4.3.1. Training Paradigm and Testing of Immediate Retention

For each of their three layouts, participants were directed through a 15 min training and testing paradigm involving (1) scales and (2) arpeggios. For each stage, this involved:

• watching the sequence once as demonstrated by audiovisual highlighting
• playing along with the audiovisual highlighted sequence three times (training)
• reproducing the sequence in the absence of audiovisual highlighting, for two consecutive performances (immediate retention task)

All demonstration sequences and participant performances were played in time with a 90 bpm metronome, and recorded as MIDI files.

4.3.2. Transfer Task

A final production task asked participants to play a well-known nursery rhyme—Frère Jacques. Participants first heard an audio recording of the nursery rhyme to confirm their knowledge of the melody. They were then given 20 s initially to explore the layout and find the correct notes before giving four consecutive performances. Again, these performances were instructed to be played in time with a 90 bpm metronome. Although this represents a fairly simple task, the nursery rhyme was chosen as it facilitated measurement of participants’ skill with each particular layout. We assume that as the participants’ memory for the melody was intact, their performance would only be affected by their memory of the layout itself.

4.4. Measuring Performance Inaccuracy

Separate measures were developed to assess performance inaccuracy in the training tasks (playing along with the audiovisual highlighted sequence), and performance inaccuracy in the retention and transfer tasks (playing along with a metronome beat). Both measures first calculate an Accuracy score by taking account of both the number of “correct” notes played, and the timing of these notes in comparison to either the audiovisual sequence (in the training tasks), or the audio metronome beat (in the retention and transfer tasks). This is then converted into an Inaccuracy score by subtracting Accuracy from the maximum possible Accuracy value (as would be achieved by a flawless performance).

4.4.1. Training Tasks

In the training tasks, the participant played along with the audiovisual highlighted sequence. Here, a “correct” performance would constitute playing the correct pitches at the correct times in the sequence; that is, matching the sequence being produced aurally and visually onscreen. For each target note in the audiovisual demo, a window of 666 ms (the interonset interval at 90 bpm), centred around the target note, was created. The first performed note within this window to match the target’s MIDI note number was used. In this way, inserted notes which did not match the expected MIDI note were not penalized if the “correct” note was played at some point in the time window. If no matching MIDI note number was identified within this window, a zero score was allocated for this target note in the sequence. The first matching MIDI note number was then assigned a score reflecting its timing accuracy—a value of 1, if played at the same time as the target; linearly reducing to 0 as the timing error increases to the boundary of the window ($\pm 333$ ms). The resulting scores are summed. As a final step, the total is normalized by dividing it by the number of notes in the target sequence and multiplied by 100. This puts accuracy onto the same scale for target sequences of different lengths, such that 100 always means the target sequence has been performed perfectly. The formula to calculate a participant’s inaccuracy in the training sequences of the scales and arpeggios is

$$\begin{array}{cc}\hfill Inaccuracy& =100-Accuracy,\mathrm{where}\hfill \\ \hfill Accuracy& =\frac{100}{N}\sum _{n=1}^{N}1-\frac{|t_p-t_n|}{333},\hfill \end{array}$$

(1)

and

• N is the total number of notes in the target sequence
• $t_n$ is the time in milliseconds of nth target note
• $t_p$ is the time in milliseconds of first performed note within the $t_n\pm 333$ time window that matches the pitch of the target note.

4.4.2. Retention and Transfer Tasks

In the immediate retention and transfer tasks, the participant played the test sequence along with a 90 bpm metronome. Here, a “correct” performance would constitute playing the correct notes in the expected sequence, as before, but with one caveat related to penalizing errors. Because these tasks were accompanied only aurally by a metronome beat, the performer might add an extra note or leave a gap so that all subsequent notes are then played one metronome beat late. In a situation such as this, it is reasonable to penalize the first late note, but not the subsequent notes, which are then correct subject to a delay. Similarly, a performer might skip a note, hence all subsequent notes will be a metronome beat early. As before, it is reasonable only to penalize the first pitch error.

To establish which notes of the performed sequence were “correctly” pitched, we used the Note Time Playing Path software [34], which uses a windowing process to identify where extra, skipped, or substituted (wrongly pitched) notes occur. For performances where there were large numbers of pitch errors (>5), this matching process was visually confirmed.

In order to count the total number of errors, we took a novel approach where only the first pitch error in a consecutive sequence of pitch errors was included. For example, consider a performer who plays the first four notes of a scale correctly (C, D, E, F). After this he/she plays three wrong notes in a row (D, D, E), but then returns to the original scale sequence one note after the last correct note (A, B). In this instance of four consecutive errors—three wrong notes (D, D, E) and one deletion (G)—only the first error is counted. The final penalty is then calculated by the total number of such “first” errors, designated by the letter E in Equation (2). This final error value is subtracted from an accuracy score calculated in the same manner as Equation (1) with the exception that it refers only to the correct notes in the performance. As before, the accuracy value is normalized by $100/N$ to ensure a perfect performance of a melody of any length has an accuracy of 100, which means the formula to calculate a participant’s inaccuracy in the retention and transfer tasks is

$$\begin{array}{cc}\hfill Inaccuracy& =100-Accuracy,\mathrm{where}\hfill \\ \hfill Accuracy& =\frac{100}{N}\left(\left(\sum _{c=1}^{C}1-\frac{|t_c-t_m|}{333}\right)-E\right),\hfill \end{array}$$

(2)

and

• N is the total number of notes in the target sequence.
• C is the total number of notes in the corrected performance.
• $t_c$ is the time in milliseconds of the cth note in the corrected performance.
• $t_m$ is the time of the metronome beat closest to the performed note $t_c$.
• E is the number of errors calculated as explained above.

We have defined two versions of Inaccuracy—one for the training tasks, one for the retention and transfer tasks. The context (e.g., in Section 2) will always make clear which of these is being used.