A Transformational Modified Markov Process for Chord-Based Algorithmic Composition

Round 1

Reviewer 1 Report

The paper presents a method for generating harmonic sequences that capture stylistic attributes from musical examples. Its main novelty relies on the exploration of a new alphabet for representing chord sequences as (temporal) interval transformations--notably those related to neo-Riemmanian theory). The temporal modeling of the harmonic content is done by a first-order Markov chain--one of the earliest algorithms adopted to model temporal music structures. The authors also introduce a new method, named UTT, which splits the chord per quality into different n-grams. 

The manuscript is decently structured, and the work is well written, notably on what concerns the notation and formal definition of the method. The use of figures and tables is appropriate and enhances readability. On the other hand, the motivation for the work and its broad musical context is misleading and terminologically incorrect at times (more details below). I believe that the paper has a few shortcomings at the musical level, which limits its potential to reach a wider audience and impact.

The method departs from music analysis methods that have been attracting the attention of the music community since the late-80s. Despite its increasing popularity in music theory circles, the alphabet adopted in the study is still new to the generative or computer-assisted algorithmic composition fields and shows promising avenues, notably due to their high degree of formalization (as their fundamental concepts rely on group theory). As such, contributions on this topic are welcome and push the boundaries of the aforementioned fields by challenging the computational thinking at the representation level of the harmonic content of musical structures. Having said that, I believe that the paper needs some carful revision before publication.

- While the authors position their contribution within machine improvisation and continuously refer to the material as improvised, they assume the concept of improvisation as being based entirely on transformative structures. While improvisation deals with degrees of transformation, I believe that the method is rather situated in the scope of computer-assisted algorithmic composition for the creation of harmonic transformations (i.e., variations). Can you provide musical examples in which the formal structure of the musicians' intension is to vary the harmonic structure overtime during improvisation? I cannot recall a single example.

Please refer to the following page for a broad categorization of computer-generated music approaches: https://cloudflare-ipfs.com/ipfs/QmXoypizjW3WknFiJnKLwHCnL72vedxjQkDDP1mXWo6uco/wiki/Computer-generated_music.html

Moreover, please refer to the context and details of this work: http://cmmr2017.inesctec.pt/wp-content/uploads/2017/09/56_CMMR_2017_paper_94.pdf

as well as others by L. Bigo.

- Furthermore, while the context of machine musicianship is brought at the beginning of the manuscript, no considerations are then recovered at the end in light of the results. How does the method can be incorporated in an ongoing improvisation setting or human-machine musical improvisation? 

- The transformational theory in music is used to explain temporal sequences by enhancing their degree of interval changes over consecutive harmonies. It does not aim and, to the best of my knowledge, does not provide a basis for compositional principles. The use of Markov chain to capture those principles by probabilities is an interesting idea; yet, considerations, discussion, or proper evaluation should be provided to show their potential in comparison with traditional alphabets for harmonic-based representations, e.g. [root, quality] and/ or pc sets. 

- The lack of consideration for the long-term structure, and notably, the level of contextual embedding to in-key, is striking. All modeled examples are in the major-minor mode and in a single key. On the other hand, the proposed transformations do not consider that hierarchy as they wave between various spaces. It is essential to discuss the quality of the transformations and the context of stylistic affiliation to the modeled source.

- The introduction and literature review does not provide sufficient background, notably because it was not clear what is the authors' contribution beyond the contribution of A. Zhang (reference entry [3] in the paper). This distinction is not clearly stated a single time in the paper.

- The model is also of limited scope in which it does not even include the complete set of diatonic triads. Is the method scalable to musical examples parsed from extensive collections? 

Terminology:

- I would relate to the concepts of entropy and probability, but to those of familiarity vs. novelty, instead of compatibility and variety. There's a substantial body of literature on the topic from cognitive and psychoacoustic and/or algorithmic composition. Please refer to the literature on the use of the Wundt curve to describe musical appreciation. I would explain the balance between μ1 and μ2 in light of these concepts.

- You should explain in better detail what you mean by 'smooth and natural' chord transitions. In fact, the choice of chord progressions derives from a corpus of existing music. Whatever is highly probable in the corpus is considered smooth.

Author Response

See attachment.

Author Response File: Author Response.pdf

Reviewer 2 Report

The article proposes a framework for the generation of triadic sequences based on a Markov-like decision system, balancing compatibility with data corpus and variety of outcome.
The paper is clear and well written, the subject is well targeted and the bibliography is sound. The idea behind the generation is simple but interesting and well carried out.

I have however some concerns about some specific issues, and some minor and major remarks.

Two general remarks:
1) the authors propose a system to generate chord sequences based on 1st order markov chains, balancing weights between compatibility and variety. It seems that they evaluate this model on "how well" it reproduces the sequence of input data. I wondered whether this is a good metric for computer-aided improvisation tools, however I understand that it's very hard to find any other meaningful metric, and at the end the real metric is the composers' ears. Also, this is done on very little data, so I'm not sure that the evaluation is significant. The authors mention the fact that a larger evaluation is a subject for another article, and I disagree with that. I think it should be the subject of this article as well. Moreove my concern is whether this approach, with this metric, is any better than a higher order Markov chain. It should be made more clear why the approach is better or in any case why did the authors choose it instead of using a higher order markov chain.

2) The authors use the term "interval" and "interval-based" in an unclear way. They define both intervals between pitch class and Gamma-Intervals, but then they only refer to "interval" and "interval-based" approaches (e.g. introduction or conclusion) and it is a bit unclear to me whether this is a synonym for transformation (i.e. interval=Gamma-interval) or pitch-interval, or something else. It looks as if the latter is the case, since the introduction deals with the the difference between a transformational approach and an interval-based approach is. But then why are the two exclusive? Indeed in sec. 2 we learn that a (Gamma-)interval is indeed a transformation, so one is a bit lost... It should definitely be made more clear.

Here below is the list other remarks.
Some of them are minor (-), some are rather important (*), some are major (**).

ABSTRACT:
- "of the transformational approach are demonstrated discussed" -> *AND* discussed

- UTT: the authors should not assume the familiarity with the acronym so early, and it's better to use "uniform triadic transformation" in the abstract instead

1. INTRODUCTION

- I am a bit skeptical about the seemingly interchangeable usage of the word "approach", as in the title, and "algorithm", as in the first line of the introduction. An approach is not an algorithm: if the paper presents an algorithm, then the title should be changed; otherwise the first line of the introduction should be changed.

* In general, the authors uses the introduction to enumerate a lot of related bibliographical references (which is always useful!), but it would be also interesting to have more words about the problem itself: what is the specific problem they are tackling, on what side, and why is the problem interesting (there's something only in the first paragraph). Some assertions seems to have something more to do with conclusion instead of introduction, such as "Combining the Markov model with the transformational approach and the suggested optimization algorithm significantly expands the possible musical outputs. Associating this with a UTT-based approach improves the musicality and style of the result.", and more importantly: "Further research may also include other types of information including more complex harmonies and rhythm and melody, which may be expressed in terms of group theory, and developed thereon."
Again, to reiterate: these assertions should be saved for the end, and the introduction should be rewritten to better explain what the problem is, why it is important (citations about Markov or UTT are not enough to express that), and what is the authors' approach to it. Some of it is already there, but it's blurred.

- "[9] presents...": I'm not sure whether this is formally OK to start a sentence with a reference, but that pertains more to the editors, of course.

- "may be extended in several ways to include larger databases" -> I assume they mean databases of chords/chordal progression, but it should be made more clear (perhaps choosing another word, instead of "databases").

- "which considers compatibility, which is represented..., which is represented..." -> a lot of "which"s... I'd change to: "which considers compatibility, represented by the average transition probability, and variety, represented by entropy".

SECTION 2

- "The triads can be enumerated as: Γ = {0,...,11}⊎{12,...,23}," -> I wonder whether this enumeration is actually needed, since this is not elegant and it seems unnecessary to me. As it is never used in the rest of the paper, it should probably be dropped.

- "Permuting the three intervals" -> It should be made more clear that these are the three intervals between the triadic notes.

* "The uniform triadic transformations provide a unified notation for arbitrary transformations on Γ": this doesn't seem to be true, unless one needs transormation to have a group-like structure. One could imagine, indeed, a transformation that sends (r, +)->(r, -) and that fixes (r, -)->(r, -). This is not a UTT. Am I mistaken? If not, one should point this out more clearly and redefine UTT in a more rigourous way.

SECTION 3:

* "Let hC⃗ be the distribution of consecutive pairs of triads (or, as described later, triadic transformations)": I think that h_C should be defined more properly.

Section 3.1
- "Write δi for the probability that δi assigns to ci ∈ Γ" ->typo: probability that δ assigns...

** I have a hard time understanding the -mu_2/ln 2 term in the equation just after the "Therefore". Everything else looks ok. Also, that term disappears abruptly in the final equation of page 5, which makes me wonder whether the final equation is correct and somehow that term was wrong in the middle passages. Please check the calculation.

- There is a typo in the equation right after "to optimal distribution vector by": the second e (exponent basis) should be 2.

* Equation 5, and from there onwards for all the rest of the article, uses two parameters (mu1, mu2) with a 1-sum constraint. I wonder why a single parameter isn't used instead (say, mu and 1-mu). It looks small as a change, but this would make everything more readable in the following sections and graphs. The authors could perhaps find an appropriate name for the parameter to emphasize is role between standardness and entropy.

- Figure 1: It is not very clear why we reach a uniform distribution with mu = 0.5. Shouldn't it be at mu_2 = 1? If not, why not? If yes: maybe the example should be chosen more carefully?

SECTION 4:

- "The UTT-based method, constructs" -> remove the comma

** "Large databases can be covered in future work." -> I believe that the validation of this work on larger datasets would be an important addition for the paper. It doesn't have to be a huge report, a few pages will suffice. In a sense, I have the feeling that the three cherry-picked examples shown later on display not just the process, but very much the limits of its oversimplification (but more on this later), and I think that a validation on larger datasets may prove benificial to understanding its value.

- Figure 2 (and following), the Tf. pairs may be shifted more to the right, so that they lie exactly in the middle between two Td. pairs.

* "Average transition probability was calculated separately for the triads and for the transformations." -> I don't see this here, I guess the best phrasing is "will be calculated separately", since it appears in the next subsection.

* I personally find the approach (section 4.3) of "Average transition probability matrices given major and minor triads - UTT-based approach." quite inelegant. A lot has been said in section 2 to frame the problem within the context of group theory, but here one has to split the behavior on two classes of elements which were supposed to have been unified. The whole idea of having a single set Gamma and transformation inside it is now essentially pushed back to having two different sets and something that acts on them. In a sense, this is no longer transformational, because it relies on triadic information, and this should be pointed out perhaps more clearly ("UTT-based approach" seems to imply that it is purely transformational...). I wonder whether there's no better formal way to describe this. (Then again, if that approach works in practice, I'm not against it, but it somehow destroys the previous formal elegance, and maybe it should be pointed out more honestly)

- "All three methods may be used in an improvisational environment. This environment may typically include tempo and rhythm variation and voice leading. The improvisational environment may be extended later, or, alternatively, it may be included in an existing setup" -> The sentence is a bit sketchy, I would go deeper into the topic or perhaps remove the sentence altogether?

- Figures 9, 10, 11, 12: I think they should have no Repeat sign at the end.

- FIgure 10: What's the rationale behind the choice of A# in meas. 2 and Bb in meas 4? It's probably worth explaining or remove the difference by making it uniform.

SECTION 5

- "and the third is the chorus of the Beatles’ song ‘Hello Goodbye’, introduced in the previous section" --> where? I cannot find it. There seems to be no mention of it in the previous section.

- "given 0 < μ1 < 1 (so that 1 > μ2 > 0)" -> I think the parenthesis doesn't really add much information, and just looks bizarre...

- "Figure 13 shows excerpt from the Liszt composition ‘Wilde Jagd’" -> I wouldn't say that. I'd say that Fig. 13 shows a simplified harmonic progression from the ... Same is true for all other figures.

Sections 5.2 and 5.3
** These examples to me are somehow problematic.

1) Morning Bell - Radiohead: the C#m the authors assert, has actually a very big returning A in the bass. That honestly make the simplification close to useless, because that A changes the progression very much: when it arrives it gets us back to a Amaj7 region which does not appear in the simplified progression.

2) Hello, Goodbye: The chord progression seems wrong to me; the G chord at the end of the 2nd measure is actually a C chord with G in the bass (it's very clear from the melody). (Also the last Bb is actually a Bb9, which makes quite a difference in that context, although of course I understand the simplification to triads....

I recognize that the authors laid out their premises very clearly, with their reduction to triads. However, I think that in a context where the risk of over-simplification are too high, examples should be chosen more carefully. This also shows, for me, that a triadic theory is somehow too small a theory even for pop/rock music. The low A in the Radiohead song is monumental, and there's a huge expectation "game" between the C#m and the moment in which the low A comes to complete it to Amaj7 - I believe this cannot be ignored by any harmonic analysis.

- "The structure of the ‘hello goodbye’ chorus" -> Hello, Goodbye must be capitalized

Author Response

See attachment.

Author Response File: Author Response.pdf

Round 2

Reviewer 2 Report

Thank you very much for the time you took to undergo the reviewers' requests.

Just a single additional point: you still claim that "The uniform triadic transformations provide a unified notation for arbitrary transformations on Γ": it still doesn't seem to be true to me. I've gone through the reference. Again, UTTs seem to me just a sliver of transformations within Γ (there are 24! transformations in Γ, but much less UTTs). What am I missing? In what sense do they provide "a unified notation for arbitrary transformations on Γ"? I'm surely wrong, but I would love to have a feedback on this.