Predicting Discrimination in L3 Portuguese by Hungarian Speakers: The Effect of Perceptual Overlap
Round 1
Reviewer 1 Report
Comments and Suggestions for AuthorsThis is a very interesting paper, which addresses two languages that are uncommonly found in the L3 phonology literature. My main concerns with this contribution are twofold. First, the writing could be much clearer in terms of explaining things. I found myself not entirely sure what was done exactly with the response variables, for example. This needs to be fixed. Second, the organization of the methods sections is unusual, and the procedures sections really need to be split up into the relevant sections.
I have the following detailed feedback for the author(s):
Lines 130-142: It might be helpful to have a figure to show the vowel chart; note also that some treatments of European Portuguese vowel contrasts include an eighth oral vowel, which is a central high vowel.
Lines 143-157: It might be helpful to have a figure to show the vowel chart.
Line 231: One potential concern in what is in Table 1 is that the syllables have been manipulated so that all vowels have the same duration, basically. The problem here is that, all things being equal, low vowels are of longer duration than high vowels. In normal speech, language speakers don’t notice this, but would this manipulation seem odd to them?
Lines 246-257: It seems like it is more typical to report the participants before anything else. It is especially important here because in Section 2.1.1, there are references to the participants’ native languages, and when that is discussed prior to the participants, it makes it difficult to understand what is meant here. Another issue that I see here is that it is not clear where the demographic information about the participants come from; I’m going to guess that there was some sort of questionnaire, but that is not mentioned, which is a problem.
Lines 254-257: What did the stimuli look like for the EP native speakers? Surely they were not presented with choices of Hungarian words.
Line 255: It strikes me as problematic that a ‘standard dialectal area’ is referred to here. What is “Standard European Portuguese” and who speaks it? I would assume that we are referring to the area around Lisbon, but of course it is unreasonable to assume that there is not sociolinguistic variation in Lisbon. More specificity is clearly needed.
Lines 274-278: This seems like part of the materials. Really, the procedure should focus on what the participants saw, not what was done to prepare it.
Lines 276-278: The question remains as to what the choices were for the EP speakers since they likely cannot have been Hungarian words.
Line 279-286: It seems like this should be a separate subsection for analysis.
Line 280: Missing apostrophe after ‘participants’
Line 282-284: Why were chi-squared tests used instead of logistic mixed-effects regressions? Instead of a bunch of chi-squared tests, you could have a unified logistic mixed-effects regression, which would reduce the concern of a Type I error.
Lines 294-295: EP /o/ and /u/ may be perceptually similar, but was the token of [o] a good specimen?
Line 308: This table shows what orthography was presented to the native EP listeners, but this needs to be explained beforehand.
Line 316: It strikes me that the EP vowels are a good deal more fronted. Was any sort of normalization procedure employed or are we comparing apples to oranges here?
Line 373: Strikethrough comma after “[a]”
Line 381: misspelled “proportion”
Line 387: as a modifier, “identifications” should not be plural; no hyphen for “chance-level”
Line 390: It’s still a little unclear how this perceptual overlap score was calculated. Note that while all perceptual overlap scores are equal to the lower percentage to the left, the last line does not match.
Line 394-402: It seems like this may need to be unpacked some because it isn’t very clear.
Lines 449-496: This section seems to include materials, procedure, and analysis, which are usually separate sections.
Line 497: This is numbered the same as the previous section. This section should probably go before the previous section.
Line 574: two <l> in “Finally”
Line 651-657: Something to keep in mind is that, while English does not have a vowel that is typically transcribed as [ɐ], it does have mid central vowels, and ultimately the IPA symbol is merely that, a symbol. The vowel space is ultimately scalar in nature.
Lines 734-754: Align right all number columns; these tables are not large, so I am not sure why they are in appendices; it seems like the reader will want to see these straight away, and it is inconvenient to have to flip back and forth.
Comments on the Quality of English LanguageIn general, the English seems fine, although I did notice a few minor typos. The writing is at times dense and becomes difficult to unpack, so this is something that the author(s) should work on for the final version.
Author Response
Dear reviewer,
We would like to thank you for your detailed revision. Below, we answer to your corrections and suggestions, they have contributed undoubtably to improve the quality of our manuscript.
Comments 1: Lines 130-142: It might be helpful to have a figure to show the vowel chart; note also that some treatments of European Portuguese vowel contrasts include an eighth oral vowel, which is a central high vowel.
Response: We added a vowel chart for European Portuguese.
Comments 2: Lines 143-157: It might be helpful to have a figure to show the vowel chart.
Response: We also added a vowel chart for Hungarian.
Comments 3: Line 231: One potential concern in what is in Table 1 is that the syllables have been manipulated so that all vowels have the same duration, basically. The problem here is that, all things being equal, low vowels are of longer duration than high vowels. In normal speech, language speakers don’t notice this, but would this manipulation seem odd to them?
Response: A group of Portuguese native speakers controlled for stimuli nativeness, and no major problems were reported. The exception was in the identification of /o/, where the analysis revealed a speaker effect.
Comments 4: Lines 246-257: It seems like it is more typical to report the participants before anything else. It is especially important here because in Section 2.1.1, there are references to the participants’ native languages, and when that is discussed prior to the participants, it makes it difficult to understand what is meant here. Another issue that I see here is that it is not clear where the demographic information about the participants come from; I’m going to guess that there was some sort of questionnaire, but that is not mentioned, which is a problem.
Response: Changed according to this suggestion.
Comments 5: Lines 254-257: What did the stimuli look like for the EP native speakers? Surely they were not presented with choices of Hungarian words.
Response: This information was added.
Comments 6: Line 255: It strikes me as problematic that a ‘standard dialectal area’ is referred to here. What is “Standard European Portuguese” and who speaks it? I would assume that we are referring to the area around Lisbon, but of course it is unreasonable to assume that there is not sociolinguistic variation in Lisbon. More specificity is clearly needed.
Response: A footnote was added to clarify this question.
Comments 7: Lines 274-278: This seems like part of the materials. Really, the procedure should focus on what the participants saw, not what was done to prepare it.
Response: We moved this information to the beginning of the Procedure section, explaining what the participants saw in the experiment.
Comments 8: Lines 276-278: The question remains as to what the choices were for the EP speakers since they likely cannot have been Hungarian words.
Response: See response to comment for Lines 254-257.
Comments 9: Line 279-286: It seems like this should be a separate subsection for analysis.
Response: Changed according to this suggestion.
Comments 10: Line 280: Missing apostrophe after ‘participants’
Response: Corrected.
Comments 11: Line 282-284: Why were chi-squared tests used instead of logistic mixed-effects regressions? Instead of a bunch of chi-squared tests, you could have a unified logistic mixed-effects regression, which would reduce the concern of a Type I error.
Response: In the categorization task, there was no correct or incorrect answers. For each trial (= each EP token), nine responses were possible, one for each L1 word. This makes the analysis of the data in a single model very difficult, especially considering that there were only three trials for each EP token, for each participant (we refer to this limitation in the Discussion section). The option for a chi-square test was based on the nature of the data.
Comments 12: Lines 294-295: EP /o/ and /u/ may be perceptually similar, but was the token of [o] a good specimen?
Response: One of the /o/ tokens was deviant and we refer to this in lines 294–298. Since the aim of the first experiment was to observe the identification of the EP [ɐ] and [ɨ] by Hungarian listeners, and detect perceptual overlap involving these two vowels, we did not repeat the experiment replacing the deviant /o/ token.
Comments 13: Line 308: This table shows what orthography was presented to the native EP listeners, but this needs to be explained beforehand.
Response: See response to comment for Lines 254-257.
Comments 14: Line 316: It strikes me that the EP vowels are a good deal more fronted. Was any sort of normalization procedure employed or are we comparing apples to oranges here?
Response: The explanation for the EP vowels as being more fronted is described below, in Figure 3 (lines 327–347). The data for the Hungarian vowels was retrieved from a perceptual study using synthetised stimuli and no information was given about the production of the stimuli by male or female speakers. From the calculation of formant dispersion, we could determine that it was likely to correspond to values produced by a Hungarian male speaker.
Comments 15: Line 373: Strikethrough comma after “[a]”
Response: Corrected.
Comments 16: Line 381: misspelled “proportion”
Response: Corrected.
Comments 17: Line 387: as a modifier, “identifications” should not be plural; no hyphen for “chance-level”
Response: Corrected.
Comments 18: Line 390: It’s still a little unclear how this perceptual overlap score was calculated. Note that while all perceptual overlap scores are equal to the lower percentage to the left, the last line does not match.
Response: The calculations for perceptual overlap can be confusing, especially considering that different methods are used. We made some changes and expect that it is clearer now.
Comments 19: Line 394-402: It seems like this may need to be unpacked some because it isn’t very clear.
Response: See comment above.
Comments 20: Lines 449-496: This section seems to include materials, procedure, and analysis, which are usually separate sections.
Response: We made changes according to this suggestion.
Comments 21: Line 497: This is numbered the same as the previous section. This section should probably go before the previous section.
Response: Corrected.
Comments 22: Line 574: two <l> in “Finally”
Response: Corrected.
Comments 23: Line 651-657: Something to keep in mind is that, while English does not have a vowel that is typically transcribed as [ɐ], it does have mid central vowels, and ultimately the IPA symbol is merely that, a symbol. The vowel space is ultimately scalar in nature.
Response: We added a comment in the Discussion, following this suggestion.
Comments 24: Lines 734-754: Align right all number columns; these tables are not large, so I am not sure why they are in appendices; it seems like the reader will want to see these straight away, and it is inconvenient to have to flip back and forth.
Response: In lines 734–754 there are no tables. The tables closer to those lines are the ones in the Appendices, and these tables would occupy one and a half page in the main text, which we believe would make reading very difficult. However, the text includes the ANOVA results.
Comments on the Quality of English Language
In general, the English seems fine, although I did notice a few minor typos. The writing is at times dense and becomes difficult to unpack, so this is something that the author(s) should work on for the final version.
Response: We simplified some sentences, and hopefully improved the language and comprehensibility.
Reviewer 2 Report
Comments and Suggestions for Authors
This paper examines perception of European Portuguese vowels by Hungarian listeners. The authors report the results of two experiments: a vowel categorization test with goodness ratings and a vowel discrimination test. The results of the categorization test operationalized as “overlap scores” are then used to predict discrimination difficulties of these vowel contrasts. Overall, the methods, statistical analysis and procedures are appropriate and my opinion is that the study is a significant contribution to the L2/L3 speech perception literature. However before the ms. is accepted for publication, the authors must address some content and minor issues listed below and strengthen the discussion section.
CONTENT ISSUES
Line 263: what is the rationale for the four-point Likert scale? Previous studies use a 7-point or 9-point-scale for goodness ratings. Please provide a convincing explanation for the four-point scale.
Line 324: I would recommend calculating the Euclidean distances between the Hungarian and the EP vowels. Since the speakers do not match as far as sex is concerned I suggest normalizing the data first.
Line 371: Please provide the rationale for the fit-index (Guion et al. 2000). It is not clear to me, why you include this metrics in the analysis if your predictions of discrimination are based on percent overlap. Besides, as argued in Tyler (2021), the fit-index can be problematic:
When the categorization percentage is high, the fit index is easily interpretable. For example, if the categorization percentage is 100 per cent and the goodness rating is 2, then the fit index of 2 entirely reflects the perceived phonetic goodness-of-fit. A fit index of 2 would also be obtained with a categorization percentage of 40 per cent and a goodness rating of 5. By combining the two values, the assumption is that the perceived similarity to the L1 is qualitatively the same in all
three cases where the fit index is 2. That may be true, but it is possible that the measure is conflating measures that are qualitatively different. (page 168).
Line 498: The participants of the categorization and discrimination tests were not the same, right? If I understood it correctly, the participants of the categorization test had no prior knowledge of Portuguese but the participants of the discrimination test had some experience with this language. Ideally, to make predictions of discrimination based on categorization, the same group of listeners should have done both tests. This issue must be addressed at some point and the limitations must be acknowledged accordingly.
Discussion section: I missed implications of your findings to the L2 speech learning models reviewed in section 1. The PAM-L2 is especially relevant to your study. Please discuss how your findings relate or add to the existing literature (i.e. other empirical work that you review in the introduction). One aspect that you should highlight in the discussion is that many previous studies are limited to L2-English, so your study makes a significant contribution to the L2-L3 speech perception field because you provide data other than L2-English.
Minor issues
Line 42: replace “acquisition” with “production”
Line 34: add “linguistic” before “perception model”
Lines 60: Aside from Faris et al. 2018, there is a more recent study examining the relationship between % overlap and discrimination (see suggested references below).
Line 167: replace “acquire” with “perceive”
Line 177: replace “among” with “as instances of “Hungarian”
Line 366: please provide the results of the chi-square test
Line 616-17: as I said earlier, one possible explanation is that the discrimination and categorization tasks were administered to two different groups of listeners with varying experience with Portuguese.
REFERENCES
Rallo Fabra, L., Tyler, M. D. (2023). Predicting discrimination difficulty of Californian English vowel contrasts from L2-to-L1 categorization. Ampersand, 10, 100109.
Tyler, M. D. (2021). Phonetic and phonological influences on the discrimination of non-native phones. In R. Wayland (Ed.), Second Language Speech Learning. Theoretical and Empirical Progress (pp. 157–174). Cambridge University Press.
Comments on the Quality of English Language
The manuscript is quite well-written pending some minor edits.
Author Response
Dear reviewer,
We would like to thank you for your detailed revision. Below, we answer to your corrections and suggestions, they have contributed undoubtably to improve the quality of our manuscript.
Comments 1: Line 263: what is the rationale for the four-point Likert scale? Previous studies use a 7-point or 9-point-scale for goodness ratings. Please provide a convincing explanation for the four-point scale.
Response: We believe this is already explained in the text: “By using a four-point Likert scale, our goal was to force participants to express a clearly positive (3 and 4) or negative (1 and 2) judgment and to avoid the face-saving don’t know effect (Sturgis et al. 2014).”
Comments 2: Line 324: I would recommend calculating the Euclidean distances between the Hungarian and the EP vowels. Since the speakers do not match as far as sex is concerned I suggest normalizing the data first.
Response: In a preliminary version of the article, we had perceptual overlap calculations based on Euclidean distances. However, we ended up excluding it for the following reasons: First, the aim of the study was not to test the best method for determining perceptual overlap scores (as it is in Elvin et al. 2021, mentioned in our text), but rather investigate if perceptual overlap affects discrimination in L3. Second, in Elvin et al. 2021, the authors concluded that perceptual overlap scores obtained with a categorization task were a better predictor for the Spanish listeners, who have a smaller vocalic space than the Australian English speakers, for whom predictions based on Euclidean distances were more accurate. Although more complex than the Spanish vowel inventory, Hungarian has nine vowel qualities (against the 13 for the Australian in Elvin et al.’s study). Consequently, perceptual overlap based on categorization task is a more appropriate method in for our study than based on Euclidean distances.
Comments 3: Line 371: Please provide the rationale for the fit-index (Guion et al. 2000). It is not clear to me, why you include this metrics in the analysis if your predictions of discrimination are based on percent overlap. Besides, as argued in Tyler (2021), the fit-index can be problematic:
When the categorization percentage is high, the fit index is easily interpretable. For example, if the categorization percentage is 100 per cent and the goodness rating is 2, then the fit index of 2 entirely reflects the perceived phonetic goodness-of-fit. A fit index of 2 would also be obtained with a categorization percentage of 40 per cent and a goodness rating of 5. By combining the two values, the assumption is that the perceived similarity to the L1 is qualitatively the same in all three cases where the fit index is 2. That may be true, but it is possible that the measure is conflating measures that are qualitatively different. (page 168).
Response: We added a comment on this in the text.
Comments 4: Line 498: The participants of the categorization and discrimination tests were not the same, right? If I understood it correctly, the participants of the categorization test had no prior knowledge of Portuguese but the participants of the discrimination test had some experience with this language. Ideally, to make predictions of discrimination based on categorization, the same group of listeners should have done both tests. This issue must be addressed at some point and the limitations must be acknowledged accordingly.
Response: As we describe in 500–503, participants were at the onset of learning, and they completed the experiment in the first week of the course. The first experiment was conducted in the Spring term, and it was a preliminary experiment. The results were used to design Experiment 2, which was the first step of a perceptual training study. Since the two experiments were conducted in two different moments, and due to anonymization, it was impossible to contact the participants who completed the first one. We add a comment on this in the limitations of the study.
Comments 5: I missed implications of your findings to the L2 speech learning models reviewed in section 1. The PAM-L2 is especially relevant to your study. Please discuss how your findings relate or add to the existing literature (i.e. other empirical work that you review in the introduction). One aspect that you should highlight in the discussion is that many previous studies are limited to L2-English, so your study makes a significant contribution to the L2-L3 speech perception field because you provide data other than L2-English.
Response: Regarding the first suggestion, it was not our aim to test models such as SLM, PAM-L2, or L2LP. Although it would be relevant, the description and discussion of results in light of any of those models would take space in the article and eventually distract the reader from our main goal: to observe if the effect of perceptual overlap attested in L2 is also an issue in L3. As for the second suggestion, the phrase in the conclusion - “In closing, this study is original in observing L3 perception in two languages that have been remarkably understudied: European Portuguese and Hungarian” - refers this issue.
Comments 6: Line 42: replace “acquisition” with “production”
Response: We kept “perception and acquisition” because both are key words for the models we refer. Although the Perceptual Assimilation model (Best 1995) initially target non-native perception, later it was extended to acquisition (Best and Tyler 2007)
Comments 7: Line 34: add “linguistic” before “perception model”
Response: Corrected.
Comments 5: Lines 60: Aside from Faris et al. 2018, there is a more recent study examining the relationship between % overlap and discrimination (see suggested references below).
Response: Although the study of Rallo Fabra and Tyler (2023) also investigates non-native vowel perception, its aim is to discuss this issue in light of the SLM-r, PAM-L2, and the L2LP model, which is out of our study scope. As for Tyler (2021) it’s a book chapter, focusing on PAM.
Comments 8: Line 167: replace “acquire” with “perceive”
Response: Corrected.
Comments 9: Line 177: replace “among” with “as instances of “Hungarian”
Response: Corrected.
Comments 10: Line 366: please provide the results of the chi-square test
Response: There were no significant effects of knowledge of German in the categorization of the 3 EP vowels. We added a sentence to make this clear.
Comments 11: Line 616-17: as I said earlier, one possible explanation is that the discrimination and categorization tasks were administered to two different groups of listeners with varying experience with Portuguese.
Response: Please see comment for Line 498.
Reviewer 3 Report
Comments and Suggestions for AuthorsThe study examines how perceptual overlap affects Hungarian listeners' perception of European Portuguese vowels as a third language (L3). It finds that nonnative vowels pose challenges, while prior knowledge of German aids in recognizing specific vowels. Neither group nor individual-based overlap measures reliably predicted discrimination difficulties. The paper would benefit from the following comments:
Introduction: Consider adding studies about the perception of Portuguese or other nonnative sounds by Hungarian listeners.
Lines 32-34: Also check the Universal Perceptual Model: Georgiou, G. P., Giannakou, A., & Alexander, K. (2024). Perception of second language phonetic contrasts by monolinguals and bidialectals: A comparison of competencies. Quarterly Journal of Experimental Psychology. doi: 10.1177/17470218241264566
Lines 44-87: For another perspective on perceptual overlap, see the following work: Georgiou, G. P. & Dimitriou, D. (2023). Perception of Dutch vowels by Cypriot Greek listeners: To what extent can listeners’ patterns be predicted by acoustic and perceptual similarity? Attention, Perception, and Psychophysics, 85, 2459-2474. doi: 10.3758/s13414-023-02781-7
Lines 130-170: It would be better to include vowel trapezia representing the vowels of the two languages.
Line 249: Report the mean age and standard deviation.
Line 274: Online experiments can differ significantly from lab-based experiments. How did you ensure that the quality of the experimental procedure was maintained (e.g., noiseless rooms, use of headphones or earphones, participant attention to the experiment, etc.)?
Lines 498-500: The results of the two tests may not be directly comparable since they included different individuals; explain how you obtained reliable results.
Lines 616-623: Please explain this inconsistency.
Author Response
Dear reviewer,
We would like to thank you for your detailed revision. Below, we answer to your corrections and suggestions, they have contributed undoubtably to improve the quality of our manuscript.
Comments 1: Consider adding studies about the perception of Portuguese or other nonnative sounds by Hungarian listeners.
Response: To the best of our knowledge, there are no studies on non-native vowel perception with Hungarian native speakers.
Comments 2: Lines 32-34: Also check the Universal Perceptual Model: Georgiou, G. P., Giannakou, A., & Alexander, K. (2024). Perception of second language phonetic contrasts by monolinguals and bidialectals: A comparison of competencies. Quarterly Journal of Experimental Psychology. doi: 10.1177/17470218241264566
Response: We reviewed this article and included it in the article.
Comments 3: Lines 44-87: For another perspective on perceptual overlap, see the following work: Georgiou, G. P. & Dimitriou, D. (2023). Perception of Dutch vowels by Cypriot Greek listeners: To what extent can listeners’ patterns be predicted by acoustic and perceptual similarity? Attention, Perception, and Psychophysics, 85, 2459-2474. doi: 10.3758/s13414-023-02781-7
Response: We reviewed this model; however, we decided not to include or change the methods for perceptual overlap due to the limited space available to test all methods for perceptual overlap.
Comments 4: Lines 130-170: It would be better to include vowel trapezia representing the vowels of the two languages.
Response: We added two vowel charts: one for European Portuguese and another one for Hungarian.
Comments 5: Line 249: Report the mean age and standard deviation.
Response: We did not collect the exact age of the participants. Age was asked in a multiple-choice, interval format (18-25, 26-30, 31-40, 41-45), to simplify the questionnaire – since the goal was not to investigate age effect. However, we ended up realizing it was not a good decision and in Experiment 2 we asked the age as an open question.
Comments 6: Line 274: Online experiments can differ significantly from lab-based experiments. How did you ensure that the quality of the experimental procedure was maintained (e.g., noiseless rooms, use of headphones or earphones, participant attention to the experiment, etc.)?
Response: We provided precise instructions on the use of headphones/earbuds and the importance of completing the task in a silent environment. In addition, participants were asked to close all tabs and browser windows while doing the experiment, and to keep the sound volume the same during the tasks.
Comments 7: Lines 498-500: The results of the two tests may not be directly comparable since they included different individuals; explain how you obtained reliable results.
Response: We added a comment on this in the limitations of the study.
Comments 8: Lines 616-623: Please explain this inconsistency.
Response: We elaborated on these inconsistencies in lines 624-643.
