Adaptation of Gap Predictions in Filler-Gap Dependency Processing during Reading

Round 1

Reviewer 1 Report

This series of studies uses eye-tracking during reading to investigate syntactic adaptation and whether adaptation is robust to changes in task and changes of the type of sentences tested. In Experiment 1, 3 groups are exposed to filled gaps (with the gap at the preposition), object gaps, or filler sentences, respectively, and then tested on filled gaps. The group that had been exposed to preposition gaps was the only group not to show difficulty with the filled gap in the test phase, suggesting that these readers had adjusted their processing as a function of exposure. Experiment 2 and 3 make the participants believe that the exposure phase was a different task. In addition, the exposure sentences were embedded in a narrative.   Exp 3 also had different test sentences in that plausibility at the verb rather than filled gaps were manipulated. Neither of the latter two experiments showed effects of adaptation. The authors conclude that adaptation to syntactic structures can occur, but may be very specific to the task.

This paper is very clearly written and I have no major comments. My only somewhat major questions to the authors are:

First, what the motivation was for Experiment 3 (changing both exposure phase and test phase from Exp 1), since Experiment 2 (changing only exposure phase) yielded no effects of exposure. 

Second, the authors interpret the findings in Experiment 1 as a “dampening of gap predictive behavior” (p 21). This actually nicely fits in with the idea of predictive utility: if too often the prediction is not borne out, you may stop predicting (see Kuperberg & Jager 2016; Kaan & Gruter 2021). An alternative interpretation is that the readers still predict DO gaps, but that the revision takes less effort as a result of exposure, or that people get used to the anomalies, and/or perhaps do not bother revising them. Is there any way that these alternative explanations can be ruled out on the basis of the current study?

The following are mostly minor remarks:

Description of participants (in all experiments): more information should be provided. How was “native English speaker” defined? What was the age, gender? Were the groups matched in terms of age, gender, education, SES, knowledge of other languages, …? How many participant data sets were retained in each group?

Table 3: typo: “was named”

Cond 3a: I garden-pathed here myself since I read the verb as “wrote about”  If such misinterpretations were systematic in the DO gap exposure group, would this have affected the interpretation of the results you find in the test phase (Experimental block)?

Analysis (p 7). Was Trial number centered?

Section 2.3.2 , first para (p 8): were the numbers of missing data equally distributed over the 6 cells (groups x conditions)?

Page 10: be careful in the wording of the results. Effects are relative to the reference level (filler exposure and trial 0 (if not centered) ). The effect of fronting type, for instance, can therefore not be interpreted as a main effect but only holds for the reference level.

Related to this, some of the pairwise analyses are redundant. The effect of fronting type on the filler group is already given in the main analysis.   Another way to follow-up on the interactions is to use the same model as in the main analysis but to recode the exposure groups such that the reference level is the PO exposure group.

Exp 2, procedure (Section 3.1.1.3). What was the duration of the break between the the exposure task and the test phase? Was this different from Exp 1?

Exp 2, procedure (Section 3.1.1.3). Why did the exposure task involve reading sentences out loud? Would that have distracted attention away from the sentences? May that have contributed to the absence of the exposure effects?

Exp 2 analysis (Section 3.1.1.4) “ The analysis procedure for the eye tracking portion was identical to that  from Experiment 1.” But we only have 2 groups now. How were these groups coded in the models?

P 17, ln, 625: typo: minimal

Section 4.2 p 20. I ‘d drop this section, or in any case the paragraph ln 743-757. This is not relevant for the main point of the paper, and hard to follow if one if not familiar with slash categories. I also think that the “problem” addressed is an easy thing to fix in such approaches.

P 22, ln 820: Chun 2018 does not look at talker-specificity

P22, ln 837: the use of “talker” is rather confusing here. I would incorporate endnote 3 into the main text to make clearer what is meant; or replace “talker” with “author” or “language producer”

Same para: the authors could insert some ideas of how talker (in this case: writer)-specificity of syntactic adaptation could be tested.

 References cited:

Kaan, E, and Grüter, T (2021) Prediction in second language processing and learning: Advances and directions. In E. Kaan & T. Grüter (Eds.), Prediction in Second Language Processing and Learning (pp. 1-24). Amsterdam: John Benjamins.

Kuperberg, GR, and Jaeger, TF (2016) What do we mean by prediction in language comprehension? Language, Cognition & Neuroscience 31(1), 32-59. doi:10.1080/23273798.2015.1102299

Author Response

Please see the attached PDF.

Author Response File: Author Response.pdf

Reviewer 2 Report

The authors in this eye-tracking study sought to generalize syntactic adaptation findings to filler gap structures by exposing participants to a greater proportion of PO gaps than is found in natural speech (as evidenced by a corpus analysis). Experiment one had a test block showing that participants exposed to more PO gaps had a reduced penalty for reading NP-fronted PO gaps in the test block, suggesting they adapted to the structure. Experiments 2 and 3 sought to determine whether this adaptation persisted across tasks. In both experiments, the exposure block was a sentence recognition task consisting of passages that included the target structures. After reading these passages, participants were prompted to answer whether or not they saw a given sentence. This task was, importantly, not eye-tracking. The test block in Experiment 2a was identical to the test block in Experiment 1, whereas the test block in Experiment 2b used eye-tracking to test for a plausibility mismatch effect in filler-gaps. In both experiments, no group differences were found at the critical region, suggesting participants did not adapt to the PO gaps across tasks.

Overall, I think this paper would make a great contribution to the special issue on syntactic adaptation in Languages. The overall approach makes sense, the theoretical motivation, although not exhaustive, is logical, and the conclusions derived from the models seem to check out. That being said, there are a number of minor, yet crucial, revisions that I would like to see before publication. These are outlined below.

1. In the introduction, the authors write about how Harrington-Stack et al. (2018) failed to replicate Fine et al.’s (2013) findings. This is an issue I am particularly nitpicky on, because this has confused many people in the past. Harrington-Stack et al. did not fail to replicate initial adaptation toward a structure (and it has been corroborated by Dempsey et al., 2020, 2023; Yan & Jaeger, 2020). They specifically failed to replicate the penalty effect found in Fin et al.’s Experiment 2 showing an increase in reading times for MV resolutions after exposure to RR resolutions, what I like calling a “penalty effect.” I believe this is what the authors refer to as a “transfer of learning to a new context,” but this needs to be clarified. It might help to explicitly state in the introduction that this study is concerned with adaptation toward structures and not with subsequent adaptation away from its alternatives, although this is mentioned briefly in the general discussion.

2. What was the procedure for when mixed effects models did not converge? Did this ever happen?

3. There needs to be a multiple comparison correction for p-values at the very least when getting your simple effects. In fact, because eye-tracking measures are highly correlated, you might want to consider multiple comparison corrections for the models in general (see von der Malsburg & Angele, 2017).

4. Why are participants paid more in Experiment 2? Motivation has been shown to affect syntactic adaptation (), so this should be mentioned as something that might change things (although it would arguably make adaptation more likely…).

5. Was the story recognition task verbatim or no? Did it always rely on the difference of one word? The authors should include more details here.

6. What was the rationale for participants to read the stories in Experiment 2 out loud? Please talk about how this could influence results, if at all.

7. The predictions for Experiment 2b are confusing and I think there is a typo – the two competing predictions seem to say the same thing (e.g., no mismatch effect in test block). I read it a few times, but maybe I’m wrong. In either case, this could maybe be rewritten for more clarity.

8. Statistical power needs to be addressed here, especially given that Harrington-Stack’s paper is mentioned. I understand that some of the major findings are main effects of Group and are therefore less underpowered than, say, interaction effects, but it still needs to be addressed.

9. The design of Experiments 2a and 2b don’t allow for any test of initial adaptation before the switch between tasks, which is concerning because we do get this test in Experiment 1. I don’t expect the authors to run more experiments, but this needs to be addressed as a caveat.

10. Section 4.2 is a bit off topic, but if this issue was brought to readers’ attention in the introduction as well, then it would be fine.

11. In section 4.3, the authors could also consider alternative possibilities for why there may not be a penalty effect after syntactic adaptation. For example, satiation in comprehension explains how people become more accustomed to difficult, novel, or ungrammatical constructions after exposure (e.g., Boland et al., 2023; Do & Kaiser, 2017; Fraundorf & Jaeger, 2016; Kaschak & Glenberg, 2004).

12. The authors should include a conclusion section that summarizes the main findings and explains why they matter in a few sentences.

Final Note (not necessary to address in revisions): The major findings in this study rest on the inability to reject the null hypothesis. Importantly, this is not evidence against the null hypothesis, it is only a lack of evidence for the null hypothesis. If I may jump behind my Bayesian soapbox for a bit, this is one reason why frequentist analyses may not be the best choice if you think there’s a good chance of finding null effects. Bayes Factors, for instance, can give a probabilistic measure for or against the null hypothesis.

Referenced Articles:

Dempsey, J., Liu, Q., & Christianson, K. (2020). Convergent probabilistic cues do not trigger syntactic adaptation: Evidence from self-paced reading. Journal of Experimental Psychology: Learning, Memory, and Cognition, 46(10), 1906.

Dempsey, J., Liu, Q., & Christianson, K. (2023). EXPRESS: Syntactic adaptation leads to updated knowledge for local structural frequencies. Quarterly Journal of Experimental Psychology, 17470218231172908.

Yan, S., & Jaeger, T. F. (2020). Expectation adaptation during natural reading. Language, Cognition and Neuroscience, 35(10), 1394-1422.

Von der Malsburg, T., & Angele, B. (2017). False positives and other statistical errors in standard analyses of eye movements in reading. Journal of memory and language, 94, 119-133.

Do, M. L., & Kaiser, E. (2017). The relationship between syntactic satiation and syntactic priming: A first look. Frontiers in psychology, 8, 1851.

Fraundorf, S. H., & Jaeger, T. F. (2016). Readers generalize adaptation to newly-encountered dialectal structures to other unfamiliar structures. Journal of memory and language, 91, 28-58.

Kaschak, M. P., & Glenberg, A. M. (2004). This construction needs learned. Journal of Experimental Psychology: General, 133(3), 450.

Boland, J. E., Atkinson, E., De Los Santos, G., & Queen, R. (2023). What do we learn when we adapt to reading regional constructions?. Plos one, 18(4), e0282850.

Author Response

Please see attached PDF.

Author Response File: Author Response.pdf