Review Reports

Round 1

Reviewer 1 Report

Comments and Suggestions for Authors

Please see the attached document.

Comments for author File: Comments.pdf

Author Response

We thank Reviewer 1 for very useful comments and suggestions.  Our reply can be found in the attachment.

Author Response File: Author Response.pdf

Reviewer 2 Report

Comments and Suggestions for Authors

This is an interesting study testing for comprehension of conditional sentences with different connectives (if, if not, unless) under positive quantificational contexts (e.g., every) in Turkish-speaking 5-year-olds. Overall, the paper is well-written, it has a very informative literature review and is easy to follow along (even though the topic is highly complex). I also think that there is a real gap in our understanding about how preschoolers understand conditionals, so I am excited to see this data published (especially since it concerns data from languages other than English).

However, I have a big concern about the design of the study as I explain below. I also have some suggestions that can improve the intelligibility of the paper overall.

My big concern, as I mentioned is the design. It is unclear to me why the authors chose to test understanding of conditionals and universal quantifiers together, given that we don’t know that much about how young children (preschoolers) reason about conditionals (introduced simply with if – without any additional complicating factors). I understand that this is an extension of adult work, designed to test different semantic theories, but the complexity of the task makes it very hard to interpret the findings. Specifically, it is not clear why children (unlike) adults, do not distinguish between the different connectors (if, if not, unless) and the authors spend the whole discussion trying to speculate about the reason of this null finding, without having a definite conclusion. Perhaps the authors should consider a follow-up experiment where children are presented with simple conditional sentences introduced by the different connectors (without any quantifiers). This kind of follow-up would at least eliminate one of the speculations in the discussion.

Introduction

I really liked the introduction of this paper, I thought it was very clear and well-written. However, I have some suggestions to improve clarity and make the purpose of the study and the predictions more explicit for the reader.

I suggest that the authors introduce subsections in the introduction. I felt that the transition from the semantic theories concerning the meaning of different connectives to the developmental findings (mostly in the context of mental model theory) was a little abrupt. Introducing separate subsections would help the reader separate the different types of literature and findings. The developmental findings section should end with what the gap in the literature currently is.

Finally, the introduction needs a “current study” section. In this section the authors should outline what the main purpose of the study is and how the manipulations serve these purposes. More specifically, I would like to see a discussion about developmental predictions for each type of conditional connector. The authors simply state what each semantic theory predicts for adults but adults are not tested in this study, and it is not clear how children would behave for each connector. Also, it would be particularly useful if predictions are discussed in terms of the ratio manipulation. It is ok if some information is repeated from a previous section where adult findings are discussed but specific developmental predictions for each of the ratio condition needs to be introduced again. I had to go back and forth in the paper to try and understand the ratio manipulation.

 

Analyses

In general, the authors used appropriate modeling of the data but there was no need for doing the same analysis over and over again trying to find significant differences across conditions, when it is clear from the graph that there are no such differences. In fact, for experimental research, stepwise elimination of factors is a bad idea. The design is the design, and a maximal effects structure should be used (including all relevant interactions). If no significant interaction is found, the analysis should stop. I understand that the authors really wanted to prove that there is no difference whatsoever, but I was convinced from the very first model.

 

Discussion

Overall, the discussion was not at the level of the introduction and it needs significant elaboration and more careful consideration of the actual findings and their implications.

Lines 341-355: I found the discussion here odd. Studies on conditional reasoning show persistent difficulties with conditionals, as mentioned in the introduction (e.g., 9-year-olds having only a conjunctive interpretation of conditionals). It is extremely likely that these difficulties present in the 9-year-olds are also present (even more so) in the 5-year-olds. I wouldn’t be so easy to reject this possibility.

Also, it would be useful for the authors to talk about how children interpret universal quantifiers. It is assumed that they interpret them like adults and that this is an early achievement but is this the case? I would like to see some more in depth discussion of this issue (after all, comprehension of both conditionals and universal quantification is tested here and this should be acknowledged).

Lines 356-379: The second stipulation is even more speculative. I am not sure I agree that this has to do with working memory, and even if it did, working memory was not tested here, so it is hard to say what role working memory could be playing. In particular, working memory in the mental models literature was found to correlate with the number of models children (and adults) were able to construct. The discussion here is suggesting that it may also have to do with processing the quantifier as well. These suggestions are a little generic, as they could potentially apply to all tasks that mix different types of hard meanings and do not find the anticipated results.  

Author Response

We thank Reviewer 2 for very useful comments and suggestions.  Our reply can be found in the attachment.

Author Response File: Author Response.pdf

Round 2

Reviewer 1 Report

Comments and Suggestions for Authors

I had two major concerns in my previous review that determined my decision: (i) the ambiguity in the items and (ii) subject and item size. The authors addressed my concern regarding the scopal ambiguity and mentioned that the choice of materials was an informed one and the ambiguity was intentionally included. This was not clear in the previous draft. The present version is improved in this regard. The writing of the manuscript has also improved. But I am afraid the study’s subject and item size are still a concern. The authors added 2 more child participants as well as 15 new adult participants. The addition of adult participants is a welcome revision as it puts the study in a clearer trajectory. Without any adult participants, or other age groups to compare to, the study’s motivations and contributions were not clear. Below, I have detailed my concern for subject and item size and relatively less major (but still important) other concerns.

 

 

Major concerns:

 

1) The subject size has been increased to 16 (from 14) from the previous draft but the item size remains the same (6). I understand the authors’ concern regarding children’s attention span but that does not solve the generalizability/power problem the study faces. The authors could have provided several breaks during data collection, for instance, in ways that would suit the target population’s attention span.

 

In hypothesis testing we examine the fixed effects (e.g., the type of conditional connective in the present study) against random effects, i.e., an error term that captures the variability of effect across individuals. In repeated measures designs for instance, F = treatment effects + random, unsystematic difference / random, unsystematic differences (Wallnau & Gravetter, 2017). In psycholinguistic experiments variability of linguistic materials also need to be taken into consideration because just like participants they have certain variability that contribute to the error rates in the analyses (Clark, 1973).

 

In traditional ANOVAs, by-subject F values do not suffice and it has been common practice to report F1 and F2 values for both by-subject and by-item analyses, both of which was required to pass a min .05 alpha level. Barr et al. (2013), a reference that the authors also cite, show that this is also true for linear mixed effects models: “LMEMs generalize best when they include the maximal random effects structure justified by the design”; “to generalize across subjects and items, only min-F′ and maximal LMEMs can be said to be fundamentally sound … F1-only and random-intercepts-only LMEMs are fundamentally flawed”; and “failure to include maximal random-effect structures in linear mixed effects models (when such random effects are present in the underlying populations) would inflate Type I error rates”.

 

The present item size as well as the subject size (which is relatively better but still small) makes the analyses prone to increased error rates.

 

I am not sure when the study was conducted. If it is possible to reach out to the same participants (as well as adding a few more participants), without jeopardizing the age group tested, it may be reasonable and practical to collect more data with more items from the same subjects increasing power by items. This, although not ideal, can be considered as a break and would allow the authors to use the current data. An ideal scenario would be to rerun the experiments with 18-24 sentences (please see Barr et al., 2013 for simulations on 12-24 sample size), perhaps with breaks that would not challenge the age group’s attention span much, but I am not sure if the authors would prefer that revision.

 

For the ideal subject and item size the authors may as well conduct a power analysis as suggested in Brysbaert and Stevens (2018).  

 

The authors mention that “there are many relatively recent child language studies published in good journals prior to Covid 19 that have 4-6 items with around 20 participants in one age group (e.g., Goodman & Frank, 2016). This is typical in child language studies because it is not as easy to reach children as reaching adults who mostly come from unrepresentative undergraduate students.”

 

I disagree with this statement as this does not eliminate the power/generalizability issue. There are many experimental child language acquisition studies with good statistical power (see for instance Trueswell et al. (1999), Novick et al. (2008), Pozzan et al., (2015), Wang and Trueswell (2022)). There are even more difficult to reach populations such as sign-language users and even such studies have better subject and item sample size (see work by Kadir Gökgöz and colleagues, for instance). This note also does not justify the small item size.

 

The issue is also not the relative difficulty of collecting data from children compared to adult participants. It is a statistical generalizability/power problem. (The adult data also suffers from the same statistical generalizability problem and this note is contradictory in that regard.) When the data cannot generalize to target subjects or items, the conclusions drawn from the data/analyses would not be warranted.

 

The design is really complex with many conditions when pictures/ratios are also taken into consideration and I understand possible reluctance to this suggestion but if statistical generalizability is not warranted, neither are the conclusions.

 

References:

Barr D. J., Levy R., Scheepers C., Tily H. J. (2013). Random effects structure for confirmatory hypothesis testing: Keep it maximal. J Mem Lang. 68(3).

 

Brysbaert, M. & Stevens, M. 2018. Power analysis and effect size in mixed effects models: A tutorial. Journal of Cognition, 1(1), 1–20.

 

Gravetter, f. J. & Wallnau, L. B. (2017). Statistics for the behavioral sciences. Cengage.

 

Novick, J. M., Sharon L., Thompson-Schill, Trueswell, J. C. (2008). Putting lexical constraints in context into the visual-world paradigm, Cognition, 107(3), 850-903.

 

Wang, F. H., & Trueswell, J. (2022). Being suspicious of suspicious coincidences: The case of learning subordinate word meanings. Cognition, 224, 105028.

 

Pozzan, L., Gleitman, L. R. &. Trueswell, J. C. (2015): Semantic ambiguity and syntactic bootstrapping: the case of conjoined-subject intransitive sentences, Language Learning and Development,

 

Trueswell, J. C., Sekerina, I., Hill, N. M., & Logrip, M. L. (1999). The kindergartenpath effect: studying on-line sentence processing in young children. Cognition, 73, 89-134.

 

 

2) I wonder if the authors did any analyses on influential subjects and items as well as inspection of residuals during model fitting as is standard for regression analyses. I am not sure if this is mentioned in the report.

 

3) I am also not sure if the authors addressed my concern regarding the complexity of the experimental sentences. I believe they agree that these sentences are rather complex. Is there independent evidence showing that the children’s responses truly reflected how they processed/comprehended the uni/bi-conditionals in positive quantifier contexts? That is, could the data also indicate that they did not understand the sentences and their response data indeed perhaps reflected some good-enough processing? (When experimental sentences are too complex, even adults are reported to give up on comprehension.) Was there an independent task, question, etc. that ensured that children did comprehend the sentences, in an age/cognitive-level-appropriate manner?

 

4) p. 6-7, the authors revised the materials presentation, and the revised version is improved, but a few things are still not very clear. I am sorry for insisting on this, but it is important that the design is delivered properly. Did each child see 72 pictures in total and make a binary decision on those pictures? If so, each child heard the same experimental sentence twice in each list, is that correct? (because you mention that in each list there were 36 sentences.) Given that there are 6 sentences each in 3 connective types, then the participants heard a particular sentence more than once in each list. Is this correct? If so, since there were also no fillers, could children’s decision have been affected by their response to the same sentence they heard before (e.g., potential order/recall effects)?

 

 

 

Minor concerns:

 

p.5, ln. 234, final sentence: I think a better terminology would be “adult speaker” instead of “adult processor”.

 

p. 14, note 4: I think the adv “rather” must come before “limited”.

 

-----------

 

Finally, some notes on the response to reviewer letter:

 

Please note that the reviewers put in unpaid hours to provide feedback. As such, I find the following comments rather unfair and personal:

 

-        “Our reviewer is hinting that the present study is unnecessary from a “simple language processing perspective”. We agree that the issue is complex for researchers who are interested in adult syntactic processing and it raises more questions and puzzles than providing clear-cut answers for these issues. Yet, we would expect from our readers to have at least some familiarity with the simple logical terms and the semantic processing theories presented in this study to find some interest in the puzzles raised by the findings

 

-        “The target reader is not a psychologist who does not have much information about theoretical semantic issues, and similarly, the target reader is not an adult language processing researcher who does not have much knowledge in logical/semantic issues and developmental studies”

 

Although the statistical generalizations are a concern, I think the revised version of your manuscript does provide answers. The previous version was more confusing than informing. Also, I did not hint, by any means, that your research was “unnecessary”. The manuscript was simply very difficult to follow, as the motivations of your project, the predictions you made and the contributions you were aiming at were not clearly presented. One condition that you were manipulating was not even clear and misled me to consider it as a confound. This could have been due to the way the manuscript was written or due to some oversights. To ensure that it is not due to the latter, the manuscript must be easy to read/follow and leave no room for misunderstandings. I believe the authors would agree with this.

 

Regarding the authors’ note on the target audience: my comments were about the lack of focus in the way the study was reported, not related to the presumed field of study by reviewers. That is, the motivations for your study were informed by cognitive science, adult comprehension of conditionals, semantics/logic, and child language acquisition research. But the readers need a bridge between these fields and see the relevance and importance of the study for child language acquisition research as this seems to be the major motivation of the study. That bit needed to come through in the introduction, predictions and conclusions but was missing in the previous report, making it difficult to comprehend your work and its contributions to the field. The way the manuscript was written did not construct that bridge nor did it highlight that you were addressing an important question for a different target population. It does so now; clearing the clouds.

 

 

Author Response

Thanks to our reviewer for the valuable comments. Please see the attachment for our responses.

Author Response File: Author Response.pdf

Reviewer 2 Report

Comments and Suggestions for Authors

The authors did a great job addressing the comments from the previous round of reviews and the manuscript is much clearer and easier to follow. Happy to see this published

Author Response

Thanks to our reviewer for their comments and time.

Round 3

Reviewer 1 Report

Comments and Suggestions for Authors

Please see the attached file.

Comments for author File: Comments.pdf

Author Response

We would like to thank our reviewer for the comments. Our responses can be found in the attachment. Please see the attachment.

Author Response File: Author Response.pdf