This experiment consisted of a scalar Grammaticality Judgment task in which participants used a five-point Likert scale (1: Strongly Agree, 2: Agree, 3: Neither Agree nor Disagree, 4: Disagree, 5: Strongly Disagree) to evaluate the un/grammaticality of forty-six English sentences presented to them in a random order. Similar to the TVJ task in Experiment 1, the main goal of this experiment was to test the sensitivity of L2ers to Superiority and Subjacency violations associated with wh-interrogatives in English. In addition, this task also tested whether our participants were sensitive to the grammaticality distinction across the two kinds of superiority violations in English wh-questions, as in (32).
4.3. Results
In preparation for the statistical analysis, we computed mean scores for each participant as he/she judged the grammaticality of wh-questions for the three conditions. In order to do this, participants’ judgments on the five-point scale (strongly agree = 1 to strongly disagree = 5) were averaged.
Figure 2 shows the mean choices of the answers for the three participant groups.
When both kinds of Superiority violations (Test column in C1 in
Figure 1) are considered together, both native controls (Mean = 3.8, SD = 0.96) and L2ers (Adv/L2: Mean = 3.9, SD = 1.0, Inter/L2: Mean = 3.8, SD = 0.67) showed very similar rates of rejection of ungrammatical sentences resulting from Superiority violations. Regarding control items in C1, English monolinguals (NS mean = 2.8, SD = 1.0) and L2ers (Adv/L2: Mean = 3.6, SD = 1.2, Inter/L2: Mean = 3.6, SD = 0.80) were slightly different only in that English monolingual showed a lower rate of acceptance of arguably grammatical counterparts to Superiority violations.
In the test of Subjacency (C2), both English monolinguals and L2ers performed very similarly in both the test and control conditions, in rejecting the test sentences with Subjacency violations. All three groups are in the higher end of the five-point acceptability scale: English monolinguals (Mean = 4.5, SD = 0.49), advanced L2ers (Mean = 4.5, SD = 0.46), and intermediate L2ers (Mean = 4.1, SD = 0.72) all strongly and consistently rejected question sentences with Subjacency violations in this experiment. A very similar pattern is observed in C3 (Subjacency + Superiority violations) for both L2ers and native controls. Finally, notice that for all three groups, the mean difference between test vs. control items in C1 is not as substantial as what is observed in the other two conditions, when the two types of Superiority conditions are considered together.
Given the pattern that we have observed in descriptive statistics, we submitted these mean judgment scores to a repeated measures ANOVA with proficiency (3 levels: Native, Adv/L2, and Inter/L2) as between-subject factor, condition (3 levels: Superiority, Subjacency, combined), and grammaticality (2 levels: test vs. control) as within-subject factors. Both by-participant and by-item analyses showed significant effects of grammaticality (F1 (2,64) = 231.08, p < 0.001, F2 (1, 4) = 386.86, p < 0.001), condition (F1 (2,29) = 10.14, p < 0.001, F2 (2,3) = 9.93, p < 0.04), interactions of proficiency and grammaticality (F1 (2,65) = 15.27, p < 0.001, F2 (2,3) = 209.79, p < 0.001), condition and grammaticality (F1 (2,64) = 231.08, p < 0.001, F2 (2,3) = 665.76, p < 0.001), and a three-way significant interaction between condition, grammaticality, and proficiency (F1 (4, 12) = 5.47, p < 0.001, F2 (4,16) = 8.13, p < 0.001). However, proficiency alone (F1 (2, 65) = 0.72, p > 0.48, F2 (2, 3) = 4.1, p > 0.13) was not significant either in the participant or the item analyses. We take these results to imply that our participants in general judged grammatical conditions (test) differently from ungrammatical conditions (control), though at least one group showed a different level of sensitivity to the grammaticality distinction (test vs. control) in one or more conditions. In order to further explore the nature of this interaction involving proficiency, we conducted 3 (proficiency levels) × 2 (grammaticality: test vs. control) repeated measures ANOVAs for each condition separately.
As far as C1 (Superiority violations) is concerned, the interaction between proficiency and grammaticality proved to be significant (F1 (2, 65) = 10.21, p < 0.001, F2 (2, 3) = 19.80, p < 0.01). Proficiency had a main effect only in the item analysis (F1 (2, 65) = 1.75, p > 0.18, F2 (2, 3) = 8.95, p < 0.05), while grammaticality was only significant in the participant analysis, F1 (2, 65) = 26.50, p < 0.001, F2 (1, 4) = 1.76, p > 0.25. Subsequent post hoc (paired t-test) comparisons revealed that in C1, the grammaticality distinction (test vs. control) was significant for the native controls, t (30) = 6.87, p < 0.001, but not for the advanced L2 group, t (13) = 1.49, p > 0.15, or the intermediate L2 group, t (21) = 1.33, p > 0.19. Given this difference, we carried further analysis of the results in C1 considering the difference between the two types of superiority violations (argument-over-argument and argument-over-adjunct) as we discuss later in this section.
Meanwhile, C2 (Subjacency) also revealed an interaction of proficiency and grammaticality, F1 (2, 65) = 5, 30, p < 0.001, F2 (2, 8) = 14.56, p < 0.002 (due primarily to the results from Inter/L2 subjects, as we return to below). The effect of grammaticality also proved to be significant, F1 (1, 65) = 1063, 31, p < 0.001, F2 (1, 4) = 210.82, p < 0.001. However, crucially, in this condition, L2 groups, like native controls, showed strong sensitivity to the grammaticality distinction, as each group judged test items to be significantly worse than control items (Adv/L2: t (13) = 23.17, p < 0.001; Inter/L2: t (21) = 13.36, p < 0.001; Native: t (30) = 27.88, p < 0.001), as we discussed above.
In C3 (combined Superiority and Subjacency), we found a similar interaction between proficiency and grammaticality, F1 (2, 65) = 13.95, p < 0.001, F2 (2, 3) = 28.88, p < 0.01 (due to how Inter/L2 subject performed, as further discussed in the next paragraph). Despite this, for each of the three groups, the difference between grammatical and ungrammatical items was significant, Native: t (30) = 21.29, p < 0.001, Adv/L2: t (13) = 9.74, p < 0.001, Inter/L2: t (21) = 12.61, p < 0.001. That is, each group clearly rejected the test items involving subjacency and superiority violations, as opposed to the grammatical control items. Further analyses considering both C2 and C3 showed that the interaction between proficiency and grammaticality was significant only in C2 and C3, because the intermediate group performed slightly differently from the other two groups in their judgment of both test and control items. Despite this difference, the intermediate group was also sensitive to the grammaticality distinction in both C2 and C3.
Summarizing our results on the three conditions so far, native controls, as predicted, showed sensitivity to the grammaticality distinction in all three conditions. Meanwhile, L2ers were sensitive to this distinction only in C2 (Subjacency) and C3 (combined Subjacency and Superiority). However, we return below to C1 results and show that L2ers were in fact sensitive to a more fine-grained distinction among the test items in that condition.