Assessing Engineering Students’ Conceptual Understanding of Introductory Quantum Optics

Abstract

Quantum technologies have outgrown mere fundamental research in laboratories over recent years, and will facilitate more and more potentially disruptive applications in a wide range of fields in the future. In foresight, qualification opportunities need to be implemented in order to train qualified specialists, referred to as the future quantum workforce, in various fields. Universities world-wide have launched qualification programmes for engineers focusing on quantum optics and photonics. In many of these programmes, students attend courses on quantum physics contextualized via quantum optics experiments with heralded photons, because: (1) their experimental and physical foundations may be directly leveraged to teaching a number of quantum technology applications, and (2) physics education research has provided empirical evidence, according to which such quantum optics-based approaches are conducive to learning about quantum concepts. While many teachers are confident about the effectiveness of their concepts, there is little empirical evidence due to the lack of content-area-specific research tools. We present a 16-item concept inventory to assess students’ conceptual understanding of quantum optics concepts in the context of experiments with heralded photons adopted from a test instrument published in the literature. We have administered this Quantum Optics Concept Inventory as a post-test to $N=216$ students after instruction on quantum optics as part of an undergraduate engineering course. We evaluated the instruments’ psychometric quality, both in terms of classical test theory, and using a Rasch scaling approach. The Quantum Optics Concept Inventory enables a reliable measure ($\alpha =0.74$), and the data gathered show a good fit to the Rasch model. The students’ scores suggest that fundamental quantum effects pose striking learning hurdles to the engineering students. In contrast, most of the students are able to cope with the experimental and technical foundations of quantum optics experiments with heralded photons and their underlying principles, such as the coincidence technique used for the preparation of single-photon states. These findings are in accordance with prior research, and hence, the Quantum Optics Concept Inventory may serve as a fruitful starting point for future empirical research with regard to the education of the future quantum workforce.

1. Introduction

Quantum technologies have evolved dynamically in recent years [1]: From purely fundamental research in laboratories, they have increasingly moved toward potential disruptive applications in a wide range of fields until today [2,3]. To establish quantum technologies in future industry sustainably, it becomes necessary to train qualified specialists—the future quantum workforce [4], e.g., quantum engineers, since an increase in the “engineering talent flow [...] could vastly accelerate the development of quantum technologies” [5] (p. 3). According to [5] the term “quantum engineering” refers to the “application of engineering methods and principles to quantum information systems and problems” [5] (p. 3). Hence, qualification opportunities need to be implemented for this target group, both in terms of training at universities and in terms of training for professionals.

1.1. Educating the Future Quantum Workforce

Many initiatives have been implemented that aim at boosting the development of quantum ecosystems, and addressing the industry’s request for a quantum-literate workforce. A topical overview of the quantum workforce landscape is provided in Ref. [6]. For example, in the US, the Quantum Information Science and Technology Workforce Development National Strategic Plan [7] was set up, and with the National Q-12 Education Partnership, a consortium has been established with the aim of preparing “America’s next generation workforce with the tools to succeed in the industries of the future” [8]; see [9] for a comprehensive overview of Quantum Information Science courses at US institutions. In Europe, the Quantum Flagship [10] is flanked by the Coordination and Support Action for Quantum Technology Education (QTEdu CSA) [11], which aims at “bringing together quantum professionals from different walks of quantum education, research, and industry to help define the future of Europe’s quantum workforce as the second quantum revolution unfolds” [11]. With Canada’s National Quantum Strategy [12], the Sydney Quantum Academy in Australia [13], India’s National Mission on Quantum Technologies and Applications [14], or Singapore’s Quantum Engineering Program [15], similar efforts aimed at educating the future quantum workforce have been launched in other parts of the world.

In prior research, the industry needs for the future quantum workforce have been explored in different studies [16,17,18]. The results of such studies are essential for future efforts, since they may serve as a starting point for the development of target-group-specific educational programmes on quantum technologies, as has recently been shown with the development of a quantum engineering (undergraduate) programme presented in Ref. [5]. In the European context, a comprehensive Delphi study has been conducted within the QTEdu CSA, in order to collect quantum-specific competences that experts from academia and industry demand for the future quantum workforce [19,20]. The Delphi study results contributed to the development of the European Competence Framework for Quantum Technologies [21]. One prominent competence domain highlighted by the Competence Framework refers to quantum optics as a basis for quantum technologies, and closely related, its application in quantum photonics [21].

Quantum photonics and the underlying quantum optics fundamentals are highly relevant for almost all applications of quantum technologies, either as a core component of the application, e.g., in quantum imaging or quantum communication, or as an enabling technology [22]. Thus, quantum photonics are not limited to a specific application field. On the other hand, quantum photonics are the technology on the basis of which commercial applications can already be expected in the near future [2,22], e.g., in quantum sensor technology. For this reason, we believe that strengthening education in the field of quantum optics in (undergraduate) engineering programmes might have an impact on boosting developments in the field.

Different universities world-wide have already launched qualification programmes for engineering studies focusing on quantum optics and photonics [23,24,25,26]. Common are programmes that introduce students to the essential content domains of quantum physics via quantum optics fundamentals, most often starting from experiments with heralded photons (see [27]) for two main reasons:

(1)

The experimental and physical foundations of experiments with heralded photons may directly be leveraged to quantum technology applications within teaching scenarios, e.g., with regard to quantum computing [28], quantum metrology [29], or quantum information [30,31]. This content-specific argument is especially important with regard to undergraduate courses within study programmes for prospective quantum engineers.

(2)

Such experiments “provide the simplest method to date for demonstrating the essential mystery of quantum physics” [32] (p. 1), and “elegantly illustrate the fundamental concepts of quantum mechanics such as the wave-particle duality of a single photon, single-photon interference, and the probabilistic nature of quantum measurement” [33] (p. 1). Hence, such quantum optics-based approaches are likely to be conducive to circumvent widespread learning difficulties regarding quantum concepts, as has already been indicated by empirical results presented in Ref. [34].

1.2. Aim of This Study

Despite the above-mentioned advantages that potentially emerge from the integration of quantum optics courses with a focus on experiments with heralded photons into quantum engineering study programmes, no empirical results regarding the evaluation of such study programmes have been published in the literature so far. Not least, this is due to a lack of target-group- and content-domain-specific concept inventories that allow for the investigation of students’ learning progression. Therefore, the development of a concept inventory that can be used to inform instructors about engineering students’ mastery of quantum optics concepts constitutes a research desideratum to date. This is where this project comes in: We present a 16-item concept inventory to assess students’ conceptual understanding of quantum optics, with a focus on experiments with heralded photons on a qualitative level. We administered this Quantum Optics Concept Inventory as a post-test to $N=216$ students in the end of a four-week programme on quantum optics that is embedded into an undergraduate engineering course leading students from the basics of heralded photons experiments to the first insights into quantum technologies applications. Consequently, the objectives of the study presented in this paper are twofold: From a Rasch scaling approach, we

(1)

investigate as to how our concept inventory psychometrically functions on a sample of engineering students,

(2)

establish a difficulty scale regarding concepts covered by quantum optics experiments with heralded photons suggested by students’ scores on the concept inventory.

In Section 2, we provide insights into the research background regarding learning about quantum optics content domains, before the development of the Quantum Optics Concept Inventory for engineering students is sketched in Section 3. The research questions are formulated in Section 4, and, in Section 5, we outline the methodology employed in our study. The results obtained with regard to the research questions are given in Section 6, and finally, we discuss our findings against the backdrop of the literature in Section 7.

2. Research Background

In recent years, there has been an increase in the number of research projects aimed at introducing students of different levels and target groups to quantum physics, starting from a quantum optical perspective [32,35,36,37,38,39]. Although empirical research on teaching and learning quantum physics is gaining pace in general [40], there have been no empirical studies examining engineering students’ learning in such quantum optics courses to date. Contributing to closing this gap is a major goal of this study. In the next two Subsections, we present some results of research on the teaching and learning of quantum optics in secondary and undergraduate physics courses, in order to situate the study presented in this paper against the backdrop of the existing literature.

2.1. Assessment of Students’ Conceptual Understanding of Quantum Optics

Various concept inventories that allow for the assessment of students’ conceptual understanding of quantum physics have been published so far; a detailed overview of these concept tests aimed at the secondary school level or at the undergraduate physics level is given in Ref. [41]. These instruments either cover basic content domains of quantum physics (see [42,43,44,45]) or address aspects of the quantum formalism (see [46,47,48]).

The only test instrument on the introductory aspects of quantum optics is presented in Ref. [49]. This instrument consists of single-choice items, and has been developed in an iterative process, including different rounds of piloting. In terms of content, it covers both theoretical and experimental aspects of experiments with heralded photons. The content validity of the instrument has been ensured by means of an expert survey, and the tests’ psycometric properties have been analyzed in terms of classical test theory, based on data from $N=86$ undergraduate engineering students after instruction in introductory quantum optics. The test was found to enable a reliable measure ($\alpha =0.78$), and has good item statistics. This instrument serves as a starting point for the project presented in this paper: In Section 3, we describe how we adapted it for use on a sample of engineering students.

2.2. Students’ Conceptions on Quantum Optics Aspects

Two reviews provide a comprehensive overview of learning difficulties that students may encounter in quantum physics: one review focusing on secondary and lower elementary levels [41], and another one focusing on upper-level undergraduate students [50]. Currently, only a few studies with relatively small samples have been published exploring obstacles that students encounter with regards to quantum optics [33,51,52,53,54]. An exploratory interview study identifying $N=25$ students’ difficulties regarding different quantum optics content domains is reported in Ref. [55]. In [55], it was shown that (a) students are predominantly unprejudiced towards the experimental and technical foundations of quantum optics experiments, e.g., with regard to the preparation of single-photon states in experiments with heralded photons via coincident events on binary detectors, and that (b) most of the students can cope with the underlying principles [55]. These indications are also supported in further studies, e.g., exploiting the technique of probing acceptance [51]. In contrast, students specifically showed learning difficulties with regard to quantum effects and photons, which are in accordance with the findings from earlier studies into students’ difficulties with aspects of quantum physics (see, e.g., [56,57]), for example:

• a part of the learners interpret single-photon interference via photons that divide and then overlap with themselves [55] (p. 216);
• a part of the respondents equate photons with waves and conclude from this the necessity of observing interference phenomena [55] (p. 221);
• some learners claim that the photons’ localization would fail due to the small size of the photons [55] (p. 227);
• furthermore, a part of the participants believe that quantum anticorrelation at a beam splitter cube would be caused by photons behaving like haptical particles, being either reflected or transmitted [55] (p. 232).

With respect to the technical basics of quantum optics, learning difficulties regarding the detection of photons have been reported in the literature [51,58]: for example, a widespread misconception is that a pulse or click of a single-photon detector proves the detection of a single photon, but “this is not in agreement with the theory. A photo multiplier tube (PMT) or avalanche photo diode (APD) sends clicks, because engineers designed them to do so” [51] (p. 2).

For the development of the Quantum Optics Concept Inventory presented in this paper (see Section 3), we particularly referred to difficulties that students encounter with aspects of quantum optics that are documented in the literature, as presented in this subsection.

3. The Quantum Optics Concept Inventory for Engineering Students

We used the instrument described in Ref. [49], which is the result of a comprehensive process of development and piloting (described in Section 2), as a starting point for the development of our Quantum Optics Concept Inventory, focusing on qualitative aspects of experiments with heralded photons. We adopted the instrument for use in the primary target group of engineering students. Namely, we took two steps for the development:

(1)

we updated the existing items (and the corresponding distractors) from Ref. [49], in terms of language, and

(2)

beyond the content domains covered in the test of Ref. [49], namely (a) the theoretical and (b) the experimental basics of quantum optics, we developed new items to incorporate the thematic area of the technical basics of quantum optics experiments into the instrument, since we believe this content domain to be relevant to future quantum engineers.

In total, the Quantum Optics Concept Inventory consists of 16 single-choice items. In tier one, the participants are asked to select exactly one out of three answer options. In tier two, the participants are asked to rate their answer confidence on a five-point rating scale (1 stays for “I guessed”, 2 for “not sure”, 3 for “undecided”, 4 for “sure”, and 5 for “very sure”). A point is assigned if and only if the correct answer option was chosen in tier one, plus the respondent was at least sure with the response, meaning that a minimum of 4 was chosen on the confidence scale in tier two. Comparable confidence scales have previously been used for concept inventories, and they lead to the combination of two fundamental advantages [49,59,60,61]: On the one hand, the confidence scale

(1)

helps to minimize the effect of guessing, and hence, is “useful for gauging the quality of students’ understanding” [59] (p. 3); on the other hand,

(2)

it allows for the exploration of learning difficulties regarding the content area under investigation, namely by analyzing incorrect answers that were given confidently [62]. This point is postponed for the future research.

From a content-specific perspective, the Quantum Optics Concept Inventory aimed at engineering students is developed to test three content domains regarding quantum optics with heralded photons: technical-experimental foundations (domain 1), preparation of single-photon states (domain 2), and quantum effects (domain 3). Each domain is addressed by at least five items. An overview of the three content areas and the topics covered are given in

Table 1. Overview of the content domains covered in the Quantum Optics Concept Inventory, and the topics addressed with the related items. For a didactically prepared overview of the subject-specific topics represented in the test, e.g., the anticorrelation of single-photon states (see [18,51]). The list of items is given in Appendix A.

Content Domain	No. of Items	Topics Covered (Related Items)
Technical-experimental foundations	5	Set-up/adjustment of quantum optical experiment (items 1, 2, and 4), Single-photon detection (items 5 and 8)
Preparation of single-photon states	6	Parametric downconversion and energy conservation (items 3, 6, and 7), Coincidence technique (items 9, 10, and 15)
Quantum effects	5	Interference of single quanta (items 11, 12, and 13), Anticorrelation of single-photon states (items 14 and 16)

.

The content validity of the Quantum Optics Concept Inventory addressing the context of experiments with heralded photons was checked by means of an expert survey with $N=8$ faculty members [63]. For the investigation of further aspects of validity, in particular with regards to construct validity, we checked the fit of the data gathered in our study to a one-dimensional Rasch model. Therefore, we refer the reader to our methods and results in Section 5 and Section 6. Beyond that, we also describe reliablity issues in the results in Section 6.

4. Research Questions

The psychometric characterization of the Quantum Optics Concept Inventory (see Section 3) with respect to the primary target group of engineering students is the main goal of this study. Hence, the first research question (RQ1) to be addressed is:

RQ1:

How does the Quantum Optics Concept Inventory function on a sample of engineering students?

In the literature, qualitative approaches have been taken so far to explore difficulties that students encounter with quantum optics concepts (see Section 2). We have outlined findings from prior studies according to which students seem to be unprejudiced towards experiments with heralded photons, in particular, regarding the preparation of single-photon states, and most of the students can cope with the underlying principles. Conversely, quantum physical effects pose more difficulties to learners. Hence, to draw on earlier studies, and to shed new light on the difficulties that engineering students encounter with certain content aspects of quantum optics experiments with heralded photons using our concept inventory, we approach a clarification of the following second research question (RQ2) in this paper:

RQ2:

What difficulty scale of quantum optics content aspects is suggested by students’ scores on the concept inventory?

5. Methods

5.1. Test Administration and Sample

The Quantum Optics Concept Inventory used was administered to $N=216$ second-year engineering students (144 males, 68 females, 4 not specified) as a post-test after the students had participated in a four-week program about quantum optics with heralded photons. This four-week program was integrated in a regular introductory physics lecture for future engineers. In Section 5.2, a brief overview of the four-week intervention is given while in Appendix B, we provide an in-depth description of the educational pathway and the content aspects covered. None of the participants had any prior instruction in quantum physics beyond school physics.

The participation in our study was voluntary, and was not financially recompensed. In addition, the students were informed about anonymity, as well as about processing of their data, and were asked to give their consent to participate.

5.2. Intervention

The educational pathway of the four-week program was based on an introductory quantum optics course originally developed for the secondary school level as well as for the lower undergraduate levels, and is described in detail in [51]. For the purpose of this study, this concept was adapted in terms of content and depth—specifically, two important modifications dedicated to future engineers have been made to the original version [51] for the four-week program. Namely,

(1)

while the original teaching concept exclusively covered aspects of experiments with heralded photons qualitatively to make quantum effects accessible [51], during the course implemented in this study, the same topics were also covered on a formal level, as proposed in [64];

(2)

to conclude, the course implemented in this study provided future engineers with an outlook on quantum technologies, namely by dealing with quantum cryptography, e.g., the BB84 protocol with single-photons [65,66].

The four-week program was divided into two consecutive parts, and culminated in the discussion of the 1986 paper by Grangier et al. [67]. Hence, from a theoretical point of view, the program was based on the theory of optical coherence, which “investigates the properties of light in terms of correlation functions between two distinct fields or of a single field at different points in time and space” [68] (p. 144).

It is noteworthy that the development of a well-grounded proposal for educating the future quantum workforce is beyond the scope of this paper. However, to provide a deeper background of the intervention implemented in this study, we sketch the essential topics that were covered in the four-week quantum optics course, in Appendix B. We reflect on the role of the educational path for the results of this study in the limitation (see Section 8.1).

5.3. Data Analysis

In the following, we present the analysis tools incorporated into our research, bundled for each research question.

5.3.1. Analysis Carried out to Answer RQ1

In a first step, we provide a characterization of the instrument in the sense of classical test theory; i.e., Cronbach’s $\alpha$, as well as the items’ point-biserial coefficents and items difficulties are presented. Here, we refer to the accepted ranges of $0.2$ to $0.8$ for the item difficulty [69] and values $\ge$0.20 for the point-biserials [70]. For Cronbach’s $\alpha$, values above $0.7$ are considered as acceptable [71].

In a further step, the goal is to divorce statements regarding the concept inventory from the sample it was piloted with. This is enabled via dichotomous Rasch scaling. To justify this analysis, we investigated the pre-conditions of Rasch scaling (see [72]) by verifying that:

• the skewness and kurtosis of the items do not exceed the range of $-2$ to $+2$,
• the items are locally independent, and
• the uni-dimensionality of the construct can be assumed.

It is noteworthy that a verification of our instrument’s uni-dimensionality also supports construct validity [60].

After the pre-conditions of Rasch Scaling are met, the participants’ ability levels and the items’ difficulties are estimated. The goodness-of-fit parameters, Outfit mean-square (MNSQ) and Infit MNSQ, further describe how well the items fit the model—their expected value is 1; values below this indicate unmodeled noise, and values above this indicate redundancy, meaning that the respective item does not contribute much to estimating the participants’ latent trait levels [73]. The accepted ranges for these fit statistics are between $0.7$ and $1.3$ [74]. All parameters were calculated using the software R (Version 4.1.2) and its packages ltm (1.0-2) [75], TAM 4.0-16 [76], and eRm (1.0-2) [77].

5.3.2. Analysis Carried out to Answer RQ2

In our concept test, three quantum optics content domains are addressed with at least five items each (see Table 1). To compare the difficulties of the items related to each of these content domains, we calculated their average values, $\mu$ (in logits), and their uncertainties in terms of standard errors (SEs), as has been performed in prior studies [78].

Since the average difficulty of all items is set to zero in the Rasch modeling approach, item groups with negative average item difficulties are easier to solve for the sample students than the test average, whereas item groups with positive average item difficulties are more difficult than the test average.

6. Results

6.1. Psychometric Characterization

First, a distractor analysis was conducted, which revealed that for most of the items of our concept inventory, each answer option was selected by at least $5\%$ of the participants, which constitutes a widely used criterion in test development (see [49,60]). Next, we analyzed item difficulties and point-biserial coefficents in the sense of classical test theory (see

Table 2. Each item’s difficulty and point-biserial-coefficent, as well as the adjusted Cronbach’s $\bar{\alpha }$ if the respective item is dropped. See text for details.

Item No.	Difficulty	Point-Biserial	$\bar{\mathit{\alpha }}$
1	0.15	0.35	0.73
2	0.37	0.30	0.73
3	0.24	0.46	0.72
4	0.91	0.22	0.74
5	0.43	0.48	0.72
6	0.63	0.39	0.73
7	0.82	0.33	0.73
8	0.37	0.29	0.74
9	0.80	0.21	0.74
10	0.59	0.20	0.75
11	0.54	0.34	0.73
12	0.20	0.34	0.73
13	0.35	0.44	0.72
14	0.32	0.31	0.73
15	0.35	0.34	0.73
16	0.61	0.36	0.73

). It can be observed that Cronbach’s $\alpha$ increases only slightly when item 10 is omitted. The item difficulties and the point biserials show a reasonable range, overall.

A dichotomous Rasch model was justified by the data. Values of skewness and kurtosis are given in

Table 3. Overview of the relevant parameters for a dichotomous Rasch Model; “SE” stands for the standard error of item difficulty. See text for details.

Item No.	Skewness	Kurtosis	Difficulty	SE	Infit	Outfit
1	1.99	2.00	2.096	0.206	0.929	0.916
2	0.54	−1.72	0.648	0.156	1.056	1.042
3	1.22	−0.51	1.393	0.174	0.902	0.772
4	−2.83	6.07	−2.697	0.247	0.985	0.873
5	0.30	−1.93	0.364	0.152	0.902	0.889
6	−0.54	−1.72	−0.652	0.155	0.951	1.139
7	−1.63	0.67	−1.787	0.189	0.956	0.952
8	0.54	−1.72	0.648	0.156	1.079	1.048
9	−1.48	0.20	−1.649	0.183	1.080	1.109
10	−0.38	−1.87	−0.462	0.153	1.148	1.197
11	−0.15	−2.00	−0.184	0.151	1.022	1.002
12	1.48	0.20	1.647	0.183	0.987	0.907
13	0.63	−1.62	0.745	0.157	0.924	0.931
14	0.80	−1.37	0.948	0.161	1.028	1.003
15	0.63	−1.62	0.745	0.157	1.017	0.973
16	−0.46	−1.81	−0.556	0.154	1.005	0.967

. Item 4 does not meet the criterion mentioned in Section 5; however, it was retained for content reasons, as we elaborate on in Section 7. Local independence was verified by checking the $Q_3$ correlation matrix for values above $0.2$ [79,80]. Uni-dimensionality was confirmed using the R-package sirt (version 3.9-4), finding weighted indices $\mathrm{DETECT}=-0.041$ (<0.20), $\mathrm{ASSI}=-0.017$ (<0.25), and $\mathrm{RATIO}=-0.036$ (<0.36) [81]. In summary, the pre-conditions of Rasch scaling were met.

The “expected a posteriori/plausible value” (EAP/PV) reliability was found to be $0.746$, and the “weighted likelihood estimates” (WLE) reliability was $0.724$, exceeding the lower threshold of $0.5$ according to [74]. An overview of all the estimated parameters by the Rasch model is presented in Table 3. The values of the Infit MNSQ and Outfit MNSQ all lie in the accepted range indicating a good Rasch homogeneity. The Infit statistic varies between $0.902$ (item 3) and $1.148$ (item 10), with a mean value of $\langle \mathrm{Infit}\rangle =1.004$. The Outfit statistic varies between $0.772$ (item 3) and $1.197$ (item 10), with a mean value of $\langle \mathrm{Outfit}\rangle =0.983$. A graphical representation of the findings is given in Figure 1.

The item difficulties may be investigated by visualizing their relationship to the solving probability. In this regard, each item can be described by a logit function, where the solving probability is plotted against the latent trait level on a continuous scale. The resulting graph of this function is characteristic for the item, and is thus called the item characteristic curve [74]. The curves are shown in Figure 2.

The item difficulty ranges from $-2.697$ logits (item 4) to $2.096$ logits (item 1). More important, however, is how the items correspond to the latent trait levels within the sample. This connection is investigated in Figure 3 using a Wright Map. The Wright Map shows the distribution of item difficulty and respondent latent trait level on a common scale, and thus enables an assessment of how well the instrument is suited to measuring different areas of ability. For example, Figure 3 shows that in densely populated areas, the item density reacts accordingly, but in the upper end of the ability scale (≥2 logits), no item corresponds, meaning that our concept inventory is not suited to distinguishing between participants with very high trait levels. In other words, in the future, this concept inventory may be refined by developing at least one more difficult item.

In summary, the results presented in this Subsection may contribute to a clarification of the research question RQ1. We judge the overall psychometric quality of the Quantum Optics Concept Inventory and its functioning on our sample of engineering students in Section 7 below referring to the literature on concept inventories.

6.2. Difficulty Scale of Quantum Optics Concepts

The Rasch scaling of our concept inventory was further leveraged to investigate the mean difficulty of the different content domains covered in our diagnostic tool (see Table 1), as has been performed in prior research using concept inventories (see [82]).

The most difficult content domain appears to be domain 3 (quantum effects), with an average item difficulty of ($\mu =0.52$, $\mathrm{SE}=0.35$), as is graphically shown in Figure 4. The next content domain according to difficulty is the domain 1 on technical and experimental foundations of quantum optics experiments with heralded photons ($\mu =0.21$, $\mathrm{SE}=0.71$). This content domain, in total, appears to be slightly more difficult than the average. In contrast, the content domain 2 on the preparation of single-photon states ($\mu =-0.40$, $\mathrm{SE}=0.47$) lays well below the average difficulty, and hence, the corresponding items did not pose a problem for a majority of the students. In summary, these results allow for the establishment of a difficulty scale of quantum optics concepts with respect to experiments with heralded photons, and hence, they contribute to a clarification of research question RQ2.

In Section 7 just below, we discuss the observed differences between the three content domains covered in our concept inventory with regard to their average item difficulties. There, we provide evidence that the findings of our study are consistent with prior research, and derive implications for the future. Based on the results obtained, it is not possible to resolve statistically significant differences regarding the mean item difficulties on a $5\%$ level. This problem can be addressed by adding further items to the concept inventory covering the three content domains, respectively. This point is addressed in Section 8.1.

7. Discussion

7.1. Discussion of RQ1

From the standpoint of classical test theory, the item difficulties and point-biserials lie in the desired range for all items (except for item 4) and constitute a scale with good internal consistency expressed by Cronbach’s $\alpha =0.74$.

The Rasch analysis of the concept inventory showed a good degree of functioning for almost all test items. Only item 4 may be revisited due to its skewness and kurtosis values, even though it shows good fit statistics. The Wright Map (see Figure 3) illustrates how well the items can be ordered along the logit scale; they measure a conceptual understanding of quantum optics on different trait levels, and most items measure around the 0 logit area, which is most densely populated. We conclude that the test is well designed for the population that it is intended to be applied on. Merely for very high trait levels, at least one further item should be developed in the future. Additionally, one could make a case for also developing items in the region of about $-2.5$ logits and about $-1.5$ logits, as visual gaps can be observed both in the Wright Map, as well as in the overview of the item characteristic curves (see Figure 2). Further refinement to close this gap, and thus, to contribute to an overall continuous item difficulty distribution may result in a further refined concept inventory for quantum optics in engineering curricula.

In essence, only item 4 turned out to be problematic. It seemed too easy for our sample, from the perspective of classical test theory (item difficulty $0.91$). In addition, its high skewness and kurtosis suggest a deviation from the normal distribution. However, the Wright Map substantiates that items of such difficulty ($-2.697$ logits from the Rasch perspective) are in fact necessary, and no other item measures similarly at the lower end of the scale. Furthermore, item 4 is important for content reasons, since it is the only item of the instrument that asks for a differentiation between single-photon from further types of quanta, e.g., single electrons, in this case. The mix-up of photons and electrons, however, has been proven to be a learning obstacle to students in [83,84]. To this end, this item is to be retained in order to ensure content validity. Additionally, such outlier items are considered in a categorial judgement scheme provided in [70]. We use this judgement scheme to summarize the above discussed observations, allowing for a holistic judgement regarding all relevant concept inventory characteristics. In this regard, we follow Ref. [60], where this judgement scheme has been expanded to Rasch scaling parameters. The overall quality of our concept inventory is summarized in

Table 4. Categorial judgment scheme and assignment rules for evaluating a concept inventory according to [70]. Values in parenthesis indicate the number of items that can fall outside of this recommendation. The judgement of our concept inventory is presented in the last column labeled QOCI.

Analysis			Excellent	Good	Average	Poor	QOCI
Classical Test theory
	Item Statistics
		Difficulty	$0.2$–$0.8$	$0.2$–$0.8$ (3)	$0.1$–$0.9$	$0.1$–$0.9$ (3)	good
		Discrimination	>0.2	>0.1	>0	>−0.2	excellent
	Total score reliability
		$\alpha$ of total score	>0.9	>0.8	>0.65	>0.5	average
		$\alpha$-with-item-deleted	All items less than overall $\alpha$	(3)	(6)	(9)	good
Item Response Theory
	Individual item measures
		Infit MNSQ	$0.7$–$1.3$	$0.6$–$1.4$	$0.5$–$1.5$	–	excellent
		Outfit MNSQ	$0.7$–$1.3$	$0.6$–$1.4$	$0.5$–$1.5$	–	excellent
		All items fit the model	(2)	(4)	(6)	(8)	excellent

.

7.2. Discussion of RQ2

The Quantum Optics Concept Inventory is developed to test students’ conceptual understanding of quantum optics basics, with a focus on experiments with heralded photons focusing on three content domains, namely technical-experimental foundations (domain 1), the preparation of single-photon states (domain 2), and quantum effects (domain 3).

The data analysis presented in Section 6 revealed that the content domain 3 addressing quantum effects was the most challenging one for the study participants (average item difficulty, $\mu =0.52$). The content domain was covered by six items, three of which address single-photon interference (items 11, 12, and 13), and two of which focus on single-photons and the anticorrelation effect (items 14 and 16). Analyzing these items in more detail, two observations become striking.

(1)

The item difficulties of the items 12 ($1.647$ logits), and 13 ($0.745$ logits) both lay well above average—hence, single-photon interference obviously poses conceptual difficulties to the study participants. This observation is in accordance with prior research, and can be enriched by findings from various qualitative studies exploring student difficulties on the interference of single quanta [33,55].

(2)

Item 14, focusing on the anticorrelation effect of single-photon states at an optical beam splitter cube, has an item difficulty of $0.948$ logits. Hence, the concept that photons are either transmitted or reflected at a beam splitter cube seems to be difficult for students. This observation fits well with results presented in an earlier contribution [51], where the authors used a micro-intervention to explore students’ understanding of single-photons’ anticorrelation, using the technique of probing acceptance. Beyond that, this observation is particularly striking against the background of the finding described above regarding single-photon interference: while items 12 and 13 address the wave nature of photons, item 14 can be associated with a particle nature of photons. Hence, the three items represent one of the fundamental issues of quantum physics: wave-particle duality.

Taken together, the observations fit well with previous studies into students’ conceptual hurdles regarding the wave-particle duality of quantum entities (see [55,85,86]; for a review of studies on this topic, see [41]). In summary, it is not a surprise that in Ref. [87], the authors demand “a radical reconstruction” (p. 257) of learners’ initial knowledge when it comes to learning about quantum physics.

The items related to the content domain 1 on technical-experimental foundations had an average item difficulty of $\mu =0.21$. The content domain is covered by five items of the concept inventory, which cover the whole logit scale regarding item difficulties, ranging from $-2.697$ logits (item 4) at the lower end of the difficulty scale, to $2.096$ logits (item 1) at the upper end. While the more difficult items of this content domain (item 1, with $2.096$ logits, and item 2, with $0.648$ logits) cover experimental aspects, e.g., the use of beam splitter cubes (item 1) or the adjustment of optical experiments (item 2), the other items covering technical aspects appear to have low-to-average item difficulties ($-2.697$ logits for item 4, $0.364$ logits for item 5, and $0.648$ logits for item 8).

Finally, the items related to the content domain 2 on the preparation of single-photon states had an average item difficulty of $\mu =-0.40$, and hence, were easier than average. This observation is in accordance to the results of prior (qualitative) studies, which revealed that students can predominantly cope well with the basic ideas underlying the preparation of single-photon states in experiments with heralded photons via coincident events on binary detectors [51,55]. To this end, we believe that the results obtained provide a further argument in favor of using the experimental setting of heralded photons as a gateway towards quantum effects in teaching quantum physics. The easiest items related to this content domain were items 6 ($-0.652$ logits) and 7 ($-1.787$ logits), addressing the concept of energy levels and energy conservation in the context of nonlinear optics. In contrast, the most difficult, item 3 ($1.393$ logits), referred to parametric downconversion (PDC), a nonlinear effect leading to the emission of photon pairs from a nonlinear crystal irradiated by a laser at a certain wavelength. An in-depth analysis of the students’ response behavior on the answer options of item 3 shows that the students tend to conceptually mess up PDC with reflection and diffraction from geometrical optics.

8. Conclusions

We presented a 16-item Quantum Optics Concept Inventory aimed at assessing engineering students’ conceptual understandings of the fundamental aspects of quantum optics experiments with heralded photons. This tool has been adopted from an instrument previously published elsewhere [49]. We added items related to the technical aspects of single-photon experiments, e.g., addressing single-photon detection, making it a valuable tool for evaluating teaching-learning scenarios.

• aimed at engineering university students, and
• designed to introduce these students to quantum physics and modern quantum technologies via quantum optics experiments with heralded photons.

8.1. Limitations

There are some limitations of our study that may serve as a springboard for future research. These limitations either stem from the study design itself (Section 8.1.1), or are related to the concept inventory evaluated in this study (Section 8.1.2).

8.1.1. Limitations of this Study

Without a doubt, the evaluation of the Quantum Optics Concept Inventory presented in this paper is—at least to some extent—dependent on the quantum optics instruction that the study participants had received prior to test administration. This a known concern in educational research in general [88]—especially when it comes to piloting novel assessment tools. However, in the current study, this issue was mitigated by means of Rasch scaling utilized for the psychometric characterization of the Quantum Optics Concept Inventory, since the Rasch model permits the separation of person and item parameters [89].

Against the backdrop of the above argumentation, it becomes apparent that the difficulty scale of quantum optics concepts established on the basis of the participants’ test results (see Section 6.2) should not be confused with statements about the perceived difficulty of quantum optics topics. Rather, this difficulty scale should be understood to shed more light on (a) the items of the Quantum Optics Concept Inventory, and (b) on possible learning difficulties that might pose obstacles to student learning in the context of introductory quantum optics (see Section 7.2). Taken together, the results presented in this study may not only lead to a revision of the Quantum Optics Concept Inventory, but they might also initiate the refinement of the four-week program implemented in a lecture for future engineers in the course of this study.

8.1.2. Limitations Related to the Quantum Optics Concept Inventory

We merged the viewpoints of classical test theory with those of probabilistic test theory to provide a psychometric characterization of the Quantum Optics Concept Inventory. Based on the data gathered from $N=216$ engineering students who took the Quantum Optics Concept Inventory as a post-test after instruction, we found our instrument to exhibit average-to-excellent psychometric properties following the categorical judgment scheme for evaluating a concept inventory of Ref. [70]. In particular, the Rasch analysis results revealed that the items discriminate well among students from lower to medium–high competence levels (see Figure 3). However, the items do not manage to discriminate well between students at the upper end of the ability scale. Hence, in order to refine our instrument accordingly, items with a higher difficulty should be added.

Analyzing the data in more detail with respect to the content domains covered in our inventory (see Table 1), findings from prior research could be replicated, adding further evidence for the functioning of our concept inventory: The average item difficulty for the items related to the content domain 3 on quantum effects lies well above the average. Specifically, we found that items addressing different aspects of wave-particle duality, i.e., items 12, 13, and 14, covering single-photon interference and the anticorrelation of single-photon states, were found at the upper end of the difficulty scale. This is consistent with the wave-particle duality posing several conceptual hurdles to learners from different target groups, as is well documented in the literature [41]. However, more differentiated statements on student conceptions, especially regarding quantum optics concepts, are difficult to make with the current version of our concept inventory: since for now, all topics are covered with only two or three items each, further items would be required in order to allow for a more comprehensive insight into students’ conceptions regarding the respective isolated topics. In contrast, the brevity of our concept inventory allows for economic practical use aimed at the evaluation of teaching–learning environments. Against this backdrop, the creation of additional items is always to be considered carefully, in order to maintain the practical applicability of the instrument.

Finally, at the qualitative level, an important limitation must be considered, which stems from the fact that the items of the Quantum Optics Concept Inventory represent a compromise of the following criteria:

(i)

intelligible wording,

(ii)

coverage of overarching concepts of quantum optics in the heralded photon realm, rather than addressing all the details of a topic,

(iii)

subject-specific correctness.

Item 7 of the Quantum Optics Concept Inventory (see Appendix A), which deals with PDC, may serve as a prime example of this tradeoff: While PDC photons in general have different frequencies, in this item, we focus on the particular situation where the two PDC photons have equal frequencies. Hence, this item is intended to evaluate whether students have understood that energy is conserved in the PDC process. In contrast, assessing students’ conceptual understanding of details of phasematching conditions is outside the scope of this item (see bullet 2 in the above list). Again, developing additional items that cover further topics and aspects related to quantum optics experiments with heralded photons could help alleviate the constraints imposed by the aforementioned compromise.

8.2. Outlook

In the future, standardized feedback formats need to be established, such that outcomes can easily be communicated to various stakeholders, e.g., to teachers or to their students [90]. For this purpose, instead of presenting statistics based on raw test values, feedback formats that map the distribution of student performance on categorically graded levels of proficiency are required. Future studies may draw on the determination of such levels of proficiency with respect to the content area under investigation, in order to describe (and predict) students’ learning progressions [91].

The number of proficiency levels regarding quantum optics experiments with heralded photons that can be empirically separated using our concept inventory depends on the person separation reliability of our concept inventory. According to [92], the person seperation reliability, s, can be used to calculate the G-value,

$$G=\sqrt{\frac{s}{1-s}},$$

which allows for the determination of the number of empirically separable proficiency levels via

$$\left[\frac{4G+1}{3}\right].$$

Here, $[-]$ is the closest integer; for details, see [74]. For the Quantum Optics Concept Inventory under investigation, we obtained $s=0.75$, and hence, $G=\sqrt{3}$, meaning that it allows us to differentiate between

$$\left[\frac{4G+1}{3}\right]=3$$

proficiency levels among engineering students. After a revision of our instrument to tackle the above-mentioned weaknesses, future research will address the definition of the proficiency levels regarding quantum optics experiments with heralded photons based on data from larger samples.