Molecular Beam Epitaxial Growth and Nonlinear Optical Signatures of Single-Domain Bi₂Se₃

Abstract

We report a new approach to enhance the photonic response of thin-film topological insulator Bi₂Se₃ by significantly reducing twin domains and antiphase disorder. The strategy employs closely lattice-matched trigonal substrates combined with surface structuring to preferentially seed a single rotational domain before epitaxy. Characterization using optical second harmonic generation (SHG), nonlinear optical tensor analysis, X-ray diffraction, and atomic force microscopy confirms the near-single crystal Bi₂Se₃ heteroepitaxial layers. These results show a clear six-fold symmetric sin²(3ϕ) SHG pattern at normal incidence, and a vanishingly small 100-to-1 peak-height ratio from X-ray pole-scans showing negligible twinning. These results show that this approach can yield near perfect single crystal heteroepitaxial Bi₂Se₃ whose photonic properties converge to those of bulk-grown single crystals.

1. Introduction

The unique band structures of topological insulator (TI) materials enable transformative applications in photonics and electronics. These new applications leverage novel transport phenomena such as spin-momentum-locked carriers and immunity to backscattering, which arise from the Dirac-like energy-momentum dispersion of the metal-like surface state, coupled with an insulating bulk state. Within the (Bi,Sb)₂(Te,Se)₃ family of van der Waals TI materials, Bi₂Se₃ is particularly significant due to its large bulk energy gap and topologically protected surface states consisting of a single massless Dirac fermion [1].

These attributes of topological insulators have inspired new electronics architectures that could transcend CMOS limitations, such as Magnetoelectric Spin–Orbit (MESO) logic. MESO devices leverage strong spin–orbit coupling to convert between spin currents and charge currents, potentially overcoming the Boltzmann subthreshold slope that limits Moore’s law [2,3]. More interestingly, however, TIs enable advanced photonic devices that stem from their nonlinear properties and spin-polarized responses, including saturable absorbers with low saturation intensity [4], compact polarization-sensitive detectors [5], novel lasers [6,7] and broad-band infrared detectors [8].

Because of their unique non-linear optical responses and low saturation thresholds, the nascent application of topological insulators as saturable absorbers has shown unusually strong promise by demonstrating high-power and short-pulse laser generation at technologically important laser wavelength bands [9,10]. With continued refinement of materials’ quality to enhance thermal dissipation and optimize nonlinear optical performance, Bi₂Se₃ materials have potential to strongly compete with other 2D-material-based saturable absorber candidates [11,12].

Despite the potential of these new device applications, transitioning TIs to chip-scale technologies requires highly controlled thin-film synthesis. While early insights were derived from long-range order of bulk-grown crystals that provide well-defined band structures, the historical prior-art thin film growth by molecular beam epitaxy (MBE) often introduces significant microstructural defects and is replete with antiphase disorder and crystallographic twin domains.

For photonics applications, these defects are detrimental; they reduce the efficiency of photonics response and induce parasitic scattering between surface and bulk states [13,14]. Twinning domains and anti-phase disorder have also been shown to degrade the overall photonic response of Bi₂Se₃ because they introduce competing nonlinear optical (NLO) responses [14]. Second harmonic generation (SHG) is an all-optical probe of the NLO susceptibility tensor and topological surface states and can serve as a direct figure-of-merit to probe the material’s crystal symmetries and the effectiveness of reducing twinning and antiphase disorder [15,16].

Prior studies of both Bi₂Se₃ bulk-grown crystals and thin films have used SHG to interrogate the surface, interface and bulk contributions to the NLO response [13,15,16,17,18]. Thickness-dependent SHG responses with clear six-fold symmetric responses for thin films with thicknesses less than 20 nm and attributable in-plane and out-of-plane contributions have been reported [18]. A deviation from this behavior is observed in highly twinned thin films with an isotropic, circularly symmetric SHG emission profile with essentially no azimuthal dependence [13]; that result is attributed to a high degree of twinning domains.

To improve the understanding of these interactions, foundational theoretical models have evolved that explore the relative magnitude of NLO tensor elements in SHG to describe the contribution from the bulk and from surface electrons on the SHG signal in bulk-grown crystals [15]. This approach was subsequently applied to investigate the effect of surface charge and to probe the time-reversal symmetry properties of Bi₂Se₃ using excitations caused by circularly polarized photons [16]. Further, excitations using circularly polarized photons can robustly probe the time-reversal symmetry properties of a TI. In comparative studies, separating bulk and surface contributions to the SHG signal, researchers observed that surface contributions dominate in thin film samples, though the overall signal intensity is reduced relative to that from bulk samples [17]. Based on these findings, prior literature has proposed SHG as an in situ probe of TIs, provided that an appropriate microscopic model is employed to explain the relative bulk vs. surface contributions.

In this paper, we demonstrate a novel epitaxial growth strategy designed to overcome these microstructural limitations and synthesize high-quality Bi₂Se₃ thin films. By utilizing a closely lattice-matched substrate with trigonal symmetry and precisely structuring the surface before epitaxy, we preferentially seed a single rotational domain. We employ SHG as a primary figure-of-merit, alongside structural diagnostics including X-ray diffraction (XRD) and atomic force microscopy (AFM), to quantify the resulting crystalline quality. Our measurements confirm that our growth technique achieves a significant reduction in twinning domains and a six-fold symmetric SHG pattern. We perform a detailed nonlinear tensor analysis to support these results. This work establishes that engineered thin-film Bi₂Se₃ can achieve photonic properties equivalent to those of bulk single crystals, removing a major barrier to the chip-scale integration of topological insulators.

2. Materials and Methods

2.1. Reducing Twin Domains

The weak interlayer bonding of two-dimensional (2D) van der Waals materials enables the epitaxial integration of thin film layers on any substrate, independent of lattice matching and crystal symmetries. However, this flexibility typically results in oriented but highly textured films. For example, while Bi₂Se₃ films grown on amorphous substrates have some crystallinity, they are highly textured with significant crystallographic anti-phase disorder and a high degree of random in-plane rotations [19]. Because bulk-grown Bi₂Se₃ is nominally single domain, achieving bulk-equivalent photonic responses in thin films requires elimination of these microstructural defects [13,14].

One of the legacy interests of 2D van der Waals materials is monolayer thickness devices (e.g., transistors). Monolayer-thickness devices have shown remarkable success, but many photonics applications (e.g., polarization-sensitive photodetectors and saturable absorbers) benefit from thicker, bulk-like films. While there has been a dearth of research into thick film growth, and defects, there have been few studies on how to control twinning defects in 2D materials.

Single crystal substrates with trigonal surface symmetry have been employed to reduce defects like twinning in 2D van der Waals materials because they can guide the orientation and crystallinity of the subsequent film [20]. Redwing’s group reported that by modifying the surface vacancy content of hexagonal boron nitride substrates, twin domains could be reduced for epitaxial layers of WSe₂ despite the large lattice misfit [21]. They also explored using step edges on trigonally symmetric sapphire to align WSe₂ deposits and control the rotational anisotropy by chemically modifying sapphire’s top-most atomic surface and found reasonable control but still reported large-scale polycrystalline disorder with grain boundaries [22]. Thus, it can be concluded that the single-crystal substrate approach can be somewhat successful even with large lattice misfit for reducing twins and antiphase disorder in 2D van der Waals materials because it reduces the rotational symmetries that are obtained compared to the completely random case. This is an improvement because a much smaller subset of domain rotations is obtained.

For the case of rhombohedral (001) Bi₂Se₃, InP (111)B is a particularly attractive candidate due to its low in-plane lattice misfit of 0.16%. While this symmetry reduces the number of possible domain rotations, it generally results in two distinct triangular shaped domains that are mirror-symmetric by one 180° rotation.

Recent efforts to suppress these twinning domains have relied on specialized surface preparations. Randomly roughening InP (111)B surface and subsequent Bi₂Se₃ MBE growth has been reported to result in single-domain films [23]. However, the dimensionality of the required roughness is not quantified, so repeatability and scalability appear limited. Another study employs a complex thermal treatment to form a metastable In₂Se₃ heterostructure buffer layer to bridge InP and Bi₂Se₃ layers and was reported to yield single-domain Bi₂Se₃ [24]. This approach lacks repeatability due to the varied phase transitions of metastable In₂Se₃ and its incipient ferroelectric properties, which can introduce parasitic effects to the desired photonic response.

This work introduces a new technique that is repeatable and precise, utilizing a crystallographic vicinal misorientation of the InP (111)B substrate. Our strategy builds upon prior findings that explored a range of unusually high-index oriented substrates and reported that Bi₂Se₃ epitaxial films preferentially nucleate on the InP (111)B step ledges, which naturally form along <110> type directions [25]. While these prior studies that use high-index orientations yield surfaces that are far too rough for photonic devices, our approach uses a small-angle vicinal surface that seeds one preferential domain orientation, while simultaneously maintaining the atomic smoothness that is essential for photonic devices. This novel approach yields bulk-quality Bi₂Se₃ in thin-film form and eliminates the need for random surface roughness or complicated heterostructure buffer layers.

2.2. Molecular Beam Epitaxy and Carrier Transport

Thin 10 nm films of Bi₂Se₃ were grown by MBE on epi-ready InP (111)B with either 2° offcut, or nominally on-axis (±0.5°). After oxide desorption, the substrate was cooled to the growth temperature of 493 K and Bi₂Se₃ heteroepitaxy was initiated with a Se:Bi flux ratio of 100:1 to reduce selenium vacancy formation. After MBE, the obtained specular films were continuously cooled to room temperature under an impinging flux of selenium molecules such that a protective capping layer of semi-crystalline selenium adsorbs on the surface before removal.

Although the primary focus of this work is the photonic response, carrier transport was evaluated as an independent metric of material quality and to ensure consistency across different growths. Room-temperature (300 K) Hall effect measurements in the van der Pauw geometry yielded a Hall mobility of 560 cm²/V-s for the low-twinned sample and 566 cm²/V-s for the high-twinned sample, and similar carrier densities between 5 × 10¹⁸/cm³ and 1 × 10¹⁹/cm³. Prior studies focus on improving the Hall mobility and carrier density at the cost of twinning defects. Because twinning in Bi₂Se₃ has been shown to obscure topological photocurrent responses [14] and degrade the performance of intrinsic NLO-based devices, its characterization and analysis is emphasized here.

2.3. Nonlinear Optical Tensor Measurement and Analysis

While macroscopic transport properties provide a baseline indicator of material quality, NLO susceptibility tensors, which are experimentally accessible by SHG, offer a more microscopic probe of the crystalline symmetry of Bi₂Se₃. Twinned domains introduce a mirror symmetry in the crystallographic structure that is evident in the SHG response, making characterization of the rotational anisotropy and magnitude of the SHG signal essential. In large-area, low-twinned Bi₂Se₃ films, these SHG responses converge to those seen in cleaved bulk crystal samples with $C_{3v}$ point group symmetry [15], thereby offering a promising path for realizing microelectronic and photosensors that utilize TIs’ inherent polarization sensitivity.

The pattern measured via SHG is dictated by the form of the third rank tensor ${\sigma }_{ijk}^{SHG}$, where $i,j,k\in \{x,y,z\}$. The non-zero elements of ${\sigma }_{ijk}^{SHG}$ are readily determined by symmetry analysis, and for the $C_{3v}$ surface point group of Bi₂Se_3, the tensor is given by the following expression [26]:

$${\sigma }^{SHG}=\left(\begin{array}{ccc}\left(\begin{array}{c}{\sigma }_{xxx}\\ 0\\ {\sigma }_{xxz}\end{array}\right)& \left(\begin{array}{c}0\\ -{\sigma }_{xxx}\\ 0\end{array}\right)& \left(\begin{array}{c}{\sigma }_{xzx}\\ 0\\ 0\end{array}\right)\\ \left(\begin{array}{c}0\\ -{\sigma }_{xxx}\\ 0\end{array}\right)& \left(\begin{array}{c}- {\sigma }_{xxx}\\ 0\\ {\sigma }_{xxz}\end{array}\right)& \left(\begin{array}{c}0\\ {\sigma }_{xzx}\\ 0\end{array}\right)\\ \left(\begin{array}{c}{\sigma }_{zxx}\\ 0\\ 0\end{array}\right)& \left(\begin{array}{c}0\\ {\sigma }_{zxx}\\ 0\end{array}\right)& \left(\begin{array}{c}0\\ 0\\ {\sigma }_{zzz}\end{array}\right)\end{array} \right).$$

(1)

To model the experiment, we define $C_{3v}$ symmetry to be generated by three-fold rotations about the z-axis and mirror symmetry along the y-axis. As we rotate the thin film by an azimuthal angle $\varphi$, the effective SHG tensor ${\sigma }_{ijk}^{SHG}$ transforms according to the rotation $R\left(\varphi \right)$ matrix:

$${\sigma }_{ijk}^{SHG}\left(\varphi \right)={\sum }_{i^{\prime },j^{\prime },k^{\prime }}{R}_{i{i}^{\prime }}\left(\varphi \right)R_{j{j}^{\prime }}\left(\varphi \right)R_{k{k}^{\prime }}\left(\varphi \right){\sigma }_{i^{\prime }{j}^{\prime }{k}^{\prime }}^{SHG}$$

(2)

We then obtain the following expression:

$${\sigma }^{SHG}\left(\varphi \right)=\left(\begin{array}{ccc}\left(\begin{array}{c}\mathit{cos}3\varphi {\sigma }_{xxx}\\ \mathit{sin}3\varphi {\sigma }_{xxx}\\ {\sigma }_{xxz}\end{array}\right)& \left(\begin{array}{c}\mathit{sin}3\varphi {\sigma }_{xxx}\\ -\mathit{cos}3\varphi {\sigma }_{xxx}\\ 0\end{array}\right)& \left(\begin{array}{c}{\sigma }_{xzx}\\ 0\\ 0\end{array}\right)\\ \left(\begin{array}{c}\mathit{sin}3\varphi {\sigma }_{xxx}\\ -\mathit{cos}3\varphi {\sigma }_{xxx}\\ 0\end{array}\right)& \left(\begin{array}{c}- \mathit{cos}3\varphi {\sigma }_{xxx}\\ -\mathit{sin}3\varphi {\sigma }_{xxx}\\ {\sigma }_{xxz}\end{array}\right)& \left(\begin{array}{c}0\\ {\sigma }_{xzx}\\ 0\end{array}\right)\\ \left(\begin{array}{c}{\sigma }_{zxx}\\ 0\\ 0\end{array}\right)& \left(\begin{array}{c}0\\ {\sigma }_{zxx}\\ 0\end{array}\right)& \left(\begin{array}{c}0\\ 0\\ {\sigma }_{zzz}\end{array}\right)\end{array} \right).$$

(3)

In our experimental configuration, we measure SHG along the same polarization axis as the excitation. Labeling this as the y-axis, $E^{in}=\widehat{y} E_0$, the SHG pattern is then given by the following expression:

$$E_y^{SHG}\left(\varphi \right)={\sum }_{jk}{\sigma }_{yjk}^{SHG}\left(\varphi \right)E_j^{in}{E}_k^{in} .$$

(4)

Experimentally, we measure the intensity of SHG, yielding ${\left|E_y^{SHG}\left(\varphi \right)\right|}^2={\displaystyle \frac{E_0^2}{2}}(1-\mathit{cos}6\varphi )$ for the $C_{3v}$ surface point group of single crystal Bi₂Se₃, which results in a perfectly six-fold symmetric SHG plot.

3. Results

3.1. Atomic Force Microscopy (AFM)

The surface topography of MBE Bi₂Se₃ was measured after ex situ thermal desorption of the semi-crystalline selenium capping layer in a flow of inert, dry nitrogen gas. Figure 1 shows the atomic-force microscope (AFM) image of the Bi₂Se₃ and highlights two important points: (1) the domain structure appears self-aligned and of a single domain with steps running downward in only one direction; (2) the measured step height along the indicated blue line trace is 1 nm and is consistent with the height of one quintuple layer suggesting step-flow epitaxial growth. The terrace width is 30 nm and coupled with the 1 nm step height indicates the Bi₂Se₃ has adopted the same 2° vicinal offset angle as the InP.

These points are in stark contrast to previously published literature. The most commonly reported observation in prior work is that discrete, randomly misoriented triangular islands of Bi₂Se₃ form during the initial phase of deposition [27,28]. For the case of the triangular island deposition, as growth proceeds, twin defects form at the coalescing boundaries of neighboring islands and are permanently locked into the film’s microstructure. For the new work reported here, growth initiates at the step edges of the vicinal substrate and epitaxial growth proceeds outward from the step edge with no island coalescence. As this step-flow epitaxial growth mode proceeds, this yields essentially twin-free, single-domain Bi₂Se₃ with atomically flat terraces, as shown in Figure 1.

3.2. X-Ray Diffraction (XRD)

More quantitative evidence on residual strain is provided by the crystallography and domain structure obtained from X-ray diffraction measurements. A high-resolution symmetric scan showing the (006) reflection in Figure 2 exhibits a slight shift of Bragg angle of +0.13° relative to the bulk value, which informs us that the c-axis lattice constant of Bi₂Se₃ has slightly decreased from the unstrained value (2.88 nm) to 2.86 nm, which is consistent with the small ~0.6% residual tensile strain. We assume that this obtained strain evolves from the mismatch in thermal expansion coefficients upon cooling after epitaxial growth. Well-defined Laue fringes emerge on the Gaussian tail bounds of the (006) peak and indicate near atomic flatness of both the epitaxial interface and the terraces of the sample surface observed in AFM. Figure 3 captures the (0015) and (0018) reflections from the Bi₂Se₃ reflections near the (222) reflection from InP, which confirms the Bi₂Se₃/InP epitaxial alignment and also shows prominent Laue fringing.

The spacing of the Laue fringing on the sides of the (006), (0015) and (0018) reflections, as indicated by the arrows in Figure 3 for example, can be used to determine that the layer thickness is 11 nm. A separate measurement of the thickness was made by X-ray reflectivity measurement and yielded a thickness value of 9 nm, averaging a final thickness of 10 nm.

By capturing X-ray pole scan data, including an azimuthal rotation scan at a constant Bragg angle for the (015) reflection, one can determine a quantitative index for the relative proportion of twin domains. The results from the pole scan from the 10 nm film grown on vicinal InP (111)B + 2° are presented as the top (red) data in Figure 4.

There are two sets of peaks each spaced by 120° with one set corresponding to one domain orientation with a relatively high intensity, and a second set corresponding to a twinned domain orientation rotated with respect to the first by 180°. An index to quantify the relative proportion of twins can be obtained by taking the ratio of peak heights of these two sets of peaks. For the pole scan in Figure 4, one peak from the higher intensity (at 300°) and one from the lower intensity (at 240°) are indicated.

Taking the peak height of each and subtracting the constant background gives a peak height ratio of about 100:1. This gives a relative index that suggests the population of the dominant twin orientation is one hundred times more prevalent than the minor twin orientation. Figure 4 shows a similar pole scan from a different sample (lower, blue) that was epitaxially grown on non-vicinal InP (111)B on-axis to within ±0.5°. For the latter case, a large twin density is expected. For the lower (blue) pole-scan, we readily identify two sets of peaks spaced by 120°. In this case, the peak height ratio is different: a smaller ratio of only about 3 is obtained. This informs that the on-axis substrate yields a textured film with almost equal proportions of 180° twin domains.

The combination of AFM and X-ray diffraction measurements provides evidence for a new highly controllable and reproducible pathway to obtain essentially single-domain, single-crystal heteroepitaxial films of Bi₂Se₃ on InP. The surfaces have an RMS roughness of 1 nm (one quintuple layer height) and have sufficient atomic flatness to display robust Laue fringes in X-ray measurements.

3.3. Second Harmonic Generation (SHG)

To validate these structural findings against the material’s NLO properties, SHG measurements were performed on both low- and high-twinned Bi₂Se₃ samples (shown in Figure 5). The experiment was performed in a normal incidence geometry using a Ti:Sapphire laser (800 nm, <100 fs pulse width, 80 MHz repetition rate, 26 mW average power). Crucially, the laser spot size (20 μm) is orders of magnitude larger than the usually observed domain size for commonly grown thin-film Bi₂Se₃, ensuring that the SHG signal represents an ensemble average of the film’s symmetry. The incident and collected light were co-polarized (vertical linear polarization), and the SHG signal near 400 nm was spectrally filtered and detected with an electron multiplying charged coupled device (EMCCD) camera (20 s integration time).

As derived from the $C_{3v}$ tensor elements (Section 2.3), the SHG intensity for a single-crystal Bi₂Se₃ surface in a co-polarized normal-incidence geometry should exhibit a strictly sin²(3ϕ) dependence. Figure 5a shows the integrated SHG intensity as a function of the azimuthal angle ϕ for the low-twinned sample. The data show a clear sin²(3ϕ) six-fold symmetry, which is expected from a domain-free, single-crystal Bi₂Se₃. However, some two-fold dependence still emerges in the SHG as shown in Figure 5a; the lobes around 160° and 340° are smaller. Figure 5b plots the integrated SHG intensity in the high-twinned sample, where additional contributions from mirrored domains add additional symmetries. This twinning is expected to obscure the $C_{3v}$ SHG response, resulting in an overall lower SHG intensity that has approximately 1/3 the peak signal intensity and 1/24 integrated area over all azimuthal directions, relative to the low-twinned sample, as well as a weaker ϕ dependence. Interestingly, we see that the SHG in the high-twinned sample has a pronounced two-fold symmetry with additional six-fold lobes. These deviations from a purely six-fold symmetric SHG pattern in ϕ require additional analysis of the underlying NLO contributions.

4. Discussion

The efficacy of utilizing small-angle vicinal InP (111)B to produce bulk-quality, single-domain Bi₂Se₃ via MBE is conclusively demonstrated by the convergence of our structural and optical metrics. The X-ray pole scans (Figure 4) reveal a twin ratio of 100:1, representing a strong preferential domain. This is corroborated by SHG measurements, which show a well-defined six-fold symmetry and stronger intensity (Figure 5a) in the low-twin sample. This is indicative of an epitaxial material with vanishingly small twin domain structure and reduced antiphase disorder comparable with essentially single-domain, bulk single-crystal Bi₂Se₃ h

Both the vicinal and the non-vicinal InP substrates have a nominal offcut. The vicinal substrates have a 2° offcut that results in narrower terraces and more step edges to seed preferential domains, resulting in a low twinning density. For the non-vicinal, the offcut is within ±0.5° of the zone axis, resulting in steps that have much wider terraces, and fewer step edges to seed preferential domains, allowing more random domains to form. These step edges can act as a partial polarizer of the emitted SHG field.

Single-crystal Bi₂Se₃ is expected to produce a perfectly six-fold symmetric SHG plot. In Figure 5a, we observe a slight deviation from that behavior seen as a directional bias, where four lobes are larger than the other two. This most likely originates from the step edges that partially polarize the SHG field. In this case, we need to apply partial polarization to our SHG output by applying the operator $M={\lambda }_1 I+{\lambda }_2 M_P(\varphi +\Delta )$, where ${\lambda }_i$ represents the constants, $I$ is the identity operator and $M_p(\varphi +\Delta )$ is the polarization tensor at angle $\varphi$ with offset $\Delta$:

$$M_p= \left(\begin{array}{ccc}{\mathit{cos}}^2(\varphi +\Delta )& \mathit{cos}(\varphi +\Delta )\mathit{sin}(\varphi +\Delta )& 0\\ \mathit{cos}(\varphi +\Delta )\mathit{sin}(\varphi +\Delta )& {\mathit{sin}}^2(\varphi +\Delta )& 0\\ 0& 0& 1\end{array}\right).$$

(5)

We assume the step edges observed in Figure 1 define the permitted polarization angle and set $\Delta ={\displaystyle \frac{\pi }{2}}$ to align the steps with the edges of the Bi₂Se₃ triangular domains. Then we plot ${\left|M_p\cdot E_y^{SHG}\left(\varphi \right)\right|}^2$ in Figure 5c and find strong agreement between experiment and theory for the low twinned sample.

The azimuthal dependence of the SHG in the high-twinned samples in Figure 5b presents a significantly more pronounced two-fold behavior. To account for this variation, we must take into consideration the opposite twinning domain in the material. The SHG tensor for this contribution, ${\sigma }_{twin}^{SHG}\left(\varphi \right)$, is computed by subjecting the original ${\sigma }^{SHG}\left(\varphi \right)$ to a $\pi$ rotation using Equation (2). We can then add the two SHG tensors in unequal contributions to obtain a total SHG tensor [10]:

$${\sigma }_{Tot}^{SHG}\left(\varphi \right)={\sigma }^{SHG}\left(\varphi \right)+\gamma {\sigma }_{twin}^{SHG}\left(\varphi \right) ,$$

(6)

where the second twinning domain has a proportion $\gamma$ to the first. We can now compute ${\left|M_p\cdot E_{twin,y}^{SHG}\left(\varphi \right)\right|}^2$ which we plot in Figure 5d, emulating the observed SHG dependence seen in Figure 5b for the high-twinned sample. In the high-twinned case, we are unsure along which crystal axis the step edges fall; however, for an arbitrary choice of $\Delta$ in $M_p$, we find that there will be two large, biased lobes mimicking the data.

While not studied here, localized planar stacking fault-type defects may also exist pointing out-of-plane within individual domains. This disorder is also equivalent to twinning and probably has a very small impact on the X-ray pole-scan measurement, but more simply acts to induce broadening of the major Bragg peaks observed in symmetric 2-theta scans similar to Figure 2 and Figure 3. Because broadening is caused by many factors including thickness and other defects, understanding the effect of only one factor causing broadening would seem intractable.

Such stacking-fault twin disorder has been predicted to have small, but impactful perturbations on the dispersion of the topological surface state [29]. Separate magnetotransport measurements show that carrier mobility in this family of materials decreases with increasing temperature that is characteristic of phonon dominated, or lattice scattering, and unlikely to impact electrical behavior [30].

The novelty and significance of this work is that enhanced photonic response from MBE thin-film Bi₂Se₃ can be obtained by the improved new epitaxial approach described here. In this simple approach, neither substrate roughness nor complicated hetero-structure buffer layers are employed or needed to yield repeatable Bi₂Se₃ thin-film heterostructures whose photonic quality and performance converge to those of idealized perfect bulk-single crystal material.

5. Conclusions

The nonlinear optical response of thin film topological insulator materials can be adjusted to achieve the idealized bulk response by reducing both antiphase disorder and twin domains. We demonstrate the efficacy of the new approach described here by obtaining a vanishingly small peak-height ratio from X-ray pole-scans showing negligible twinning, new aligned step-flow, terraced growth using atomic force microscopy, and strong SHG signals at the inversion-breaking surface. We corroborate this result with group theoretic analysis of nonlinear optical responses, which shows a dominant six-fold symmetry response in low-twinned samples. The effect of crystalline symmetries is diminished in the high twinned samples, with reduced SHG intensity and a strong preferential SHG axis attributed to the vicinal step structures caused by the miscut of the substrate. The results of this work show that this approach can yield near-perfect single-crystal heteroepitaxial Bi₂Se₃ whose nonlinear optical responses reflect those of bulk single crystals.

Understanding the impact of these NLO responses on material properties that correlate with photonic behaviors and further refining material quality is a crucial step for realizing chip-scale integrated sensors with strong nonlinear and polarization-dependent responses. A complete picture can be obtained through correlations with, for example, direct photocurrent measurements via time-domain THz spectroscopy on bare samples [9,10] or fabricated devices under polarization-dependent illumination.