Optical Bistability in a Quantum Dot–Metallic Nanoshell–Cell Membrane Hybrid System: Applications for High-Performance Biosensing

Abstract

We investigate optical bistability (OB) in a hybrid system comprising a semiconductor quantum dot (SQD), a metallic nanoshell (MNS), and a cell membrane within the framework of the multipole approximation. Bistability phase diagrams plotted in the system’s parameter subspaces demonstrate that, in the weak exciton–phonon coupling regime, dynamic switching of bistable states among no-channel, single-channel, and dual-channel configurations can be achieved via precise modulation of the MNS’s dielectric shell thickness. Especially, a critical sensing threshold is identified: the absorption peak disappears and a bistable effect emerges when only 1.82% of normal cells undergo malignant transformation. Furthermore, the bistable region exhibits a gradual broadening trend with an increasing proportion of cancerous cells, yielding a quantitative and ultra-sensitive readout that underpins a highly accurate strategy for early cancer diagnosis. These findings not only deepen our fundamental understanding of bistability regulation in hybrid quantum-plasmonic systems interfaced with biological materials but also offer valuable insights for the development of next-generation optical switches and biomedical sensing platforms.

1. Introduction

With the rapid development of nanophotonics, researching functional optical phenomena has received unprecedented focus [1,2,3]. Optical bistability (OB), a compelling optical phenomenon with substantial fundamental research significance, has been demonstrated to holds great potential for practical applications in fields including optical switching and optical memory devices. A substantial amount of theoretical and experimental work has been conducted on OB in various systems including graphene-based structures [4,5], optomechanical systems [6,7], and waveguide [8,9]. Among these, a hybrid system composed of semiconductor quantum dots (SQDs) and metal nanoparticles (MNPs) has garnered growing interest due to its distinctive optical bistable properties. Artuso and Bryant demonstrated that further increasing the sizes of both the SQD and MNP leads to strengthened coupling, thereby revealing a bistable operational regime [10]. Malyshev et al. showed that OB becomes discernible when the optical hysteresis of Rayleigh scattering is measured [11]. Li et al. found that the bistable nonlinear absorption response is observable and the corresponding bistable region can be precisely modulated by tailoring the size of the MNP, the interparticle distance, and the intensity of control field [12]. Nugroho et al. achieved both the construction of 2D bistability phase diagrams within the system’s parameter space and the visualization of the switching process between the lower and upper stable branches [13]. Asadpour and Rahimpour Soleimani demonstrated that a 2D array of metal-coated dielectric nanospheres embedded in a unidirectional ring cavity exhibits both OB and optical multistability (OM) behaviors [14]. Paspalakis et al. showed that the optical rectification of this hybrid system is bivalued, with a bistable behavior manifesting alongside it [15]. Mohammadzadeh and Miri established that the fluorescence spectrum and the intensity–intensity correlation function can be substantially tailored in the regions exhibiting bistability [16]. Zhao et al. investigated OB enabled by multipole polarizations in an SQD-MNP hybrid system, and their findings reveal that multipole effects lead to significant broadening of the pump intensity-dependent bistable region under strong exciton–plasmon coupling [17]. They further demonstrated that, for a given intermediate scaled pumping intensity in an SQD–metallic nanoshell (MNS) hybrid system, multipole polarizations not only narrow the bistable zone but also enlarge the corresponding thresholds [18].

In our previous work [18], we systematically explored the dependence of OB on the dielectric shell thickness of the MNS. He and Zhu proposed a highly sensitive method for cancer cell detection based on a peptide QD–cell membrane coupled system, leveraging the unique optical responses of hybrid biomaterial structures [19]. Beyond these advances, SQD/MNP hybrid nanosystems have emerged as promising candidates for biosensing applications [20,21,22]. For example, Zhao et al. demonstrated that rigid DNA spacers can bridge Ag NPs and CdS QDs, thereby enabling the targeted stimulation of exciton–plasmon interactions in photoelectrochemical systems, offering a viable mechanism for novel DNA sensing protocols [20]. Popov et al. presented a comprehensive overview of signal amplification strategies for electrochemical immunosensors, in which MNPs and QDs act as labeling agents, with a specific focus on recent advances in the ultrasensitive detection of biomarkers [21]. However, to the best of our knowledge, no studies have yet focused on OB behaviors in an SQD-MNS–cell membrane. Investigating OB in this system is not only scientifically meaningful but also practically valuable. Additionally, the bistable properties in this system may enable the development of highly sensitive optical switches tailored for biomedical applications. Specifically, the synergistic coupling between the SQD, MNS, and cell membrane is expected to endow the system with enhanced optical responsiveness, which cannot be achieved by binary hybrid system alone. In the present paper, we therefore investigate the influences of key parameters (including pump intensity, interparticle distance, exciton–phonon coupling strength, and dielectric shell thickness) on the bistable effect in an SQD-MNS–cell membrane hybrid system. Particularly, we discuss the feasibility of leveraging the tunable OB behaviors of this system for cancer cell detection, aiming to provide a new theoretical basis and technical reference for the design of next-generation biosensors.

2. Theoretical Model and Method

A schematic of the SQD-MNS–cell membrane hybrid system under investigation is provided (Figure 1) with the system being simultaneously irradiated by a strong pump field (E_pu, ω_pu) and a weak probe field (E_pr, ω_pr). The SQD is modeled as a two-level system, encompassing a ground state |g> and a single exciton state |e>. The exciton transition frequency is represented as ω_ex. The MNS features an outer dielectric shell with radius r₁ and a metallic inner core with radius r₀, where the shell thickness is defined as Δr = r₁ − r₀. Specifically, adjusting the dielectric shell thickness enables precise control over the dielectric environment surrounding the metallic core, a parameter that governs the exciton–plasmon coupling efficiency in the SQD-MNS–cell membrane hybrid system. Additionally, the distance between the SQD and the MNS is denoted as d. When radiation pressure from the strong pump field acts on the cell membrane, the membrane exhibits vibrational motion. The fundamental vibrational frequency associated with the cell membrane is given by ω_c = 1/2 × {(ϕ/η) × [(1/L₁)² + (1/L₂)²]}^1/2 [19], wherein ϕ and η stand for the average surface tension and average surface density of the cell membrane, respectively. L₁ and L₂ refer, respectively, to the length and width of the region housing the SQD. Furthermore, ε₀, ε₁, ε_m, ε_s, and ε₂ represent the dielectric constants of vacuum, the background medium, the metallic core, the dielectric shell, and the SQD, respectively.

Within the rotating frame, the total Hamiltonian corresponding to the SQD/MNS/cell membrane hybrid system is expressed as follows:

$$H=\hslash {\mathsf{\Delta }}_{pu}{\sigma }_z+\hslash {\omega }_c{a}^{+}a+\hslash \lambda {\sigma }_{z}Q-\mu \left({\tilde{E}}_{ex}{\sigma }_{eg}+{\tilde{E}}_{ex}^{*}{\sigma }_{ge}\right),$$

(1)

Here, the Hamiltonian of the exciton is denoted as ħΔ_puσ_z, whereas the Hamiltonian descripting the phonon mode of the cell membrane is given by ħω_ca⁺a. The interaction Hamiltonian between phonons and the exciton is represented by the term ħλσ_zQ, and the interaction Hamiltonian governing the coupling between the exciton and the applied electric field is specified as −μ(E_exσ_eg + E_ex*σ_ge), where μ represents the electric dipole moment of the SQD, and E_ex denotes the amplitude of the electric field acting on the exciton. Additionally, the phonon quadrature operator is defined as Q = a⁺ + a, in which a⁺ and a correspond to the phonon creation and annihilation operators, respectively. The frequency detuning between the exciton and the pump field is given by Δ_pu = ω_ex − ω_pu. The transition operator σ_ij = |i><j| (where i, j = g, e) describes the transition process from the initial state |i> to the final state |j>. Furthermore, the coupling strength between the exciton in the SQD and the phonon modes of the cell membrane is quantified as λ = λ₀ω_c, where λ₀ is a dimensionless coupling coefficient, and σ_z = (σ_ee − σ_gg)/2 refers to the half population inversion of the exciton. The total field experienced by the exciton (i.e., E_ex) is denoted as [23]

$$\begin{gathered} { E}_{ex}={\displaystyle \frac{1}{{\epsilon }_{eff}}}\left[1+{\displaystyle \frac{2r_1^3{\gamma }_1\left(\omega \right)}{d^3}}\right]\left(E_{pu}+E_{pr}{e}^{-i{\delta }_{\mathrm{pr}}t}\right) \\ +{\displaystyle \frac{1}{{8\pi {ϵ}_0{ϵ}_1ϵ}_{eff}}}\times \sum _{n=1}^{\infty }n\left(n+1\right){\left(n+1\right)}^2{\displaystyle \frac{r_1^{2n+1}{\gamma }_n\left(\omega \right)}{d^{2n+4}}}{P}_{SQD} \\ =\mathsf{\Lambda }\left(E_{pu}+E_{pr}{e}^{-i{\delta }_{\mathrm{pr}}t}\right)+\mathsf{\Theta }{P}_{SQD} \end{gathered}$$

(2)

Here, Λ is defined as [1 + 2γ₁(ω)r₁³/d³]/ε_eff with Λ_I = Im[Λ] and Λ_R = Re[Λ], while Θ takes the form [Σ_nn(n + 1)(n + 1)²γ_n(ω)r₁²ⁿ⁺¹]/[8πε₀ε₁ε_effd²ⁿ⁺⁴]. In these expressions, ε_eff = (ε₂ + 2ε₁)/(3ε₁) [24], P_SQD = μσ_ge, and δ_pr = ω_pr − ω_pu represents the probe-pump detuning. Additionally, the nth order (n = 1, 2, 3…) polarization factor γ_n(ω) reads as [23,25]

$${\gamma }_n\left(\omega \right)={\displaystyle \frac{n\left[{\epsilon }_{mc}\left(\omega \right)-{\epsilon }_b\right]}{n{\epsilon }_{mc}\left(\omega \right)-{\left(n+1\right)\epsilon }_b}},$$

(3)

$${\epsilon }_{mc}\left(\omega \right)={\epsilon }_m\times {\displaystyle \frac{\left[n{\epsilon }_m+{\left(n+1\right)\epsilon }_s\right]r_1^{2n+1}+n\left({\epsilon }_m-{\epsilon }_s\right)r_0^{2n+1}}{\left[n{\epsilon }_m+{\left(n+1\right)\epsilon }_s\right]r_1^{2n+1}-\left(n+1\right)\left({\epsilon }_m-{\epsilon }_s\right)r_0^{2n+1}}},$$

(4)

Using the Heisenberg equation of motion with the commutation relations including [a, a⁺] = 1, [σ_ge, σ_z] = σ_ge, [σ_eg, σ_z] = −σ_ge, and [σ_eg, σ_ge] = 2σ_z [26], the temporal evolutions of operators σ_z, σ_ge, and Q can be expressed as

$$\dot{{\sigma }_z}=-{\mathsf{\Gamma }}_1\left({\sigma }_z+{\displaystyle \raisebox{1ex}{$1$}\!\left/ \!\raisebox{-1ex}{$2$}\right.}\right)+i{\mathsf{\Omega }}_{pu}\left({\mathsf{\Lambda }{\sigma }_{ge}^{*}-{\mathsf{\Lambda }}^{*}\sigma }_{ge}\right)+{\displaystyle \frac{i\mu }{\hslash }}\left({\mathsf{\Lambda }{E}_{pr}{e}^{-i{\delta }_{\mathrm{pr}}t}{\sigma }_{ge}^{*}-{\mathsf{\Lambda }}^{*}{E}_{pr}^{*}{e}^{i{\delta }_{\mathrm{pr}}t}\sigma }_{ge}\right)+i\left(\mathsf{\Pi }-{\mathsf{\Pi }}^{*}\right){\sigma }_{ge}^{*}{\sigma }_{ge},$$

(5)

$${\dot{\sigma }}_{ge}=-{[\mathsf{\Gamma }}_2+i\left({\mathsf{\Delta }}_{pu}+\lambda Q\right)]{\sigma }_{ge}-2i{\mathsf{\Omega }}_{pu}\mathsf{\Lambda }{\sigma }_z-{\displaystyle \frac{2i\mu }{\hslash }}\mathsf{\Lambda }{E}_{pr}{e}^{-i{\delta }_{\mathrm{pr}}t}{\sigma }_z-2i\mathsf{\Pi }{\sigma }_z{\sigma }_{ge}+{\widehat{f}}_1,$$

(6)

$$\ddot{Q}+{\gamma }_c\dot{Q}+{\omega }_c^{2}Q=-2\lambda {\omega }_c{\sigma }_z+{\widehat{f}}_2,$$

(7)

Here, Π = μ²Θ/ħ is defined as the feedback parameter with Π_I = Im[Π] and Π_R = Re[Π], Γ₁ (Γ₂) denotes to the exciton relaxation (dephasing) rate, γ_c refers to the decay rate of the cell membrane vibration, and Ω_pu = μE_pu/ħ represents the Rabi frequency of the pump field. I_pu = Ω_pu² stands for the scaled pump intensity, where c is the speed of light in a vacuum. The Langevin noise operator, denoted as ${\widehat{f}}_1$, satisfies the relations $〈{\widehat{f}}_1〉=0$ and $〈{\widehat{f}}_1\left(t_1\right){\widehat{f}}_1^{+}\left(t_2\right)〉=\delta \left(t_1-t_2\right)$. In addition, ${\widehat{f}}_2$ corresponds to a zero-mean Brownian random force, which adheres to the correlation relation $〈{\widehat{f}}_2^{+}\left(t_1\right){\widehat{f}}_2\left(t_2\right)〉={\displaystyle \frac{{\gamma }_c}{2\pi {\omega }_c}}\int \omega \times e^{-i\omega \left(t_1-t_2\right)}\left[1+\coth \left({\displaystyle \frac{\hslash \omega }{2k_{B}T}}\right)\right]d\omega$. To solve Equations (5)–(7), the following ansatz is adopted [26]: σ_z = σ_z⁽⁰⁾ + σ_z⁽⁻¹⁾e^iδprt + σ_z⁽¹⁾e^−iδprt, σ_ge = σ_ge⁽⁰⁾ + σ_ge⁽⁻¹⁾e^iδprt + σ_ge⁽¹⁾e^−iδprt, and Q = Q₀ + Q₋₁e^iδprt + Q₁e^−iδprt. When these expressions are substituted into Equations (5)–(7),the formula for the linear susceptibility is derived

$${\chi }_{eff}^{\left(1\right)}={\displaystyle \frac{\mu {\sigma }_{ge}^{\left(1\right)}}{{\epsilon }_0{E}_{pr}}}=C_8\times \left({\displaystyle \frac{C_3{C}_4{C}_7{w}_0-2C_1{C}_3{C}_7\mu {\sigma }_{ge}^{\left(0\right)\mathrm{*}}}{C_1{C}_5{C}_{7-}{C}_2{C}_4{C}_7+C_1{C}_6}}\right)+C_9{w}_0$$

(8)

where w₀ = 2σ_z⁽⁰⁾, σ_ge⁽⁰⁾ = (Λ_I − iΛ_R)Ω_puw₀/(Γ₂ − Π_Iw₀ + iΔ_pu − iλλ₀w₀ + iΠ_Rw₀), C₁ = (Γ₂ − Π_Iw₀) − i(δ_pr − Δ_pu − Π_Rw₀ + λλ₀w₀), C₂ = (Λ_I − iΛ_R)μΩ_pu + (Π_I − iΠ_R)μσ_ge⁽⁰⁾ − iλ²ζμσ_ge⁽⁰⁾, C₃ = μ²(Λ_I − iΛ_R)/ħ, C₄ = −(4Π_Iμσ_ge⁽⁰⁾* + 2μΩ_puΛ_I) − 2iμΩ_puΛ_R, C₅ = (−iδ_pr + Γ₁)μ², C₆ = 2μΩ_puΛ_I + 4Π_Iμσ_ge⁽⁰⁾ − 2iμΩ_puΛ_R, C₇ = [(Γ₂ − Π_Iw₀) − i(δ_pr + Δ_pu + Π_Rw₀ − λλ₀w₀]/[(μΩ_puΛ_I + μΠ_Iσ_ge⁽⁰⁾*) + i(Λ_RμΩ_pu + μΠ_Rσ_ge⁽⁰⁾* + λ²ζμσ_ge⁽⁰⁾*)], C₈ = 1/ε₀ × [iλ²ζμσ_ge⁽⁰⁾ + (−Λ_I + iΛ_R)μΩ_pu + (−Π_I + iΠ_R)μσ_ge⁽⁰⁾]/[(−Γ₂ + Π_Iw₀) + i(δ_pr − Δ_pu − Π_Rw₀ + λλ₀w₀)], C₉ = μ²/(ε₀ħ) × (−Λ_I + iΛ_R)/[(−Γ₂ + Π_Iw₀) + i(δ_pr − Δ_pu − Π_Rw₀ + λλ₀w₀)], and ζ = −ω_c/(ω_c² − iδ_prγ_c − δ_pr²). Imχ_eff⁽¹⁾ denotes the linear absorption, and Reχ_eff⁽¹⁾ corresponds to the dispersion.

The exciton-population inversion (w₀) can be calculated using the equation provided below

$${\mathsf{\Gamma }}_1\left(w_0+1\right)\left[{\left({\mathsf{\Gamma }}_2-{\mathsf{\Pi }}_I{w}_0\right)}^2+{\left({\mathsf{\Delta }}_{pu}-{\lambda }_0\lambda w_0+{{\mathsf{\Pi }}_{R}w}_0\right)}^2\right]+4{\left|\mathsf{\Lambda }\right|}^2{\mathsf{\Omega }}_{pu}^2{\mathsf{\Gamma }}_2{w}_0=0.$$

(9)

3. Results and Discussion

Subsequently, we investigate a practical hybrid system consisting of an SQD, an MNS, and a cell membrane. All parameter values employed in this study are experimentally achievable. Regarding the MNS, it comprises a Au core and a SiO₂ shell with the dielectric constant of the SiO₂ shell set to ε_s = 2.16 [27,28], the core radius to r₀ = 12 nm. The exciton resonant frequency is chosen to 2.36 eV, which is close to the broad plasmon frequency of Au. Here, ε_m denotes the dielectric constant of the Au core [29]. Regarding the SQD, its parameters are configured as follows: ε₂ = 6, μ = 1.0 × 10⁻²⁸ C·m, Γ₁ = 1.25 ns⁻¹, and Γ₂ = 3.33 ns⁻¹ [30]. The phospholipid bilayer-based cell membrane is distinguished by two key vibrational parameters: a fundamental mode frequency ω_c = 4.90 MHz and a decay rate γ_c = 0.04 MHz [31]. The total system is located in an air environment with a background dielectric constant of ε₁ = 1. Additionally, the subsequent analysis focuses on the scenario where the pump and probe fields have the same frequency (i.e., ω_pu = ω_pr).

Figure 2 illustrates the dependence of linear absorption on pump intensity for three distinct nanosystems, which are SQD, SQD-MNS, and SQD-MNS–cell membrane, under both dipole and multipole approximations. We first discuss the scenario of dipole approximation. As presented in Figure 2a, under the dipole approximation (i.e., N = 1), the absorption of the SQD system weakens monotonically as the pump intensity I_pu increases. For the SQD-MNS system, a sharp peak is observed at I_pu = 2.2 GHz². This peak may be attributed to the localized surface plasmon resonance (LSPR) of the MNS. This resonance enhances light–matter interaction at a specific pump intensity to induce a pronounced absorption maximum, and its emergence likely stems from the synergistic interaction between the excitonic transitions of the SQD and the LSPR of the MNS. The SQD-MNS–cell membrane system, however, exhibits distinctly different behavior: an interesting bistable effect occurs, suggesting the system can exist in two different absorption states when exposed to the same pump intensity. Next, we analyze the linear absorption behavior of these three systems under the multipole approximation (corresponding to N = 12) [Figure 2b], which accounts for higher-order multipole interactions and provides a more accurate description of light–matter interaction in complex nanostructures. The selection of a multipole order N = 12 in the present work is based on our previously published findings [18], where we rigorously demonstrated that optical spectral lines achieve complete convergence and overlap at N = 7. Therefore, we have opted for N = 12 to ensure the fulfillment of the convergence condition. Unlike the dipole approximation case, the SQD-MNS system displays an “H-shaped” bistable region due to the multipole effect, and its lower and upper bistable branches nearly overlap [Inset in Figure 2b]. This result may stem from the spatially inhomogeneous near-field distribution around the SQD, which is induced by these high-order multipoles. Such a distribution modulates the exciton transition probability, consequently narrowing the energy gap between the two bistable states. In particular, the constructive interference between the dipole and high-order multipole fields serves to reduce the hysteresis loop width of the OB curve, driving the lower and upper branches of the bistable response toward near-overlap. For the SQD-MNS–cell membrane system, bistable behavior is observable across an extremely wide region, which suggests that OB can be easily achieved with an appropriate pump intensity. Additionally, Compared with the N = 1 case, the bistable region for N = 12 shifts toward higher I_pu values. This shift implies that higher-order multipole interactions require a larger pump intensity to trigger OB, and this is an important consideration for optimizing the operating conditions of OB-based devices.

Figure 3a illustrates the variation in the linear absorption curve with pump intensity I_pu by adjusting the spacing between the SQD and the MNS. As is all known, the exciton–plasmon interaction is strongly dependent on the interparticle spacing [32,33,34]: a gradual increase in this spacing weakens the interaction, while a decrease strengthens it. For a spacing d = 26 nm, the bistable effect is initiated when the pump intensity I_pu rises to a critical value of 2.148 × 10³ GHz²; this threshold denotes the point where the bistable response of the coupled system surpasses the dominance of linear absorption. When d increases to 30 nm, the critical I_pu required to generate OB increases to 2.761 × 10³ GHz². This observation confirms that a larger d results in weaker exciton–plasmon coupling, which in turn requires a stronger pump intensity to induce OB. This also reveals a core relationship that underpins the tunability of bistable behavior in the SQD-MNS–cell membrane system. To further quantify the dependence of the bistable threshold on the SQD-MNS distance and provide a comprehensive visualization of parameter-space behavior, in Figure 3b we plot a bistability phase diagram within the system’s parameter subspace [I_puc; d; N = 12; Δr = 2 nm; λ = 2 GHz]. For d = 20 nm, OB appears only when the lower bistable threshold I_puc0 = 6.162 × 10³ GHz². However, as d increases gradually from 20 nm, I_puc0 undergoes an abrupt decline. When d reaches 22.09 nm, I_puc0 drops to merely 0.1 GHz², representing the minimum threshold for OB in this system. Notably, at this specific d value, a narrow bistable region I_puc ∈ [0.1, 205.2] GHz² is formed, indicating that the system can only maintain bistable states within a limited range of pump intensities. When d exceeds 22.09 nm, the lower bistable threshold further increases as d continues to rise, as the system demands higher pump energy to sustain OB.

Inspecting Equation (9), it is evident that the bistable effect is strongly correlated with the dielectric shell thickness. Figure 4a illustrates how the linear absorption curve varies with the interparticle distance d for different dielectric shell thicknesses Δr. For a specific shell thickness (i.e., Δr = 2 nm), a distinct bistable effect emerges in the absorption spectrum, characterized by two bistable absorption states that respond to tuning of d. However, as Δr is further increased to 4 nm, the bistable region vanishes completely and is replaced by a sharp peak precisely centered at d = 24.72 nm. When Δr is increased to 6 nm, the maximum intensity of the absorption peak is significantly suppressed (by approximately 50% relative to that at Δr = 4 nm), and the peak profile becomes broadened, reflecting weakened exciton–plasmon interaction and increased energy dissipation in the thicker dielectric layer. To further visualize the influence of shell thickness on OB, we plot a bistability phase diagram in the parameter subspace [d_c; Δr; N = 12; λ = 2 GHz; I_pu = 100 GHz²]. A distinctly different scenario can be observed. Herein, a narrow bistable region appears, indicating that our proposed system can function as an optical bistable switch. To further elucidate the underlying physical properties, we divide this region into three subspaces. Specifically, in region ① (corresponding to 0 nm ≤ Δr ≤ 2.13 nm), the bistable switch exhibits only a single channel with one bistable region. In contrast, a dual-channel bistable state emerges in region ②, which is characterized by two independent bistable regions. When Δr exceeds 2.86 nm, the system enters region ③, where the bistable effect is smeared out, thereby indicating that the bistable switch is turned off. Driven by the synergistic effect of the MNS shell dielectric confinement and the exciton–phonon coupling, the sequential evolution of the system’s bistable channels with dielectric shell thickness arises from the combined effect of exciton–plasmon and exciton–phonon couplings.

In contrast to our previously published work, the cell membrane assumes a pivotal role in the proposed system. Consequently, clarifying the influence of the coupling between the cell membrane and the exciton on OB of the system holds considerable significance. Figure 5a illustrates the variation curve of linear absorption with the exciton–phonon coupling strength λ for different shell thicknesses Δr. For Δr = 2 nm, the absorption curve exhibits two isolated bistable regions; however, as Δr increases to 4 nm, these two regions shift toward larger λ values. Especially, near the left critical point of the left bistable region, the absorption intensity is greatly suppressed compared to that observed at Δr = 2 nm. Unexpectedly, when Δr is further increased to 6 nm, these two bistable regions disappear entirely. It should be noted that the bistable effect disappears when the exciton–phonon coupling is absent (i.e., λ = 0 GHz) under fixed conditions. However, the bistable effect emerges once the cell membrane is incorporated into the system. In the SQD-MNS system [18], the absence of interfacial charge transfer dynamics removes the critical support needed to sustain exciton–phonon coupling and stabilize the bistable state. In contrast, integrating the cell membrane into the system establishes such interfacial dynamics, thereby restoring both the coupling and the bistable response. These observations demonstrate that the cell membrane is an indispensable component for the precise modulation of OB. To clarify the effects of shell thickness and exciton–phonon coupling strength on the critical bistable conditions, we construct a bistability phase diagram in the parameter subspace [λ_c; Δr; N = 12; d = 28 nm; I_pu = 100 GHz²]. In the absence of the dielectric shell (i.e., Δr = 0 nm), the bistable effect arises. It is not until Δr increases to 1.23 nm that a dual-channel bistable state begins to be observed. Specifically, for a fixed Δr = 1.23 nm, the bistable state can be achieved under the conditions of 0 GHz ≤ λ_c ≤ 0.024 GHz or 0.516 GHz ≤ λ_c ≤ 0.571 GHz. Consistent with the conventional definition in nanophotonic coupling systems, the weak coupling regime corresponds to λ < Γ₁, while the strong coupling regime is demarcated by λ > Γ₁. Our experimental results align well with this standard classification. It should be noted that the bistable state is found to be favored in the weak exciton–phonon coupling regime. As Δr is further increased beyond this threshold, two narrow bistable regions emerge simultaneously. Prior studies [35,36,37] have demonstrated that an SiO₂ shell with a thickness under 8 nm exerts only a negligible influence on the optical properties of Au@SiO₂ core–shell NPs, implying that tuning the shell thickness across the range of 0 to 8 nm induces only a marginal change in the dielectric constant of the SiO₂ layer. Collectively, these results demonstrate that such dual-channel bistable states exist only when the coupling strength between the exciton and the cell membrane reach certain specific values, thereby offering novel insights for switching single-channel and dual-channel bistable states.

By examining Equation (9), it is evident that the vibrational frequency of the cell membrane may play a key role in regulating OB within the proposed system. As is well known, the malignant transformation of normal cells into cancer cells is accompanied by a corresponding change in their membrane vibrational frequency ω_c. Specifically, during this transformation process, the average surface tension ϕ of the cell membrane undergoes alteration, which in turn induces a concomitant change in ω_c that conforms to the relationship: ω_c = (1 − β)ω_c0 + βω_c1 [38]. Herein, ω_c0 and ω_c1 correspond, respectively, to the fundamental mode vibrational frequencies of the cell membrane derived from normal and cancer cells [19], while β denote the ratio of cancer cells to total cells. In our model, β increase as the fraction of normal cells undergoes malignant transformation rises. Specifically, β = 0 corresponds to a fully normal cell membrane, whereas β = 1 represents a completely cancerous one. When normal cells have not yet started to become cancerous, the vibrational frequency of normal cells is 4.90 MHz prior to the initiation of carcinogenesis; however, as these cells undergo partial transformation into cancer cells, this vibration frequency gradually increases. Notably, the bistable effect is absent when ω_c = 4.90 MHz but emerges as ω_c rises to 5.325 MHz. In fact, a larger ω_c induces a broader bistable region. This observation indicates that monitoring the evolution of the bistable region offers a feasible approach to tracking cellular carcinogenesis.

To enhance the visualization of these results, we plot a bistability phase diagram for cancer cell detection in the system’s parameter subspace [d_c; β; N = 12; Δr = 2 nm; λ = 5 GHz; I_pu = 640 GHz²] (Figure 6b). In a strong exciton–phonon coupling regime, the bistable effect can be observed at point C (d_c(C) = 19.62 nm), where β equals only 1.82%, suggesting that OB appears accordingly when 1.82% of normal cells are transformed into cancer cells. Furthermore, as the proportion of cancerous cells increases, the bistable region broadens progressively. Figure 6c illustrates how the critical value β_c changes with SiO₂ shell thickness Δr, where β_c denotes the proportion of cancerous cells corresponding to the exact onset of OB. At Δr = 2 nm, β_c equals 1.82%. When Δr increases to 2.1 nm, β_c rises to nearly 100%, which demonstrates that β_c is highly sensitive to the SiO₂ shell thickness. This sensitivity can be attributed to the strengthened exciton–plasmon coupling induced by the adjustment of the dielectric shell thickness. In contrast, the relationship between β_c and the decay rate of the cell membrane γ_c follows a starkly different pattern (Figure 6d). β_c remains almost constant at 1.82% even as γ_c varies, a behavior stemming from the negligible effect of the cell membrane’s decay rate on OB modulation. Furthermore, as the proportion of cancerous cells increases, the bistable region exhibits a progressively broadening trend. Collectively, these findings establish a highly accurate and practical approach for the early diagnosis of cancer cells. Complementarily, the distinct features of bistable signals enable the reverse deduction of cellular malignant transformation status, obviating the need for both biomarker labeling and broadband spectral scanning.

In contrast to conventional optical bistable devices, our SQD-MNS–cell membrane system offers three key advantages for cancer detection applications: (1) it enables label-free biosensing, where cell membrane biophysical parameters directly modulate bistable properties; (2) it supports low-power operation, as the synergistic effects of exciton–plasmon and exciton–phonon coupling efficiently lower switching thresholds; (3) it provides integrated diagnostic functionality, facilitating the simultaneous qualitative and quantitative analysis of cellular malignant transformation.

Based on our theoretical framework and the latest advances in nanophotonics and biomaterials, four core engineering and materials science barriers must be addressed to advance the practical translation of the proposed SQD-MNS–cell membrane hybrid system. (1) The scalable, reproducible fabrication of such heterostructures with uniform optical properties is hindered by random conjugation and non-specific binding in large-scale processes. These limitations contravene the model’s assumptions of precisely regulated SQD-MNS spatial coupling and SQD characteristics. A feasible solution lies in optimized surface functionalization via site-specific covalent linkers. This approach would not only enhance structural uniformity but also directly tailor the coating composition and surface morphology of the MNS. This precise structural modulation is critical because our findings demonstrate that it dictates the plasmonic response of the hybrid system and in turn its biosensing performance for early cancer diagnosis. (2) Retaining the structural and optical integrity of the hybrid system over extended periods in complex biological microenvironments is a prerequisite for clinical translation. Notably, the stability of the MNS coating, including its composition and morphological robustness, directly governs the persistence of OB which is the core sensing mechanism of the system. Future efforts to engineer durable, biocompatible MNS coatings will therefore be pivotal to bridging advanced coating science with real-world biomedical sensing applications. (3) High-affinity, target cell membrane-specific binding is required to eliminate off-target interference and ensure high signal-to-noise ratios. This goal is achievable through the rational design of surface-conjugated targeting ligands. When integrated with precisely engineered MNS coatings, such ligand-functionalized heterostructures can synergistically optimize both the specificity and sensitivity of the biosensing platform. This integration further reinforces the intrinsic link between coating design, plasmonic behavior, and biosensing efficacy. (4) Downsizing the laboratory-scale optical platforms currently used to quantify absorption peak variations and optical bistability is a key step toward deploying the system for point-of-care and intraoperative sensing applications. Miniaturized devices would need to retain the capability to resolve subtle changes in bistable behavior triggered by MNS coating modulation. This technical challenge underscores the need for cross-disciplinary collaboration between materials scientists specializing in advanced coatings, nanophotonics researchers, and clinical diagnosticians.

Collectively, these challenges and their proposed solutions highlight a central contribution of this work. We have established an explicit, mechanistic link between MNS coating composition, surface morphology-dependent plasmonic responses, and the biosensing performance of quantum-plasmonic–biological hybrid systems. By delineating this connection, our study not only advances the fundamental understanding of OB regulation in such complex systems but also forges a critical bridge between advanced coating science and cutting-edge biomedical sensing.

4. Conclusions

In conclusion, this work analyzes the optical bistability (OB) of a hybrid system comprising a semiconductor quantum dot (SQD), a metallic nanoshell (MNS), and a cell membrane under the multipole approximation. Key results are summarized as follows: First, bistability phase diagrams are plotted in the system’s parameter subspaces, explicitly illustrating the dynamic switching of the bistable state among no-channel, single-channel, and dual-channel configurations. This switching behavior is achievable via precise modulation of the MNS dielectric shell thickness within the weak exciton–phonon coupling regime. Second, we identify a critical sensing threshold for early cancer diagnosis: when as little as 1.82% of normal cells undergo malignant transformation, the characteristic absorption peak vanishes, and a distinct bistable effect emerges concomitantly. Furthermore, the bistable region broadens progressively with an increasing proportion of cancerous cells. Collectively, these results deepen the fundamental understanding of bistability regulation in hybrid quantum-plasmonic systems interfaced with biological materials, while establishing a quantitative, ultra-sensitive readout that enables highly accurate early cancer diagnosis.

Notably, this study establishes an explicit, direct connection between coating composition (i.e., MNS dielectric shell thickness), surface morphology-dependent plasmonic responses, and the hybrid system’s biosensing performance. By correlating structural parameters of the MNS coating with the tunability of optical bistability and the resultant sensing sensitivity for cancer cell detection, our work forges a critical bridge between advanced coating science and cutting-edge biomedical sensing. Beyond these findings, our results lay a solid foundation for advancing the development of next-generation optical switches and high-performance biomedical sensing platforms tailored to clinical diagnostics and biotechnological applications.