A ray of light in air is incident with an angle θ on one side of a right-angle prism with refractive index n, as shown in Figure 1.

The light ray is refracted into the prism and it propagates toward the hypotenuse surface of the prism. At that surface, there is a boundary between the prism and air. If the angle of incidence at the boundary is θ₁, then we have: (1) θ 1 = 45 ° + sin − 1 ( sin   θ n )

Here the signs of θ₁ and θ are defined as positive if they are measured clockwise from a surface normal. If θ₁ is larger than the critical angle θ_C, the light is totally reflected at the boundary. According to Fresnel’s equations, the phase difference between s- and p-polarizations is given as: (2) ϕ 1 = 2   tan − 1 { sin 2 [ 45 ° + sin − 1 ( sin   θ n ) ] − 1 n 2 tan [ 45 ° + sin − 1 ( sin   θ n ) ]   sin [ 45 ° + sin − 1 ( sin   θ n ) ] } .

From Equation (2), the variation Δθ of the incident angle can be written as: (3) Δ θ ≅ ( n 2   tan 2 θ 1 − 1 ) ( n 2   sin 2 θ 1 − 1 ) 1 2 2 n   sin   θ 1 [ 2 − ( n 2 − 1 )   tan 2   θ 1 ] × ( n 2 − sin 2   θ ) 1 2 cos   θ Δ ϕ 1 = A ( θ ) Δ ϕ 1where Δθ₁ is the phase difference variation and: (4) A ( θ ) = ( n 2   tan 2   θ 1 − 1 ) ( n 2   sin 2   θ 1 − 1 ) 1 2 2 n   sin   θ 1 [ 2 − ( n 2 − 1 ) tan 2   θ 1 ] × ( n 2 − sin 2   θ ) 1 2 cos   θ ..

In this paper, a right-angle prism with a four-layer device [prism-titanium(Ti)-gold(Au)-air] in the Kretchmann’s configuration [6] is used. For the Kretchmann configuration of the four-layer system as shown in Figure 2, the surface plasmons are excited when α equals to the resonant angle α_sp.

From Maxwell’s equations, the reflection coefficients of p- and s-polarizations can be expressed as [7]: (5) r 1234 t = r 12 t + r 234 t e i 2 k z 2 d 2 1 + r 12 t r 234 t e i 2 k z 2 d 2 , (6) r 234 t = r 23 t + r 34 t e i 2 k z 3 d 3 1 + r 23 t r 34 t e i 2 k z 3 d 3where r ij t = E i t − E j t E i t + E j t, d₂ and d₃ are the thicknesses of medium 2 and medium 3, respectively, and t = p, s, (7) E I t = { n I 2 / k z I t = p k z I t = s ,     I = i ,   j ; i ,   j = 1 , 2 , 3 , 4 .

In Equation (7)k_zi(j) is the component of the wave vector in a medium i(j) in the z direction and is given as: (8) k zi ( j ) = k 0   ( n i ( j ) 2 − n 1 2   sin 2   α ) ,where n₁ is the refractive index of the prism, n₂ is the refractive index of Ti metal, n₃ is the refractive index of Au metal, n₄ is the refractive index of air and k₀ is the wave vector in vacuum. If the amplitude reflection coefficients r 1234 p and r 1234 s are written as: (9) r 1234 p = | r 1234 p | e i ϕ p , r 1234 s = | r 1234 s | e i ϕ s ,then the phase difference ϕ between p- and s- polarization components is: (10) ϕ = ϕ p − ϕ s

Besides, the reflectivities of p- and s- polarization components are R p = | r 1234 p | 2 and R s = | r 1234 s | 2, respectively.

Because the phase difference variation Δϕ₂ due to the SPR effect is a function of the rotation angle or deviation angle Δθ [8], the phase difference variation can be given by: (11) Δ ϕ 2 = B ( θ ) Δ θwhere B(θ) is a parameter dependent on the initial incident angle.

Figure 3 shows the structure of a DP; it consists of an objective lens with a focal length f and a mirror located near the focal plane of an objective lens. A light ray coming from a heterodyne light source is incident on the objective lens and is reflected by the mirror. The reflected light passes through the objective lens again. And it is in the anti-parallel direction with the incident beam if the mirror is just located at the focal plane. Otherwise it will have a deviation angle Δθ with the anti-direction of the incident beam. From the refraction ray- tracing equation, we can achieve: (12) Δ z ≅ − f 2 D Δ θ

From Equation (12), it can be seen that the displacement Δz is almost proportional to Δθ.

Figure 4 shows the scheme of the small-displacement sensor. The small-displacement sensor is made of a right-angle prism with a refractive index of 1.51509 at the wavelength λ = 632.8 nm. A right-angle side of the right-angle prism isn’t coated with any metal film, but the other side is coated with two layers of metal film. If the light reflects from the DP (see Figure 3), it later enters the small-displacement sensor.

Suppose that the defocusing amount Δz exists, the reflected light coming from the DP is convergent or divergent. Suppose the reflected light is divergent, the incident angle on the hypotenuse of the right-angle prism is not unique. In Figure 4, the incident angles of the two marginal rays that lie on the plane perpendicular to the hypotenuse of the right-angle prism are θ₁ = θ₀ + Δθ and θ₂ = θ₀ − Δθ, respectively, where θ₀ is the initial incident angle as Δz = 0. It must be noted that the initial incident angle θ₀ is chosen when the incident angle α equals to the resonant angle α_sp, i.e. θ₀ = sin⁻¹[nsin(α_sp−45°)]. This is because that the SPR effect has a very steep phase change around the resonant angle [8].

We can then obtain the incident angles of the two marginal rays that are incident at one side of the right-angle prism (the surface of the side is uncoated metal) are β 1 = 45 ° + sin − 1 [ sin ( θ 0 + Δ θ ) n ] and β 2 = 45 ° + sin − 1 [ sin ( θ 0 − Δ θ ) n ], respectively. If the phase difference variation is Δϕ₁ for one marginal ray indent on the side of the right-angle prism at the incident angle β₁, then we can obtain the phase difference variation Δϕ₁^′ = −Δϕ₁ for the other marginal ray at the incident angle β₂. Thus we can achieve the phase difference variation on account of the TIR effect is: (13) Δ ϕ t 1 = Δ ϕ 1 − Δ ϕ 1 ′ ≈ 2 Δ ϕ 1 .

Combining Equation (3) with (13), we have: (14) Δ ϕ t 1 ≈ 2 Δ θ A ( θ )

Similarly, the incidence angles of the two marginal rays that are incident at the other side (coated with two metal layers) are α 1 = 45 ° − sin − 1 [ sin ( θ 0 + Δ θ ) n ] and β 2 = 45 ° − sin − 1 [ sin ( θ 0 − Δ θ ) n ], respectively.

If the phase difference variation is Δϕ₂ for one marginal ray incident on the side of the right-angle prism at the incidence angle α₁, then we can obtain the phase difference variation Δϕ₂^′ = −Δϕ₂ for the other marginal ray at the incidence angle α₂. Thus we can achieve the phase difference variation on account of the SPR phenomenon as: (15) Δ ϕ t 2 = Δ ϕ 2 − Δ ϕ 2 ′ ≈ 2 Δ ϕ 2 .

From Eqs (11) and (15), Δϕ_t2 can be expressed by: (16) Δ ϕ t 2 ≈ 2 B ( θ ) Δ θ

Therefore, the total phase difference variation Δϕ_t is given as: (17) Δ ϕ t = Δ ϕ t 1 + Δ ϕ t 2 ≈ 2 ( 1 A ( θ ) + B ( θ ) ) ⋅ Δ θ = C ( θ ) ⋅ Δ θwhere C ( θ ) = 2 ( 1 A ( θ ) + B ( θ ) ). According to Eqs. (12) and (17), the total phase difference variation Δϕ₂ due to the displacement Δz can be written as: (18) Δ ϕ t ≈ − C ( θ ) ⋅ D f 2 ⋅ Δ z .

The first order error accompanying the phase range 0 ∼ 2π has the same cycle. The errors are frequency mixing error and polarization mixing error due to:

The light source is not complete linear polarization [10];

Suppose that the polarizer or polarization beam-splitter is imperfect, it can be found that a small amount of the TE state exists in the TM arm and vice versa. Thus, the electric fields of the TM and TE arms are given by, respectively [10,12–13]: (20) E TM = ( Ae i ω t / 2 + α e − i ω t / 2 ) e i ω 0 t (21) E TE = ( Be − i ω t / 2 + βe i ω t / 2 ) e i ω 0 t ,where ω and ω₀ are the angular frequencies of heterodyne light and optical light source, respectively; A = |A|e^iϕ_A and B = |B|e^iϕ_B are the amplitudes of p- and s-polarizations, respectively; and α = |α|e^iϕ_α and β = |β|e^iβ_β are the polarization mixing amplitudes of s- and p-polarizations, respectively.

If the transmission axis of analyzer with respect to x-axis is equal to 45° and all phase are the same, such as ϕ_α = ϕ_A, ϕ_β = ϕ_B and ϕ_A = ϕ_B, then the reference beam signal I_r is given by [14]: (22) I r = | E r | 2 = 1 2 [ ( | A | 2 + | B | ) 2 + ( | α | 2 + | β | ) 2 + 2 ( | A | | α | + | B | | β | + | A | | B | + | B | | β | ) cos ω t ]Similarly, the test beam signal I_t is: (23) I t = | E t | 2 = 1 2 [ | A | 2 + | B | 2 + | α | 2 + | β | 2 + 2 ( | A | | α | + | B | | β | ) cos   ω t + 2 ( | A | | β | + | B | | α | ) cos   ϕ               + 2 | A | | B | cos ( ω t + ϕ ) + 2 | α | | β | cos ( ω t − ϕ ) ] (24) = | A | 2 + | B | 2 + | α | 2 + | β | 2 + 2 ( | A | β | + | B | | α | | ) cos   ϕ       + 2 ( | A | | α | + | B | | β | + ( | A | | B | + | α | | β | ) cos ϕ ) 2 + ( | A | | B | + | α | | β | ) 2   sin 2   ϕ cos ( ω t + ϕ ′ ) ,where ϕ′ and ϕ are the measured value and the theoretical value, respectively. And ϕ′ is given by: (25) ϕ ′ = tan − 1 [ ( | A | | B | − | α | | β | ) sin   ϕ ( | A | | α | + | B | | β | ) + ( | A | | B | + | α | | β | ) cos ϕ ] .

Thus, the first order error due to polarization mixing is Δϕ_m = ϕ′ − ϕ. For convenience, we set |A| = |B| and |α| = |β|. Figure 9 shows the first order error as a function of the phase difference ϕ. It is evident that the maximum error is 0.36° if the extinction ratio is equal to 1 × 10⁻⁵, i.e. | α | | A | = 0.0032.

Second harmonic error is due to the polarization rotation between s- and p-polarizations. And the second harmonic error Δϕ_r is given by [15]: (26) Δ ϕ r = tan − 1 ( sin   ϕ cos   2 θ r   cos   ϕ ) − ϕ ,where θ_r is the polarization rotation angle. As shown in Figure 10, the second harmonic error Δθ_r is a function of the phase difference ϕ for different polarization rotation angles. It is clear that the second harmonic error Δθ_r is close to zero if the polarization rotation angle θ_r = 0.1°.

Besides, if the phase difference error only results from the resolution of the lock-in amplifier, the phase difference error Δϕ is given by: (27) Δ ϕ = d ( Δ ϕ t ) d z Δ z = d ( Δ ϕ t 1 + Δ ϕ t 2 ) d z Δ z .

Because the resolution Δϕ of the lock-in amplifier is equal to 0.01°, then the small-displacement measurement error Δz, of the system can be obtained by: (28) Δ z r = Δ z = Δ ϕ d ( Δ ϕ t 1 + Δ ϕ t 2 ) / dz .

As a matter of fact, the Δz can be regarded as the solution of the system if the inherent nonlinear errors have been adjusted to zero by choosing high performance optical components and aligning the optical system carefully. According to Equation (28), the theoretical displacement resolution of the small-displacement sensor can reach 0.45 nm in the displacement range of −500 nm ≤ Δz ≤ +500 nm, as shown in Figure 11.