Stereo Digital Image Correlation in MATLAB

Abstract

Digital Image Correlation (DIC) has found widespread use in measuring full-field displacements and deformations experienced by a body from images captured of it. Stereo-DIC has received significantly more attention than two-dimensional (2D) DIC since it can account for out-of-plane displacements. Although many aspects of Stereo-DIC that are shared in common with 2D DIC are well documented, there is a lack of resources that cover the theory of Stereo-DIC. Furthermore, publications which do detail aspects of the theory do not detail its implementation in practice. This literature gap makes it difficult for newcomers to the field of DIC to gain a deep understanding of the Stereo-DIC process, although this knowledge is necessary to contribute to the development of the field by either furthering its capabilities or adapting it for novel applications. This gap in literature acts as a barrier thereby limiting the development rate of Stereo-DIC. This paper attempts to address this by presenting the theory of a subset-based Stereo-DIC framework that is predominantly consistent with the current state-of-the-art. The framework is implemented in practice as a 202 line MATLAB code. Validation of the framework shows that it performs on par with well-established Stereo-DIC algorithms, indicating it is sufficiently reliable for practical use. Although the framework is designed to serve as an educational resource, its modularity and validation make it attractive as a means to further the capabilities of DIC.

1. Introduction

Digital Image Correlation (DIC) is an optical metrology technique capable of determining full-field displacements and deformations experienced by a body from images captured of the body’s surface. The accuracy of DIC results is greatly influenced by the quality of the speckle pattern [1], the internals of the DIC algorithm [2,3], the appropriateness of the chosen processing parameters [4] and the quality of the calibration process [5,6,7], among others. However, despite this, DIC, a non-interferometric technique, is more robust in withstanding vibrations and ambient light variations than interferometric techniques, such as Electronic Speckle Pattern Interferometry and Moiré interferometry, allowing it to be used outside the confines of a laboratory. Additionally, its non-contact, full-field nature; its ability to function at a range of scales (from scanning electron microscopy images [8,9] to satellite images [10]); and the rapid improvement of cameras has resulted in it becoming one of the most favored techniques in experimental solid mechanics.

Although DIC has received the most attention and thus development within experimental solid mechanics, it has found widespread use across many fields. Some applications of DIC include: (i) investigating mechanical properties of biological tissues [11,12,13,14,15]; (ii) deflection measurement and structural health monitoring of bridges [16,17,18,19,20,21,22]; (iii) investigating cracks in masonry structures [23,24,25]; (iv) landslide monitoring [26,27,28,29]; (v) determining material properties of woven fabrics (such as cotton) [30,31]; vibrational analysis of components [32,33,34]; and (vii) in situ damage detection of components and structures [35,36,37] to name a few.

Following the introduction of two-dimensional (2D) DIC in 1982 by Peters and Ranson [38] it was extended to three dimensions (Stereo-DIC) by Luo et al. [39] in 1993 and to digital volume correlation (DVC) by Bay et al. [40] in 1999. The correlation aspect of DIC has undergone significant refinement over the years. Current state-of-the-art DIC algorithms as identified by Pan [41] make use of: (i) bi-quintic b-spline interpolation which was shown by Schreier et al. [3] to produce the most accurate sub-pixel displacements; (ii) the first and second-order shape functions (SFs) proposed by Peters et al. [42] and Lu and Carry [43] respectively; (iii) the zero-mean normalized sum of squared difference (ZNSSD) correlation criterion suggested by Tong [44]; (iv) Gaussian pre-filtering of images to reduce bias in the displacements caused by high frequency noise in the images as proposed by Pan [45]; and (v) the inverse compositional Gauss–Newton (IC-GN) optimization method introduced by Baker and Matthews [46]. Although the IC-GN optimization method is theoretically equivalent to the Newton–Raphson method [46], in practice it has greater efficiency, accuracy and robustness to withstand noise [47] making it preferable.

Although DIC is a complex process, the papers proposed by Pan et al. [48], Gao et al. [49], Solav et al. [50], and Blaber et al. [51] provide an in-depth explanation of the theory of the DIC process, which coupled with the Good Practices Guide for DIC [52], enables newcomers to the field of DIC to gain the necessary knowledge to effectively apply it in established use cases. However, in order for a newcomer to contribute to the field of DIC by developing new algorithms which improve its capabilities or adapting it for new applications, he or she must gain a deep understanding of DIC. Gaining such a deep understanding is cumbersome due to the lack of resources which directly bridge the gap between the theory of DIC and its coded implementation. More specifically, although some publications release code that is consistent with the theory presented [50,51], these codes focus on robustness and ease of use which, despite making them more suitable for real world applications, results in a complex code which is ineffective as a learning resource. This lack of resources acts as a barrier to newcomers intending to further the capabilities of DIC, thereby limiting the development rate of the field.

It is for this reason that the authors published a paper bridging the gap between the theory of a 2D DIC framework (ADIC2D) and its practical implementation as a modular 117 line MATLAB code [53]. However, 2D DIC, being limited to determining in-plane displacements, is susceptible to errors in the presence of out-of-plane motion, which is generally unavoidable especially outside the confines of a laboratory [54]. As such, Stereo-DIC has become favored since it accounts for out-of-plane motion, allowing for more reliable use across a broader range of applications and fields. This is supported by the significantly larger number of publications of Stereo-DIC relative to 2D DIC.

This paper builds upon the work of the preceding paper by extending ADIC2D to Stereo-DIC with the aim of bridging the gap between the theory and implementation of Stereo-DIC. This is done by first presenting the theory of a subset-based, Stereo-DIC framework that is predominantly consistent with current state-of-the-art techniques. How this theory is implemented as a modular 202 line MATLAB code is then discussed. These theory and implementation sections focus on aspects of Stereo-DIC which differ from that of ADIC2D. Thereafter the framework is validated using image sets provided by the Stereo-DIC Challenge [55]. More specifically, ADIC3D’s results are compared to those of LaVision’s StrainMaster software and DICe [56] in order to make observations about its capabilities. Finally, the proposed framework’s modularity, limitations, and overall performance are discussed.

The framework, referred to as ADIC3D, was developed in MATLAB since its simple syntax lets the reader focus on the core mathematics of the code. Furthermore, MATLAB’s efficient built-in functions are leveraged to simplify the code and reduce its computation time. The code is designed to be modular, so that the link between the theory and code is clear and enables readers to systematically develop an understanding of the code. Additionally, this modularity allows the code to be easily altered, urging readers to further the capabilities of DIC.

2. Framework Theory

Stereo-DIC is fundamentally an extension of 2D DIC, which relies upon two or more cameras simultaneously capturing images of the specimen (where specimen refers to the body being tracked) from different perspectives in order to determine its in-plane and out-of-plane displacements. As such, there are aspects of Stereo-DIC which are identical to those of 2D DIC; namely calibration and correlation. Stereo-DIC relies upon epipolar geometry to determine in-plane and out-of-plane displacements in the real world from in-plane displacements determined within images. Section 2.5 discusses how calibration and correlation are employed by Stereo-DIC based on epipolar geometry and how triangulation methods are used to perform displacement transformations.

Although Stereo-DIC can employ more than two cameras [50], ADIC3D is limited to the conventional two-camera implementation. Variables that need to be defined on a per camera basis have the subscript $j$ to indicate that they are associated with the $j$^th camera (i.e., camera 1 and 2). Within this work, an image series is defined as the collection of images captured by a single camera and an image set refers to the two corresponding image series. Thus, an image pair refers to the two images, one from each image series, which were captured simultaneously.

2.1. Homogeneous Coordinates

Calibration and triangulation use homogeneous coordinates [57], to represent coordinates in projective space, such that affine and projective transformations can be implemented through matrix multiplication. Appending a scaling variable of unity to a $n$ dimensional vector in Euclidean space converts it to an $n+1$ dimensional vector in projective space. Converting back to Euclidean space, represented as function $\mathsf{\Phi }$, entails dividing the vector’s elements by the scaling variable before eliminating it to reduce the vector to $n$ dimensions. Underlined variables denote homogenous coordinate vectors.

2.2. Calibration

Calibration, explained in terms of a single camera since each camera is calibrated separately, determines the parameters of the camera model, which involves the four coordinate systems (CSs) depicted in Figure 1: (i) the three-dimensional (3D) world CS having arbitrary orientation and position; (ii) the 3D camera CS with its origin at the aperture of the camera and its z-axis aligned with the optical axis; (iii) the 2D ideal sensor CS which is coincident with the plane of the charge coupled device (CCD) representing the image in the absence of distortion; and (iv) the 2D distorted sensor CS which is coincident with the ideal sensor CS but represents the actual image by accounting for radial distortion. No symbols have been assigned to the camera CS since it is not directly involved in DIC.

The camera model uses the pinhole camera model to project a 3D coordinate of a point on the specimen in the world CS to its corresponding 2D location in the ideal sensor CS before the radial distortion model determines its location in the distorted sensor CS. Although this camera model, and subsequent calibration process, is rather simple, it performs sufficiently well as shown in Section 4. However, the accuracy the displacements determined could be improved by using a more sophisticated camera calibration method which provides a more complete description of the camera optics such as the generic camera calibration method [58,59].

2.2.1. Pinhole Camera Model

The projection matrix ${\mathit{Q}}_j$, consisting of intrinsic (${\mathit{K}}_j$) and extrinsic (${\mathit{V}}_j$) parameter matrices, relates a coordinate in the world CS, ${\underset{¯}{\widehat{\mathit{x}}}}_w={\left[\begin{array}{cccc}{\widehat{x}}_w& {\widehat{y}}_w& {\widehat{z}}_w& 1\end{array}\right]}^T$, to a coordinate in the ideal sensor CS, ${\underset{¯}{\widehat{\mathit{x}}}}_{s_j}={\left[\begin{array}{ccc}{\widehat{x}}_{s_j}& {\widehat{y}}_{s_j}& 1\end{array}\right]}^T$, as

$${\alpha }_j{\underset{¯}{\widehat{\mathit{x}}}}_{s_j}={\mathit{Q}}_j{\underset{¯}{\widehat{\mathit{x}}}}_w={\mathit{K}}_j{\mathit{V}}_j{\underset{¯}{\widehat{\mathit{x}}}}_w=\left[\begin{array}{ccc}{\xi }_{x_j}& c_{s_j}& c_{x_j}\\ 0& {\xi }_{y_j}& c_{y_j}\\ 0& 0& 1\end{array}\right]\left[\begin{array}{cccc}{R}_j^{11}& R_j^{12}& R_j^{13}& T_j^1\\ R_j^{21}& R_j^{22}& R_j^{23}& T_j^2\\ R_j^{31}& R_j^{32}& R_j^{33}& T_j^3\end{array}\right]{\underset{¯}{\widehat{\mathit{x}}}}_w.$$

(1)

Here ${\alpha }_j$ is the arbitrary scaling variable of the homogenous coordinates. First, the rotation matrix, ${\mathit{R}}_j$, and translation vector, ${\mathit{T}}_j$, transform ${\underset{¯}{\widehat{\mathit{x}}}}_w$ to the camera CS. Thereafter this 3D coordinate is projected to the ideal sensor CS using ${\mathit{K}}_j$ which performs four operations: (i) projects the 3D coordinate to a 2D coordinate on the plane of the CCD; (ii) scales the coordinate from metric units to units of pixels using ratios ${\xi }_{x_j}$ and ${\xi }_{y_j}$ in the x- and y-directions respectively; (iii) applies a translation, $c_{x_j}$ and $c_{y_j}$ in the x- and y-directions respectively, placing the origin of the ideal sensor CS at the top left of the image; and (iv) converts from the orthogonal camera CS to a skew ideal sensor CS using $c_{s_j}$. In this work $c_{s_j}=0$ since an orthogonal ideal sensor CS is assumed.

${\mathit{K}}_j$ is only dependent on the camera system and does not change if the camera position changes. In contrast, ${\mathit{V}}_j$ depends on the relative position and orientation between the camera and the world CS.

2.2.2. Radial Distortion Model

The radial distortion model is sufficient to account for distortion in the images caused by imperfections of the lens system [60]. First the normalized, ideal image coordinates ${\widehat{\mathit{x}}}_{n_j}={\left[\begin{array}{cc}{\widehat{x}}_{n_j}& {\widehat{y}}_{n_j}\end{array}\right]}^T$ are computed as

$${\widehat{\mathit{x}}}_{n_j}=\left[\begin{array}{c}{\widehat{x}}_{n_j}\\ {\widehat{y}}_{n_j}\end{array}\right]=\mathsf{\Phi }\left({\mathit{K}}_j^{-1}{\underset{¯}{\widehat{\mathit{x}}}}_{s_j}\right).$$

(2)

Thereafter, the unit-less radial distortion parameters, ${\kappa }_j^1$ and ${\kappa }_j^2$, are used to compute the normalized, distorted image coordinates, ${\mathit{x}}_{n_j}={\left[\begin{array}{cc}{x}_{n_j}& y_{n_j}\end{array}\right]}^T$, as [61]

$${\mathit{x}}_{n_j}=\left(1+{\kappa }_j^1{\widehat{\mathit{x}}}_{n_j}^T{\widehat{\mathit{x}}}_{n_j}+{\kappa }_j^2{\left({\widehat{\mathit{x}}}_{n_j}^T{\widehat{\mathit{x}}}_{n_j}\right)}^2\right){\widehat{\mathit{x}}}_{n_j}.$$

(3)

Finally, the distorted coordinates in the distorted sensor CS, ${\mathit{x}}_{s_j}={\left[\begin{array}{cc}{x}_{s_j}& y_{s_j}\end{array}\right]}^T$, are obtained as

$${\mathit{x}}_{s_j}=\left[\begin{array}{c}{x}_{s_j}\\ y_{s_j}\end{array}\right]=\mathsf{\Phi }\left({\mathit{K}}_j\left[\begin{array}{c}{x}_{n_j}\\ y_{n_j}\\ 1\end{array}\right]\right).$$

(4)

2.2.3. Determining Calibration Parameters

Prior to performing calibration, a calibration image series must be captured of the calibration plate. The calibration plate is a planar object containing distinct, point-like features, called calibration targets (CTs), at known 3D locations which define the position and orientation of the world CS. A corresponding set of 2D coordinates in the distorted sensor CS are obtained by locating the CTs in the calibration images. These sets of 3D and 2D coordinates are used to solve for the parameters of the camera model, which describe the relationship between the two. Using at least 50 calibration images is recommended since this inverse problem is sensitive to noise in the 3D and 2D coordinates [41,62].

Calibration is performed using two steps. First, initial estimates for ${\mathit{K}}_j$ and ${\mathit{V}}_j$ are obtained using the method proposed by Zhang [61] while ${\kappa }_j^1$ and ${\kappa }_j^2$ are set to zero. Secondly, non-linear, least-squares optimization is used to optimize all the parameters by minimizing the total projection error, $E_{proj}$, given as

$$E_{proj}={{\displaystyle \sum }}_{l=1}^L\left({\left(x_l^{calc}-x_l^{true}\right)}^2+{\left(y_l^{calc}-y_l^{true}\right)}^2\right).$$

(5)

Here ${\mathit{x}}_l^{true}={\left[\begin{array}{cc}{x}_l^{true}& y_l^{true}\end{array}\right]}^T$ and ${\mathit{x}}_l^{calc}={\left[\begin{array}{cc}{x}_l^{calc}& y_l^{calc}\end{array}\right]}^T$ are the locations of the $l$^th CT in the distorted sensor CS determined from the calibration image series and predicted by the camera model respectively. There is a total of $L$ many CTs visible within the calibration image series.

2.3. Correlation

ADIC3D uses correlation to track a subset, a cluster of pixels, between images. The specimen’s surface requires an isotropic, random pattern with high information content such that subsets are tracked reliably. Correlation operates on two images: a reference image, $F$, representing the specimen in its relaxed state and a deformed image, $G$, representing the specimen after experiencing displacement and/or deformation. Correlation determines how a subset of the reference image, called a reference subset ($\mathit{f}$), must displace and deform such that it matches a corresponding subset in the deformed image, called an investigated subset ($\mathit{g}$). This is referred to as the correspondence problem, which is mathematically defined as the objective function. For a more detailed discussion on the correspondence problem, the reader is referred to the authors’ previous paper [53].

Note that correlation operates solely within the distorted sensor CS and is unconcerned with which camera the distorted sensor CS belongs to. Thus, in Section 2.3 and Section 3.2, the subscript $s_j$ is dropped from ${\mathit{x}}_{s_j}$ (i.e., $\mathit{x}$) to simplify the mathematics.

To understand how the objective function is comprised of the SF, the interpolation method and correlation criterion first consider that the $i$^th pixel position of $\mathit{f}$, ${\mathit{x}}_i={\left[\begin{array}{cc}{x}_i& y_i\end{array}\right]}^T$, is represented by the reference subset center position, ${\mathit{x}}^o={\left[\begin{array}{cc}{x}^o& y^o\end{array}\right]}^T$, and the distance from ${\mathit{x}}^o$ to ${\mathit{x}}_i$, $\Delta {\mathit{x}}_i={\left[\begin{array}{cc}\mathsf{\Delta }{x}_i& \mathsf{\Delta }{y}_i\end{array}\right]}^T$, as

$${\mathit{x}}_i=\Delta {\mathit{x}}_i+{\mathit{x}}^o.$$

(6)

The SF, $\mathit{W}$, determines how the pixel positions of $\mathit{f}$ displace as a result of the displacement and deformation of $\mathit{f}$, which is quantified by the shape function parameters (SFPs), $\mathit{P}$. The SF does this by modifying $\Delta {\mathit{x}}_i$ as

$$\Delta {\mathit{x}}_i^{\prime }=\left[\begin{array}{c}\mathsf{\Delta }{x}_i^{\prime }\\ \mathsf{\Delta }{y}_i^{\prime }\end{array}\right]=\mathit{W}\left(\Delta {\mathit{x}}_i,\mathit{P}\right).$$

(7)

The pixel positions of $\mathit{f}$ after displacement and deformation, ${\mathit{x}}_i^{\prime }={\left[\begin{array}{cc}{x}_i^{\prime }& y_i^{\prime }\end{array}\right]}^T$, (referred to as query points) are determined as

$${\mathit{x}}_i^{\prime }=\Delta {\mathit{x}}_i^{\prime }+{\mathit{x}}^o.$$

(8)

Thus $\Delta {\mathit{x}}_i^{\prime }$ represents the distance from the center of $\mathit{f}$ to ${\mathit{x}}_i^{\prime }$. The light intensity values of $G$ are interpolated at ${\mathit{x}}_i^{\prime }$ to obtain $\mathit{g}$. Thus, $\mathit{g}$ represents a portion of $G$ at a location described by the displacement SFPs with the deformation defined by the SFPs having been removed. The correlation criterion quantifies the degree of similarity between $\mathit{f}$ and $\mathit{g}$.

2.3.1. Shape Function

The main SFs are the zero (${\mathit{W}}^{SF0}$), first (${\mathit{W}}^{SF1}$), and second-order SFs (${\mathit{W}}^{SF2}$) given as [43]

$$\begin{gathered} {\mathit{W}}^{SF0}\left(\Delta {\mathit{x}}_i,{\mathit{P}}^{SF0}\right)=\left[\begin{array}{ccc}1& 0& u\\ 0& 1& v\end{array}\right]\left[\begin{array}{c}\Delta x_i\\ \Delta y_i\\ 1\end{array}\right], \\ {\mathit{W}}^{SF1}\left(\Delta {\mathit{x}}_i,{\mathit{P}}^{SF1}\right)=\left[\begin{array}{ccc}1+u_x& u_y& u\\ v_x& 1+v_y& v\end{array}\right]\left[\begin{array}{c}\Delta x_i\\ \Delta y_i\\ 1\end{array}\right] \\ \mathrm{and}{\mathit{W}}^{SF2}\left(\Delta {\mathit{x}}_i,{\mathit{P}}^{SF2}\right)=\left[\begin{array}{cccccc}\frac{1}{2}{u}_{xx}& u_{xy}& \frac{1}{2}{u}_{yy}& 1+u_x& u_y& u\\ \frac{1}{2}{v}_{xx}& v_{xy}& \frac{1}{2}{v}_{yy}& v_x& 1+v_y& v\end{array}\right]\left[\begin{array}{c}\Delta x_i^2\\ \Delta x_i\Delta y_i\\ \Delta y_i^2\\ \Delta x_i\\ \Delta y_i\\ 1\end{array}\right]. \end{gathered}$$

(9)

Their sets of SFPs are given as

$$\begin{gathered} {\mathit{P}}^{SF0}={\left[\begin{array}{cc}u& v\end{array}\right]}^T, \\ {\mathit{P}}^{SF1}={\left[\begin{array}{cccccc}u& u_x& u_y& v& v_x& v_y\end{array}\right]}^T \\ \mathrm{and}{\mathit{P}}^{SF2}={\left[\begin{array}{cccccccccccc}u& u_x& u_y& u_{xx}& u_{xy}& u_{yy}& v& v_x& v_y& v_{xx}& v_{xy}& v_{yy}\end{array}\right]}^T. \end{gathered}$$

(10)

Here $u$ and $v$ represent the displacement of $\mathit{f}$ in the x- and y-directions respectively. Furthermore, $u_x$, $u_{xx}$, $v_y,$ and $v_{yy}$ define stretching of $\mathit{f}$, whereas $u_y$, $v_x$, $u_{yy}$, $v_{xx}$, $u_{xy}$, and $v_{xy}$ define shearing. Although higher-order SFs more reliably determine displacements in complex displacement fields by accounting for more complex deformation, the DIC process is only concerned with the displacements determined by correlation.

2.3.2. Interpolation

ADIC3D employs the bi-cubic b-spline interpolation method detailed in the work of Hou et al. [63]. Letting $F$ and $G$ represent interpolation functions, the light intensity values of pixel $i$ of the reference, $f_i$, and investigated subsets, $g_i$, are interpolated as

$$f_i=F\left({\mathit{x}}^o+\Delta {\mathit{x}}_i\right) \mathrm{and}{g}_i=G\left({\mathit{x}}^o+\mathit{W}\left(\Delta {\mathit{x}}_i,\mathit{P}\right)\right).$$

(11)

Images are filtered using a Gaussian low-pass filter, detailed by Pan [45], since otherwise the sensitivity of bi-cubic b-spline interpolation to high frequency noise would lead to bias in the displacements. The Gaussian filtering parameters, being the window size ($\beta$) and the standard deviation of the Gaussian function (${\sigma }^g$), should be chosen to reduce bias without increasing variance.

2.3.3. Correlation Criterion

The objective function employs the more computationally efficient ZNSSD correlation criterion ($C_{ZNSSD}$), having a range of $\left\{C_{ZNSSD}\in ℝ|0\le C_{ZNSSD}\le 4\right\}$ where smaller values indicate higher similarity, given as

$$C_{ZNSSD}={{\displaystyle \sum }}_{i=1}^I{\left[\frac{f_i-\overline{f}}{\tilde{f}}-\frac{g_i-\overline{g}}{\tilde{g}}\right]}^2,$$

(12)

where

$$\overline{f}=\frac{{{\displaystyle \sum }}_i^I{f}_i}{I},\overline{g}=\frac{{{\displaystyle \sum }}_i^I{g}_i}{I},$$

(13)

$$\tilde{f}=\sqrt{{{\displaystyle \sum }}_{i=1}^I{\left(f_i-\overline{f}\right)}^2}\mathrm{and}\tilde{g}=\sqrt{{{\displaystyle \sum }}_{i=1}^I{\left(g_i-\overline{g}\right)}^2}.$$

(14)

Here there are $I$ many pixels per subset. The correlation coefficient is reported as the zero-mean normalized cross correlation criterion (ZNCC), $C_{ZNCC}$, since its range, $\left\{C_{ZNCC}\in ℝ|-1\le C_{ZNCC}\le 1\right\}$ where larger values indicate higher similarity, is more intuitive. These correlation criteria are related as [64]

$$C_{ZNCC}=1-\frac{C_{ZNSSD}}{2}.$$

(15)

2.3.4. Objective Function

The objective function, based on Equation (12), is given as

$$C_{ObjFun}={{\displaystyle \sum }}_{i=1}^I{\left[\frac{f_i-\overline{f}}{\tilde{f}}-\frac{G\left({\mathit{x}}^o+\mathit{W}\left(\Delta {\mathit{x}}_i,\mathit{P}\right)\right)-\overline{g}}{\tilde{g}}\right]}^2.$$

(16)

Therefore, correlation seeks the SFPs that define $\mathit{g}$ to contain a light intensity pattern similar to $\mathit{f}$ thereby optimizing $C_{ZNSSD}$. However, the IC-GN optimization method requires modifying the objective function in order to derive an iterative optimization equation from it. More specifically, the current estimate of the SFPs, $\mathit{P}$, is applied to the investigated subset while the iterative improvement of the SFPs, $\mathsf{\Delta }\mathit{P}$, is applied to the reference subset as

$$C_{ObjFun}={{\displaystyle \sum }}_{i=1}^I{\left[\frac{F\left({\mathit{x}}^o+\mathit{W}\left(\Delta {\mathit{x}}_i,\mathsf{\Delta }\mathit{P}\right)\right)-\overline{f}}{\tilde{f}}-\frac{G\left({\mathit{x}}^o+\mathit{W}\left(\Delta {\mathit{x}}_i,\mathit{P}\right)\right)-\overline{g}}{\tilde{g}}\right]}^2.$$

(17)

2.3.5. Optimization Equation

The optimization equation derived from Equation (17), derivation detailed in [53], is given as

$$\Delta \mathit{P}=-{\mathit{H}}^{-1}{{\displaystyle \sum }}_{i=1}^I{\left(\nabla {\mathit{f}}_i\frac{\partial {\mathit{W}}_i}{\partial \mathit{P}}\right)}^T\left[f_i-\overline{f}-\frac{\tilde{f}}{\tilde{g}}\left(G\left({\mathit{x}}^o+\mathit{W}\left(\Delta {\mathit{x}}_i,\mathit{P}\right)\right)-\overline{g}\right)\right].$$

(18)

Here $\mathit{H}$ is the Hessian, defined in Equation (19), and the remaining terms (within the summation) form the Jacobian, $\mathit{J}$. $\mathit{H}$, being independent of $\mathit{P}$, is precomputed prior to beginning iterations.

$$\mathit{H}={{\displaystyle \sum }}_{i=1}^I\left[{\left(\nabla {\mathit{f}}_i\frac{\partial {\mathit{W}}_i}{\partial \mathit{P}}\right)}^T\left(\nabla {\mathit{f}}_i\frac{\partial {\mathit{W}}_i}{\partial \mathit{P}}\right)\right].$$

(19)

$\nabla {\mathit{f}}_i=\left[\begin{array}{cc}\frac{\partial f_i}{\partial x}& \frac{\partial f_i}{\partial y}\end{array}\right]$ is the light intensity gradient of $\mathit{f}$ at pixel $i$. $\frac{\partial {\mathit{W}}_i}{\partial \mathit{P}}$ is the Jacobian of the SF for pixel $i$ and based on the SF order is given as

$$\begin{gathered} \frac{\partial {\mathit{W}}_i^{SF0}}{\partial {\mathit{P}}^{SF0}}=\left[\begin{array}{cc}1& 0\\ 0& 1\end{array}\right], \\ \frac{\partial {\mathit{W}}_i^{SF1}}{\partial {\mathit{P}}^{SF1}}=\left[\begin{array}{cccccc}1& \Delta x_i& \Delta y_i& 0& 0& 0\\ 0& 0& 0& 1& \Delta x_i& \Delta y_i\end{array}\right] \\ \mathrm{and}\frac{\partial {\mathit{W}}_i^{SF2}}{\partial {\mathit{P}}^{SF2}}=\left[\begin{array}{cccccccccccc}1& \Delta x_i& \Delta y_i& \frac{\Delta {x_i}^2}{2}& \Delta x_i\Delta y_i& \frac{\Delta {y_i}^2}{2}& 0& 0& 0& 0& 0& 0\\ 0& 0& 0& 0& 0& 0& 1& \Delta x_i& \Delta y_i& \frac{\Delta {x_i}^2}{2}& \Delta x_i\Delta y_i& \frac{\Delta {y_i}^2}{2}\end{array}\right]. \end{gathered}$$

(20)

Although, each iteration of Equation (18) attempts to determine how $\mathit{f}$ should displace and deform (according to $\mathsf{\Delta }\mathit{P}$) such that it matches $\mathit{g}$, $\mathit{f}$ is not displaced or deformed in practice. Instead $\mathsf{\Delta }\mathit{P}$ is used to determine the updated SFPs, ${\mathit{P}}_{update}$, which serves as the current estimate in the next iteration. The stopping criterion deems when further iterations are redundant.

2.3.6. Updating the SFPs

${\mathit{P}}_{update}$ is obtained by composing the inverse of $\Delta \mathit{P}$ with $\mathit{P}$ as

$${\mathit{P}}_{update}=\mathit{\omega }\left(\mathit{P}\right) \mathit{\omega }{\left(\Delta \mathit{P}\right)}^{-\mathbf{1}}.$$

(21)

Here $\mathit{\omega }$, being dependent on the SF order, uses the SFPs to populate a square matrix as [43]

$$\begin{gathered} {\mathit{\omega }}^{SF0}\left({\mathit{P}}^{SF0}\right)=\left[\begin{array}{ccc}1& 0& u\\ 0& 1& v\\ 0& 0& 1\end{array}\right], \\ {\mathit{\omega }}^{SF1}\left({\mathit{P}}^{SF1}\right)=\left[\begin{array}{ccc}1+u_x& u_y& u\\ v_x& 1+v_y& v\\ 0& 0& 1\end{array}\right] \\ \mathrm{and}{\mathit{\omega }}^{SF2}\left({\mathit{P}}^{SF2}\right)=\left[\begin{array}{cccccc}1+A_1& A_2& A_3& A_4& A_5& A_6\\ A_7& 1+A_8& A_9& A_{10}& A_{11}& A_{12}\\ A_{13}& A_{14}& 1+A_{15}& A_{16}& A_{17}& A_{18}\\ \frac{1}{2}{u}_{xx}& u_{xy}& \frac{1}{2}{u}_{yy}& 1+u_x& u_y& u\\ \frac{1}{2}{v}_{xx}& v_{xy}& \frac{1}{2}{v}_{yy}& v_x& 1+v_y& v\\ 0& 0& 0& 0& 0& 1\end{array}\right], \end{gathered}$$

(22)

where

$$A_1=2u_x+u_x^2+u{u}_{xx},$$	$$A_2=2u{u}_{xy}+2\left(1+u_x\right)u_y,$$
$$A_3=u_y^2+u{u}_{yy},$$	$$A_4=2u\left(1+u_x\right),$$
$$A_5=2u{u}_y,$$	$$A_6=u^2,$$
$$A_7=\frac{1}{2}\left(v{u}_{xx}+2\left(1+u_x\right)v_x+u{v}_{xx}\right),$$	$$A_8=u_y{v}_x+u_x{v}_y+v{u}_{xy}+u{v}_{xy}+v_y+u_x,$$
$$A_9=\frac{1}{2}\left(v{u}_{yy}+2\left(1+v_y\right)u_y+u{v}_{yy}\right),$$	$$A_{10}=v+v{u}_x+u{v}_x,$$
$$A_{11}=u+v{u}_y+u{v}_y,$$	$$A_{12}=uv,$$
$$A_{13}=v_x^2+v{v}_{xx},$$	$$A_{14}=2v{v}_{xy}+2v_x\left(1+v_y\right),$$
$$A_{15}=2v_y+v_y^2+v{v}_{yy},$$	$$A_{16}=2v{v}_x,$$
$$A_{17}=2v\left(1+v_y\right)$$	$$\mathrm{and}{A}_{18}=v^2.$$

2.3.7. Stopping Criterion

The stopping criterion, $\Vert \mathsf{\Delta }\mathit{P}\Vert$, determines the magnitude of the change in the SFPs between iterations, based on $\mathsf{\Delta }\mathit{P}$, as [49]

$$\begin{gathered} \Vert \mathsf{\Delta }{\mathit{P}}^{SF0}\Vert ={\left[\mathsf{\Delta }{u}^2+\mathsf{\Delta }{v}^2\right]}^{0.5}, \\ \Vert \mathsf{\Delta }{\mathit{P}}^{SF1}\Vert ={\left[\mathsf{\Delta }{u}^2+{\left(\mathsf{\Delta }{u}_x\zeta \right)}^2+{\left(\mathsf{\Delta }{u}_y\zeta \right)}^2+\mathsf{\Delta }{v}^2+{\left(\mathsf{\Delta }{v}_x\zeta \right)}^2+{\left(\mathsf{\Delta }{v}_y\zeta \right)}^2\right]}^{0.5} \\ \mathrm{and}\Vert \mathsf{\Delta }{\mathit{P}}^{SF2}\Vert ={\left[\mathsf{\Delta }{u}^2+{\left(\mathsf{\Delta }{u}_x\zeta \right)}^2+{\left(\mathsf{\Delta }{u}_y\zeta \right)}^2+{\left(\frac{1}{2}\mathsf{\Delta }{u}_{xx}{\zeta }^2\right)}^2+{\left(\mathsf{\Delta }{u}_{xy}{\zeta }^2\right)}^2+{\left(\frac{1}{2}\mathsf{\Delta }{u}_{yy}{\zeta }^2\right)}^2+\mathsf{\Delta }{v}^2+{\left(\mathsf{\Delta }{v}_x\zeta \right)}^2+{\left(\mathsf{\Delta }{v}_y\zeta \right)}^2+{\left(\frac{1}{2}\mathsf{\Delta }{v}_{xx}{\zeta }^2\right)}^2+{\left(\mathsf{\Delta }{v}_{xy}{\zeta }^2\right)}^2+{\left(\frac{1}{2}\mathsf{\Delta }{v}_{yy}{\zeta }^2\right)}^2\right]}^{0.5}. \end{gathered}$$

(23)

Here $\zeta$ is the largest value of $\mathsf{\Delta }{x}_i$ or $\mathsf{\Delta }{y}_i$ for the subset. Iterations cease once $\Vert \mathsf{\Delta }\mathit{P}\Vert$ falls below a predefined value called the stopping criterion value.

2.4. Epipolar Geometry

Epipolar geometry is the projective geometry between the ideal sensor CSs, i.e., between camera 1 and 2. Consider the following definitions aided by Figure 2: (i) the baseline is formed between the two origins of the camera CSs; (ii); the epipolar plane is formed between the 3D coordinate and the baseline; (iii) the epipolar lines, ${\underset{\_}{\mathit{\lambda }}}_j$, are the intersection of the epipolar plane and the x-y plane of the ideal sensor CSs; and (iv) the epipoles, given as ${\underset{\_}{\mathit{e}}}_{s_j}={\left[\begin{array}{ccc}{e}_{s_j}^1& e_{s_j}^2& e_{s_j}^3\end{array}\right]}^T$ in homogeneous coordinates, are located at the intersection of the baseline and the x–y plane of the ideal sensor CSs.

For 2D coordinates in the ideal sensor CSs of the first, ${\widehat{\underset{\_}{\mathit{x}}}}_{s_1}$, and second cameras, ${\widehat{\underset{\_}{\mathit{x}}}}_{s_2}$, to correspond to the same ${\widehat{\mathit{x}}}_w$ requires that these coordinates satisfy the epipolar constraint defined as

$${{\underset{¯}{\widehat{\mathit{x}}}}_{s_2}}^T\mathit{B}{\widehat{\underset{\_}{\mathit{x}}}}_{s_1}=0.$$

(24)

More specifically, since the fundamental matrix, $\mathit{B}$, maps a point in one ideal sensor CS to its corresponding epipolar line in the other ideal sensor CS, the 2D coordinates must lie on the epipolar lines of their respective ideal sensor CSs. $\mathit{B}$ encapsulates the intrinsic and extrinsic relation between the ideal sensor CSs.

Computing $\mathit{B}$ requires applying a homography, $\mathsf{\Omega }$, to the projection matrices of the cameras to transform them to canonical form (${\mathit{Q}}_{c_j}$), where the world CS coincides with the camera CS of the first camera, as [65]

$${\mathit{Q}}_{c_1}={\mathit{Q}}_1\mathsf{\Omega }=\left[\mathit{I}|\mathbf{0}\right]\mathrm{and}{\mathit{Q}}_{c_2}={\mathit{Q}}_2\mathsf{\Omega }=\left[\mathit{M}|\mathit{m}\right]$$

(25)

where

$$\mathsf{\Omega }={\left[\begin{array}{cccc}{Q}_1^{11}& Q_1^{12}& Q_1^{13}& Q_1^{14}\\ Q_1^{21}& Q_1^{22}& Q_1^{23}& Q_1^{24}\\ Q_1^{31}& Q_1^{32}& Q_1^{33}& Q_1^{34}\\ 0& 0& 0& 1\end{array}\right]}^{-1}.$$

(26)

Here $\mathit{I}$ is a $3\times 3$ identity matrix. ${\mathit{Q}}_{c_2}$, which consists of a $3\times 3$ matrix $\mathit{M}$ and a $3\times 1$ vector $\mathit{m}$, transforms a coordinate from the camera CS of the first camera to its location in the ideal sensor CS of the second camera. $\mathit{B}$ is given as

$$\mathit{B}={\left[\mathit{m}\right]}_{\times }\mathit{M},$$

(27)

where ${\left[\mathit{m}\right]}_{\times }$ is the skew-symmetric matrix of $\mathit{m}={\left[\begin{array}{ccc}{m}^1& m^2& m^3\end{array}\right]}^T$ defined as

$${\left[\mathit{m}\right]}_{\times }=\left[\begin{array}{ccc}0& -m^3& m^2\\ m^3& 0& -m^1\\ -m^2& m^1& 0\end{array}\right].$$

(28)

2.5. Stereo-DIC Overview

The core operations of Stereo-DIC are subset matching, calibration, and displacement transformation. For a point on the specimen, subset matching determines the two sets of in-plane, pixel displacements it experiences in each distorted sensor CS which, along with the calibration parameters determined by the calibration process, are used by displacement transformation to determine the in-plane and out-of-plane (thus 3D) metric displacements it experiences in the world CS. Note that with regards to subset matching and displacement transformation, the term in-plane refers to the x–y plane of the distorted sensor CSs and world CS respectively.

Subset matching is composed of stereo and temporal matching which utilize correlation differently. First, subset matching breaks up the first image of the first image series (FIS1) into subsets. Second, stereo matching determines the location of the corresponding subsets in the first image of the second image series (FIS2). Such subsets, one from each image of an image pair which correspond to the same point on the specimen, are termed a subset pair. Lastly, temporal matching, processing each image series separately, determines the displacements experienced by these subsets through their respective image series. This is achieved by sequentially stepping through an image series, starting at the second image, and at each step performing correlation by treating the current image as the deformed image while the reference image is dictated by the reference image strategy as discussed in Section 2.6.1. Thus, stereo matching is unique to Stereo-DIC while temporal matching is identical to that of 2D DIC.

The calibration method is identical to that of 2D DIC. However, in order to determine the epipolar geometry from the calibration parameters requires that the two calibration image series, used to calibrate the cameras separately, form part of the same calibration image set.

Displacement transformation, illustrated in Figure 3, steps through image pairs of the image set similarly to temporal matching. More specifically, considering a single reference subset pair with known locations, ${\mathit{x}}_{s_j}^o$, it uses the in-plane displacements determined by temporal matching ($u_j$ and $v_j$ in the x- and y-directions) between the reference images (of the reference image pair) and the deformed images (of the deformed image pair) to determine the locations of the investigated subset pair, ${\mathit{x}}_{s_j}^d$.

In contrast to 2D DIC, distortion correction alone is insufficient to determine the ideal coordinate of a subset in the ideal sensor CS. More specifically, discretization of light to form an image introduces noise in the images [66] and displacements measured by correlation, which are used to determine the locations of subsets, inherently contain some noise. Thus, correcting ${\mathit{x}}_{s_j}^o$ and ${\mathit{x}}_{s_j}^d$ for radial distortion results in the measured coordinates of the reference, ${\check{\mathit{x}}}_{s_j}^o$, and investigated subset pair, ${\check{\mathit{x}}}_{s_j}^d$, in the ideal sensor CS respectively. Generally, these measured coordinates of a subset pair, represented as ${\check{\mathit{x}}}_{s_j}$, do not satisfy the epipolar constraint of Equation (24) since their back-projection rays do not intersect. Thus, triangulation, which determines the 3D coordinate in the world CS, ${\widehat{\mathit{x}}}_w$, from ${\check{\mathit{x}}}_{s_j}$, is performed in two steps. First the polynomial triangulation method, detailed in Section 2.7, determines the ideal coordinates of a subset pair, ${\widehat{\mathit{x}}}_{s_j}$, which lie as close to ${\check{\mathit{x}}}_{s_j}$ as possible while satisfying the epipolar constraint. Thereafter the linear triangulation method, detailed in Section 2.8, determines ${\widehat{\mathit{x}}}_w$ to which ${\widehat{\mathit{x}}}_{s_j}$ correspond as the intersection of their back-projected rays. Performing triangulation on ${\check{\mathit{x}}}_{s_j}^o$ and ${\check{\mathit{x}}}_{s_j}^d$ results in ${\widehat{\mathit{x}}}_w^o$ and ${\widehat{\mathit{x}}}_w^d$ respectively.

Finally, ${\widehat{\mathit{x}}}_w^o$ is subtracted from ${\widehat{\mathit{x}}}_w^d$ to determine the displacement experienced by the point on the specimen, in the x-, y-, and z-directions represented as ${\widehat{u}}_w$, ${\widehat{v}}_w,$ and ${\widehat{w}}_w$ respectively, between the time stamps at which the reference and deformed image pairs were captured. Note that subscript $w$ indicates coordinates or displacements in the world CS, whereas variable $w$ indicates displacement in the z-direction. This is done for all subset pairs across all image pair combinations, analyzed by temporal matching, to determine the full-field metric displacements experienced by the specimen throughout the image set.

2.6. Subset Matching

The optical flow determined by temporal and stereo matching is a result of the displacement and/or deformation experienced by the specimen (over time) and the perspective change between the cameras respectively. They differ predominantly in the method used to obtain SFP initial estimates.

2.6.1. Temporal Matching

There are two reference image strategies for temporal matching: absolute and incremental. The absolute and incremental strategies define the reference image as either the first image of the image series or the previous image, relative to the current deformed image, respectively. The incremental strategy more reliably tracks more severe deformations between images but incurs accumulative errors when determining the displacement relative to the first image.

Temporal matching uses the Phase Correlation Method (PCM) to determine initial estimates for the displacements between two subsets. PCM efficiently computes the correlation coefficients in the frequency domain for a range of integer displacements of up to half the subset size in each direction between the two subsets. The integer displacements with the best correlation coefficient are identified and used as initial estimates for the displacement SFPs while the deformation SFPs are set to zero. Consult the work of Foroosh et al. [67] for a discussion on PCM. For the absolute strategy, PCM is only used for the first correlation run. Thereafter the SFPs of the previous correlation run, of the same image series, are used as an initial estimate for the SFPs of the current correlation run. The incremental strategy uses PCM for each correlation run.

2.6.2. Stereo Matching

Determining reliable out-of-plane displacements in the world CS requires an angle between the optical axes of the cameras of between 15° and 35° [5,68]. As such, stereo matching uses the second-order SF since subsets experience complex deformation between the FIS1 and FIS2 due to this perspective change [41].

Traditionally, template matching, which utilizes image rectification, was used to determine SFP initial estimates for stereo matching [69]. However, image rectification is susceptible to errors in the calibration parameters, can cause aliasing in the rectified images and interpolation during image rectification can lead to errors in the determined SFPs [70].

As such, feature matching methods are becoming favored [41]. ADIC3D’s feature matching method is based on that proposed by Zhou et al. [71]. It uses the scale-invariant feature transform (SIFT) feature matching algorithm [72] to identify corresponding keypoints between the images (where a keypoint is a unique feature in an image) and the affine transformation mapping model to mathematically relate the coordinates of these keypoint pairs between the images.

The SIFT algorithm is used because it is robust in translation, rotation, image scaling, variation in illumination, affine transformation and moderate perspective change. SIFT identifies keypoints that are scale invariant and computes a descriptor, in the form of a 128-dimensional vector, for each keypoint based on the light intensity gradient information in the vicinity of the keypoint.

For each keypoint of the FIS1, the Euclidean distance between its descriptor and that of every keypoint of the FIS2 is determined. The two keypoints of the FIS2 with the smallest descriptor distances are identified. If the ratio of the smallest to second smallest descriptor distance is less than 0.8, the keypoint of the FIS2 with the smallest descriptor distance is designated as the matching keypoint of the keypoint of the FIS1 [72]. Otherwise, this keypoint of the FIS1 is discarded.

The affine transformation relates a keypoint pair as

$$\left[\begin{array}{c}{x}_{s_2}^k\\ y_{s_2}^k\end{array}\right]=\left[\begin{array}{ccc}1+a_1& a_2& a_3\\ a_4& 1+a_5& a_6\end{array}\right]\left[\begin{array}{c}{x}_{s_1}^k\\ y_{s_1}^k\\ 1\end{array}\right].$$

(29)

Here ${\mathit{x}}_{s_1}^k={\left[\begin{array}{cc}{x}_{s_1}^k& y_{s_1}^k\end{array}\right]}^T$ and ${\mathit{x}}_{s_2}^k={\left[\begin{array}{cc}{x}_{s_2}^k& y_{s_2}^k\end{array}\right]}^T$ are the locations of the $k$^th keypoint in the FIS1 and FIS2 respectively while $a_1$ through $a_6$ are the affine transformation parameters. Given three or more keypoint pairs, the affine transformation parameters can be solved either as a system of linear equations or through linear least-squares respectively.

The $K$ many ${\mathit{x}}_{s_1}^k$ that are nearest to the subset’s center position are used to determine the affine transformation parameters of the subset. However, since SIFT often returns false keypoint pairs, the m-estimator sample consensus (MSAC) method [73] is used to remove these false keypoint pairs when determining the affine transformation parameters. False keypoint pairs are identified as those which have a squared error distance greater than a predefined threshold, $\tau$. The squared error distance of the $k$^th keypoint pair, $E_k^{dist}$, is calculated as

$$E_k^{dist}={\left(x_{s_2}^k-\left(1+a_1\right)x_{s_1}^k-a_2{y}_{s_1}^k-a_3\right)}^2+{\left(y_{s_2}^k-a_4{x}_{s_1}^k-\left(1+a_5\right)y_{s_1}^k-a_6\right)}^2.$$

(30)

ADIC3D uses $K=20$, $\tau =1$ pixel² and a confidence, specifying the probability that MSAC finds the maximum number of inliers, of 99.5%. Second-order SFP initial estimates are determined from the affine transformation parameters and the subset position in the FIS1, ${\mathit{x}}_{s_1}={\left[\begin{array}{cc}{x}_{s_1}& y_{s_1}\end{array}\right]}^T$, as

$$\begin{array}{cc}u=a_1{x}_{s_1}+a_2{y}_{s_1}+a_3,& v=a_4{x}_{s_1}+a_5{y}_{s_1}+a_6,\\ u_x=a_1,& v_x=a_4,\\ u_y=a_2,& v_y=a_5,\\ u_{xx}=0,& v_{xx}=0,\\ u_{xy}=0,& v_{xy}=0,\\ u_{yy}=0& \mathrm{and}{v}_{yy}=0.\end{array}$$

(31)

Once correlation has optimized these SFPs, the corresponding subset position in the FIS2, ${\mathit{x}}_{s_2}={\left[\begin{array}{cc}{x}_{s_2}& y_{s_2}\end{array}\right]}^T$, is determined as

$$\left[\begin{array}{c}{x}_{s_2}\\ y_{s_2}\end{array}\right]=\left[\begin{array}{c}{x}_{s_1}\\ y_{s_1}\end{array}\right]+\left[\begin{array}{c}u\\ v\end{array}\right].$$

(32)

2.7. Polynomial Triangulation Method

The polynomial triangulation method, proposed by Hartley and Sturm [66], determines ${\widehat{\mathit{x}}}_{s_j}$ by minimizing the geometric error cost function, $D$, given as

$$D=\delta {\left({\check{\mathit{x}}}_{s_1},{\widehat{\mathit{x}}}_{s_1}\right)}^2+\delta {\left({\check{\mathit{x}}}_{s_2},{\widehat{\mathit{x}}}_{s_2}\right)}^2\mathrm{subject}\mathrm{to}{{\underset{¯}{\widehat{\mathit{x}}}}_{s_2}}^T\mathit{B}{\widehat{\underset{\_}{\mathit{x}}}}_{s_1}=0.$$

(33)

Here function $\delta$ determines the Euclidean distance between two points. Determining ${\widehat{\mathit{x}}}_{s_j}$ by minimizing the distance between the coordinates ${\widehat{\mathit{x}}}_{s_j}$ and ${\check{\mathit{x}}}_{s_j}$ within the ideal sensor CSs ensures that the polynomial triangulation method is projective-invariant. Projective-invariance implies that the projective frame within which ${\widehat{\mathit{x}}}_w$ is defined does not affect the values of ${\widehat{\mathit{x}}}_{s_j}$ [65]. The polynomial method assumes that the calibration parameters are known with greater accuracy than the measured coordinates [66]. Furthermore, it is assumed that neither ${\widehat{\mathit{x}}}_{s_j}$ nor ${\check{\mathit{x}}}_{s_j}$ coincides with an epipole since this would lead to an unfavorable solution for the 3D coordinate.

It is known that a pair of ideal coordinates which satisfy the epipolar constraint lie on a pair of corresponding epipolar lines. Additionally, the epipolar plane can only rotate about the baseline since the camera CSs, and thus epipoles, are fixed. Thus, there is a family of epipolar lines upon which the idea coordinates can lie. Furthermore, the projective-invariance of the polynomial triangulation method allows applying a separate rigid transformation to each ideal sensor CS in order to parameterize the family of epipolar lines in terms of a single variable $t$. As such, homogeneous coordinates are utilized and Equation (32) becomes

$$D=\delta {\left({\check{\underset{\_}{\mathit{x}}}}_{s_1},{\underset{\_}{\mathit{\lambda }}}_1\left(t\right)\right)}^2+\delta {\left({\check{\underset{\_}{\mathit{x}}}}_{s_2},{\underset{\_}{\mathit{\lambda }}}_2\left(t\right)\right)}^2,$$

(34)

where ${\underset{\_}{\mathit{\lambda }}}_j\left(t\right)$ is a vector representing an epipolar line in the ideal sensor CS as a function of $t$ while $\delta \left({\check{\underset{\_}{\mathit{x}}}}_{s_j},{\underset{\_}{\mathit{\lambda }}}_j\left(t\right)\right)$ represents the distance from ${\check{\underset{\_}{\mathit{x}}}}_{s_j}$ to its orthogonal projection on ${\underset{\_}{\mathit{\lambda }}}_j\left(t\right)$. The rigid transformations first apply a translation matrix, ${\mathit{T}}_j^{pm}$, to each ideal sensor CS translating ${\check{\underset{\_}{\mathit{x}}}}_{s_j}$ to the origin of its CS.

$${\mathit{T}}_1^{pm}=\left[\begin{array}{ccc}1& 0& -{\check{x}}_{s_1}\\ 0& 1& -{\check{y}}_{s_1}\\ 0& 0& 1\end{array}\right]\mathrm{and}{\mathit{T}}_2^{pm}=\left[\begin{array}{ccc}1& 0& -{\check{x}}_{s_2}\\ 0& 1& -{\check{y}}_{s_2}\\ 0& 0& 1\end{array}\right].$$

(35)

The original fundamental matrix, $\mathit{B}$, determined using Equation (27) needs to be updated as

$${\mathit{B}}_1={{\mathit{T}}_2^{pm}}^{-T}\mathit{B}{{\mathit{T}}_1^{pm}}^{-1}.$$

(36)

The epipoles ${\underset{\_}{\mathit{e}}}_{s_1}$ and ${\underset{\_}{\mathit{e}}}_{s_2}$ of the first and second cameras are computed, using singular value decomposition, as the right and left null-space of ${\mathit{B}}_1$ respectively. Thereafter they are normalized such that ${\left(e_{s_1}^1\right)}^2+{\left(e_{s_1}^2\right)}^2=1$ and ${\left(e_{s_2}^1\right)}^2+{\left(e_{s_2}^2\right)}^2=1$.

The second step of the rigid transformation applies rotations ${\mathit{R}}_1^{pm}$ and ${\mathit{R}}_2^{pm}$ to the ideal sensor CSs of the first and second cameras respectively in order to place their epipoles on the x-axes of their respective CSs as ${\mathit{R}}_1^{pm}{\underset{\_}{\mathit{e}}}_{s_1}={\left[\begin{array}{ccc}1& 0& e_{s_1}^3\end{array}\right]}^T$ and ${\mathit{R}}_2^{pm}{\underset{\_}{\mathit{e}}}_{s_2}={\left[\begin{array}{ccc}1& 0& e_{s_2}^3\end{array}\right]}^T$.

$${\mathit{R}}_1^{pm}=\left[\begin{array}{ccc}{e}_{s_1}^1& e_{s_1}^2& 0\\ -e_{s_1}^2& e_{s_1}^1& 0\\ 0& 0& 1\end{array}\right]\mathrm{and}{\mathit{R}}_2^{pm}=\left[\begin{array}{ccc}{e}_{s_2}^1& e_{s_2}^2& 0\\ -e_{s_2}^2& e_{s_2}^1& 0\\ 0& 0& 1\end{array}\right].$$

(37)

The fundamental matrix is updated as

$${\mathit{B}}_2={\mathit{R}}_2^{pm}{\mathit{B}}_1{{\mathit{R}}_1^{pm}}^T.$$

(38)

Since ${\mathit{B}}_2{\left[\begin{array}{ccc}1& 0& e_{s_1}^3\end{array}\right]}^T=\left[\begin{array}{ccc}1& 0& e_{s_2}^3\end{array}\right]{\mathit{B}}_2={\left[\begin{array}{ccc}0& 0& 0\end{array}\right]}^T$ the elements of the fundamental matrix are represented as

$${\mathit{B}}_2=\left[\begin{array}{ccc}{e}_{s_1}^3{e}_{s_2}^3{\phi }_4& -e_{s_2}^3{\phi }_3& -e_{s_2}^3{\phi }_4\\ -e_{s_1}^3{\phi }_2& {\phi }_1& {\phi }_2\\ -e_{s_1}^3{\phi }_4& {\phi }_3& {\phi }_4\end{array}\right].$$

(39)

An epipolar line of the first camera, ${\underset{\_}{\mathit{\lambda }}}_1\left(t\right)$, passing through point ${\left[\begin{array}{ccc}0& t& 1\end{array}\right]}^T$ on the y-axis of the ideal sensor CS is represented as

$${\underset{\_}{\mathit{\lambda }}}_1\left(t\right)={\left[\begin{array}{ccc}0& t& 1\end{array}\right]}^T\times {\left[\begin{array}{ccc}1& 0& e_{s_1}^3\end{array}\right]}^T={\left[\begin{array}{ccc}t{e}_{s_1}^3& 1& -t\end{array}\right]}^T.$$

(40)

The corresponding epipolar line of the second camera, ${\underset{\_}{\mathit{\lambda }}}_2\left(t\right)$, is computed as

$${\underset{\_}{\mathit{\lambda }}}_2\left(t\right)={\mathit{B}}_2{\left[\begin{array}{ccc}0& t& 1\end{array}\right]}^T={\left[\begin{array}{ccc}-e_{s_2}^3\left({\phi }_{3}t+{\phi }_4\right)& {\phi }_{1}t+{\phi }_2& {\phi }_{3}t+{\phi }_4\end{array}\right]}^T.$$

(41)

Thus, $\delta {\left({\check{\underset{\_}{\mathit{x}}}}_{s_1},{\underset{\_}{\mathit{\lambda }}}_1\left(t\right)\right)}^2=\frac{t^2}{1+{\left(t{e}_{s_1}^3\right)}^2}$ and $\delta {\left({\check{\underset{\_}{\mathit{x}}}}_{s_2},{\underset{\_}{\mathit{\lambda }}}_2\left(t\right)\right)}^2=\frac{{\left({\phi }_{3}t+{\phi }_4\right)}^2}{{\left({\phi }_{1}t+{\phi }_2\right)}^2+{e_{s_2}^3}^2{\left({\phi }_{3}t+{\phi }_4\right)}^2}$ and so Equation (34) has the form

$$D\left(t\right)=\frac{t^2}{1+{\left(t{e}_{s_1}^3\right)}^2}\frac{{\left({\phi }_{3}t+{\phi }_4\right)}^2}{{\left({\phi }_{1}t+{\phi }_2\right)}^2+{e_{s_2}^3}^2{\left({\phi }_{3}t+{\phi }_4\right)}^2}.$$

(42)

Finding the optimal value of $t$ involves taking the derivative Equation (42) in terms of $t$, collecting terms over a common denominator and equating the resulting numerator to 0 as

$$t{\left({\left({\phi }_{1}t+{\phi }_2\right)}^2+{e_{s_2}^3}^2{\left({\phi }_{3}t+{\phi }_4\right)}^2\right)}^2-\left({\phi }_1{\phi }_4-{\phi }_2{\phi }_3\right){\left(1+{\left(t{e}_{s_1}^3\right)}^2\right)}^2\left({\phi }_{1}t+{\phi }_2\right)\left({\phi }_{3}t+{\phi }_4\right)=0.$$

(43)

The 6 roots of Equation (43) are solved for and Equation (42) is evaluated at the real parts of these roots to determine the value of $t$ which corresponds to a minimum. The asymptotic value of Equation (42) should also be evaluated as $t\to \infty$ which corresponds to an epipolar line in the first image of $e_{s_1}^3{\check{x}}_{s_1}=1$ [65].

The optimal $t$ is used to determine ${\widehat{\mathit{x}}}_{s_1}$ and ${\widehat{\mathit{x}}}_{s_2}$ as

$${\widehat{\mathit{x}}}_{s_1}=\mathsf{\Phi }\left({{\mathit{T}}_1^{pm}}^{-1}{{\mathit{R}}_1^{pm}}^T\left[\begin{array}{c}{t}^2{e}_{s_1}^3\\ t\\ {\left(t{e}_{s_1}^3\right)}^2+1\end{array}\right]\right)$$

(44)

$$\mathrm{and} {\widehat{\mathit{x}}}_{s_2}=\mathsf{\Phi }\left({{\mathit{T}}_2^{pm}}^{-1}{{\mathit{R}}_2^{pm}}^T\left[\begin{array}{c}{e}_{s_2}^3{\left({\phi }_{3}t+{\phi }_4\right)}^2\\ -\left({\phi }_{1}t+{\phi }_2\right)\left({\phi }_{3}t+{\phi }_4\right)\\ {\left({\phi }_{1}t+{\phi }_2\right)}^2+{e_{s_2}^3}^2{\left({\phi }_{3}t+{\phi }_4\right)}^2\end{array}\right]\right).$$

(45)

Note that the inverse of the rigid transformations are applied during Equations (44) and (45) such that ${\widehat{\mathit{x}}}_{s_1}$ and ${\widehat{\mathit{x}}}_{s_2}$ are in the original ideal sensor CSs.

2.8. Linear Triangulation Method

Denoting the $n$^th row of ${\mathit{Q}}_{c_j}$ as ${\mathit{q}}_{c_j}^n$ the projection of ${\widehat{\mathit{x}}}_w$ to ${\widehat{\mathit{x}}}_{s_1}$ is described by the following three equations

$${\alpha }_1{\widehat{x}}_{s_1}={{\mathit{q}}_{c_1}^1}^T{\widehat{\mathit{x}}}_w,{\alpha }_1{\widehat{y}}_{s_1}={{\mathit{q}}_{c_1}^2}^T{\widehat{\mathit{x}}}_w\mathrm{and}{\alpha }_1={{\mathit{q}}_{c_1}^3}^T{\widehat{\mathit{x}}}_w.$$

(46)

Similarly, for ${\widehat{\mathit{x}}}_{s_2}$ this is given as

$${\alpha }_2{\widehat{x}}_{s_2}={{\mathit{q}}_{c_2}^1}^T{\widehat{\mathit{x}}}_w,{\alpha }_2{\widehat{y}}_{s_2}={{\mathit{q}}_{c_2}^2}^T{\widehat{\mathit{x}}}_w\mathrm{and}{\alpha }_2={{\mathit{q}}_{c_2}^3}^T{\widehat{\mathit{x}}}_w.$$

(47)

Using the third relation of Equations (46) and (47) to remove the scaling variables, ${\alpha }_1$ and ${\alpha }_2$, the following relation is obtained.

$$\left[\begin{array}{c}{\widehat{x}}_{s_1}{{\mathit{q}}_{c_1}^3}^T-{{\mathit{q}}_{c_1}^1}^T\\ {\widehat{y}}_{s_1}{{\mathit{q}}_{c_1}^3}^T-{{\mathit{q}}_{c_1}^2}^T\\ {\widehat{x}}_{s_2}{{\mathit{q}}_{c_2}^3}^T-{{\mathit{q}}_{c_2}^1}^T\\ {\widehat{y}}_{s_2}{{\mathit{q}}_{c_2}^3}^T-{{\mathit{q}}_{c_2}^2}^T\end{array}\right]{\widehat{\mathit{x}}}_w=\mathit{S}{\widehat{\mathit{x}}}_w=0$$

(48)

${\widehat{\mathit{x}}}_w$ is computed, using singular value decomposition, as the unit eigenvector of ${\mathit{S}}^T\mathit{S}$ associated with the smallest eigenvalue [65].

2.9. Displacement Transformation

Consider a single subset pair and its two sets of in-plane displacements determined by temporal matching between the reference and deformed image pairs under consideration. First ${\mathit{x}}_{s_j}^d={\left[\begin{array}{cc}{x}_{s_j}^d& y_{s_j}^d\end{array}\right]}^T$ is calculated based on ${\mathit{x}}_{s_j}^o={\left[\begin{array}{cc}{x}_{s_j}^o& y_{s_j}^o\end{array}\right]}^T$ as

$$\left[\begin{array}{c}{x}_{s_j}^d\\ y_{s_j}^d\end{array}\right]=\left[\begin{array}{c}{x}_{s_j}^o\\ y_{s_j}^o\end{array}\right]+\left[\begin{array}{c}{u}_j\\ v_j\end{array}\right].$$

(49)

Thereafter ${\mathit{x}}_{s_j}^o$ and ${\mathit{x}}_{s_j}^d$ are undistorted to obtain ${\check{\mathit{x}}}_{s_j}^o={\left[\begin{array}{cc}{\check{x}}_{s_j}^o& {\check{y}}_{s_j}^o\end{array}\right]}^T$ and ${\check{\mathit{x}}}_{s_j}^d={\left[\begin{array}{cc}{\check{x}}_{s_j}^d& {\check{y}}_{s_j}^d\end{array}\right]}^T$ respectively. This is performed using non-linear, least-squares optimization since an exact solution for the inverse of Equation (3) does not exist because it requires determining the roots of a polynomial of degree greater than four [74]. Triangulation, represented by function $\mathsf{\Psi }$, determines the 3D coordinates in the world CS to which the reference subset pair, ${\widehat{\mathit{x}}}_w^o={\left[\begin{array}{ccc}{\widehat{x}}_w^o& {\widehat{y}}_w^o& {\widehat{z}}_w^o\end{array}\right]}^T$, and investigated subset pair, ${\widehat{\mathit{x}}}_w^d={\left[\begin{array}{ccc}{\widehat{x}}_w^d& {\widehat{y}}_w^d& {\widehat{z}}_w^d\end{array}\right]}^T$, correspond as

$${\widehat{\mathit{x}}}_w^o=\mathsf{\Psi }\left({\check{\mathit{x}}}_{s_1}^o,{\check{\mathit{x}}}_{s_2}^o\right)\mathrm{and}{\widehat{\mathit{x}}}_w^d=\mathsf{\Psi }\left({\check{\mathit{x}}}_{s_1}^d,{\check{\mathit{x}}}_{s_2}^d\right).$$

(50)

Finally, the displacements ${\widehat{u}}_w$, ${\widehat{v}}_w,$ and ${\widehat{w}}_w$ are determined as

$$\left[\begin{array}{c}{\widehat{u}}_w\\ {\widehat{v}}_w\\ {\widehat{w}}_w\end{array}\right]=\left[\begin{array}{c}{\widehat{x}}_w^d\\ {\widehat{y}}_w^d\\ {\widehat{z}}_w^d\end{array}\right]-\left[\begin{array}{c}{\widehat{x}}_w^o\\ {\widehat{y}}_w^o\\ {\widehat{z}}_w^o\end{array}\right].$$

(51)

3. Implementation

ADIC3D’s framework, presented in Appendix A, is invoked from MATLAB’s command prompt as “ProcData = ADIC3D(FileNames1, FileNames2, Mask, GaussFilt, StepSize, SubSize, SubShape, SFOrder, RefStrat, StopCritVal, WorldCTs, ImgCTs);” with the input variables defined in

Table 1. Required input variables of the ADIC3D framework.

Variable	Variable Description
FileNames1	Cell array of character vectors containing the image file names of the first image series. All images need to be the same size.
FileNames2	Cell array of character vectors containing the image file names of the second image series. All images need to be the same size.
Mask	Logical matrix, which is the same size as the images, indicating which pixels should not be analyzed during correlation.
GaussFilt	Defines the standard deviation and window size for the Gaussian filter in pixels as [FiltSigma, FiltSize] respectively where {FiltSigma ∈ ℝ+|FiltSigma > 0} and {FiltSize ∈ ℕ}.
StepSize	Step size in pixels {StepSize ∈ ℕ}.
SubSize	Subset size in pixels {SubSize = 2k + 1|k ∈ ℕ}.
SubShape	Subset shape {SubShape ∈ 'Square','Circle'}.
SFOrder	Dictates the SF order {SFOrder ∈ ℤ|0 ≤ SFOrder ≤ 2}.
RefStrat	Logical statement dictating reference image strategy (Section 2.6.1).
StopCritVal	Defines the stopping criterion value {StopCritVal ∈ ℝ|StopCritVal > 0}.
WorldCTs	Location of CTs in the world CS defined according to MATLAB’s estimateCameraParameters function.
ImgCTs	Location of CTs in the distorted sensor CSs defined according to MATLAB’s estimateCameraParameters function.

.

ADIC3D returns a structured array ResultData containing four main fields: structured variable Stereo holding stereo-matching data as detailed in

Table 2. Accessing the stereo matching data contained in ResultData.Stereo for subset q.

Variable	Variable Description
P(b,q)	SFPs (b = 1 for $u$ and b = 7 for $v$).
C(q)	ZNCC coefficient.
Iter(q)	Number of iterations until stopping criterion is satisfied (maximum of 100 iterations).
StopVal(q)	Final stopping criterion value for subset q.

; structured arrays ProcData1 and ProcData2, detailed in

Table 3. Accessing the processing data for temporal matching (ResultData.ProcData1(d) and ResultData.ProcData2(d)) for image d and subset number q.

Variable	Variable Description
ImgName	Deformed image name.
ImgSize(b)	Image size (b = 1 for rows and b = 2 for columns).
ImgFilt(b)	Standard deviation (b = 1) and window size (b = 2) for the Gaussian filter respectively in pixels.
SubSize(q)	Subset size in pixels.
SubShape(q,:)	Subset shape.
SFOrder(q)	SF order.
Xos(b,q)	Reference subset position in the distorted sensor CS of the relevant camera (b = 1 for $x^o$ and b = 2 for $y^o$).
P(b,q)	SFPs (b = 1 for $u$ and b = 7 for $v$).
C(q)	ZNCC coefficient.
Iter(q)	Number of iterations until stopping criterion is satisfied (maximum of 100 iterations).
StopVal(q)	Final stopping criterion value.

, holding the processing data for temporal matching of subsets of the first and second image series respectively; and structured array DispTrans containing displacement transformation data detailed in

Table 4. Accessing the displacement transformation data for image d (contained in ResultData.DispTrans(d)) for subset q.

Variable	Variable Description
Xow(b,q)	Reference subset position in the world CS (b = 1 for ${\widehat{x}}_w^o$, b = 2 for ${\widehat{y}}_w^o$, and b = 3 for ${\widehat{z}}_w^o$).
Uw(b,q)	Displacement in the world CS (b = 1 for ${\widehat{u}}_w$, b = 2 for ${\widehat{v}}_w,$ and b = 3 for ${\widehat{w}}_w$).
CamParams	Calibration parameters (extrinsic, intrinsic, and radial distortion parameters).

.

Refer to Appendix B for brief discussion of utilizing parallel processing to reduce computation time of ADIC3D.

3.1. ADIC3D Function

The main function ADIC3D, outlined in

Table 5. ADIC3D algorithm summary.

Line Numbers	Task Performed
Lines 2–3	Compute image names of both image series;
Lines 4–5	Compute number of images and size of the first image in first image series;
Lines 6–7	Create regularly spaced reference subset positions, Xos;
Line 8	Remove subsets containing invalid pixels which are defined by Mask;
Line 9–12	Pre-assign ResultData structure array;
Line 13	Call subroutine StereoMatch to perform stereo matching;
Line 15	Call subroutine ImgCorr to perform temporal matching of first image series;
Line 17	Call subroutine ImgCorr to perform temporal matching of second image series;
Line 18	Call subroutine CSTrans to perform displacement transformation from the distorted sensor CSs to the world CS;

, sets up the DIC problem and calls the appropriate subroutines. The output variables are preassigned in lines 9–12 to save input data and efficiently store computed data. ADIC3D allows subsets to be analyzed in a flexible manner by assigning Xos, SubSize, SubShape, and SFOrder on a per subset and per image basis. However, despite being capable of this, ADIC3D assigns the same SubSize, SubShape, and SFOrder to each subset as is conventional for DIC. Subsets containing invalid pixels, identified by Mask, are eliminated.

The subroutine StereoMatch is called to perform stereo matching. StereoMatch has input variables n (the total number of images in each image series), ResultData, FileNames1, FileNames2, and StopCritVal. It returns the subset positions in the FIS2 stored as ProcData2(d).Xos and stereo matching correlation results P, C, Iter, and StopVal stored in Stereo.

Temporal matching is performed on each image series separately in lines 15 and 17 using the ImgCorr subroutine. ImgCorr’s inputs are n, ProcData1 or ProcData2, FileNames1 or FileNames2, RefStrat and StopCritVal. It returns P, C, Iter, and StopVal for each subset throughout the image series stored in ProcData1 or ProcData2 for the first and second image series respectively.

Finally, subroutine CSTrans determines coordinates and displacements in the world CS based on inputs n, ResultData, WorldCTs, ImgCTs, and RefStrat. CSTrans returns Xow, Uw, and CamParams stored in DispTrans. Note that within the subroutines ResultData is shortened to RD and ProcData is shortened to PD where appropriate.

3.2. Correlation Implementation

Both stereo and temporal matching make use of correlation, detailed in Section 2.3, which is implemented using the three subroutines: SubShapeExtract, SFExpressions, and SubCorr. These subroutines are discussed briefly since they are identical to those used in ADIC2D [53].

SubShapeExtract, detailed in

Table 6. SubShapeExtract algorithm summary.

Line Numbers	Task Performed
Line 2	switch SubShape;
Line 3	caseSubShape = 'Square', do
Line 4–6	Extract $f_i$ and $\nabla {\mathit{f}}_i$ using SubExtract;
Line 7	Compute $\Delta {\mathit{x}}_i$ using SubSize;
Line 8	caseSubShape = 'Circle', do
Line 9–11	Extract $f_i$ and $\nabla {\mathit{f}}_i$ using SubExtract;
Line 12	Compute $\Delta {\mathit{x}}_i$ using SubSize;
Line 13	Determine mask of elements that fall within the circular subset;
Line 14–16	Use mask to extract appropriate data for circular subset;
Line 17	end switch

, determines $f_i$, $\nabla {\mathit{f}}_i$ and $\Delta {\mathit{x}}_i$ of a subset based on inputs SubSize, SubShape, Xos, $F$, $\nabla F,$ and SubExtract. Note that variables with subscript $i$ refer to the full set of this variable for the subset (i.e., $f_i$ refers to $f_i\forall i\in I$) throughout Section 3.2. $\nabla F$ is the light intensity gradient of the reference image whereas SubExtract is an anonymous function which extracts a square subset from a matrix based on the size and position of the subset. For SubShape defined as ‘Circle’, SubSize specifies the diameter.

SFExpressions returns anonymous functions for expressions that are dependent on the SF order, as outlined in

Table 7. SFExpressions algorithm summary.

Line Numbers	Task Performed
Line 2	switch SFOrder
Line 3–8	caseSFOrder = 0, do assign functions for zero-order SF;
Line 9–14	case SFOrder = 1, do assign functions for first-order SF;
Line 15–20	case SFOrder = 2, do assign functions for second-order SF;
Line 21	end switch

. These anonymous functions are as follows: W defines Equation (9); dFdWdP defines $\nabla {\mathit{f}}_i\frac{\partial {\mathit{W}}_i}{\partial \mathit{P}}$; SFPVec2Mat defines Equation (22); Mat2SFPVec extracts the SFPs from SFPVec2Mat; and StopCrit defines Equation (23). ADIC3D allows for the zero, first, or second-order SFs to be used by setting SFOrder to 0, 1, or 2 respectively.

SubCorr performs the core operations of correlation for a subset as outlined in

Table 8. SubCorr algorithm summary.

Line Numbers	Task Performed
Line 2	Call SFExpressions to assign equations dependent on the SF order;
Line 3	Compute $\nabla {\mathit{f}}_i\frac{\partial {\mathit{W}}_i}{\partial \mathit{P}}$;
Line 4	Compute ${\mathit{H}}^{-\mathbf{1}}$, Equation (19);
Line 5	Compute $\overline{f}$ and $\tilde{f}$;
Line 6	Initialise flag ← 0, Iter ← 1 and $\Delta \mathit{P}\leftarrow 1$;
Line 7	whileflag = 0, do
Line 8	Compute $\Delta {\mathit{x}}_i^{\prime },$ Equation (7), using estimates of $\mathit{P}$;
Line 9	Compute $\mathit{g}$ using interpolation coefficients;
Line 10	Compute $\overline{g}$ and $\tilde{g}$;
Line 11	Compute ‖$\mathsf{\Delta }\mathit{P}$‖ using Equation (23);
Line 12	if ‖$\mathsf{\Delta }\mathit{P}$‖ < StopCritVal or Iter ≥ 100, do
Line 13	Set flag ← 1;
Line 14	Compute C, Equation (12) substituted into Equation (15);
Line 15	else, do
Line 16	Compute $\mathit{J}$, Summation expression of Equation (18);
Line 17	Compute $\Delta \mathit{P}$, Equation (18);
Line 18	Update $\mathit{P}$, Equation (21);
Line 19	end if
Line 20	Set Iter ← Iter + 1;
Line 21	end while

. Its inputs are the interpolation coefficients of the deformed image, $f_i$, $\nabla {\mathit{f}}_i$, SubSize, SFOrder, Xos, $\Delta {\mathit{x}}_i$, initial estimates for P and StopCritVal. SubCorr returns P, C, Iter, and StopVal. The size of vector P is consistent with that of the second-order SF. Thus, the elements of ${\mathit{P}}^{SF2}$, not relevant to the specified SF order, remain zero.

SubShapeExtract is called prior to calling SubCorr in order to determine data used by SubCorr while SFExpressions is called within SubCorr to define the appropriate anonymous functions on a subset basis. Thus, the effect of the subset shape and SF order as well as the core mathematical operations of correlation can be considered separately.

3.3. Stereo Matching Implementation

Excluding the subroutines used to perform correlation, stereo matching is implemented using two additional subroutines: FeatureMatch and StereoMatch. FeatureMatch returns SFP initial estimates, determined according to the feature matching method outlined in Section 2.6.2, based on inputs ProcData1, d (the image to be analyzed), F, G, and SubExtract. StereoMatch refines these SFPs through correlation and uses them to determine the subset positions in the FIS2.

3.3.1. StereoMatch Function

StereoMatch, detailed in

Table 9. StereoMatch algorithm summary.

Line Numbers	Task Performed
Line 2	Define SubExtract function to extract square subset data;
Line 3–6	Load the FIS1 and FIS2 and perform Gaussian filtering on them;
Line 7	Compute interpolation coefficients of G using MATLAB’s griddedInterpolant function;
Line 9	Call subroutine FeatureMatch to determine SFP initial estimates;
Line 10	Initialise temporary storage variables used to save correlation information during the for loop;
Line 11	for subset number q = 1 to number of subsets, do
Line 12	if FeatureMatch determined SFP initial estimates for subset q, do
Line 13	Call subroutine SubShapeExtract;
Line 14	Call subroutine SubCorr;
Line 15	end if
Line 16	end for
Line 17	Save correlation information to RD.Stereo variable;
Line 18	for image number d = 1 to d = n, do
Line 19	Compute the subset positions in the FIS2 using Equation (32);
Line 20	end for
Line 21–22	Determine subsets which fail stereo matching and subsets which pass;
Line 23–26	Set the subset position of subsets which fail stereo matching to NaNs throughout both image series;
Line 27	Display information for stereo matching;

, loads the FIS1 and FIS2 as F and G, respectively, and performs Gaussian filtering on these using MATLAB’s imgaussfilt function. Lines 4 and 6 can be changed to use alternative filtering methods. Bi-cubic b-spline interpolation coefficients are determined for the deformed image using MATLAB’s griddedInterpolant function. In line 7 ‘spline’ can be replaced with ‘linear’ or ‘cubic’ for bi-linear or bi-cubic polynomial interpolation respectively. Moreover, line 7 can be replaced with an alternative interpolation method such as MATLAB’s spapi function. Line 9 of SubCorr may need to be updated if alternative interpolation methods are employed.

Lines 11 to 16 cycle through each subset of the FIS1, for which FeatureMatch in line 9 determined SFP initial estimates, performing correlation using subroutines SubShapeExtract and SubCorr. These displacement SFPs are used to determine the reference subset positions in the second image series.

Subset pairs which achieve a ZNCC coefficient below 0.6, or which fall outside the bounds of the FIS2, are assumed to have failed stereo matching. These subsets are determined in lines 21–22 and their subset positions in both image series are set to NaN (not a number) in lines 23–26 such that they are not analyzed during temporal matching or displacement transformation.

StereoMatch returns ResultData with the subset positions in the FIS2 assigned to ProcData2(d).Xos; the correlation results for stereo matching (P, C, Iter, StopVal) are assigned to Stereo.

3.3.2. FeatureMatch Function

FeatureMatch, outlined in

Table 10. FeatureMatch algorithm summary.

Line Numbers	Task Performed
Line 2	if VLFeat library is not setup, then return an error;
Line 3–4	Compute keypoint locations and descriptors for F and G using vl_sift;
Line 5	Determine vector KptsInVacinity identifying the keypoints of F which fall within the perimeter of the square subsets, equivalent in size to that specified for the subset under consideration, of F;
Line 6	Eliminate keypoints (and their associated descriptors) of F which do not fall within the perimeter of any of the subsets;
Line 7–8	Determine matching keypoints using vl_ubcmatch;
Line 9	Determine the 20 nearest keypoints for each subset stored in matrix relevantKpts;
Line 10–11	Define anonymous functions RansacModel and RansacError for determining affine transformation parameters (of Equation (29)) and evaluating Equation (30) respectively;
Line 12	Initialise P as NaNs;
Line 13	for subset number q = 1 to number of subsets, do
Line 14	try
Line 15	Use MSAC to determine affine transformation parameters for subset q from its relevant keypoints;
Line 16	Convert affine transformation parameters to second-order SFPs using Equation (31);
Line 17	end try
Line 18	end for
Line 19	Display information for SIFT feature matching;

, implements SIFT feature matching using codes available from the VLFeat open-source library of computer vision algorithms. More specifically, VLFeat’s vl_sift function is used to determine the keypoint locations and descriptors of F and G in lines 3–4. The keypoints of F which fall within the perimeter of at least one subset (square subsets equivalent in size to that specified for the subset under consideration are used) are identified in line 5. Only these keypoints of F are retained, in line 6, in order to reduce the computation time of VLFeat’s vl_ubcmatch function in determining matching keypoint pairs between F and G in lines 7–8.

For each subset, the 20 closest keypoints to its center position are identified in line 9. The for loop of line 13 cycles through the subsets using MSAC to determine reliable affine transformation parameters, which are used by Equation (31) to compute the corresponding second-order SFPs.

MSAC is implemented using MATLAB’s ransac function. Two anonymous functions are used in the call to ransac. RansacModel, defined in line 10, determines the affine transformation parameters from the keypoint locations by solving Equation (29). RansacError, defined in line 11, evaluates Equation (30) to determine each keypoint’s squared error distance (for keypoints identified to be relevant to the subset). The fifth argument of ransac sets $\tau =1$ pixel² and the final argument specifies the confidence that the maximum number of inliers are determined. The fifth and sixth arguments of ransac, as well as the number of nearest keypoints used for each subset (line 9), can be altered to suit the reader’s needs. The try statement of line 14 catches instances where ransac fails to identify three keypoint inliers. As such, the SFP initial estimates of these subsets, stored in P, remain NaN as initialized on line 12.

3.4. Temporal Matching Implementation

Excluding the subroutines used to perform correlation, temporal matching is implemented using PCM and ImgCorr. ADIC3D’s implementation of temporal matching is discussed briefly since it is almost identical to that of ADIC2D [53] with the exception that PCM and ImgCorr are modified to avoid analyzing subsets which fail stereo matching.

3.4.1. PCM Function

PCM determines an initial estimate of the displacement SFPs of a subset based on inputs F, G, SubSize, the subsets position, and SubExtract as outlined in

Table 11. PCM algorithm summary.

Line Numbers	Task Performed
Line 2	if subset positions do not contain NaNs, do
Line 3	Compute normalized cross-power spectrum in the frequency domain;
Line 4	Convert back to spatial domain;
Line 5	Find index of the maximum correlation coefficient;
Line 6	Compute index vector which relates indices of the correlation coefficient matrix to the displacements they correspond to;
Line 7–8	Obtain displacements using index of the maximum correlation coefficient;
Line 9	else, do
Line 10	Set output displacements to NaN;
Line 11	end if

. Subset positions are passed as separate x and y positions as required by MATLAB’s arrayfun function which is used to call PCM. The if statement of line 2 avoids analyzing subsets which contain NaN positions and returns NaN displacements in this case.

3.4.2. ImgCorr Function

ImgCorr, outlined in

Table 12. ImgCorr algorithm summary.

Line Numbers	Task Performed
Line 2	Define SubExtract function to extract square subset data;
Line 3	for image number d = 2 to d = n, do
Line 4	Define $G$;
Line 5	Perform Gaussian filtering on $G$ using MATLAB’s imgaussfilt function;
Line 6	Compute interpolation coefficients of $G$ using MATLAB’s griddedInterpolant function;
Line 7	if first image of correlation run or RefStrat = 1, do
Line 8	Define $F$;
Line 9	Perform Gaussian filtering on $F$ using MATLAB’s imgaussfilt function;
Line 10	Compute gradients for $F$ (compute $\nabla F$);
Lines 11–12	Displace Xos with previous image correlation run displacement SFPs (incremental strategy);
Line 13	Call subroutine PCM to compute initial estimates of displacement SFPs;
Line 14	else, do
Line 15	Set P(d) ← P(d-1);
Line 16	end if
Line 17	Initialise temporary storage variables used to save correlation information during the inner loop;
Line 18	for subset number q = 1 to number of subsets, do
Line 19	if subset q passed stereo matching, do
Line 20	Call subroutine SubShapeExtract;
Line 21	Call subroutine SubCorr;
Line 22	end if
Line 23	end for
Line 24	Save correlation information to PD variable;
Line 25–26	Display results for image d correlation;
Line 27	end for

, makes use of a nested for loop to perform temporal matching. The outer loop cycles through the images of an image series, starting at the second image, preparing data for correlation. This includes loading the appropriate reference and deformed images; filtering the images; determining the interpolation coefficients of $G$; determining the reference image gradient; shifting the reference subset positions by the displacement SFPs determined in the previous correlation run (these displacement SFPs are rounded down such that the reference subset need not be interpolated [75]); and using PCM to determine initial estimates of the SFPs for the current correlation run. In the case of the absolute reference image strategy, the reference subset positions are not shifted and PCM is only used to obtain an initial estimate of the displacement SFPs for the first correlation run. For subsequent iterations, the SFPs of the previous correlation run are used as initial estimates.

The inner for loop cycles through the subsets calling SubShapeExtract and SubCorr to perform correlation of the subsets. The if statement of line 19 ensures that only subsets which pass stereo matching are analyzed within the for loop.

3.5. Displacement Transformation Implementation

Displacement transformation is implemented using subroutines CSTrans and Triangulation.

3.5.1. CSTrans Function

CSTrans, outlined in

Table 13. CSTrans algorithm summary.

Line Numbers	Task Performed
Line 2	Compute calibration parameters using MATLAB’s estimateCameraParameters function;
Line 3–4	Compute canonical projection matrices of first and second cameras;
Line 5	Load the fundamental matrix;
Line 6	for image number d = 1 to d = n, do
Line 7–8	Compute ${\mathit{x}}_{s_1}^d$ and ${\mathit{x}}_{s_2}^d$, Equation (49);
Line 9	Determine indices of subsets which do not contain NaNs in either ${\mathit{x}}_{s_1}^d$ or ${\mathit{x}}_{s_2}^d$;
Line 10	if first image pair of image set or RefStrat = 1, do
Line 11	Compute ${\widehat{\mathit{x}}}_w^o$ (for subsets identified in line 9) using the Triangulation subroutine and MATLAB’s undistortPoints function;
Line 12	else
Line 13	Assign ${\widehat{\mathit{x}}}_w^o$ of previous image pair to current image pair;
Line 14	end if
Line 15	Compute ${\widehat{\mathit{x}}}_w^d$ (for subsets identified in line 9), using the Triangulation subroutine and MATLAB’s undistortPoints function, and subtract ${\widehat{\mathit{x}}}_w^o$ from it to determine ${\widehat{u}}_w$, ${\widehat{v}}_w$ and ${\widehat{w}}_w$ using Equation (51);
Line 16	Save calibration parameters;
Line 18	end for

, uses MATLAB’s estimateCameraParameters function to determine the calibration parameters according to Section 2.2 and the fundamental matrix according to Section 2.4. The canonical form of the projection matrices are used such that the world CS is aligned with the camera CS of the first camera.

Lines 6–18 cycle through image pairs of the image set performing displacement transformation according to Section 2.9. Line 9 identifies subsets which do not contain NaN values in either their position or SFPs (subsets which pass stereo matching) which are to be analyzed during displacement transformation. MATLAB’s undistortPoints function is used to remove distortion from the subset positions while the subroutine Triangulation uses polynomial and linear triangulation to determine the 3D coordinate in the world CS corresponding to a subset pair. The if statement of line 10 avoids recomputing the reference subset positions for the absolute reference image strategy.

CSTrans is susceptible to particular issues which can cause a fatal error. Refer to Appendix C for a discussion of a function which improves the robustness of CSTrans.

3.5.2. Triangulation Function

Triangulation, outlined in

Table 14. Triangulation algorithm summary.

Line Numbers	Task Performed
Line 2	for subset pair number q = 1 to number of subsets, do
Lines 3–4	Determine translation matrices of Equation (35);
Line 5	Apply translation matrices to fundamental matrix according to Equation (36);
Lines 6–8	Compute epipoles and normalize them;
Lines 9–10	Compute rotation matrices according to Equation (37);
Line 11	Apply rotation matrices to fundamental matrix according to Equation (38);
Lines 12–13	Compute polynomial coefficients of Equation (43);
Lines 14–15	Compute real roots of the polynomial;
Line 16	Evaluate polynomial of Equation (42) at these roots;
Line 17	Determine root corresponding to minimum of Equation (42);
Lines 18–20	Compute ideal subset positions according to Equations (44) and (45);
Lines 21–22	Determine 3D position of noiseless subset positions using linear triangulation method;
Line 23	end for

, cycles through the subset pairs determining their ideal coordinates in the ideal sensor CSs using the polynomial triangulation method according to Section 2.7, in lines 3–20 (based on the code of Lourakis [76]), before performing linear triangulation according to Section 2.8, in lines 21–22, to determine their 3D coordinate in the world CS.

4. Validation

The temporal matching aspect of ADIC3D has been shown to be sufficiently robust to contrast changes and noise while offering a compromise between noise suppression and spatial resolution consistent with that of established DIC algorithms [53]. Thus, Samples 1 and 2 of the Stereo-DIC Challenge [55], exhibiting rigid body translations which avoid introducing errors during temporal matching due to complex displacement fields, are used to assess the remaining aspects of ADIC3D in a quantitative manner. Sample 5, exhibiting complex displacement fields, assesses ADIC3D in a qualitative manner.

ADIC3D was used by setting StopCritVal = 10⁻⁴ (limited to 100 iterations per subset), FiltSize = 0, RefStrat = 0, SubShape = ‘Square’, SFOrder = 1, SubSize = 61, and StepSize = 5. A larger subset size was used to reduce errors during temporal matching. Increasing the subset size beyond 61 pixels produced only marginal improvements.

Additionally, DICe (Version 2.0-beta.16) [56] and LaVision’s StrainMaster (version 1.3) were used to analyze the samples. DICe is well-established [77,78] and its modularity enables it to use the same subset positions in the FIS1 as ADIC3D prior to stereo matching. DICe was run using the same parameters as ADIC3D. StrainMaster, a streamlined version of LaVision’s DaVis software, was used to analyze Samples 1 and 2. Its streamlined nature prevents modifying some correlation parameters such that it employed circular subsets, a stopping criterion of 0.01, and a maximum of 200 iterations. Although this prohibits its results from being directly comparable to ADIC3D, it is included to reflect the capabilities of a commercial code. All codes used identical masks for each sample.

For ADIC3D, subsets achieving a ZNCC criterion below 0.8 for stereo matching were eliminated while DICe and StrainMaster have their own methods of eliminating subsets which fail stereo matching. The final sets of valid subset pairs for ADIC3D and DICe were determined in two steps such that the results of the two codes are directly comparable.

First, a preliminary set of valid subset pairs was determined for each code as the subsets which achieved a ZNCC criterion above 0.8 for temporal matching. Additionally, for Samples 1 and 2 of DICe, Grubbs’s test [79] was used to remove outliers, in terms of displacement error, to reduce outliers occurring in its preliminary set. Secondly, the final set of valid subset pairs for both codes was determined as the intersection of these preliminary sets based on the subset positions in the FIS1.

Successive images of all samples exhibited a jump in displacement such that PCM failed to determine SFP initial estimates. As such, subroutine FeatureMatch was used to obtain SFP initial estimates during temporal matching by inserting “[PD(d).P]=FeatureMatch(PD,d,F,G, SubExtract);” after line 16 in subroutine ImgCorr. DICe also used feature matching to determine SFP initial estimates.

The provided calibration image sets, of dot-grid-style calibration plates, were used by each code to perform calibration. ADIC3D used the STEP1_CalcDLTparameters function of MultiDIC, an open-source Stereo-DIC software proposed by Solav et al. [50], to locate the CTs in the distorted sensor CSs. All experimental image sets were captured using the FLIR Grasshopper 2 model Gras-50S5M cameras with (Edmund Optics) lenses having a focal length of 35 mm.

4.1. Samples 1 and 2

Sample 1 captures images of a complex specimen geometry, consisting of a flat base plate with protruding 3D features, as it is displaced in increments of 10 mm in the x- and z-directions according to

Table A2. Mean and standard deviation of the imposed displacements measured by the stage’s optical encoder for Sample 1 [55].

Step	${\widehat{\mathit{u}}}_{\mathit{w}}\mathbf{Mean}\left(\mathbf{mm}\right)$	U STD (nm)	${\widehat{\mathit{w}}}_{\mathit{w}}\mathbf{Mean}\left(\mathbf{mm}\right)$	W STD (nm)
1	0	7.01	0	6.76
2	0	7.69	10	6.16
3	0	6.30	20	6.21
4	0	7.67	−10	6.12
5	0	6.74	−20	6.33
6	−10	4.91	0	6.83
7	−20	5.71	0	7.27
8	10	6.53	0	6.79
9	20	5.69	0	7.37
10	−10	5.99	−10	4.57
11	−20	14.65	−20	25.19
12	10	7.65	10	6.43
13	20	6.10	20	6.54
14	−10	5.70	10	6.08
15	−20	5.14	20	6.45
16	10	6.29	−10	5.01
17	20	5.99	−20	6.07
18	0	6.36	0	7.59

in Appendix D. The specimen is displaced by an Aerotech nano-positioning stage (model ANT130-160-XY) with a high-precision linear encoder and position feedback control such that the true displacements are known with high certainty. Sample 2 simulates the experiment of Sample 1 using the synthetic image generator documented by Balcaen et al. [62] to minimize error sources in the image set.

The world CS of each DIC code differs from that within which the displacements were imposed. Thus, for each code Pythagoras’s theorem is used to compute the true, ${\widehat{\mu }}_{w_q}^{true}$, and calculated, ${\widehat{\mu }}_{w_q}^{calc}$, displacement magnitudes of each subset $q$. The resulting errors in displacement magnitude are quantified as bias, computed as the mean of the absolute error ($MAE$), and variance, computed as the standard deviation of the absolute error ($\sigma$).

$$MAE=\frac{{{\displaystyle \sum }}_{q=1}^Q\left|{\widehat{\mu }}_{w_q}^{calc}-{\widehat{\mu }}_{w_q}^{true}\right|}{Q}\hspace{1em}\sigma =\sqrt{\frac{{{\displaystyle \sum }}_{q=1}^Q{\left(\left|{\widehat{\mu }}_{w_q}^{calc}-{\widehat{\mu }}_{w_q}^{true}\right|-MAE\right)}^2}{Q-1}}$$

(52)

Here $Q$ is the number of valid subset pairs in the image pair analyzed. For Samples 1 and 2, ADIC3D and DICe had 71,715 and 66,850 valid subsets, respectively, while StrainMaster had 112,021 and 108,085 valid subsets respectively.

Table 15. Displacement error metrics for Sample 1 reported at ×10⁻³ mm.

Step	ADIC3D		DICe		Strain Master
	Bias	Variance	Bias	Variance	Bias	Variance
3	47.2	9.17	37.1	23	55.5	14.1
7	44.2	7.65	41.1	21.6	50.2	13
15	61.6	13.4	57.1	33.5	77.4	20.1

and

Table 16. Displacement error metrics for Sample 2 reported at ×10⁻³ mm.

Step	ADIC3D		DICe		Strain Master
	Bias	Variance	Bias	Variance	Bias	Variance
3	1.55	2.48	5.14	4.59	11.3	5.89
7	1.4	2.16	4.48	4.4	3.07	6.47
15	2.35	3.52	9.92	6.86	7.03	21.2

report errors metrics for Samples 1 and 2, respectively, for steps 3, 7, and 15 which correspond to displacement extremes.

The improved error metrics of Sample 2 relative to Sample 1 for each code, with the exception of StrainMaster’s variance for step 15, reflects the impact of the reduced error sources of the synthetic image set. For Sample 1, ADIC3D performs on par with DICe (up to 27% higher bias) and StrainMaster (at least 12% lower bias). In contrast, for Sample 2 ADIC3D shows improved error metrics relative to DICe and StrainMaster (at least 69% and 54% lower bias respectively). StrainMaster’s higher bias and variance are a result of it analyzing more subsets in the vicinity of protruding features, which are challenging to track, which are removed for ADIC3D and DICe.

4.2. Sample 5

Sample 5 captures images of an ASTM tensile specimen made of Aluminium 2024-T6 as it is loaded in tension. The displacements computed by ADIC3D and DICe are directly comparable since both align their world CS with the camera CS of the first camera.

Figure 4 shows the ${\widehat{u}}_w$ displacement field of image pair 1000 superimposed on the FIS1. The mean and maximum of the percentage difference of the ${\widehat{u}}_w$, ${\widehat{v}}_w,$ and ${\widehat{w}}_w$ displacements, between ADIC3D and DICe, are reported in

Table 17. Percentage difference of computed displacements between ADIC3D and DICe.

Image Pair	250		500		750		1000
Metric	Mean	Max	Mean	Max	Mean	Max	Mean	Max
${\widehat{u}}_w$	0.0342	0.131	0.0354	0.113	0.0373	0.118	0.0374	0.122
${\widehat{v}}_w$	0.374	2.19	0.6	2.57	0.875	2.97	1.04	3.84
${\widehat{w}}_w$	0.093	0.443	0.193	0.744	0.291	0.926	0.345	1.02

for selected image pairs.

The cameras being positioned apart horizontally in the x-direction is reasoned to cause the larger percentage difference of ${\widehat{v}}_w$ and ${\widehat{w}}_w$ relative to ${\widehat{u}}_w$ which is consistent with the findings of Balcaen et al. [5]. The low percentage difference (remaining below 2% mean and 4% maximum) indicates that ADIC3D performs consistently with DICe.

5. Discussion

ADIC3D’s modularity is threefold. Firstly, the main tasks are performed by separate subroutines, closely linked to the mathematical theory presented, allowing readers to progressively build up their understanding of ADIC3D. For instance, correlation is separated into three subroutines enabling the reader to consider: (i) how data is prepared based on the subset shape (SubShapeExtract); (ii) the SF order’s influence on expressions for variables used during correlation (SFExpressions); and (iii) the core operations of correlation (SubCorr) separately. Furthermore, subroutines ImgCorr and StereoMatch show how correlation is utilized differently by temporal and stereo matching.

Secondly, the subset shape, SF order, interpolation method, and Gaussian filtering parameters can be readily changed. Although the influence of these on displacement results is well documented in experimental solid mechanics applications [3,4,45], this allows investigating their influence in novel applications to fine-tune ADIC3D’s setup.

Thirdly, the SF order, subset size, and subset shape can be changed on a per subset and per image basis enabling straightforward integration with adaptive strategies. Adaptive strategies, which are gaining interest in the field of DIC, attempt to assign optimal parameters to each subset such that the resulting displacements are, theoretically, as reliable as possible and independent of the user.

In order for the code to retain its simplicity, four aspects, identified by Pan [41] as state-of-the-art requirements, were omitted. First, calibration does not use bundle adjustment, proposed by Furukawa and Ponce [80], to refine the calibration parameters. Additionally, Vo’s method [81] of combing the frontal image plane and DIC to more accurately locate the CTs is not implemented. It should be noted that although the camera model presented in Section 2.2 only accounts for radial distortion, accounting for tangential distortion in ADIC3D is straightforward, as outlined in Appendix E.

Secondly, although the incremental reference image strategy is included, ADIC3D does not automatically employ it if the ZNCC coefficient falls below a certain threshold as recommended [82]. Thirdly, ADIC3D uses bi-cubic b-spline interpolation whereas bi-quintic b-spline interpolation is recommended [3] since increasing the degree of the interpolation method reduces the errors in the “ultimate error regime” [83] especially for smaller subsets.

Lastly, ADIC3D does not employ Pan’s [84] reliability guided displacement tracking strategy (RGDT). ADIC3D’s temporal matching uses the SFPs of each subset of the previous correlation run as initial estimates in the current correlation run. In contrast, RGDT only does this for the subset achieving the best ZNCC in the previous correlation run. This subset’s optimized SFPs are used as an initial estimate to correlate its neighboring subsets. Of the correlated subsets, having uncorrelated neighbors, the one with the best ZNCC has its SFPs used as an initial estimate to correlate its neighbors. This is repeated to correlate all subsets in the current correlation run. As noted with ADIC2D [53], this exclusion makes ADIC3D susceptible to propagating spurious SFPs of a subset throughout the image series.

Furthermore, RGDT is excluded such that ADIC3D is path-independent (where each subset is correlated independently of its neighbors). Thus, this avoids the need to specify seed points and the issue path-dependent methods have, with complex specimen geometries, of transferring SFPs across subsets which experience different deformation modes. This also allows ADIC3D to leverage MATLAB’s parfor loop to utilize parallel processing to reduce computation time (as discussed in Appendix B).

Additionally, although not a state-of-the-art requirement, ADIC3D’s feature matching method can be improved. Sample 1 and 2 highlight its limitation as it determines invalid SFP initial estimates for subsets in the vicinity of protruding features that experience a large perspective change. Iniyan et al. [85] recommend using the DeepFlow [86] feature matching algorithm combined with the homography transform and MSAC.

Despite these omissions, ADIC3D’s performance is on par with established codes. More specifically, Samples 1 and 2 show that ADIC3D determines displacement magnitudes in the world CS with bias and variance consistent with that of DICe and StrainMaster, while Sample 5 shows that ADIC3D determines displacement vectors within a 4% maximum difference of those determined by DICe. Although Samples 1 and 2 reveal limitations of ADIC3D’s stereo matching, it performs sufficiently well in the presence of a complex specimen geometry.

This, coupled with ADIC2D’s validation of temporal matching and thus the correlation method of ADIC3D indicates that ADIC3D performs sufficiently well for practical use in the field of experimental solid mechanics. Moreover, due to its modularity ADIC3D can be readily adapted to a wide range of applications across many fields. Furthermore, validation of this modular framework makes it both attractive as an education resource and as a basis to further the capabilities of DIC.

6. Conclusions

The theory of a subset-based, Stereo-DIC framework, that is predominantly consistent with current state-of-the-art techniques, and its implementation as a modular 202 line MATLAB code, was presented. This framework is an extension of the previously presented ADIC2D framework and as such is also comprised of square or circular subset shape selection, assigning of Gaussian filtering parameters, altering of the interpolation method, the zero, first and second-order SFs, reference image strategy selection, the Phase Correlation Method to determine SFP initial estimates for temporal matching and calibration through using MATLAB’s image calibration toolbox. ADIC3D, being capable of performing Stereo-DIC to determine both in-plane and out-of-plane displacements in the world CS, is additionally comprised of stereo matching, SIFT feature matching to determine SFP initial estimates for stereo matching and displacement transformation using the polynomial and linear triangulation methods. Furthermore, ADIC3D allows for assignment of SF order, subset shape, and subset size on a per image and per subset basis enabling straightforward integration with adaptive strategies. The framework is modular, enabling the reader to gain a deeper understanding of DIC as well as encouraging readers to adapt ADIC3D for new applications. Validation of the full framework coupled with its modularity makes it appealing not only as an educational resource, but also as a starting point to develop the capabilities of DIC.