Rigorous Line-Based Transformation Model Using the Generalized Point Strategy for the Rectification of High Resolution Satellite Imagery

Abstract

High precision geometric rectification of High Resolution Satellite Imagery (HRSI) is the basis of digital mapping and Three-Dimensional (3D) modeling. Taking advantage of line features as basic geometric control conditions instead of control points, the Line-Based Transformation Model (LBTM) provides a practical and efficient way of image rectification. It is competent to build the mathematical relationship between image space and the corresponding object space accurately, while it reduces the workloads of ground control and feature recognition dramatically. Based on generalization and the analysis of existing LBTMs, a novel rigorous LBTM is proposed in this paper, which can further eliminate the geometric deformation caused by sensor inclination and terrain variation. This improved nonlinear LBTM is constructed based on a generalized point strategy and resolved by least squares overall adjustment. Geo-positioning accuracy experiments with IKONOS, GeoEye-1 and ZiYuan-3 satellite imagery are performed to compare rigorous LBTM with other relevant line-based and point-based transformation models. Both theoretic analysis and experimental results demonstrate that the rigorous LBTM is more accurate and reliable without adding extra ground control. The geo-positioning accuracy of satellite imagery rectified by rigorous LBTM can reach about one pixel with eight control lines and can be further improved by optimizing the horizontal and vertical distribution of control lines.

1. Introduction

Nowadays, with the successful launch of many high resolution land observation satellites, such as Pleiades-II, WorldView-3, Jilin-1, etc., higher requirements for image processing technology have been put forward. High Resolution Satellite Imagery (HRSIs) has high spatial, radiometric and temporal resolution. It can supply accurate and abundant information of the Earth’s surface [1,2] and has been an important data source for accurate mapping at large scales (up to 1:10,000) and map updating (up to 1:5000) [3]. Thus, the geometric rectification technology of HRSIs needs improvement to fit more various and higher precision demands in actual applications.

The imaging geometric models for geometric rectification of HRSIs are mainly divided into two categories: rigorous sensor models and non-rigorous mathematical models. The modeling approach of rigorous sensor models has matured. The systematic geo-positioning accuracy of QuickBird imagery rectified by rigorous sensor model reaches 23 m CE90 (a circular error at 90% probability) [4,5]; whereas, rigorous sensor models are limited in applications because of their sophisticated imaging geometry, complicated sensor physical structure and precision instability caused by self-correlated model parameters [5,6]. Since high resolution land observation satellites possess the characteristics of stable orbit, long focal length, large flight height and narrow field of view, the strict imaging geometric relationship of HRSIs can be simplified by non-rigorous mathematical models with satellite parameters hidden for technology protection, such as the Affine Transformation Model (ATM) [7], the Direct Linear Transformation (DLT) model [8], the Rational Function Model (RFM) [9], and so on. Many improved models have been proposed based on these, as well: some of them focus on refining the description of imaging geometric relationship, such as the self-calibrating DLT model [10] and strict ATMs [11,12,13]; while some of them optimize RFM by removing correlated parameters from Rational Polynomial Coefficients (RPCs) [14,15].

At present, most of the non-rigorous mathematical models are constructed based on point features and mainly in two ways: the terrain independent method and the terrain-dependent method [9,16]. For the terrain independent method, hundreds of 3D grid points are generated by projection from the image plane to the virtual layered object space using a rigorous sensor model with the aid of attitude and orbit data, then the parameters of the non-rigorous mathematical model are calculated by least squares adjustment. In general, RPCs are generated using the terrain independent method, and they are provided with sensor correction products of commercial high resolution satellites. Experiments show that RFM agrees with the rigorous sensor model to within a fraction of a pixel [4,9]. While for the terrain-dependent method, the imaging geometric relationship between image space and object space is fit by appropriate model parameters, which are determined by control conditions captured from a map or field measurement. The solution of the terrain-dependent method is highly dependent on the number, accuracy and distribution of control conditions [5,17]. Application practices indicate that the geo-positioning accuracies of SPOT5 and IKONOS imagery rectified by RFM can achieve about one pixel with well-distributed Ground Control Points (GCPs) in urban areas [15,18]. The strict geometric model based on Affine Transformation (rigorous ATM) can also achieve an accuracy level corresponding to the Ground Sampling Distance (GSD) of IKONOS and CBERS-2 satellite imagery with only ten model parameters [11,12].

There are mainly two ways of improving the direct geo-positioning accuracy of HRSIs: when the original RPCs and one or more GCPs are provided, the RPCs can be refined by a direct correction method (such as the batch iterative least squares solution with regularization method [19]) or an indirect correction method (such as the bias compensation method [20,21]). For example, the regenerated RPCs with the affine bias correction model provide an accuracy of about one pixel of QuickBird imagery with no less than three well-distributed GCPs [21]. Another situation is when the original RPCs are not available; the terrain-dependent method is the only option, though more GCPs are required. For instance, at least forty GCPs are demanded for solving the RPCs of RFM.

As can be seen above, GCPs play an important role in high precision geometric rectification of HRSIs by Point-Based Transformation Models (PBTMs). It is high cost to layout and maintain numerous GCPs. The automatic extraction and matching of GCPs are complicated and unreliable, as well, due to image orientation variation, relief displacement, multi-sensor or multi-temporal imagery and texture lacking areas [22,23,24]. By contrast, line features possess many unique advantages: first of all, line features in image space are easier to detect and extract automatically with sub-pixel accuracy across the direction of the edge; secondly, linear features can be matched exactly, either within overlapping images or between image space and object space; thirdly, line features can be implicitly described by segments along lines to avoid certain obscured or changed parts, making the control scheme more flexible; besides, line features in object space possess more sufficient attributes and semantic information; they can increase the redundancy and improve the robustness of image processing; last, but not least, there exists plenty of line features in the man-made environment. They can also be directly derived from existing geographic spatial data, such as a GIS database, large-scale maps, a terrestrial mobile mapping system, and so on, which can largely reduce the cost of ground control work [22,23,24,25]. Given these, line features have been widely used in photogrammetric activities, including space intersection and resection [23,25], triangulation [22,25], relative and exterior orientation [26,27,28], image matching and registration [22,29,30], camera calibration [22,31], etc.

As basic control conditions instead of point features, line features have been introduced into the field of image rectification, as well. 3D affine LBTM, 2D LBTM and 2D conformal LBTM were first proposed by Shaker [32,33] in 2004. These LBTMs are all constructed by line features in the image space and the object space in vector form. Rectified by 3D affine LBTM with twelve evenly-distributed control lines, the geo-positioning accuracy of IKONOS imagery on flat terrain is approximately two and seven pixels respectively in the X and Y directions, which is comparable with those rectified by existing relevant PBTMs. Shi et al. [34] applied 2D LBTM and 2D conformal LBTM to image-to-image registration of HRSIs. One GCP is used to correct the translation parameters, and several control lines are used to correct the rotation and deformation parameters. The geo-positioning accuracies of rectified IKONOS and QuickBird imagery are better than two pixels; whereas, these LBTMs above cannot accommodate the adverse effect from relief displacements due to the variations in sensor inclination and terrain elevation. Elaksher [35,36] further designed line-based parallel projection, extended parallel projection and DLT models. In an area where the average elevation range is about 175 m, the average Root Mean Square (RMS) errors of image rectification of SPOT5, IKONOS and QuickBird imagery are about 1.7 pixels based on manually- or automatically-extracted line features. In addition, Long et al. [24] proposed a generic framework for image precise rectification using multiple features, which include points, straight line segments, freeform curves and areal regions. The rigorous sensor model of Landsat-5 L2 and bias-compensated RFM using affine transformation in image space are utilized as examples, and sub-pixel accuracy is achieved for SAR and optical imagery registration. Teo [37] introduced control lines into direct RFM and bias-compensated RFM in vector and parametric forms. The geo-positioning accuracies of rectified IKONOS and QuickBird imagery achieve pixel-level accuracy with sufficient control lines.

In this study, a novel rigorous LBTM is proposed to improve the geometric rectification accuracy of HRSIs by further compensating the adverse impacts caused by sensor inclination and terrain variation. This paper is organized as follows: in Section 2, rigorous LBTM is constructed by introducing line features into rigorous ATM using a generalized point strategy. In Section 3, reasonable initial values of some parameters in rigorous LBTM are obtained by the calculation of the six-parameter LBTM in the preprocessing step. Then, their precise values are solved by least squares adjustment. The process of high precision geometric rectification of HRSIs based on rigorous LBTM is illustrated in detail. In Section 4, the experiments of different satellite data, different terrain conditions, different numbers and distributions of control lines are carried out and discussed. The geo-positioning accuracies of IKONOS, GeoEye-1 and ZiYuan-3 satellite imagery rectified by rigorous LBTM are verified in contrast with other relevant LBTMs and PBTMs. Finally, in Section 5, some conclusions arising from the present study are given, and the prospects of future work are addressed.

2. Construction of the Rigorous LBTM

2.1. Strict Geometric Model Based on Affine Transformation

LBTMs proposed by Shaker [32,33] are mainly deduced from point-based linear models, such as ATM and the conformal transformation model, since these models are suitable to describe line features in vector form. ATM adopts parallel projection instead of central projection. It can eliminate the substantial correlations between the positioning and orientation parameters in the imaging geometric processing of HRSIs [6]. Nevertheless, ATM simplifies the 3D problem to a 2D problem in object space without taking into account the influence of terrain variation.

To solve this problem, Zhang et al. [11,12] put forward the rigorous ATM, which is under the hypothesis that the perspective projection relationship of HRSI can be seen as the central projection in the scanning direction and the parallel projection in the flight direction. In this study, we deduce rigorous ATM simplistically by two steps: first, a point in object space is projected to the image plane using affine transformation; next, the coordinates of the same image point are obtained by parallel projection and central projection along the scanning direction, respectively. Figure 1 presents the imaging geometric relationship of a linear array scanner along the scanning direction.

As shown in Figure 1, $S$ is the projection center, and $O$ is the principle point of the image. $\overrightarrow{SO}$ shows the direction along parallel projection and perpendicular to the original image plane. $B$ is an object point with elevation $H$. $C$ is the corresponding image point of $B$ on the original image plane by central projection. $A^{\prime }$ is the intersection point of $\overrightarrow{SA}$ and the virtual horizontal image plane. $A^{\prime }{B}^{\prime }$, which is parallel to $AB$, can be considered as the corresponding line segment of $AB$ with the proper scale in the image space.

$\overrightarrow{B^{\prime }D}$ is parallel to $\overrightarrow{SO}$ through $B^{\prime }$. $E$ is the intersection point of $\overrightarrow{B^{\prime }D}$ and the original image plane, as well as the corresponding image point of $B$ by parallel projection. The difference between central projection and parallel projection in the scanning direction can be eliminated by building the relationship between line segments $OE$ and $OC$ [11,12].

The rigorous ATM is constructed as follows:

$$\{\begin{array}{c}\frac{f-(Z-Z_{ave})/(m*\cos (\omega ))}{f-x\tan (\omega )}x=b_{1}X+b_{2}Y+b_{3}Z+b_4\\ y=b_{5}X+b_{6}Y+b_{7}Z+b_8\end{array}$$

(1)

where $(x,y)$ is the coordinate of an image point in the image space coordinate system, of which the origin is at the image principle point $O(x_0,y_0)$; $b_1,b_2,\dots ,b_8$ are the parameters of ATM; $f$ is the equivalent focal length; $\omega$ is the tilt angle of the linear array scanner in the scanning direction; $m$ is the imaging scale denominator; $Z$ is the elevation coordinate of $B$ in the object space; and $Z_{ave}$ is the average elevation of the coverage area, which meets $Z-Z_{ave}=H$.

In general, $m$ can be approximately expressed by the ratio of GSD and pixel size. The initial values of $f$ and $m$ are easy to acquire from the released parameter files of different commercial high resolution satellites.

2.2. Rigorous LBTM Based on the Generalized Point Strategy

Line features in LBTMs are mainly described by conjugate unit vectors in the image space and the object space. Their imaging geometric relationship can be conveniently constructed through subtracting equations of the point-based linear model between two points along the same line feature [18,19,20]; whereas, this method is not adequate for the non-linear models, including the rigorous ATM.

The generalized point strategy was first proposed by Zhang et al. [38,39] for bundle adjustment to combine the line feature with the point feature in collinearity equations. The line feature is regarded as an extension of the point feature in the line’s parameter equation. Thereby, the individual point and the point on the line can be expressed in the same form for constructing collinearity equations. The generalized point strategy can be widely applied for exterior orientation, industrial part measurement, building reconstruction, and so forth. In this study, the generalized point strategy is attempted to reform the rigorous ATM into the line-based form by introducing line features into nonlinear equations.

According to the perspective imaging principle, if an image point lies on a line in the image space, the conjugate object point is certain to lie on the conjugate line in the object space. Their correspondence is shown in Figure 2.

As shown in Figure 2, $(x_1,y_1)$ and $(x_2,y_2)$ are the coordinates of two image points on line $l$ in the image space coordinate system; $(X_1,Y_1,Z_1)$ and $(X_2,Y_2,Z_2)$ are the coordinates of the conjugate object points on conjugate line $L$ in the object space coordinate system; $(X_1^{\prime },Y_1^{\prime },Z_1^{\prime })$ and $(X_2^{\prime },Y_2^{\prime },Z_2^{\prime })$ are the coordinates of the other two object points on line $L$.

Based on the line’s parameter equation, $(X_1,Y_1,Z_1)$ can be described by $(X_1^{\prime },Y_1^{\prime },Z_1^{\prime })$ and a variable $t_1$ in Equation (2):

$$\{\begin{array}{l}{X}_1=X_1^{\prime }+A_X{t}_1\\ Y_1=Y_1^{\prime }+A_Y{t}_1\\ Z_1=Z_1^{\prime }+A_Z{t}_1\end{array}$$

(2)

where $(A_X,A_Y,A_Z)$ is the unit vector of line $L$ in the object space coordinate system; $t_1$ is the proportion coefficient.

In rigorous ATM, the correspondence of conjugate points $(x_1,y_1)$ in the image space and $(X_1,Y_1,Z_1)$ in the object space can be expressed by Equation (3):

$$\{\begin{array}{c}\frac{f-(Z_1^{\prime }+A_Z{t}_1-Z_{ave})}{f-x_1\tan (\omega )}{x}_1=b_1(X_1^{\prime }+A_X{t}_1)+b_2(Y_1^{\prime }+A_Y{t}_1)+b_3(Z_1^{\prime }+A_Z{t}_1)+b_4\\ y_1=b_5(X_1^{\prime }+A_X{t}_1)+b_6(Y_1^{\prime }+A_Y{t}_1)+b_7(Z_1^{\prime }+A_Z{t}_1)+b_8\end{array}$$

(3)

Equation (3) is the basic form of the rigorous LBTM, in which the unit vector is introduced to avoid strict correspondence between conjugate points. The constraint condition is relaxed compared with point-based rigorous ATM, on the condition that the corresponding points are on the conjugate line features. $b_1,b_2,\dots ,b_8$ in both rigorous LBTM and ATM have the same meaning.

Suppose there are $j$ control lines and $i$ points on each control line. Since two equations as Equation (3) can be constructed by each point, $2ij$ equations can be constructed by $j$ lines in all. Besides, two equations as Equation (1) are determined by one GCP; hence, the total number of equations increases to $2ij+2$. On the other hand, each point corresponds to a proportion coefficient $t$, and a total of $ij$ proportion coefficients are required for $j$ control lines. In addition, unknowns in these equations also include eight affine transformation parameters $b_1,b_2,\dots ,b_8$, the equivalent focal length $f$ and the tilt angle of the linear array scanner in the scanning direction $\omega$. The total number of unknowns is $ij+10$. The number of control lines should satisfy $ij\ge 8$ for solving the parameters of rigorous LBTM. For instance, if there are two points selected on each control line, four lines at least are needed for the calculation. If $ij>8$, the parameters of rigorous LBTM can be calculated based on least squares adjustment [40].

3. Image Rectification by Rigorous LBTM

3.1. Preprocessing by Six-Parameter LBTM

Supposing the imaging geometric relationship of the push broom scanner is approximately a parallel projection [13], the relationship of the conjugate points in the image space and the object space constructed by ATM takes on the following form:

$$\{\begin{array}{l}{x}_i=b_1{X}_i+b_2{Y}_i+b_3{Z}_i+b_4\\ y_i=b_5{X}_i+b_6{Y}_i+b_y{Z}_i+b_8\end{array}$$

(4)

where $\begin{array}{cc}(x_i,y_i)& (i=1,2,\dots ,n)\end{array}$ is the coordinate of an image point in the image space coordinate system; $\begin{array}{cc}(X_i,Y_i,Z_i)& (i=1,2,\dots ,n)\end{array}$ is the coordinate of the conjugate object point in the object space coordinate system; $b_1,b_2,\dots b_8$ are the parameters of ATM.

If there are two points $A(x_1,y_1,X_1,Y_1,Z_1)$ and $B(x_2,y_2,X_2,Y_2,Z_2)$, $b_4$ and $b_8$ can be removed by the subtraction of two corresponding equations constructed by $A$ and $B$. Then, Equation (4) is translated into another form as Equation (5):

$$\{\begin{array}{l}{x}_2-x_1=b_1(X_2-X_1)+b_2(Y_2-Y_1)+b_3(Z_2-Z_1)\\ y_2-y_1=b_5(X_2-X_1)+b_6(Y_2-Y_1)+b_7(Z_2-Z_1)\end{array}$$

(5)

Set $M=\sqrt{{(x_2-x_1)}^2+{(y_2-y_1)}^2}$ and $N=\sqrt{{(X_2-X_1)}^2+{(Y_2-Y_1)}^2+{\left(Z_2-Z_1\right)}^2}$; the expression of Equation (5) is refined to be another form:

$$\{\begin{array}{l}\frac{M}{N}\frac{x_2-x_1}{M}=b_1\frac{X_2-X_1}{N}+b_2\frac{Y_2-Y_1}{N}+b_3\frac{Z_2-Z_1}{N}\\ \frac{M}{N}\frac{y_2-y_1}{M}=b_5\frac{X_2-X_1}{N}+b_6\frac{Y_2-Y_1}{N}+b_7\frac{Z_2-Z_1}{N}\end{array}$$

(6)

Set $\left(a_x,a_y\right)$ as the unit vector of line segment $AB$ in the image space, where $a_x=\left(x_2-x_1\right)/M$ and $a_y=\left(y_2-y_1\right)/M$; $\left(A_X,A_Y,A_Z\right)$ is the unit vector of $AB$ in the object space, where $A_X=(X_2-X_1)/N$, $A_Y=\left(Y_2-Y_1\right)/N$ and $A_Z=\left(Z_2-Z_1\right)/N$; $\lambda =M/N$ represents the scale factor. Then, Equation (6) can be simplified as Equation (7):

$$\{\begin{array}{l}\lambda a_x=b_1{A}_X+b_2{A}_Y+b_3{A}_Z\\ \lambda a_y=b_5{A}_X+b_6{A}_Y+b_7{A}_Z\end{array}$$

(7)

Equation (7) is the basic form of the six-parameter LBTM. The rotation and scaling parameters between $\left(a_x,a_y\right)$ and $\left(A_X,A_Y,A_Z\right)$ are considered. As, for example, in Shaker’s experiments [32,33,34,35], the values of two translation parameters $b_4,b_8$ are small enough to ignore. Six-parameter LBTM is proven to eliminate the over-parameterized problem existing in eight-parameter LBTM.

In rigorous LBTM, the initial values of $b_1,b_2,b_3,b_5,b_6,b_7$ can be calculated by Equation (7) with more than three non-collinear control lines. Then, the initial values of $b_4$ and $b_8$ can be calculated by Equation (4) with one GCP. At last, the initial values of $t_i$ in the related line’s parameter equations can be calculated as follows:

First, Equations (2) and (4) are combined together as Equation (8):

$$\{\begin{array}{l}{x}_1=b_1(X_1^{\prime }+A_X{t}_1)+b_2(Y_1^{\prime }+A_Y{t}_1)+b_3(Z_1^{\prime }+A_Z{t}_1)+b_4\\ y_1=b_5(X_1^{\prime }+A_X{t}_1)+b_6(Y_1^{\prime }+A_Y{t}_1)+b_7(Z_1^{\prime }+A_Z{t}_1)+b_8\end{array}$$

(8)

Then, all of the $t_i$ are calculated by Equation (8) on the basis of least squares adjustment.

3.2. Overall Adjustment of Rigorous LBTM

The precise values of all of the parameters in rigorous LBTM can be further calculated by least squares overall adjustment. First, Equation (3) is converted into the form of error equations and linearized using Taylor series expansion as the following form:

$$V=\left[\begin{array}{cc}A& C\end{array}\right]\left[\begin{array}{c}X\\ T\end{array}\right]-LP$$

(9)

where $V={\left[\begin{array}{cc}{v}_x& v_y\end{array}\right]}^T$ is the correction matrix of observations in the $x$ and $y$ direction; $X={[\begin{array}{llllllllll}\mathrm{\Delta }{b}_1\hfill & \mathrm{\Delta }{b}_2\hfill & \mathrm{\Delta }{b}_3\hfill & \mathrm{\Delta }{b}_4\hfill & \mathrm{\Delta }{b}_5\hfill & \mathrm{\Delta }{b}_6\hfill & \mathrm{\Delta }{b}_7\hfill & \mathrm{\Delta }{b}_8\hfill & \mathrm{\Delta }f\hfill & \mathrm{\Delta }\omega \hfill \end{array}]}^T$ is the correction matrix of $b_1,b_2,\dots ,b_8$, $f$ and $\omega$; $T={\left[\begin{array}{cc}\mathrm{\Delta }{t}_{1i}& \mathrm{\Delta }{t}_{2i}\end{array}\right]}^T$ is the correction matrix of $t_i$ determined by the control lines; $A$ and $C={\left[\begin{array}{cc}{c}_{1i}& c_{2i}\end{array}\right]}^T$ are the designed matrices of the correction matrices $X$ and $T$; $L={\left[\begin{array}{cc}-F_{x0}& -F_{y0}\end{array}\right]}^T$is the constant matrix calculated by the initial values of the unknowns; $P$ is the weighting matrix of the observation equations. The detailed form of $A$ is presented as:

$$A=\left[\begin{array}{cccccccccc}{a}_{10}& a_{11}& a_{12}& a_{13}& a_{14}& a_{15}& a_{16}& a_{17}& a_{18}& a_{19}\\ a_{20}& a_{21}& a_{22}& a_{23}& a_{24}& a_{25}& a_{26}& a_{27}& a_{28}& a_{29}\end{array}\right]$$

(10)

where all of the parameters $a_{10}~a_{19},a_{20}~a_{29}$ in matrix $A$ and $c_{1i},c_{2i}$ in matrix $C$ are the partial derivatives of the corresponding parameters in Equation (3).

$$\{\begin{array}{l}{a}_{10}=\partial F_x/\partial b_1=-X^{\prime }-A_X{t}_i,a_{24}=\partial F_y/\partial b_5=-X^{\prime }-A_X{t}_i\hfill \\ a_{11}=\partial F_x/\partial b_2=-Y^{\prime }-A_Y{t}_i,a_{25}=\partial F_y/\partial b_6=-Y^{\prime }-A_Y{t}_i\hfill \\ a_{12}=\partial F_x/\partial b_3=-Z^{\prime }-A_Z{t}_i,a_{26}=\partial F_y/\partial b_7=-Z^{\prime }-A_Z{t}_i\hfill \\ a_{13}=\partial F_x/\partial b_4=1.0,a_{27}=\partial F_y/\partial b_8=1.0\hfill \\ a_{14},\dots ,a_{17}=0.0,a_{20},\dots ,a_{23},a_{28},a_{29}=0.0\hfill \\ a_{18}=\partial F_x/\partial f=\frac{x_i(Z_1^{\prime }+A_Z{t}_i-Z_{ave})-x_i^2\tan ({\omega }_0)}{{(f_0-x_i\tan ({\omega }_0))}^2}\hfill \\ a_{19}=\partial F_x/\partial \omega =\frac{x_i^2{f}_0-x_i^2(Z_1^{\prime }+A_Z{t}_i-Z_{ave})}{{(f_0-x_i\tan ({\omega }_0))}^2{\cos }^2({\omega }_0)}\hfill \\ c_{1i}=\partial F_x/\partial t_i=\frac{x_i{A}_Z}{x_i\tan \left({\omega }_0\right)-f_0}-b_1^0{A}_X-b_2^0{A}_Y-b_3^0{A}_Z\hfill \\ c_{2i}=\partial F_y/\partial t_i=-b_5^0{A}_X-b_6^0{A}_Y-b_7^0{A}_Z\hfill \end{array}$$

(11)

where $b_1^0,b_2^0,b_3^0,b_5^0,b_6^0,b_7^0$ are the initial values of $b_1,b_2,b_3,b_5,b_6,b_7$ calculated by six-parameter LBTM in the preprocessing step, as illustrated in Section 3.1; $f_0$ and ${\omega }_0$ are the initial values of $f$ and $\omega$ acquired from the released parameter files of the satellite.

Error equations as Equation (9) can be constructed by each GCP and point on the control lines. They can be further normalized and calculated on the basis of least squares adjustment [40] through overall iteration. Distinguished from existing LBTMs, it is of significance to select more than two points on each control line to strengthen the control network of the overall adjustment in the solution process of rigorous LBTM.

3.3. Image Rectification by Rigorous LBTM

Since all of the precise values of model parameters in rigorous LBTM are obtained in Section 3.1 and Section 3.2, the coordinate of the object points corresponding to each image point of HRSI can be calculated precisely by rigorous ATM.

Under the condition of flat terrain or the Digital Elevation Model (DEM) of the coverage area is unavailable, the horizontal coordinates of the object point corresponding to image point $\left(x_1,y_1\right)$ can be calculated simply as Equation (12):

$$\left[\begin{array}{cc}{b}_1& b_2\\ b_5& b_6\end{array}\right]\left[\begin{array}{c}{X}_1\\ Y_1\end{array}\right]=\left[\begin{array}{c}\frac{fx}{f-x_1\tan (\omega )}-b_3{Z}_{ave}-b_4\\ y_1-b_7{Z}_{ave}-b_8\end{array}\right]$$

(12)

where $Z_{ave}$ is the average elevation of the coverage area.

When the DEM of the coverage area is provided, the geometric rectification processing of HRSI would be more accurate with the following three steps: first, in order to determine the geographical coverage area of rectified HRSI, four corner image points are projected to the DEM surface by the forward solution method of image rectification [41] using the inverse transform of rigorous ATM. The accurate elevation coordinates of conjugate object points are approached gradually from $Z_{ave}$ by iterative computation of Equation (12); secondly, the minimum enclosing rectangle of the coverage area is divided into regular square grids. The coordinate of the image point corresponding to each object grid point is calculated by the inverse solution method of image rectification [41] using rigorous ATM. The pixel value of this rectified image point is calculated by the bilinear interpolation of four adjacent pixels in HRSI; at last, the pixel of each image point in rectified HRSI is calculated by the bilinear interpolation of four pixels of image points corresponding to four adjacent object grid points. The flow chart of image rectification based on rigorous LBTM is shown in Figure 3.

4. Results and Discussion

4.1. Data Source and Experiment Scheme

To verify the performance of the proposed rigorous LBTM for image rectification, four HRSI datasets are employed for experiments in this paper, which include the IKONOS imagery of Hong Kong/China area (dataset IKONOS_HK), the ZiYuan-3 imagery of Hong Kong/China area (dataset ZIYUAN_HK), the IKONOS imagery of Hobart/Australia area (dataset IKONOS_HB), and the GeoEye-1 imagery of Hobart/Australia area (dataset GEOEYE_HB). The elevation range of coverage area of Hong Kong/China and Hobart/Australia are about 200 m and 500 m, respectively. The basic specifications of the four HRSI datasets are shown in

Table 1. Specifications of the four HRSI datasets.

Datasets	IKONOS_HK	ZIYUAN_HK	IKONOS_HB	GEOEYE_HB
Sensor Type	IKONOS (Pan)	ZiYuan-3 (Pan)	IKONOS (Pan)	GeoEye-1 (Pan)
Coverage Area	Hong Kong, China	Hong Kong, China	Hobart, Australia	Hobart, Australia
Terrain Variation	Undulated	Undulated	Hilly	Hilly
Elevation Range (m)	200	200	500	500
Coordinate System	WGS 84 UTM	WGS 84 UTM	WGS 84 UTM	WGS 84 UTM
GSD (m)	1.0	2.1	1.0	0.5
Focal Length (m)	10.0	1.7	10.0	13.3
Image Frame (pixel)	11,604 × 10,280	4372 × 6165	13,148 × 12,124	31,668 × 26,928
Pixel Size (μm)	12.0	7.0	12.0	8.0

Note: Dataset ZIYUAN_HK is intercepted from a standard scene of the ZiYuan-3 image product, which covers the same areas of Hong Kong as dataset IKONOS_HK for comparison.

.

Numerous evenly-distributed GCPs have been observed for each dataset by fast static Global Positioning System (GPS) technology, which include 42 GCPs in the coverage area of dataset IKONOS_HK and ZIYUAN_HK and 50 GCPs in the coverage area of dataset IKONOS_HB and GEOEYE_HB. The GCPs are located on the ground surface of road intersections, landmarks and some well-known features. Measurements are carried out by two Trimble dual-frequency GPS receivers with differential GPS technology: one reference receiver lays in the middle of the network and is used as the base station; the other rover receiver moves around for collecting GCPs with the baseline less than 6 km. The average observation time of each GCP is about twenty minutes. After surveying, the GPS coordinates of each GCPs are determined and corrected by GPS baseline adjustment with Trimble Geomatics Office (TGO) software. Coordinates are provided in the WGS84 datum and Universal Transverse Mercator (UTM) projection with the RMS errors being less than 0.05 m in the horizontal direction and 0.08 m in the vertical direction. The image points corresponding to each GCP on the four datasets are identified manually with the accuracy of about 0.5 pixels.

In rigorous LBTM, the unit vector of a control line in the image space or object space can be defined by any two points along the line. The construction of the unit vector is not affected by some obscured or changed parts along the control line. For convenience in most experiments, the unit vector in the image space is calculated by the image coordinates of two GCPs, and the conjugate unit vector in the object space is calculated by the object coordinates of the same two GCPs separately; whereas, the strict correspondence between image points and object points on the conjugate control lines is not needed. Relevant validation experiments are made by nine real control lines (in which the image points and object points for constructing conjugate control lines in the image space and the object space are different) in Section 4.3 for a complement explanation.

4.2. Performance of Rigorous LBTM to Different Datasets

Rigorous LBTM is compared with relevant LBTMs (including eight-parameter LBTM (8-param LBTM) and six-parameter LBTM (6-param LBTM)) and relevant PBTMs (including ATM and rigorous ATM) to investigate their performance under the conditions of different sensors, GSDs and terrain variations.

During the process of image rectification based on LBTMs, GCPs are used for three purposes: one of them is selected as a single GCP to determine the datum in the object space; some are used for constructing control lines; and others are used as check points to examine the actual positioning accuracy of image rectification. Both control lines and check points are selected as uniformly as possible in the coverage area of the four datasets to get better experimental results. As for image rectification based on PBTMs, the first two groups of GCPs in LBTMs are both used as control points. The check points are the same as those in LBTMs. The numbers of control conditions of Control Schemes I to IV on four HRSI datasets are shown in

Table 2. Numbers of control conditions of Control Schemes I to IV on four HRSI datasets. PBTM, Point-Based Transformation Model.

Control Schemes	Datasets	Numbers in LBTMs			Numbers in PBTMs
		Control Lines	Single GCP	Check Points	Control Points	Check Points
Control Scheme I	IKONOS_HK	8	1	29	13	29
Control Scheme II	ZIYUAN_HK	8	1	29	12	29
Control Scheme III	IKONOS_HB	12	1	20	24	20
Control Scheme IV	GEOEYE_HB	12	1	20	24	20

. The corresponding spatial distributions of control conditions are shown in Figure 4.

The performance of image rectification by LBTMs and PBTMs is measured in terms of relative actual accuracy, which is defined by dividing actual positioning accuracy by GSD, and calculated by RMS error [40]. The value of relative actual accuracy is approximately equivalent to the pixel offset.

Table 3. Relative actual accuracies of Control Schemes I to IV by different LBTMs and PBTMs. Param, Parameter; ATM, Affine Transformation Model.

Control Schemes	Datasets	Relative Actual Accuracy	LBTMs			PBTMs
			8-Param LBTM	6-Param LBTM	Rigorous LBTM	ATM	Rigorous ATM
Control Scheme I	IKONOS_HK	XRMS (pixel)	1.4714	1.4485	0.9928	0.9961	0.9922
		YRMS (pixel)	1.3009	1.2508	0.9381	0.9389	0.9351
Control Scheme II	ZIYUAN_HK	XRMS (pixel)	2.4784	2.2105	1.4164	1.3959	1.3901
		YRMS (pixel)	1.1845	1.1634	1.3177	1.2101	1.1296
Control Scheme III	IKONOS_HB	XRMS (pixel)	6.3520	5.8500	0.5029	1.0475	0.4977
		YRMS (pixel)	3.7699	3.6705	0.4353	0.4976	0.3660
Control Scheme IV	GEOEYE_HB	XRMS (pixel)	8.0303	7.3253	1.5470	1.7821	1.4574
		YRMS (pixel)	5.5918	5.5669	0.8728	1.0073	0.5215

Note: X_RMS represents the RMS error in the scanning (X) direction of the sensor; Y_RMS represents the RMS error in the flight (Y) direction, which is perpendicular to the scanning direction.

shows the relative actual accuracies of Control Schemes I to IV by different LBTMs (including 8-param LBTM and 6-param LBTM) and PBTMs (including ATM and rigorous ATM).

As shown in Table 3, the relative actual accuracies of the four datasets rectified by rigorous LBTM are similar to those rectified by two PBTMs and much higher than those rectified by the other two LBTMs. Though the geometric constraints of control lines are relatively less strict than those of control points, the performance of rigorous LBTM does not decrease obviously and is better than that of ATM.

The relative actual accuracies of the four datasets rectified by rigorous LBTM are generally higher than 1.5 pixels and as stable as those rectified by rigorous ATM, whether in flat, undulated or hilly areas. The relative actual accuracy of dataset IKONOS_HB achieves about 0.5 pixels. In contrast, the relative actual accuracies of the four datasets rectified by ATM, 8-param LBTM and 6-param LBTM appear significant decreased in hilly areas. The experimental results show that rigorous LBTM and rigorous ATM have a stronger compatibility to complex terrain situations than other models, owing to the compensation of geo-positioning deviation caused by sensor inclination and terrain variation.

Rectified by different LBTMs and PBTMs, the relative actual accuracies of the four datasets in the Y direction are higher than those in the X direction. Through the comparison, the relative actual accuracies by rigorous LBTM in the X direction decrease more slightly than those by other LBTMs. There exists more deformation in HRSI along the scanning direction than along the flight direction. Hence, it is meaningful for rigorous LBTM to take special correction in the X direction by adding model parameters, such as the equivalent focal length and tilt angle of the optical sensor, as illustrated in Section 2.1.

Besides, relative actual accuracies of the four datasets rectified by 6-param LBTM are slightly superior to those rectified by 8-param LBTM. 6-param LBTM removes the two duplicated translation parameters from 8-param LBTM, thus avoiding their adverse effects on least squares adjustment as illustrated in Section 3.1. It is feasible for 6-param LBTM to provide reliable initial values of the model parameters of rigorous LBTM.

In brief, experimental results of Control Schemes I to IV show that rigorous LBTM is more accurate and robust than existing relevant LBTMs and PBTMs without adding extra ground control conditions. Rigorous LBTM is more suitable for high precision geometric rectification of HRSIs.

4.3. Accuracy Verification of Rigorous LBTM Based on Real Control Lines

To further verify the effectiveness of rigorous LBTM for real control lines, nine control lines are collected evenly in the Hong Kong area in practice. Each control line in the object space is determined by connecting two object points, which are measured by fast static GPS technology, like GCPs, and each corresponding control line in the image space is constructed by the other two image points, which are measured manually on datasets IKONOS_HK and ZIYUAN3_HK. The detailed measuring methods of object points and image points are the same as those illustrated in Section 4.1. The distribution of real control lines in the Hong Kong area is shown in Figure 5.

Real control lines and control lines constructed by control points are respectively used for constructing Control Schemes V and VI. Their distributions are similar to each other. The relative actual accuracies of Control Schemes V and VI by rigorous LBTM are shown in

Table 4. Relative actual accuracies of Control Schemes V and VI by rigorous LBTM.

Control Schemes	Relative Actual Accuracy	Datasets
		IKONOS_HK	ZIYUAN_HK
Control Scheme V	XRMS (pixel)	0.9922	1.4162
	YRMS (pixel)	0.9351	1.3168
Control Scheme VI	XRMS (pixel)	0.9923	1.4165
	YRMS (pixel)	0.9380	1.3172

Note: X_RMS represents the RMS error in the scanning (X) direction of the sensor; Y_RMS represents the RMS error in the flight (Y) direction, which is perpendicular to the scanning direction.

.

As shown in Table 4, the relative actual accuracies of image rectification by rigorous LBTM in Control Schemes V and VI are basically on the same level. Control lines constructed by control points have the same control effect on image rectification as real control lines. Hence, the experimental schemes carried out in Section 4.2 are reliable.

4.4. Influence of Control Lines Number to LBTMs

Dataset IKONOS_HB is used for investigating the geo-positioning accuracy variation of image rectification by different LBTMs with the increase of control lines. The number and layout of single GCP and check points are the same as those in Figure 4c. With the number of control lines increasing from four to twenty at intervals of two, the relative actual accuracy variations of image rectification by different LBTMs (including 8-param LBTM, 6-param LBTM and rigorous LBTM) in the X and Y directions are shown in Figure 6.

In this set of experiments, the relative actual accuracies of image rectification by LBTMs improve with the increase of control lines. The relative actual accuracies of dataset IKONOS_HB rectified by rigorous LBTM stabilized at about 0.5 pixels in both the X and Y directions when the number of control lines reaches eight; while the relative actual accuracies of dataset IKONOS_HB rectified by 8-param LBTM and 6-param LBTM tend to be stable at the level of about six pixels in the X direction and four pixels in the Y direction, respectively, with the number of control lines increasing to twelve. Experimental results demonstrate that rigorous LBTM is more accurate and stable in comparison with other relevant LBTMs, and 6-param LBTM performs slightly better than 8-param LBTM. Besides, the relative actual accuracies of image rectification by the three models in the Y direction are higher than those in the X direction, as illustrated in Section 4.2.

4.5. Influence of Control Lines’ Distribution on LBTMs

To study the influences of the spatial distribution of control lines on the performance of geometric rectification of HRSI by different LBTMs (including 8-param LBTM, 6-param LBTM and rigorous LBTM), five different control schemes based on dataset IKONOS_HB are laid out. The distribution of single GCP and twenty check points is the same, while the distribution of eight control lines is different. Limited by the distribution of GCPs observed in the coverage area of dataset IKONOS_HB, the uniformity and consistency of the distribution of control conditions, including single GCP, control lines and check points, are not strictly demanded. Their detailed spatial distributions are shown in Figure 7.

The relative actual accuracies of Control Schemes VII to XI by different LBTMs (including 8-param LBTM, 6-param LBTM and rigorous LBTM) are shown in

Table 5. Relative actual accuracy of Control Schemes VII to XI by different LBTMs.

Control Schemes	Relative Actual Accuracy	LBTMs
		8-Param LBTM	6-Param LBTM	Rigorous LBTM
Control Scheme VII	XRMS (pixel)	6.5383	6.0626	0.5600
	YRMS (pixel)	4.1577	3.6783	0.4434
Control Scheme VIII	XRMS (pixel)	15.1325	10.3079	0.8513
	YRMS (pixel)	7.6346	5.9687	0.4843
Control Scheme IX	XRMS (pixel)	8.3384	7.5454	0.6724
	YRMS (pixel)	7.2757	5.4367	0.4823
Control Scheme X	XRMS (pixel)	12.7432	9.7182	0.8262
	YRMS (pixel)	4.9636	4.2063	0.4595
Control Scheme XI	XRMS (pixel)	15.6346	11.6248	1.0497
	YRMS (pixel)	9.7341	7.0535	0.6671

Note: X_RMS represents the RMS error in the scanning (X) direction of the sensor; Y_RMS represents the RMS error in the flight (Y) direction, which is perpendicular to the scanning direction.

.

As shown in Table 5, the relative actual accuracies of Control Scheme VII are higher than those of Control Schemes VIII, IX, X and XI. By comparison between Control Schemes VII and VIII, the relative actual accuracies of Control Scheme VIII in the X and Y directions are both decreased significantly, which illustrates that it is significant to increase the elevation differences of control lines for improving the geo-positioning accuracy. By comparison among Control Schemes VII, IX and X, the relative actual accuracies of Control Scheme IX in the Y direction decrease obviously while those in the X direction decrease a little. In contrast to Control Scheme IX, the relative actual accuracies of Control Scheme X in the X direction decrease obviously while those in the Y direction decrease a little. If the control lines are mainly toward a certain direction, the control strength in this direction may be weakened obviously. Therefore, it is meaningful to layout control lines as evenly as possible in the directions. By comparison between Control Schemes VII and XI, the relative actual accuracies of Control Scheme XI in the X and Y directions both decrease obviously. Therefore, it is meaningful to layout control lines as evenly as possible in positions all over the coverage area. The experimental results provide a practice basis and beneficial reference for the layout of control lines.

4.6. Discussion

A rigorous LBTM is proposed in this study for high precision geometric rectification of HRSI based on line features. There is no need to know complicated imaging geometric parameters, like rigorous sensor models do, and no need to collect numerous GCPs, like various non-rigorous mathematical models based on point features do. Rigorous LBTM fully utilizes the merits of line features, by reducing the workloads of implantation and the maintenance of control conditions and raising the efficiency of image rectification procedures significantly. At the same time, the geo- positioning accuracy of HRSIs by rigorous LBTM is about one pixel with at least eight well-distributed control lines. In this paper, a practical processing scheme of image rectification based on rigorous LBTM is put forward. This scheme is highly valued in applications, particularly under the condition that the rigorous imaging geometry or original RPCs are unknowable.

The essence of LBTM is to build a strict conjugate relationship of line features in the image space and the object space, so as to precisely describe the correspondence between imagery and a complicated terrain surface. Compared with existing LBTMs, rigorous LBTM makes several improvements: First of all, rigorous LBTM provides an efficient way of constructing non-linear LBTM. 3D affine LBTM, 2D LBTM and 2D conformal LBTM proposed by Shaker and Shi [32,33,34] are all based on linear equations. The conjugate relationship of unit vectors is built by simple subtraction of corresponding equations of two points on conjugate line features and further unitization. Instead, non-linear rigorous LBTM is constructed based on a generalized point strategy. The strict conjugate relationship between the image point and the object point is removed by adding one variable in the line’s parameter equation. Rigorous LBTM can simulate the complex imaging relationship and terrain conditions more efficiently than linear LBTMs, thus its precision and reliability are higher than those of linear LBTMs. Secondly, rigorous LBTM is constructed based on the relatively strict imaging geometric relationship of point-based rigorous ATM. In contrast, line-based parallel projection, extended parallel projection and DLT models proposed by Elaksher [35,36] are constructed by conjugate unit vectors in the image space and the object space directly in forms that are similar to the relevant PBTMs. Though they are non-linear LBTMs, the relationship between conjugate line features is another mathematical one, not the relatively strict one as in PBTM. Thereby, the precision and reliability of these models may be negatively influenced, as well. Thirdly, rigorous LBTM improves the accuracy and robustness remarkably with minimum cost. It is simple in form and easy to implement. The dominant factors of geometric positioning deviation caused by sensor inclination and terrain variation are compensated by two parameters as much as possible. There is only the need to know the approximate values of equivalent focal length and the tilt angle of the linear array scanner in the scanning direction, then their precise values can be obtained through overall adjustment. In comparison with line-based direct RFM and bias-compensated RFM proposed by Long et al. [24] and Teo [37], rigorous LBTM can reach a relatively accessible precision level with simplified parameters and four control lines at least. Besides, most of the parameters in rigorous LBTM are consistent with those in six-parameter LBTM and rigorous ATM. It is convenient to gain their initial values by the calculation of six-parameter LBTM in the preprocessing step, as well as to rectify HRSI precisely by point-to-point correspondence based on rigorous ATM. Last, but not least, if a higher precision level of image rectification is required, more rigorous and accurate nonlinear LBTMs can be constructed similar to the modeling scheme of rigorous LBTM in this paper. For example, rigorous LBTM can be regarded as an effective preprocessing model, then line features can be introduced into bias-compensated RFM based on the generalized point strategy and least squares overall adjustment. Higher geo-positioning precision and reliability are to be expected.

Based on the study of the construction and solution method of the proposed rigorous LBTM, experiments are carried out systematically in this paper to get a better understanding of its characteristics and applications. The performance of image rectification by rigorous LBTM and other relevant LBTMs and PBTMs on accuracy and robustness are mainly concluded in five aspects: firstly, under the conditions of different sensor types (include IKONOS, GeoEye-1 and ZiYuan-3 satellite), different imaging resolutions (1 m, 0.5 m and 2.1 m) and different terrain variations (include undulated and hilly terrains), the geo-positioning accuracies by rigorous LBTM are all far better than those by relevant LBTMs (include eight-parameter LBTM and six-parameter LBTM), especially in hilly areas; secondly, rigorous LBTM can achieve approximate equivalent geo-positioning accuracies to rigorous ATM, like the accuracy consistency between existing LBTMs and their corresponding PBTMs illustrated by Shaker, Shi [32,33,34] and Elaksher [35,36]; thirdly, the geo-positioning accuracies of LBTMs improve continuously with the number of control lines increase and become stable when control lines reach a certain amount. The convergence rate of accuracies by rigorous LBTM is faster than those by other relevant LBTMs, and the accuracy of IKONOS imagery by rigorous LBTM reaches up to about 0.5 pixels with at least eight evenly-distributed control lines; finally, the geo-positioning accuracy by rigorous LBTM is affected by the spatial distribution of positions, directions and elevation differences of the control lines. Hence, it is meaningful to select control lines distributed as evenly as possible both in the horizontal and vertical directions. Experimental results of eleven control schemes verify the above theoretical analysis about rigorous LBTM and other relevant LBTMs and PBTMs effectively.

All in all, a novel rigorous LBTM is presented in this paper for high precision geometric rectification of HRSI by combining the advantages of previous LBTMs and PBTMs. On the one hand, rigorous LBTM and six-parameter LBTM in the preprocessing method both take the ideas of geometric modeling based on line features in vector form instead of control points, which are applied in previous LBTMs [32,33,34,35,36]. On the other hand, rigorous LBTM transforms rigorous ATM into the line-based form based on a generalized point strategy and provides a more rigorous solution of image rectification based on non-linear LBTM and least squares overall adjustment. Rigorous LBTM is very suitable for image rectification of HRSI; nevertheless, it should be noted that rigorous LBTM is not the most precise one compared with other more complicated LBTMs, such as line-based direct RFM and bias-compensated RFM. The control capability of rigorous LBTM is limited due to its simplified modeling form, leading to geo-positioning accuracy decreases obviously when faced with too large coverage areas or too complicated terrain conditions. Rigorous LBTM is a better choice when taking into account the control work cost, accuracy requirement and coverage areas. Further studies of line-based direct RFM and bias-compensated RFM based on a generalized point strategy are in progress for higher precision demand in actual applications.

5. Conclusions

In this paper, a novel rigorous line-based transformation model has been proposed for high precision geometric rectification of high resolution satellite imagery. The integrated solutions of the modeling method, the parameters’ solving method and the image rectification process are illustrated in detail. Taking line features as the basic control conditions, the rigorous line-based transformation model is constructed from the strict geometric model based on affine transformation utilizing a generalized point strategy. With more than four uniformly-distributed control lines, the parameters of the rigorous line-based transformation model are precisely solved by overall adjustment based on the least squares method, for which the initial values are acquired from the released parameter files of the satellite and the calculation of the six-parameter line-based transformation model in the preprocessing step. Finally, the satellite imagery is processed accurately by forward and inverse solutions of image rectification using the strict geometric model based on affine transformation.

The rigorous line-based transformation model can eliminate the negative impact of elevation parameters on geo-positioning accuracy by taking the correction of sensor inclination and terrain variation into consideration. The equations of the rigorous line-based transformation model are simple, uniform and easy to solve with some modification. Both theoretical analysis and experimental results demonstrate that this model is more strict, accurate and reliable than previous line-based transformation models, particular in rolling areas. With eight control lines at least, the geo-positioning accuracy of IKONOS, GeoEye-1 and ZiYuan-3 imagery rectified by the rigorous line-based transformation model can achieve a level corresponding to the ground resolution on average, and it can be further improved by optimizing the control scheme.

As future work, more rigorous point-based transformation models, such as the rational function model and other improved models, are considered to transform into line-based forms using the generalized point strategy, to improve the geo-positioning accuracy of image rectification. Combined with the characteristics of control lines, the model parameters will be further optimized to remove correlations among them. Their accuracy and reliability need to be further studied, especially in comparison with existing LBTMs. In addition, automatic recognition, extraction and description of line features from satellite imagery and vector maps and the method of introducing them into line-based transformation models will be investigated.