Long Integral Time Continuous Panorama Scanning Imaging Based on Bilateral Control with Image Motion Compensation

: Urban remote sensing with moving carriers enables comprehensive monitoring of an urban area. High spatial resolution and wide covering are always required to improve the performance and e ﬃ ciency of remote sensing. Continuous scanning imaging is a feasible solution. However, imaging motion degrades the performance of a remote sensing system. Rotating motion leads to the loss of key urban morphology information of a panorama imaging. Image translation results in blurry images. For high spatial resolution and high e ﬃ ciency imaging with low illumination condition, such as imaging at dusk, long-focus lens and long integral time must be further utilized, which makes the problem more severe. In this paper, a novel image motion compensation method is proposed to compensate for image rotation and image translation simultaneously. A quantitative description of image motion, including both image rotation and image translation, is ﬁrst developed based on the principle of geometrical optics and then analyzed in detail through numerical simulations. Furthermore, a comprehensive image rotation compensation method is developed based on four-channel bilateral control with sliding mode controller, at the same time image translation compensation is performed according to the quantitative relationship of the motion of the scan mirror and image translation compensator. The experimental results show that the proposed method provides e ﬀ ective compensation for image rotation and image translation. This enables acquisition of high spatial resolution urban panoramic images. of during the scan of of over ms, stage a trigger completed.


Introduction
Urban remote sensing involves the simultaneous use of multiple remote sensing technologies to achieve comprehensive remote sensing monitoring of urban morphology. It provides digital information of urban environmental protection, urban construction management, land use and so on [1][2][3]. It is a critical and necessary technique in the dynamic collection and monitoring of urban spatial information [4,5]. To achieve the flexibility and high efficiency, an airborne carrier, e.g., an unmanned aerial vehicle (UAV), is usually employed to realize the remote sensing, especially to monitor the changed land cover. For high spatial resolution with low illumination condition, such as imaging at dusk, long-focus lens and long integral time are utilized. For wide coverage, scanning motion should be used.
Scanning imaging is used to achieve wider coverage under a limited field of view [6]. Traditional frame-scanning imaging (FSI) realizes scanning imaging by controlling the motion of the entire camera.
This results in multiple drawbacks, including a large moment of inertia, high power consumption, and low efficiency, which is not suitable for large-scale high-efficiency urban remote sensing [7]. Rapid panoramic scanning imaging can be achieved by continuously rotating a scanning mirror installed inside the imaging system. Nonetheless, the swing of the normal line will cause the object image to rotate relative to the imaging medium, which is called image rotation. Such rotation issue will, in turn, affect the stitching of panorama images [8,9]. The causes of image rotation during the imaging process and the associated compensation methods are explored in literature [10].
To meet the increasing demand for high spatial resolution urban remote sensing images, especially for those taken in low light, the integral time of the imaging medium must be increased to improve the signal-to-noise ratio (SNR). The rapid rotation of the scanning mirror will induce a relative motion between the object and imaging medium over a long integral time. This relative motion will result in blurred images and smearing effects, both of which create image translation [11]. Image rotation and image translation can be collectively referred to as image motion. To ensure that the staring image of the target object over a long integral time within the exposure time is sufficiently clear, image motion including image rotation and translation must be compensated for at the same time.
During the imaging process, the movement of the imaging medium within the exposure time can be controlled by a controllable actuator to achieve synchronous image translation compensation. This method, known as continuous scanning step-stare imaging (CSSSI), allows obtaining clear panoramic images over a long integral time with continuous scanning, which achieves higher efficiency than the mode of "step-stop". For a moving carrier of sensors, such as the UAV, high speed continuous scanning imaging not only realizes a wide cover imaging, but also makes higher speed of flight be possible. Wide cover and high speed of moving are very beneficial to reduce the operation time and improving the efficiency of imaging operation. For constantly changed urban areas, high efficiency of imaging operation is the base of high temporal resolution remote sensing. However, the impacts of image rotation and image translation on the quality of panoramic images must be compensated for during the scanning process to obtain high spatial resolution.
Based on the principle of image motion compensation, common methods for compensating image motion can be classified into two categories. One is image processing-based method in which the distorted object image is directly modified by image processing techniques. An image distortion correction based on the mapping relationship between the pixel coordinates of the scan image and ground coordinates was proposed in [12]. In [13], an image post-processing method is used to eliminate blurred images resulting from image translation and to improve image quality. Image processing-based compensation methods do not require any additional compensation units, but the processing time is highly related to the amount of image data. In addition, when performing image rotation compensation, using an image processing-based method will reduce the SNR of the image and result in partial loss of edge features in the field of view. The rotation of the image itself will also affect the positioning accuracy of the target object [14]. These effects pose significant challenges when restoring an original image in the presence of both image rotation and image translation.
Another solution is based on motion control, in which continuous compensation for image motion is realized by directly controlling the synchronized motion of image compensation units and scanning mirrors [15]. In traditional remote sensing cameras, synchronized movement between different motion units is generally achieved by mechanical linkage. Synchronized image rotation compensation between the focal plane and scanning mirror is achieved using a dual-axis gear linkage and dual-motor servo control method [16]. The accuracy of gear transmission and the dual-motor servo control system proposed in this study is 34'' and 1', respectively. However, the synchronization accuracy of such a method is dependent on the accuracy of the mechanical transmission unit itself. An increasing number of motion units makes the design of the mechanical structure increasingly complicated which makes it difficult to maintain consistency in the accuracy of the synchronized components over a long time period of time. In contrast, direct drive solutions are more advantageous for realizing image compensation as they do not require mechanical transmission units. A direct-drive method based on synchronous rotation of a prism is applied to compensate for image rotation [17]. Nevertheless, it is difficult to achieve highly precise synchronized motions by controlling the scanning mirror and de-rotation component independently, which yields limited image compensation effect in practice. A highly precise compensation method for a moving base imaging system is proposed to ensure a synchronized position change of the two rotating units for suppressing any rotation effects of the image [18]. However, this method only enables the exchange of position information of the motion units. Therefore, the effect of synchronous rotation will be affected once any motion unit is influenced by any external disturbance. The image motion caused by the forward movement of an aircraft has been compensated for by controlling a fast mirror with good stiffness and high bandwidth [19].
Because each motion unit often has a different operating principle, size, inertia, and range of motion, the mechanism and scale of disturbances also differ. Therefore, when controlling multiple motion units, it is difficult to achieve an ideal synchronized control if their control systems are designed separately. Moreover, to achieve the better synchronization performance, both position control based on position information of each motion unit and torque control based on torque information applied on the motors of each motion unit should be together. By extending the bilateral control principle proposed for remote teleoperation of robots to the field of optical imaging, we proposed a new idea for image motion compensation based on position and torque control. Such method can theoretically enable perfectly synchronized motions between the scanning mirror and compensation unit, thereby yielding high-precision compensation for image rotation [20]. Nonetheless, this method still employs a traditional proportion-integration-differentiation (PID) control for designing the controller. Such an approach fails to fully consider the robustness of the controller design as well as the effect of image translation on the spatial resolution over a long integral time.
Some studies have achieved compensation for image rotation and image translation by using a rotating imaging medium and controlling the motion of a compensation mirror, respectively. However, the controllers used to compensate for image rotation and image translation were controlled separately. In other words, these approaches did not consider the synchronized motion of different image compensation units. In the CSSSI system, a synchronized control strategy must be designed to integrate these two compensation systems. The synchronized control of multiple image compensating components can theoretically enable continuous and synchronous compensation of different types of image motions without image quality loss from the physical perspective.
In this paper, we propose a novel synchronized control method of multiple image compensation units to resolve the image rotation and image translation issues that occur during the long integral time in the CSSSI process. In particular, a four-channel bilateral control method based on sliding mode control and disturbance observer was proposed for image rotation compensation. The accuracy of the synchronized motion is less affected by model uncertainties and external disturbances. At the same time, we quantitatively analyzed the effects of the movement of the scanning mirror on image translation according to the principle of geometrical optics. Subsequently, equivalent velocity and acceleration feedforward control method based on the nonlinear tracking differentiator was applied to compensate for image translation. Thus, high spatial resolution panoramic imaging with a long integral time is achieved using continuous scanning. This paper is organized as follows. The problem is formulated in Section 2. Then, the control strategy of image motion is discussed in Section 3. The imaging experiments are carried out in Section 4. A discussion is proposed in Section 5. The paper is concluded in Section 6. Figure 1 shows a schematic diagram of a panoramic scanning imaging system. During the process of panoramic scanning imaging, the carrier such as a plane flies along the flight direction at a constant speed, while the imaging system is controlled to scan continuously along the scanning motion direction. In a practical aerial system, image forward motion caused by carrier forward flight, carrier attitude vibrations, line of sight jitter caused by external disturbances and image motion caused by active Remote Sens. 2019, 11, 1924 4 of 20 scanning imaging are all kinds of motions that effect imaging performance. These motions could be considered and compensated separately.  Continuous scanning step-stare imaging enables high spatial resolution and high imaging efficiency. Due to the carrier motion, image forward motion along the carrier flight direction is caused by the relative movement between scene and imaging medium. Image forward motion can be compensated by several methods. Time Delay and Integration (TDI) is an effective and commonly used method. Meanwhile, image motion along scanning motion direction including image ration and image translation is caused by scanning imaging as mentioned above. The working principle of image forward motion and image motion caused by scanning imaging is different, which can be compensated separately by decoupling.

Problem Formulation
In the panoramic scanning imaging system, TDI-based compensation method could be applied to compensate for image forward motion, carrier attitude vibrations can be isolated by gyro-based frame stabilization method, external and internal disturbances can be compensated by disturbance estimated and rejection method. Thus, for high spatial resolution and high efficiency panoramic scanning imaging system, the core problem becomes image motion compensation in scanning direction. However, the image rotation and image translation caused by scanning imaging is coupled complexly, which is hard to be compensated. Therefore, in the following analysis, simulation and experiment, we focus on the motion compensation caused by scanning imaging to obtain high spatial resolution panoramic scanning images. In this paper, we use several actuators to compensate image rotation and image translation caused by scanning imaging. OY axis of the imaging medium, respectively. L is the distance between the object and the scanning mirror, l is the distance between the scanning mirror and the lens, and f is the focal distance of the lens. According to Equations (A6) and (A7) in Appendix A, it is found that the relative motions of the image point are generated in both the x-axis direction and the y-axis direction when the scanning mirror rotates. The rotational and translational motions are coupled, and it is difficult to completely decouple them. The movement of the image point is related to the focal distance of the lens, distance between the object point and the lens, coordinate location of the object point, and rotation angle of the scanning mirror. Continuous scanning step-stare imaging enables high spatial resolution and high imaging efficiency. Due to the carrier motion, image forward motion along the carrier flight direction is caused by the relative movement between scene and imaging medium. Image forward motion can be compensated by several methods. Time Delay and Integration (TDI) is an effective and commonly used method. Meanwhile, image motion along scanning motion direction including image ration and image translation is caused by scanning imaging as mentioned above. The working principle of image forward motion and image motion caused by scanning imaging is different, which can be compensated separately by decoupling.
In the panoramic scanning imaging system, TDI-based compensation method could be applied to compensate for image forward motion, carrier attitude vibrations can be isolated by gyro-based frame stabilization method, external and internal disturbances can be compensated by disturbance estimated and rejection method. Thus, for high spatial resolution and high efficiency panoramic scanning imaging system, the core problem becomes image motion compensation in scanning direction. However, the image rotation and image translation caused by scanning imaging is coupled complexly, which is hard to be compensated. Therefore, in the following analysis, simulation and experiment, we focus on the motion compensation caused by scanning imaging to obtain high spatial resolution panoramic scanning images. In this paper, we use several actuators to compensate image rotation and image translation caused by scanning imaging. Figure 2 shows the schematic of image formation. During the imaging process, the 45 • scanning mirror rotates around the O 0 Z 0 axis, which coincides with the optical axis of the lens below the scanning mirror. The O 0 X 0 axis and O 0 Y 0 axis are parallel to the coordinate axes on the O 1 X 1 axis and O 1 Y 1 axis of the imaging medium, respectively. L is the distance between the object and the scanning mirror, l is the distance between the scanning mirror and the lens, and f is the focal distance of the lens.
According to Equations (A6) and (A7) in Appendix A, it is found that the relative motions of the image point are generated in both the x-axis direction and the y-axis direction when the scanning mirror rotates. The rotational and translational motions are coupled, and it is difficult to completely decouple them. The movement of the image point is related to the focal distance of the lens, distance between the object point and the lens, coordinate location of the object point, and rotation angle of the scanning mirror. Numerical simulations are performed to verify the above results. The simulation parameters are summarized in Table 1. Table 1. Parameters for the simulation the urban remote sensing imaging system.

Parameter
Simulation Value Unit  Table 2.  Figure 3 shows the simulated image movement of the image points on the image plane at exposure times of 0.1, 1, and 5 ms, respectively. As shown in Figure 3, the moving distance of the image point on the image plane increases with an increase in the exposure time. In addition, the moving distance of the image point along the x-axis is significantly greater than that along the y-axis. Further comparison of the points A, B, and C shows that the magnitude of translational movement Numerical simulations are performed to verify the above results. The simulation parameters are summarized in Table 1. Table 1. Parameters for the simulation the urban remote sensing imaging system.

Parameter
Simulation Value Unit distance between the object and the scan mirror L To compare the magnitude of image motion at different positions on the image plane, we analyzed five image points (A-E). The coordinates of these five image points on the image plane are listed in Table 2. Table 2. Coordinates of the image points A-E on the image plane (unit: pixels).

Image Points
Coordinate of the Image Point (x F0 ,y F0 ) Figure 3 shows the simulated image movement of the image points on the image plane at exposure times of 0.1, 1, and 5 ms, respectively. As shown in Figure 3, the moving distance of the image point on the image plane increases with an increase in the exposure time. In addition, the moving distance of the image point along the x-axis is significantly greater than that along the y-axis. Further comparison of the points A, B, and C shows that the magnitude of translational movement of each point is significantly larger than the magnitude of the rotational movement in the exposure time. Therefore, the image translation along the x-axis is the main cause of image blur during the exposure time. The longer the exposure time, the larger the image translation. of each point is significantly larger than the magnitude of the rotational movement in the exposure time. Therefore, the image translation along the x-axis is the main cause of image blur during the exposure time. The longer the exposure time, the larger the image translation.  Table 3 summarizes the values of image translation for points A-E along the x-axis and y-axis when the exposure time is 0.1, 1, and 5 ms. From the simulation results, the movements of image points A-E along the x-axis and y-axis are both less than one pixel when the exposure time is 0.1 ms. In this case, the image translation has minor influences on the imaging quality. However, when the exposure time is 1 ms and 5 ms, the image translation of points A-E along the x-axis is considerably larger than one pixel while that along the y-axis is still less than one pixel. Therefore, when the exposure time is longer than 1 ms, it is necessary to compensate for image translation along the x-axis during the exposure time to obtain a high spatial resolution image. Further, it can also be seen that regardless of location of the point, the image translation along the x-axis does not differ significantly during the same exposure time. Therefore, to simplify the image compensation strategy and ensure high imaging quality in the center of the image plane, the magnitude of image translation of point A along the x-axis is selected as the reference to generate an image translation compensation command for the image translation compensator (ITC) during the exposure time.

Control Strategy of Image Motion
The control strategies used for image motion compensation in the CSSSI system include a four-channel bilateral control-based image rotation compensation and an image translation compensation based on the equivalent velocity and acceleration feedforward control method using the nonlinear tracking differentiator, as shown in Figure 4. Where four-channel bilateral control  Table 3 summarizes the values of image translation for points A-E along the x-axis and y-axis when the exposure time is 0.1, 1, and 5 ms. From the simulation results, the movements of image points A-E along the x-axis and y-axis are both less than one pixel when the exposure time is 0.1 ms. In this case, the image translation has minor influences on the imaging quality. However, when the exposure time is 1 ms and 5 ms, the image translation of points A-E along the x-axis is considerably larger than one pixel while that along the y-axis is still less than one pixel. Therefore, when the exposure time is longer than 1 ms, it is necessary to compensate for image translation along the x-axis during the exposure time to obtain a high spatial resolution image. Further, it can also be seen that regardless of location of the point, the image translation along the x-axis does not differ significantly during the same exposure time. Therefore, to simplify the image compensation strategy and ensure high imaging quality in the center of the image plane, the magnitude of image translation of point A along the x-axis is selected as the reference to generate an image translation compensation command for the image translation compensator (ITC) during the exposure time.

Control Strategy of Image Motion
The control strategies used for image motion compensation in the CSSSI system include a four-channel bilateral control-based image rotation compensation and an image translation compensation based on the equivalent velocity and acceleration feedforward control method using the nonlinear tracking differentiator, as shown in Figure 4. Where four-channel bilateral control means both position information and torque information are applied to transmission and feedback control in the bilateral control system. Four channels refer to the position and torque transmission Remote Sens. 2019, 11,1924 7 of 20 channels from the scan mirror to image rotation compensator, and the position and torque transmission channels from image rotation compensator to the scan mirror, respectively. In Figure 4, J* denotes the nominal equivalent moment of inertia of control motor and C* denotes the controller for different control purpose. The descriptions and explanation of every parameters and controllers is discussed in Appendix B in detail.
Remote Sens. 2019, 11, x FOR PEER REVIEW 7 of 20 means both position information and torque information are applied to transmission and feedback control in the bilateral control system. Four channels refer to the position and torque transmission channels from the scan mirror to image rotation compensator, and the position and torque transmission channels from image rotation compensator to the scan mirror, respectively. In Figure 4, J* denotes the nominal equivalent moment of inertia of control motor and C* denotes the controller for different control purpose. The descriptions and explanation of every parameters and controllers is discussed in Appendix B in detail. On one side, the high-precision de-rotation problem can be decoupled into two independent aspects based on acceleration: first, the scan mirror and image rotation compensator (IRC) have the same rotation responses; second, both these motion units experience the same torques. Through this decomposition, the complex bilateral control structures can be transformed into two general feedback control forms. In this case, a four-channel bilateral control image rotation compensation strategy based on position/torque information is applied to control the synchronous movements of the scan mirror and IRC in order to compensate for the image rotation.
The image translation compensator is then controlled to switch between the two operating modes according to the imaging working sequence. During the exposure time, coordinate transformation is first performed on the position information of the scan mirror according to the results of the geometrical optical analysis. Subsequently, image translation is synchronously compensated by an equivalent velocity and acceleration feedforward control strategy based on the nonlinear tracking differentiator. At the end of the exposure time, a control command is generated to move the image translation compensator to its initial position.
By applying these image rotation compensation and image translation compensation control strategies, the system achieves an equivalent rotation-free and translation-free continuous scanning step-stare imaging during the given exposure time. Then, high spatial resolution urban remote sensing panoramic images can be obtained by splicing the compensated remote sensing images.
A control flow chart of image motion compensation in the continuous scanning step-stare imaging system with a long integral time is shown in Figure 5. The design of controller is discussed in Appendix B in detail. On one side, the high-precision de-rotation problem can be decoupled into two independent aspects based on acceleration: first, the scan mirror and image rotation compensator (IRC) have the same rotation responses; second, both these motion units experience the same torques. Through this decomposition, the complex bilateral control structures can be transformed into two general feedback control forms. In this case, a four-channel bilateral control image rotation compensation strategy based on position/torque information is applied to control the synchronous movements of the scan mirror and IRC in order to compensate for the image rotation.
The image translation compensator is then controlled to switch between the two operating modes according to the imaging working sequence. During the exposure time, coordinate transformation is first performed on the position information of the scan mirror according to the results of the geometrical optical analysis. Subsequently, image translation is synchronously compensated by an equivalent velocity and acceleration feedforward control strategy based on the nonlinear tracking differentiator. At the end of the exposure time, a control command is generated to move the image translation compensator to its initial position.
By applying these image rotation compensation and image translation compensation control strategies, the system achieves an equivalent rotation-free and translation-free continuous scanning step-stare imaging during the given exposure time. Then, high spatial resolution urban remote sensing panoramic images can be obtained by splicing the compensated remote sensing images.
A control flow chart of image motion compensation in the continuous scanning step-stare imaging system with a long integral time is shown in Figure 5. The design of controller is discussed in Appendix B in detail.

Experimental Results
The proposal aims at airborne remote sensing systems, especially the unmanned aerial vehicle carried optical imaging instrument. However, the proposal method is not only feasible in airborne systems, but also useful in a space-borne system or a land-based system. The proposal is a general approach to realize an image rotation free and image translation free continuous scanning step-stare imaging during long integral time. Due to the universality of the proposed technique and high cost of a flight experiment, a land-based device is used in the experiments. Figure 6 shows the experimental principle and device. Two brushless motors were employed: one as the scan motor and the other as the IRC motor. The focal distance of the lens was 300 mm. A Complementary Metal Oxide Semiconductor (CMOS) detector was used as the imaging element. The rotation angle of the scan mirror and IRC were measured using an optical-electricity encoder with a resolution of 0.00007°. The overall algorithms were implemented on a computer by programming in Windows-RTX, which is a real-time operating system. The sampling time was 1 ms. Other parameters of the proposed control algorithm are listed in Table 4.

Experimental Results
The proposal aims at airborne remote sensing systems, especially the unmanned aerial vehicle carried optical imaging instrument. However, the proposal method is not only feasible in airborne systems, but also useful in a space-borne system or a land-based system. The proposal is a general approach to realize an image rotation free and image translation free continuous scanning step-stare imaging during long integral time. Due to the universality of the proposed technique and high cost of a flight experiment, a land-based device is used in the experiments. Figure 6 shows the experimental principle and device. Two brushless motors were employed: one as the scan motor and the other as the IRC motor. The focal distance of the lens was 300 mm. A Complementary Metal Oxide Semiconductor (CMOS) detector was used as the imaging element. The rotation angle of the scan mirror and IRC were measured using an optical-electricity encoder with a resolution of 0.00007 • . The overall algorithms were implemented on a computer by programming in Windows-RTX, which is a real-time operating system. The sampling time was 1 ms. Other parameters of the proposed control algorithm are listed in Table 4.     Figure 7 shows the urban remote sensing mosaic image of two images with image rotation compensation only under long integral time. As shown in the image, image rotation could be compensated, but increasing integral time results in severe image blurring and smearing effects, which significant degrade the quality of the panoramic image and result in failure on the panoramic image mosaic of more than two images. Therefore, image translation compensation is necessary during the imaging process in CSSSI with a long integral time. Case I. Urban remote sensing mosaic images with image rotation compensation under long integral time. Figure 7 shows the urban remote sensing mosaic image of two images with image rotation compensation only under long integral time. As shown in the image, image rotation could be compensated, but increasing integral time results in severe image blurring and smearing effects, which significant degrade the quality of the panoramic image and result in failure on the panoramic image mosaic of more than two images. Therefore, image translation compensation is necessary during the imaging process in CSSSI with a long integral time. Case II. Urban remote sensing panoramic images obtained with image rotation compensation and image translation compensation. Figure 8 shows an urban remote sensing panoramic image under long integral time. Due to the limitation of the display length of the article, here we make a panoramic mosaic of five images. Under the proposed image motion compensation strategy, a clear panoramic remote sensing image of the urban area without the loss of key urban morphology information is obtained, the digital information of the urban buildings and other urban constructions is shown.  Figure 9 shows the motion response of each motion unit during the urban remote sensing panoramic imaging process with image rotation compensation and image translation compensation. Figure 9a shows the motion response of the scan mirror and IRC in 10 s. Under the proposed four-channel bilateral control method based on the sliding mode control, the scan mirror and IRC can realize the high precision image rotation compensation. Figure 9b shows the corresponding ITC control command and position response. During the scan imaging stage, the control system controls the reciprocating motion of the ITC under the proposed equivalent velocity and acceleration feedforward control method based on the nonlinear tracking differentiator. When the speed of the ITC matches that of the scan mirror according to Equation (A7), the control system generates a trigger impulse which controls the system for imaging, the image translation is compensated and high spatial resolution panorama images are obtained. Figure 9c shows a partially enlarged drawing of the IRC velocity and ITC equivalent velocity. It can be seen that during the scan imaging stage, the  Case I. Urban remote sensing mosaic images with image rotation compensation under long integral time. Figure 7 shows the urban remote sensing mosaic image of two images with image rotation compensation only under long integral time. As shown in the image, image rotation could be compensated, but increasing integral time results in severe image blurring and smearing effects, which significant degrade the quality of the panoramic image and result in failure on the panoramic image mosaic of more than two images. Therefore, image translation compensation is necessary during the imaging process in CSSSI with a long integral time. Case II. Urban remote sensing panoramic images obtained with image rotation compensation and image translation compensation. Figure 8 shows an urban remote sensing panoramic image under long integral time. Due to the limitation of the display length of the article, here we make a panoramic mosaic of five images. Under the proposed image motion compensation strategy, a clear panoramic remote sensing image of the urban area without the loss of key urban morphology information is obtained, the digital information of the urban buildings and other urban constructions is shown.  Figure 9 shows the motion response of each motion unit during the urban remote sensing panoramic imaging process with image rotation compensation and image translation compensation. Figure 9a shows the motion response of the scan mirror and IRC in 10 s. Under the proposed four-channel bilateral control method based on the sliding mode control, the scan mirror and IRC can realize the high precision image rotation compensation. Figure 9b shows the corresponding ITC control command and position response. During the scan imaging stage, the control system controls the reciprocating motion of the ITC under the proposed equivalent velocity and acceleration feedforward control method based on the nonlinear tracking differentiator. When the speed of the ITC matches that of the scan mirror according to Equation (A7), the control system generates a trigger impulse which controls the system for imaging, the image translation is compensated and high spatial resolution panorama images are obtained. Figure 9c shows a partially enlarged drawing of the IRC velocity and ITC equivalent velocity. It can be seen that during the scan imaging stage, the  Figure 9 shows the motion response of each motion unit during the urban remote sensing panoramic imaging process with image rotation compensation and image translation compensation. Figure 9a shows the motion response of the scan mirror and IRC in 10 s. Under the proposed four-channel bilateral control method based on the sliding mode control, the scan mirror and IRC can realize the high precision image rotation compensation. Figure 9b shows the corresponding ITC control command and position response. During the scan imaging stage, the control system controls the reciprocating motion of the ITC under the proposed equivalent velocity and acceleration feedforward control method based on the nonlinear tracking differentiator. When the speed of the ITC matches that of the scan mirror according to Equation (A7), the control system generates a trigger impulse which controls the system for imaging, the image translation is compensated and high spatial resolution panorama images are obtained. Figure 9c shows a partially enlarged drawing of the IRC velocity and ITC equivalent velocity. It can be seen that during the scan imaging stage, the velocity of IRC and the equivalent velocity of IRC can match for over 20 ms, during which stage a trigger imaging is completed.

Discussion
This paper describes a novel image motion compensation method for continuous scanning step-stare imaging in urban remote sensing. Through quantitative analysis, system modeling, formulation and imaging experiments, it can be verified that the proposed method can effectively compensate for image rotation and image translation caused by continuous scanning imaging under the condition of low illumination. This enables acquisition of high spatial resolution urban panoramic images.
The limitation of the proposal is the constraint of the motion range of the actuators and the field of view of the lens. Faster scanning speed and longer integral time need wider motion range of the actuator. However, the sensor should be always in the field of view of the lens. Therefore, the motion range is limited. In the future, higher speed of the controlled actuator and wider field of view lens design will be focused on.

Discussion
This paper describes a novel image motion compensation method for continuous scanning step-stare imaging in urban remote sensing. Through quantitative analysis, system modeling, formulation and imaging experiments, it can be verified that the proposed method can effectively compensate for image rotation and image translation caused by continuous scanning imaging under the condition of low illumination. This enables acquisition of high spatial resolution urban panoramic images.
The limitation of the proposal is the constraint of the motion range of the actuators and the field of view of the lens. Faster scanning speed and longer integral time need wider motion range of the actuator. However, the sensor should be always in the field of view of the lens. Therefore, the motion range is limited. In the future, higher speed of the controlled actuator and wider field of view lens design will be focused on.

Conclusions
In this paper, a novel four-channel bilateral control method based on sliding mode control and a disturbance observer for image rotation compensation and an equivalent velocity and acceleration feedforward control method based on the nonlinear tracking differentiator for image translation compensation were proposed to resolve the image motion problem in the continuous scanning step-stare imaging system for urban remote sensing imaging.
The method proposed in this paper does not require any change in the optical structure of the scanning system. Based on the modern control theory and the sliding mode control theory, a four-channel bilateral sliding mode control method was adopted to perform image rotation compensation based on position and torque information. Subsequently, a quantitative analysis of the image rotation and image translation process of the scanning imaging system is performed according to the principles of geometrical optics. The image translation compensation component is controlled correspondingly according to the analysis results to realize high-precision image translation compensation. The algorithm proposed is validated in a real-time imaging experiment. The experimental results demonstrate the effectiveness of the proposed method in realizing efficient and high-precision remote sensing imaging. Furthermore, the experiments proved that the images obtained with image motion compensation can provide urban morphology information with improved spatial coverage and spatial resolution. Therefore, the proposed image motion compensation method can satisfy various remote sensing imaging requirements for collecting and monitoring dynamic high spatial resolution urban spatial information.

Conflicts of Interest:
The authors declare no conflict of interest.

Appendix A. Working Principle of the System
Appendix A.1. Working Principle of the System Figure A1 shows the schematics of the working principles for the frame-scanning imaging (FSI) mode and continuous scanning imaging mode (CSI). For the FSI mode shown in Figure A1a, the movement of the camera is controlled by a series of instructions given by the control system. Once the camera arrives at the target imaging area, it will maintain a staring state and start collecting remote sensing images. For the CSI mode shown in Figure A1b, a fast scanning imaging, realized by continuously rotating the scanning mirror, is performed to increase the coverage width within the limited field of view under a strong ambient light and short exposure time.
However, for poor photographing environments (e.g., low environmental light) and high requirements for the imaging spatial resolution, the imaging integral time must be increased to improve the image SNR. Under such conditions, simply using the CSI mode will cause image translation, which results in image blur. Therefore, to ensure a high image spatial resolution with a long integral time, the relative positional relationship between the imaging medium and the object must be strictly maintained to be constant. In other words, the imaging system is maintained in the staring state. The working principle of the traditional stepping-based staring imaging mode is to first move the scanning mirror to a predetermined position by controlling its acceleration and deceleration and then maintain zero velocity during the time of exposure. After the exposure step, the scanning mirror is again controlled to move to a new location associated with the next imaging frame. However, this stepping-based staring scanning imaging method has poor imaging efficiency. The frequent start and stop of the motor results in a lot of pressure on the control strategy. To improve the scanning efficiency, a rigid and high-frequency voice coil unit or a highly precise piezoceramic motion unit can be added to the imaging medium. When the scanning mirror is rotating continuously during the exposure time, the controlling unit will push the imaging medium to compensate for the image motion, thereby ensuring a constant relative position between the object image and the imaging medium. This imaging approach is called a continuous scanning step-stare imaging (CSSSI). rotating continuously during the exposure time, the controlling unit will push the imaging medium to compensate for the image motion, thereby ensuring a constant relative position between the object image and the imaging medium. This imaging approach is called a continuous scanning step-stare imaging (CSSSI).
(a) (b) Figure A1. Schematic of the working principles: (a) the frame-scanning imaging (FSI) system; (b) the continuous scanning imaging (CSI) system. Figure A2 shows the schematic of the working principle of the continuous scanning step-stare imaging mode. The scan mirror is controlled to rotate to realize a panorama scanning. An image rotation compensator (IRC) is designed to eliminate the image rotation. An image translation compensator (ITC) is employed to eliminate the image motion and to realize an equivalent stare during integral time. To ensure high imaging quality, a high-precision synchronous control strategy must be investigated to synchronize the control of the actuators.
. Figure A2. Imaging schematic diagrams of the continuous scanning step-stare imaging mode.
The working sequence of the scan mirror, ITC, IRC, optical axis, and trigger signal in CSSSI system is shown in Figure A3. The scanning mirror maintains a constant angular scanning speed, s ω , during the scanning state, while IRC rotates at the same angular speed, s ω , as the scan mirror to compensate for the image rotation. After the velocity of scanning mirror becomes stable, the ITC is accelerated by the control system to move in the reverse direction of the imaging medium at the same velocity. Once the velocity of ITC is achieved, a trigger signal is then emitted, and the ITC compensates for the image translation at a constant speed so that the optical axis always remains Figure A1. Schematic of the working principles: (a) the frame-scanning imaging (FSI) system; (b) the continuous scanning imaging (CSI) system. Figure A2 shows the schematic of the working principle of the continuous scanning step-stare imaging mode. The scan mirror is controlled to rotate to realize a panorama scanning. An image rotation compensator (IRC) is designed to eliminate the image rotation. An image translation compensator (ITC) is employed to eliminate the image motion and to realize an equivalent stare during integral time. To ensure high imaging quality, a high-precision synchronous control strategy must be investigated to synchronize the control of the actuators.
Remote Sens. 2019, 11, x FOR PEER REVIEW 13 of 20 rotating continuously during the exposure time, the controlling unit will push the imaging medium to compensate for the image motion, thereby ensuring a constant relative position between the object image and the imaging medium. This imaging approach is called a continuous scanning step-stare imaging (CSSSI).
(a) (b) Figure A1. Schematic of the working principles: (a) the frame-scanning imaging (FSI) system; (b) the continuous scanning imaging (CSI) system. Figure A2 shows the schematic of the working principle of the continuous scanning step-stare imaging mode. The scan mirror is controlled to rotate to realize a panorama scanning. An image rotation compensator (IRC) is designed to eliminate the image rotation. An image translation compensator (ITC) is employed to eliminate the image motion and to realize an equivalent stare during integral time. To ensure high imaging quality, a high-precision synchronous control strategy must be investigated to synchronize the control of the actuators.
. Figure A2. Imaging schematic diagrams of the continuous scanning step-stare imaging mode.
The working sequence of the scan mirror, ITC, IRC, optical axis, and trigger signal in CSSSI system is shown in Figure A3. The scanning mirror maintains a constant angular scanning speed, s ω , during the scanning state, while IRC rotates at the same angular speed, s ω , as the scan mirror to compensate for the image rotation. After the velocity of scanning mirror becomes stable, the ITC is accelerated by the control system to move in the reverse direction of the imaging medium at the same velocity. Once the velocity of ITC is achieved, a trigger signal is then emitted, and the ITC compensates for the image translation at a constant speed so that the optical axis always remains Figure A2. Imaging schematic diagrams of the continuous scanning step-stare imaging mode.
The working sequence of the scan mirror, ITC, IRC, optical axis, and trigger signal in CSSSI system is shown in Figure A3. The scanning mirror maintains a constant angular scanning speed, ω s , during the scanning state, while IRC rotates at the same angular speed, ω s , as the scan mirror to compensate for the image rotation. After the velocity of scanning mirror becomes stable, the ITC is accelerated by the control system to move in the reverse direction of the imaging medium at the same velocity. Once the velocity of ITC is achieved, a trigger signal is then emitted, and the ITC compensates for the image translation at a constant speed so that the optical axis always remains stationary in space. During the exposure time t e , the imaging system maintains the staring state, and the image of the target object is formed. After the image is formed, ITC moves back to its initial position under a position control command, which marks the end of a photographing cycle T.
Remote Sens. 2019, 11, x FOR PEER REVIEW 14 of 20 stationary in space. During the exposure time e t , the imaging system maintains the staring state, and the image of the target object is formed. After the image is formed, ITC moves back to its initial position under a position control command, which marks the end of a photographing cycle T. Figure A3. Working sequence of each motion unit in the continuous scanning step-stare imaging system.

A.2. Formation Principle of Imaging Motion
Assume that the unit normal vector of the scanning mirror in the coordinate system 0 0 0 X Y Z at the initial scanning state is [0, ( 2 / 2), ( 2 / 2)] T N = − − , the incident light vector along the incident The reflection law in three-dimensional space is According to Equation (A1), the emergent light vector 0 In the imaging system, the incident light focuses on the image plane through the lens. The coordinates of the image point 0 F on the image plane are According to Equation (A1), the corresponding emergent light vector '' 0 A after the rotation of the scanning mirror is given by Figure A3. Working sequence of each motion unit in the continuous scanning step-stare imaging system.

Appendix A.2. Formation Principle of Imaging Motion
Assume that the unit normal vector of the scanning mirror in the coordinate system X 0 Y 0 Z 0 at the initial scanning state is N = [0, −(

the incident light vector along the incident light
The reflection law in three-dimensional space is According to Equation (A1), the emergent light vector A 0 of A 0 is In the imaging system, the incident light focuses on the image plane through the lens. The coordinates of the image point F 0 on the image plane are Assume that the scanning mirror rotates for an angle of θ s about the O 0 Z 0 axis, the normal line of the mirror plane then becomes N , which is given by According to Equation (A1), the corresponding emergent light vector A 0 after the rotation of the scanning mirror is given by x E0 cos 2 θ s + y E0 sin θ s cos θ s + z E0 sin θ s x E0 sin θ s cos θ s + y E0 sin 2 θ s − z E0 cos θ s x E0 sin θ s − y E0 cos θ s After the scanning, the coordinates of the corresponding image points on the image plane are given by During exposure time t e , the rotation angle of the scanning mirror is θ s = ω s t e . According to Equations (A3) and (A6) and by substituting the coordinates of point A (x F0_A , y F0_A ) = (0, 0) into the equations, the relationship between the actual rotational speed of the scanning mirror measured by the rotation sensor and image translation compensation magnitude by the ITC is obtained as:

Appendix B. Controller Design
The image rotation compensation and image translation compensation control strategies ensure high-precision synchronization of the motions for all three units.
To achieve high-precision synchronization between the motion units, it is necessary to first ensure high-precision motion for each unit independently. Since the remote sensing imaging system is usually mounted on a moving carrier base such as an airplane, the working environment is relatively complicated. External disturbances such as carrier vibrations and wind resistance as well as internal disturbances such as friction, mass imbalance, and wire-wound friction will all affect system performance. The actual control object will also inevitably be affected by uncertainties. Therefore, it cannot be replaced by an accurate mathematical model. Let G in (s) be the nominal model of the actual control plant G ip (s). The models are expressed using transfer function. Then, the difference between the actual and nominal models of the system is dynamic model uncertainty ∆ i (s), (i = s, r, m). The relationship between these models is described by Then, the controlled plant can be modeled as: where Θ i is the control output, J in and B in denote the nominal equivalent moment of inertia and nominal equivalent motor damping coefficient respectively. D ext i is the external disturbance. The expressions of Θ i , U i , and D ext i in the time domain are θ i (t), u i (t), and d ext i (t), respectively. The subscripts s, r, and m represent the scan mirror motor, IRC motor, and ITC motor, respectively.
The external disturbances D ext i and model uncertainties ∆ i (s) can be collectively referred to as the equivalent disturbance D i of the system.
Then, the dynamic model of the actuators can be rewritten as A robust disturbance observer can compensate for the equivalent disturbances. The purpose of the robust compensators is to normalize the dynamics from U i (i = s, r, m) to Θ i as In this work, a first-order disturbance observer is used as the robust compensator. The same observer is used for the motion torque information estimator of each component. Figure A4 shows a first-order disturbance observer and the corresponding equivalent model after compensation. Herein, Y i is either the angular position output of the scan mirror or IRC or the displacement output of the ITC.
as the equivalent disturbance i D of the system. In this work, a first-order disturbance observer is used as the robust compensator. The same observer is used for the motion torque information estimator of each component. Figure B1 shows a first-order disturbance observer and the corresponding equivalent model after compensation. Herein, i Y is either the angular position output of the scan mirror or IRC or the displacement output of the ITC. According to the principle of superposition, the system control output i Y shown in Figure B1 can be expressed as Therefore, the transfer function from the control parameter ' i U to the output i Y and the transfer function from the system i D to the output i Y are given by Figure A4. Block diagram of the disturbance observer.
According to the principle of superposition, the system control output Y i shown in Figure A4 can be expressed as Therefore, the transfer function from the control parameter U i to the output Y i and the transfer function from the system D i to the output Y i are given by Based on Equations (A13)-(A16), it is observed that the filter Q i (s) is very important to the design of the disturbance observer. Assuming that Q i (s) is an ideal low-pass filter, Q i (s) = 1 for low frequencies and Q i (s) = 0 for high frequencies.
At low frequencies, it follows that This result indicates that at low frequencies, even if G ip (s) G in (s), the existence of the disturbace observer ensures that the response of the actual control plant is consistent with the expected response. In other words, the controller is robust to changes in the parameters of the control plant. G D i Y i (s) = 0 also indicates that the disturbance observer can completely suppress the equivalent disturbances in the low frequency band.
At high frequencies, Q i (s) = 0, then This suggests that the disturbance observer does not have any inhibitory effects on model uncertainty and equivalent disturbances at high frequencies.
Subsequently, we can derive the sensitivity function of the system as Therefore, the complementary sensitivity function is given by According to the robust stability theorem, the sufficient and necessary condition for ensuring robust stability of the system is where · ∞ is the norm. With the disturbance observer, the uncertainty and disturbances are compensated. Then, controllers should be designed. As shown in Figure 4, C 1 and C 2 are the position controllers; C 3 , C 4 , C 5 , and C 6 are the torque controllers; and C 7 and C 8 are the position and velocity controllers of the ITC, respectively.
The position and torque controls of the bilateral control scheme can be decomposed into two independent control tasks [21]. Therefore, the control target of the bilateral control system can be transformed into two general feedback forms. Thus, the position and torque controllers are designed separately in this study. The torque controllers are designed as where K t is the torque gain. In this work, the control strategy of the scan mirror is given by where Θ sr is the position command of the scan mirror, and T s is the control value of the torque controller. Θ sr and T s are the expressions of the scan command θ sr and torque τ s in the frequency domain, respectively. The torque controller of the scan mirror is designed as C s (s) = K ps + K vs s (A24) The velocity compensators C vs and C vr are used to ensure that the scan motor and IRC motor have the same damping coefficients. Their velocity controllers can be described by Thereafter, both the scan motor and IRC motor will share the same damping coefficient. This suggests that the transfer function between the control input and output becomes Then, position controllers C 1 and C 2 are designed. In remote sensing imaging, the increasing demand of more accurate panoramic image poses greater requirements on the synchronous motion accuracy of the scan mirror and IRC. Traditional control strategies such as proportional integral derivative control (PID) that only use the proportional, integral and derivative of the control error as the control law are insufficient to meet the requirements of high precision control systems. Combining modern control theory with the sliding mode control (SMC) method, this study adopted SMC for controller design in C 1 and C 2 in a four-channel bilateral control system of the scan mirror and IRC. The left part of Figure 4 shows the block diagram of the four-channel bilateral SMC of the scan mirror and IRC.
SMC is essentially a special type of nonlinear control. It does not depend on the system structure but maintains a sliding trajectory according to the preset "sliding mode" based on the current state of the system. Compared to the PID method, the SMC is advantageous in terms of its fast response, insensitivity to variation in parameters and external disturbances, and simple implementation. In this system, there is still disturbance residual, although the disturbance observer compensated most of the disturbance. The strong robustness of SMC method achieves better performance under the disturbance residual.
In the time domain, the position errors in the bilateral control of the scan mirror e s (t) and IRC e r (t) (for simplicity, (t) is neglected in the following equations) are defined as e s = θ r − θ s , e r = θ s − θ r (A27) Then, the velocity error of these two components can be expressed as Here, z s and z r are the sliding mode. After the system reaches the sliding mode surface, the position error of the IRC and the scan mirror gradually converges to zero. The position controllers C 1 and C 2 used in the four-channel bilateral control system in this study are designed as C 1 = b 1 z r + b 2 sign(z r ), C 2 = b 1 z s + b 2 sign(z s ) (A30) Remote Sens. 2019, 11, 1924 19 of 20 where, b 1 and b 2 are constants and sign(·) is a symbolic function. Therefore, we can derive that C 1 = −C 2 . The control strategy described in Equation (A30) ensures that the system exhibits a sliding mode. According to Equations (A22), (A26), and (A30), the dynamic equation of the SMC system of the scan mirror and IRC can be derived as The stability of the system given in Equation (A33) can be determined according to the Lyapunov stability criterion.
The control block diagram of the ITC is shown in the right part of Figure 4. With the disturbance observer, the actual control model of the ITC control system can be considered to be consistent with its nominal model. Based on this, an equivalent speed and acceleration feedforward controller with velocity information from the equivalent nonlinear tracking differential estimator is designed, which realizes a high-precision steady-state tracking performance of the ITC at low frequencies. Generally, the differential algorithms for signal differentiation are not accurate with a weak resistance to disturbances. Their sensitivity to signal noise increases substantially with an increasing sampling period. A nonlinear tracking differentiator is used in this paper [22]. This method is insensitive to signal distortion, avoiding transient influence from command distortion and ensuring a highly stable control system.
The controllers of the ITC are designed as C 7 = K pm , C 8 = K vm (A34) Therefore, the control system shown in the right part of Figure 4 is a typical proportionaldifferential control system with velocity and acceleration feedforward.
During actual operation, the motion control command of the ITC switches with the system working state. After the scan mirror rotates at a constant speed, the image rotation is compensated by the IRC. Simultaneously, an acceleration command y p is generated by the control system according to the rotation velocity of the scan mirror, which will control the ITC to accelerate. When the ITC is accelerated to the expected velocity, it will then switch to the image translation compensation state. Meanwhile, the control system will issue a trigger signal to commence imaging. When the imaging process is complete, the ITC will be switched back to the return-to-zero state and the control system will control the ITC to return to its initial position under a smoothened command. This marks the end of one imaging cycle.
According to above design, acceleration, constant velocity movement, and switching back to the return-to-zero state all involve transitioning from one stable state to another. The control strategy adopted in this study reserves enough dwell time for the ITC to switch between different moving states. Therefore, the ITC can be controlled stably during all processes.