# Side-Scan Sonar Image Mosaic Using Couple Feature Points with Constraint of Track Line Positions

^{1}

^{2}

^{3}

^{*}

## Abstract

**:**

## 1. Introduction

## 2. SSS Image Mosaic Method

#### 2.1. Coarsely Mosaicking Process Based on the Image Pixel Coordinates

#### 2.2. Refined Adjustment Based on the CFPs and Track Line Positions

#### 2.2.1. Determination and Segment of the Overlapping Area

#### 2.2.2. Image Registration in the Segmented Overlapping Areas

#### 2.2.3. The Constraint of Track Line Positions

#### 2.2.4. Adjustment Model Using CFPs and Track Line Positions

_{i}, y

_{i}) are given in the plane, a TPS function can be defined in Equation (1).

_{i}and b

_{i}are the transformation parameters to be solved by this equation.

- (1)
- The local coordinate dislocations in segmented overlapping areas are considered by using the CFPs when establishing the adjustment model.
- (2)
- The transformed sensed image will remain stable globally because of using the track line positions as the constraint.
- (3)
- The established model based on TPS function can satisfy the above two requirements.

#### 2.2.5. Gap Filling and Image Fusion

#### 2.3. The SSS Image Mosaic Process

## 3. Experiments and Analysis

#### 3.1. Experimental Data

#### 3.2. Mosaic Process Using Adjacent Strips

#### 3.3. Evaluation of the Mosaic Method

#### 3.3.1. The Consistency of CFP Coordinates

#### 3.3.2. The Variation of Track Line Positions

#### 3.4. Comparison of the Proposed Method to Others

#### 3.4.1. Comparison Experiment 1

#### 3.4.2. Comparison Experiment 2

#### 3.5. Multi-Strip Image Mosaics and Analysis

## 4. Discussion

#### 4.1. Image Quality and Preprocessing

#### 4.2. Position Accuracy of the Mosaicked Image

_{layback}is the projected horizontal length of the cable, L is the cable length, h is the height of the GPS receiver above the sea surface, f

_{d}is the towfish depth, A is the heading data that can be recorded by the compass, and (x

_{towfish}, y

_{towfish}) and (x

_{vessel}, y

_{vessel}) are separately the coordinates of the towfish and the vessel.

#### 4.3. Impact of Overlapping Ratios

#### 4.4. Interpolation to Fill Gap

## 5. Conclusions

## Author Contributions

## Funding

## Acknowledgments

## Conflicts of Interest

## References

- Philippe, B. The Handbook of Sidescan Sonar; Springer: Berlin/Heidelberg, Germany; New York, NY, USA, 2009; ISBN 978-3-540-42641-7. [Google Scholar]
- Lurton, X.; Jackson, D. An Introduction to Underwater Acoustics, 2nd ed.; Springer-Praxis: New York, NY, USA, 2008; ISBN 3540429670. [Google Scholar]
- Cervenka, P.; Moustier, C.D. Sidescan sonar image processing techniques. IEEE J. Ocean. Eng.
**1993**, 18, 108–122. [Google Scholar] [CrossRef] - Fakiris, E.; Papatheodorou, G.; Geraga, M.; Ferentinos, G. An Automatic Target Detection Algorithm for Swath Sonar Backscatter Imagery, Using Image Texture and Independent Component Analysis. Remote Sens.
**2016**, 8, 373. [Google Scholar] [CrossRef] - Wang, X.; Zhao, J.; Zhu, B.; Jiang, T.; Qin, T. A Side Scan Sonar Image Target Detection Algorithm Based on a Neutrosophic Set and Diffusion Maps. Remote Sens.
**2018**, 10, 295. [Google Scholar] [CrossRef] - Coiras, E.; Petillot, Y.; Lane, D. Multiresolution 3-D Reconstruction from Side-Scan Sonar Images. IEEE Trans. Image Proc.
**2007**, 16, 382–390. [Google Scholar] [CrossRef] - Zhao, J.; Shang, X.; Zhang, H. Reconstructing Seabed Topography from Side-Scan Sonar Images with Self-Constraint. Remote Sens.
**2018**, 10, 201. [Google Scholar] [CrossRef] - Yuan, X.; Martínez-Ortega, J.-F.; Fernández, J.A.S.; Eckert, M. AEKF-SLAM: A New Algorithm for Robotic Underwater Navigation. Sensors
**2017**, 17, 1174. [Google Scholar] [CrossRef] [PubMed] - Chavez, P.S.C., Jr; Isbrecht, J.A.; Galanis, P.; Gabel, G.L.; Sides, S.C.; Soltesz, D.L.; Ross, S.L.; Velasco, M.G. Processing, mosaicking and management of the monterey bay digital sidescan-sonar images. Mar. Geol.
**2002**, 181, 305–315. [Google Scholar] [CrossRef] - Reis, J.; Morgado, M.; Batista, P.; Oliveira, P.; Silvestre, C. Design and Experimental Validation of a USBL Underwater Acoustic Positioning System. Sensors
**2016**, 16, 1491. [Google Scholar] [CrossRef] [PubMed] - Krzysztof, N.; Aleksander, N. The Positioning Accuracy of BAUV Using Fusion of Data from USBL System and Movement Parameters Measurements. Sensors
**2016**, 16, 1279. [Google Scholar] [CrossRef] [PubMed] - Wang, A. Research on 3D Seafloor Terrian Recovery from the Side Scan Sonar Image; Wuhan University: Wuhan, China, 2014. [Google Scholar]
- Cobra, D.; Oppenheim, A.; Jaffe, J. Geometric distortions in side-scan sonar images: A procedure for their estimation and correction. IEEE J. Ocean. Eng.
**1992**, 17, 252–268. [Google Scholar] [CrossRef] - Clarke, J. Dynamic Motion Residuals in Swath Sonar Data: Ironing out the Creases. Int. Hydrogr. Rev.
**2003**, 4, 6–23. [Google Scholar] - Cervenka, P.; Moustier, C.; Lonsdale, P. Geometric corrections on sidescan sonar images based on bathymetry: Application with SeaMARC II and Sea Beam data. Mar. Geophys. Res.
**1995**, 17, 217–219. [Google Scholar] [CrossRef] - Cervenka, P.; Moustier, C. Postprocessing and corrections of bathymetry derived from sidescan sonar systems: Application with SeaMARC II. IEEE J. Ocean. Eng.
**1994**, 19, 619–629. [Google Scholar] [CrossRef] - Barrodale, I.; Kuwahara, R.; Poeckert, R.; Skea, D. Side-scan sonar image processing using thin plate splines and control point matching. Numer. Algorithms
**1993**, 5, 85–98. [Google Scholar] [CrossRef] - Daniel, S.; Leannec, F.; Roux, C.; Soliman, B. Side-scan sonar image matching. IEEE J. Ocean. Eng.
**1998**, 23, 245–259. [Google Scholar] [CrossRef] - Chailloux, C.; Caillec, J.; Gueriot, D.; Zerr, B. Intensity-Based Block Matching Algorithm for Mosaicing Sonar Images. IEEE J. Ocean. Eng.
**2011**, 36, 627–645. [Google Scholar] [CrossRef] - Schwind, P.; Suri, S.; Reinartz, P.; Siebert, A. Applicability of the SIFT operator to geometric SAR image registration. Int. J. Remote Sens.
**2010**, 31, 1959–1980. [Google Scholar] [CrossRef] - Zhao, J.; Wang, A.; Zhang, H.; Wang, X. Mosaic method of side-scan sonar strip images using corresponding features. IET Image Proc.
**2013**, 7, 616–623. [Google Scholar] [CrossRef] - Ye, X.; Li, P.; Zhang, J.; Shi, J.; Guo, S. A feature-matching method for side-scan sonar images based on nonlinear scale space. J. Mar. Sci. Technol.
**2016**, 21, 38–47. [Google Scholar] [CrossRef] - Tao, W.; Liu, Y. Combined imaging matching method of side scan sonar images with prior position knowledge. IET Image Proc.
**2018**, 12, 194–199. [Google Scholar] [CrossRef] - Burt, P.; Adelson, E. A Multiresolution spline with application to image mosaics. ACM Trans. Graph.
**1983**, 2, 217–236. [Google Scholar] [CrossRef] - Davies, J.; Baxter, J.; Bradley, M.; Connor, J.; Khan, E.; Murray, W.; Sanderson, C. Marine Monitoring Handbook; Joint Nature Conservation Committee: Peterborough, UK, 2001; ISBN 1-86107-5243. [Google Scholar]
- Bay, H.; Ess, A.; Tuytelaars, T.; Gool, L. Speeded-Up Robust Features (SURF). Comput. Vis. Image Underst.
**2008**, 110, 404–417. [Google Scholar] [CrossRef] - Luo, J.; Gwun, O. A Comparison of SIFT, PCA-SIFT and SURF. Int. J. Image Proc.
**2013**, 3, 143–152. [Google Scholar] - Brook, A.; Bendor, E. Automatic Registration of Airborne and Spaceborne Images by Topology Map Matching with SURF Processor Algorithm. Remote Sens.
**2011**, 3, 65–82. [Google Scholar] [CrossRef][Green Version] - Fischler, M.; Bolles, R. Random sample consensus: A paradigm for model fitting with applications to image analysis and automated cartography. Commun. ACM
**1981**, 24, 381–395. [Google Scholar] [CrossRef] - Barrodale, I.; Skea, D.; Berkley, M.; Kuwahara, R.; Poeckert, R. Warping digital images using thin plate splines. Pattern Recognit.
**1993**, 26, 375–376. [Google Scholar] [CrossRef] - Bookstein, F.L. Principal warps: Thin-plate splines and the decomposition of deformations. IEEE Trans. Pattern Anal. Mach. Intell.
**1989**, 11, 567–585. [Google Scholar] [CrossRef] - Zala, C.; Barrodale, I. Warping aerial photographs to orthomaps using thin plate splines. Adv. Comput. Math.
**1999**, 11, 211–227. [Google Scholar] [CrossRef] - Richards, J. Remote Sensing Digital Image Analysis; Springer: Heidelberg, Germany; New York, NY, USA, 2010; ISBN 978-3-642-30061-5. [Google Scholar]
- Zhao, J.; Yan, J.; Zhang, H.; Meng, J. A new radiometric correction method for side-scan sonar images in consideration of seabed sediment variation. Remote Sens.
**2017**, 9, 575. [Google Scholar] [CrossRef] - Li, X.; Hu, Z. Rejecting Mismatches by Correspondence Function. Int. J. Comput. Vis.
**2010**, 89, 1–17. [Google Scholar] [CrossRef] - Li, J.; Hu, Q.; Ai, M. Robust Feature Matching for Remote Sensing Image Registration Based on Lq-Estimator. IEEE Geosci. Remote Sens. Lett.
**2016**, 13, 1989–1993. [Google Scholar] [CrossRef]

**Figure 4.**Four SSS geocoded images. (I) (II) (III) and (IV) are four SSS images with 50% overlapping

**Figure 6.**The CFPs detected in three segmented overlapping areas (

**a**–

**c**); no CFPs are detected in area (

**d**); the S-L means the starboard side of left SSS image; the S-R means the starboard side of the right SSS image; the blue arrows mean the track line directions.

**Figure 8.**Coordinate deviations of CFPs. “☆” and “▷” symbols denote the coordinate deviations in east–west and north–south directions between the reference image and transformed sensed image; “○” and “◇” symbols denote these between the reference image and raw sensed image.

**Figure 10.**The comparison of image matching results using the MI and CR and the proposed method. (I) and (II) are three image matching results using the MI and CR and the proposed method.

**Figure 11.**Transformed sensed image with (

**a**) and without (

**b**) constraint of track line positions. The black lines denote raw positions and the red lines denote transformed positions.

**Figure 13.**Coordinate deviations of CFPs. “☆” and “▷” symbols denote the coordinate deviations in east–west and north–south directions between the reference image and transformed sensed image; “○” and “◇” symbols denote these between the reference image and raw sensed image.

**Figure 16.**Image registration operation for the raw SSS images (

**a**) and radiation corrected ones (

**b**).

**Figure 18.**The image matching results with different overlapping ratios: (

**A**–

**C**) are from Figure 5; and (

**D**,

**E**) are from the SSS images measured in Bohai Sea.

Max. (m) | Min. (m) | Mean (m) | STD (± m) | |
---|---|---|---|---|

dE1 | 3.10 | −3.08 | 0.00 | 1.06 |

dN1 | 4.31 | −4.30 | 0.00 | 1.76 |

dE2 | 16.58 | −11.75 | −0.62 | 5.76 |

dN2 | 15.13 | −16.94 | −0.13 | 6.16 |

**Table 2.**Statistical parameters of coordinate deviations of 1200 track line positions after adjacent image mosaics.

Max. (m) | Min. (m) | Mean (m) | STD (± m) | |
---|---|---|---|---|

dE | 0.19 | −0.05 | 0.00 | 0.02 |

dN | 0.12 | −0.15 | 0.00 | 0.03 |

Area (A) | Area (B) | Area (C) | |
---|---|---|---|

Image size | 701 × 337 pixels | 701 × 337 pixels | 701 × 337 pixels |

MI and CR | 62.33 | 62.63 | 64.54 |

SURF and RANSAC | 24.03 | 23.67 | 22.34 |

**Table 4.**Statistical parameters of coordinate deviations between raw and transformed track line positions with and without constraint.

Max. (m) | Min. (m) | Mean (m) | STD (± m) | ||
---|---|---|---|---|---|

With constraint | dE | 0.03 | −0.04 | 0.00 | 0.01 |

dN | 0.02 | −0.04 | 0.00 | 0.02 | |

Without constraint | dE | 7.98 | −5.60 | 1.50 | 3.16 |

dN | 10.05 | −3.43 | 2.62 | 3.07 |

**Table 5.**Statistical parameters of coordinate deviations of 45 CFPs and 1200 track line positions after multi-strip image mosaics.

Max. (m) | Min. (m) | Mean (m) | STD (± m) | |
---|---|---|---|---|

dE1 | 1.08 | −3.90 | −0.85 | 1.48 |

dN1 | 4.07 | −3.36 | 0.21 | 1.59 |

dE2 | 3.92 | −8.62 | −1.42 | 3.85 |

dN2 | 10.88 | −10.26 | 0.84 | 4.67 |

dE | 0.13 | −0.10 | 0.00 | 0.02 |

dN | 0.19 | −0.18 | 0.00 | 0.04 |

**Table 6.**The computing time and statistical parameters of coordinate deviations of 1200 track line positions after multi-strip image mosaics using different mosaicking methods.

Geocoding Method | MI and CR | Without Constraint | With Constraint | |
---|---|---|---|---|

Computing time | 83.22 s | 495.80 s | 268.54 s | 282.69 s |

Std (dE) | 0 m | 3.64 m | 3.69 m | 0.02 m |

Std (dN) | 0 m | 3.71 m | 3.61 m | 0.04 m |

**Table 7.**The standard deviations of the interpolation results using different methods to fill the gaps.

Gap SizeMethod | 5 × 5pixels | 10 × 10pixels | 15 × 15pixels | 20 × 20pixels |
---|---|---|---|---|

Nearest | 18.07 | 23.05 | 22.90 | 22.41 |

Linear | 13.63 | 17.58 | 19.02 | 19.32 |

4-Mean | 13.42 | 17.32 | 19.30 | 19.75 |

© 2018 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (http://creativecommons.org/licenses/by/4.0/).

## Share and Cite

**MDPI and ACS Style**

Zhao, J.; Shang, X.; Zhang, H.
Side-Scan Sonar Image Mosaic Using Couple Feature Points with Constraint of Track Line Positions. *Remote Sens.* **2018**, *10*, 953.
https://doi.org/10.3390/rs10060953

**AMA Style**

Zhao J, Shang X, Zhang H.
Side-Scan Sonar Image Mosaic Using Couple Feature Points with Constraint of Track Line Positions. *Remote Sensing*. 2018; 10(6):953.
https://doi.org/10.3390/rs10060953

**Chicago/Turabian Style**

Zhao, Jianhu, Xiaodong Shang, and Hongmei Zhang.
2018. "Side-Scan Sonar Image Mosaic Using Couple Feature Points with Constraint of Track Line Positions" *Remote Sensing* 10, no. 6: 953.
https://doi.org/10.3390/rs10060953