Robotic Coverage Path Planning for Ultrasonic Inspection

Automatic robotic inspection of arbitrary free-form shapes is relevant for many quality control applications in different industries. We propose a method for planning the motion of an industrial robot to perform ultrasonic inspection of varying 3D shapes. Our method starts with the calculation of a set of sub-paths. These sub-paths are derived from streamlines. The underlying vector field is deduced from local curvature of the inspected geometry. Intermediate robot motions are planned to connect individual sub-paths to obtain a single complete inspection path. Coverage is calculated via ray tracing to simulate the propagation of ultrasound signals. This simulation enables the algorithm to proceed adaptively and to find a good trade-off between path length and coverage. We report experiments for four different geometries. The results indicate that shorter paths are achieved by using ray tracing for adaptive adjustment of streamline density. Our algorithm is tailored to ultrasonic inspection. However, the main concept of exploiting local surface curvature and streamlines for coverage path planning generalizes to other robotic inspection problems.


Introduction
Coverage path planning (CPP) deals with the problem of finding a path for a robot 18 to cover some region of interest. Depending on the application, coverage may relate to 19 visiting, inspecting, or anyhow treating (mowing, painting, etc.) the complete region. 20 Hence, the definition of what region is covered by which robot configuration or position 21 strongly depends on the application. For example, the region covered by a lawn mower 22 robot at a specific position is a disk-shaped area around the center point of the rotating 23 blade. For a robot performing optical inspection, the covered area is derived from the 24 camera position, the field of view, and obstacles within the field of view. In any case, the 25 definition of coverage is an important topic for all coverage planning algorithms. 26 We introduce a method for coverage path planning that is built upon a heuristic 27 based on streamlines. Our main application is robotic inspection via ultrasonic testing 28 (UT). In order to calculate coverage, we rely on a ray tracing approach. Via ray tracing 29 it is possible to generate a volumetric representation of covered regions inside the 30 inspected part. While the ray tracing approach is very specific to UT, the heuristic of 31 using streamlines is applicable to other coverage planning problems. 32 This article is structured in the following way. In section 2 we review related 33 work on coverage path planning and coverage calculation. In section 3 we provide an 34 overview about streamlines in general. To derive streamlines for our CPP algorithm, 35 we build upon a vector field defined by local surface curvature. Surface curvature is 36 discussed in section 4. Another building block for our method is ray tracing to simulate 37 propagation of ultrasound waves -outlined in section 5. Section 6 explains how we put 38 Version November 15, 2021 submitted to Appl. Sci.

48
Many coverage planning algorithms address the problem of a mobile robot that is 49 required to visit all locations in the region of interest. The region of interest is discretized 50 so that it is represented by a finite number of points or grid elements [6,7]. Depending 51 on the application, additionally the orientation of inspection positions needs to be con-52 sidered. This enlarges the space of possible solutions leading to increased computational 53 complexity. For on-machine inspection with a 5-axis system, a method was proposed 54 that considers multiple orientations per inspected point [8]. Subsequently, the points 55 with different orientations are connected by solving a traveling salesman problem.

56
Robotic inspection using ultrasound is relevant for multiple quality control appli-57 cations in industry. An example for robotic UT is the inspection of components of a 58 thermonuclear reactor [9]. Even in the medical field, ultrasonic scanning by a robot is an 59 interesting topic. A recent method [10] is based on force-feedback during the scanning 60 process.

61
In the context of robotic scanning, recent work involves in-line dimensional in-62 spection via Spatio-Temporal Adaptive Sampling (STAS) [11]. This method estimates 63 deviations of the whole part based on partial measurement of a free form surface. In 64 addition, the method adaptively selects the next region to be measured in each step. An 65 interesting extension of coverage path planning deals with multi-robot environments 66 [12,13]. For most applications this requires some kind of synchronization of robots. Al-67 though there could be some potential of using our method in a multi-robot environment, 68 we do not further address this scenario.

69
As computational complexity is often high in the context of 3D surface inspection, 70 some papers propose heuristics that iteratively grow the inspected region. One method 71 uses a geometric random tree to sample possible future configurations [14]. This was 72 shown to provide good scaling properties with respect to the complexity of scenarios. In 73 general, the concept of iteratively growing the covered area with influence from control 74 theory is referred to as receding horizon. A recent work [15] deploys this concept for online 75 path planning to adaptively acquire information from the surface. Information already 76 acquired is inserted into a manifold Gaussian process and influences the progressive 77 calculation of the path.

78
A problem that is related to coverage path planning is classical point-to-point robot complexity. There are well-known algorithms available, e.g. rapidly exploring random 83 trees (RRT) [16]), that solve the point-to-point path planning problem. In the following, 84 we deploy standard solutions for point-to-point robot motion planning. This is mainly 85 relevant for situations where sub-paths that cover separate regions need to be connected.

86
The main intention of such a connecting motion is not to increase coverage but to perform 87 a transition from one region or robot configuration to another.

88
An adequate definition of coverage is an important topic for CPP algorithms.

89
Definitions of coverage can be very diverse ranging from point-wise coverage [12] 90 to drones that spray desinfectants [17]. In the present work we focus on volumetric method [19,20]. This is especially useful to evaluate low frequency effects like diffraction.

107
A geometric acoustic approach on the other hand is suitable for high frequency waves, i.e.

108
if the examined wave lengths are short in relation to the objects the wave is interacting 109 with. In order to additionally account for low frequency effects, geometric and wave 110 based approaches can be combined [21].

111
For our purpose, we use a geometric approach, which already is a widely studied 112 field, motivated by a variety of use cases, e.g. auralization for audio post processing and 113 virtual environments or simulation of classrooms helping the design process [19,22,23].
where t is the parameter of the streamline. In general, at any point along the 124 streamline s t the vector v(s t ) is parallel or anti-parallel to the derivative ds t dt . Therefore,

125
both vectors differ at most in their absolute size: the vector field is zero. Figure 1 shows an example for a streamline in 2D. The first   are passed by rays that meet these criteria are marked as captured, see Figure 3.
174 Figure 3. Illustration of coverage calculation via ray tracing. Two rays that are emitted by the sensor are shown. At any surface, transmission and reflection occur. Hence, each ray spans a tree. Voxels that are traversed by rays are shown in color. Red voxels correspond to a tree of rays that do not return to the sensor -at least not within the first four levels of the tree. Blue voxels are covered by rays that return back to the sensor. In contrast to red voxels, we consider blue voxels to be covered.
The ultrasound sensor used throughout this paper is a linear phased array (PA) the geometry in such a way that they hit a phased array element are marked as captured 181 by the sensor. As illustrated in Figure 4, the modeled PA sensor is placed at a given 182 working distance relative to a streamline that is calculated for the mesh surface. Emitted    voxelgrid ← voxelize mesh ▷ mark all voxels as not inspected 3: surface ← discretize outer surface 4: seeds ← calculate curvature vector field for surface 5: sort seeds using the curvature value in descending order 6: for s in seeds do 7: if s is too close to an existing streamline then 8: skip s 9: if curvature of s > threshold then 10: streamline ← expand streamline from s 11: else ▷ Note: specimen is now subdivided into flat areas 12: A ← connected flat area containing s 13: c ← center of A, re-projected onto the mesh 14: v ← vector pointing along the principal component of A

15:
streamline ← expand streamline from c with initial direction v 16: for p in streamline do 17: inspected voxels ← run ray-tracing algorithm for pose p 18: update voxelgrid using inspected voxel 19: for s in seeds do ▷ Only the seed points underneath the sensor are considered 20: n ← #voxels along the surface-normal marked as not inspected 21: if n = 0 then

235
Once the Cartesian paths are created, the second phase of the algorithm is executed. 236 We deploy a greedy search algorithm that generates the corresponding robot path. In a first step the algorithm calculates multiple valid robot paths per streamline. In a second In the final stage of the algorithm the generated path segments are connected.

254
Starting from a randomly selected path segment, the final complete robot path is created.

255
In each iteration the path is expanded at one of its endpoints by the path segment which  For flat areas, such as the object depicted in Figure 5a, the baseline algorithms achieve 289 full coverage with short paths. However, for more challenging areas, e.g. for the highly 290 curved center area on the mesh EDGE1, the baseline algorithms require an even smaller inspection radius to achieve full coverage. Figure 6 visualizes the voxels that are missed 292 by algorithm papp-base-60. papp-adaptive and papp-base-30 both achieve full coverage. However, in this case the 301 papp-adaptive version requires a slightly larger path length than papp-base-30.

302
(a) Front view (b) Side view Figure 6. Visualization of the 163 missed voxels on the mesh EDGE1 for the path planned using papp-base-60. 303 Coverage path planning for inspection of complex 3D geometries is a challenging 304 task. The exact solution of this problem is computationally not feasible. In this paper we 305 presented an approach that builds upon a heuristic using streamlines. Local curvature 306 is taken as the measure to evolve individual streamlines. An important parameter of 307 our algorithm is the distance at which streamlines are placed. This can either be a fixed 308 value or can be adjusted adaptively. In the latter case, coverage is approximated via 309 ray tracing in order to estimate the actual coverage for a path. While ray tracing is a 310 relatively simple method to simulate ultrasound propagation it provides a good trade-off 311 between physical correctness and computational effort. An evaluation on four selected 312 geometries shows that adaptive distances of streamlines can bring an improvement in 313 the range of 17 to 22 percent shorter path length with the same or even better coverage 314 on non-flat geometries. While the method does not provide an optimal solution to the 315 computationally intractable coverage path planning problem, our method is well suited 316 to many practical robotic ultrasonic scanning problems. Moreover, the method of using 317 streamlines lends itself well for adaptation of similar inspection tasks.

318
Future work will cover topics related to improvements of the ray tracing algorithm 319 for better coverage calculation. Specifically, we plan to compare the calculated coverage 320 with real ultrasound scans.