Towards Autonomous Bridge Inspection: Sensor Mounting Using Aerial Manipulators

Round 1

Reviewer 1 Report

This paper proposes a method to mount sensors on the bridge wall and  the method is using two UAVs to achieve the goal. There are model and control derivation and flying simulations of UAV. However, the title is "Aerial Manipulation Approach for Autonomous bridge inspection" which seems not to match the content of the paper. The paper mainly talks about two UAVs attach sensors, not inspect the bridge. Please check the appropriateness of the title.

Author Response

We thank the reviewer for pointing out the title is not matching the content of the paper. We agree that the title is too broad for the paper as it is. The Reviewer 2 also noted that the paper presents a partial solution for mounting sensors on the bridge. Therefore, we changed the title to define our solution more precisely:

Towards Autonomous Bridge Inspection: Sensor Mounting using Aerial Manipulators

Reviewer 2 Report

The concept and ideas presented are very ambitious but interesting at the same time.

The paper provides some very interesting work related to the issues the authors wish to resolve.

Although the work presented has relevant content in the field, there is a lack of practical implementation that would enforce the proposed results obtained via simulations.

There are also a few aspects regarding the research that are short sighted and need to be addressed on a fundamental level, these aspects are related to the technical challenges imposed.

One of these aspects is related to the autonomy of the UAV, as stated by the authors, for achieving the goals of the application, the drone would need to maintain constant force for several minutes to form a bond of the adessive. Current state of the art drones have a very limited flight time even without compensating to achieve constant pressure while interacting with fixed objects. This issue needs to be addressed for the application to be feasible.

Another aspect that is not addressed is related to the environment itself, as also mentioned by the work cited, most applications where drones perform physical interactions are at an experimental level in an indoor environment. References cited indicate that accuracy even in an indoor environment is far from what the proposed application needs. The proposed system will be applied in an outdoor environment, where wind and turbulence impose many more dynamic issues than in an indoor environment.

The main issue would be to achieve a stable flight control, including a 3 DOF manipulator or any element that modifies the center of gravity of the drone would drastically hinder the drones flight controller. As seen in the cited work, some applications minimize lateral mass by placing the effector in the center of the drone, this idea might be a good alternative to the current concept proposed in figure 1.

There is an inconsistency in the statements that need additional explanation. At one point the authors imply that the proposed application would be a more efficient solution by mounting sensors as opposed to the use of inspection trucks that usually inspect for corrosion, cracks, voids, weakening connections, and concrete delamination. There is no clear understanding of what exactly the sensors will inspect. What exactly is the inspection task that the sensors would perform compared to the conventional approach? One could assume that the equipment required for inspection is considerably heavy, if it required a truck to be operated.

 The most sensitive problem (from a conceptual point of view), which from my point of view must be solved is that of the compensation of the yaw (and also roll) movement of the UAV (around the ZB axis). In the event of such a movement (yaw), the system cannot maintain a correct contact with the surface with the sensor attached to its end-effector, because the designed manipulator cannot compensate for such a movement.
Indeed, the relationship (4) provided by the authors will ensure the distribution of the takeover of the system movement between the UAV and the manipulator, but only in the plan in which the manipulator works! (can compensate only the pitch movement of the UAV)
There are solutions to solve this shortcoming, but the authors will have to modify the constructive solution of the system!
There is also the option to specify from the beginning this partial solution of the problem - to leave the paper as it is - and to the conclusions to introduce future research directions that will completely solve the problem.
I suggest the second option.

Section titles are self-explanatory, thus the need of section 1.3. is redundant.
On page 4, paragraphs beginning on lines 140 and 149, respectively, must be aligned accordingly.
In the explanations in Table 1, d1 must be replaced by d3.
 Please provide a DOI for the 11th reference.

Contribution section claims need to be more specific and at its current state of the art I suggest some references from MDPI journals as:

UAVs with manipulators
https://doi.org/10.3390/s19194253.

https://doi.org/10.3390/app11136220 

https://doi.org/10.3390/app9112230

 https://doi.org/10.3390/s20174708

flight stability

https://doi.org/10.3390/sym10070291

https://doi.org/10.3390/app9173600

Overall, I propose the paper undergoes mentioned changes and acceptance if updates address all issues mentioned in the review.

Author Response

To provide clearer responses, we divided the reviewer's comments and suggestions into several remarks. As part of the response, we are also uploading the revised manuscript with changes suggested by this reviewer, highlighted in green color.

Remark 1

One of these aspects is related to the autonomy of the UAV, as stated by the authors, for achieving the goals of the application, the drone would need to maintain constant force for several minutes to form a bond of the adessive. Current state of the art drones have a very limited flight time even without compensating to achieve constant pressure while interacting with fixed objects. This issue needs to be addressed for the application to be feasible.

Answer

We thank the reviewer for concerns about the autonomy of the UAV. It is true that indoor multirotors have a quite limited flight time. This flight time is further reduced by attaching a manipulator because of the added weight and power consumption. It is also important to note that indoor multirotors are typically smaller than their outdoor counterparts, the main reason being limited flight environment and narrow corridors.

However, as the final application scenario of this paper is outdoor sensor mounting, size and payload capability of outdoor multirotors increase. Although this will inevitably lead to the manipulator redesign due to its size, the flight times of such vehicles are much greater. In our previous work(10.1109/LRA.2021.3068923) we used a custom built quadcopter with a flight time of around 30min. As we plan to use the same vehicle for future outdoor experiments, we believe that the total flight time of this vehicle will be sufficient to locate the mount point, attach the sensor and maintain contact for several minutes. 

Since this has the potential to spark confusion in future readers, we addressed it in conclusions.

Remark 2

Another aspect that is not addressed is related to the environment itself, as also mentioned by the work cited, most applications where drones perform physical interactions are at an experimental level in an indoor environment. References cited indicate that accuracy even in an indoor environment is far from what the proposed application needs. The proposed system will be applied in an outdoor environment, where wind and turbulence impose many more dynamic issues than in an indoor environment.

Answer

We thank the reviewer for this remark. We agree that the interaction with the environment is indeed mostly limited to the indoor environment. In this paper we are trying to mount the sensor as accurately as possible because the accuracy will certainly drop in the outdoor environment. One of the main reasons for testing blob detection and tracking is to get the idea how it will work in an outdoor environment. Using this visual detection can help us improve the accuracy in future work. Also, if a larger area is sprayed with the first glue component, we can allow for lower accuracy as long as the end-effector is parallel to the plane normal.

However, as the reviewer stated, being in an outdoor environment introduces its own challenges in terms of position controller accuracy and disturbances imposed on the system. For successful outdoor application the manipulator will have to be augmented with at least two more degrees of freedom compensating for roll and yaw angles of the UAV. We have already designed such a manipulator. All of the aforementioned has not been stated in the paper and we see how it could confuse a future reader. We have addressed this at the end of section 5.

Remark 3

The main issue would be to achieve a stable flight control, including a 3 DOF manipulator or any element that modifies the center of gravity of the drone would drastically hinder the drones flight controller. As seen in the cited work, some applications minimize lateral mass by placing the effector in the center of the drone, this idea might be a good alternative to the current concept proposed in figure 1.

Answer

We thank the reviewer for pointing out stability concerns. The real world 3DOF (and even the future 5DOF) manipulator is constructed using lightweight carbon fiber links and fairly light Dynamixel XM-430 motors. Furthermore, the inspection sensors such as accelerometers, strain gauges, tiltmeters, all can be found in lightweight design. Therefore, we believe the final manipulator carrying the sensor will be lightweight and suitable for use on a multirotor UAV. 

The outdoor UAV we plan to use has an operational weight of 9.5kg, with up to 4kg of payload. Although the manipulator is envisioned to be mounted on top of the UAV, there is still enough payload redundancy to mount a counterweight which would bring the center of mass closer to the geometric center of the UAV. As for the lateral mass displacement, the attitude and position control will see this displacement as a disturbance and compensate for it. 

Placing the end-effector at the center of the drone limits the number of surfaces we can attach a sensor to. Most of these surfaces are underneath the bridge or vertical, and would not be reachable with such design. These are the main reasons we opted for attaching the manipulator on top of the UAV, and have been briefly outlined at the end of section 3.1.

Remark 4

There is an inconsistency in the statements that need additional explanation. At one point the authors imply that the proposed application would be a more efficient solution by mounting sensors as opposed to the use of inspection trucks that usually inspect for corrosion, cracks, voids, weakening connections, and concrete delamination. There is no clear understanding of what exactly the sensors will inspect. What exactly is the inspection task that the sensors would perform compared to the conventional approach? One could assume that the equipment required for inspection is considerably heavy, if it required a truck to be operated.

Answer

We thank the reviewer for pointing out this problem. Specialized trucks have a crane with a basket for human inspectors. This crane is able to reach higher up the construction or underneath the bridge and allow the inspectors to review conditions there. Some of the non-destructive methods require mounting small sensors (accelerometers, strain gauges, tilt meters, and various transducers i.e. for acoustic or pressure measurements). After mounting these sensors, the bridge is excited with vibrations, tapping, sound waves, etc. and responses are recorded. Analyzing these responses gives insight in the bridge state, revealing cracks, voids, weakening connections, concrete delamination etc.

We can see how mentioning trucks can imply these sensors are large or bulky, but they are mostly used to access hard-to-reach spots on the bridge. We have added clarifications in the Introduction section, paragraph before subsection 1.1. 

Remark 5

The most sensitive problem (from a conceptual point of view), which from my point of view must be solved is that of the compensation of the yaw (and also roll) movement of the UAV (around the ZB axis). In the event of such a movement (yaw), the system cannot maintain a correct contact with the surface with the sensor attached to its end-effector, because the designed manipulator cannot compensate for such a movement.

Indeed, the relationship (4) provided by the authors will ensure the distribution of the takeover of the system movement between the UAV and the manipulator, but only in the plan in which the manipulator works! (can compensate only the pitch movement of the UAV)

There are solutions to solve this shortcoming, but the authors will have to modify the constructive solution of the system!

There is also the option to specify from the beginning this partial solution of the problem - to leave the paper as it is - and to the conclusions to introduce future research directions that will completely solve the problem.

I suggest the second option.

Answer

We thank the reviewer for this suggestion. As a response to Remark 2 we have already added a short description of this problem to the manuscript. We do agree that in order to compensate for errors in yaw, roll and y-axis, the manipulator has to be augmented with additional degrees of freedom. However, the focus of this paper is on the development of a model-based motion planner that is able to work with the impedance control. We agree that the design of the manipulator is preliminary and will be adapted in future work. We have addressed this remark in the conclusion section.

Remark 6 - smaller remarks

Section titles are self-explanatory, thus the need of section 1.3. is redundant.

Answer: We agree and section 1.3 has been removed.

Actions: To je section za organizaciju papera. Mislim najmanji problem je to maknuti.

On page 4, paragraphs beginning on lines 140 and 149, respectively, must be aligned accordingly.

Answer: The mentioned paragraphs, along with several others have been properly aligned.

Actions: Da, ti paragrafi nisu uvučeni iz nekog razloga. Uvući.

In the explanations in Table 1, d1 must be replaced by d3.

Answer: Yes, this typo has been corrected.

Please provide a DOI for the 11th reference.

Answer: The DOI for the reference in question is still not available since the ICRA2021 proceedings are yet to be published. Due to added references, this is not reference [15].

Actions: Na još nekim referencama fali DOI. Dodati. Baš na ovoj referenci 11 nema DOI-ja jer još nisu došli proceedingsi od icra-e.

Contribution section claims need to be more specific and at its current state of the art I suggest some references from MDPI journals as:

UAVs with manipulators

https://doi.org/10.3390/s19194253.

https://doi.org/10.3390/app11136220 

https://doi.org/10.3390/app9112230

 https://doi.org/10.3390/s20174708

flight stability

https://doi.org/10.3390/sym10070291

https://doi.org/10.3390/app9173600

Answer: We included several suggested references in the state of the art overview, within section 2. We have also rewritten contributions to state them in a more clear manner.

Actions: 

Overall, I propose the paper undergoes mentioned changes and acceptance if updates address all issues mentioned in the review.

Author Response File: Author Response.pdf

Round 2

Reviewer 2 Report

The authors have corrected/responded to all the observations made in the review and, therefore, I recommend that the paper be accepted in present form.