Simulation Framework for Real-Time PHM Applications in a System-of-Systems Environment

Round 1

Reviewer 1 Report

The paper is fairly well written and well organized. It provides good correlation with previous related work. The sequence in the paper is intuitive, making it easy to follow. The language is clear and concise.

 

It describes a simulation framework that represents the wear and tear of individual UAVs of a fleet. This topic is relevant to the field, given the recent increase of interest in UAV swarm strategies. The simulation requires modeling of different component degradation progressions, which result in a number of possible failures for each UAV. By changing the maintenance strategies, different outcomes can be obtained. Then, it is possible to investigate the impact of maintenance on the mission.

 

The baseline strategy is the run-to-failure approach. Then, a predetermined maintenance strategy (1) and a condition-based maintenance (CBM) strategy (2) are evaluated. Strategy (1) does not take any knowledge about the UAV status or health into account and maintenance is performed in fixed intervals, whereas in strategy (2), maintenance procedures are initiated through a threshold regarding component health of the UAV. The results show that the use of metrics can indicate the impact of different maintenance strategies on fleet availability in real-time.

About this part:"[...] the experiment was conducted ten times for each maintenance strategy [...]":

The authors are talking about Monte-Carlo experiments. However, why 10 experiments? Typically, one would perform the experiment several times and check the mean and standard deviations. After a sufficient number of experiments, these values should become stable. Can the authors present the evolution of these parameters along the 10 experiments? Or present an explanation regarding why the number 10 was chosen.

 

The results are largely dependent on the models postulated and assumptions. The author made it clear that several improvements can be implemented to improve the simulation fidelity. For future work, perhaps the biggest challenge is how to extract component failure models from experimental data. The author could include a comment in this regard as well.

 

 

Please improve for clarity:

"Over several missions run to failure data is produced that due to different fault constellations the remaining useful life (RUL) differs every time."

Maybe: "Over several missions, run-to-failure data are produced. Due to different fault constellations, the remaining useful life (RUL) differs every time."

 

The real-time data storage during mission execution can be seen as black dots in Figure 3 and Figure 4.

(Cannot see this)

 

This behavior can be seen in Figure 3 and Figure 4, where the discharge over time is represented through the blue slope

(blue slope?)

 

changing environment However,

changing environment. However,

 

 

As an early approach for PHM driven UAV swarm management only the actual battery capacity

As an early approach for PHM driven UAV swarm management, only the actual battery capacity

 

A comparison of the results is given in Chapter 6.

Section? (many other instances of "chapter" should be fixed)

 

Detailed descriptions of the strategies follow in the next subchapters.

Subsection? (many other instances of "subchapter" should be fixed)

 

[...] after the mission, where the health grade falls [...]

[...] after the mission, during which the health grade falls [...]

 

Both metrices are simple

Both metrics are simple

 

While the UAV get deployed randomly for the predetermined strategy the CBM strategy 

While the UAV get deployed randomly for the predetermined strategy, the CBM strategy 

 

Figure 9:

Probability of Occurance

Probability of Occurrence 

 

Abbreviations

RUL - In the text: remaining useful life

RUL - In the list of abbreviations: remaining useful lifetime

(please pick one, for consistency)

Please add:

SoC state of charge?

Author Response

About this part:"[...] the experiment was conducted ten times for each maintenance strategy [...]":

Most useful feedback, I wasn't happy with the "ten times" either. Conducted the experiment several times now until mean and std values of experiments became stable. Added additional plots to proof this as well as a correction of Fig. 9

 

 

For future work, perhaps the biggest challenge is how to extract component failure models from experimental data. The author could include a comment in this regard as well.

Included a comment in the section "Conclusion"

"Over several missions run to failure data is produced that due to different fault constellations the remaining useful life (RUL) differs every time."

Fixed this

The real-time data storage during mission execution can be seen as black dots in Figure 3 and Figure 4.

Clearly not visible. Cross reference to section 4.1

 

This behavior can be seen in Figure 3 and Figure 4, where the discharge over time is represented through the blue slope

Also not visible, corrected this to "black curve"

 

changing environment. However,

Fixed this

 

 

As an early approach for PHM driven UAV swarm management, only the actual battery capacity

Fixed this

 

A comparison of the results is given in Chapter 6.

Section? (many other instances of "chapter" should be fixed)

Changed all "Chapters" to "Sections"

 

Detailed descriptions of the strategies follow in the next subchapters.

Subsection? (many other instances of "subchapter" should be fixed)

Fixed this as well.

 

[...] after the mission, where the health grade falls [...]

[...] after the mission, during which the health grade falls [...]

Fixed

 

Both metrices are simple

Both metrics are simple

Fixed

 

While the UAV get deployed randomly for the predetermined strategy, the CBM strategy 

Fixed punctuation

 

Figure 9:

Probability of Occurrence 

Fixed typo

 

Abbreviations

RUL - In the text: remaining useful life

RUL - In the list of abbreviations: remaining useful lifetime

(please pick one, for consistency)

Picked remaining useful life for consistency

Please add:

SoC state of charge!

Added State of Charge abbreviation as well as others:

• ARINC
• STD

Reviewer 2 Report

The submitted paper presents a simulation framework integrating PHM methods for UAV swarm operation and fleet management. The presented framework enables situational awareness, condition-driven asset management, and higher mission reliability. For example, it can simulate and compare different PHM methods and maintenance strategies. The framework validation is based on the case study of a fleet of Skyhunter VTOL UAVs performing observation and SaR missions.

The topic is relevant and contributes to optimizing UAV fleet and swarm operation management.

The presented framework aims to consider dynamically changing UAV configurations, capabilities, task allocations, and trajectories during mission execution. The focus is on task scheduling and task allocation within the fleet with component degradation and weather considerations. However, it needs to be clarified what the novelty of the presented work is compared to the published literature.

The conclusion summarize the achievements, focusing on the outcomes of the simulation, together with the current limitations and corresponding future work directions.

References appear appropriate.

Figure 5 misses a legend and the values of the time and EOL probability axes. It should also clarify the failure degradation scale. For example, degradation of 1.0 means fully degraded or as new?

Figure 6 misses a legend and the values of the time axis. It looks like the UAV subsystems have different health at time 0. Is it true? If yes, please explain where does the difference come from. If not, please correct the figure.

Figure 7 misses the units of the time delta (y-axis).

Figure 8 misses the units of the dwell time (y-axis).

Figure 9 is missing the x and y-axis' gradation and units. Avoid using acronyms (CBM) in the figure. Typo, y-axis label should read occurrence.

Additional comments:

The paper is nicely written and structured. The background, problem statement, approach, contribution, and significance of the work are clearly presented through the introduction, body, and conclusion. However, the abstract should be revised to more clearly provide the purpose of the article. Also, it should describe the main elements and novelty of the presented framework and explain the significance of the work.

The novelty of the presented simulation framework needs to be clarified. This aspect could be a weakness of the submitted work.

Based on the overall high merit and quality of the presented work, I recommend an acceptance with minor revision (abstract).

Author Response

Figure 5 misses a legend and the values of the time and EOL probability axes. It should also clarify the failure degradation scale. For example, degradation of 1.0 means fully degraded or as new?

Legend added, values for EoL Probability axes added. Failure Degradation Scale explained in text after figure. x-Axes generalized with "Time" only to avoid confusion between real time and simulation time. Explanation added in text.

 

Figure 6 misses a legend and the values of the time axis. It looks like the UAV subsystems have different health at time 0. Is it true? If yes, please explain where does the difference come from. If not, please correct the figure.

Legend would be too big and confusing, explanation of different curves is given in the following text section. Values of time axis were neglected due to the same reason as mentioned before. Subsystems have a different initial health (due to variations in the manufacturing processes) to represent a more realistic behaviour of the system. Explanation is added in the text.

 

Figure 7 misses the units of the time delta (y-axis).

Units have been added

Figure 8 misses the units of the dwell time (y-axis).

Units have been added

 

Figure 9 is missing the x and y-axis' gradation and units. Avoid using acronyms (CBM) in the figure. Typo, y-axis label should read occurrence.

Y-axis gradation and units have been added. X-axis gradation and units are shown for specific values. This helps to identify important values and to make them comparable. CBM corrected to Condition Based Maintenance. Type corrected as well.

 

However, the abstract should be revised to more clearly provide the purpose of the article. Also, it should describe the main elements and novelty of the presented framework and explain the significance of the work.

Abstract has been revised in order to point out the purpose of the arcticle, to clarify the advantages of the proposed simulation and to manifest the novelty of the work.

Reviewer 3 Report

The work describes a simulation framework for predicting the degradation of wear and tear of individual UAVs in a fleet of UAVs that are used for search and delivery operations. The paper is recommended for publication with the following minor revision.

1) In line 96, there is a typo of none-linear instead of non-linear.

2)  The article list the type of uncertainties in line 99-100 without proper reference. Please include the references.

3)  Please mention the multithreaded package used for the simulation framework.

Author Response

1) In line 96, there is a typo of none-linear instead of non-linear.

Corrected

 

2)  The article list the type of uncertainties in line 99-100 without proper reference. Please include the references.

The reference was mentioned in the sentence before but the allocation was not clear. The sentence was shortened and linked so that the reference is now mentioned in the relevant context.

 

3)  Please mention the multithreaded package used for the simulation framework.

The python threading package was used. It is now mentioned in section 2.3. Simulation Framework Overview and also referenced.