Fully Convolutional Approaches for Numerical Approximation of Turbulent Phases in Solar Adaptive Optics

Round 1

Reviewer 1 Report

1. Interesting research in a poorly assembled manuscript that needs some easily accomplished mild revisions concentrating on three aspects: 1) poorly stated results, 2) the very low-quality figures (image quality, but also professional presentation) and 3) citation issues.
2. Abstract is okay but is not likely to entice the readership to continue reading the rest of the manuscript.
   • The benefits of the proposed methods are decently stated. “…Avoid approximations and lose of information…” is a decent reduced description of transforming the height and width of the intermediate layer feature map back to the size of input image through the transposed convolution layer, so that the predictions have a one-to-one correspondence with input image in spatial dimension (height and width).
   • Results are only presented in weak, qualitative fashion. Highest quality expression of main conclusions or interpretations is quantitative results discussed in the broadest context possible, e.g., percent performance improvement compared to a declared benchmark. “…research shows the improvements of performance…” is very weakly stated results compared to “…xxx percent performance improvement over benchmarked methods was achieved….”. Quantitative results presented in figures should be reduced to such a ubiquitously understandable figure of merit.
3. Introduction is decently done with some omitted very recent literature and some extremely mild abuse of multi-citation without elaboration that readers might not appreciate.
   • Lines 28-29 introduce the broad topic of A.I. without citation, but before quickly focusing on ANN’s, deterministic artificial intelligence was omitted as a competing alternative to the stochastic methods proposed despite recent, broad applicability: unmanned underwater vehicles, actuator DC motors, and even space systems, e.g. https://doi.org/10.3390/jmse8080578. The assertion continues after the lit review, where another mini-lit review continues in the Materials and Methods section quickly justifying an appropriately reduced focus to FCNs for image reconstruction to smoothly flow the reader to the focused topic of the manuscript. Line 32 seems to imply (and over-claim) ANN’s are synonymous with A.I. rather than being one option amongst many stochastic and deterministic instantiations that use measurement information to ensure optimality.
4. Continued citation throughout the document is strong with some issues.
   • Please add the reference for the complex conjugate of Fourier Transform cited as [26] in line 251.
   • Please add the reference for Durham optics for night and solar observations cited as [27] in line 268.
   • Please add the reference for Gregor Solar Telescope cited as [28] in line 296.
   • Please add the reference for expected system correction times cited as [29] in line 477.
   • Please add the reference for translating phase of Zernike coefficients cited as [30] in line 483.
5. Equations are scientifically sound and well presented, enhancing the manuscript quality.
   • By the time the reader reaches equation (4), they need an acronym list that includes variable definitions. Please consider adding such to the appendix.
6. Figures are decently attempted with some mandatory improvements to ensure the readership has access to the content.
   • Internal font size is illegibly small and blurry, and this is generally true of every figure in the manuscript negating their usefulness. This assertion is true when the manuscript is read on an electronic device with zoom capability. As currently embodied, the figures are likely completely useless to readers of printed hardcopies, particularly in black and white.
   • Please notice the smallest font size permissible in the manuscript template (to ensure legibility by the reader) is the figure caption which provides a conveniently proximal prototype for sizing figures.
   • Data styles, fills, line thicknesses, and data markers are all identical in figures 6-8 rendering the disparate data indistinguishable when the manuscript is read in printed hardcopy (particularly in black and white) negating the value of the figures.
7. Tables are decently done to present results but omitted to introduce problem formation (aiding repeatability). Great job defining acronyms in the table 1 caption.  Readers will likely gravitate towards table 1, so this convenience will certainly be appreciated.  Please add the definition of WFE red, (rad), WFE residual, (ms), r_0, and (cm) to increase self-sufficiency of the table.

Author Response

REVIEWER 1:

Thank you for your kind review. Your in-depth revision pointed several flaws of the document, which has been significantly enriched thanks to your comments and suggestions. We deeply appreciate your time and effort. Our responses and changes according to your comments are detailed below:

1. Interesting research in a poorly assembled manuscript that needs some easily accomplished mild revisions concentrating on three aspects: 1) poorly stated results, 2) the very low-quality figures (image quality, but also professional presentation) and 3) citation issues.

You are right, and thus, we performed some changes through the document to improve the quality. We focused on the three aspects that you point out above.

2. Abstract is okay but is not likely to entice the readership to continue reading the rest of the manuscript.
   • The benefits of the proposed methods are decently stated. “…Avoid approximations and lose of information…” is a decent reduced description of transforming the height and width of the intermediate layer feature map back to the size of input image through the transposed convolution layer, so that the predictions have a one-to-one correspondence with input image in spatial dimension (height and width).

We have improved the abstract, regarding your following suggestion. More details are explained in the following answer.

• Results are only presented in weak, qualitative fashion. Highest quality expression of main conclusions or interpretations is quantitative results discussed in the broadest context possible, e.g., percent performance improvement compared to a declared benchmark. “…research shows the improvements of performance…” is very weakly stated results compared to “…xxx percent performance improvement over benchmarked methods was achieved….”. Quantitative results presented in figures should be reduced to such a ubiquitously understandable figure of merit.

Thank you for your comment, regarding your suggestions, we have made more quantitative analysis of the results obtained and a comparison with the results obtained by LS, the most common reconstructor used nowadays, has been added. It allowed us to show better the improvement that this kind of reconstructors offers to adaptive optics.

This new study has been included, as also data from figure 10 and table 3, comparing the residual WFE obtained by both ANN methods in comparison with the most used algorithm in real Solar AO System based on the least-squares (LS) method; also providing the data from Average of the residual wavefront error obtaining by the Least-Squares (LS) method in simulations of a Solar SCAO AO system.

This also has used to improve the scientific sound of the abstract (which has been changed accordingly), providing more accurate information about the improvements.

3. Introduction is decently done with some omitted very recent literature and some extremely mild abuse of multi-citation without elaboration that readers might not appreciate.
   • Lines 28-29 introduce the broad topic of A.I. without citation, but before quickly focusing on ANN’s, deterministic artificial intelligence was omitted as a competing alternative to the stochastic methods proposed despite recent, broad applicability: unmanned underwater vehicles, actuator DC motors, and even space systems, e.g. https://doi.org/10.3390/jmse8080578. The assertion continues after the lit review, where another mini-lit review continues in the Materials and Methods section quickly justifying an appropriately reduced focus to FCNs for image reconstruction to smoothly flow the reader to the focused topic of the manuscript. Line 32 seems to imply (and over-claim) ANN’s are synonymous with A.I. rather than being one option amongst many stochastic and deterministic instantiations that use measurement information to ensure optimality.

Thank you for pointing this out, as you mention, in the introduction we focus directly into neural networks; for fixing this issue, we improved the introduction, mentioning other artificial intelligence techniques; in particular with the reference you suggest, and also with the followings:

• Sands, T. Deterministic Artificial Intelligence; 2020;
• Sands, T. Development of Deterministic Artificial Intelligence for Unmanned Underwater Vehicles (UUV). J. Mar. Sci. Eng. 2020, 8, 578.
• Smeresky, B.; Rizzo, A.; Sands, T. Optimal Learning and Self-Awareness Versus PDI. Algorithms 2020, 13, 23.

As you comment, Line 32 can be interpreted as ANN’s are synonymous with A.I. so we have changed the paragraph to solve this issue.

4. Continued citation throughout the document is strong with some issues.
   • Please add the reference for the complex conjugate of Fourier Transform cited as [26] in line 251.
   • Please add the reference for Durham optics for night and solar observations cited as [27] in line 268.
   • Please add the reference for Gregor Solar Telescope cited as [28] in line 296.
   • Please add the reference for expected system correction times cited as [29] in line 477.
   • Please add the reference for translating phase of Zernike coefficients cited as [30] in line 483.

Thank you for noticing this. It seems that we had a problem with the end of the document, which affected those last references. It should be corrected now. We apologize for this issue.

5. Equations are scientifically sound and well presented, enhancing the manuscript quality.
   • By the time the reader reaches equation (4), they need an acronym list that includes variable definitions. Please consider adding such to the appendix.

You are right, perhaps we had used lots of acronyms during the paper that made the reading too complicated, and we have not realized that. The appendix A has been added at the end of the manuscript with the definitions of all the acronyms of the text.

6. Figures are decently attempted with some mandatory improvements to ensure the readership has access to the content.
   • Internal font size is illegibly small and blurry, and this is generally true of every figure in the manuscript negating their usefulness. This assertion is true when the manuscript is read on an electronic device with zoom capability. As currently embodied, the figures are likely completely useless to readers of printed hardcopies, particularly in black and white.

This issue has been addressed, changing the figures accordingly. More details are explained in the following answer.

• Please notice the smallest font size permissible in the manuscript template (to ensure legibility by the reader) is the figure caption which provides a conveniently proximal prototype for sizing figures.

Both points above have been addressed. All the figures of the research have been changed for ones with better quality and definition that avoid their blurriness. In some cases, as figures 4 and 5 they also have been increased to make the font size bigger. If necessary and the journal indicates so, original will be provided to the editorial team.

• Data styles, fills, line thicknesses, and data markers are all identical in figures 6-8 rendering the disparate data indistinguishable when the manuscript is read in printed hardcopy (particularly in black and white) negating the value of the figures.

Thank you for noticing this, since as we had not seen the images in black and white we have not realized of it. In the case of bars graphics, an explanation of the order of the bars (that is always the same for each graphic) has been added in their caption to ease the reading. In the case of lines graphs, the data markers for each case have been changed. Now, each case has its own data marker. In general, several captions have been improved to enhance clarity in the figures.

7. Tables are decently done to present results but omitted to introduce problem formation (aiding repeatability). Great job defining acronyms in the table 1 caption.  Readers will likely gravitate towards table 1, so this convenience will certainly be appreciated.  Please add the definition of WFE red, (rad), WFE residual, (ms), r_0, and (cm) to increase self-sufficiency of the table.

The definition has been added to the caption and the new appendix already included.

Also, to improve quality of the paper, more detailed information was included, as for example , details about DASP as an standard tool used for adaptive optics simulation, How is the method going to work when implemented, and about the selection of the topology used.

Reviewer 2 Report

In this paper, AO reconstruction for solar scenarios has been addressed. Generally, the paper is well written, but I suggest many points to improve the quality of this paper. 

1) I recommend the authors do more simulations to enhance their proposed method.

2) The main contribution of the paper should be highlighted and emphasized. 

3) How the MLPs and backpropagation are implemented in this paper.

4) Explain the values of variables considered in equation 2.

5) Explain the significance and importance of figure 2.

6) What is the importance of residual wavefront error (WFE).

Author Response

Comments and Suggestions for Authors 

In this paper, AO reconstruction for solar scenarios has been addressed. Generally, the paper is well written, but I suggest many points to improve the quality of this paper.  

Thank you for your review, you have pointed several deficiencies of our document that once solved, they have enriched the paper. The changes made according to your comments are detailed below each point. 

1. I recommend the authors do more simulations to enhance their proposed method.

Thank you for your suggestion, we have made more simulations to check the correct performance of the reconstructor and, to enrich de document, we have compared the method with the most common used one, the least-squares method. New information about the experiments performed is included at the end of the results section, including a figure and a table that accompanies and completes the new information.

2. The main contribution of the paper should be highlighted and emphasized. 

You are right, we have not highlighted enough the benefits of our research. Continuing with the previous point, thanks to your suggestion we have used the new added comparison to emphasize the quality of the results obtained, apart from the comparison between both new methods that had already been done. Information of the improvements of the reconstructions has also been added in the abstract of the document and in the conclusions. 

3. How the MLPs and backpropagation are implemented in this paper.

Thank you for your question, perhaps we have not made it clear but, in our research, we only make use of fully-convolutional neural networks. An introduction about MLPs is now presented in the paper as we consider that they are the base concept for more complex ANN topologies and, we also use them to introduce the backpropagation. We have employed MLPs it in previous adaptive optics research, now referenced in the paper, but in not in this one. In the case of the backpropagation algorithm, it was used to modify the kernels during the training process of the FCN. 

4. Explain the values of variables considered in equation 2.

Thank you for your suggestion, you are right, we have not realized that some variables of equation 2 were not explained in the document. This has been corrected and included in the text of the manuscript.

5. Explain the significance and importance of figure 2.

Figure 2 represents the difference in the information obtained by the wavefront sensor between diurnal and nocturnal observations. Thanks to your suggestion, we have emphasized in the document that, as the data received are not similar, the algorithms used by the sensors in night AO are not valid for the diurnal case, being that one of the most important difficulties present in solar adaptive optics. We have included in the text improved versions of the images, to improve quality and clarity. More detailed captions have also been included to improve the issue that you point out

6. What is the importance of residual wavefront error (WFE).

Thank you for your question, the residual WFE consist in the RMSE of the difference between the reconstructed phase and the original (simulated) one. It allows to know the quality of the reconstructions made, as in the case of a perfect reconstruction its value is 0. It is the magnitude that measured how big is difference between both phases. 

Reviewer 3 Report

The manuscript presents a numerical study of solar Adaptive Optics correction with phase recovery using Fully-Convolutional Neural Network Models. The paper is well written and of scientific interest, exploring the capabilities of neural networks when applied for physics. I would recommend its publication after the following suggestions:

• Check image quality. Images with small values do not have enough resolution.
• Check references. Last 5 references are cited but are not listed in the bibliography.
• I see that DASP is used for the simulations. Is it a standard tool used for adaptive optics simulation? Does it allow to make an implementation on a real situation? Please elaborate on that.
• How is the method going to work when implemented? Is there any expectation on how it should be implemented to obtain reasonable computation times?
• Check redaction. Section 2.4 includes different punctuation styles (20,000m / 3000m). Please revise the whole document regarding this.
• The selected topology is included, however the process/decision to select it is not detailed. Please elaborate the topology selection.
• More clarity on results is expected. A summary of the improvements must be specified clearly for the conclusions. Consider adding some figures that represent those improvements performed by the networks.
• To improve results, a comparison with an already stablished algorithm is recommended.
• In results, figure 4, in can slightly appreciate some “grid-patterns” in the recovered phases, which I do not appreciate in the phases from figure 5. Is there any reason for that? Is it studied how that affects the results?
• In discussion you can find “4.5ms”, another punctuation style. Please check.
• In conclusions, as possible future work, an optical bench is mentioned before the implementation on the telescope. How does it work and why is it relevant in the process from simulation to real implementation?

Author Response

The manuscript presents a numerical study of solar Adaptive Optics correction with phase recovery using Fully-Convolutional Neural Network Models. The paper is well written and of scientific interest, exploring the capabilities of neural networks when applied for physics. I would recommend its publication after the following suggestions: 

• Check image quality. Images with small values do not have enough resolution.

Thank you for your comment, we have not realized that in a printed document the images did not have enough quality and resolution for a good viewing. All the images of the document have been changed for new ones with more quality and the size of some of them have also been increased. 

• Check references. Last 5 references are cited but are not listed in the bibliography.

You are right, we have had an issue with the software used for the references and the last five had not been listed. The issue has been solved and we have checked that all the references are listed now. 

• I see that DASP is used for the simulations. Is it a standard tool used for adaptive optics simulation? Does it allow to make an implementation on a real situation? Please elaborate on that. 

Yes, DASP is a currently used simulation platform for forthcoming real instruments as the ELT (Extreme Large Telescope) or the Chinese Large Optical Telescope. Due to the quality of the simulations made by the platform, telescope systems trained with simulations made by DASP can be then implemented in real situations, as some previous ANN reconstructors developed for night observations by our group. We have improved the document regarding your suggestion. 

• How is the method going to work when implemented? Is there any expectation on how it should be implemented to obtain reasonable computation times?

Usually, FPGAs are used to implement networks on-sky instruments. However, previous to this phase, a check in optical bench is required, as stated in conclusions in the manuscript

• Check redaction. Section 2.4 includes different punctuation styles (20,000m / 3000m). Please revise the whole document regarding this.

Thank you for your comment, you are right, we have not realized that we have employed several punctuation styles along the document. All the paper has been reviewed and everything is consistent now.  

• The selected topology is included, however the process/decision to select it is not detailed. Please elaborate the topology selection.

Although explicitly omitted in the document, a grid search with the aid of a genetic algorithm, over the possible hyper-parameters of the network, has been performed, since it is the standard procedure to select the adequate topology

• More clarity on results is expected. A summary of the improvements must be specified clearly for the conclusions. Consider adding some figures that represent those improvements performed by the networks.
• To improve results, a comparison with an already stablished algorithm is recommended. 

Thank you for your suggestions, we have made a unique answer to the previous two recommendations as we have solved it together. The section 4 of the document has been increased to compare the results obtained by the new reconstructors with the one most commonly used, the least-squares method. Both methods based on ANNs have obtained lower errors in their reconstructions as it has now showed in the document. Moreover, a figure has been added which has the errors obtained by all the methods for the several turbulence situations represented, where the improvements can be seen and its corresponding explanation. The improvements obtained by the new reconstructors has also been specified in the abstract of the document and in the conclusions, as you have recommended us. 

• In results, figure 4, in can slightly appreciate some “grid-patterns” in the recovered phases, which I do not appreciate in the phases from figure 5. Is there any reason for that? Is it studied how that affects the results? 

Thank you for your comment, you are right, that “grid-patterns” are appreciated in figure 4 in the phases obtained by the ANN. In previous research made, we have already obtained that “issue” in the outputs of the networks, probably due to how the kernels of deconvulational layers act to several patterns of the feature maps obtained by the convolutional layers. The consequence is that in the borders of the kernels that “grid-patterns” mentioned appear. The contribution of them is measured in the residual WFE as it is calculated as the difference between both images, the net one and the original one. Anyway, in a real implementation, that patterns Will not affect the performance of the reconstructor as the information needed by the AO system are the positions of the DM actuators, an approximation of the turbulence phase where those small alterations would not affect.  

• In discussion you can find “4.5ms”, another punctuation style. Please check.

Thank you for the comment, as we have mentioned before, all the document has been reviewed to have a consistent punctuation style along it. 

• In conclusions, as possible future work, an optical bench is mentioned before the implementation on the telescope. How does it work and why is it relevant in the process from simulation to real implementation?

An optical bench is a system that allow us to obtain data of a light source propagated along several turbulence layers with the same characteristics as the ones that will be received in the real telescope. We need that the components have the same features as the telescope’s ones (as the number of subapertures of the sensor, number of pixels, etc). The bench is limited in the number of different images we can obtain, as it depends on the number of turbulence layers pieces we have, to situate them between the source and the wavefront sensor and the different combinations that can be made with them. It allows us to test the reconstructor with real images without having to implements all the system on the telescope’s own computers. Detailed explanations have been included in the text of the manuscript regarding this issue.

Round 2

Reviewer 1 Report

Thank you for making revisions satisfying reviewer recommendations.