Geometric Calibration of Thermal Infrared Cameras: A Comparative Analysis for Photogrammetric Data Fusion

Round 1

Reviewer 1 Report

Comments and Suggestions for Authors

The paper evaluates the impact of different calibration targets (calibration methods) on the results of intrinsic and extrinsic parameter calibration. It holds certain reference value for the industry. However, as a research article, its novelty is insufficient.  At the same time, some expressions are unclear.

1.The abstract mentions the proposal of 3D field calibration target, but the article does not clearly demonstrate where the novelty lies.

2.The abstract states that 3D evaluation metrics can better assess IO and RO, but it lacks a theoretical explanation of why 3D evaluation metrics are more effective in decoupling the evaluation of IO and RO.

Author Response

We thank the reviewer for their comments and have compiled the following responses to the remarks:

1. We acknowledge the position of the reviewer, and as a result we have amended the paper to place greater emphasis on the application of calibration parameters as opposed to the 3D calibration target. The novelty of the 3D calibration test fields (and custom markers) is that the multi-purpose markers solve several open challenges for IRT-3DDF, notably: (1) their use as configurable calibration markers for lab- and field-based geometric calibration procedures, (2) their use as survey markers for photogrammetric scaling, referencing and validation, and (3) their use for temperature ground truth markers for radiometric calibration, correction and verification. We have amended Section 2.2.2 to highlight this, as well as improving the Discussion.
2. We have expanded the references to projective coupling in the Introduction and Discussion to provide greater clarity on the differences between 2D and 3D approaches, the assessment of calibration error, and photogrammetric measurement accuracy. We have also amended the final paragraph of the Introduction to provide clarity on the suggested point. Within the abstract, we have included ‘projective coupling’ as the initial ‘problem statement’ for 2D boards that we wish to address throughout the article.

Reviewer 2 Report

Comments and Suggestions for Authors

 

This work has comparatively evaluated existing 2D- and novel 3D calibration targets, detailing construction, calibration and application for IRT-3DDF. Results demonstrate the success of the proposed 3D field calibration target for the calculation of both IO and RO parameters required for data fusion. The overall structure is complete and logically coherent, with a reasonable methodology that demonstrates a certain level of innovation. Overall, this manuscript can be considered for publication in your journal after undergoing major revision. Specific suggestions for revision are as follows:

1. The review of existing research in the introduction is somewhat fragmented and lacks systematic summarization. It is recommended to consider merging the content of Section 2 into the introduction.
2. Why not consider using passive infrared thermography for geometric calibration? Passive infrared thermography utilizes the infrared radiation emitted by objects themselves for imaging, offering advantages such as no need for an external heat source and ease of operation.
3. The annotations and resolution of some figures (e.g., Figures 5 and 6) are not sufficiently clear, which affects readability. It is recommended to optimize the design of the figures to ensure that key information is immediately apparent.
4. The manuscript does not address the impact of environmental factors on calibration accuracy. Thermal infrared cameras are sensitive to ambient temperature, humidity, and lighting conditions. It is unclear whether different seasons, weather conditions, and locations would affect the calibration results.
5. Section 4.1 suffers from verbose and loosely structured writing, making it difficult for readers to quickly grasp the key points of the research.
6. Although indicators such as MRE and RMSE are provided, the sources of error do not appear to have been analyzed. It is recommended to include an error analysis.
7. Is there a potential limitation in validating the effectiveness of the method using only two case studies?
8. The formatting of some references is inconsistent. It is recommended to standardize the format according to the target journal’s requirements.
9. There are some details or formatting issues in the paper, and the author is advised to carefully revise them.
10. The conclusion does not sufficiently emphasize the innovation of the research. It is recommended to clearly summarize the study’s contributions to the field.
11. The references could be further enriched. The following references are suggested to be added to reflect the timeliness and depth of the paper:

https://doi.org/10.3390/s23073479

https://doi.org/10.1016/j.jrmge.2025.03.023.

https://doi.org/10.1007/s11629-023-8484-9

Author Response

We thank the reviewer for their comments. We have addressed their points as follows:

1. As this paper focuses on both geometric calibration and photogrammetric data fusion, we acknowledge the duality of the Introduction and State-of-the-Art. In response to the reviewer, we have re-ordered the content from the SOTA into the Introduction and built a flow through from calibration to photogrammetric data fusion.
2. Certainly, the use of passive thermography would have been preferred, but the ability for meaningful contrast to be generated indoors without active thermography was insufficient. Without active thermography, the targets showed no significant contrast (thermal equilibrium). To address this point, we have included in the discussion the limitations present for active thermography (Section 4.1.1).
3. We have increased the resolution and annotation sizes of Figures 5 and 6. With a detailed caption, we believe for those keen to explore the figures in more detail, the resolution and description is now sufficient. We have also printed a copy of the manuscript and believe the resolution here is sufficient for the central points to be visible.
4. Although we have included a note about varying sensor temperature in the discussion, we have included environmental conditions, notably Wan et al.’s controlled lab experiments, and the influence of external factors on calibration as a note for future investigations (Section 4.1.2).
5. Section 4.1 (now 3.1) has been re-written to address verbose and structure. As this section represents a considerable amount of work yet to be published, we appreciate the summary may have reduced critical points.
6. References to MRE, RMSE and their importance have been improved in Sections 3.1, 3.2, 4.1 and 4.2.
7. Certainly, two methods may not be sufficient to wholly determine the best approach for TIR geometric calibration, but this intends to be a gateway to additional methods for validation. We have stressed this in the Section 4 but improved wording to state the relevance of your comment.
8. The formatting of references has been addressed and now follows the MDPI guidelines.
9. We have reviewed the paper to address obvious formatting issues, but if the reviewer has any particular instances we would be happy to address them specifically. We have centred results in tables where we deemed it appropriate, but will be led by MDPI (as per the MDPI Style Guide) if the article is accepted.
10. The conclusion has been re-written to address innovation in the paper (notably comparative analysis and multi-modal fusion method).
11. We thank the reviewer for their suggested references. However, the first is already included and the other references were deemed unsuitable for this article.

Reviewer 3 Report

Comments and Suggestions for Authors

Please see the attachment. 

Comments for author File: Comments.pdf

Author Response

Thank you for your comments. Instances where in-text citations have been used in the specified manner have been changed accordingly. We have used the authors names (e.g., Usamentiaga et al.’s [1]…) to improve readability and help the reader identify critical studies.

Reviewer 4 Report

Comments and Suggestions for Authors

The first sentence of the introduction is a little confusing: 'The determination of precise and reliable intrinsics for thermal infrared (TIR) cameras...' What do you mean by the terms precision and reliability? The question arises because the terms are not commonly used in demographic practice. Also, the introduction says in line 26: '(1) the inherently small dimensions of TIR sensors' Perhaps LongWave should be added, because the sensors today have a commercial resolution of 2560*2048, a thermal resolution of 15 mK. What is missing in the introduction is the thematic limitation of the research and the indication that every thermographic camera has an integrated optical camera and that almost all manufacturers offer a fusion of thematic and visual images to compensate for the limitations due to the low resolution. The standards for the maintenance of photovoltaic modules stipulate the minimum resolution of the optical camera in the thermal imaging camera. The biggest problem of the research objective is to circumvent the fact and not name the physical diversity of the generation of each image. While photography depends on the reflection of light radiation, thermal radiation depends on the angle of observation and changes with the angle.
Chapter 2 begins with a good sentence containing the clumsy term 'TIR geometric camera calibration'. Calibration of the thermographic camera is usually done by calibrating individual microbolometers at one temperature value, and then calibrating the entire sensor at different temperatures. I can only assume that the authors meant the calibration of the thermal camera optics, correct me if I'm wrong? If I am correct, the 'State- of- the-Art' must explicitly state this. Since the chapter introduces thermal imaging windows with different frequencies (wavelengths) in the abstract itself and in the water, it should be stated which camera is used in the paper: LWIR. I am writing the review as I read the paper, so the above comments are partly discussed in chapter 3.1. I found the literature review surprisingly well done, although personally some paragraphs do not seem sufficiently coherent to me, but as they are not the subject of the paper there is no point in complaining as they would only complicate the review unnecessarily. It would be good to give the factory calibration setting and to point out that emissivity is not constant at all wavelengths (and that it depends on viewing angle).
In the sentence 'Although the Workswell WIRIS Pro features two sensors, WWP is used to refer to the thermal sensor in the camera. ' is not the stated meaning of the abbreviation 'WWP' and I would not write two sensors but visual and thermal sensor because we can confuse people dealing with 3D. Looking at picture 1 and the cameras side by side, I wonder what kind of focus Workswell has? It's a reputable European manufacturer that made a top of the line agricultural camera, but not knowing anything about the design, I'm just assuming it's a fixed focus that is calibrated during production? The Sony and Nikon images may not be necessary as they are commonly known devices. And now, thanks to Table 1, we come to the backbone of the first objection to the work, i.e. It must be stated how the authors methodically solved the differences in pixel size, IFOV of the thermal imager and the camera, resulting in an inaccurate reading of the temperature of the studied object due to the interruption of the reference, which is clearly visible on the photo and smeared on the thermography; clearly visible in Figure 2?
Table 2. in the delta emissivity of 0.20 hides a greater mystery manifested in the fact that the emissivity depends on the angle, and if the indication of the deletion makes the table accurate and precise, unfortunately this is not the case with the procedure itself; for the observed analysis this may not be important, but it should be mentioned in the text of the paper, because the question arises especially when looking at Figure 12. At the very end, I must commend the authors for their good research and suggest omitting Figure 1. because it somewhat diminishes the seriousness of the paper and people reading the paper will surely know what the cameras described in Table 1 look like.

Author Response

We thank the reviewer for their comments. We have tried to address all the listed comments; however, as many address concerns relating to radiometric calibration, we have included in the discussion a clear statement that future work must address the radiometric calibration of sensors for IRT-3DDF to be complete (Section 4.2). As this paper was designed to focus on geometric calibration of TIR cameras, we believe this is appropriate. We have addressed additional comments as follows:

1. We have reviewed terms and phrases deemed misleading or inflated. We specify ‘precise and reliable’ to note the stability of calibration procedures and their durability for photogrammetric applications, respectively.
2. We have included LWIR in the table of camera specifications to reiterate the spectral range being discussed. We have also refined the wording of the listed sentence and included references for the application of these methods to various spectral wavelengths (Section 1, para. 1).
3. Although mentioned in the introduction, we have clarified the sentence regarding the WWP designation to make it clear about proximal sensors.
4. We state that the WWP is focussed at infinity in Section 2.3, fixed during manufacturing (although changeable with a specific tool form Workswell if necessary), we hope this is sufficient to cover this point.
5. Finally, regarding the inclusion of Figure 1, although we appreciate readers will likely know what the cameras look like, we find this graphic helpful in reiterating the fusion of independent cameras and useful for visualising the term designations.

Round 2

Reviewer 1 Report

Comments and Suggestions for Authors

Thank you for the author's response. However, the revised manuscript did not adequately address the two key issues I raised.

1. While the author claims that the core innovation of this paper lies in the "3D Field" and its related applications, the method of calibrating targets by distributing markers in space is not original to the authors. In fact, existing calibration methods (e.g., using checkerboards placed at different spatial positions and capturing multiple images) can already achieve accurate parameter calibration—a technique widely adopted in current practices.

2. Regarding the second issue, planar targets introduce the problem of projective coupling. Although the author added some explanations, I believe they failed to clarify this issue sufficiently, either through empirical demonstration or theoretical analysis.

For these reasons, I must maintain my decision to reject this paper.

Author Response

Thank you for your comments. we have endeavoured to address your concerns and have expanded both points below with reference to specific sections:

1. 3D Calibration Fields: Certainly, we do not wish to state the concept of a 3D field as our novelty, and we do not believe this assumption has been made. We know this is a well-established (and preferable) practice not only for traditional digital camera calibration but for thermal cameras also. However, even with this knowledge, they are still seldom used for IRT photogrammetric data fusion approaches. The novelty of our approach was to present a design(notably the individual markers themselves) for others to adopt and refine, importantly for cases when simultaneous calibration is required:
   
   “low-cost, easy-to-manufacture and highly configurable, [using] multi-purpose markers [that] solve several open challenges for IRT-3DDF, notably: (1) their use as configurable calibration markers for lab- and field-based geometric calibration procedures, (2) their use as survey markers for photogrammetric scaling, referencing and validation, and (3) their use as ground truth for radiometric calibration, correction and verification.” (Section 2.2.2)
   
   We believe through the calibration experiments and accompanying IRT-3DDF case studies, we have demonstrated where each of those points have been implemented, presenting the use of the markers not only for calibration, but as practical tools central to this field.
   
   
2. Projective Coupling: We appreciate the reviewer’s comments regarding projective coupling. Frustratingly, MATLAB does not provide any ability to generate a covariance matrix for intrinsics or extrinsic correlations, and the generation of these statistics are outside the scope of this work. Therefore, we have amended the paper to both acknowledge this fact and clarify the inclusion of the term.
   
   Firstly, we believe the references to “susceptibility to projective coupling”, and the accompanying citations, are warranted and well-supported. These have been included to note that the camera network generated when imaging a 2D board (both in terms of geometry and number of images) is proven to produce more imprecise results than a 3D field. This notion has been included to demonstrate that we have attempted to mitigate this through appropriate geometry.
   
   Secondly, we have removed the singular reference to the 2D board having projective coupling between IO and EO values in Section 3.1. Although this is suggested by the significantly poorer performance of the 2D board, we acknowledge we cannot access the values to support this conclusion. Upon running the 2D board points through Australis (as it is not possible in MATLAB), we have seen notable increases between IO/EO correlations when the WWP IO values are generated from 40 or 20 images (removing images with rotation, scale and varying perspective), suggesting greater propensity for projective coupling with fewer images and weaker network (as supported by Dlesk et al., 2021. Transformations…).
   
   To address this, we have included the following statement regarding software choice in Section 4.1.2: “(Australis was the only software to provide access to both IO and IO/EO covariance matrices for the assessment of intra- and inter-image projective coupling).”

We believe that these amendments should satisfy your concerns and clarify the desired points for the reader.