Review Reports

Round 1

Reviewer 1 Report

Comments and Suggestions for Authors

The authors compared magnetic field maps by direct measurement and Boundary Element Methods (BEM) and showed differences between the two approaches. The authors' conclusion is “the reconstructed fields can be calculated to within 1 mT rms of the directly measured fields”, but in many cases, the relative error is more important. 
The number of points used for fitting in Figure 10 is insufficient. Using cubic fit with four points will obviously over-fitting, and using quadratic fitting with four points is also not appropriate. The authors should appropriately increase the number of mesh size elements before fitting. Other similar questions in other figures should also be checked, it makes the extrapolation to a "0 mm" mesh size is not credible.

Author Response

We thank the reviewer for their comments and their indication that they are willing to sign the report is appreciated. The reviewer has raised two points, both of which we agree highlighted areas where the manuscript could be improved.

Comment 1: The authors compared magnetic field maps by direct measurement and Boundary Element Methods (BEM) and showed differences between the two approaches. The authors' conclusion is “the reconstructed fields can be calculated to within 1 mT rms of the directly measured fields”, but in many cases, the relative error is more important.

Response 1:  We agree with this point. Figures 3c and 19c (previously 2c and 15c) have been updated to show the relative error between the field maps, rather than the absolute error. We originally struggled to quantify the overall relative error due to the particularly large relative errors encountered when measuring very small field components. These large errors are shown by the new plots in Figures 15, 16, 24 and 25. We have decided to calculate the rms relative differences between the fields, but only include measured fields above 1 mT in the calculations to avoid giving large rms errors which are dominated by measurements of the fields outside of the magnet fringe fields. Additional discussion on the relative errors has been added in lines 383-400 for the ZEPTO quadrupole and lines 531-543 for the ECRIS dipole. An updated conclusion that the transverse field components can be calculated to within 5% have been included in the abstract and conclusion and an additional column has been added to the summary Table 1 in the discussion.

Comment 2: The number of points used for fitting in Figure 10 is insufficient. Using cubic fit with four points will obviously over-fitting, and using quadratic fitting with four points is also not appropriate. The authors should appropriately increase the number of mesh size elements before fitting. Other similar questions in other figures should also be checked, it makes the extrapolation to a "0 mm" mesh size is not credible.

Response 2:  We thank the reviewer for this comment and agree that the number of points used were insufficient. We have updated this plot to have data for mesh sizes between 0.3 mm and 1 mm in 0.1 mm steps. This increases the number of points from 4 to 8, and focuses on lower mesh sizes. The 2 mm mesh size used previously was large in comparison to the 6 mm volume dimension. Through this updated plot, it is apparent that there is no quadratic or cubic relationship between the mesh size and difference in calculated fields. Therefore, only a linear fit has been used. Only the linear fit is used for the further discussion on the extrapolation technique and for the plots in Figures 12, 13 and 14 to represent this change. Using only the linear fit, the same conclusion remains that the extrapolation does not lead to better results than can be achieved with a smaller mesh size.

Reviewer 2 Report

Comments and Suggestions for Authors

The present manuscript reports on the reconstruction of the magnetic field using the Boundary Element Method. Two case studies, a quadrupole and a dipole magnet, are examined in detail to test different the accuracy of the BEM and its dependence on various parameters, in particular the mesh.

This rather technical work is sound and carried out systematic. The magnet examples chosen are complementary.

I recommend that this this work be considered for publication in Metrology.

I also suggest the authors to consider the following points:

1)The manuscript covers the topic thoroughly, but in a somewhat lengthy style. Please try to shorten the text. For instance, the time advantage of BEM over direct measurements, or the boundary exceeding the ROI by one mesh size, are mentioned in several places. Consider limiting the redundancies.

2) Several comments are made about the increased computational time and practical computational limits reached when performing calculations for small meshes, for instance L177-L187 or L279. Figure 5 even presents the time taken for part of the calculations process. These remarks would be more informative to the reader if the actual hardware used for these calculations was described in section 2.

3) Figures 10 and 11. I question the legitimacy of performing cubic or quadratic fits on only 4 data points, especially when the points look more randomly scattered than expressing a tendency. In my opinion, the discussion L287 to L350 is made on very shaky ground. This part could be greatly simplified by using only a linear extrapolation and shortened to focus on the essential point: the extrapolation shows little improvement compared to the smallest mesh size examined. Figure 10 could be optimized so as not to show the same caption three times, while enlarging the actual plots.

Author Response

We thank the reviewer for their positive comments and their indication that the manuscript should be published in Metrology. The reviewer has also made a number of constructive suggestions that we have made edits to address.

Comment 1: The manuscript covers the topic thoroughly, but in a somewhat lengthy style. Please try to shorten the text. For instance, the time advantage of BEM over direct measurements, or the boundary exceeding the ROI by one mesh size, are mentioned in several places. Consider limiting the redundancies.

Response 1: We have made the effort to delete repeat statements throughout the manuscript, including the statements: “apart from within one mesh boundary”, “As seen previously, the accuracy of the BEM solutions breaks down within approximately one mesh size of the boundary.”, “The BEM gradients at a point depend on all of the field measurements made around the measurement volume, and the radial basis function used to interpolate between them.”, “This is again due to the magnetic scalar potential calculated in the BEM being a solution to Laplace’s equation, hence ensuring that the divergence of the magnetic field is zero and that random errors are canceled by integration over the boundary.”, “As with the ZEPTO-Q3 results, points at a distance of greater than one mesh length from the boundary must be used to ensure the accuracy of the BEM fields.”, “As discussed for the ZEPTO quadrupole, BEM is likely a more accurate method for determining areas where the field changes regularly due to the BEM forming a magneto-static solution to the reconstructed fields.” There is still some repetition of these points from the main body of the text in the discussion and conclusion sections as these sections are intended to elaborate and summarise the points made in the main body of the text respectively.

Comment 2: Several comments are made about the increased computational time and practical computational limits reached when performing calculations for small meshes, for instance L177-L187 or L279. Figure 5 even presents the time taken for part of the calculations process. These remarks would be more informative to the reader if the actual hardware used for these calculations was described in section 2.

Response 2: The second point was that the reader would benefit from knowledge of the computing hardware used for these calculations. We agree that this influences the results found here, particularly with respect to the achievable mesh sizes. An additional sentence has been added in the Materials and Methods sections to read: “The calculations were performed using a PC with AMD Ryzen 9 7900 X3D 12-core processor, 64 GB physical memory and solid state drive with 1.81 TB available memory. The package Bempp also supports GPU acceleration through OpenCL, however this was not exploited in this work.”

 

Comment 3: Figures 10 and 11. I question the legitimacy of performing cubic or quadratic fits on only 4 data points, especially when the points look more randomly scattered than expressing a tendency. In my opinion, the discussion L287 to L350 is made on very shaky ground. This part could be greatly simplified by using only a linear extrapolation and shortened to focus on the essential point: the extrapolation shows little improvement compared to the smallest mesh size examined. Figure 10 could be optimized so as not to show the same caption three times, while enlarging the actual plots.

Response 3: We thank the reviewer for this comment and agree that the number of points used were insufficient. We have updated this plot to have data for mesh sizes between 0.3 mm and 1 mm in 0.1 mm steps. This increases the number of points from 4 to 8, and focuses on lower mesh sizes. The 2 mm mesh size used previously was large in comparison to the 6 mm volume dimension. Through this updated plot, it is apparent that there is no quadratic or cubic relationship between the mesh size and difference in calculated fields. Therefore, only a linear fit has been used. Only the linear fit is used for the further discussion on the extrapolation technique and for the plots in Figures 12, 13 and 14 to represent this change. Using only the linear fit, the same conclusion remains that the extrapolation does not lead to better results than can be achieved with a smaller mesh size.

Reviewer 3 Report

Comments and Suggestions for Authors

The authors present the application of the BEM in two complex study cases, with both a numerical model and an accurate experimental setup to check for the adequation between simulation and experience. A particular effort has been made to quantify which parameters of the BEM allow for a more precise simulation of the measured fields.

I have two suggestions to give to the authors to improve their work:

1. The measurement system is not shown in the paper, which focuses mainly on the BEM implementation. There is the reference [21] that shows the Hall sensor, but a drawing showing how it is linked to the motion system and to the acquisition system would be useful.
2. The position precision has been given by the authors, without explaining how it has been determined. Such information can be useful to better assess the given precision.

Author Response

We thank the reviewer for their positive comments and instructive suggestions for improvement.

Comment 1: The measurement system is not shown in the paper, which focuses mainly on the BEM implementation. There is the reference [21] that shows the Hall sensor, but a drawing showing how it is linked to the motion system and to the acquisition system would be useful.

Response 1:  A photograph of the measurement system has been added to the manuscript as the new Figure 1. The photograph shows the Hall sensor mounted to the motion stage and connected to the Teslameter. Lines 116-119 and 128-130 have been added to further explain the details of how the probe was mounted and connected to the acquisition system.

Comment 2: The position precision has been given by the authors, without explaining how it has been determined. Such information can be useful to better assess the given precision.

Response 2: The position precision is defined by a tolerance on the set and read positions of the stages set in the software used to control them. We have added the following sentences in lines 121-126 to describe this: “The achieved precisions are defined by an allowable tolerance set in the motion control software between the set and measured absolute position of the stages. The software polls the read position of the stages after sending a movement command. If the read position is not within the given precision set by the tolerance within a set time, a time out error is raised. The set precisions have been found to be practical limits which allow large 3D scans to be performed without timeout errors being regularly raised”

Reviewer 4 Report

Comments and Suggestions for Authors

The paper compares the results of calculation of magnetic field distributions for two different configurations of field sources in the form of permanent magnets with the results of direct measurements. The quadrupole and dipole configurations of field sources of different sizes are considered. It is shown that the experimental determination of the magnetic field distribution requires significantly more time than its modelling. Sources of errors and ways to reduce them are discussed. The proposed approach can be useful in the design of, for example, ondulators.

For the first modelled area of 6x6 mm cross-section the mesh spacing of 2 mm seems to be too coarse, we would like to see the justification of this choice. Perhaps we should have limited the mesh spacing to 1 mm (0.25 to 1).

It is not clear why the calculation error at the edges of the modelled area increases if the calculation is based on the results of measurements at the boundary of the area (figure 3).

Unfortunately, the article lacks information about the size of the used Hall sensor and its error. With a stated displacement step of 25 microns, the size of the sensor can create a noticeable error.

It is not clear how the influence of the Earth's magnetic field was taken into account. It may be the source of the 0.06 mT error (lines 300-301).

Figures must preferably be referenced with ‘Figure’ rather than just a number.

Author Response

The reviewer has made a number of insightful and constructive comments that we have addressed in the following points and we greatly appreciate their indication that they are willing to sign their review.

Comment 1: For the first modelled area of 6x6 mm cross-section the mesh spacing of 2 mm seems to be too coarse, we would like to see the justification of this choice. Perhaps we should have limited the mesh spacing to 1 mm (0.25 to 1).

Response 1:  We agree that this mesh size is large with respect to the 6 mm cross-section dimension as it leaves only a 2x2 mm cross section in which the fields can be evaluated. This mesh size was originally considered in the manuscript due to the good visual agreement between the results and the measured fields in Figure 3 in the original manuscript. We have performed the BEM calculations at a number of different mesh sizes now running from 0.3 mm to 1.0 mm in 0.1 mm steps. It was found that the calculations could be run at 0.3 mm mesh size without error. Figures 4, 5, 7, 8, 10, 11, 12, 13 and 14 have all been updated to consider mesh sizes only in the range 0.3 mm to 1 mm. This allows direct comparison of the rms differences between directly measured and BEM fields over the same range of directly measured points within 1 mesh size of the boundary and hence more accurately demonstrates the advantages of reducing the mesh size.

Comment 2: It is not clear why the calculation error at the edges of the modelled area increases if the calculation is based on the results of measurements at the boundary of the area (figure 3).

Response 2: The second point was concerning the calculation error near the boundaries. This error is due to a term in the boundary element equation (Equation 3) which depends on the distance of the evaluation point from the boundary. This point becomes singular when the evaluation point is on the boundary. For evaluation points within one mesh size of the boundary, this singularity becomes dominant and the results diverge. The following sentence has been added with a reference to lines 214-215 in the manuscript to explain this point: “This behaviour can be explained by the divergence of the boundary integral equations at the boundaries due to the singularity of the Green’s function [31].”

Comment 3: Unfortunately, the article lacks information about the size of the used Hall sensor and its error. With a stated displacement step of 25 microns, the size of the sensor can create a noticeable error.

Response 3: The third point asks for further information on the Hall sensor used. The following sentences have been added to lines 110-115 in the Materials and Methods to give further details on the probe and Teslameter specification as defined by the manufacturer: “The type C probe contains 3 individual sensors and so can measure the three orthogonal field components simultaneously. The spacial resolution of the probe, as stated by the manufacturer, is 30 x 5 x 30 μm3 for the By field component and 100 x 10 x 100 μm3 for the Bx and Bz field components. The planar Hall effect is suppressed through the application of the spinning current technique. The 3MH6 Teslameter provides an accuracy of 0.01% and a precision of 1 μ T.” The reviewer also refers to a stated displacement step of 25 um. It is apparent from the probe resolution that this step size would indeed cause an error. However, we are not certain where in the text the reviewer is referencing this step size. The step sizes between points were 1 mm in the x and y directions and 5 mm in the z direction for the quadrupole and 2 mm in all directions for the dipole.

Comment 4: It is not clear how the influence of the Earth's magnetic field was taken into account. It may be the source of the 0.06 mT error (lines 300-301).

Response 4: The fourth comment asked for clarification on the treatment of Earth’s field in the analysis. We agree that this should be discussed in this manuscript, but do not believe this to be a large source of error. The measurements were performed in a laboratory that is not shielded from Earth’s field, and it is known to have a magnitude of approximately 60 uT. The fields measured using the Hall sensor were the fields directly present in the laboratory and so were a superposition of the Earth’s field and the field from the permanent magnets. As both systems were permanent magnets, it was impossible to turn them “off” to determine the background field within the yoke structure. As the BEM should be capable of reconstructing arbitrary field shapes, the additional presence of the Earth’s field should not have an impact on the accuracy of these results. An additional paragraph in the Discussion section in lines 639-647 has been added to address this in the manuscript.

Comment 5: Figures must preferably be referenced with ‘Figure’ rather than just a number.

Response 5: All references to figures in the manuscript have been changed to refer to “Figure”, rather than just the number.

Reviewer 5 Report

Comments and Suggestions for Authors

The study presents comparison of magnetic field maps by direct measurement and reconstruction using BEM (Boundary Element Methods). The fields reconstructed using BEM were compared to the fields directly measured during the Hall probe scans for two example magnets. The reconstructed fields can be calculated to within 1 mT rms of the directly measured fields.

The study is original and could be interesting for the readers of the journal, however, it cannot be published in the present form.

Comments:

1. The authors clearly explain the BEM advantages for reconstruction of magnetic fields, however, the Introduction lacks more detailed information on the other computational methods.
2. The description of measurements using Hall sensors (“Measurements have been performed using a Senis Type C Hall sensor and 3MH6 Teslameter [21] to calculate the flux density vectors at a set of discrete, regularly spaced points within cuboid measurement domains inside two test magnets”) lacks information about position and distance between sensors, only motion distances were indicated.
3. Shape of the mesh element (trapezoidal, rectangular etc. ) should be accented. Can the BEM use any mesh shape?
4. Fig.2. Right axis would be more informative showing deltaB/IBI, not absolute value.
5. Lines 180-181 please insert “Figure” before “5”: “as shown in Figure 5”.
6. Lines 208-209 : Statement “Therefore, the BEM fields have the correct physical behaviour of the real magnetic fields because they are magneto-static solutions to the field. “ is not clear enough. Please explain in more details.
7. Fig.14. It is difficult to understand where the magnet is placed.
8. It would be good to show comparison for some points with analytical solution.

Author Response

We thank the reviewer for their indication that this manuscript is of interest and for their insightful suggestions in how to improve the manuscript to allow it to be suitable for publication.

Comment 1: The authors clearly explain the BEM advantages for reconstruction of magnetic fields, however, the Introduction lacks more detailed information on the other computational methods.

Response 1: We were unsure of the specific meaning of the reviewers comment here, so we have made the assumption that the reviewer is referring to the use of multipole field harmonics as an alternative for reconstructing the magnetic fields for particle tracking. We have expanded the arguments in lines 26-39 to further comment on the advantages of using stretched wire and rotating coil magnetometer techniques for measuring field harmonics, and the use of field harmonics to calculate fringe fields using generalised gradient techniques. We would be very appreciative if the reviewer could kindly provide further clarity if these lines address the reviewer’s point, or if they refer to some other computational methods for evaluating field measurements.

Comment 2: The description of measurements using Hall sensors (“Measurements have been performed using a Senis Type C Hall sensor and 3MH6 Teslameter [21] to calculate the flux density vectors at a set of discrete, regularly spaced points within cuboid measurement domains inside two test magnets”) lacks information about position and distance between sensors, only motion distances were indicated.

Response 2: The Hall sensor used was a Senis type C 3-axis Hall sensor which contains 3 individual sensors packaged in the same magnetic-field sensitive volume. The following sentences have been added to lines 110-115 in the Materials and Methods to give further details on the probe and Teslameter specification as defined by the manufacturer: “The type C probe contains 3 individual sensors and so can measure the three orthogonal field components simultaneously. The spacial resolution of the probe, as stated by the manufacturer, is 30 x 5 x 30 μm3 for the By field component and 100 x 10 x 100 μm3 for the Bx and Bz field components. The planar Hall effect is suppressed through the application of the spinning current technique. The 3MH6 Teslameter provides an accuracy of 0.01% and a precision of 1 μ T.”

Comment 3: Shape of the mesh element (trapezoidal, rectangular etc. ) should be accented. Can the BEM use any mesh shape?

Response 3: The current version of the programme we have used to perform the calculations, Bempp, only supports triangular surface meshing. The following sentences in lines 139-145 have been added to explain this point: “The package also supports the generation of the triangular mesh used to evaluate the Neumann and Dirichlet data on the boundary of a set of standard grid shapes through the meshio library. The current version of Bempp only supports the generation of triangular surface meshing. Custom triangular meshes can be defined for arbitrary shaped volumes. The restriction to triangular meshes is specific to the software package used and is not a limitation of BEM in general, which can use other mesh shapes.”.

Comment 4: Fig.2. Right axis would be more informative showing deltaB/IBI, not absolute value.

Response 4: We agree with the comment that in many cases the relative error is more informative than the absolute error. Arguments about the relative error were not originally included as the calculated relative errors are large for measurements of low field components (e.g. axial components, points in the quadrupole magnetic centre and points outside the magnet fringe fields). We have added Figures 15, 16, 24 and 25 to demonstrate how the relative errors are over-estimated when considering low fields. This has resulted in the updated conclusion that the transverse field components greater than 1 mT can be reconstructed within 5 % of the direct measurements. The differences in the contour plots in Figures 3 and 19 (previously 2 and 15) have been updated to show the relative error.

Comment 5: Lines 180-181 please insert “Figure” before “5”: “as shown in Figure 5”.

Response 5: All references to figures in the manuscript have been changed to refer to “Figure”, rather than just the number.

Comment 6: Lines 208-209 : Statement “Therefore, the BEM fields have the correct physical behaviour of the real magnetic fields because they are magneto-static solutions to the field. “ is not clear enough. Please explain in more details.

Response 6: The statement has been expanded to read: “For the BEM fields, the output of the BEM model is the scalar potential, which is a solution to Laplace’s equation (Equation 2). The magnetic fields are calculated by differentiation of this potential using Equation 1. This means that the solution for the BEM fields is forced to have 0 divergence. Therefore, these fields conform to the Maxwell equations and form a valid magneto-static solution. Consequently, the BEM fields demonstrate a realistic physical behaviour, which could describe a real magnetic field.” In lines 253-259. We hope this explanation more clearly explains how the solution of the BEM forces the calculated fields to have 0 divergence, which is the behaviour expected of the magnetic field vectors.

 

Comment 7: Fig.14. It is difficult to understand where the magnet is placed.

Response 7: Figure 18 has been added to provide a photograph of the magnet on the measurement bench along with an explanation of the positioning in lines 425-429.

Comment 8: It would be good to show comparison for some points with analytical solution.

Response 8: We are unable to provide an exact analytical solution for the expected fields as both magnets are formed from permanent magnets with soft-magnetic steel used to shape the field. Therefore, the fields measured depend on the BH curves of both the permanent magnets and the steel in the assembly. We are however able to compare the measured and reconstructed field shapes with a finite element simulation (OPERA) of the quadrupole and a boundary element simulation (Radia3D) of the dipole. Figure 7 has been updated to indicate the gradient strength predicted in the Opera model of the quadrupole and lines 245-249 comment on the lower strength seen in the direct and reconstructed measurements. Lines 479-485 have been added to comment on the differences between the strength of the directly measured dipole fields and those predicted using a Radia3D model of the magnet. It is shown that the differences between the measurements and BEM reconstruction (0.01 mT) are less than the differences between the direct measurements and the modelling (0.5 mT).