Fringe Projection Method for 3D High-Resolution Reconstruction of Oil Painting Surfaces

: The fringe projection (FP) method is an outstanding tool for reconstructing painted surfaces. This technique, which has been used for conservation and digitization, does not damage the artwork and can reach sub-millimeter accuracy. To carry out this type of analysis, it is necessary to achieve the most accurate measurements possible. Measuring the precision that a projector-camera-object arrangement can achieve is a complex task. In this paper, we show an experimental method used to measure the accuracy of this technique with instrumentation within the reach of most conservation laboratories. The method consists of capturing, as a reference model, a stepped cylindrical Nylamid ® pyramid, as a construction whose shape, size, and manufacturing accuracy are known with high precision. The pyramid has eight well-deﬁned steps, which are fashioned with an accuracy more exact than that of the fringe projection method. The height of each step was measured, obtaining the mean and variance of the height measurements ﬁtted to a Gaussian distribution. In this work, we show the measured heights of the steps, obtained by varying the period of the fringes. The smallest detectable step height was less than 44.1 µ m; however, this was obtained with a variance in the order of the step height. The smallest detectable step height with a small variance was 0.1008 mm. In addition to this accuracy measurement, a qualitative evaluation of a painting was carried out, ﬁnding the presence of possible superimposed thin layers, fabric, and microcracks, which commonly occur in the drying and aging processes. Further research would provide an experimental measurement of the method’s accuracy and its variance as essential for obtaining a conﬁdence criterion that could then be applied to the model of the painting’s surface.


Introduction
The best possible reconstruction of the shape of an art piece has been a subject of great importance since ancient times, when such reproductions could be done only by mechanical pantograph.Initially, the focus was on obtaining the most precise copy of the artwork.Today, the possibility of replicating the external aspects and the surface of a cultural heritage object with high precision allows for various 3D applications [1].One of the most common applications is to create digital models of the object for museography and cataloging purposes.For this type of use, very high precision is not necessarily required, since a resolution of one millimeter provides acceptable quality [2,3].
Digital models allow a precise memory of the artifact to be preserved in the event of deterioration or loss (due to natural or accidental causes).In addition, they enable databases to be generated that can be used to recognize replicas of original artworks and to create digital museums.These make art accessible to people all over the world by allowing virtual tours and observation of objects from different points of view and with a certain degree of closeness.Lastly, with the advent of 3D printers, the possibility of transforming three-dimensional models into faithful copies of scanned works is now viable, both for the public [4,5] and for reconstructing and reintegrating missing pieces [6].
One of the first institutions to use 3D scanner techniques applied to conservation was the National Research Council of Canada (NRCC).They first utilized this technology in the 1980s to measure artifacts in order to make replicas of them [7].Italy began to explore this field with some interesting and iconic projects, such as the Michelangelo Project (1998) and the Pietà Rondanini Project (1998), both focused on documenting Michelangelo Buonarroti's artworks in Italy.Additionally, other pieces of cultural heritage have been digitally reconstructed for study, such as relics from the Qin Shi Huang Mausoleum and the Terracotta Warriors and Horses Museum (1999), "La Minerva di Arezzo" (2001), parts of the San Giovanni Baptistery in Florence (2001), and some sculptures by Donatello and San Giovanni Pisano (2004) [7].
The possibility of obtaining 3D models that are faithful to the respective artworks enables virtual restoration of specific works and their reintegration through image processing, the return of damaged polychrome sculptures to their original appearance, or the relocation of them virtually to their original architectural context [8,9].
More recent advanced applications of this type of technology in the field of cultural assets include quantitative morphometric analysis for research purposes in support of various scientific fields (e.g., archeology, art history, conservation, and restoration).However, digital models with different resolutions are required for cultural asset applications, depending on the scale and type of the study to be carried out [10][11][12][13].For example, an archaeologist or an art historian can carry out studies with models that have precisions ranging from millimeters to as many as five hundred microns.In contrast, the restorer who wants to study the deformation of the pictorial surface related to a specific type of lining or the state of a wall painting before and after chemical treatment requires details of some tens of microns to at most two hundred microns, an accuracy that not all methods can achieve.
Today, digital scanners based on structured light projection and user-friendly photogrammetric methods are available to many professional laboratories.Structured light projection involves projecting a known pattern of pixels (horizontal lines, grids, or bars) onto a scene [14].The way the pixel pattern deforms when it is projected onto the surface enables computer vision systems to calculate the surface depth of the objects in the scene [15].
Most people think of paintings as two-dimensional because this is what the eye perceives at first glance, but they are in fact three-dimensional objects.As a base support, they may have metal, canvas, or wood panels, or other less common materials such as bone or stone, on which a preparation base and several layers of paint are applied, creating a substrate that has its own topography.Artists, either deliberately or unintentionally, create 3D textural effects on the surface by applying the brushstrokes in certain ways [16][17][18].
Paintings are made up of different layers and materials that significantly influence not only their appearance, but also their chemical and physical behavior during the drying process and, later, the aging process, which involves eventual degradation that is reflected in the surface appearance [18,19].In order to be able to characterize and quantify these phenomena, the professional field of restoration has ventured into applying image analysis systems to photographic documentation, obtaining semi-quantitative results for this purpose [20].
To be able to study these painting surfaces, a three-dimensional model with a very high resolution must be created, in such a way that we can observe and monitor the topographic characteristics of each painting, as well as the potential aging or damage caused by agents such as chemical cleaners, mechanical abrasion, or relining processes [21,22].
The main goal of the present study was to apply and generate a fringe projection (FP) system to create a three-dimensional model with an axial and plane resolution of 100 µm in which the original details and physical characteristics of the objects can be observed.This is very important in order to be able to monitor the state of conservation of an artwork, as well as the success of potential treatments applied to solve specific problems, which can sometimes modify the impasto of the pictorial layer or cause other types of damage.This paper presents the general method, describes the methodology as applied to an oil painting on canvas, and shows how the results obtained can be useful in meeting the objectives.

Materials and Methods
In this paper, we use fringe projection as a 3D reconstruction methodology to be applied in the preservation of cultural heritage.This technique has been employed in various disciplines [23][24][25][26][27][28][29][30][31][32][33] due to its simplicity and measurement accuracy.The general experimental setup and basic principles are described in the following section.

Fringe Projection
Fringe projection is a three-dimensional reconstruction method used to generate a 3D digital model of an object with excellent accuracy and simplicity [29,[34][35][36][37][38][39][40][41][42].A typical setup for a fringe projection system consists of a projection unit (projector) and an image acquisition unit (CCD camera), as shown in Figure 1, as well as an analysis system (computer).The camera is placed perpendicular to the plane where the object under study is located at a distance H.The projector must be situated on the camera axis at a distance D and angle α.
Heritage 2023, 6, FOR PEER REVIEW 3 To be able to study these painting surfaces, a three-dimensional model with a very high resolution must be created, in such a way that we can observe and monitor the topographic characteristics of each painting, as well as the potential aging or damage caused by agents such as chemical cleaners, mechanical abrasion, or relining processes [21,22].
The main goal of the present study was to apply and generate a fringe projection (FP) system to create a three-dimensional model with an axial and plane resolution of 100 µm in which the original details and physical characteristics of the objects can be observed.This is very important in order to be able to monitor the state of conservation of an artwork, as well as the success of potential treatments applied to solve specific problems, which can sometimes modify the impasto of the pictorial layer or cause other types of damage.This paper presents the general method, describes the methodology as applied to an oil painting on canvas, and shows how the results obtained can be useful in meeting the objectives.

Materials and Methods
In this paper, we use fringe projection as a 3D reconstruction methodology to be applied in the preservation of cultural heritage.This technique has been employed in various disciplines [23][24][25][26][27][28][29][30][31][32][33] due to its simplicity and measurement accuracy.The general experimental setup and basic principles are described in the following section.

Fringe Projection
Fringe projection is a three-dimensional reconstruction method used to generate a 3D digital model of an object with excellent accuracy and simplicity [29,[34][35][36][37][38][39][40][41][42].A typical setup for a fringe projection system consists of a projection unit (projector) and an image acquisition unit (CCD camera), as shown in Figure 1, as well as an analysis system (computer).The camera is placed perpendicular to the plane where the object under study is located at a distance H.The projector must be situated on the camera axis at a distance D and angle α.The 3D reconstruction model begins by projecting a sinusoidal pattern (Figure 2) with a well-defined frequency onto the reference plane, and then onto the object surface placed on the reference plane.Images of fringe patterns are acquired with and without the object.Here, the distribution of the heights of the object modulates the pattern phase (distorted pattern).The distorted fringe patterns can be modeled as: I x, y a x, y b x, y cos ϕ x, y Δϕ , The 3D reconstruction model begins by projecting a sinusoidal pattern (Figure 2) with a well-defined frequency onto the reference plane, and then onto the object surface placed on the reference plane.Images of fringe patterns are acquired with and without the object.Here, the distribution of the heights of the object modulates the pattern phase (distorted pattern).The distorted fringe patterns can be modeled as: where a(x, y) is the background light (ambient light), b(x, y) is the modulation of the fringes (fringe contrast), φ(x, y) is the phase term, and ∆φ is the phase shift produced by the object's surface being a certain height above the reference plane.The background and modulation are adjusted to a gray scale to generate black-and-white theoretical sinusoidal fringes (Figure 2).In practice it is difficult to achieve this range of fringes; however, the optical setup must be adjusted to obtain the result closest to the ideal case.The critical task is to recover the phase of the fringe image captured, because the phase contains the information giving the height of the object.There are several techniques for phase recovery, such as stepping, triangular phase stepping, or π-shift FTP, among others [35][36][37].The phase stepping technique [35] is notable for the small number of images required and for its excellent precision.Phase stepping involves the capture of N-images (three as a minimum) with a known N-step of fringe shifting.The N-step images are modeled by the following equation in which the spatial component is omitted: where n represents the phase-shift index n = 0, 1, 2, . . ., N − 1.Based on the fringe steps, a system of equations can be solved to obtain the phase.The solution to the system of equations is expressed as: where φ w (x, y) is the wrapped phase.Figure 2 shows an example of fringe images with 3-step phase shifting, produced using Equation ( 1).
where a x, y is the background light (ambient light), b x, y is the modulation of the fringes (fringe contrast), ϕ x, y is the phase term, and Δϕ is the phase shift produced by the object's surface being a certain height above the reference plane.The background and modulation are adjusted to a gray scale to generate black-and-white theoretical sinusoidal fringes (Figure 2).In practice it is difficult to achieve this range of fringes; however, the optical setup must be adjusted to obtain the result closest to the ideal case.The critical task is to recover the phase of the fringe image captured, because the phase contains the information giving the height of the object.There are several techniques for phase recovery, such as stepping, triangular phase stepping, or π-shift FTP, among others [35][36][37].
The phase stepping technique [35] is notable for the small number of images required and for its excellent precision.Phase stepping involves the capture of N-images (three as a minimum) with a known N-step of fringe shifting.The N-step images are modeled by the following equation in which the spatial component is omitted: where n represents the phase-shift index n 0,1,2, … , N 1 .Based on the fringe steps, a system of equations can be solved to obtain the phase.The solution to the system of equations is expressed as: where ϕ w x, y is the wrapped phase.Figure 2 shows an example of fringe images with 3-step phase shifting, produced using Equation (1).The drawback of this technique is that the phase is wrapped.The main problem is that an error in unwrapping affects not only a single pixel, but propagates.When the captured object has edges or abrupt changes, the phase unwrapping process is more complicated.Fortunately, in most cultural heritage and art objects, there are few cases where the objects have abrupt changes; therefore, the phase stepping method can be applied in most cases without difficulty.In this work, we used the unwrapping method proposed by Barrientos et al. [38].

Experimental Arrangement
In the previous section, we described the geometrical arrangement of the method; here, we describe the specific configuration used in our experiments.Following the geometry shown in Figure 1, the distance between the reference plane and the camera lens was 24 cm, the illumination angle of the projector was 30° (α), and the distance between the projector lens and the object was 39 cm. Figure 3 shows the equipment used.In this arrangement, we used a Lumenera ® LW11057M high-resolution camera (4008 × 2672 pixels) with a Nikon 60 mm focal length lens and a multimedia Epson EH-TW6100 high- The drawback of this technique is that the phase is wrapped.The main problem is that an error in unwrapping affects not only a single pixel, but propagates.When the captured object has edges or abrupt changes, the phase unwrapping process is more complicated.Fortunately, in most cultural heritage and art objects, there are few cases where the objects have abrupt changes; therefore, the phase stepping method can be applied in most cases without difficulty.In this work, we used the unwrapping method proposed by Barrientos et al. [38].

Experimental Arrangement
In the previous section, we described the geometrical arrangement of the method; here, we describe the specific configuration used in our experiments.Following the geometry shown in Figure 1, the distance between the reference plane and the camera lens was 24 cm, the illumination angle of the projector was 30 • (α), and the distance between the projector lens and the object was 39 cm. Figure 3 shows the equipment used.In this arrangement, we used a Lumenera ® LW11057M high-resolution camera (4008 × 2672 pixels) with a Nikon 60 mm focal length lens and a multimedia Epson EH-TW6100 high-resolution video projector (1920 × 1080 pixels).The fringe periods varied from 0.5 to 8 mm.For all images, we employed four equally spaced phase shifts: 0, π/2, π, and 3π/2.

Processing of Captured Images
Once the fringe stepping images are captured without and with the object, the phase needs to be recovered and unwrapped, and the phase units converted to height (calibrated).The expression for recovering the phase from captured images, which in this work was four, can be obtained from Equation (3), as follows: where I 1 , I 2 , I 3 , and I 4 correspond to displacements of 0, π/2, π, and 3π/2, respectively.The phase-recovered ϕ w is involved.Fortunately, the pictorial surfaces do not present discontinuities, and drift is low.The algorithm we employed in this work was that proposed by Barrientos et al. [38]).They developed a robust method for 2D phase unwrapping using the fast cosine transform.Once the phase is unwrapped, ϕ u , it is necessary to convert from phase to metric units.The conversion expression is as follows: Δz , where Δϕ is the phase difference and T ′ the equivalent period, calculated using In the case of paint on canvas, it is necessary to separate the deformation of the canvas from the paint strokes.For this, regional minima of the whole model are detected and interpolated with a fourth-order polynomial.The interpolation contributed by the polynomial represents the overall shape of the canvas.Subtracting the shape of the canvas from the model yields the texture map corresponding to the painting.For more details on the deduction of formulas 1-3 and a description of the phase-stepping technique, we suggest the work published by Zuo et al. [43].

Resolution Tests on Reference Figures with Known Geometry
Accuracy is a crucial issue in the metrology of paintings on canvas.A high-accuracy 3D map is helpful for the conservation and restoration of works of art.To understand the precision of our digital models, it is necessary to compare them with objects whose dimensions and surface irregularities are perfectly known.For this reason, to calibrate the fringe projection method and make a reliable estimate of the accuracy obtained in the reconstruction of the models, a physical model was constructed as a reference, with the shape, size, and manufacturing accuracy known with high precision.

Processing of Captured Images
Once the fringe stepping images are captured without and with the object, the phase needs to be recovered and unwrapped, and the phase units converted to height (calibrated).The expression for recovering the phase from captured images, which in this work was four, can be obtained from Equation (3), as follows: where I 1 , I 2 , I 3 , and I 4 correspond to displacements of 0, π/2, π, and 3π/2, respectively.The phase-recovered φ w is involved.Fortunately, the pictorial surfaces do not present discontinuities, and drift is low.The algorithm we employed in this work was that proposed by Barrientos et al. [38]).They developed a robust method for 2D phase unwrapping using the fast cosine transform.Once the phase is unwrapped, φ u , it is necessary to convert from phase to metric units.The conversion expression is as follows: where ∆φ is the phase difference and T the equivalent period, calculated using In the case of paint on canvas, it is necessary to separate the deformation of the canvas from the paint strokes.For this, regional minima of the whole model are detected and interpolated with a fourth-order polynomial.The interpolation contributed by the polynomial represents the overall shape of the canvas.Subtracting the shape of the canvas from the model yields the texture map corresponding to the painting.For more details on the deduction of formulas 1-3 and a description of the phase-stepping technique, we suggest the work published by Zuo et al. [43].

Resolution Tests on Reference Figures with Known Geometry
Accuracy is a crucial issue in the metrology of paintings on canvas.A high-accuracy 3D map is helpful for the conservation and restoration of works of art.To understand the precision of our digital models, it is necessary to compare them with objects whose dimensions and surface irregularities are perfectly known.For this reason, to calibrate the fringe projection method and make a reliable estimate of the accuracy obtained in the reconstruction of the models, a physical model was constructed as a reference, with the shape, size, and manufacturing accuracy known with high precision.
The model in question was a stepped cylindrical pyramid which was used to test the resolution of the out-of-plane FP method.At the pyramid's top surface, three slots were made, each with a specific width and depth, to test the in-plane resolution.To minimize alterations produced by scattering, the material used to fabricate this model was dark gray Nylamid ® .
The circular pyramid was fabricated with highly accurate step heights.The pyramid is shown in Figure 4C.It had eight steps, with heights of 2.0993 mm, 1.0895 mm, 0.5855 mm, 0.3151 mm, 0.1836 mm, 0.1008 mm, 0.0441 mm, and 0.0141 mm.At the top step, there were three concentric channels with widths of 0.9267 mm, 0.4327 mm, and 0.2325 mm.These channels were used for the "x" and "y" calibration.A detailed diagram is shown in Figure 4A.The heights of the steps and the roughness were accurately confirmed using a high precision digital 3D-touch BFWA 10-45-2/90 • roughness meter probe.The accuracy of the height of the pyramid was 25 µm.A profile of the roughness is shown in Figure 4B.

PEER REVIEW 6
The model in question was a stepped cylindrical pyramid which was used to test the resolution of the out-of-plane FP method.At the pyramid's top surface, three slots were made, each with a specific width and depth, to test the in-plane resolution.To minimize alterations produced by scattering, the material used to fabricate this model was dark gray Nylamid ® .
The circular pyramid was fabricated with highly accurate step heights.The pyramid is shown in Figure 4C.It had eight steps, with heights of 2.0993 mm, 1.0895 mm, 0.5855 mm, 0.3151 mm, 0.1836 mm, 0.1008 mm, 0.0441 mm, and 0.0141 mm.At the top step, there were three concentric channels with widths of 0.9267 mm, 0.4327 mm, and 0.2325 mm.These channels were used for the "x" and "y" calibration.A detailed diagram is shown in Figure 4A.The heights of the steps and the roughness were accurately confirmed using a high precision digital 3D-touch BFWA 10-45-2/90° roughness meter probe.The accuracy of the height of the pyramid was 25 µm.A profile of the roughness is shown in Figure 4B.

Case Study: The Rooster.
The painting used in the experimental procedure, The Rooster, is an oil painting on linen canvas painted in part using a spatula technique.It was painted by Pablo Gonzales in 2001 and is 30 cm wide by 40 cm tall.Figure 5 shows a color image of The Rooster.

Case Study: The Rooster
The painting used in the experimental procedure, The Rooster, is an oil painting on linen canvas painted in part using a spatula technique.It was painted by Pablo Gonzales in 2001 and is 30 cm wide by 40 cm tall.Figure 5 shows a color image of The Rooster.This painting presents a number of complex issues in terms of color, with b dark and very bright areas.The surface of the painting has areas where the p applied with a soft-bristled brush, generating a very thin layer, as well as areas w dom apices and thickly plastered oil paint, which are typical of the application of a spatula.Thus, the painting covers a variety of possible scenarios.

Results and Discussions
This section presents and discusses the results obtained from testing, in ways, the fringe projection method.The quantitative results obtained by analy reference object whose dimensions and roughness were known with high prec presented.In addition, a qualitative description of how the method performed o painting with different fringe periods is provided, noting its limits and potential.gies of real details of pictorial surfaces that can be observed and measured at th tion achieved by the method are also described.

FPS Resolution Limit Test Using a Stepped Cylindrical Pyramid as Reference
In order to test the FP resolution limit and the influence of resolution on the a of the DEM, the pyramid (Figure 4C) was captured, varying the optical sensitiv fringe periods were used: 0.5 mm, 1 mm, 2 mm, 4 mm, and 8 mm.The maximum tion achieved with the experimental setup was obtained using the 0.5 mm fringe Figure 6 shows a capture of the pyramid.Figure 7 shows the reconstructions and This painting presents a number of complex issues in terms of color, with both very dark and very bright areas.surface of the painting has areas where the paint was applied with a soft-bristled brush, generating a very thin layer, as well as areas with random apices and thickly plastered oil paint, which are typical of the application of oils with a spatula.Thus, the painting covers a variety of possible scenarios.

Results and Discussions
This section presents and discusses the results obtained from testing, in different ways, the fringe projection method.The quantitative results obtained by analyzing the reference object whose dimensions and roughness were known with high precision are presented.In addition, a qualitative description of how the method performed on an oil painting with different fringe periods is provided, noting its limits and potential.Typologies of real details of pictorial surfaces that can be observed and measured at the resolution achieved by the method are also described.

FPS Resolution Limit Test Using a Stepped Cylindrical Pyramid as Reference
In order to test the FP resolution limit and the influence of resolution on the accuracy of the DEM, the pyramid (Figure 4C) was captured, varying the optical sensitivity.Five fringe periods were used: 0.5 mm, 1 mm, 2 mm, 4 mm, and 8 mm.The maximum resolution achieved with the experimental setup was obtained using the 0.5 mm fringe period.Figure 6 shows a capture of the pyramid.Figure 7 shows the reconstructions and profiles of the reference piece using different fringe periods.It can easily be observed that using small periods decreases the residual noise.The image of the three-dimensional model was processed by level sets that corresponded to the steps.At each level set the distribution of heights was obtained.Figure 8A shows an example.This distribution, as expected, has a Gaussian behavior, which enables the degree of uncertainty of the measurements to be determined.Figure 8B shows the mean and standard deviation of the Gaussian distributions of each step.
As seen in Figure 8B, for the five highest steps (≥0.1836 mm), the mean value of the measurement is consistent with the real value, and the standard deviation is smaller when the period is smallest.The 0.0141 mm and 0.0441 mm steps are detectable, but unfortunately the variance is in the order of the step height.This fringe projection setup was able to detect up to 0.1008 mm with high precision and with a degree of uncertainty up to 0.0125 mm.
In addition, to provide the actual resolution of the method, this analysis included an evaluation of how the variation in the period of the fringes influences the resolution of the method.It can be observed that, while a fringe width of 0.5 mm allows all the steps to be perceived and accurately measures the 0.1008 mm step with a fringe period of 1 mm, it is only possible to measure the 0.1813 mm step.Moreover, upon observing the errors associated with the measurements, it can be clearly seen how the uncertainty increases as the period of the fringes become progressively larger.The image of the three-dimensional model was processed by level sets that corresponded to the steps.At each level set the distribution of heights was obtained.Figure 8A shows an example.This distribution, as expected, has a Gaussian behavior, which enables the degree of uncertainty of the measurements to be determined.Figure 8B shows the mean and standard deviation of the Gaussian distributions of each step.

Result of the Experimental Arrangement Applied to The Rooster
In the previous section, the results of the quantitative measurement of the reference objects were presented from a methodological point of view.Now, we will show the results from the application of the same techniques to the painting we chose for our case study, giving a qualitative description of how the method performs, its scope, and its potential applications.As we mentioned, The Rooster is an oil painting on linen canvas, painted using in part a spatula technique.This technique generates thick applications of paint with a strong relief that provides a greater dynamic to the painting.Figure 9A is a real color image of the work.To show the application of the method, we captured the complete work with two fringe periods: 2 mm, as shown in Figure 9B, and 1 mm, as shown in Figure 9C.We discuss the process at two scales: analyzing the complete model (global) and analyzing a region (local).In the global analysis, we can see that both photographs clearly show where there is thicker paint, applied with a relief effect that results from the angles of the spatula.The As seen in Figure 8B, for the five highest steps (≥0.1836 mm), the mean value of the measurement is consistent with the real value, and the standard deviation is smaller when the period is smallest.The 0.0141 mm and 0.0441 mm steps are detectable, but unfortunately the variance is in the order of the step height.This fringe projection setup was able to detect up to 0.1008 mm with high precision and with a degree of uncertainty up to 0.0125 mm.
In addition, to provide the actual resolution of the method, this analysis included an evaluation of how the variation in the period of the fringes influences the resolution of the method.It can be observed that, while a fringe width of 0.5 mm allows all the steps to be perceived and accurately measures the 0.1008 mm step with a fringe period of 1 mm, it is only possible to measure the 0.1813 mm step.Moreover, upon observing the errors associated with the measurements, it can be clearly seen how the uncertainty increases as the period of the fringes become progressively larger.

Result of the Experimental Arrangement Applied to the Rooster
In the previous section, the results of the quantitative measurement of the reference objects were presented from a methodological point of view.Now, we will show the results from the application of the same techniques to the painting we chose for our case study, giving a qualitative description of how the method performs, its scope, and its potential applications.As we mentioned, The Rooster is an oil painting on linen canvas, painted using in part a spatula technique.This technique generates thick applications of paint with a strong relief that provides a greater dynamic to the painting.Figure 9A is a real color image of the work.
To show the application of the method, we captured the complete work with two fringe periods: 2 mm, as shown in Figure 9B, and 1 mm, as shown in Figure 9C.We discuss the process at two scales: analyzing the complete model (global) and analyzing a region (local).
In the global analysis, we can see that both photographs clearly show where there is thicker paint, applied with a relief effect that results from the angles of the spatula.The texture is relatively thick, for example, in the tail of the rooster.With respect to these characteristics, at the scale considered, there are no distinguishable differences.However, if we perform an analysis focused on certain regions of the painting, we can observe a notable difference between the models.An example can be seen in the upper left corner of the painting.
objects were presented from a methodological point of view.Now, we will show the results from the application of the same techniques to the painting we chose for our case study, giving a qualitative description of how the method performs, its scope, and its potential applications.As we mentioned, The Rooster is an oil painting on linen canvas, painted using in part a spatula technique.This technique generates thick applications of paint with a strong relief that provides a greater dynamic to the painting.Figure 9A is a real color image of the work.To show the application of the method, we captured the complete work with two fringe periods: 2 mm, as shown in Figure 9B, and 1 mm, as shown in Figure 9C.We discuss the process at two scales: analyzing the complete model (global) and analyzing a region (local).In the global analysis, we can see that both photographs clearly show where there is thicker paint, applied with a relief effect that results from the angles of the spatula.The  Figure 10B shows how changing the fringe period, which corresponds to a change in resolution, produces a significant difference in the image.The changes in the elevation pattern are evident.In Figure 10B with 1 mm stripes (higher resolution), coarse details are not recognized, and a slightly lowered vertical zone (dark stripe) can be noted in the center.It can be seen that this corresponds to a smooth area with little relief.In contrast, in the lower resolution image (2 mm stripes), the deeper central area is completely lost and artifacts appear on the right and left margins of the image where raised areas are indicated that do not actually exist.The reason for these apparent differences is the presence of details below the resolution limit being merged together, creating fictitious detail.In Figure 10C, significant differences can be observed between the images using the 1 mm and those using the 2 mm fringe period.As the period of the fringe increases, a general decrease in resolution is noted, with the disappearance of minute linear details and the appearance of broader, blurred features.Fictitious black crater-like structures, which do not exist in reality, appear in the center of the image along a narrow vertical band.A change in the overall dynamics of the image is also noticeable.It is interesting to note that in the 2 mm fringe image, periodic vertical bands related to the fringes appear.In the image with 1 mm stripes, the vertical bands can be discerned, but they are much less evident.In Figure 10D (1 mm period), very fine details are observed as aligned ridges, with a tendency to verticality, and tiny peaks on the edges of the impasto, which are probably due to irregularities on the edge of the ridges, generated by the spatula and/or the brushstrokes of the base paint which is applied as a base prior to the addition of the figure.Very slight ripples appear against an almost perfectly flat background, as seen in the large smooth gray area at the top left of the image.In the 2 mm fringe image, these very fine details lose their definition and become blurry fringes, hidden by black shadows that are not present in the higher-resolution image.The change in the sharpness of the image is striking.Figure 10E is a typical example whereby, upon decreasing the resolution (2 mm fringes), the pinpoint details become wider and more blurred, and black spots (craters) appear that do not exist in reality.The effect of the longer period is also noticeable in the appearance of a vertical stripe pattern due to the fringes.In Figure 10F, captured in a very darkly pigmented area of the painting, the limit of the fringe projection method is clearly seen.The pattern of vertical lines due to the fringes is visible in both images (1 mm and 2 mm fringe).However, the higher-resolution image (1 mm period) has finer details and the stripes are slightly less visible.
acteristics, at the scale considered, there are no distinguishable differences.However, if we perform an analysis focused on certain regions of the painting, we can observe a notable difference between the models.An example can be seen in the upper left corner of the painting.
Figure 10A again shows the original painting.The green frames indicate regions of interest in the chart that have been digitized with fringes of different periods: 1 mm (lefthand column) and 2 mm (right-hand column).These are described below.Figure 10B shows how changing the fringe period, which corresponds to a change in resolution, produces a significant difference in the image.The changes in the elevation pattern are evident.In Figure 10B with 1 mm stripes (higher resolution), coarse details are not recognized, and a slightly lowered vertical zone (dark stripe) can be noted in the center.

Conclusions
In this work, we presented a high-precision experimental method to estimate the accuracy of a capture system based on the fringe projection technique.A reference object with well-defined height levels was fabricated, and an instrument verified the levels with a resolution greater than 10 µm.To estimate the error, a Gaussian distribution was fitted to the heights of the points at each level.Using the best experimental setup (fringe period of 0.5 mm), the lowest detectable level was 0.0441 mm; however, the standard deviation was in the order of step height.The second measurable step was 0.1008 mm with a standard deviation of 0.0125 mm.In this way, it was confirmed that the precision of the method is of the order of one-tenth of a millimeter, a resolution capable of characterizing and measuring most of the details of the technical characteristics of a pictorial surface and conservation state.Some examples of these include the thickness of the paint layers, direction of the brushstrokes, and the variations in relief of the impasto throughout the pictorial surface.In terms of the conservation state, various alterations of the pictorial layer can be observed in close detail, such as microfractures, craquelure pattern, or effects of metal soaps, among others.A resolution of this order of magnitude also makes it possible to perform quantitative measurements of deformations in the painting's support or to evaluate the effects of conservative interventions (such as consolidating or lining the pictorial surface) in terms of loss of the relief dynamics of the artwork.
Statistical analysis also provided quantitative data concerning the effect of the fringe period on the resolution.Our study showed that increasing the fringe period from 0.5 to 1 mm worsens the resolution from 0.1008 to 0.1813 mm, respectively.Increasing the period of the fringes also considerably increases the uncertainty of the measurements.
A qualitative analysis of the 3D reconstruction of the oil painting The Rooster was carried out as a case study to explore the potential of the method for art restoration.Although it was a challenge to capture this painting with a fringe projection system since it displays a large color dynamic range and areas with high contrast tones, the resulting digital model was of good quality.Visual inspection of the 3D model confirmed that with the resolution used, it is possible to measure, and eventually quantify, thin layers of paint, brushstrokes, fillings, paint detachment, craquelure, metal soaps, state of the fabric, or microfractures in the paint layers generated by the drying or aging processes.In addition, it was possible to learn through visual inspection how the increase in the period of the fringes not only produces a loss of resolution, but also introduces fictitious details, which do not exist in reality and produce appreciable changes in the apparent topography of the paint.
This method has some limitations, such as the appearance of fictitious features due to subsampling, and the presence of bands that appear in the direction of the grid of the fringe.This is a problem that is always present in models obtained by the fringe projection method, although it is significantly reduced by using small fringes or by post-processing with de-striping algorithms.This method may also suffer significant loss of resolution in strong lighting conditions, such as outdoors or in paintings that have very dark, absorbent areas or strong color contrasts.
The experimental estimation, with measurement of the precision error, enabled us to establish a degree of objective certainty in capturing the characteristics and the deterioration of the surfaces of the paintings.This may prove to be of great utility and importance for conserving cultural heritage.
The fringe projection method is very effective for producing digital models of paintings with high precision and in a reasonably short time (minutes to hours, depending on the size and complexity of the artwork).This method is within the reach of any conservation laboratory.

Figure 1 .
Figure 1.Typical fringe projection setup for three-dimensional reconstruction of objects.

Figure 1 .
Figure 1.Typical fringe projection setup for three-dimensional reconstruction of objects.

Figure 2 .
Figure 2. Example of fringe images with 3-step phase shifting.

Figure 2 .
Figure 2. Example of fringe images with 3-step phase shifting.

Figure 4 .
Figure 4. Circular pyramid (8 steps) used to test fringe projection.This pyramid was manufactured from Nylamid ® using a lathe.The top step has three concentric channels which were useful for plane calibration.(A) Detailed diagram of the size of the steps and concentric channels.(B) Roughness profile.(C) Rendering of pyramid.

Figure 4 .
Figure 4. Circular pyramid (8 steps) used to test fringe projection.This pyramid was manufactured from Nylamid ® using a lathe.The top step has three concentric channels which were useful for plane calibration.(A) Detailed diagram of the size of the steps and concentric channels.(B) Roughness profile.(C) Rendering of pyramid.

Figure 5 .
Figure 5. Color image of the The Rooster (2001) by Pablo Gonzales.Technique: oil with b spatula on linen.

Figure 5 .
Figure 5. Color image of the The Rooster (2001) by Pablo Gonzales.Technique: oil with brush and spatula on linen.

Heritage 2023, 6 ,Figure 6 .
Figure 6.Example of pyramid reconstruction.(A) Three-dimensional surface plot.(B) Two-dimensional plot using color value.The color is related to the value of the height.

Figure 6 .Figure 7 .
Figure 6.Example of pyramid reconstruction.(A) Three-dimensional surface plot.(B) Two-dimensional plot using color value.The color is related to the value of the height.Heritage 2023, 6, FOR PEER REVIEW 9

Figure 7 .
Figure 7. Reconstruction of the pyramid with different fringe periods.

Heritage 2023, 6 ,Figure 8 .
Figure 8. (A) Distribution of heights for the 2.0993 mm step using the fringe with 0.5 mm period.Measured heights are shown in black and the best fit of a normal distribution in red.(B) Means and standard deviations of all steps and periods.

Figure 9 .
Figure 9.The Rooster.Oil painting on linen with spatula technique.In (A) RGB color image of the work, in (B) 3D model with false color captured with 2 mm period fringes, and in (C) the model captured with 1 mm fringes.

Figure 8 .
Figure 8. (A) Distribution of heights for the 2.0993 mm step using the fringe with 0.5 mm period.Measured heights are shown in black and the best fit of a normal distribution in red.(B) Means and standard deviations of all steps and periods.

Figure 9 .
Figure 9.The Rooster.Oil painting on linen with spatula technique.In (A) RGB color image of the work, in (B) 3D model with false color captured with 2 mm period fringes, and in (C) the model captured with 1 mm fringes.

Figure 9 .
Figure 9.The Rooster.Oil painting on linen with spatula technique.In (A) RGB color image of the work, in (B) 3D model with false color captured with 2 mm period fringes, and in (C) the model captured with 1 mm fringes.

Figure
Figure 10A again shows the original painting.The green frames indicate regions of interest in the chart that have been digitized with fringes of different periods: 1 mm (left-hand column) and 2 mm (right-hand column).These are described below.Figure10Bshows how changing the fringe period, which corresponds to a change in resolution, produces a significant difference in the image.The changes in the elevation pattern are evident.In Figure10Bwith 1 mm stripes (higher resolution), coarse details are not recognized, and a slightly lowered vertical zone (dark stripe) can be noted in the center.It can be seen that this corresponds to a smooth area with little relief.In contrast, in the lower resolution image (2 mm stripes), the deeper central area is completely lost and artifacts appear on the right and left margins of the image where raised areas are indicated that do not actually exist.The reason for these apparent differences is the presence of details below the resolution limit being merged together, creating fictitious detail.In Figure10C, significant differences can be observed between the images using the 1 mm and those using the 2 mm fringe period.As the period of the fringe increases, a general decrease in resolution is noted, with the disappearance of minute linear details and the appearance of broader, blurred features.Fictitious black crater-like structures, which do not exist in reality, appear in the center of the image along a narrow vertical band.A change in the overall dynamics of the image is also noticeable.It is interesting to note that in the 2 mm fringe image, periodic vertical bands related to the fringes appear.In the image with 1 mm stripes, the vertical bands can be discerned, but they are much less evident.In Figure10D(1 mm period), very fine details are observed as aligned ridges, with a tendency to verticality, and tiny peaks on the edges of the impasto, which are probably due to irregularities on the edge of the ridges, generated by the spatula and/or the brushstrokes of the base paint which is applied as a base prior to the addition of the figure.Very slight ripples appear against an almost perfectly flat background, as seen in the large smooth gray area at the top left of the image.In the 2 mm fringe image, these very fine details lose their definition and become blurry fringes, hidden by black shadows that are not present in the higher-resolution image.The change in the sharpness of the image is striking.Figure10Eis a typical example whereby, upon decreasing the resolution (2 mm fringes), the pinpoint details become wider and more blurred, and black spots (craters) appear that do not exist in reality.The effect of the longer period is also noticeable in the appearance of a vertical stripe pattern due to the fringes.In Figure10F, captured in a very darkly pigmented area of the painting, the limit of the fringe projection method is clearly seen.The pattern of vertical lines due to the fringes is visible in both images (1 mm and 2 mm fringe).However, the higher-resolution image (1 mm period) has finer details and the stripes are slightly less visible.

Figure 10 .
Figure 10.(A) Within the original color painting, the green boxes indicate the location of the selected areas.For these areas, by the fringe projection method, digital models were obtained using two different fringe steps.(B-F) The two different fringe steps: 1 mm fringe period (left-hand column) and 2 mm (right-hand column).The macroscopic differences in the models, using different fringe steps, can be observed.

Figure 10 .
Figure 10.(A)Within the original color painting, the green boxes indicate the location of the selected areas.For these areas, by the fringe projection method, digital models were obtained using two different fringe steps.(B-F) The two different fringe steps: 1 mm fringe period (left-hand column) and 2 mm (right-hand column).The macroscopic differences in the models, using different fringe steps, can be observed.