^{1}

^{2}

^{*}

^{1}

^{1}

^{2}

^{2}

This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution license (

3D modeling of cultural heritage objects like artifacts, statues and buildings is nowadays an important tool for virtual museums, preservation and restoration. In this paper, we introduce a method to automatically design a minimal imaging network for the 3D modeling of cultural heritage objects. This becomes important for reducing the image capture time and processing when documenting large and complex sites. Moreover, such a minimal camera network design is desirable for imaging non-digitally documented artifacts in museums and other archeological sites to avoid disturbing the visitors for a long time and/or moving delicate precious objects to complete the documentation task. The developed method is tested on the Iraqi famous statue “Lamassu”. Lamassu is a human-headed winged bull of over 4.25 m in height from the era of Ashurnasirpal II (883–859 BC). Close-range photogrammetry is used for the 3D modeling task where a dense ordered imaging network of 45 high resolution images were captured around Lamassu with an object sample distance of 1 mm. These images constitute a dense network and the aim of our study was to apply our method to reduce the number of images for the 3D modeling and at the same time preserve pre-defined point accuracy. Temporary control points were fixed evenly on the body of Lamassu and measured by using a total station for the external validation and scaling purpose. Two network filtering methods are implemented and three different software packages are used to investigate the efficiency of the image orientation and modeling of the statue in the filtered (reduced) image networks. Internal and external validation results prove that minimal image networks can provide highly accurate records and efficiency in terms of visualization, completeness, processing time (>60% reduction) and the final accuracy of 1 mm.

The documentation of archeological elements through advanced imaging techniques, finally leading to a detailed 3D model of the objects of interest, is currently a hot topic both in commercial as well as in scientific communities. Some examples of using the 3D technology in this field are listed in [

Although software tools which offer support for the entire workflow are available nowadays, a good planning of the initial image acquisition is still necessary in many applications such as in archeology in order to achieve the desired accuracy and reliability of subsequent image processing steps. Moreover, portable physical cultural finds need to be documented first

In the literature few recently presented papers deal with the proper selection and acquisition of images among a large dataset for 3D image-based modeling. Hosseininaveh,

Previously, we introduced the minimal camera network technique [

In order to find the minimal camera network for the 3D modeling of cultural heritage objects, a dense imaging network is filtered on the basis of removing redundant cameras in terms of coverage efficiency and the impact on the total accuracy in the object space or the uncertainty of cameras orientation. The following sections describe the methodology for computing the visibility status and the camera reduction (filtering) technique.

The visibility of object points from the different camera locations is an important factor during the design and filtering of the imaging network. In other words, we should carefully compute for every part of the object of interest, the imaging cameras according to their designed orientation. Different methods can be used to test the visible points like the HPR method [_{dir}. This difference is compared to a threshold (like <90°) to decide the visibility status. It must be noted that the threshold angle is also related to the baseline/depth (B/D) ratio and its magnitude can be selected to satisfy the desired small ratio for the 3D modeling [

The aim of this research is to introduce a new method of finding the minimum set of cameras within a pre-designed dense imaging network, which guarantees the sufficient coverage and accuracy of the 3D modeling of heritage objects. Fraser [

Previously, we published our filtering method for a dense imaging network [_{x}_{y}_{z}

The motivation of using the filtering for point accuracy is based on the well-known relation between the ray intersection geometry and accuracy at the intersection point as shown in

Accordingly, the filtering is based on evaluating the total error in the object space and computing the effect of each camera on this error. The least effective redundant camera in terms of accuracy will be neglected. The whole procedure of filtering will be iterated until reaching the desired accuracy (by error propagation) or when no more redundant cameras are found in the imaging network. The algorithm implementing the old strategy based on coverage and the new strategy based on accuracy is illustrated in

The summarized procedure is:

-Prepare the input information of point cloud, designed external and internal camera parameters, and the initial surface normals.

-Project the points back to the images by collinearity equations [

-Classify the points as over-covered points if they imaged by more than three cameras [

-Consequently, the cameras involved in imaging over-covered points are classified as redundant camera and is subject to filtering out according to the accuracy requirements.

-To filter the redundant cameras based on accuracy, the covariance matrix of every point per camera is computed [

Preferably, additional connection cameras to be added when modeling objects of steep connected faces like buildings [

The motivation of using this method is to find a compromise between the demands of good coverage and high accuracy in one filtering model. Sivanandam, Deepa and Sumathi [

The developed FIS will use specific fuzzy rules to produce the suitability measure (between 0 and 1) of each camera. These rules are set after testing several types and the user can modify the logical rules relying on the type of the problem. The proposed FIS filtering includes a Mamdani-type [

The input parameters are to be computed every iteration: the uncertainty of camera orientation, the number of visible points in each image, the proximity (d) of the image center (principal point p.p.) to the cluster of the object points in the image plane and the area covered through the distribution of the points as shown in

The motive for considering these parameters is illustrated in

The imaging network of

Accordingly, the membership functions are designed by using our own intuition as discussed, which is also inspired by the work of [

The linguistic variables of the number of points (

The linguistic variables of the points distribution (

The linguistic variables of the camera uncertainty (

The linguistic variables for the proximity measure are designed as:

Different rules are designed to comply with the physical problem of filtering redundant cameras in a dense imaging network like the following rules (

(

(

To implement the inference automatically another option of using the Adaptive Neuro-Fuzzy Inference System (ANFIS) can be followed [

ANFIS uses a Sugeno-type fuzzy system [_{i}_{i}

The first step to implement the ANFIS is the learning of information about a data set. The learning and checking for the minimal camera network is based on the simulated network of

To illustrate the methodology of the network reduction procedure, a simulated test is created which consists of eight points and 22 viewing cameras, as shown in ^{2} format size with a B/D ratio of 80%.

To evaluate the expected error in the object points, the standard deviations are estimated by image triangulation [

Accordingly, the two new filtering techniques are implemented to find the minimal camera network that satisfies the coverage and accuracy requirements. The error plot ellipsoid gives a good visual aid for the comparison between the different techniques. In

This simulated test showed the benefit of using each camera filtering technique in the sense of coverage and accuracy of the object points. The estimated errors seem better in the technique of filtering with FIS due to the strong ray’s intersection of 20° which means a wide base imaging. Moreover, the plots in

The study case of the presented approach is tested on the famous Iraqi sculpture “Lamassu” which means “protective spirit” in Akkadian. Lamassu is a human-headed winged bull of over 4.25 m in height. It dates back to the reign of Ashurnasirpal II’s (883–859 BC). In

The imaging network design is planned in an ordered block that results in a ground sample distance of 1mm. The camera used is a Canon 1100D with a focal length of 18 mm which constrains the depth distance between the cameras and the statue body to 2.7 m and a scale of 1/1,500. The expected accuracy of the object points _{XYZ}_{image}_{image}

For the external validation and scaling, 28 reference points are fixed on the Lamassue body of paper stickers with crosses that cannot affect the physical body. A Leica TPS 400 total station was used to measure the 3D coordinates with a local datum as shown in

The reference points are used to define the imaging area by a surface triangulation method using ball pivoting [

The minimal networks are computed by filtering the dense network of 45 images using the three techniques and results in 22 images for filtering with accuracy condition and 25 images with the ANFIS method. To verify the efficiency of the minimal networks after filtering, we used internal and external validations. Moreover, a comparison in terms of time consuming is applied for the automatically oriented imaging networks. The orientation is done by using different commercial and open source software

^{2}) in the image correspondences matching [

For the external validation, the orientation is achieved in two steps. Firstly a relative orientation is done and then followed by an absolute orientation [

The relative accuracy is evaluated in the three networks as well. The longest distance between two reference points is measured from images and compared to its reference length of the total station. The computed relative error in the three networks is 1/20,000 which indicates a highly accurate close range photogrammetric measurements.

The quality of all filtered networks is evaluated by computing the error ellipsoids of the reference points which resulted from the image orientation bundle adjustment as shown in

The results shown in

Finally, a 3D digital model is created for the final documentation of Lamassu.

The filtered camera network ends with a satisfactory 3D model as shown in

The point cloud comparison of

In this paper, we introduced two new methods for finding the sufficient number of images for the 3D modeling of cultural heritage objects like the statues and monuments. The method is based on filtering a pre-designed dense imaging network to the minimal camera network. The minimum number of cameras was attained by using two different strategies of filtering to preserve the coverage and accuracy. The first proposed filtering method classified the object points into over-covered and fair-covered according to the minimum requirement of visibility in three images. Consequently, the cameras that contain most over-covered points are considered as redundant cameras and were investigated for filtering. The filtering is done by cancelling the redundant camera that has the least impact on the total positional accuracy as described in Section 2.2.1.

The second proposed method is developed by building a fuzzy inference system FIS or adaptive neuro-fuzzy inference System ANFIS which uses four input measures of the cameras uncertainty, the number of points per image, proximity to p.p. and their coverage area. Different fuzzy rules are composed to get the final inference of the cameras in an iterative way as described in Section 2.2.2.

The developed methods are tested on an Iraqi heritage statue of Lamassu which belongs to the Ashurnasirpal II era (883–859 BC). For analysis, we designed a dense imaging network (45 images) of three strips, captured with a Canon1100 SLR camera. Three different state-of-the-art software packages are used to automatically process the data and check the possibility of a successful image orientation with less time consuming and sub pixel variance. The filtering with accuracy requirements are tested and resulted in a reduced network of 22 images while the ANFIS method resulted in 25 images (≈50% reduction). The results show a significant decrease in the processing time (approx. 60%) which is quite promising. The final 3D models produced from the minimal imaging networks were shown in

From the above discussion, we can conclude that the minimal imaging network can be computed automatically by filtering a dense imaging network which might be simulated before capture. This is proven to be sufficient for the 3D modeling purpose and accurate in terms of completeness and minimum error.

Concluding general rules to be followed for having a minimal imaging network for 3D modeling is not a practical advice. This is because of the high proficiency needed for image capturing and the effect of the object complexity on the configuration of the minimal network. Therefore, the proposed procedure of the automated simulation and filtering is more suitable to be programmed and then to be used by non-professionals in the field of cultural heritage documentation.

Further work can be investigated on the ANFIS method or other machine learning techniques to get more reliable 3D models in the sense of accuracy and object coverage. Ultimately, future work will be considered on different cultural heritage objects like ancient buildings where an additional requirements like the segmentation into facades may be applied to the task of 3D image based modeling.

The first author would like to introduce his deep gratitude to the University Assistance Fund UAF (

The Iraqi national museum is highly appreciated for their help and understanding.

The authors declare no conflict of interest.

Visibility by using the triangular surface normal vectors.

Intersection geometry and error plot. (

A methodology flowchart of filtering for coverage and accuracy requirements.

Estimated uncertainty in the object space with ellipsoid of errors. (

(

(

A methodology flowchart of filtering with FIS.

The simulated dense camera network for the verification test and the magnitude of imaging coverage and error before filtering.

The minimal networks and the estimated error ellipsoid. (

Gate of Sargon II’s citadel excavation, Khorsabad [

The dense imaging network configuration.

Measuring the reference points.

The uniform mesh of the rough model with surface normals for visibility testing.

The fully automatic orientation of the four networks with different software.

Comparison of the image orientation with different software. (

RMSE of the checkpoints.

The error ellipsoids of the checkpoints. (

The digital 3D models in the four types of networks. (

Comparison between the point clouds resulted from filtering networks and the dense network. (

The cloud to cloud distance computations.

Dense-coverage filtered | 0.002 | 0.150 | 0.009 |

Dense-accuracy filtered | 0.001 | 0.130 | 0.008 |

Dense-FIS filtered | 0.003 | 0.250 | 0.015 |