Positional Accuracy Assessment of Digital Orthophoto Based on UAV Images: An Experience on an Archaeological Area

Abstract

Rapid development in UAV (unmanned aerial vehicle) photogrammetry made it preferable in many applications including cultural heritage documentation. Usability, quickness and accuracy of digital images have grabbed also the attention of archaeologists. Especially orthoimages by UAVs have become considerably significant in the field of archaeological heritage documentation since they are fast and accurate images of the object with high detailed information. However their accuracy and quality are the most important features of these images for archaeological documentation. The aim of this paper is to evaluate horizontal and vertical accuracy of an orthophoto taken by a fixed-wing UAV in an archaeological area. The evaluation is made according to ASPRS (American Society for Photogrammetry and Remote Sensing) Accuracy Standards for Digital Geospatial Data. The archaeological area, the name of which is Kubad Abad Palace in Beyşehir Province in Konya, is the only Anatolian Seljuk Palace structure that has survived to the present day. The study describes the orthophoto generation process and positional accuracy evaluation results within the frame of the importance of accuracy for archaeological documentation.

1. Introduction

With the developments in image capturing and image processing, UAVs have become an alternative for cultural heritage documentation and have started to be used in diverse heritage projects. In a cultural heritage area, UAV applications are mainly focused on aerial documentation [1], observation, monitoring [2], mapping [3], 3D modeling [4] as well as producing digital orthophoto, digital elevation model (DEM) and digital surface models (DSMs) [5].

In archaeological projects, there is always a need for accurate documentation of the objects. Even though fast, accurate, cheap modeling and visualization of archaeological areas are a demand, there are some situations making this demand difficult. The complexity of archaeological areas, their geometric and radiometric features and their conceptual interpretation are among these situations.

Compared with traditional documentation techniques in archaeological areas, UAVs have brought stunning innovation and ease to get aerial images of the total area. The information obtained from aerial or satellite images provides an overview of the area, which is fundamental for the interpretation of archaeological structures. In fact, the images obtained by metric aerial cameras (film and digital) or by high-resolution satellite sensors have been used in archaeology for a long time [6]. It should be pointed out that such images have some limitations. Many times geometric resolution of these images is inadequate for detailed studies. Another limitation is related to the periods of acquisition which do not always correspond to given particularly useful data for the purposes of the archaeological work and ultimately to the expense [7]. Up to today, diverse kinds of manned and unmanned platforms have been used in archaeology in order to get aerial images. Among them, there are mats, poles, booms, towers as well as kites, balloons, blimps and helikites [8]. Whittlesey reported on the use of a tethered balloon for archaeological documentation for the first time in 1970, which he had actually started in 1967 [9]. Even though airborne photogrammetric surveys are used for archaeological area documentation, in some cases, they can be unfeasible because of the limited site extent, the large scale required and the expected low flight height, speed of the aircraft and the relatively high cost of the technique [10].

With the development of light unmanned aerial vehicles (UAVs), the number of studies has increased considerably in the last years. In literature, much diverse research and studies about the use of UAVs for archaeological purposes can be found. Some of them can be mentioned as generating color orthophotos, DEMs, 3D models and thermal images for observation of archaeological areas [4,11], documentation of archaeological excavations [12,13] and several ancient sites [14], burial mounds [15], modeling and recording of archaeological heritage [16,17], archaeological remains documentation [18] and 3D mapping [3].

In this study, a practical application of a UAV system was used to check the accuracy of an orthophoto for large-scale mapping of an archaeological area in Beyşehir town of Konya city.

Orthophoto Accuracy for Archaeology

Technological progress has pushed a high demand for geographic data. Both users and governments have started to require good-quality data. When it is considered to be useful for geometric information, accuracy should be required for all its components (i.e., spatial, temporal, topological and thematic). What makes UAV use appealing for archaeology and other fields is its high spatial and temporal resolution [19].

Orthophotos are images that have all distortions due to camera obliquity and terrain relief removed, and they combine the image characteristics of a photograph with the geometric qualities of a map [20]. The science of archaeology has preferred to use orthophotos since they are a detailed pictorial view of the earth’s surface and topographic information. Archaeologists prefer to use an orthophoto since:

-It is flexible and allows map information that can be shared easily;

-It contains everything from the original photograph;

-It can be produced in a short time;

-It provides up-to-date information;

-It has the geometric accuracy of a map;

-It has all the visual characteristics of an image [21].

As an orthophoto shows an image of objects in their true orthographic positions, they are geometrically equivalent to conventional line and symbol planimetric maps. The major difference between an orthophoto and a map is that an orthophoto is composed of images of features, whereas maps utilize lines and symbols plotted to scale to depict features [22,23].

In a cartographic product, it is important to assess the spatial, spectral, radiometric and temporal resolution of the product. Because of this, it is essential to know the objectives of a project in order to decide the platforms, tools and sensors to use. This is required also to assess the quality of all its components, including spatial resolution [19].

One of the most important quality features of cartographic products is positional accuracy. Positional quality is essential for generation of cartographic production. All NMAs (National Map Accuracy Standards) use statistical methods to control it. Spatial quality can be assessed using several tests or standards such as the NMAs, STANAG or NSSDA [24,25,26]. STANAG defines different accuracy ratings for absolute geometric accuracy. Each rate is identified by a letter from A to E, and A is considered the best accuracy rate. For each rate, it defines the maximum differences between any two well-defined points in map units [26]. NMAS sets a compliance accuracy value while the NSSDA (National Standard for Spatial Data Accuracy) uses root mean error (RMSE) to estimate positional accuracy [27]. RMSE is the square root of the average of a set of squared differences between data set coordinate values and coordinate values from an independent source of higher accuracy [27]. The spatial accuracy of a UAV orthophoto is mostly reported using the root mean square error [RMSE] [28].

The evaluation of accuracy and quality control are significant for digital orthophoto production. Not all orthophotos are generated at the same level, and thus their accuracy and quality change depending on the data used in the process and generation procedure.

An orthophoto accuracy is also influenced by many factors such as:

-Used methodology and workflow;

-Characteristics of used equipment;

-Quality of camera and calibration parameters;

-Flight height;

-Quality and size of a charged coupled device (CCD) used in the digital camera CCD array;

-Amount of imagery overlap;

-Quality of parallax determination or photo measurements;

-Quality of the GNSS signal;

-Quality, density and placement of ground control points;

-Quality of the aerial triangulation solution;

-Capability of the processing software to handle GNSS drift and shift;

-Capability of the processing software to handle camera self-calibration;

-Digital terrain model used for the production of orthoimagery;

-Resolution of the digital images;

-Fly height and number and placement of ground control points [21,27,29,30,31,32,33].

These factors can vary for each project. For this reason, existing accuracy measures based on map scale, ground sample distance (GSD) and scanning resolution no longer apply to current geospatial mapping practices.

Positional accuracy is the accuracy of the position of features, including horizontal and vertical positions, with respect to horizontal and vertical data. Positional accuracy is a product of the entire drone system (aircraft, sensors, operation and processing software), not any single component [31].

Positional accuracy of digital orthoimagery, which is independent of GSD, map scale or contour interval, depends on:

-Digital elevation data accuracy;

-Aerial triangulation accuracy;

-Ground control points’ accuracy;

-Required number and spatial distribution of check points based on project area;

-The camera’s inherent potential accuracy;

-The stability of the flight;

-The quality of the global navigation satellite system (GNSS) data;

-The quality of the inertial system (if the drone uses INS/IMU);

-The quality of the DEM used to make the orthophoto;

-The type and quality of processing of the raw imagery into an orthophoto;

-The number and quality of ground control points [31].

Each factor contributes some error to the ultimate positional accuracy of the final product. The sum of all errors determines the measurable positional accuracy [33].

The positional accuracy is usually determined for both horizontal and vertical coordinates. Horizontal positional accuracy is an estimate of accuracy of the horizontal positions of the spatial objects that can be measured in terms of latitude and longitude or local easting and northing coordinates [34]. Meanwhile, vertical positional accuracy refers to the uncertainty with which the vertical coordinates (ellipsoidal height/geocentric radius) of spatial objects can be measured. In general, the positional accuracy of orthophoto refers to the accuracy at which the position coordinates (latitude, longitude and ellipsoidal height) of the spatial objects that are well recognized in the orthophoto are estimated in reference to their corresponding ground-truth coordinates acquired at the same locations using an independent ground-based in-situ measurement such as GNSS observations [35].

There have been several studies done in the past in order to evaluate the accuracy of an orthophoto. Uysal et al. [17] obtained 0.062 m overall vertical accuracy from an altitude of 60 m [21]. Another study about quality assessment of UAV orthophotos was done by Mesas-Carrascosa et al. [19] by obtaining images with a vertically mounted digital camera. The orthophotos were assessed with various spatial quality tests used by national mapping agencies (NMAs) [19]. Udin and Ahmad (2014) conducted an assessment of photogrammetric mapping accuracy based on variation flying altitude using UAVs. Their objective was to determine the accuracy of the photogrammetric output, and they evaluated planimetry and vertical accuracy using RMSE depending on fly height [36]. Julge and Ellmann (2015) conducted an evaluation of the accuracy of orthophotos and 3D models from UAV photogrammetry. They evaluated the quality and accuracy of the products by comparing control measurements made with GNSS measurements on the ground, as well as high-resolution airborne laser scanning data and other available orthophotos. The results of vertical and planar comparison statistics were used to evaluate the accuracy of UAV data [30]. Another study by Sai et al. (2019) carried out a geometric accuracy assessment of orthophoto production from UAV aerial images. In their study, they made an investigation on geometric accuracy of the produced orthophotomosaic according to the ASPR standards [22]. Since acquisition of accurate spatial data from traditional data compilation techniques is expensive, Greenfeld (2001) used digital orthophoto quadrangles (DOQs) for establishing digital parcel coverages and to evaluate their accuracy [27]. GPS data are also used for checking the positional accuracy in large-scale mapping. Sisay et al. [37] used in-situ GPS data to approve the accuracy of horizontal coordinates and orientation of linear features of an orthophoto and line map of Bahir Dar city [37].

In this study, positional accuracy was checked according to ASPRS Standards, and the results were evaluated.

2. Kubad Abad Palace as Case Study

Kubad Abad Palace is a complex located on latitude 37°40′25.19″ N and longitude 31°24′15.59″ E. The palace was recognized by the famous Seljuk historian Ibn Bibi. It was constructed upon the order of Alaâddin Keykubad I (1120–1236), and it was established as the summer residence of Sultan in the Anatolian Seljuk period and is the only Seljuk palace surviving today. The palace lies in the south-west of Lake Beyşehir, in south-west Central Anatolia in Turkey, just over 100 km west of the Seljuk capital in Konya and located 3 km north of the town of Gölkaya (Figure 1 and Figure 2). The location of the palace was found by the Konya Museum Director Zeki Oral in 1949, after references about the location. The first excavations started in 1965–1966 by Katharina Ottodorn and in 1967 by Mehmet Önder. From 1980, the excavations were continued by a team of archaeologists under the direction of Prof. Rüçhan Arık.

The palace consists of various structures. It mainly consists of the Great Palace on the north side of the site and the Small Palace on the south of it, a boat house on the shore of the lake and small pavilions surrounding it, the walls of an inner courtyard and other ruined structure the functions of which have not been known yet (Figure 3). The Great Palace in size of 50 m × 35 m is the largest building of the complex and consists of three main sections: a palace building, a courtyard and a terrace extending to the lake (Figure 4). The Small Palace also has a similar plan with thicker and more regular walls (Figure 5). Both buildings have a plan of large and small rooms on either side of an axis determined by an iwan once used as the throne room and the forecourt. The structures were constructed with rubble stone [38,39] (Figure 6).

Besides the structure, the palace was very famous for its tiles decorating the walls of the structure. Most of these tiles are in the shape of a star or a square, mostly with the traditional color turquoise and in the dimension of 20 × 24 cm. They are decorated with figurative motifs in luster or underglaze techniques. The influence of Central Asian traditions is felt in decoration compositions. Typical tiles are white, eight-pointed stars and have delicate figural paintings of humans and animals, including some of the best in Anatolian Seljuk representational art. Painted with underglaze in turquoise, green, purple and blue, these figural tiles are set on a background of cruciform-shaped turquoise tiles with arabesque motifs in black [38] (Figure 7).

3. Materials and Methods

3.1. Methodology

Typical image-based aerial surveying with UAVs requires mainly to decide the project parameters, field work including mission planning (project parameters, flight planning), image acquisition, data processing and product generation (Figure 8).

Project parameters include all parameters such as flight parameters, camera calibration and ground control point (GCP) measurements, devices and digital photogrammetry parameters. Nex and Remondino (2014) prepared a general workflow and data acquisition and processing pipeline including all related parameters. After preparing all these parameters, image acquisition and then image triangulation follow the pipeline with additional parameters such as camera calibration and GCP. At the end of image triangulation, it is possible to get digital terrain models (DTMs) or digital surface models (DSMs) and digital elevation model (DEM). These outputs can be used for orthoimage generation or for different 3D modeling purposes [41].

3.1.1. Mission Planning

Flight mission is generally planned with related software, the required ground sample distance (GSD) (footprint) and parameters of the digital camera. Parameters such as image scale and flight heights are dependent on aim of the flight. Missions for detailed 3D model generation require high overlaps and low-altitude flights to achieve small GSDs; however, for emergency surveying or management purpose, flights require covering large areas in a few minutes, relatively at lower resolution [41]. In this project, ground control station software was used for flight planning and ground sample distance was 4.15 cm from 159 m height. The aim of this project was to see the ability of UAVs for archaeological projects, especially for 2D drawing and in desired scale which was 1:200 and 1:500. In order to cover the large area, a more durable and fixed-wing UAV was chosen.

Depending on the mission, UAV system type and environmental conditions, the flight can be done manually, with assistance or autonomously. We used autonomous option for this flight. Regarding the quality of the data, low-cost systems can be sufficient for little extensions while more expensive devices with long endurance are required for larger areas.

Appropriate aircraft image coverage is generally designed to provide about 60% forward overlap. This allows stereoscopic viewing of two overlapping images. Lateral overlap between 20% and 40% is required for complete area coverage. The usual overlap value cannot be directly used on the UAV system. In order to cover as much as possible of the area with the minimum number of images taken by the UAV system, it is necessary to investigate appropriate overlap values [42]. In this project, sidelap was 60% and overlap was 80% depending on the flight direction and archaeological area size (

Table 1. Project parameters.

Project Parameters
Flight Height	159 m
Ground resolution	3.77 cm/px
Coverage area	1.22 km2
Sidelap	60%
Overlap	80%
Number of strips	3
Number of images	903
Number of GCPs	14
Aligned cameras	859
Coordinate system	TUREF/TM 33 [EPSG:5255]
Dense point cloud points	247,211,018
Tie points	68,572
Projections	611,032
Reprojection error	0.772 px
RMS reprojection error	0.324 px
Faces	49,413,897
Vertices	24,715,739
DEM resolution	7.55 cm/px
Orthomosaic size	26,390 × 35,636

).

3.1.2. Image Acquisition

For image acquisition, in order to get aerial data, SmartPlanes Freya fixed-wing UAV was used with a 1.2 m span, covered EPP wing and molded polycarbonate fuselage (Figure 9). It is 1.1–1.5 kg. Its maximum flight time is between 50 min. and 1 h 40 min. It has different flight modes such as auto, assisted and manual and wind rating 13 m/s with payload capacity between 200 and 600 gr. For image acquisition, Ricoh GR 16.1 MP resolution compact camera with 18.3 mm focal length was used. Image resolution was 4928 × 3264 and pixel size 4.78 × 4.78 µm.

Flight with UAVs can normally be done in manual, assisted (semi-autonomous) or autonomous mode. Like the other parameters, this mode varies according to the goal of the flight, platform type and environmental conditions. GNSS/INS navigation devices lead to use of autonomous flight (take-off landing and navigation) and image acquisition. Image network quality is mainly based on the flight mode. Autonomous mode of flight and image acquisition provides much more regular overlap and the geometry of the acquisition. In this case study, images were acquired with autonomous flight mode in order to get more regular overlap and the geometry of block.

GCPs have a significant role in getting better photogrammetric results. Studies show that the more GCPs there are, the more accurate photogrammetric results. In the field, a total of 14 GCPs were measured with real-time kinematic global positioning method (RTK-GPS). The flight was carried out in autonomus mode with a trained pilot. Since it was asked that archaeological area was surveyed with surroundings, the flight height was calculated according to the area size. The flight height was 159 m. The flight trajectory and the image acquisition points were calculated depending on the scale, camera parameters, flight height, dimension of the area and the overlap. For the flight trajectory, ground control station software was used. At the end of field work, 903 images were taken in mainly 30 strips during 2 flights. During the first flight, 435 photos were taken, while during the second flight, 468 photos were taken. Image acquisition overlap was 80% and side lap was 60%.

Because of the area size, shape and topographical features of the area, only 14 GCPs could be applied. The archaeological area is at risk, and thus to enter the area requires special permission. GCPs were applied according to the topographical features of the area. Since RMSE values for location and altitude were substandard according to both national and international standards, no CPs was needed.

3.1.3. Data Processing (Aerial Triangulation)

Image orientation and camera calibration are two principal prerequisites for any metric image reconstruction. Camera calibration relates to the problem of pixel locations in the image sequence to points in the scene. The position of the camera and camera constant must be determined to relate image plane coordinates to absolute coordinates. In addition, the location of principal point, the aspect ratio and lens distortions must be defined to relate pixel coordinates to positions in the image [43]. Photogrammetric cameras (metric) are used where it is needed to get high-accuracy measurements, such as in topographical applications and architectural photogrammetry. Meanwhile, digital cameras are used when lower accuracy measurements are enough. In this project, for image acquisition, Ricoh GR 16.1 MP resolution compact camera with 18.3 mm focal length was used, and the calibration parameters are shown in

Table 2. Richor GR 18.3 mm camera calibration parameters.

Richor GR 18.3 mm Camera Calibration Parameters
Resolution	4928 × 3264	F	3872.84
Type	Frame	B1	0.441328
Cx	−6.25894	B2	2.37701
Cy	11.7735	P1	−0.000308219
K1	−0.0723221	P2	2.6913 × 10−5
K2	0.091206	P3	0
K3	−0.0196215	P4	0
K4	0

.

In this study, for aerial image processing, PhotoScan 1.1.5 was used. Image alignment was carried out with 859 cameras (Figure 10, Figure 11, Figure 12 and Figure 13) and 14 GCPs (Figure 14). The coordinate system of the project was selected TUREF/TM 33 (EPSG:5255). The images were aligned with 68,572 tie points and 14 GCPs (Table 2).

After dense point cloud model, mesh model was processed, and at the end, 49,813,987 faces and 24,715,739 vertices mesh was acquired (Figure 7 and Figure 8).

1.22 km² area was covered with 3.77 cm/px ground resolution and with 0.772 px reprojection error with Richor GR camera. Because of the nice weather and high stability of UAV, camera locations were quite stable on archaeological area. There was just some deformation of camera locations on the north-west part of the area for a couple of centers.

14 GCPs’ RMSE values were 4.06257 cm in X, 3.32436 cm in Y, 3.28841 cm in Z and in total 6.19432 cm and 0.324 px (

Table 3. GCP’s RMSE values in total.

Count	X Error (cm)	Y Error (cm)	Z Error (cm)	XY Error (cm)	Total (cm)	Image (pix)
14	4.06257	3.32436	3.28841	5.24937	6.19432	0.324

and

Table 4. GCP’s RMSE values.

Label	X Error (cm)	Y Error (cm)	Z Error (cm)	Total (cm)	Image (pix)
100	6.11981	5.95244	−1.88219	8.74221	0.269(23)
101	4.45511	−2.77318	3.11042	6.10027	0.310(24)
103	−4.27609	−1.04231	−2.03174	4.84761	0.266(18)
104	−3.38464	−1.1302	1.253	3.78194	0.183(21)
105	3.75177	−5.17337	−0.0764271	6.39103	0.121(22)
108	−0.994673	2.90823	6.45443	7.1489	0.346(25)
109	−8.0377	3.28298	−6.07461	10.5964	0.556(24)
110	2.599	−2.00871	−1.591775	3.65012	0.255(27)
111	2.75833	1.93097	2.94591	4.47386	0.183(16)
112	−3.95106	2.24095	−5.24227	6.93643	0.567(14)
113	2.49822	−1.3361	2.72138	3.92839	0.495(15)
114	−1.66697	4.1478	−1.49671	4.71415	0.051(4)
116	−3.39311	−0.623031	−1.28622	3.68181	0.177(17)
117	3.72089	−5.57416	1.97885	6.988	0.118(16)
Total	4.06257	3.32436	3.28841	6.19432	0.324

).

The final orthophoto size was 26,390 × 35,636 and 1.72 GB file size (Figure 15 and Figure 16). In this study, an aircraft model UAV was used in order to get large-scale mapping and 2D drawings of archaeological area. For this project, the data were quite dense, and it was difficult to manage the data. For this kind of big area, hardware should be powerful enough. Since the management of the data was difficult, data processing took time, and even taking the screenshots from the computer took time and caused crash in the computer.

With a typical photogrammetric pipeline, 3D results such as DSM (digital surface model) or DTM (digital terrain model), contour lines, textured 3D models, vector data, etc., can be produced in a reasonable automated way. Digital elevation model (DEM), one of the products of UAV photogrammetry, can be defined as a digital model or 3D representation of a terrain surface created from elevation data. A DEM can be a raster file consisting of each record with 3D element usually associated with geographic coordinates. Digital terrain model (DTM) is a DEM shape of the ground, while digital surface model (DSM) is a DEM of the shape of the surface (including vegetation, infra-structures, etc.). In photogrammetry, DEMs are a by-product resulting from the interpolation of extremely dense 3D point clouds extracted from imagery. The quality of a DEM depends on horizontal and vertical accuracy. In order to check DEM accuracy, RMSE and standard deviation (SV) calculations can be used [44]. In this project, DEM was in 7.55 cm/px resolution and 176 points/m² (Figure 17).

The resolution of the final orthophoto is high, and even single structures such as stones can be seen in orthophoto and 3D model. The main difficulties were in vertical walls; however, since their height was not much from the ground, it was possible to draw in AutoCAD for 2D purpose (Figure 18). For these kinds of parts, it would be another way to make laser scanning and to combine both TLS and aerial data sets. For future work, it is aimed to combine both data and to prepare a reconstructed model of the area. According to the experience, UAV data can be used for large-scale mapping (more than 1:500) and 2D drawing.

4. Results

4.1. Specific Requirements

To check the data accuracy, the ASPRS (American Society for Photogrammetry and Remote Sensing) published draft report in 2014 was taken as a reference. The ASPRS is a scientific association founded in 1934, and their mission is to advance knowledge and improve understanding of mapping sciences to promote responsible applications of photogrammetry, remote sensing, geographic information systems (GISs) and supporting technologies (www.asprs.org). They published ASPRS Accuracy Standards for Digital Geospatial Data on 21 March 2014 with the objective to replace the existing ASPRS Accuracy Standards for Large-Scale Maps (1990) and the ASPRS Guidelines Vertical Accuracy Reporting for Lidar Data (2004) to better address current technologies.

The standard defines specific accuracy classes and associated RMSE thresholds for digital orthoimagery, digital planimetric data and digital elevation data (Table 3). Accuracy classes have been revised and upgraded from the 1990 standard to address the higher accuracies achievable with newer technologies. The standard also includes additional accuracy measures, such as ortho seamlines, aerial triangulation accuracy, horizontal accuracy of elevation data and the required number and spatial distribution of check points. The standard is intended to be technology-independent and the base standard upon which future work can build.

When required, horizontal accuracy can be tested by comparing the planimetric coordinates of well-defined points in the data set with coordinates of the same points from an independent source of higher accuracy. Vertical accuracy can be tested by comparing the elevations in the data set with elevations of the same points as determined from an independent source of higher accuracy.

When errors are normally distributed, accuracy can be performed with RMSE values, standard deviations, mean errors, maximum and minimum errors and unitless skew kurtosis values. When errors are not normally distributed, alternative methods can be used. If the number of test points (check points) is sufficient, testing and reporting can be performed using 95th percentile errors.

Using metric units, the ASPRS recommends 100 static vertical check points for the first 2500 square kilometer area within the project. This provides a statistically defensible number of samples on which to base a valid vertical accuracy assessment. For horizontal testing of areas >2500 km², clients should determine the number of additional horizontal check points, if any, based on criteria such as resolution of imagery and extent of urbanization.

For vertical testing of areas >2500 km², add five additional vertical check points for each additional 500 km² area. Each additional set of five vertical check points for 500 km² would include three check points for NVA (non-vegetated vertical accuracy) and two for VVA (vegetated vertical accuracy) (

Table 5. Recommended number of check points based on area [37].

Project Area (Square Kilometers)	Horizontal Accuracy Testing of Orthoimagery and Planimetrics	Vertical and Horizontal Accuracy Testing of Elevation Data Sets
	Total Number of Static 2D/3D Check Points (Clearly Defined Points)	Number of Static 3D Check Points in NVA	Number of Static 3D Check Points in VVA	Total Number of Static 3D Check Points
≤500	20	20	5	25
501–750	25	20	10	30
751–1000	30	25	15	40
1001–1250	35	30	20	50
1251–1500	40	35	25	60
1501–1750	45	40	30	70
1751–2000	50	45	35	80
2001–2250	55	50	40	90
2251–2500	60	55	45	100

) [35]. The recommended number and distribution of NVA and VVA check points may vary depending on the importance of different land-cover categories and client requirements.

In the Kubad Abad project, the project area was 1.22 km², a total of 14 GCPs were used, and the area was vegetated.

4.2. Horizontal and Vertical Accuracy Assessment

4.2.1. Horizontal Accuracy Assessment

Horizontal accuracy is the horizontal component of the positional accuracy of a data set with respect to a horizontal datum, defined at the 95% confidence level. Horizontal accuracy is to be assessed using root mean square error (RMSE) statistics, while vertical accuracy is to be assessed using RMSE statistics in non-vegetated terrain and 95th percentile statistics in vegetated terrain (

Table 6. Horizontal accuracy values depending on the x values [37].

Horizontal Accuracy Class	RMSEx and RMSEy (cm)	RMSEr (cm)	Horizontal Accuracy at 95% Confidence Level (cm)	Orthoimagery Mosaic Seamline Mismatch (cm)
X-cm	≤X	≤1.41 x X	≤2.45 x X	≤2 xX

).

Horizontal accuracy,

RMSEx = sqrt[(x_data,i − x_check,i)²/n]

(1)

RMSEy = sqrt[(y_data,i − y_check,i)²/n]

(2)

RMSEx, the horizontal linear RMSE in the X direction.

RMSEy, the horizontal linear RMSE in the Y direction .

x_data,i and y_data,i are the coordinates of the ith check point in the data set.

x_check,i and y_check,i are the coordinates of the ith check point in the independent source of higher accuracy (in the model), n is the number of check points tested, and i is an integer ranging from 1 to n. In this project,

RMSE_x = 4.06 cm

RMSE_y = 3.32 cm

RMSE_z = 3. 29 cm

For testing horizontal accuracy,

RMSE _xy = [RMSE_x + RMSE_y]/2

RMSE _xy = 3.69 cm

RMSE_r = √ RMSE_x ² + RMSE_y ²

RMSE_r = 5.24 cm

According to the Standards, horizontal accuracy at a 95% confidence level should be ≤2.45 x X which for this project is

Accuracy _r = 1.7308 X 5.24

Accuracy _r = 9.06 cm

According to

Table 7. Horizontal accuracy/quality example standards for digital orthoimagery [37].

Horizontal Accuracy Class RMSEx and RMSEy (cm)	RMSEr (cm)	Orthoimage Mosaic Seamline Maximum Mismatch (cm)	Horizontal Accuracy at the 95% Confidence Level (cm)
0.63	0.9	1.3	1.5
1.25	1.8	2.5	3.1
2.50	3.5	5.0	6.1
5.00	7.1	10.0	12.2
7.50	10.6	15.0	18.4
10.00	14.1	20.0	24.5
12.50	17.7	25.0	30.6
15.00	21.2	30.0	36.7
17.50	24.7	35.0	42.8
20.00	28.3	40.0	49.0
22.50	31.8	45.0	55.1
25.00	35.4	50.0	61.2
27.50	38.9	55.0	67.3
30.00	42.4	60.0	73.4
45.00	63.6	90.0	110.1
60.00	84.9	120.0	146.9
75.00	106.1	150.0	183.6
100.00	141.4	200.0	244.8
150.00	212.1	300.0	367.2
200.00	282.8	400.0	489.5
250.00	353.6	500.0	611.9
300.00	424.3	600.0	734.3
500.00	707.1	1000.0	1223.9
1000.00	1414.2	2000.0	2447.7

horizontal accuracy examples for digital orthophotos, the values of the Kubad Abad project are between the required values of ASPRS Standards with the values

RMSE _xy = 3.69 cm

RMSE_r = 5.24 cm

Accuracy _r = 9.06 cm

Ground sample distance (GSD) is the linear dimension of a sample pixel’s footprint on the ground. In ASPRS Standards, GSD is assumed to be the value computed using the camera focal length and camera height above average mean terrain. In the Kubad Abad project, GSD can be calculated as

GSD = (H/f) µ

(3)

µ = W/Sw = H/Sh

(4)

where H is flight height (m), f is focal length (mm), GSD is ground sample distance (m), µ is pixel size (μm), W is the width of CCD (mm), H is the height of CDD(mm), Sw is the number of pixels for W, and Sh is the number of pixels for H.

In this project, GSD = 4.15 cm.

According to ASPRS Standards, depending on the horizontal accuracy values, the GSD value should be between 1.25 and 5.0 cm when RMSExy value is between 2.5 and 5 cm (

Table 8. Horizontal accuracy/quality examples for high-accuracy digital planimetric data.

ASPRS 2014				Equivalent to Map Scale in
Horizontal Accuracy Class RMSEx and RMSEy (cm)	RMSEr (cm)	Horizontal Accuracy at the 95% Confidence Level (cm)	Approximate GSD of Source Imagery (cm)	ASPRS1990 Class 1	ASPRS1990 Class 2
0.63	0.9	1.5	0.31 to 0.63	1:25	1:12.5
1.25	1.8	3.1	0.63 to 1.25	1:50	1:25
2.5	3.5	6.1	1.25 to 2.5	1:100	1:50
5.0	7.1	12.2	2.5 to 5.0	1:200	1:100
7.5	10.6	18.4	3.8 to 7.5	1:300	1:150
10.0	14.1	24.5	5.0 to 10.0	1:400	1:200
12.5	17.7	30.6	6.3 to12.5	1:500	1:250
15.0	21.2	36.7	7.5 to 15.0	1:600	1:300
17.5	24.7	42.8	8.8 to 17.5	1:700	1:350
20.0	28.3	49.0	10.0 to 20.0	1:800	1:400
22.5	31.8	55.1	11.3 to 22.5	1:900	1:450
25.0	35.4	61.2	12.5 to 25.0	1:1000	1:500
27.5	38.9	67.3	13.8 to 27.5	1:1100	1:550
30.0	42.4	73.4	15.0 to 30.0	1:1200	1:600

). In our study, GSD value 4.15 cm is within this interval with 3.69 cm RMSE xy.

4.2.2. Vertical Accuracy Assessment

Vertical accuracy is the measure of positional accuracy of a data set with respect to a specified vertical datum. According to ASPRS accuracy standards (Table 9), elevation data have mainly ten accuracy levels relevant to technologies such as mobile mapping systems, unmanned aerial systems, airborne or satellite imagery, Lidar or IFSAR (interferometric synthetic aperture radar).

RMSE z = sqrt [(z_data,i − z_check,i)2/n]

(5)

where

z_data,i is the vertical coordinate of the ith check point in the data set.

z_check,i is the vertical coordinate of the ith check point in the independent source of higher accuracy.

n = the number of points being checked.

i is an integer from 1 to n.

It is assumed that systematic errors were eliminated as best as possible. If the vertical error is normally distributed, the factor 1.9600 is applied to compute linear error at the 95% confidence level (Greenwalt and Schultz, 1968). Therefore, vertical accuracy, Accuracy z, reported according to the NSSDA, should be computed by the following formula:

Accuracy z =1.96 X RMSEz.

Table 9. Accuracy standards for digital elevation data.

Vertical Accuracy Class	Absolute Accuracy			Relative Accuracy (Where Applicable)
	RMSEz Non-Vegetated (cm)	NVA at 95% Confidence Level (cm)	VVA at 95th Percentile (cm)	Within-Swath Hard Surface Repeatability (Max Diff) (cm)	Swath-to-Swath Non-Vegetated Terrain (RMSDz) (cm)	Swath-to-Swath Non-Vegetated Terrain (Max Diff) (cm)
X-cm	≤X	≤1.96 x X	≤3.00 x X	≤0.60 x X	≤0.80 x X	≤1.60 x X

Table 9. Accuracy standards for digital elevation data.

Vertical Accuracy Class	Absolute Accuracy			Relative Accuracy (Where Applicable)
	RMSEz Non-Vegetated (cm)	NVA at 95% Confidence Level (cm)	VVA at 95th Percentile (cm)	Within-Swath Hard Surface Repeatability (Max Diff) (cm)	Swath-to-Swath Non-Vegetated Terrain (RMSDz) (cm)	Swath-to-Swath Non-Vegetated Terrain (Max Diff) (cm)
X-cm	≤X	≤1.96 x X	≤3.00 x X	≤0.60 x X	≤0.80 x X	≤1.60 x X

Absolute accuracy is a measure that accounts for all systematic and random errors in a data set (

Table 10. Vertical accuracy/quality examples for digital elevation data.

Vertical Accuracy Class	Absolute Accuracy			Relative Accuracy (Where Applicable)
	RMSEz Non-Vegetated (cm)	NVA at 95% Confidence Level (cm)	VVA at 95th Percentile (cm)	Within-Swath Hard Surface Repeatability (Max Diff) (cm)	Swath-to-Swath Non-Veg Terrain (RMSDz) (cm)	Swath-to-Swath Non-Veg Terrain (Max Diff) (cm)
1 m	1.0	2.0	3	0.6	0.8	1.6
2.5-cm	2.5	4.9	7.5	1.5	2	4
5-cm	5.0	9.8	15	3	4	8
10-cm	10.0	19.6	30	6	8	16
15-cm	15.0	29.4	45	9	12	24
20-cm	20.0	39.2	60	12	16	32
33.3-cm	33.3	65.3	100	20	26.7	53.3
66.7-cm	66.7	130.7	200	40	53.3	106.7
100-cm	100.0	196.0	300	60	80	160
333.3-cm	333.3.	653.3	1000	200	266.7	533.3

).

In the Kubad Abad porject, RMSE_z = 3.29 cm

Accuracy_z = 1.96 X 3.29 = 6.49 cm (NVA at 95% confidence level)

3X3.29 = 9.87 cm (VVA at 95th percentile)

According to ASPRS Standards, horizontal and vertical accuracy should be reported as in the following.

This data set was tested to meet ASPRS Positional Accuracy Standards for Digital Geospatial Data (2014) for a 5 (cm) RMSE_x/RMSE_y horizontal accuracy class. Actual positional accuracy was found to be RMSE_x = 4.06 cm and RMSE_y = 3.32 cm which equates to ±9.04 cm at a 95% confidence level.

This data set was tested to meet ASPRS Positional Accuracy Standards for Digital Geospatial Data (2014) for a 5 (cm) RMSE_z vertical accuracy class. Actual NVA accuracy was found to be RMSE_z = 3. 29 cm, equating to ±6.49 cm at a 95% confidence level. Actual VVA accuracy was found to be ±9.87 cm at the 95th percentile.

4.3. A General Algorithm of the Case Study

Depending on the methodology, a general algorithm of the study is prepared and showed as a table (Figure 19) [45].

5. Results

Orthophoto has been used for many years in large-scale mapping and archaeological projects. With rapid development in technology, it has started to use digital orthophotos as a map since they are all corrected images. The ability of direct measurement in orthophotos has made them indispensable for complex heritage documentation such as large archaeological areas. For this reason, the factors affecting the accuracy of orthophoto should be considered in the production process.

The aim of this study was to check the positional accuracy of an orthophoto created by the images from a UAV and assess the positional accuracy of the orthophoto aiming for large-scale archaeological documentation. In this study, high-resolution aerial images of an archaeological area were collected with a fixed-wing UAV. Ground control points were established and measured using the RTK-GPS method. A high-resolution orthophoto was created. The horizontal and vertical positional accuracy of the orthophoto was evaluated mainly in terms of RMSE and checked according to the ASPRS. Both the horizontal and vertical accuracy results are at a 95% confidence level. From the result obtained, UAVs could be used for large-scale mapping and documentation of large archaeological areas. In particular, even when the area is very large, the acquired accuracy level is extremely good and suitable for archaeological 2D drawings.