Digital Airborne Photogrammetry—A New Tool for Quantitative Remote Sensing?—A State-of-the-Art Review On Radiometric Aspects of Digital Photogrammetric Images

Abstract

The transition from film imaging to digital imaging in photogrammetric data capture is opening interesting possibilities for photogrammetric processes. A great advantage of digital sensors is their radiometric potential. This article presents a state-of-the-art review on the radiometric aspects of digital photogrammetric images. The analysis is based on a literature research and a questionnaire submitted to various interest groups related to the photogrammetric process. An important contribution to this paper is a characterization of the photogrammetric image acquisition and image product generation systems. The questionnaire revealed many weaknesses in current processes, but the future prospects of radiometrically quantitative photogrammetry are promising.

1. Introduction

Great progress is occurring in all fields of geospatial imaging, i.e., passive and active imaging from spaceborne, stratospheric, airborne, UAV, terrestrial and ubiquitous platforms. Various imaging techniques have their pros and cons, and it is anticipated that future mapping and monitoring processes will fuse various methodologies. This investigation concerns high-resolution, airborne photogrammetric imaging. In this area, the recent revolutionary technical advancement was the transition from film imaging to digital imaging [1].

An important novel feature of the digital systems, in comparison to analog systems, is their high radiometric potential, which was empirically proven by Honkavaara [1] and Markelin et al. [2]. In these baseline investigations, radiometric properties of all commercial first generation photogrammetric large-format sensor types were studied using imagery collected in 2004 and 2005. These studies revealed that the serious problems hindering the quantitative use of the image radiometry included the insufficiently described sensors and processing lines, insufficient calibration, and insufficient processing chains. Also, some sensor-related problems were detected. The conclusion was that developments are needed in all fields of radiometric processing.

The requirements for accurate radiometry are a thorough understanding of the measurement problem, a complete description and understanding of the instruments, and mechanisms for comparing and assessing results [3]. Accurate radiometry is a new issue in photogrammetric processes. Well-established radiometric processing approaches exist for remote sensing systems (e.g. satellite and airborne hyper-spectral imaging systems) [4,5,6], but they are not directly applicable in photogrammetric processing lines due to the special features of photogrammetric data acquisition [7]. The fundamental requirements of mainstream photogrammetric applications are great geometric accuracy, high spatial resolution, stereoscopy, and high efficiency and reliability. Hundreds of constantly improving photogrammetric sensors are in operation. A large number of data providers collect imagery from different platforms, using different systems and principles. In a typical photogrammetric project, even thousands of images may be collected during several acquisition days. In many processes, a huge amount of data is collected yearly, e.g. over entire countries every few years as a repeat cycle. High-quality sensors with large image format and optimized processes ensure accurate and efficient data production. Because of high productivity requirements, the image collection is not always carried out in optimum atmospheric or illumination conditions. Photogrammetric sensors have large field of view, which highlights object reflectance anisotropy. Images are typically arranged in image blocks with 20-80% side and forward overlaps, providing multiple views to objects.

It is expected that the rigorous treatment of image radiometry could significantly improve the automation potential of photogrammetric applications, such as national topographic mapping, 3D environmental model generation, and orthophoto production, and open new application areas for the photogrammetric imagery, e.g. in the fields of environmental monitoring and natural resources assessment [8,9,10,11]. The existing photogrammetric production lines with efficient, repetitive image collection, rigorous geometric processing, great geometric accuracy and reliability, high spatial resolution, and stereoscopy with large observation angles is an appealing and practical environment for the accurate radiometric processing and utilization.

The European Spatial Data Research organization (EuroSDR) launched a project on radiometric aspects of digital photogrammetric airborne images in May 2008 [12,13]. Objectives of this investigation are to: (1) improve knowledge on radiometric aspects of digital photogrammetric cameras, (2) review existing methods and procedures for radiometric image processing, (3) compare and share operative solutions through a comparison of these techniques on a same test data set, and (4) analyze the benefit of radiometric calibration and correction in different applications (quantitative remote sensing, classification, change detection etc.). The project is realized in two phases. In the first phase, a review on radiometric aspects of digital photogrammetric images is performed. In the second phase, a comparative, multi-site, empirical investigation is conducted.

The objective of this article is to present a state-of-the-art review on the radiometric aspects of digital photogrammetric images. In Section 2, the photogrammetric imaging process is discussed on a general level, based on existing literature; after drawing a framework for the entire process, three cornerstones of accurate radiometry, i.e., sensors, calibration and radiometric correction, are discussed in more detail. The literature does not extensively cover the entire photogrammetric process; these issues are emphasized in Section 3, based on results of a questionnaire submitted to various interest groups. A sample of five existing photogrammetric production lines of national mapping agencies (NMAs) is evaluated in the framework. The discussion in Section 4 completes this study. This article is the first state-of-the-art review on radiometric aspects of the entire digital photogrammetric processing line; a short summary of the results were presented by Honkavaara et al. [13]. As the result of this analysis, we expect that the knowledge on radiometric aspects of photogrammetric imagery will increase and widely spread among the remote sensing community, and this will lead to improvements in radiometric processing chains and applications.

Figure 1. A photogrammetric process (from [1]).

Figure 1. A photogrammetric process (from [1]).

2. Imaging Process

2.1. A digital Photogrammetric Airborne Imaging System

A photogrammetric process is a measurement process whose central sub-processes are image acquisition, referencing, and measurement and interpretation (Figure 1) [1]. The image acquisition process provides new image data. In the referencing process, the data is georeferenced and radiometrically corrected; example outputs are orthophotos, stereomodels and image blocks. The image products are utilized in the measurement and interpretation process. The photogrammetric process interacts with a geographical information system (GIS) by utilizing GIS tools and information and by storing the process outputs in it. Calibration can be considered as one sub-process in the photogrammetric production line. The sub-processes are presented as overlapping, because they are not necessarily isolated.

Two definitions can be given for the digital photogrammetric airborne imaging system: an image acquisition system or an image product generation system consisting of the image acquisition and referencing systems [1]. The central hardware components, in addition to the sensor, are the vehicle, sensor mount, and direct orientation system. The system calibration is considered as a component of the system. If the product generation is considered as part of the imaging process, then georeferencing, restoration, and radiometric correction also become parts of the system. The central factors influencing the output of an airborne system are summarized in Table 1 [1].

The basic requirements for the imagery are set by applications and they concern especially spectral, geometric, radiometric, spatial resolution, and temporal properties, and efficiency aspects, e.g. the image size. Simulation is an efficient method for determining system parameters for a certain application [14,15,16].

Table 1. Components of a digital photogrammetric airborne imaging system and central factors influencing photogrammetric system performance (adapted from [1]).

System	Sensor	Lens, detector, filter, beam splitter, shutter, temeperature/pressure stabilization
	Other system components	Sensor mount, camera port window, direct orientation system (GNSS, IMU), vehicle
	Calibration	Models, parameters, and methods for geometry, spatial resolution and radiometry
	Data post-processing	Image post-processing, direct orientation post-processing, georeferencing*, restoration*, radiometric correction*
Photogram­metric network	Block structure	Number of flight lines, number of images, side and forward overlaps, relative orientations
	Control	GCPs*, direct orientation observations, GNSS base stations, atmospheric observations*, in situ reflectance and illumination measurements, reflectance reference targets*, spatial resolution reference targets*
Conditions	System settings	Aperture, exposure time, FMC, in-flight data processing (e.g. compression)
	System environment	Altitude, vibrations and swing, velocity, temperature, pressure, humidity
	Atmosphere	Refraction, Mie and Rayleigh scattering (visibility), absorption, turbulence, clouds, temperature, pressure, humidity
	Illumination	Direct sunlight, diffuse light, solar elevation angle, spectral distribution of light
Object		Structure, contrast, anisotropy, topography, adjacent objects

* The factors only concern the image product generation system.

Table 1. Components of a digital photogrammetric airborne imaging system and central factors influencing photogrammetric system performance (adapted from [1]).

System	Sensor	Lens, detector, filter, beam splitter, shutter, temeperature/pressure stabilization
	Other system components	Sensor mount, camera port window, direct orientation system (GNSS, IMU), vehicle
	Calibration	Models, parameters, and methods for geometry, spatial resolution and radiometry
	Data post-processing	Image post-processing, direct orientation post-processing, georeferencing*, restoration*, radiometric correction*
Photogram­metric network	Block structure	Number of flight lines, number of images, side and forward overlaps, relative orientations
	Control	GCPs*, direct orientation observations, GNSS base stations, atmospheric observations*, in situ reflectance and illumination measurements, reflectance reference targets*, spatial resolution reference targets*
Conditions	System settings	Aperture, exposure time, FMC, in-flight data processing (e.g. compression)
	System environment	Altitude, vibrations and swing, velocity, temperature, pressure, humidity
	Atmosphere	Refraction, Mie and Rayleigh scattering (visibility), absorption, turbulence, clouds, temperature, pressure, humidity
	Illumination	Direct sunlight, diffuse light, solar elevation angle, spectral distribution of light
Object		Structure, contrast, anisotropy, topography, adjacent objects

* The factors only concern the image product generation system.

The following review begins with a short discussion on image radiometry. After that, sensors, radiometric sensor calibration and radiometric image correction are discussed briefly. The existing photogrammetric literature describes the sensors in many details, but the radiometric calibration and correction issues are not described thoroughly in most cases. The review is completed by the state-of-art survey in Section 3.

2.2. Image Radiometry in Image Collection Process

Radiometry means the measurement of radiance. A digital imaging sensor measures incoming radiance and stores the result of the measurement as digital number (DN). Two central phases in imaging process are the radiance transfer from object to a system and the transfer of the radiance entering the system (at-sensor radiance) to DNs inside the system. [1]

Figure 2. Radiation components (adopted from [19]).

Figure 2. Radiation components (adopted from [19]).

Principles of an imaging event of a passive imaging sensor is illustrated in Figure 2. Irradiance at object (O) is composed mainly of direct sunlight (A), skylight (B), multiple scattering (D), and light reflected from adjacent objects (F) [4,5,6,7,17,18,19]. The incident irradiance is reflected by scene objects according to their spectral, directional (typically anisotropic) reflectance characteristics. The elementary reflectance quantity is the bi-directional reflectance distribution function (BRDF), which models the dependence of the object reflectance on the directions of illumination and observation [20,21]. The radiance entering the imaging system composes mostly the radiance reflected from the object (G) and adjacent objects (E) and of skylight (C). In all phases of the radiative transfer in the atmosphere, the illumination and atmospheric properties and their changes influence the radiance; important object related disturbances are surface topography and shadowing by adjacent objects (Table 1).

The radiance enters the imaging system thorough the camera port, which is equipped or not with a glass window; conditions in the aircraft can thus be similar to the surrounding atmosphere or pressure and temperature stabilized. The radiation entering the sensor is controlled by the sensor aperture and exposure time. During the exposure, the sensor is subject to both forward and angular movements, which can be compensated for by using stabilizing sensor mounts and forward motion compensation (FMC). The incoming radiance enters through the sensor optics and spectral filters to the detector located at the focal plane, where the image is formed. The electronic signal in a certain band is amplified electronically by gain and offset values, and filtered by an electronic point spread function. Finally, the amplified and filtered signal is sampled and quantized to DNs, using an appropriate sampling distance (pixel size) and number of quantization levels (pixel depth). Central sensor and system properties, system settings and environmental factors influencing the radiometric output are given in Table 1 [1,4]. More detailed descriptions of the digital imaging can be found in the literature [4,22].

2.3. Airborne Imaging Sensors

Technical realizations, geometric, radiometric and spectral properties, and image formats of airborne imaging sensors vary greatly [23]. Characteristics for photogrammetric sensors (Section 2.3.1) have been a high geometric performance level, whereas for remote sensing sensors (e.g. multi and hyper-spectral imaging sensors), spectral and radiometric properties are highlighted. However, these sensor types are now approaching each other [1,8,9,10,11]. Digital large-format airborne photogrammetric sensors are emphasized in the following section.

2.3.1. Digital large-format photogrammetric sensors

Digital large-format photogrammetric sensors have replaced the institutional, analog 23 × 23 cm² format frame cameras [7]. These sensors entered the commercial markets during 2001–2003, and by the end of 2008, approximately 300 systems were in operational use world-wide. The general design principles of these sensors included a calibrated geometry with sub-pixel accuracy potential of up to 1 cm, a ground sample distance (GSD) potential of up to 2 cm, accurate stereoscopic data, an image width of more than 10,000 pixels, multi-spectral imagery on red (R), green (G), blue (B) and near infrared (NIR) regions of the electromagnetic spectra, and radiometry with linear response, large dynamic range, high resolution, and suitable for visual and quantitative applications [8,9].

The image width requirement had a fundamental role in directing the technical realizations. The production of sufficiently large area CCD arrays is still impossible, so large-format digital sensors are built either as multi-head systems by fusing several smaller area CCD arrays and cameras (frame sensors) or as pushbroom scanners by using linear CCD arrays. The leading commercially available large-format sensors are ADS (ADS40, ADS80) from Leica Geosystems [24,25], DMC from Intergraph [26,27], and UltraCam (UltraCamD, UltraCamX, UltraCamXp) from Microsoft [28,29]. New large-format sensors are entering the market, and some organizations are developing their own systems, e.g. the Institut Géographique National, France (IGN) [30,31]. Also, small- and medium-format area sensors are used in photogrammetric applications, and especially the medium-format sensors are approaching large-format sensors [23,32]; however, typical application areas of these small- and medium-format sensors are different from those of large-format sensors. Integrated systems give interesting possibilities as well, e.g. integration of vertical and oblique cameras to provide multiple views of objects or integration of cameras with range sensors. The commercial large-format sensors and the IGN’s sensor are focused in this study.

The ADS is a pushbroom scanner while the DMC, UltraCam and IGN’s sensor are multi-head frame sensors. Image widths (swath width) are 12,000 (ADS) to 17,310 pixels (UltraCamXp). The ADS has similar CCD-lines for panchromatic and multi-spectral channels; stereoscopic, multi-angular views are provided by a three-line principle; up to 12 CCD lines are available. In the cases of multi-head systems, the large-format, panchromatic image is composed of images from several individual cameras, and there is own camera for each multi-spectral channel. All channels of the ADS have the same GSD, but in the cases of the frame sensors, the GSD of multi-spectral channels is 3-4 times larger than that of the panchromatic channel. The requirements of quantitative remote sensing and classical mapping applications were taken into account in the construction of the ADS; its radiometric and spectral qualities are based on specially designed filters and beam splitter, the temperature and pressure stabilization, a telecentric lens, and accurate calibration. There are not so much information available about technical details of the frame sensors; they apply high-quality lenses and time delay integration (TDI) based forward motion compensation (FMC). The ADS provides wide-band panchromatic imagery and multi-spectral channels are relatively narrow, non-overlapping and optimized for both visual and remote sensing applications [14]. Spectral bands of the frame sensors are wider and more overlapping, and especially optimized to provide true colors for visual applications. In the case of ADS, the exposure control is based on the integration time; for DMC and UltraCam, various aperture and exposure settings are used; in the IGN’s cameras, the aperture is constant and exposure time is varying.

2.4. Radiometric Calibration

Radiometric calibration determines the radiometric response of an individual imaging system [1,4,5,6,33,34,35]. The major task is the determination of absolute and relative radiometric response models. In addition, spectral and colorimetric models and PSF are necessary information in radiometric processing. Calibration should also evaluate other factors that have influence on the system radiometric response, e.g. the shutter. Absolute radiometric calibration determines for each channel the models and parameters that are needed to transform the DNs into the units of radiance [W/(m² sr nm)]; typically, a linear model with gain and offset parameters is appropriate for CCD sensors [4]. Relative radiometric calibration normalizes the output of the sensor so that an uniform response is obtained in the entire image area when the focal plane of the sensor is irradiated with a uniform radiance field; for a single band, the corrections are determined at least for sensitivity differences of individual cells of a CCD array, defect pixels, light falloff, and dark signal [36,37]. Spectral response calibration determines the system’s response as a function of wavelength for each channel [33,35] and colorimetric calibration determines the relationship between the sensor and standard color spaces [38]. PSF-calibration determines the system’s response to a point source [1]. Various non-uniformities, such as spectral non-uniformity, temporal non-uniformity or PSF non-uniformities would be of interest to achieve high absolute calibration accuracies [39]. The exact parameterization is always system dependent. The following discussion emphasizes absolute and relative radiometric calibration.

The principle of radiometric calibration is to capture images of a flat, known radiance field at various intensity levels, using the system and by evaluating the system’s DN response to this radiance field to determine the radiometric calibration parameters [1]. In the rigorous calibration, the radiance field is traceable to international radiance standards [3,34,35]. Well-known radiometric calibration approaches are laboratory, on-board, test field (vicarious) and self-calibration (on-the-job). For each approach, different equipment and methods are used, and they provide different parameters and accuracy. Laboratory calibration determines the sensor calibration in an indoor facility using typically integrating spheres or hemispheres as light sources [34,35,40]. On-board calibration determines the sensor calibration in fight conditions using various on-board calibrators or natural light sources (the Sun, the Moon) [34,40]. Vicarious methods determine the system calibration in flight conditions utilizing targets present in the scene, typically either artificial targets or natural targets, such as playa, desert sand and salt flats or clouds; to determine accurately the radiance entering the system, vicarious methods require either accurate information on atmospheric conditions and object reflectance (reflectance-based method), or simultaneous determination of the at-sensor radiance by a calibrated radiometer (radiance-based method) [1,2,34,41,42]. Self-calibration is a concept commonly used with geometry [43,44] but can be generalized to concern radiometry as well; it means the determination or improvement of system calibration using the actual mapping data.

Figure 3. A photogrammetric test field with permanent and temporal radiometric and spatial resolution reference targets in Sjökulla, Finland [50]. Photo by the National Land Survey, Finland.

Figure 3. A photogrammetric test field with permanent and temporal radiometric and spatial resolution reference targets in Sjökulla, Finland [50]. Photo by the National Land Survey, Finland.

In an appropriate laboratory calibration facility, it is possible to determine the sensor’s response accurately under a wide range of conditions. In an ideal situation (accurate, stable sensor and rigorous calibration facility), the laboratory calibration could be the only radiometric calibration method needed. However, other calibration methods are needed because sensor/system properties can change with time and/or because the sensor parameters determined in laboratory can differ from the system parameters in operational conditions. On-board and vicarious methods are crucial for the re-calibration of satellite sensors that cannot be brought to a laboratory after the launch; several characterized test sites are available around the world for vicarious calibration of satellite systems [42]. Studies have indicated invalidity of laboratory calibration in flight conditions also for airborne systems [45,46,47]. Calibration is an active research topic at the moment [42]

2.4.1. Radiometric calibration approaches of photogrammetric sensors

The manufacturers of photogrammetric sensors have established laboratory-based calibration approaches for the radiometry. The calibration provides various corrections that are applied to the images after image collection (Figure 1).

A detailed description has been given of the laboratory calibration process of the ADS [36,48]. The spectral calibration is performed using a spectral measurement unit applying a National Institute of Standards and Technology (NIST) traceable light source. An Ulbricht sphere providing NIST traceable radiances is used for the relative and absolute radiometric calibration. The dark signal correction is determined partially at the laboratory and partially in flight.

The laboratory calibration processes have been described in fewer details for other systems. For the DMC, the relative calibration is performed using an Ulbricht sphere for each aperture, temperature and TDI settings [37,49]. The laboratory calibration of the UltraCam determines relative radiometric calibration for various aperture settings by using flat field images provided by nor­mal light lamps with known spectral illumination curves. For the IGN camera, the imaging of a white and uniform light source provides the gain, performance of shutter, linearity, noise level, sensitivity of each pixel and lens falloff; white balance is determined using a Solux lamp.

In some empirical studies, vicarious radiometric calibration and characterization of photogrammetric systems has been carried out. In Finland, comprehensive campaigns with various systems have been carried out at the Sjökulla test field [1,2,13,50]. Institut Cartogràphic Catalonia (ICC) has performed comprehensive campaigns with DMC at the Banyoles test field [38,51,52]. The German Society for Photogrammetry and Remote Sensing (DGPF) carried out test flights with many systems at the Vaihingen/Enz test field in Germany in the summer of 2008 [53]. Reference targets in these campaigns have been gray targets and color panels as reflectance reference and Siemens star and bar targets as resolution targets; in situ reference measurements have been performed using spectroradiometers, and various equipment (e.g. atmospheric lidar and sun tracking photometer) have been used to measure atmospheric state. In some of the campaigns photogrammetric and calibrated radiometers (hyperspectral sensors) have been operated simultaneously. An example of a photogrammetric test field with permanent and transportable reflectance and spatial resolution targets is shown in Figure 3.

In conclusion, the information about the calibration of photogrammetric systems given by the existing literature is not sufficient for quantitative processes. For example, information about calibration process is insufficient in many cases and there does not exist information on validity of laboratory calibration in actual operational conditions.

2.5. Radiometric Correction

As discussed in Section 2.2, many factors influence the imaging process. In Earth remote sensing applications, the objective of radiometric image correction is to eliminate those effects from images that disturb rigorous quantitative and visual evaluation of scene objects. The images can be processed to various processing levels. For quantitative applications, the objective is to obtain either absolute reflectance information of the scene elements or to obtain correct relative magnitudes of the reflectance of scene elements in a single channel, in different channels, in different images taken in one mission, or in images taken at different times. For visual applications, the objective is often to obtain natural colors.

The first step in the radiometric processing chain is to apply the instrument corrections. It is a resampling process consisting of geometric and radiometric corrections, which can be based on sensor calibration, information collected during the flight mission, and image measurements (Section 2.4). This is an integral part of the image acquisition process (Figure 1). If the absolute radiometric calibration is known, the DNs can be transformed to the units of radiance.

The fundamental task of radiometric image correction is to eliminate the disturbances caused by the atmosphere (Section 2.2). A physically based approach for atmospheric correction is the radiative transfer modeling by e.g Modtran [54] or 6S [55]; the inversion of the radiative transfer code retrieves the directional bottom of atmosphere reflectance from the radiometrically-calibrated imagery. Commonly used approaches also are dark object methods [56], empirical line methods [57] and histogram matching methods [58]; reviews of different methods can be found in many sources [4,5,6,59]. The variations in atmospheric conditions, both in space and time, make the atmospheric correction challenging. In many applications it is necessary to eliminate the influences of object reflectance anisotropy (BRDF-correction), for example, in order to produce uniform image mosaics. The reflectance anisotropy can be substantial on images that have been collected with sensors with a large field of view, such as photogrammetric sensors. Various physical and empirical BRDF-models are available for BRDF-correction [6,11,48]; the special challenge of the BRDF-correction is that different BRDF-models should be used for different objects, thus the image content should be known before making the correction. In image interpretation tasks also the shadows have to be considered [60]. In all these steps, the object topography has to be taken into account. For different sensors, different methods are optimum [6]. An example of commercial correction software based on atmospheric models is ATCOR [61]; a review of available atmospheric correction software and methods for remote sensing systems is given by Gao et al. [59]. Radiometric correction is an active research topic at the moment.

For relative radiometric correction (or normalization), popular methods are those based on invariant objects, empirical line methods and histogram matching [4,5,6,62].

A future trend is to store relatively or absolutely radiometrically corrected multi-source and multi-temporal data in remote sensing image databases. Haest et al. [11] and Biesemans et al. [63] have recently presented a prototype system that can handle photogrammetric data sets.

2.5.1. Radiometric correction in photogrammetric systems

Radiometric processing in photogrammetric processes follows the principles described above. In all systems the instrument correction is performed after image collection using software provided by the sensor manufacturer [28,36,49].

Software developments are under way to enable efficient and accurate correction of the radiometry of photogrammetric image blocks. Leica Geosystems has already presented a processing chain from raw images to reflectance images for the ADS. After applying the absolute calibration parameters determined in the laboratory, the atmospheric correction is made by utilizing radiative transfer modeling, and finally BRDF-correction is performed [36,48]. Manufacturers of DMC and UltraCam have not presented quantitative radiometric processing chains. Photogrammetric software packages include modules for radiometric balancing, which were originally developed for producing uniform orthophoto mosaics from film images; they are based on statistical adjustment and combine atmospheric and BRDF-corrections to single step (e.g. Intergraph Image Station PixelQue [64], BAE Systems Socet Set Dodger [65] and Ortho Vista [66]). Several organizations are currently developing radiometric block adjustment software [11,38,51,67]; methods from remote sensing systems are being modified and they are entering the photogrammetric processes.

Additional radiometric manipulations, mainly used for visual applications, include gamma correction, tonal transformation, transformation from the 16-bit to 8-bit domain, pansharpening, and image enhancement and restoration [49,64,68].

To summarize, the radiometric processing methods of photogrammetric imagery are under development. The general view is that the physically based methods would provide the best results, but the existing literature does not give information about the suitability of various methods for photogrammetric imagery or about the absolute or relative radiometric accuracy of corrected photogrammetric imagery.

3. Questionnaire on Radiometric Processing in Photogrammetric Production Lines

Part of the EuroSDR project is a questionnaire sent to various interest groups dealing with photogrammetric images. The questionnaire was delivered to several large and medium format photogrammetric sensor manufacturers, photogrammetric software providers, NMAs and Universities in October 2008. This questionnaire was considered crucial, because existing literature covers only partially the modern photogrammetric process and it does not give information about radiometric processing in operational processes.

Objectives of the questionnaire were to: (1) obtain a picture of the actual situation; (2) detect main weaknesses of existing digital camera radiometric processing; (3) look for main trends on existing and future development in this field; (4) know what the advantages of better radiometric processing are and find which applications ask for better radiometric processing. Based on the characterization of the photogrammetric process (Figure 1), the questions were classified under five themes: sensor, calibration, image collection, post-processing and utilization of the images. Under each theme, the questions were further divided into questions related to the current situation and to the desired situation.

Table 2. Participants of the EuroSDR radiometry questionnaire.

Participant	Organization	Role
Institut Cartogràphic Catalonia (ICC)	NMA	Software developer, Data provider, Data user, Research
Institut Géographique National, France (IGN)	NMA	Sensor manufacturer, Software developer, Data provider, Data user, Research
National Survey and Cadastre, Denmark (KMS)	NMA	Data user
National Land Survey, Finland (NLS)	NMA	Data provider, Data user
Ordnance Survey, Great Britain (OS)	NMA	Data provider, Data user
Land Survey of Switzerland (Swisstopo)	NMA	Data provider, Data user
ReSe Applications Scläpfer, Switzerland (ReSe)	Company	Software, consultant
Finnish Geodetic Institute, Finland (FGI)	Research	Research
Institut für Geoinformatik und Fernerkundung, Universität Osnabrück (IGF)	University	Research

Table 2. Participants of the EuroSDR radiometry questionnaire.

Participant	Organization	Role
Institut Cartogràphic Catalonia (ICC)	NMA	Software developer, Data provider, Data user, Research
Institut Géographique National, France (IGN)	NMA	Sensor manufacturer, Software developer, Data provider, Data user, Research
National Survey and Cadastre, Denmark (KMS)	NMA	Data user
National Land Survey, Finland (NLS)	NMA	Data provider, Data user
Ordnance Survey, Great Britain (OS)	NMA	Data provider, Data user
Land Survey of Switzerland (Swisstopo)	NMA	Data provider, Data user
ReSe Applications Scläpfer, Switzerland (ReSe)	Company	Software, consultant
Finnish Geodetic Institute, Finland (FGI)	Research	Research
Institut für Geoinformatik und Fernerkundung, Universität Osnabrück (IGF)	University	Research

The organizations that replied to the questionnaire are shown in Table 2. The widest response was obtained from NMAs, most of which are both data providers and users; some also have their own software development, and IGN is manufacturing its own sensor. NMAs that responded the questionnaire cover a relatively large portion of the Europe (Catalonia, Denmark, Finland, France, Great Britain and Switzerland). ReSe is a software company behind the atmospheric correction software ATCOR for spaceborne and airborne scanner images [61]. They are specialized in processing and utilization of imaging spectroscopy data. Finally, two research organizations, IGF and FGI, responded to the questionnaire. IGF’s focus is to use imagery in land cover and land use identification and classification at different scales. FGI is specialized in photogrammetric test fields, extensive empirical campaigns and goniospectrometry, and utilize images in interpretation applications. In total, five responses were obtained from data providers, six from data users, one from a sensor manufacturer, one from a radiometric software manufacturer, and two from research organizations.

In the following, the responses of the NMAs are first analyzed; the issues related to DMC are dominating the analysis because most of the responses were obtained from DMC users. Analysis emphasizes topics that are not covered in existing literature.

3.1. Sensor

The questions concerned the sensor construction, technical details of various sensor components, and taking the radiometric aspects into account in sensor construction (Table 1). Furthermore, recommended system set up (e.g. camera mount), operating conditions, performance indicators, and intended application areas were enquired.

3.1.1. Current situation

Three of the NMA:s have their own DMC, one has ADS40, IGN develops imaging systems themselves, and KMS purchases the imagery collected with frame sensors (DMC or UltraCamD). The principles of these sensors are given in Section 2.3.1.

3.1.2. Limitations and desired sensor properties

Some issues concerned all sensors. Data users were asking for additional channels besides regular PAN, R, G, B and NIR channels; reasons for these requests or desired channel properties were not specified. The only specification was to have well-defined spectral bands without overlap. The users who perform extensive national projects wished to have a wider image format (image width more than 12,000 to 14,500 pixels) in order to reduce the production costs.

A general issue related to frame sensors was the lower resolution of the multispectral channels. Also, the merging of several images to form the large format images was considered problematic. The channels of the DMC were not considered perfect; improvements were requested for PAN and NIR channels. A better pansharpening ratio was requested for the DMC.

Swisstopo considered the technical realization of the ADS40 SH52, having the R, G, B and NIR channels with the same viewing angle but the PAN channel 2° separated, as problematic. Another problem with the ADS40 is that in the typical image collection mode, the pixel depth is reduced from 12-bit to 8-bit using a lossy compression due to the data storage speed limitations. This reduces the radiometric quality. It is possible to collect imagery without the compression by limiting the number of channels or by decreasing the flying speed, but this is not typically acceptable solution. The manufacturer has announced that in the latest version of the sensor the data storage speed has been increased so that this problem is no longer relevant [25].

Manufacturers were requested to be more open with respect to the technical realization of the systems. Getting this information might be problematic especially for data users that purchase imagery collected by different kinds of continually changing imaging systems; data providers might be more aware of the technical realization of their systems and sensors.

3.2. Calibration

Questions concerned the phases (laboratory, on-board, test field, self-calibration) of the radiometric, color, spectral and spatial resolution calibration of the sensor and system (Section 2.4). For each phase and property, the details of the calibration method (instrumentation, calculation method), parameters and their accuracy, and quality indicators, and recommended calibration interval were requested.

3.2.1. Current situation in calibration

Currently, operational radiometric calibration is based solely on laboratory calibration performed by sensor manufacturers (Section 2.4.1). Test field calibrations/validations of radiometry are rarely performed. NLS, ICC and IGN reported on test field calibration or validation of the radiometry and spatial resolution (Section 2.4.1). OS has used resolution targets for resolving power determination.

3.2.2. Limitations and desired calibration process

The general conclusion concerning the calibration method was that laboratory calibration is the requirement for the most accurate calibration; e.g. quality requirements for the relative calibration accuracy of multi-head systems are very high (on the level of 1/1000; IGN).

Some shortcomings were reported concerning the calibration process of the DMC. It involves only relative calibration separately within each band; absolute radiometric calibration would be necessary. Sensitivities and color balance of the DMC multispectral channels do not correspond to the human visual system, which makes the colorimetric calibration necessary to obtain true colors.

Test field calibration (vicarious calibration) and validation was considered as an important validation method, which should be utilized in many phases of the sensor life cycle. The manufacturer should make a radiometric test flight before they deliver the system to the customer so that they could test the systems and give appropriate instructions for the data providers. For example, NLS had significant problems in determining the appropriate exposure and aperture settings for their new DMC in autumn of 2008 (Section 3.3.2). There also should be suitable test fields, where data providers could validate the performance of their systems. Guidelines for reference targets, reference measurements, as well as tolerances for the acceptable results are needed.

Also, calibration approaches for each mapping project were requested. It would be desirable to be able to carry out self-calibration, similar to the geometric self-calibration process, for each target flown. A platform calibration method for radiometry in the image acquisition post-processing phase, e.g. to compare relatively each channel, was suggested to be a potential method for the confirmation of the laboratory calibration. Specifications of methods and reference targets are needed for self-calibration and platform calibration processes. These methods are needed if the system calibration is not valid in operational conditions.

As the result of the calibration, the desired situation is to have at least the relative and absolute radiometric calibration parameters, spectral sensitivity, colorimetric calibration, electronic and thermal noise levels, and accuracy estimates of the calibration and sensor performance. For the data providers and data users, it is important that the calibration process is fully documented. For the users of data from different systems, it is important that calibration information from various sensors is comparable; a standardized calibration process would be desirable. Currently, the calibration procedures in most cases are not sufficiently documented or transparent, and calibration documentations of different systems are not comparable.

3.3. Image Collection

The image collection is a fundamental step in the radiometry chain and thus far uncovered in the literature in the cases of digital photogrammetric sensors. Several factors influence the radiometry in the image collection phase (Table 1). The questions concerned system configuration, conditions where the image collection is carried out, the system settings, and the on-the-fly quality control methods. Furthermore, descriptions of reflectance reference targets and reference measurements during the image collection were enquired.

3.3.1. Current situation in image collection

System configurations are presented in Table 3. Various aircrafts are used. The sensors are mounted on stabilized camera mounts and GNSS/IMU-systems for direct position and attitude measurement is integrated into the systems. The frame sensors apply TDI-based FMC. Some aircraft have pressure-stabilized cabins with a glass window on the camera port; others have no pressure-stabilized cabins or glass windows.

Central technical limits of the evaluated systems are presented in Table 4. In most cases, the minimum possible GSD is approximately 5 cm; it is limited by the lowest possible flight speed and flying height, and illumination conditions; further limitations are set in the case of frame sensors by the minimum frame rate required by stereoscopy and in the case of ADS40 by the smallest possible integration time and speed of data storage. The maximum GSD is limited by the maximum flying height of the aircraft, and in the example cases, it is 43–100 cm. It should be noticed that in the case of DMC and UltraCamD the GSD of the multispectral channels is 3–4 times larger than the nominal GSD of the PAN-channel (Section 2.3.1). The maximum flying altitude is 4.3–10 km from sea level, and the flying speed limits are 110–240 knots.

Table 3. System configurations of data providers. FMS: flight management system (CCNS, Integraph and FCMS are commercial flight management systems).

Organization	Sensor	Vehicle	GNSS/IMU	Gyro stab. mount	FMS	Pressu­rized cabin	Camera port glass
ICC	DMC	Partenavia P-68	Yes	Yes	CCNS	No	No
		Cessna Citation I	Yes	Yes	CCNS	Yes	Yes
		Cessna Caravan 208N23	Yes	Yes	CCNS	No	No
IGN	IGN	Beechcraft Super King Air 200T	GNSS	Yes	Own	Yes	Yes
		Beechcraft Super King Air B200T	GNSS	Yes	Own	Yes	Yes
		Beechcraft Super King Air B200	GNSS	Yes	Own	Yes	Yes
NLS	DMC	Turbo Commander	Yes	Yes	Intergraph	Yes	Yes
OS	DMC	Cessna 404	Yes	Yes	Intergraph	No	No
Swisstopo	ADS40	Beechcraft Super King Air 350C	Yes	Yes	FCMS	Yes	Yes
		Twin Otter DHC 6-300D	Yes	Yes	FCMS	No	No

Table 3. System configurations of data providers. FMS: flight management system (CCNS, Integraph and FCMS are commercial flight management systems).

Organization	Sensor	Vehicle	GNSS/IMU	Gyro stab. mount	FMS	Pressu­rized cabin	Camera port glass
ICC	DMC	Partenavia P-68	Yes	Yes	CCNS	No	No
		Cessna Citation I	Yes	Yes	CCNS	Yes	Yes
		Cessna Caravan 208N23	Yes	Yes	CCNS	No	No
IGN	IGN	Beechcraft Super King Air 200T	GNSS	Yes	Own	Yes	Yes
		Beechcraft Super King Air B200T	GNSS	Yes	Own	Yes	Yes
		Beechcraft Super King Air B200	GNSS	Yes	Own	Yes	Yes
NLS	DMC	Turbo Commander	Yes	Yes	Intergraph	Yes	Yes
OS	DMC	Cessna 404	Yes	Yes	Intergraph	No	No
Swisstopo	ADS40	Beechcraft Super King Air 350C	Yes	Yes	FCMS	Yes	Yes
		Twin Otter DHC 6-300D	Yes	Yes	FCMS	No	No

Table 4. Technical limits of the systems. Aperture and exposure time settings: A: Automatic, M: Manual.

Organi­zation	Aperture	Exposure time	Flight Speed [knt]	Maximum flight altitude [m]	GSD range [cm]	Refl. ref. targets/Atm. obs
ICC*	A	A	110-150	4300	5-43	No
	A	A	145-240	8800	5-70
	A	A	110-160	7200	5-88
IGN*	Fixed	M	140-180	9000	20-100**	No
	Fixed	M	140-180	10650	20-100**
	Fixed	M	140-180	10650	20-100**
NLS	A/M	A/M	120-200	10000	5-100	No
OS	A/M	A/M	110-140	3000	5-25	No
Swisstopo*	Fixed	A/M	140-180	10700	5-100	No
	Fixed	A/M	110-140	8100	5-81

• * Different specifications are related to aircrafts in Table 3.
• ** GSD range for the IGN’s version 1 camera, values will be different for the version 2.

Table 4. Technical limits of the systems. Aperture and exposure time settings: A: Automatic, M: Manual.

Organi­zation	Aperture	Exposure time	Flight Speed [knt]	Maximum flight altitude [m]	GSD range [cm]	Refl. ref. targets/Atm. obs
ICC*	A	A	110-150	4300	5-43	No
	A	A	145-240	8800	5-70
	A	A	110-160	7200	5-88
IGN*	Fixed	M	140-180	9000	20-100**	No
	Fixed	M	140-180	10650	20-100**
	Fixed	M	140-180	10650	20-100**
NLS	A/M	A/M	120-200	10000	5-100	No
OS	A/M	A/M	110-140	3000	5-25	No
Swisstopo*	Fixed	A/M	140-180	10700	5-100	No
	Fixed	A/M	110-140	8100	5-81

• * Different specifications are related to aircrafts in Table 3.
• ** GSD range for the IGN’s version 1 camera, values will be different for the version 2.

Exposure and aperture settings appeared to be critical issues for the frame sensors (Table 4). They have high impact on radiometric quality, and they have to be accounted for in the radiometric image correction. In the case of DMC, the aperture and exposure times are variable. With Intergraph’s flight management software these parameters can be controlled manually by giving a “light value” (exposure value), from which the exposure time and aperture are calculated, but with the ICC’s system the parameters are selected automatically. The approach of the OS is to use automatic parameters, excluding the water features and predominant topographic features (e.g. quarries), where manual settings are used. In the IGN’s system the aperture is fixed and the global exposure time is set manually based on evaluations of collected images. The systems record different information about system conditions during the flight (e.g. aperture value, exposure time, number of TDI steps and system temperature).

There are some possibilities for on-the-fly quality control of collected data. In the case of DMC, thumbnail images provided by an attached video camera can be used to check instant cloud cover and the image quality can be checked from a sample of acquired images. With IGN’s system, acquired images are displayed in real time, and it is then possible to visually check saturated areas on the fly. In the end, the camera operator’s experience is used to evaluate the appropriate conditions.

The settings for flight campaigns are dependent on the application. An important application area of photogrammetry is the national topographic mapping and map updating programs, including also countrywide orthophoto generation; parameters of these programs of the participants are presented in Table 5. GSD is 10–50 cm; it is typically smaller in urban areas than in rural or mountain areas. Requirements for flight conditions are clear atmosphere, no clouds and no haze. IGN has given a recommendation for the visibility superior to 15 km, with no cirrus cloud, but states that the production constraints sometimes involve compromises. The recommendations for the minimum allowable solar elevation angles from horizon are 25°–40°. Update intervals of data sets are 1–10 years. Specifications for imaging seasons vary greatly: in some cases imaging is performed in spring during non-leaf season while in some cases the imaging season extends from early spring to late autumn.

In the standard photogrammetric production process, radiometric reference targets are not used, and atmospheric state is not measured (Table 4).

Table 5. Specifications for countrywide image collection programs of various participants.

Organization	Visibility/Clouds	Solar elevation [º]	Season [month]	GSD [cm]	Temporal resolution [years]	Application
ICC	No cloud	30	mid 3-	25	1	Orthophotos
			mid 10			Stereomapping
						Classification
IGN	Visibility > 15 km Clouds: <5%/image, <1%/mission	30	mid 4-	20-50	5	Orthophotos
			mid 10			Stereomapping
						Classification
KMS	Good visibility No cloud	25	3-4	10, 20	1, 3	Orthophotos
			no leaf			Stereomapping
NLS	No cloud	30 (25)	mid 4-8	30, 50	5-10	Orthophotos
						Stereomapping
OS	No cloud	25	3-11	15, 20, 25	2-8	Orthophotos
						Stereomapping
Swisstopo	No cloud	40	4-9	25, 50	3	Orthophotos
						Stereomapping

Table 5. Specifications for countrywide image collection programs of various participants.

Organization	Visibility/Clouds	Solar elevation [º]	Season [month]	GSD [cm]	Temporal resolution [years]	Application
ICC	No cloud	30	mid 3-	25	1	Orthophotos
			mid 10			Stereomapping
						Classification
IGN	Visibility > 15 km Clouds: <5%/image, <1%/mission	30	mid 4-	20-50	5	Orthophotos
			mid 10			Stereomapping
						Classification
KMS	Good visibility No cloud	25	3-4	10, 20	1, 3	Orthophotos
			no leaf			Stereomapping
NLS	No cloud	30 (25)	mid 4-8	30, 50	5-10	Orthophotos
						Stereomapping
OS	No cloud	25	3-11	15, 20, 25	2-8	Orthophotos
						Stereomapping
Swisstopo	No cloud	40	4-9	25, 50	3	Orthophotos
						Stereomapping

3.3.2. Limitations and desired image collection process

The importance of efficient tools for the evaluation of the exposure settings on-the-fly was emphasized. If the data quality is not checked on-the-fly, it is possible that the entire flight mission does not fulfill the quality requirements. On the other hand, digital imaging gives many possibilities for the on-the-fly quality control. As on-the-fly quality control methods, the data providers suggested tools for making quick checks for all images during the flight mission and possibilities to calculate statistics to check the correctness of the settings (e.g. histograms and saturation).

The NLS faced problems in the determination of feasible exposure parameters for their new DMC (other DMC users have faced similar problems); the test flights performed with automatic exposure settings were seriously overexposed; the situation was not observed during the flight. The specific problem of the ICC is that it is not possible to manually set the exposure settings through their flight management system. The manual exposure parameters should be available when needed, because the automatic parameters do not provide acceptable image quality on special conditions, e.g. in the areas with water features.

The current systems do not support radiometric correction to optimum effect. Ideally, the atmospheric conditions could be recorded in the aircraft at time of data capture. For instance, sensors for measurement of irradiance and illumination conditions could be installed on the top of the aircraft and attached to the sensor. Additional channels could be integrated to enable measurement of the atmospheric water vapor.

The operational use of reflectance reference targets and measurements of the atmospheric state are new issues for the mainstream photogrammetric image collection. Because these tasks are laborious, the processing methods should be developed so that they are not necessary. However, in some applications reflectance reference targets might be needed; for these situations specifications are needed for the brightness range, material, size, number, and spectral, radiometric and angular properties of the targets.

3.4. Post-processing

Different post-processing steps and the specific processes for various image products (orthophotos, stereomodels) and different applications (aerial triangulation, visual interpretation, classification) were requested. Methods and indicators to characterize the quality of the imagery and quality requirements were enquired. Two types of post-processing are relevant: the post-processing of the data acquisition system and the post-processing of the image product generation system (Figure 1; Section 2.5).

3.4.1. Current approaches for post-processing

The radiometric processing steps in the evaluated systems are presented in Table 6 and discussed below.

The post-processing in the data acquisition process is performed using the manufacturer-provided software as described in Section 2.5.1. Outputs of this process are DNs.

A very complicated radiometric processing is necessary for orthophoto mosaics where the radiometric uniformity is of interest; complete radiometric corrections are not always made for stereo models. ICC and IGN utilize their own radiometric block adjustment methods (Section 2.5.1) [38,51,67]. Other software used are the methods included in photogrammetric software (e.g. BAE Systems Socet Set Dodger and OrthoVista) and methods included in general-purpose image processing software (Agfa Apertune, Adobe Photoshop) (Section 2.5.1). The comprehensive radiometric correction chain of Leica Geosystems [48] is not yet used by Swisstopo in the post-processing of ADS images.

In visual applications, various further image enhancements are performed for the radiometrically corrected imagery, including gamma corrections, histogram operations, sharpening, color balancing etc. DMC users use pansharpened images in visual applications. The pixel depth is typically 8 bits/pixel/channel.

Only ICC and IGN reported on the use of the imagery in quantitative (classification) applications. ICC uses vegetation indices to eliminate the influences of radiometric variability and uses non-pansharpened imagery. ICC is also planning to use a calibrated radiometer in simultaneous flight to obtain real atmospheric parameters. IGN uses the same imagery both in visual and quantitative work, because for logistics reasons, it is not possible to produce many variations.

Quality control of imagery includes the evaluations of dynamic range, saturation, noise, continuity, histograms and evaluations of the information loss in shadows. Swisstopo evaluates colorimetric quality by comparing images to color model images; they also control information loss caused by radiometric processing in shadows.

3.4.2. Limitations and desired post-processing approach

The desired output of the data acquisition system is a system-corrected radiance image. None of the data providers are producing radiance images.

Image users requested two kinds of image products: georeferenced, either absolutely corrected reflectance images or true color images. The radiometric block adjustment methods (Section 2.5.1) aim at producing these outputs automatically. For instance, the IGN’s method is already in operational use, but several improvements are still necessary [67]. It appeared that development and investigations are needed to operationally produce accurate reflectance and true-color images.

An important issue appeared to be the geometric transformations, with interpolation and resampling steps. To avoid degradation of image radiometry, the number of resampling steps should be minimized. As many operations as possible should be stored in the image header files and performed on the operating system level. For instance, typically, orthophotos are rectified to map projection, which in the case of tilted images provide huge data files with black pixels; the image rotation information in the header files could perform the rectification. The same approach could be used to provide different radiometric processing levels.

The desirable situation is to have 16 bits data dynamics. Several users are using 8 bits/pixel/channel imagery; this does not completely utilize the dynamic range of the new sensors. Support is needed in the entire image production and utilization chain to exploit greater than 8-bit pixel depth.

The post-processing should be automatic and efficient, because huge amounts of images are processed. It is necessary to improve the accuracy and efficiency of the processing methods. However, from the operational point of view, the post-processing systems should provide also necessary semi-automatic and interactive tools, because automatic processes do not always succeed.

Table 6. Post-processing systems of five NMAs. PPS is the Intergraph’s software for DMC and GPRO is Leica Geosystems software for ADS40; other software are well-known photogrammetric, image processing or cartographic software, or explained in the text.

ICC	Post-processing after image collection by Intergraph PPS
	Instrument correction; color balancing; gamma correction; pan sharpening; 16 to 8 bit conversion
	Orthophotos by own software (pansharpened, 4-band images)
	Radiometric block adjustment: hotspot and vigneting, color balance, color continuity, BRDF, relative radiometric adjustment between various bands, different parts of the images, different images of a single acquisition, images from different acquisitions, final local image enhancement
	Quality control: Spatial resolution, dynamic range, saturation on extreme values, radiometric artifacts (blooming, radiometric noise, etc.), good (natural) color balance
	Remote sensing images by own software (non-pansharpened, 4 band images)
	Use of vegetation index to reduce radiometric variability. Atmospheric correction: use of calibrated sensors in the same flight in order to recover physical values.
	Quality requirements: No interpolation in post-processing, if possible
IGN	Post-processing after image collection by own software
	Instrument correction
	Orthophotos, stereo models and classification images by own software (8 bit, 4-band images)
	Radiometric block adjustment: empirical BRDF-correction, global haze variations; 16 to 8 bit conversion; gamma correction. All radiometric corrections are applied simultaneously
	Additional cosmetic local correction and image enhancement by Adobe Photoshop
	Quality control: Histogram evaluations, visual assessment using hardware calibrated monitors
NLS	Post-processing after image collection by Intergraph PPS
	Instrument correction; color balancing; gamma correction; pan sharpening; 16 to 8 bit conversion
	Stereomodels and orthophotos by BAE Systems Socet Set/ORIMA (pansharpened, 8-bit, 4-band images)
OS	Post-processing after image collection by Intergraph PPS
	Instrument correction; color balancing; pansharpening; 16 to 8 bit conversion
	Orthophotos by BAE Systems Socet Set and Intergraph ISAT (8 bit images)
	Statistical radiometric image wise correction by Adobe Photoshop, Agfa Apertune, BAE Systems Socet Set Dodger and Intergraph Dodger. Additional image enhancement by Adobe Photoshop.
	Quality control: In-house imagery testing system: tolerances for radiometric accuracy and image appearance
Swiss­topo	Post-processing after image collection by Leica Geosystems GPRO
	Instrument correction
	Orthophotos by Leica Geosystems GPRO (8 bit RGB images with all corrections, NIR channel with less processing)
	Image enhancement by Adobe Photoshop interactively for each flight line: histogram clip, color balancing, contrast optimization, sharpening, 16 to 8 bit conversion, geometric restoration (bridges, cliffs, lakes and rivers)
	Radiometric block adjustment of flight lines by Ortho Vista, cosmetic editing by Adobe Photoshop
	On-the-fly color enhancements by ArcInfo ArcMap
	Quality control: Visual check of final orthophoto mosaic using hardware calibrated monitors using a color model. The loss of information in the shadows and highlights should not exceed 0.01% of all pixels in one image unit (tile).
	Stereomodels by Leica Geosystems GPRO
	Color enhancement by ArcInfo ArcMap during measurement

Table 6. Post-processing systems of five NMAs. PPS is the Intergraph’s software for DMC and GPRO is Leica Geosystems software for ADS40; other software are well-known photogrammetric, image processing or cartographic software, or explained in the text.

ICC	Post-processing after image collection by Intergraph PPS
	Instrument correction; color balancing; gamma correction; pan sharpening; 16 to 8 bit conversion
	Orthophotos by own software (pansharpened, 4-band images)
	Radiometric block adjustment: hotspot and vigneting, color balance, color continuity, BRDF, relative radiometric adjustment between various bands, different parts of the images, different images of a single acquisition, images from different acquisitions, final local image enhancement
	Quality control: Spatial resolution, dynamic range, saturation on extreme values, radiometric artifacts (blooming, radiometric noise, etc.), good (natural) color balance
	Remote sensing images by own software (non-pansharpened, 4 band images)
	Use of vegetation index to reduce radiometric variability. Atmospheric correction: use of calibrated sensors in the same flight in order to recover physical values.
	Quality requirements: No interpolation in post-processing, if possible
IGN	Post-processing after image collection by own software
	Instrument correction
	Orthophotos, stereo models and classification images by own software (8 bit, 4-band images)
	Radiometric block adjustment: empirical BRDF-correction, global haze variations; 16 to 8 bit conversion; gamma correction. All radiometric corrections are applied simultaneously
	Additional cosmetic local correction and image enhancement by Adobe Photoshop
	Quality control: Histogram evaluations, visual assessment using hardware calibrated monitors
NLS	Post-processing after image collection by Intergraph PPS
	Instrument correction; color balancing; gamma correction; pan sharpening; 16 to 8 bit conversion
	Stereomodels and orthophotos by BAE Systems Socet Set/ORIMA (pansharpened, 8-bit, 4-band images)
OS	Post-processing after image collection by Intergraph PPS
	Instrument correction; color balancing; pansharpening; 16 to 8 bit conversion
	Orthophotos by BAE Systems Socet Set and Intergraph ISAT (8 bit images)
	Statistical radiometric image wise correction by Adobe Photoshop, Agfa Apertune, BAE Systems Socet Set Dodger and Intergraph Dodger. Additional image enhancement by Adobe Photoshop.
	Quality control: In-house imagery testing system: tolerances for radiometric accuracy and image appearance
Swiss­topo	Post-processing after image collection by Leica Geosystems GPRO
	Instrument correction
	Orthophotos by Leica Geosystems GPRO (8 bit RGB images with all corrections, NIR channel with less processing)
	Image enhancement by Adobe Photoshop interactively for each flight line: histogram clip, color balancing, contrast optimization, sharpening, 16 to 8 bit conversion, geometric restoration (bridges, cliffs, lakes and rivers)
	Radiometric block adjustment of flight lines by Ortho Vista, cosmetic editing by Adobe Photoshop
	On-the-fly color enhancements by ArcInfo ArcMap
	Quality control: Visual check of final orthophoto mosaic using hardware calibrated monitors using a color model. The loss of information in the shadows and highlights should not exceed 0.01% of all pixels in one image unit (tile).
	Stereomodels by Leica Geosystems GPRO
	Color enhancement by ArcInfo ArcMap during measurement

The traceability and comparability of data collected with different data providers is especially a problem for users that order images. These users require transparency from both manufacturers and data providers for the entire image processing chain. Also, it is an issue that the image processing and thus the result depend on the subjective choices of the operators. To allow quantitative use, image enhancement operations (sharpening, color adjustments etc.) should not be applied without having the possibility to resolve for the radiometry. The ideal post-processing would be standardized and internationally accepted, automated, objective procedures.

3.5. Utilization of the Images

The applications and the basic image products for each application were requested. Also, the expected benefit of better radiometric processing was requested.

3.5.1. Current situation in photogrammetric applications

The major tasks of national photogrammetric processes are the production of orthophotos and stereomodels, and various topographic mapping and map updating tasks using this data (Table 5). The methods for orthophoto and stereomodel production are given in Section 3.4.1; the participants did not give details of the automation level of the topographic mapping processes.

3.5.2. Limitations and desired applications

Limitations of the radiometric processing in the orthophoto and stereomodel generation were presented in Section 3.4.2. Participants did not describe limitations of other topographic mapping tasks.

The expected benefit of accurate radiometric processing is more automatic and efficient imagery post-processing, better visual image quality (less visual “color-borders” in orthophoto mosaics) and more accurate classification.

4. Discussion and Conclusions

This investigation provides a state-of-the-art review on radiometric aspects of digital photogrammetric images. The analysis is based on literature research and a questionnaire to various interest groups. An important contribution was the characterization of the photogrammetric image acquisition and image product generation systems, and evaluation of properties of systems of five data providers and six image users in this framework. Central parameters of six national topographic photogrammetric image acquisition programs were also presented.

The results showed that there are several fundamental problems in photogrammetric processes, which hinder the quantitative utilization of image radiometry, make the radiometric processing complicated and laborious, and decrease the quality of output products. Shortcomings were observed in all evaluated aspects, i.e., sensor, calibration, image collection and image post-processing. Furthermore, problems appeared in the interfaces of different interest groups of the photogrammetric process.

For conventional photogrammetric applications, the large image format, good spatial resolution, high geometric accuracy, and true colors are crucial. These requirements are, to some extent, in contradiction with optimal, quantitative remote sensing sensors: there is especially the trade-off between the spatial and spectral resolution, and the channels optimized for quantitative studies are not optimum for visual applications. However, the most fundamental sensor related limitation hindering the quantitative use of the radiometry is, in most cases, the incomplete description of the measurement system.

The general problems related to the current calibration procedures with most systems were that all necessary parameters are not determined and there is not information about the quality of the calibration. Further issues are that there are not widely accepted procedures for calibration, and the calibration documentations are not comparable, transparent nor complete. The laboratory calibration is practically the only calibration method used in practice. However, in a truly quantitative process, calibration and validation should be performed in various phases of the process (in-flight, platform, vicarious/test field and self-calibration).

The image collection process should be properly analyzed to identify the steps that influence the radiometry. The sensor related limitations should be understood. Other components of the system, e.g. the aircraft type, can also limit the radiometric quality. The selection of image collection parameters (e.g. season, solar elevation angle, atmospheric conditions) influences the radiometric quality and potential of imagery.

Efficient and rigorous commercial radiometric corrections software, tuned for photogrammetric imagery, is largely missing. The radiometric processing is largely performed by statistical, not physically based, methods and the process involves subjective, interactive decisions. A physically based radiometric processing chain is available for the ADS40 [48], but there does not yet exist scientific proof on the performance of this process. Other approaches for radiometric block adjustment are also being developed.

The results indicated that it is necessary to identify the interest groups related to the photogrammetric process. The fundamental processes are the sensor manufacturing, software development, photogrammetric image acquisition, photogrammetric image product generation (orthophotos, stereomodels), applications and research. The main interest groups are data users, data providers, sensor manufacturers, software developers and research organizations. Each interest group can be further divided into different subclasses based on the tasks they undertake. For example, the data user can undertake all phases of the process (sensor manufacturing, image collection, software development, image product generation, applications) or he can concentrate on only the application. Each of these groups has a different possibility to influence or discover the details of the radiometric processing chain. Important interest groups are presented in Table 7; the interest groups of the participants of the questionnaire are shaded.

All data users expressed concern about the traceability of the radiometry. An important comment comes from the organization who purchases all the imagery: “We lack information on the entire data processing and also lack technical information on the integrated sensor system (e.g. position of GNSS/IMU related to the image sensor) and how the resulting image frame is computed”. The traceability and comparability of data collected with various sensors is especially a problem for users that order images. For the IGN, who is manufacturing its own sensor, all relevant information is available when needed.

Table 7. Various interest groups dealing with image radiometry. The groups that are covered in the questionnaire are shaded. U1-U5: different classes of users, P1-P4: different classes of image producers. R1: research; SW1: software developer; M1: sensor manufacturer.

Type	Sensor manufacturing	Software development	Data collection	Image products	Applications	Research
U1	x	x	x	x	x	(x)
U2		x	x	x	x	(x)
U3			x	x	x	(x)
U4				x	x	(x)
U5					x	(x)
P1	x	x	x			(x)
P2	x	x	x	x		(x)
P3			x			(x)
P4			x	x		(x)
R1						x
SW1		x				(x)
M1	x	x				(x)

Table 7. Various interest groups dealing with image radiometry. The groups that are covered in the questionnaire are shaded. U1-U5: different classes of users, P1-P4: different classes of image producers. R1: research; SW1: software developer; M1: sensor manufacturer.

Type	Sensor manufacturing	Software development	Data collection	Image products	Applications	Research
U1	x	x	x	x	x	(x)
U2		x	x	x	x	(x)
U3			x	x	x	(x)
U4				x	x	(x)
U5					x	(x)
P1	x	x	x			(x)
P2	x	x	x	x		(x)
P3			x			(x)
P4			x	x		(x)
R1						x
SW1		x				(x)
M1	x	x				(x)

The expected benefits of accurate radiometric processing are more automatic and efficient imagery post-processing, better visual image quality, more automatic and accurate applications, and new applications.

The data users’ and data providers’ aspects should be taken into account in the future developments of radiometric processing lines. Data users need validated, reliable image products that fulfill the requirements of the intended application. Data providers need validated, reliable, efficient production lines. A possible approach is to identify different output products, e.g. radiance images, true color images and reflectance images, and quality indicators for these products. Processes and software could then be tuned so that the desired products and quality levels are obtained.

In many photogrammetric applications, typically huge areas are processed; thus the reliability and efficiency are of fundamental importance. The complexity of the photogrammetric image collection process has to be taken into account in the new radiometric processing methods and applications. Fundamental challenges include the variability of atmospheric conditions, seasons, sensors, and processes.

To improve and validate sensors, systems and image post-processing methods, controlled flight campaigns are necessary. The results of the rigorous flight campaigns performed in context of the EuroSDR investigation in 2008 will provide new recommendations for the radiometric processing issues [12,13,52].

Results of this investigation showed that photogrammetric data providers and data users are asking for standardized processes. There are several activities in progress, which aim at developing standards for geometry and radiometry of airborne and space-borne imagery. The activities of the EuroSDR are rising from the needs of the mapping community; the current investigations include the work of the European digital aerial camera certification (EuroDAC) group [69], the radiometry project discussed in this article and a project concerning medium format cameras [32]. Important objectives of the EuroSDR projects are the standardization of the calibration documentation of photogrammetric sensors and establishment of calibration and validation test fields for airborne photogrammetric systems. The International Society for Photogrammetry and Remote Sensing (ISPRS) is a non-governmental organization devoted to the development of international cooperation for the advancement of photogrammetry and remote sensing and their applications [70]. Terms of reference of several working groups of the ISPRS include calibration and validation issues. The Committee on Earth Observation Satellites (CEOS) coordinates civil space-borne observations of the Earth [71]. CEOS calibration and validation activities are emphasizing especially satellite sensors; an example of recent CEOS achievements is a Catalog of Worldwide Test Sites for Sensor Characterization, which is available thorough the Internet [42]. European Fleet for Airborne Research (EUFAR) is an integrating activity of the 7^th framework program of the European Union, aiming at bringing together a large number of European institutions involved in airborne research [72]. National mapping authorities are developing standards and guidelines for new digital photogrammetric systems; examples of these activities are the work by the United States Geological Survey (USGS) [73] and the German DGPF [53]. The International Organization for Standardization (ISO) has several standardization projects related to geospatial imaging; for example, one of the new work item proposals is a standard related to calibration and validation, entitled “Geographic information–Calibration and validation of remote sensing imagery sensors and data” [74,75]. Investigations and co-operation of different actors is needed in order to develop optimized solutions that fulfill the needs of different stakeholders and to avoid overlapping activities.

The new sensors have shown excellent radiometric potential. We anticipate that rapid development will continue in all fields of airborne image processing. We expect that the high resolution, geometrically and radiometrically accurate, multi-spectral, multi-angular photogrammetric imagery could provide new possibilities for remote sensing applications. The Internet-based orthophoto and environmental model servers have an important role in providing up-to-date information for large public audiences. These nationwide databases could also be one component of a more general Earth analysis process, integrated with spaceborne images, hyper-spectral images, laser point clouds and terrestrial data, and all other types of geospatial information. A lot of investigation, development and co-operation are needed in this area, but there are many interesting possibilities, and the future prospects are promising.