Road Asphalt Pavements Analyzed by Airborne Thermal Remote Sensing: Preliminary Results of the Venice Highway.

This paper describes a fast procedure for evaluating asphalt pavement surface defects using airborne emissivity data. To develop this procedure, we used airborne multispectral emissivity data covering an urban test area close to Venice (Italy).For this study, we first identify and select the roads' asphalt pavements on Multispectral Infrared Visible Imaging Spectrometer (MIVIS) imagery using a segmentation procedure. Next, since in asphalt pavements the surface defects are strictly related to the decrease of oily components that cause an increase of the abundance of surfacing limestone, the diagnostic absorption emissivity peak at 11.2μm of the limestone was used for retrieving from MIVIS emissivity data the areas exhibiting defects on asphalt pavements surface.The results showed that MIVIS emissivity allows establishing a threshold that points out those asphalt road sites on which a check for a maintenance intervention is required. Therefore, this technique can supply local government authorities an efficient, rapid and repeatable road mapping procedure providing the location of the asphalt pavements to be checked.


Introduction
According to the European Asphalt Pavement Association (EAPA; http://www.eapa.org/), asphalt pavement is commonly referred to as a mixture of bitumen and mineral matter [12,48] that for Italy asphalts is mainly composed of silicates and limestone [2]. The primary surface defects that occur on the asphalt pavement mixture are raveling, flushing and polishing [67]. Raveling is defined as "the progressive loss of pavement material from the asphalt surface caused by (a) stripping of the bituminous film from the aggregate, (b) asphalt hardening due to aging, (c) poor compaction especially or insufficient asphalt content". Flushing is the "excess asphalt on the surface caused by a poor initial asphalt mix design". Polishing is defined as "a smooth oily surface caused by traffic wearing off sharp edges of aggregates" [67].
The management and maintenance of transportation infrastructures are based on detailed and accurate information about the road network. The pavement type and road surface conditions are the most common variables required to provide detailed road mapping. This data is critical to the management decision process that involves billions of euros of assets, and maintenance budgets of millions of euros each year. Moreover, street maintenance work looks still today like something hurriedly thrown and only based on the roadman job. Often the maintenance is carried out when the pavement is approaching its collapse point and with renewal interventions linked to the worker experience [57].
Remote sensing can solve the road condition mapping by applying relatively cheap methods to evaluate the surface defects of asphalt pavements [25,29,63].
Haas et al. [19] were the first authors that investigated and extracted a pavement condition index (PCI) by connecting the road physical parameters (cracking, rutting and raveling) gathered from field observations with the Global Positioning System (GPS). This common technology provides detailed and geo-referenced information about road condition even though the low cost and easy managing requirements remain unsolved. Furthermore, land-based mobile mapping systems with an extensive set of sensors (including laser reflectometers, ultrasonic sensors, accelerometers, global positioning systems, gyroscopes, video and machine vision systems) and computers, such as Automated Road Analyser (developed by Roadware GRP of Paris, Ontario, Canada), have been commercially available for road mapping application [44,62]. However, the high cost of road network inspection requires the development of innovative remote sensing data analyses that are reasonably priced, easy to manage for the local authorities and valuable for the road condition mapping. For this purpose, many studies have been conducted to discriminate road surface distresses [25] and to analyze the spectral features of urban materials and their separability [4,20,[22][23][24]51,56,58].
Much progress has been made in understanding the nominal range of wavelengths suitable for detecting a variety of minerals and the relationship between spectral absorption feature intensity and mineral abundance using remote sensed data [27,32,34,40,41]. Among others, the TIR spectral region is becoming increasingly more important, thus enhancing the use of multi-channel remote sensing TIR instruments to discriminate geologic surface materials including carbonates, sulfates, clays, and felsic vs. mafic silicate minerals [3,14,49].
In this framework, the research will be focused on developing, implementing and validating the effectiveness of emission spectroscopy, in the TIR spectral range from 8.18μm to 12.70μm, to provide a rapid assessment of the asphalt surface distress. Fast and non destructive methods, such as emission spectroscopy, offer potentially useful alternatives to time-consuming chemical methods of asphalt analysis. The characteristics of asphalt pavement emissivity spectra are controlled by mineral composition, water (hydration, hygroscopic, and free pore water) and particle size distribution.
In particular, the airborne MIVIS sensor, with its high spectral and/or spatial resolution [5], allows reliable quantitative measurements of specific absorption features of urban materials [1]. Moreover, a spatial resolution at least of 5m is optimum for urban applications [59,69], since "the spectral mixing space" becomes more complex with larger pixels [58,59]. Therefore, MIVIS sensor was used in order to retrieve a threshold to individuate those asphalt pavements to be checked for maintenance as it records emitted radiation in the TIR range, using a total of 10 bands with 2mrad of IFOV (Instantaneous Field Of View).
Since the asphalt pavements aging can be related to the loss of oily components [60] and to the sealing tar surface [25], and the decrease of oily components leads to an increase of several types of limestone deposits that are identifiable in the TIR range [36], a simple and fast method was developed in order to define a threshold on the basis of the band depth analysis [9] at 11.2μm (i.e., the limestone absorption peak in the TIR range).
For this purpose, an airborne MIVIS imagery covering a test area close to Venice city (Italy) was used for identifying in the TIR spectrum region the diagnostic asphalt emissivity features.

Study area
The study area ( Figure 1) is characterized by a mixture of urban land cover types and surface materials, including many asphalted roads and, in particular, two main highways with asphalt pavements of different ages and conditions (i.e., more or less weathered and corroded). The study area corresponds to a MIVIS (Table 1)

Image preprocessing
MIVIS data preprocessing was performed as follows: (i) radiometric calibration of the raw data; (ii) atmospheric correction of the Thermal Infra-Red (TIR) data [18]; (iii) calibration to apparent emissivity by separating temperature and emissivity according to the methods described by [16,17].
The radiometric calibration of the airborne MIVIS TIR raw data was performed using a two-point calibration technique that is based on the linearity of the detector response over the dynamic range of the instrument [47]. To this goal, the maximum and minimum reference values of the radiance were acquired at the beginning and end of each scansion line to satisfy the calibration accuracy requirements.
The retrieved pixel spectral radiance (nW cm -2 sr -1 nm -1 ) ( pixel R λ ), expressed by a linear equation obtained jointing the two reference points (i.e., the maximum and minimum reference values expressed by the radiance value and the corresponding Digital Number), is shown in Equation 1.

( )
Where, the pixel DN λ is the pixel raw data to be radiometrically calibrated; the CBB DN λ and HBB DN λ are the spectral raw data measured for the cold (minimum reference value) and hot (maximum reference value) blackbody, respectively; CBB B λ and HBB B λ are the spectral radiance values for each blackbody as predicted by Planck's law with emissivity equal to one (ε λ =1) and known temperature (T i ).
As regards the atmospheric attenuation of the TIR spectral radiance that includes atmospheric transmission and upwelling atmospheric radiance, the ISAC (In-Scene Atmospheric Compensation) algorithm [31,36,64,70] was employed for MIVIS TIR atmospheric correction. This algorithm assumes two pixels of the scene to be blackbodies on which neither locations nor temperatures are known. The ISAC algorithm is suitable also when the atmospheric radiative conditions are not available during the acquisition time. For this study, we followed the "most hits" method as described by Johnson [31] and pixels whose emissivity was equal to 1 at the wavelength were used as a marker.
Once the image was atmospherically corrected, we used the method developed by Johnson [31], and revised by Hook et al. [27] and Kahle et al. [34] to retrieve apparent emissivities. The method is based on the Planck's law for gray body radiator (ε λ ≠ 1). The standard formulation of this law for the spectral radiance (L λ ) of each pixel (i) is described by the following equation: where ε λ,i is the surface spectral emissivity of pixel (i); B λ (T i ) is the blackbody spectral radiance at related to the emission of the atmosphere itself that reaches directly the sensor or is reflected by the surface before being acquired by the sensor and they are also assumed to be independent from the view angle (i.e., pixel location) [36,64]. In order to extract pixel emissivity from the pixel radiance, the requirement is to discriminate from Equation (2) the emission that depends on the kinetic T and other atmospheric parameters as well as the ε value for each image band [16]. This multivariable problem is solved by using the assumption of ISAC algorithm about the linear relationship between the observed radiance and the Planck's function, whose slope is related to the atmospheric transmission (τ λ ), and whose offset is the upwelling atmospheric radiance ( λ ATM L ), at that wavelength: The emissivity can be calculated from this linear equation, in terms of the other variables: Substituting in Eq. (4) the Eq.
(2) and solving for T, then gives As the goal of this study is the assessment of the spectral feature shapes and the band depth analysis, we need to retrieve relative emissivities. In this context, several methods have been proposed in literature for deriving emissivities such as the reference channel method [32], the emissivity normalization method [17], the temperature-independent spectral indices [3], the thermal log residuals and alpha residuals [26], and the spectral emissivity ratios [68]. Several of these methods are compared and reviewed by Gillespie [15][16][17], by Hook et al. [26,27] and by Li et al. [45].
In this study we applied the emissivity normalization routine proposed by Realmuto [50], Hook et al. [26], Kealy and Hook [35] and Gu et al. [18] and implemented in the ENVI 4.4. [41] image processing software. This routine, first, derives the brightness temperature of each pixel from the pixel radiance. Afterward, the apparent emissivity image is obtained by normalizing the radiance of each pixel to the Planck's curve that is generated from the pixel with the maximum brightness temperature with an emissivity value set to 0.96 (i.e., a reasonable hypothesis for exposed mineral surfaces).

Image classification
To develop a method for automated image analysis of road asphalt pavements on the basis of their emissivity spectral features, (a) an object-oriented approach and (b) a band-depth analysis were used (Figure 2).

Object-oriented approach
An object-oriented approach was first adopted in order to individuate and select on MIVIS imagery the asphalt roads [6,43], which was also used as an input mask later on in the band-depth analysis.
Object-oriented approaches can represent a valuable alternative to the conventional pixel-based classification methods, as they consider the spatial context. Segmentation methods divide a study area into adjoining clusters of pixels, called segments or regions, based on similarity or dissimilarity of their single or multiple-layer pixel values [61].
The segmentation approach allows for: (a) quantifying the spatial heterogeneity within the data at various scale levels; (b) delineating homogeneous patches also involving a certain spatial generalization; (c) implementing an explicit hierarchal structure between segments at different spatial scales. As a result, spatial information is very important in classification processes to produce reliable maps [46].
For this study, according to Jensen [30], we used an object-oriented approach with a segmentation procedure followed by classification as implemented in the Feature Extraction module of the ENVI 4.4 software package [42]. In more detail, the procedure consists of a combined process of segmenting the image into regions of pixels, computing attributes for each region to create objects, and last classifying the objects. In order to identify only road asphalt pavements with the requirement of a sufficient neighboring number of pixels showing a homogeneous asphalt mixture, we chose only highways and exits asphalt pavements for the further band-depth analysis. For this purpose, a workflow consisting of two main tasks was adopted.
(i) The "find objects" task (i.e. segmentation; [30]) that was divided, in its turn, into four steps: "segment", "merge", "refine", and "compute attributes". The "segment" and "merge" steps of this task were used to divide the image into segments corresponding to real-world objects and for solving oversegmentation problems and then the adjacent segments were grouped on the basis of their brightness value.
(ii) The "rule-based classification" task (i.e. classification; [30]) was used to extract only the highways and exits objects and then to export them onto a raster image.
In this final step, the rules criteria with the relative appropriate scale factors (i.e., weights) were identified using both spectral and spatial attributes for classifying the highways/exits asphalt pavements objects.

Band Depth analysis on asphalt roads
In Italy road paving asphalts are made of a mixture of mineral aggregate (mainly of silicates and limestone granules) and bitumen [2]. Asphalt pavements surface defects [67] are strictly related to the decrease of the oily components of the bitumen, thus increasing the surface abundance of the limestone granules (see Figure 3). Therefore, for this study we chose the outcrop of the limestone granules, which certainly highlights distressed paving asphalts, as an indicator for those asphalt pavements to be checked for maintenance. The applied procedure consists, first, of the analysis of the available John Hopkins University (JHU) spectral library [55] information allowed for retrieving the emissivity spectral features (8-13μm TIR range) of the asphalt paving material. An absorption band can be described by characteristics, such as the position, depth, width and asymmetry [8][9][10]68]. The presence of an absorption feature and its position in the reflectance/emissivity spectrum provides valuable information about the chemical composition of a material [21]. Figure 4 depicts the JHU emissivity spectra convolved to MIVIS bandpasses in order to show how their occurrence would affect MIVIS detectability of the major limestone absorption feature. Looking at Figure 4a, it is evident that the main difference between the emissivity spectral feature, in the 8-13μm range, of new and old pavement asphalts is the spectral contrast centered at 11.2μm. Moreover, the study of Kirkland et al. [36] confirms that the 11.2μm is the diagnostic emissivity band for the limestone (Figure 4b). In view of this information, a methodology was developed to map by airborne remote sensing the asphalt roads showing surface defects on the basis of the diagnostic 11.2μm limestone emissivity peak.
Therefore, once selected the highway/exits asphalt roads by means of the object-oriented approach, a Band-Depth (BD) analysis was performed on MIVIS emissivity data to measure the spectral contrast of the limestone peak at 11.2μm that is centered, if convolved to MIVIS bandpasses, at 10.93μm (Figure 4b).
The BD analysis primarily requires application of the continuum-removal method that consists of: (a) "fitting a straight line hull to represent the reflectance or emissivity background using two continuum tie points on either side of the absorption feature" [8,39] and (b) dividing the spectrum by this fitted continuum line. Thus, continuum-removed absorption features can be directly overlapped to one another by scaling them to the same depth at the band centre allowing a comparison of the shapes of their absorption features.
The absorption band depth (D) was calculated from: where Rc is the emissivity of the continuum at the band centre and Rb is the emissivity at the band centre (Figure 4b).

Object-oriented classification results
In this application, the highways/exits asphalt roads feature classification, shown in Figure 5, was obtained by assigning to the band ratio criterion (concerning the MIVIS emissivity channels centered at 10.93 and 11.94μm) a weight of 0.5, whereas the spatial criterion received the remaining weight of 0.5 (length 0.4, form factor 0.2 and elongation 0.4). These optimal segmentation parameters were determined using a systematic trial and error approach validated by the visual inspection of the quality of the output image objects, i.e., how well the asphalt pavements matched feature boundaries in the image [46]. Moreover, on the obtained classification image some residual errors were manually corrected on the image in order to achieve a mask of only asphalted highways and exits ( Figure 5).
This object-oriented approach used for the extraction of asphalt roads is very cost-effective, because it reduces the necessity for laborious on-screen digitizing that is by far the most expensive task of the standard photo-interpretation process.

Application requirements and Band-Depth results
In order to be confident that MIVIS instrument technical characteristics allow to recognize the peculiar asphalt absorption features at 11.2μm (i.e., the outcropping limestone granules) and to apply BD analysis, a Band Detection Limit (BDL) was first calculated according to the Kirkland et al. [36,38] method.
BDL is the percentage of the absorption features, in our case the 11.2μm limestone peak, to be detected with the desired confidence level [38]. The resulting value depends on the BDL related to the width and depth of the spectral contrast exhibited by the target. The BDL is calculated using: where BDL = the minimum band depth required for detection of a given band width and center; Confidence Factor (CF) = the contrast relative to the SNR level that a feature should exhibiting to be distinguished from background; Band FWHM = the full-width target band at the half maximum of the band depth; Sampling Interval = the instrumental spectral sampling interval.
According to Kirkland et al. [38], the BDL value depends on the instrument signal-to-noise ratio (SNR), instrument spectral resolution, target spectral band depth, band width, desired signal level above the noise (CF) and atmospheric compensation. Lower numbers for the BDL indicate that lower spectral contrast is required for detection. The CF influences the BDL such that a higher CF requires greater band contrast for acceptance (i.e., a CF = 1 represents a signal level comparable with noise).
The BDL values were calculated by assuming that the target surface covered the instrument's field of view, and no atmospheric attenuation influenced the data. It is clear, however, that even though the noise levels for a given sensor are generally fixed, for remote sensing data application, the signal portion of the SNR is affected by other external factors such as view angle, atmospheric attenuation and scattering and surface emissivity, which can modify the sensor perceiving signal [11].
Since the sensor SNR [13] cannot be modeled without specific knowledge of the instrument characteristics, the following calculations were made using MIVIS sensor characteristics to establish its capability to retrieve spectral absorption features in the TIR region.
The MIVIS sensor DN values, acquired in-flight on the internal blackbodies, were converted to emissivity following the same procedure used in § 3.1 [26,35], except for atmospheric correction. Atmospheric correction of the blackbody data was not taken into account, because the distance between the sensors and the blackbodies was negligible. The SNR of the TIR bands was, consequently, estimated by dividing the emissivity mean spectra of all masked asphalt pavement pixels (i.e., signal) by the standard deviation of blackbody emissivity (i.e., noise). Figure 6 shows MIVIS SNR calculated for the TIR spectral range on the masked asphalt pavements of the study area. Regarding the spectra recorded by the MIVIS scanner on the masked asphalt pavements, to have a spectral feature that can be detected with the desired CF, the asphalt material has to depict a spectral contrast greater than the MIVIS scanner BDL. Therefore, a BDL value of 0.0012 was obtained by using, in Equation 7, MIVIS instrument characteristics within the desired TIR interval (as shown in Figure 4b), i.e., a FWHM of 0.4 μm, a band sampling interval at 10.93 μm of 0.2μm, a SNR at 10.93μm above 1100 and with a CF of 2.
The 0.0012 BDL value is the minimum limestone fractional exposure on the asphalt material required for being detected by MIVIS instrument characteristics.
Once the BDL analysis confirms that MIVIS characteristics allows for recognizing the limestone diagnostic emissivity peak, the MIVIS emissivity data of the masked highways/exits pavements were analyzed by means of the band-depth procedure to establish a suitable threshold level for discriminating those asphalt pavements to be checked for maintenance intervene.
For this purpose, an extensive field survey was carried out on all masked asphalt roads to visually check the asphalt pavements conditions. Figure 7a shows the two selected areas with certainly surface defects on which retrieving the BD threshold level that identifies those asphalt pavements (i.e., pixels) where to check for an asphalt maintenance intervention. On the two selected test areas the distribution function of the BD (at 10.93μm) analysis was calculated. Next, a threshold, which is based on the BD values distribution and that allows for discriminating in both test areas distressed asphalt pavements pixels (visually checked by field surveys), was assessed by using: μ -σ (8) where, μ is the mean value of the test areas BD distribution and σ is the corresponding standard deviation. As a consequence, the BD value of 0.020 is the identified threshold for determining on all masked pixels (i.e., the highways/exits) the asphalt pavements to be checked for maintenance work. Figure 7b shows in red the distressed asphalt pixels individuated using the above mentioned threshold of 0.020. For example, in Figure 8, asphalt pavements of a highway and the relative exits, different from the two chosen as test areas, are shown as a particular case as they demonstrate the ability of the proposed procedure in detecting the different surface defects of the same asphalt. All the areas classified as distressed (depicted in red in Figure 7b) were further visually checked by means of a field survey carried out on the accessible roads mostly for safety reasons. Based on the visual field accuracy assessment, the asphalts to be checked for maintenance can be properly identified in all the masked study area with accuracy ranging from 75 to 90%, corresponding respectively to a total surface area of about 70,200 m 2 and 84,240 m 2 (total area is 93,600m 2 ).
In particular, both MIVIS classification and field surveys allowed observing that surface defects were greater for the highways exit asphalts than for the highways' ones.

Conclusions
The preliminary results of this study demonstrate the utility of airborne remote sensing for identifying asphalt roads and discriminating those on which checking for a maintenance work. The potentiality of assessing the surface defects of asphalt pavements by using airborne Thermal Infra-Red data was established. This method can be surely applied to roads (e.g. highways, secondary roads) crossing rural environments and in large parking areas and wide streets within urban centres, whereas in case of densely settled urban areas it is difficult to successfully exploit this procedure. In fact, the presence of heat islands and other major drawbacks (e.g., brightness effects) would not allow obtaining suitable results.
The integration of object-oriented classification (i.e., segmentation followed by classification) applied to MIVIS multispectral TIR data permitted the identification of all the road asphalt pavements with a very cost-effective and accurate procedure.
For the surface defects assessment of the identified asphalt roads we chose the surfacing limestone granules occurrence as a suitable indicator. Therefore, on the primary emissivity absorption feature of the limestone at 11.2μm a band-depth analysis of the continuum-removed absorption feature was performed on a MIVIS airborne TIR imagery. The detection limit analysis showed that MIVIS technical characteristics in the TIR range allow discriminating this spectral feature. However, a more accurate discrimination of the limestone diagnostic absorption feature can be achieved using more advanced hyperspectral thermal instruments, such as AHI and SEBASS airborne sensors.
In conclusion, the proposed combination of segmentation procedure and emissivity shape-based analysis used in this preliminary study allows for a rapid discrimination (i.e., a suitable threshold) of the location of distressed asphalt pavements. Therefore, the encouraging results may let public institutions and private companies to adopt this procedure, applied to airborne remote sensing data, to rapidly control and monitoring the surface defects of the road asphalt pavements. Such large-scale monitoring can be also included in the more general framework of critical and civil infrastructures management and protection.
Future application of the proposed procedure will include field spectral analyses, using a portable micro-Fourier transform infrared radiometer: (a) to construct a spectral library of asphalt pavements with different surface defects useful for calibrating and validating further thermal remote sensing data acquired on this study area and on other test sites; and (b) to provide asphalt samples for laboratory mineral abundance analysis and for determining their deterioration/aging level.