1. Introduction
Change detection analyzes a pair of images of the same subject acquired at different times to detect eventual changes occurred between the two data collections. In remote sensing applications, one deals with the same geographical area trying to perform, for example, environmental monitoring [
1,
2,
3], study on land use/land cover dynamics [
4,
5], agricultural surveys [
6], analysis of forest or vegetation changes [
7,
8,
9], damage assessment [
10,
11] or analysis of urban changes [
12,
13,
14].
In most of the literature of common knowledge, the comparison between two or more images is performed on data acquired by exactly the same sensor [
2,
9,
14] or by sensors of the same type [
13,
15] but at present
multi-sensor image change detection, using for instance optical and synthetic aperture radar (SAR) images [
5], has also become a concrete possibility, at least in terms of data availability, due to the increasing number of operational airborne and spaceborne sensors. Hence, there is the need for the development of technical “tools” to exploit such data in a combined way taking advantage of their characteristics and complementarity. A relevant example of the importance of multi-sensor methodologies is the following: in the presence of a natural disaster for which rapid mapping of the damages is needed, it may be given that optical data from an archive are available for the “before” scenario, but only SAR data are available for the “after” scenario due to adverse atmospheric conditions.
We present here a novel approach for change detection based on the use of
similarity measures. Such measures have been applied until now only to image coregistration [
16,
17,
18,
19] (and mainly in the field of medical imagery). In particular, one of their principal properties is their capability to operate in the multi-sensor case. Our idea is then to profit from this property and to use the correspondence between the same points in the two images not to correct the relative displacement but, already given their precise coregistration, to detect eventual changes occurred between the data acquisitions.
Another reason of interest for similarity measures is that basically no example of their use for change detection has been yet reported in the literature. The works published in this field describe methods that can be finally divided into two main groups:
those that operate a preliminary feature extraction or classification of the images and then search for transitions of the pixels from one feature to another (hence, permitting a boolean comparison and a direct yes/no response for the change/no-change definition);
methods that estimate the difference of the radiometric values of the image pixels (via a straightforward subtraction or using ratios, also in logarithmic form, as is common practise for SAR images) and then establish if a change occurred based on thresholding criteria. Similarity measures belong to this second group.
However, even the very extensive review of change detection techniques by Lu
et al. [
15] does not mention similarity measures as a possible way to perform this task nor cites any reference to them. Thus, a summary of the theory on similarity measures, their systematic investigation and a comparative analysis of their performance represent, in our opinion, a useful scientific contribution. Given these motivations, optical and SAR images acquired at different times were coregistered and a series of similarity images were derived and used as indicators of the changes. The experiments were carried out on two data sets relative to the test sites of Toulouse, France, and Oberpfaffenhofen, Germany, and both qualitative and quantitative analyses were conducted. Open questions and suggestions for further investigations are mentioned along with our observations.
In
Section 2 and
Section 3 we begin by reviewing the theoretical background of the similarity measures objective of this study and, after describing the experimental procedure in
Section 4, we report the results obtained for the two test cases: those based on the Toulouse data are discussed in
Section 5 and those relative to the Oberpfaffenhofen data in
Section 6. Final considerations and comments are provided in
Section 7.
2. Similarity Measures
The leading principle of similarity measures is stated in [
19] and may be summarized as the consideration that, although two images of the same scene acquired by different sensors can be characterized by completely different radiometric properties, a common “basis” is shared by the two data sets since they are different representations of the same reality. The key question is then to correctly retrieve this correspondence.
In registration procedures [
20], where the goal is to find a spatial transformation relating two images, a common way to proceed is to extract features from each image with a segmentation step and minimize the distance criterion between these features. More recent approaches are based directly on the intensity values of the image pixels and do not need the preliminary feature extraction. Since image coregistration and change detection have many aspects in common, the following considerations will be valid for both topics.
By definition, given two images
I and
J, a similarity measure is any scalar strictly positive function
that quantifies how similar are the images according to the criterion
c.
f has an absolute maximum when
I and
J are identical according to
c [
19]. The selection of the similarity criterion, and hence the definition of the function
f, can vary according to the type of images under analysis, the application (e.g., image registration or change detection) and the parameters used to define it (radiometric values, features characteristics, etc.).
In general,
I and
J are mappings of the type [
18]:
where
d is the dimension of the image (for instance, in many medical imaging applications one deals with the 3D case) and
is its size along the
k-th axis. We will then denote a general spatial coordinate as
. The domain
is the set of intensities of
I and, typically, for the two images it is:
, referring to the grey levels.
I and
J may be regarded as two random variables, taking their values in
and
, which have marginal probability distributions:
and
, respectively, and joint probability distribution:
.
Based on the probability distributions of the images, several similarity criteria
c can be defined. The necessary
a priori assumption is the following: the two images are linked, thus the link is maximal when there are no differences (due to changes or registration errors) between the two. The link may be evaluated by the notion of
dependence. Intensity distributions are dependent if a variation in one distribution leads to the variation in the other. The case of maximal dependence is given when
I and
J are related by a one-to-one mapping
T such that:
the case of statistical independence when:
Inglada [
21] already discussed the change detection between two data collected by the same optical sensor. The solution proposed there, and common to most of the measures discussed in [
19], was to use the statistics of each pixel’s neighbours. Indeed,
,
and
can be estimated for the subsets of the whole images, then a pixel with a given intensity
i is defined as changed if the probability
calculated with respect to that of the neighbours is not the same as that of the corresponding pixel in the image
J (i.e., if it changes beyond a given threshold). To evaluate this, for each pair of corresponding pixels in the two images
I and
J, an estimation window has to be fixed (see
Figure 1). Then, the marginal and joint probabilities are calculated and used in the definition of several functions
f.
Figure 1.
Calculation of the marginal and joint probabilities.
Figure 1.
Calculation of the marginal and joint probabilities.
More in detail, similarity measures may be distinguished based on only the probability estimations or on the combined use of the probabilities and the radiometric information (its mean value in the estimation window or its variance). In the following, we will report the results obtained using five measures of these two main groups, namely:
Measures using only the probabilities:
Measures combining probabilities and radiometric values:
The approach adopted here to the statistical analysis necessary to calculate the similarity measures is analogous to that of [
21] and [
19]: histograms of the radiometric values for corresponding estimation windows are used to define the probabilities by simple normalization. Alternatively, in [
22] these were derived from the probability distribution functions obtained by a parameterized formula (made explicit by calculating its parameters from the pixels of the estimation windows).
2.1. Distance to Independence
As anticipated, the condition of statistical independence is formalized by Equation (
3). Hence, the difference of the two terms, the product of the marginals and the joint probability, directly measures the degree of independence of the two images:
2.2. Mutual Information
By expressing the same difference in logarithmic form, it is possible to refer to the concept of entropy introduced by Shannon [
23]:
which is a measure of the amount of uncertainty about the random variable
I. The entropy is null when an event (in our case, a given pixel value) is certain, i.e.,
, and has a maximum when all events have the same probability:
, where
n is the number of possible values of
I.
The relationship between the difference of the probabilities and the entropy is given when defining the quantity [
16]:
for which the following equations hold:
is the joint entropy of
I and
J, whereas
and
are the conditional entropy of
I given
J and of
J given
I, respectively. These are defined as:
where
is the conditional probability of
I given
J.
Since measures the uncertainty left in I when knowing J, the mutual information estimates the reduction of the uncertainty of one random variable by the knowledge of the other, or the amount of information that one contains about the other.
2.3. Cluster Reward Algorithm
A further measure based on only the joint and the marginal probabilities is the cluster reward algorithm, which is defined as:
As stated in [
19], the
index has a large value when the joint histogram has little dispersion. This can be the result of a good correlation (histogram distributed along a line) or the clustering of the image intensities within the histogram. In both cases, it is possible to predict the values of one image from those of the other.
An observed advantage of this measure, with respect to the previous ones, is that the joint histogram noise resulted from the estimation has a weaker influence, thus smaller windows may be considered to derive the histogram [
19].
2.4. Woods Criterion
Along with the probabilities, the radiometric values of the image pixels can also be directly used to estimate the correlation between two images. At this scope, it is practical to introduce the conditional mean:
and the conditional variance:
By their means it is then possible to define a measure of the variability of the pixel intensity in one image, given a certain value of the homologous ones in the other [
24]. The assumption is that this variability is larger in the presence of differences (again, changes or misregistration results) between the images. The measure is thus introduced as:
2.5. Correlation Ratio
Finally, the correlation ratio is defined in a similar way using the conditional mean and variance:
but also the variance
of the radiometric values of one of the images.
3. Robust Measures
An assumption necessary to the application of the Woods criterion is the
inter-image uniformity, which means that pixels of corresponding areas in the two images should have proportionally similar radiometric values. For example, in the specific case of [
24], this yields that voxels of 3D medical images representing the same human tissue must have similar intensities within each image. This is a relatively strong requirement that is rarely verified in other cases of medical imagery coregistration. In fact, it may be more often observed that the joint histograms of the images to be coregistered represent a mix of several populations even when considering an homogeneous region in one of the images [
17]. Consequently, the calculation of the averages and variances used in Equation (15) is influenced by the presence of several populations with different distributions.
In order to improve the robustness of the similarity estimation, especially in the case of multi-sensor data, averages and variances have to be calculated in such a way to minimize the effects of outliers altering the distribution of the “main” population. This is possible by using the Geman-McClure estimator, which is defined as:
In Equation (17), x is the grey level residual given by the difference between corresponding pixels in the case of mono-sensor images and by the difference with the conditional mean for multi-sensor data (i.e., in this case, it is implicit in the hypothesis of uniformity of the grey levels in the estimation window) and C is a scale parameter.
Given its shape, the function
permits the reduction of the relevance of the largest distribution deviations. Indeed, it is easy to verify that, as the amplitude of the residual errors increases,
tends to a constant value [
17].
3.1. Robust Woods Criterion
Based on the properties of robust estimators, an alternative formulation of the Woods criterion is possible as described in [
17]. Conditional mean and conditional variance can be calculated in robust form as:
with
and
The parameters
are defined as the median values of the absolute values of the residual errors:
This last expression derives from the fact that the median value of the absolute values of a large number of samples normally distributed and with unitary standard deviation is just equal to
[
17,
25].
The robust definition of the Woods criterion is:
7. Summary
In this work, a new approach for the change detection of remotely sensed data has been presented and the feasibility of multi-sensor images change detection at pixel level was verified; indeed, alternative procedures are also possible [
15], for instance, to operate at feature level after a preliminary segmentation or classification process [
31]. Methods normally used for image coregistration, the similarity measures, have been applied, investigating their performance when the data are provided by different sensors. The general interest is motivated by the need, in the near future, for techniques which permit the exploitation of complementary optical and SAR data from satellites planned to work in a cooperative way like the Pléiades and COSMO-SkyMed ones. A further reason of interest is that these measures express, by their very definition, the degree of similarity between two images and hence are natural candidates to estimate their difference (their “dissimilarity”). Since basically no example of their application has been reported in the literature, we decided to fill this gap and to study them providing also a basic review of their theory.
Our observations do not allow, at this stage, to draw definitive conclusions but suggest a methodology able to cope with a major remote sensing issue, which opens the way to several interesting research perspectives. The presented results are indeed promising and indicate similarity measures as possible tools to detect changes of the Earth surface [
27,
30].
We could see that the considered algorithms perform differently and that they do not offer an “absolute” measure of the changes. In fact, they depend more on the type of the sensor than on the time difference between the data takes. Also the selection of the dimensions of the estimation windows (for the pixel statistics and the similarity measure calculation) affects the results, in particular, when using measures based only on the probabilities. The definition of the optimal dimensions for the estimation window is also an open question to be further investigated. It is also worth noticing that we used a straightforward definition of the neighbourhood of a given pixel (based on square estimation windows) for the statistical analysis and the similarity estimation, and that the accuracy of these steps may be improved using any of the techniques that actively redefine region boundaries. These algorithms present the advantage of effectively identifying homogeneous areas and reducing the smearing of the radiometric values due to the straddling of the fixed window on two or more different regions. Our observations would remain valid and the use of each similarity measure could in this way be refined.
Although not originally intended to accomplish this task, the finding that similarity images can distinguish vegetated areas from man-made structures suggests their application for classification purposes in a similar way as for interferometric coherence SAR data (hence permitting, e.g., forest classification in area with limited availability of clouds-free optical images). However, since the characteristics of natural targets lead to confusion with the response from changing man-made structures, step-wise procedures are suggested to firstly establish the nature of the targets and then their eventual changes. Dedicated studies are presently in progress on both these topics.