1. Introduction
Generalization of building objects has been a topic in recent years because (1) the efficient rendering of building objects requires representation of buildings at different levels of detail (LoDs); and (2) for the visualization of buildings on small display devices such as mobile phones, PDAs, etc., abstractive buildings models have to be generated.
Since Staufenbiel [
1] proposed a rule-based approach for the simplification of 2D building ground plan, a number of algorithms have been made available for generalization of building models. The early works focused on developing techniques for generalizing building ground plans in 2D [
2,
3,
4,
5]. Using these techniques, the amount of detail in the ground plan can be reduced by removing line segments with some criteria, i.e., minimum length of a façade. For instance, Sester [
6] proposed a two-step procedure: (1) removing the minimal forms guides by rules; and (2) adjusting the form of the simplified building to the original form using least-squares adjustment. In this second step, certain characteristics of the buildings can be preserved or even emphasized, e.g., rectangularity and parallelism or size. Other approaches apply methods of pattern recognition to replace the original ground plan with a standard shape [
7].
In the recent years, a number of algorithms have been proposed specifically for the generalization of 3D building models. Lal and Meng [
8] defined some rules and constraints for 3D generalization. However, the generalization is restricted on one operation, namely, aggregation. Kada [
9] extended Sester’s approach [
6]. He developed rules to remove structures that were too small in the 3D polyhedron at first then adjust the simplified building to its original shape.
On the basis of the polyhedron segmentation proposed by Ribelles et al. [
10], Thiemann and his colleagues suggested decomposing a building into basic 3D primitives and eliminating those with small volumes [
11,
12]. Kada [
13,
14] proposed a structurally similar approach. He defined parts of simplified buildings as intersections of half-planes based on which cell decomposition and primitive instancing are applied. More related research can be found in [
15,
16].
The above mentioned studies focus mainly on simplification of buildings by removing smaller wall elements. The features on building façades are seldom concerned, but they also need generalization e.g., windows need to be enlarged so that they are legible in a reduced display space. However, enlargement may cause feature overlaps. Typification may be used to circumvent this problem. On the other hand, with the further shrinkage of display space, buildings have to be generalized in groups. For this case, the operation of typification is preferred when buildings are distributed regularly [
15].
The operator of typification is defined as a process of replacing a large number of objects with a smaller number of uniformly shaped objects while preserving the appropriate characteristics of the pattern. Regarding the approach for typification, in [
17], every four polygons neighboring each other were replaced by a new polygon created by connecting the center points of the four polygons. Regnauld’s typification algorithm [
4] was based on the minimum spanning tree in graph theory. In [
18] the procedure of typification was divided into two steps: positioning and representation, while the positioning step determines the number and the positions of the buildings based on Delaunay triangulation, in the representation step the size and orientation for the replacement will be calculated. Besides, most previous works like [
19,
20,
21,
22] determined the density of objects using Tröpfer’s radical law [
23]. Furthermore, this kind of operation was used in many literatures for generalization and various results were presented in [
5,
11,
14,
24]. However, it is not discussed why their results are reasonable. Meanwhile, there exists a similar problem in the field of computer science. For layout management of Graphical User Interface (GUIs) in graphics, typification is used to rearrange buttons (or icons) with the changing size of GUIs for different devices or scales. A number of algorithms are currently available for this issue; for instance, Luyten et al. [
25] described a method to combine abstract User Interface (UI) descriptions and constraint-based layout management system for different devices. Unfortunately, there is no investigation of finding a reasonable result of typfication either.
In our work a user survey has been conducted to find out what kind of representation after typification is best associated to the original dataset with respect to the human visual perception. In order to verify the results of the user survey, Attributed Relational Graphs (ARGs) are generated from the original dataset and the candidate representation after typification. Then the Nested Structure of Earth Mover’s Distance (NEMD) [
26] between these ARGs is calculated. The NEMD values denote the similarities between the typified façades and the original façades and can be regarded as a quantitative measure to guide the automatic approach of typification.
The rest of the paper is structured as follows: the user survey for typification is presented in
Section 2 at first. In
Section 3, the ARG and NEMD algorithms are described in order to verify the results of the user survey. The automatic approach of typification based on the results of our user survey is elicited in
Section 4 with the experimental results shown in
Section 5. Finally, the conclusion and work to be done in the future are given in
Section 6.
3. Verification of the User Survey Using ARG and NEMD Algorithm
To quantify the visual similarity between façades, pattern recognition methods are employed. Attributed Relational Graph (ARG) has been widely used to represent objects or structures to be recognized in computer vision and pattern recognition [
27]. In this paper, ARG is used to represent windows of the façade. Therefore the visual similarity evaluation between the typified façade and the original one can be quantified by matching their ARGs. Lots of algorithms are proposed for ARG matching such as the graduated assignment graph matching (GAGM) [
28], and least squares graph matching (LSGM) [
29]. The Nested Earth Mover’s Distance (NEMD) algorithm is chosen since it shows a better performance in ARG matching according to the results in [
26].
An ARG G is defined as G = {V,R} in which V = {vi|1 ≤ I ≤ n} and R = {rij|1 ≤ I ≤ n, 1 ≤ j ≤ n}. V is the set of n nodes and each vi represents a window in the façade. R is an matrix and each rij is the relationship between the window vi and vj. In this application the node contains attributes about the window such as width and height and the relationship between nodes will represent the spatial and topological relations between the windows on the façade.
The ARG matching can be implemented as a two-step procedure, constructing a distance matrix and establishing the correspondence based on the distance matrix. More specifically, NEMD consists of inner EMD (Earth Mover’s Distance) and outer EMD. The inner EMD reflects the difference between corresponding nodes from two ARGs. The outer EMD is composed of inner EMD distances of all node pairs, and we can establish the correspondence between nodes in the two ARGs and get the distance of the two ARGs by selecting and adding the minimum element in each column or row of outer EMD. The details about NEMD calculation is given in [
26,
30].
A simplified example of NEMD calculation of 2 ARGs shown in
Figure 4 is given by Kim et al. 2004. The difference between the example in
Figure 4 and the method proposed for generalization evaluation is the definition of distance between nodes and relationships. In order to simplify the calculation, the node feature only contains one figure and the distance between the nodes is the difference value of their figures. It is the same for the relationships in
Figure 4. For quality assessment, we just replace the distance functions Δ
node and Δ
relation in Equation (1) with the functions defined in
Section 3.2.
In
Figure 4,
G’ is the sub-graph of
G with nodes 1, 2, 3 in
G’ corresponding to nodes 4, 2, 1 in
G. The distance between nodes and distance between the relationships are the differences of their values given in
Figure 4.
First, the inner EMD between every pair of nodes in
G and
G’ is calculated from inner matrix D
inner, in which every element is generated from Equation (1). For example, the D
inner of node
v1 in
G and
v’1 in
G’ (
vi and
v’i’) is given in Equation (3), in which the 1st row and 2nd column of inner matrix
Dinner, Δ
inner(1, 2) can be calculated with Equation (2).
In which, I = 1, I’ = 1, j = 1, j’ = 2, Δnode(v1, v’2) = |0.8 − 0.3| = 0.5, Δrelation(r11,r’12) = |0 − 0.2| = 0.2, α = 0.5. Therefore, Δinner(1, 2) = 0.35. Similarly, we can calculate all Δinner(j, j’) and compose the Dinner for the node pair v1 and v’1 as shown in Equation (3). Based on that, the inner EMD of node v1 in G and v’1 in G’ is 0.1 + 0.3 + 0.35 = 0.75 (the minimum sum of minimum value in each column or row).
The D
outer of G and G’ is also given in Equation (3), in which the first element is 0.75 according to previous calculation. The EMD between G and G’ is 0 based on the D
outer, because G’ is the sub graph of G. But in our application, not only partial but also overall difference between ARGs should be considered. Therefore, the difference between two ARGs is the maximum sum of the minimum value in each column or row of the D
outer, e.g., 0.05 for G and G’ in
Figure 4.
This rest of the section focuses on creating the ARG of façade and constructing the distance matrix of nodes and relations.
3.1. ARG Generation
Since windows are all rectangles in our test façade data, the width and height of the window are saved as attributes of the node. vi = (wi, hi) in which is vi represent the i-th window in the façade; wi and hi are the width and height of the window. The relationship between two windows is set to the ratio of the distance between two polygons of the windows and their area sum. However, absolute distance alone is not sufficient to reflect the visual relationship between two windows on the façade since two large windows would look more similar to each other than two smaller ones even if they have same distance.
3.2. Distance Definition
Three types of distance are required by NEMD, and they are (1) distance between nodes; (2) distance between relationships; and (3) distance combining the previous two types. Node distance represents the difference between the shapes of each window pair; relationship distance represents the spatial distribution and topological difference of the window group; the combined distance is the weighted sum of node distance and relationship distance. All these distance values are normalized from 0 to 1in which 0 represent exactly the same and 1 represent completely different.
Node distance Δ
node is composed by two parts: shape distance Δ
shape and area distance Δ
area. Assuming
vi and
vj are two nodes, Δ
node can be calculated as follows:
As shown in
Figure 5, Δ
shape equals the sum area of
D1 and
D2 (the shaded part in
Figure 5c). Since rectangle
P1 and
P2 are normalized to the rectangle with 1 as their longest edge, Δ
shape is a value between 0 (means exactly the same) and 1 (completely different). In Equation (8),
and
are the total width and length of the façade which contains
, so are
and
. Δ
area is the normalized area difference.
α is a number between 0 and 1 which gives the weight of the shape and area distance in final node distance.
Relationship distance
is the normalized difference between two relative distances. Assume
is the relationship between two polygons
and
, so is
. Then the relationship distance between
and
can be calculated as follows:
In Equation (11),
is the area of the polygon. Δ
ij and Δ
pq indicate respectively the node distance between
and
and the node distance between
vp and
vq. The combined distance is the same as Equation (1),
, where α is a number between 0 and 1 and gives the weight of the node and relationship distance. In our implementation, α is set to 0.5 in Equation (10) because the importance of shape and area are considered to be the same. For the combined distance in Equation (1), α is set to 10/17 according to the value in
Table 2, in which the importance value is 10 for the windows and 7 for the relationship between windows. Therefore, the weight for node is set to 10/(10 + 7) and weight for relationship is set to 7/(10 + 7). If there is not apredefined weight, the default weights for the NEMD calculation are identical in the process, e.g., α = 0.5 in Equation (10). Otherwise, the weights are generated to reflect the rational, e.g., α = 10/17 in Equation (1).
3.3. Similarity Values
The NEMD values between the original façade and the typified ones are calculated based on the distance definitions in
Section 3.2. We will verify the NEMD values by the user survey results.
Table 3 demonstrates the NEMD values calculated for six options of typification of the test ground plans in comparison to the results of the user survey.
The NEMD values in
Table 3 denote the dissimilarities of the six options to the original distribution, while the values of the user survey present similarities. In order to make these two sets of values comparable, the NEMD values are linearly transformed and the values of the user survey are transformed into dissimilarities. Assume that an NEMD value is a and a user survey value is b. Let a’ = a/5 − 7.5 and b’ = 10 − b. This linear transformation is used to map the NEMD and user survey value into the same range and to illustrate their correlation. a’ and b’ of all six options are given in
Figure 6, from which it is shown that the proposed NEMD method can correctly reflect the user’s visual perception of similarity in our case. In
Figure 6, the lower NEMD indicates a better association between the original and typified façade, while the user survey value (b) is converted into (10-b) that means the higher user survey (b) suggests the better association.
5. Experiments and Evaluation
The algorithm for automatic typification has been implemented using Matlab (version Matlab 7.4) on a number of façades. A selection of examples is presented in this section.
Figure 9 shows the result of typification for a façade in which all the windows are distributed regularly in rows and columns.
Figure 9b denotes the new distribution of windows after typification for a scale reduction by two times.
In order to evaluate the proposed approach, two alternative results for the typification of
Figure 9a are generated manually (
Figure 9c,d). Then the similarities 9b, 9c and 9d to the original distribution (
Figure 9a) are calculated using the algorithm presented in
Section 3.2. The NEMD values of
Figure 9b, 9c and 9d to 9a are 42.41, 75.82, and 76.83 respectively. It clearly denotes that the result of our approach is better than other typification solutions.
Another type of quite common façade with windows might be when the windows are distributed evenly in one direction, but not in the other, although they are well aligned. An example is shown in
Figure 10a. In this case the whole façade had to be partitioned into several segments at first by comparing the sizes of windows and the distances among windows. Then the process of typification is carried out for each segment. At the same time the process should consider the results of typification of neighboring segments. In other words, the windows after the typification should reflect the original distribution character. In our implementation, we first calculate the distribution of windows for the segment in which there are more windows than in other segments (in case of
Figure 10a, the middle segment is typified at first). Then the number of windows in the column is determined for all the other segments. That means that the parameter
is treated as known during the process of typification for other segments.
Figure 10b presents the result of typification for the original distribution of
Figure 10a with consideration of context between segments.
Figure 11a shows a façade with well-aligned but irregularly distributed windows. Similar to the façade in
Figure 10, the windows had to be partitioned into three segments and typified segment after segment (
Figure 11b).
6. Conclusions and Further Works
This paper presented a user survey to find out which kind of representation after typification is visually best associated with the original dataset. The results of the user test revealed that “the ratio between height and width of the windows” and “the distribution pattern of window elements” are the most important clues for preserving the graphic characteristics of façades.
In order to verify the results of the user survey, ARG and NEMD algorithms are introduced to quantify the similarity between the original façade and the typified ones. The similarity values coincide very well with the results of our user survey.
Based on the user test, which reveals different significance values of constraints, an automatic approach for typification is developed which can iteratively satisfy the given constraints. The algorithm has been implemented and tested on a number of façades. Experiments show that the results of typification can effectively reflect the distribution character of the original façades.
The main contribution of this work is that the approach can, for the first time, quantitatively typify regular distributed polygonal objects on building facades. Because the method of the proposed typification fully considers the factors and their weights for the consistency of visual impression while reducing the number of façade objects and repositioning them, the results generated by the proposed approach can preserve the graphic characteristics of the façade as much as possible. This can be applied to simplify façade objects in the process of 3D generalization. In a broad sense, the method proposed in this paper can be used for the typification of polygonal objects (i.e., building footprints or urban blocks) on a 2D map in case the objects form patterns which fulfill the condition of the typification.
However, our approach is only used to typify façades with rectangular windows. For windows with complicated structures, more parameters have to be introduced. In the near future, a contextual typification will be investigated, which means that the typification for windows on a façade should consider the distributions of windows on its neighboring façades. Moreover, the developed approach will be tested on extensive ground plans of densely distributed buildings. The algorithm for automatic typification has been implemented using Matlab (version Matlab 7.4) on a number of façades. A selection of examples is presented in this section.
Figure 9 shows the result of typification for a façade in which all the windows are distributed regularly in rows and columns.
Figure 9b denotes the new distribution of windows after typification for a scale reduction by two times.