A Hybrid Tool for Visual Pollution Assessment in Urban Environments

: With increasing focus on more nuanced aspects of quality of life, the phenomenon of urban visual pollution has been progressively gaining attention from researchers and policy makers, especially in the developed world. However, the subjectivity and complexity of assessing visual pollution in urban settings remain a challenge, especially given the lack of robust and reliable methods for quantiﬁcation of visual pollution. This paper presents a novel systematic approach for the development of a robust Visual Pollution Assessment (VPA) tool. A key feature of our methodology is explicit and systematic incorporation of expert and public opinion for listing and ranking Visual Pollution Objects (VPOs). Moreover, our methodology deploys established empirical complex decision-making techniques to address the challenge of subjectivity in weighting the impact of individual VPOs. The resultant VPA tool uses close-ended options to capture the presence and characteristics of various VPOs on a given node. Based on these inputs, it calculates a point based visual pollution scorecard for the observation point. The performance of the VPA tool has been extensively tested and veriﬁed at various locations in Pakistan. To the best of our knowledge, this is the ﬁrst such tool, both in terms of quantitative robustness and broad coverage of VPOs. Our VPA tool will help regulators in assessing and charting visual pollution in a consistent and objective manner. It will also help policy makers by providing an empirical basis for gathering evidence; hence facilitating evidence-based and evidence-driven policy strategies, which are likely to have signiﬁcant impact, especially in the developing countries.


Introduction to Visual Pollution
Recent decades have witnessed an exponential growth in urbanization demands, especially for improved governance in response to an ever-increasing urban complexity in a global and multi-level context [1].In the developing countries, economic development, urban growth and dynamic economic transformation are accompanied by many other phenomena [2].Usually, amenities are provided to the citizens without a well-managed plan of service provision; this results in the visual deterioration of Sustainability 2019, 11, 2211; doi:10.3390/su11082211www.mdpi.com/journal/sustainability the urban fabric.For example, marketing agencies install various kinds of outdoor advertisements particularly hoardings and billboards, which in the absence of strict enforcement policies, often emerge as eye-sores and visual blights [3].Similarly, provision of other services without strong management result in solid waste dumps, cluttered hanging communication wires, encroachments, slums, bad road infrastructures, dilapidation of historical buildings, etc. Resultantly, urban life has come across as a disaster of aesthetic deterioration, transportation issues, air pollution, fumes, smog, and land and water pollution.However, in contrast to other types of pollutions, visual pollution has remained the most neglected aspect of urban life, both in policy and enforcement.
From the definitional evolution aspect, the term 'visual pollution' has been primarily recognized and researched in the mid of 20th century.Although earlier references to air and water pollution can still be found in the literature [4].During the 19th century, recognition of types of pollution started from the laws on air and water pollution due to their harmful influences on the human environment.The era of the 1970s yielded a number of laws and treaties on 'noise pollution' that denotes the beginning of accepting other types of pollution.During the same period, the visual and light pollution was acknowledged in general, particularly in legal documents, and their management was mandated to local authorities, state agencies, and municipalities [4].
Parallel to that, since the 1960s, visual pollution has been discussed in the developed part of the world resulting in the emergence of different acts, rules and policies for the protection, preservation and enhancement of the urban visual environment [5].A school of thought defined visual pollution as the type of pollution which offends human vision, spatial orientation, physical, mental health, or has psychological and economic effects on a community [6][7][8].All those elements which a community finds unattractive, ugly, intrusive, disturbing come under visual pollution [9].While, Nami et al and Jana describe visual pollution as "unbridled and uncoordinated diversity" of form, color, light, materials and accumulation of heterogeneous visual elements which make the manmade environment and urban landscape ugly and unattractive [3,7].Chmielewski et al quote the term as compound and the resultant effect of "clutter, disorder and excess of various objects and graphics in urban landscape" [10].J ūrat ė et al define visual pollution as "negative visual impact of Visual Pollution Object (VPO) on landscape" [11].Additionally, the Supreme Court of the USA declared that "pollution is not limited to the air we breathe and the water we drink, it can equally offend the eye and ear" [12,13].The scholars engaged in the visual impact assessments of natural landscapes have used the term visual pollution objects (VPOs) to refer to the physical components that have the ability to decrease the landscape's visual quality, may contribute to diminish visual significance, or may obstruct the view of valuable natural complexes [11].A similar analogy is valid for urban visual pollution.In an urban context, we define the term 'VPOs' to refer to all the manmade features along with their physical characteristics (placement, appearance, size, color, view and functional hindrance etc.) that affect the visual quality of urban surroundings from a human's eye view.
Urban visual pollution is the negative physical condition of a number of objects which have a direct as well as an indirect relation with the quality of the built environment which ultimately has implications for humans living in that place [14][15][16][17][18].It has been reported that visual pollution objects (together or individually) impact human health [19,20], distract drivers particularly along main highways [3,14,21,22], reduce property values, deface public places, spread annoyance, encourage needless consumption, or affect the identity of places [5,13,23].It has been argued that better visual quality of a space has a relation to the safe and good behavior of residents and so as with better communities at a larger scale [24,25].
Furthermore, it is important to mention that the term visual pollution must not be confused with the concept of neighborhood disorder.As discussed earlier, visual pollution is the compound effect of disorder, excess of a number of physical elements while neighborhood disorder can be defined as "observed or perceived physical and social features of neighborhoods that may signal the breakdown of order and social control, and that can undermine the quality of life" [26].The examples of neighborhood disorder may be adult loitering, drug dealing, crime, fighting in the streets and prostitution, and physical characteristics such as abandoned cars, dilapidated buildings, or litter in the streets [26,27].Visual pollution is very much linked with the visual quality of public and community spaces [25] which includes regularity, order, beauty, symmetry, and simplicity, etc.
In previous studies, the subject of visual pollution has been explored from the dimensions of concept development, the listing of visual pollution objects and mitigation strategies [28].However, the quantification and measurement have not been explored sufficiently.We have synthesized the previous work on the measurement of visual pollution and have provided a systematic approach to the development of a visual pollution assessment tool.

Approaches to Assessing Visual Pollution
Although visual pollution is recognized as a type of pollution, most of the city governments find it hard to regulate it since they do not have any systematic way of quantifying its presence and measuring its level of intensity vis-a-vis its local impacts [10].Various scholars have used different techniques to measure visual pollution in their respective contexts, and at various scales ranging from a single street to a city.Table 1 presents a list of the relevant studies and highlights their respective scale, VPO coverage and methods.

The Challenges of Measuring Visual Pollution
It is evident from the literature review that the quantification of visual pollution at any point has always been challenging because of its subjective nature [28] and there doesn't seem to exist any standard set of guidelines for systematic assessment of visual pollution [7].This further highlights the need for the development of a unified quantitative assessment tool.Although, researchers all around the globe have contributed to the assessment of visual pollution, they have had a certain set of limitations which are discussed below: Micro vs. Macro-level research: Previous studies on visual pollution can be broadly categorized into two extremes of micro and macro scale.Some of the previous studies have on a small case study area like a commercial street, public buildings in a residential area, or a neighborhood with maybe one or two VPOs (billboards, commercial signs).In contrast, other groups of researches adopted a bigger working scale like a city, with multiple VPOs.Consequently, two (somewhat opposite) strategies with narrow and broad classes of indicators are found in the literature to assess the visual pollution of any area.
Lack of quantification: Since visual pollution is intrinsically sensitive, subjective and a complex type of pollution, no specific tool or instrument is available to measure the scale of visual pollution at any node [29].
Dependency on subjective variables: Most of the available research has used a mix of subjective and objective indicators where the proportion of subjective indicators has been considerably high.Hence, the results can be potentially influenced either by the respondent or the researchers' interpretation, expertise, interest and prevailing literature concepts.So, they may not be agreed upon or adopted in a different context.
Narrow coverage of VPOs: Visual pollution has a broad area of knowledge of built and aesthetic environments comprising multiple objects.However, most previous work has focused on the measurement of visual pollution by means of single VPOs, which has resulted in single-object assessment approaches.
The absence of structured tools: Unlike other types of pollution, structured instruments and tools are not available for the measurement of visual pollution.Moreover, in some cases a combination of different kinds of methodologies like public surveys [2,7], inter-visibility analysis, triangulation method, focus group discussions, photo comparisons [28,38], visual comparisons and experimentation, etc. [3] has been explored to assess a few (or often a single) VPO, which again indicates the need of a composite VPA tool.In most cases, statistical validity or reliability of the methods has not been assessed and discussed thoroughly.
From the above discussion, it is evident that there is a strong need to develop a comprehensive visual pollution assessment tool that can cover a wider variety of VPOs and can be used at various scales to calculate the visual pollution level at a certain point.
When it comes to subjective ranking and weighting, expert evaluation is preferred as they use their knowledge and experience for comparing objects or phenomena under study [39][40][41].The relevant literature contains several weighing methods and techniques (e.g.Delphie method, ordering method [42][43][44]) that have experts evaluation at their core but possess respective pros and cons.In contrast, an Analytical Hierarchy Process (AHP) is considered comprehensive as it is a multi-criteria decision-making approach to deal with subjective and multiple conflicting criteria.The most prominent feature of the AHP method is the ability to compare both the quantitative and qualitative variables (verbal, graphics or numerical) on the same preference scale [45].From the psychological point of view, it is also efficient and consistent to compare two alternatives at a time rather than comparing them all at once.It is based on a ratio scale instead of an interval scale unlike other methods [27,[45][46][47].The other advantages of using AHP include its ability to handle multi-fold subjects, higher consistency among choices and the ability to evaluate the composite and compound scores of alternatives [48].

Materials and Methods
A pragmatic research design was adopted with several sequential and inter-linked steps to design the VPA tool, as summarized in Figure 1.It employs a combination of visuals, public opinion and observations to enlist visual pollution objects.These methods have been used in several studies to investigate public preferences [3,25,29,35,[49][50][51][52].A carefully selected panel of experts was engaged to group, rank and weight VPOs using AHP, which is a widely tested approach to handle subjectivity [30,48,53].Furthermore, AHP findings have been arranged in the form of a VPA scorecard.To measure the reliability of the tool, it has been applied to locations with diverse land-uses, and inter-rater reliability of the tool has been calculated.Figure 1 explains the employed methods, adopted processes and obtained outputs that led to developing the final visual pollution assessment tool.Details of each stage are given below.
First, a list of VPOs has been prepared based on various sources including a literature review, personal observations, a public survey and a university-based photo competition.In literature, only a few VPOs have been frequently listed primarily including outdoor advertisements and billboards.However, through a public survey (available at https://goo.gl/forms/LjKobwAK9m1wUBZc2),107 participants were asked as to what do they find visually annoying in their neighborhood or urban fabric around them.Similarly, a photo competition was arranged among the students of urban planning to identify and capture VPOs in their surroundings.
The second step included the determination of size for the panel of experts and their identification.Literature indicates that the size of an expert panel for AHP studies may vary from just a few people to large groups depending upon the nature of the problem and availability of experts.Generally, AHP does not need more interviews as results get stable after a few responses [54].Furthermore, when the availability of experts is limited, many studies have presented their results with smaller panel sizes; n = 5 [55,56], n = 7 [57], n = 18 [58], and n = 25 [59].In our case, a group of 20 professionals (with 10 or more years of experience) was selected to help on the grouping, ranking and weighting of various VPOs.By means of variability sampling, it was ensured that due representation was achieved from various stakeholders.Consequently, five members were selected from urban planning related academia, three from city district government, three from development authorities, two from cantonments of armed forces, five from private consultants on urban planning, and two civil society members having a history of expressing concern on urban planning matters.In addition to ensuring the thematic diversity of experts, they were selected from various cities of Pakistan representing a variety of cultural and urban contexts.This diversity was particularly helpful since they have been engaged to record their opinion based on their mental images in urban areas.In the third step, the experts classified the listed VPOs into 10 groups considering the similarity of objects.For example, billboards, signboards, advertisement banners, poster and streamers have been clubbed into the group 'outdoor advertisements'.Similarly, out-of-proportion building structures, irregular building facades and an uneven skyline have been grouped into 'architecturally poor structures'.The key reason behind this classification was to reduce the number of VPO groups which can be compared to assigning ranks and weights based on their contribution to visual pollution on a site.It was not possible to handle the larger number of VPOs in AHP without this categorization.AHP was employed for the ranking and weighting of VPOs groups to remove subjectivity associated with the measurement of VPOs.AHP allows the decision maker to consider objective and subjective factors in assessing the relative importance of each VPO through a pairwise comparison.
Since there were 10 VPO groups, a 10 × 10 matrix was formed.The AHP template by Goepel [60] had been adopted for the compilation of results.Since the matrix size and number of panel experts was reasonably large, we used a commercial spreadsheet application (Microsoft Excel) for the compilation of results.To capture their opinion, each member was asked to provide (1) ranking for the VPOs and (2) pairwise comparison of all VPOs groups among themselves based on the level of a VPO's contribution to visual pollution.
Similarly, each of the criteria (VPOs group) were compared with every other criterion by means of a pair-wise comparison over a 9-point Saaty scale.Every member was thoroughly trained on the process and the definitions of scale values.To do so, a series of the dedicated session was conducted with experts where along with the scale and ranking criteria, practical examples were also discussed in detail.For example, in a typical comparison, the expert decides which among outdoor advertisement (A) and wall chalking (B) is the bigger contributor to visual pollution.Suppose the expert selects A, then the next question is that on a scale of 1-9 how much more A contributes to visual pollution than B (while 1 means equally severe, 3 means moderately more severe than the other, 5 means more strongly severe than the other, 7 means very strongly severe and 9 means extremely severe compared to the other while the values 2, 4, 6 and 8 represent the micro-scale between them).Figure 2 presents the screenshot of the excel sheet where pairwise comparisons are recorded and how they are used to generate the automated matrix.
Sustainability 2019, 11 FOR PEER REVIEW 7 compilation of results.To capture their opinion, each member was asked to provide 1) ranking for the VPOs and 2) pairwise comparison of all VPOs groups among themselves based on the level of a VPO's contribution to visual pollution.Similarly, each of the criteria (VPOs group) were compared with every other criterion by means of a pair-wise comparison over a 9-point Saaty scale.Every member was thoroughly trained on the process and the definitions of scale values.To do so, a series of the dedicated session was conducted with experts where along with the scale and ranking criteria, practical examples were also discussed in detail.For example, in a typical comparison, the expert decides which among outdoor advertisement (A) and wall chalking (B) is the bigger contributor to visual pollution.Suppose the expert selects A, then the next question is that on a scale of 1-9 how much more A contributes to visual pollution than B (while 1 means equally severe, 3 means moderately more severe than the other, 5 means more strongly severe than the other, 7 means very strongly severe and 9 means extremely severe compared to the other while the values 2, 4, 6 and 8 represent the micro-scale between them).Figure 2 presents the screenshot of the excel sheet where pairwise comparisons are recorded and how they are used to generate the automated matrix.Parallel to the ranking and weighting of VPOs, characteristics of each type of VPO were listed and rubrics for these characteristics were prepared.At this stage, panel members identified those characteristics for each VPOs that have a direct relationship with the visual impact generated by that VPO.Furthermore, rubric values have been defined against each characteristic.At the next step, intragroup VPOs were weighted and finally, the objects were arranged in the form of a scorecard to be used in the field (see Figures 7 and 8).Finally, the validation and reliability of the VPA tool were Parallel to the ranking and weighting of VPOs, characteristics of each type of VPO were listed and rubrics for these characteristics were prepared.At this stage, panel members identified those characteristics for each VPOs that have a direct relationship with the visual impact generated by that VPO.Furthermore, rubric values have been defined against each characteristic.At the next step, intra-group VPOs were weighted and finally, the objects were arranged in the form of a scorecard to be used in the field (see Figures 7 and 8).Finally, the validation and reliability of the VPA tool were assessed.The inter-observer/ inter-rater reliability (IRR) analysis (also called the inter-observer agreement) by a pilot study was applied.Reliability analysis facilitated in finding the extent to which results in scale is consistent even when observed by different observers.For IRR, the tool was also experimented with at 20 distinct locations of different land-uses in Lahore (the second largest city of Pakistan) by a group of five trained observers (Figure 3).The reason behind selecting 20 locations was to ensure that they cover different combinations of land-uses (residential, commercial, open spaces and public buildings, etc.) and land-use activity intensity.Each location was a three-or-more-legged road junction.The observer was positioned at the centre of a junction or a similarly appropriate location with a 360 o view of the location to record VPOs on the tool.The observers were final year students of urban and regional studies undergraduate programme, who were thoroughly trained on the VPO identification and assessment of their characteristics.Each observer completed the VPA exercise for all 20 locations.Resultantly, 5 assessments (filled VPA scorecards) were available for each location -it is important to highlight that the VPA tool requires capturing 205 values to represent characteristics of different VPOs.Subsequently, responses were analyzed in ten observer pairs and several agreements between each were calculated to get percentage agreement-based IRR.
Sustainability 2019, 11 FOR PEER REVIEW 8 legged road junction.The observer was positioned at the centre of a junction or a similarly appropriate location with a 360 o view of the location to record VPOs on the tool.The observers were final year students of urban and regional studies undergraduate programme, who were thoroughly trained on the VPO identification and assessment of their characteristics.Each observer completed the VPA exercise for all 20 locations.Resultantly, 5 assessments (filled VPA scorecards) were available for each location -it is important to highlight that the VPA tool requires capturing 205 values to represent characteristics of different VPOs.Subsequently, responses were analyzed in ten observer pairs and several agreements between each were calculated to get percentage agreementbased IRR.

Results
The first stage of VPO identification and listing resulted in a long list of 42 VPOs containing more localized and non-conventional visual pollution objects including hanging wires, electricity transformers, broken poles, dilapidated buildings, etc.Such VPOs have not been considered in earlier studies with such emphasis.The listed VPOs have been classified by the panel experts on the basis of the similarity of objects which resulted in 10 wider groups as presented in Table 2.A few glimpses presenting some key VPOs from the streets of Lahore, Pakistan are presented in Figure 4.More graphical evidence of these VPOs can be accessed at https://urbanvisualpollution.wordpress.com.

Results
The first stage of VPO identification and listing resulted in a long list of 42 VPOs containing more localized and non-conventional visual pollution objects including hanging wires, electricity transformers, broken poles, dilapidated buildings, etc.Such VPOs have not been considered in earlier studies with such emphasis.The listed VPOs have been classified by the panel experts on the basis of the similarity of objects which resulted in 10 wider groups as presented in Table 2.A few glimpses presenting some key VPOs from the streets of Lahore, Pakistan are presented in Figure 4.More graphical evidence of these VPOs can be accessed at https://urbanvisualpollution.wordpress.com.The identified groups were ranked by the experts and their weights were calculated.The consolidated weights and ranks generated by experts through AHP reveal that open dumping of solid waste is marked as the largest contributor to visual pollution (23.8%), followed by outdoor advertisements and signage (20.1%).Dilapidated buildings have been ranked as the third major contributing VPO (13.8%) followed by hanging and cluttered wires (11.1%).The list continues with overflown sewers and drains at fifth place (10.4%), graffiti/wall chalking at sixth place (6.9%), various poles and transformers at seventh place (4.5%), encroachments at eighth place (3.8%), and broken roads/ditches at ninth place (3.5%).The VPO group of architecturally poor structures is ranked at the tenth place with a score of 2.1%. Figure 5 presents the consolidated matrix generated from the individual responses of panel experts while Figure 6 represents the final weights and ranks for VPO groups.The identified groups were ranked by the experts and their weights were calculated.The consolidated weights and ranks generated by experts through AHP reveal that open dumping of solid waste is marked as the largest contributor to visual pollution (23.8%), followed by outdoor advertisements and signage (20.1%).Dilapidated buildings have been ranked as the third major contributing VPO (13.8%) followed by hanging and cluttered wires (11.1%).The list continues with overflown sewers and drains at fifth place (10.4%), graffiti/wall chalking at sixth place (6.9%), various poles and transformers at seventh place (4.5%), encroachments at eighth place (3.8%), and broken roads/ditches at ninth place (3.5%).The VPO group of architecturally poor structures is ranked at the tenth place with a score of 2.1%. Figure 5 presents the consolidated matrix generated from the individual responses of panel experts while Figure 6 represents the final weights and ranks for VPO groups.

VPO Group VPOs
advertisements and signage (20.1%).Dilapidated buildings have been ranked as the third major contributing VPO (13.8%) followed by hanging and cluttered wires (11.1%).The list continues with overflown sewers and drains at fifth place (10.4%), graffiti/wall chalking at sixth place (6.9%), various poles and transformers at seventh place (4.5%), encroachments at eighth place (3.8%), and broken roads/ditches at ninth place (3.5%).The VPO group of architecturally poor structures is ranked at the tenth place with a score of 2.1%. Figure 5 presents the consolidated matrix generated from the individual responses of panel experts while Figure 6 represents the final weights and ranks for VPO groups.In order to understand the pattern of ranking by each expert, an AHP consensus indicator was calculated using Shannon alpha and beta entropy [63].The consensus indicator ranges from 0% (no consensus) to 100% (full consensus).The calculated consensus turned out to be 80.1%, which reflects a high overall level of consensus among the experts.Figure 7 represents the mapping of VPO weights given by each panel member.Each line represents one expert while the bold red line shows the average value.The dispersion in the opinion of experts on certain VPOs reflects the diversity which comes in opinion because of their experience, knowledge or professional background.After ranking, rubrics were prepared to systematically measure the characteristics of VPOs.Table 3 shows some of the characteristics and rubric values listed for billboards.Similar tables were prepared for each VPO under study and vetted by the panel of experts.In order to understand the pattern of ranking by each expert, an AHP consensus indicator was calculated using Shannon alpha and beta entropy [60].The consensus indicator ranges from 0% (no consensus) to 100% (full consensus).The calculated consensus turned out to be 80.1%, which reflects a high overall level of consensus among the experts.Figure 7 represents the mapping of VPO weights given by each panel member.Each line represents one expert while the bold red line shows the average value.The dispersion in the opinion of experts on certain VPOs reflects the diversity which comes in opinion because of their experience, knowledge or professional background.In order to understand the pattern of ranking by each expert, an AHP consensus indicator was calculated using Shannon alpha and beta entropy [63].The consensus indicator ranges from 0% (no consensus) to 100% (full consensus).The calculated consensus turned out to be 80.1%, which reflects a high overall level of consensus among the experts.Figure 7 represents the mapping of VPO weights given by each panel member.Each line represents one expert while the bold red line shows the average value.The dispersion in the opinion of experts on certain VPOs reflects the diversity which comes in opinion because of their experience, knowledge or professional background.After ranking, rubrics were prepared to systematically measure the characteristics of VPOs.Table 3 shows some of the characteristics and rubric values listed for billboards.Similar tables were prepared for each VPO under study and vetted by the panel of experts.After ranking, rubrics were prepared to systematically measure the characteristics of VPOs.Table 3 shows some of the characteristics and rubric values listed for billboards.Similar tables were prepared for each VPO under study and vetted by the panel of experts.As discussed previously, the tool has been tested at 20 locations to assess its validity and reliability through IRR analysis.Figure 8 shows the level of calculated visual pollution on those 20 assessed sites.Table 4 presents the number of inter-observer agreements for each location/site.Furthermore, agreement ratio has been calculated for each observer pair and then the mean of each row has been calculated to see the overall agreement ratio at each site, as presented in Figure 9.
Sustainability 2019, 11 FOR PEER REVIEW 13 Furthermore, agreement ratio has been calculated for each observer pair and then the mean of each row has been calculated to see the overall agreement ratio at each site, as presented in Figure 9.The results of inter-rated reliability analysis reflect a very good level of agreement at 83% which indicates the potential soundness of the methodology and the resulting VPA tool.It is important to highlight that out of 100 observer pairs, the minimum agreement ratio is 73% while the highest agreement ratio is 93%.The key reason behind the higher agreement ratio is that the tool collects data on 205 variables out of which 134 can get discrete answers with almost no potential of variance in observations (if observers are well trained).
From the initial testing, it is clear that the developed VPA tool tends to mitigate the previously identified limitations related to the assessment and quantification of visual pollution.The tool can deal with the subjective problem of visual pollution in a more objective way.It offers a wider coverage of 40 VPOs in local urban settings.Furthermore, it can record detailed characteristics against each VPO and quantify them by assigning weights and calculating visual pollution score.The tool can be used at both the micro and macro level scale for the measurement of visual pollution, i.e., nodes, street, neighborhood, or a city.The effective utilization of the tool requires the collection of geolocation and pictorial evidences as complementary information.In addition to the VPA scorecard, the collection of such attributes requires handling of additional gadgets (GPS device, camera or a hybrid device).From initial testing, it has been learnt that careful attention must be given to the systematic storage and labeling of these additional pieces of information so that they can be synchronized with the score card data.

Conclusions and Future Work
In this paper, we have presented a novel methodology for the systematic development of a robust and consistent VPA tool, which provides a structured mechanism for quantification of visual pollution at any given location by measuring the presence and characteristics of various VPOs.Since the VPO characteristics have been explicitly quantified, the tool provides the resultant quantitative score representing the level of visual pollution on a scale of 1-100.The incorporation of (diverse) expert opinion makes the methodology robust and suitable for almost all urban areas of Pakistan.Moreover, given the broad coverage of the tool, it is applicable in other parts of the world, especially the developing countries.This VPA tool can help urban planners and government stakeholders to better understand the prevalence of visual pollution, assess its spatial spread, identify primary contributing VPOs per location, take any corrective actions, and (most importantly) inform policy decisions in a robust, quantifiable and evidence-driven manner.
Further to the national and global application of this tool, an interesting future research dimension will be the determination of visual pollution threshold defined by means of public opinion and statistical method.Although the tool has been primarily designed, tested and being implemented Inter-rater reliability

Sites
Site wise mean of inter-observer agreement The results of inter-rated reliability analysis reflect a very good level of agreement at 83% which indicates the potential soundness of the methodology and the resulting VPA tool.It is important to highlight that out of 100 observer pairs, the minimum agreement ratio is 73% while the highest agreement ratio is 93%.The key reason behind the higher agreement ratio is that the tool collects data on 205 variables out of which 134 can get discrete answers with almost no potential of variance in observations (if observers are well trained).
From the initial testing, it is clear that the developed VPA tool tends to mitigate the previously identified limitations related to the assessment and quantification of visual pollution.The tool can deal with the subjective problem of visual pollution in a more objective way.It offers a wider coverage of 40 VPOs in local urban settings.Furthermore, it can record detailed characteristics against each VPO and quantify them by assigning weights and calculating visual pollution score.The tool can be used at both the micro and macro level scale for the measurement of visual pollution, i.e., nodes, street, neighborhood, or a city.The effective utilization of the tool requires the collection of geolocation and pictorial evidences as complementary information.In addition to the VPA scorecard, the collection of such attributes requires handling of additional gadgets (GPS device, camera or a hybrid device).From initial testing, it has been learnt that careful attention must be given to the systematic storage and labeling of these additional pieces of information so that they can be synchronized with the score card data.

Conclusions and Future Work
In this paper, we have presented a novel methodology for the systematic development of a robust and consistent VPA tool, which provides a structured mechanism for quantification of visual pollution at any given location by measuring the presence and characteristics of various VPOs.Since the VPO characteristics have been explicitly quantified, the tool provides the resultant quantitative score representing the level of visual pollution on a scale of 1-100.The incorporation of (diverse) expert opinion makes the methodology robust and suitable for almost all urban areas of Pakistan.Moreover, given the broad coverage of the tool, it is applicable in other parts of the world, especially the developing countries.This VPA tool can help urban planners and government stakeholders to better understand the prevalence of visual pollution, assess its spatial spread, identify primary contributing VPOs per location, take any corrective actions, and (most importantly) inform policy decisions in a robust, quantifiable and evidence-driven manner.
Further to the national and global application of this tool, an interesting future research dimension will be the determination of visual pollution threshold defined by means of public opinion and

Figure 2 :
Figure 2: Screenshot of AHP sheet reflecting the capturing of pair-wise comparison and formation of a comparison matrix by one expert.A Similar process has been adopted to capture inputs of all panel members

Figure 2 .
Figure 2. Screenshot of AHP sheet reflecting the capturing of pair-wise comparison and formation of a comparison matrix by one expert.A Similar process has been adopted to capture inputs of all panel members.

Figure 3 :
Figure 3: Map showing the distribution of sites for piloting of VPA tool and IRR analysis (each blue numbered dot represents one site)

Figure 3 .
Figure 3. Map showing the distribution of sites for piloting of VPA tool and IRR analysis (each blue numbered dot represents one site).

Figure 4 .
Figure 4. Pictures from the streets of Lahore, Pakistan capturing some key VPOs including outdoor advertisements, poles, hanging and cluttered wires, architecturally poor structures, dilapidated building and encroachments.

Figure 4 .
Figure 4. Pictures from the streets of Lahore, Pakistan capturing some key VPOs including outdoor advertisements, poles, hanging and cluttered wires, architecturally poor structures, dilapidated building and encroachments.

Figure 5 :
Figure 5: Consolidated AHP matrix generated from individual responses of panel experts

Figure 5 .
Figure 5. Consolidated AHP matrix generated from individual responses of panel experts.Sustainability 2019, 11 FOR PEER REVIEW 11

Figure 6 :
Figure 6: Final weights and ranks for VPO groups

Figure 7 :
Figure 7: Mapping of weights assigned to VPO groups by each panel expert

Figure 6 .
Figure 6.Final weights and ranks for VPO groups.

Figure 6 :
Figure 6: Final weights and ranks for VPO groups

Figure 7 :
Figure 7: Mapping of weights assigned to VPO groups by each panel expert

Figure 7 .
Figure 7. Mapping of weights assigned to VPO groups by each panel expert.

Figure 8 :
Figure 8: Map showing the spatial spread of visual pollution on the 20 assessed sites (heat map color scale from red to green represents highest to lowest visual pollution)

Figure 8 .
Figure 8. Map showing the spatial spread of visual pollution on the 20 assessed sites (heat map color scale from red to green represents highest to lowest visual pollution).

Figure 9 :
Figure 9: Site wise mean agreement ratio based on inter-observer agreements

Figure 9 .
Figure 9. Site wise mean agreement ratio based on inter-observer agreements.

Table 1 .
List of studies containing components similar to visual pollution assessment.

Table 2 .
Classification of VPOs in Major VPO Groups

Table 2 .
Classification of VPOs in Major VPO Groups.

Table 3 .
Listing of VPOs Characteristics and Preparation of Rubrics for "Billboards".

Table 3 .
Listing of VPOs Characteristics and Preparation of Rubrics for "Billboards".

Table 3 .
Listing of VPOs Characteristics and Preparation of Rubrics for "Billboards".

Table 4 .
Number of inter-observer agreements

Table 4 .
Number of inter-observer agreements.