# An Optimal Rubrics-Based Approach to Real Estate Appraisal

^{1}

^{2}

^{3}

^{4}

^{5}

^{6}

^{*}

## Abstract

**:**

## 1. Introduction

## 2. Methodology

#### 2.1. Construction of the Rubrics Model

#### 2.1.1. Crowdsourcing

#### 2.1.2. Rating Model

#### 2.1.3. Rubrics

#### 2.2. Definition of Similarity

#### 2.2.1. Relevance

#### 2.2.2. Diversity

#### 2.3. Similarity Measurement

#### 2.3.1. Cosine Similarity Function

#### 2.3.2. Relevance

#### 2.3.3. Diversity

#### 2.3.4. Similarity

#### 2.4. Selection of Optimal Rubrics

#### 2.5. Crowdsourcing Appraisal

## 3. Case Study

#### 3.1. Study Area and Datasets

#### 3.2. Method Application

- P1 = (4, 1, 1, 1, 4, 2, 1, 2, 0, 1, 1, 2, 2, 1, 1, 1, 1, 1, 1),
- PA = (4, 2, 1, 1, 3, 4, 1, 1, 0, 1, 2, 1, 5, 4, 3, 4, 1, 2, 5),
- PB = (6, 2, 1, 1, 4, 4, 1, 1, 0, 1, 2, 2, 4, 2, 3, 4, 2, 3, 4),
- PC = (7, 3, 1, 2, 4, 4, 2, 2, 1, 1, 2, 1, 5, 4, 3, 4, 3, 1, 5).

#### 3.3. Validation

## 4. Discussion

## 5. Conclusions

## Acknowledgments

## Author Contributions

## Conflicts of Interest

## References

- Renigier-Biłozor, M.; Biłozor, A.; Wisniewski, R. Rating engineering of real estate markets as the condition of urban areas assessment. Land Use Policy
**2017**, 61, 511–525. [Google Scholar] [CrossRef] - Zhang, H.; Li, L.; Chen, T.; Li, V. Where will China’s real estate market go under the economy’s new normal? Land Use Policy
**2016**, 55, 42–48. [Google Scholar] [CrossRef] - Liu, F.; Matsuno, S.; Malekian, R.; Yu, J.; Li, Z. A Vector Auto Regression Model Applied to Real Estate Development Investment: A Statistic Analysis. Sustainability
**2016**, 8, 1082. [Google Scholar] [CrossRef] - Quigley, J. Real Estate Prices and Economic Cycles. In Berkeley Program Housing Urban Policy Working Paper; Institute of Business and Economic Research, UC Berkeley: Berkeley, CA, USA, 2002; pp. 1–20. [Google Scholar]
- Case, B.; Quigley, J. The Dynamics of Real Estate Prices. Rev. Econ. Stat.
**1991**, 73, 50–58. [Google Scholar] [CrossRef] - TırtırogˇLu, D. Valuation of real estate assets using the adjusted present value method. J. Prop. Financ.
**1997**, 8, 7–23. [Google Scholar] [CrossRef] - Yang, J.; Bai, Q. Research of Real Estate Appraisal Based on GIS Technology. Adv. Mater. Res.
**2013**, 859, 562–565. [Google Scholar] [CrossRef] - Pagourtzi, E.; French, N.; Hatzichristos, T.; Assimakopoulos, V. Real estate appraisal: A review of valuation methods. J. Prop. Invest. Financ.
**2003**, 21, 383–401. [Google Scholar] [CrossRef] - Gwin, C.; Maxam, C. Why do real estate appraisals nearly always equal offer price. J. Prop. Invest. Financ.
**2002**, 20, 242–253. [Google Scholar] [CrossRef] - Lind, H. The definition of market value: Criteria for judging proposed definitions and an analysis of three controversial components. J. Prop. Valuat. Invest.
**1998**, 16, 159–174. [Google Scholar] [CrossRef] - De Ruggiero, M.; Forestiero, G.; Manganelli, B.; Salvo, F. Buildings Energy Performance in a Market Comparison Approach. Buildings
**2017**, 7, 2–14. [Google Scholar] [CrossRef] - International Valuation Standards Council (IVSC). International Valuation Standards 2011; IVSC: London, UK, 2011. [Google Scholar]
- Guo, J.; Xu, S.; Bi, Z. An integrated cost-based approach for real estate appraisals. Inf. Technol. Manag.
**2014**, 15, 131–139. [Google Scholar] [CrossRef] - Bradford, D. Benefit—Cost Analysis and Demand Curves for Public Goods. Kyklos
**1970**, 23, 775–791. [Google Scholar] [CrossRef] - Giudice, V.D.; Paola, P.D.; Manganelli, B.; Forte, F.; Rosen, M.A. The Monetary Valuation of Environmental Externalities through the Analysis of Real Estate Prices. Sustainability
**2017**, 9, 229. [Google Scholar] [CrossRef] - Wu, L.; Guo, X. An Evaluation Model of Inundated Land Based on Income Approach—Residual Method. Appl. Mech. Mater.
**2012**, 209–211, 1627–1630. [Google Scholar] [CrossRef] - Suhadolc, M. Application of the income approach in the special real properties mass appraisal system. Geod. Vestnik.
**2009**, 53, 259–272. [Google Scholar] - Salvo, F.; De Ruggiero, M. Market Comparison Approach between tradition and innovation. A simplifying approach. Aestimum
**2013**, 62, 585–594. [Google Scholar] - Lin, T.; Chang, H. How do appraisers absorb market information in property valuation? Some experimental evidence. Prop. Manag.
**2012**, 30, 190–206. [Google Scholar] - Downie, M.L.; Robson, G. Automated Valuation Models: An International perspective. Counc. Mortg. Lenders (CML)
**2007**, 10, 10–11. [Google Scholar] - IAAO. Standard on Automated Valuation Models. 2003. Available online: http://www.iaao.org/media/standards/AVM_STANDARD.pdf (accessed on 3 May 2017).
- RICS. Automated Valuation Models (AVMs), 1st Edition. 2013. Available online: http://www.rics.org/Global/Automated_Valuation_Models_1st_edition_PGguidance_2013.pdf (accessed on 3 May 2017).
- Sopranzetti, B.J. Hedonic Regression Analysis in Real Estate Markets: A Primer. In Handbook of Quantitative Finance and Risk Management; Springer: New York, NY, USA, 2010; pp. 1201–1207. [Google Scholar]
- Giuffrida, S.; Ferluga, G.; Valenti, A. Clustering Analysis in a Complex Real Estate Market: The Case of Ortigia (Italy). In Proceedings of the International Conference on Computational Science and Its Applications, Guimaraes, Portugal, 30 June–3 July 2014; pp. 106–121. [Google Scholar]
- Giuffrida, S.; Ferluga, G.; Valenti, A. Capitalisation rates and ‘real estate semantic chains’: An application of clustering analysis. Int. J. Bus. Intell. Data Min.
**2015**, 10, 174–199. [Google Scholar] [CrossRef] - Piccinini, L.C.; Chang, T.F.M. The Turning point of the landscape–cultural mosaic: Renaissance, Revelation, Resilience. In Proceedings of the 19th IPSAPA/ISPALEM Conference, Naples, Italy, 2–3 July 2015; pp. 67–78. [Google Scholar]
- Kauko, T.; D’Amato, M. Mass Appraisal Methods: An International Perspective for Property Values; Wiley-Blackwell: Hoboken, NJ, USA, 2008. [Google Scholar]
- Kauko, T. On current neural network applications involving spatial modelling of property prices. J. Hous. Built Environ.
**2003**, 18, 159–181. [Google Scholar] [CrossRef] - Borst, R.; Mccluskey, W. Using Geographically Weighted Regression to Detect Housing Submarkets: Modeling Large-Scale Spatial Variations in Value. J. Prop. Tax Assess. Adm.
**2007**, 5, 21–54. [Google Scholar] - D’Amato, M. A location value response surface model for mass appraising: An “iterative” location adjustment factor in Bari, Italy. Int. J. Strateg. Prop. Manag.
**2010**, 14, 231–244. [Google Scholar] [CrossRef] - Sundfors, D.; Lind, H. Sustainability Metrics and Property Value: The Need for a Standardized Sustainability Description. Available online: http://kth.diva-portal.org/smash/get/diva2:1068052/FULLTEXT01.pdf (accessed on 13 April 2017).
- Kontrimas, V.; Verikas, A. The mass appraisal of the real estate by computational intelligence. Appl. Soft Comput.
**2011**, 11, 443–448. [Google Scholar] [CrossRef] - Goodrich, H. Understanding rubrics. Educ. Leadersh.
**1997**, 54, 14–17. [Google Scholar] - Brabham, D.C. Crowdsourcing as a model for problem solving: An introduction and cases. Convergence
**2008**, 14, 75–90. [Google Scholar] [CrossRef] - Cao, C.; She, J.; Tong, Y.; Chen, L. Whom to ask? Jury selection for decision making tasks on micro-blog services. PVLDB
**2012**, 5, 1495–1506. [Google Scholar] - Marcus, A.; Wu, E.; Karger, D.; Madden, S.; Miller, R. Human-powered sorts and joins. PVLDB
**2011**, 5, 13–24. [Google Scholar] [CrossRef] - Parameswaran, A.; Garcia-Molina, H.; Park, H. Crowd screen: Algorithms for filtering data with humans. In Proceedings of the SIGMOD Conference, Scottsdale, AZ, USA, 20–24 May 2012; pp. 361–372. [Google Scholar]
- Payne, J.W. Task complexity and contingent processing in decision making: An information search and protocol analysis. Organ. Behav. Hum. Perform.
**1976**, 16, 366–387. [Google Scholar] [CrossRef] - Porter, M.F. Chapter: An Algorithm for Suffix Stripping. In Readings in Information Retrieval; Morgan Kaufmann Publishers Inc.: San Francisco, CA, USA, 1997; pp. 313–316. [Google Scholar]
- Wang, J.; Kraska, T.; Franklin, M.; Feng, J. Crowder: Crowdsourcing entity resolution. PVLDB
**2012**, 5, 1483–1494. [Google Scholar] [CrossRef] - Das, M.; Thirumuruganathan, S.; Amer-Yahia, S.; Das, G.; Yu, C. Who tags what? An analysis framework. PVLDB
**2012**, 5, 1567–1578. [Google Scholar] - Feng, A.; Franklin, M.; Kossmann, D.; Kraska, T. CrowdDB: Query processing with the VLDB crowd. PVLDB
**2011**, 4, 1387–1390. [Google Scholar] - Doan, A.; Ramakrishnan, R.; Halevy, A. Crowdsourcing systems on the world-wide web. ACM
**2011**, 54, 86–96. [Google Scholar] [CrossRef] - Gomes, R.; Welinder, P.; Krause, A.; Perona, P. Crowdclustering. NIPS. 2011. Available online: https://las.inf.ethz.ch/files/gomes11crowd.pdf (accessed on 13 April 2017).
- Guo, S.; Parameswaran, A.; Garcia-Molina, H. So who won? Dynamic max discovery with the crowd. In Proceedings of the SIGMOD Conference, Scottsdale, AZ, USA, 20–24 May 2012; pp. 385–396. [Google Scholar]
- Kaplan, H.; Lotosh, I.; Milo, T.; Novgorodov, S. Answering planning queries with the crowd. PVLDB
**2013**, 6, 697–708. [Google Scholar] [CrossRef] - Amsterdamer, Y.; Grossman, Y.; Milo, T.; Senellart, P. Crowd mining. In Proceedings of the SIGMOD Conference, New York, NY, USA, 22–27 June 2013; pp. 241–252. [Google Scholar]
- Zhang, Q.; Yang, L.T.; Chen, Z.; Li, P. High-order Possibilistic c-Means Algorithms Based on Tensor Decompositions for Big Data in IoT. Inform. Fusion.
**2018**, 39, 72–80. [Google Scholar] [CrossRef] - Zhang, Q.; Yang, L.T.; Chen, Z.; Li, P. Deep Computation Model for Unsupervised Feature Learning on Big Data. IEEE Trans. Serv. Comput.
**2016**, 9, 161–171. [Google Scholar] [CrossRef] - Bayas, J.; See, L.; Fritz, S.; Sturn, T.; Perger, C.; Dürauer, M.; Karner, M.; Moorthy, I.; Schepaschenko, D.; Domian, D.I.; et al. Crowdsourcing In-Situ Data on Land Cover and Land Use Using Gamification and Mobile Technology. Remote Sens.
**2016**, 8, 905. [Google Scholar] - Chen, X.; Bennett, P.; Collins-Thompson, K.; Horvitz, E. Pairwise ranking aggregation in a crowdsourcing setting. In Proceedings of the Sixth ACM International Conference on Web Search and Data Mining, Rome, Italy, 4–8 February 2013; pp. 193–202. [Google Scholar]
- Dalvi, N.; Dasgupta, A.; Kumar, R.; Rastogi, V. Aggregating crowdsourcing binary ratings. In Proceedings of the 22nd International Conference on World Wide Web, Rio de Janeiro, Brazil, 13–17 May 2013; pp. 285–294. [Google Scholar]
- Buhrmester, M.; Kwang, T.; Gosling, S.D. Amazon’s Mechanical Turk: A new source of inexpensive, yet high-quality, data? Perspect. Psychol. Sci.
**2011**, 6, 3–5. [Google Scholar] [CrossRef] [PubMed] - Gunay, O.; Akbas, C.E.; Cetin, A.E. Cosine Similarity Measure According to a Convex Cost Function. Comput. Sci.
**2014**, 34, 4447–4463. [Google Scholar] - Joseph, F.; Schram, J. Real Estate Appraisal; Rockwell Publishing: Washington, DC, USA, 2011. [Google Scholar]

**Figure 2.**A well-designed sample data driven rubric. (a) The painting assignment rated B; (b) The painting assignment rated D; (c) The painting assignment rated C; (d) The painting assignment rated A.

**Figure 3.**Effect of relevance. (a) The painting assignment rated C; (b) The painting assignment rated D; (c) The painting assignment rated A; (d) The painting assignment rated B.

**Figure 4.**Effect of diversity. (a) The painting assignment rated B; (b) The painting assignment rated B; (c) The painting assignment rated B; (d) The painting assignment rated B.

**Figure 6.**Example of publication on the crowdsourcing platform. (a) The painting assignment rated B; (b) The painting assignment rated D; (c) The painting assignment rated C; (d) The painting assignment rated A.

Rubrics | TP (S) | Rubric | TP (S) |
---|---|---|---|

A, B, C | 0.240 | A, B, D | 0.070 |

A, B, E | 0.298 | A, C, D | 0.128 |

A, C, E | 0.229 | A, D, E | 0.161 |

B, C, D | 0.161 | B, C, E | 0.301 |

B, D, E | 0.189 | C, D, E | 0.231 |

Sample Point | Point 1 | Point 2 | Point 3 |
---|---|---|---|

Price (¥/m^{2}) | 4231.95 | 7525.80 | 3441.16 |

ID | 200001230119 | 200001690085 | 200201370180 |

Geographic Coordinate X | 22.950302 | 22.687230 | 22.955814 |

Geographic Coordinate Y | 113.267433 | 113.167094 | 113.097154 |

Architectural Structure | Reinforced concrete | Reinforced concrete | Frame |

Date of Registration Right | 28 May 2002 | 17 October 2001 | 1 January 1900 |

Date of Issue | 30 December 1899 | 30 December 1899 | missing |

Address | Room 201, Building 2, Fifth Road, Country Garden, Shunde District | No. 15, Unit K, Building B, Baian Road, Electronics Store, Shunde District | Room 7, Building B, No. A112, Guihua Road, Shunde District |

Recorded Time (year) | 2000 | missing | 2007 |

Number of Floors | 6 | 7 | 13 |

Area of The Building Base (m^{2}) | 0 | 0 | 0 |

Set Inside Floor Area (m^{2}) | 72.8 | 71.6 | 29.06 |

Usage of The House | Residential | Commercial | Garage |

Land Use | Residential | Residential and Commercial | Residential and Commercial |

Land Public Area (m^{2}) | 7.627 | 21.515 | 29.309 |

Land Apportioned Area (m^{2}) | 13 | 13.3 | 3 |

Building Area (m^{2}) | missing | missing | missing |

Public Apportioned Area (m^{2}) | 0 | 0 | 0 |

The Distance from the Subway Station (m) | 3669 | 27,248 | 17,042 |

The Distance from the CBD (m) | 3669 | 19,404 | 17,042 |

Density of Road Network | 3.27 | 5.66 | 3.64 |

Density of Markets | 0.32 | 0 | 1.59 |

Density of Bus Stations | 0 | 1.27 | 8.91 |

Density of Telecommunications | 0.22 | 0 | 0.88 |

Density of Supermarkets | 0 | 3.54 | 2.65 |

Density of Post Offices | 0.11 | 0.33 | 0.33 |

Density of Banks | 2.86 | 3.50 | 8.91 |

Attributes | Original Value | Discrete Value (Class) |
---|---|---|

Recorded Time (year) | 1990–2010 | 1, 2, 3, 4 |

Number of Floors | −1–42 | 1, 2, 3, 4, 5 |

Area of The Building Base (m^{2}) | 0–235.60 | 1, 2, 3, 4 |

Set Inside Floor Area (m^{2}) | 1.01–104.36 | 1, 2, 3, 4 |

Usage of The House | Office, Parking, Industrial, Commercial, Residential, etc. | 1, 2, 3, 4, 5, 6 |

Land Use | Industrial, Commercial, Residential, etc. | 1, 2, 3, 4 |

Land Public Area (m^{2}) | 0–44.27 | 1, 2, 3, 4, 5 |

Land Apportioned Area | 0–37.63 | 1, 2, 3, 4 |

Building Area (m^{2}) | 2–124.68 | 1, 2, 3, 4 |

Public Apportioned Area (m^{2}) | 0–22.22 | 1, 2, 3, 4 |

The Distance from the Subway Station (m) | 1462–27,248 | 1, 2, 3, 4, 5, 6 |

The Distance from the CBD (m) | 248–24,162 | 1, 2, 3, 4, 5, 6 |

Density of Road Networks | 0.22–13.23 | 1, 2, 3, 4, 5 |

Density of Markets | 0–2.55 | 1, 2, 3, 4 |

Density of Bus Stations | 0–22.92 | 1, 2, 3, 4, 5 |

Density of Telecommunications | 0–3.76 | 1, 2, 3, 4 |

Density of Supermarkets | 0–13.26 | 1, 2, 3, 4, 5 |

Density of Post Offices | 0–1.10 | 1, 2, 3 |

Density of Banks | 0–35.01 | 1, 2, 3, 4, 5 |

Attributes | P1 | PA | PB | PC |
---|---|---|---|---|

Recorded Time (year) | 2000 | 2000 | 2006 | 2009 |

Number of Floors | 6 | 13 | 14 | 29 |

Area of The Building Base (m^{2}) | 0 | 0 | 0 | 0 |

Set Inside Floor Area (m^{2}) | 72.8 | 58.5 | 20.8 | 372.35 |

Usage of The House | Residential | Commercial | Residential | Residential |

Land Use | Residential | Residential and commercial | Residential, commercial, and office | Residential and commercial |

Land Public Area (m^{2}) | 7.627 | 2174.1 | 1368.7 | 15285 |

Land Apportioned Area | 13 | 9.6 | 2.8 | 35.3 |

Building Area (m^{2}) | missing | missing | missing | 372.35 |

Public Apportioned Area (m^{2}) | 0 | 0 | 0 | 0 |

The Distance from theSubway Station (m) | 3669 | 8648 | 8356 | 8954 |

The Distance from the CBD (m) | 3669 | 899 | 2425 | 988 |

Density of Road Networks | 3.27 | 10.42 | 9.87 | 11.47 |

Density of Markets | 0.32 | 2.23 | 1.27 | 2.23 |

Density of Bus Stations | 0 | 11.46 | 8.91 | 10.19 |

Density of Telecommunications | 0.22 | 2.87 | 3.09 | 2.87 |

Density of Supermarkets | 0 | 0.88 | 3.54 | 5.31 |

Density of Post Offices | 0.11 | 0.44 | 1.10 | 0.33 |

Density of Banks | 2.86 | 33.10 | 25.78 | 33.10 |

Attributes | OP1 | OP2 | OP3 |
---|---|---|---|

Price (¥/m^{2}) | 4223.29 | 4206.22 | 4231.95 |

Recorded Time (year) | 2001 | 2001 | 2001 |

Number of Floors | 6 | 6 | 6 |

Area of The Building Base (m^{2}) | 0 | 0 | 0 |

Set Inside Floor Area (m^{2}) | 90.1 | 72.7 | 72.8 |

Usage of The House | Residential | Residential | Residential |

Land Use | Residential | Residential | Residential |

Land Public Area (m^{2}) | 9.252 | 7.631 | 7.631 |

Land Apportioned Area | 16.2 | 13 | 13 |

Building Area (m^{2}) | missing | missing | missing |

Public Apportioned Area (m^{2}) | 0 | 0 | 0 |

The Distance from the Subway Station (m) | 3582 | 3669 | 3669 |

The Distance from the CBD (m) | 3582 | 3669 | 3669 |

Density of Road Networks | 3.25 | 3.27 | 3.27 |

Density of Markets | 0.32 | 0.32 | 0.32 |

Density of Bus Stations | 0 | 0 | 0 |

Density of Telecommunications | 0.44 | 0.22 | 0.22 |

Density of Supermarkets | 0 | 0 | 0 |

Density of Post Offices | 0.11 | 0.11 | 0.11 |

Density of Banks | 2.86 | 2.86 | 2.86 |

Users | Feedback Results (¥/m^{2}) | Users | Feedback Results (¥/m^{2}) |
---|---|---|---|

U1 | 4250 | U17 | 4150 |

U2 | 4300 | U18 | 4200 |

U3 | 4100 | U19 | 4250 |

U4 | 4200 | U20 | 4200 |

U5 | 4250 | U21 | 4150 |

U6 | 4150 | U22 | 4300 |

U7 | 4300 | U23 | 4200 |

U8 | 4000 | U24 | 4250 |

U9 | 4100 | U25 | 4250 |

U10 | 4150 | U26 | 4200 |

U11 | 4250 | U27 | 4400 |

U12 | 4200 | U28 | 4350 |

U13 | 4200 | U29 | 4100 |

U14 | 4500 | U30 | 4150 |

U15 | 4300 | U31 | 4200 |

U16 | 4250 | U32 | 4200 |

Estimated Point | Relative Error | Estimated Point | Relative Error |
---|---|---|---|

P1 | 0.3% | P11 | 4.2% |

P2 | 3.8% | P12 | 0.8% |

P3 | 1.1% | P13 | 1.3% |

P4 | 7.4% | P14 | 3.7% |

P5 | 16.3% | P15 | 6.0% |

P6 | 1.2% | P16 | 5.5% |

P7 | 4.5% | P17 | 2.7% |

P8 | 0.7% | P18 | 4.2% |

P9 | 5.5% | P19 | 13.3% |

P10 | 1.4% | P20 | 3.1% |

Combination of Rubrics | Accuracy ($\mathit{\alpha}=0.3$) | Accuracy ($\mathit{\alpha}=0.5$) | Accuracy ($\mathit{\alpha}=0.7$) |
---|---|---|---|

k = 1 | 65% | 70% | 65% |

k = 2 | 70% | 70% | 75% |

k = 3 | 85% | 85% | 90% |

k = 5 | 80% | 75% | 80% |

k = 10 | 60% | 65% | 70% |

© 2017 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (http://creativecommons.org/licenses/by/4.0/).

## Share and Cite

**MDPI and ACS Style**

Chen, Z.; Hu, Y.; Zhang, C.J.; Liu, Y.
An Optimal Rubrics-Based Approach to Real Estate Appraisal. *Sustainability* **2017**, *9*, 909.
https://doi.org/10.3390/su9060909

**AMA Style**

Chen Z, Hu Y, Zhang CJ, Liu Y.
An Optimal Rubrics-Based Approach to Real Estate Appraisal. *Sustainability*. 2017; 9(6):909.
https://doi.org/10.3390/su9060909

**Chicago/Turabian Style**

Chen, Zhangcheng, Yueming Hu, Chen Jason Zhang, and Yilun Liu.
2017. "An Optimal Rubrics-Based Approach to Real Estate Appraisal" *Sustainability* 9, no. 6: 909.
https://doi.org/10.3390/su9060909