- freely available
Information 2016, 7(4), 67; https://doi.org/10.3390/info7040067
2.1. Search Strategy and Databases Searched
2.2. Inclusion and Exclusion Criteria
2.3. Data Extraction
2.4. Quality Assessment
3. Systematic Review Process
3.1. GPS Data Processing Procedure
3.2. Data Preprocessing Procedure
3.3. Trip/Segment Identification
3.4. Travel Mode Detection
4. Quality of Reviewed Studies
5. Limitations and Strengths
6. Discussion and Conclusions
Conflicts of Interest
- Xiao, G.; Juan, Z.; Zhang, C. Travel mode detection based on GPS track data and Bayesian networks. Comput. Environ. Urban Syst. 2015, 54, 14–22. [Google Scholar] [CrossRef]
- Yue, Y.; Lan, T.; Yeh, A.G.O.; Li, Q.Q. Zooming into individuals to understand the collective: A review of trajectory-based travel behavior studies. Travel Behav. Soc. 2014, 1, 69–78. [Google Scholar] [CrossRef]
- Wolf, J.L. Using GPS Data Loggers to Replace Travel Diaries in the Collection of Travel Data. Ph.D. Thesis, Georgia Institute of Technology, School of Civil and Environmental Engineering, Atlanta, GA, USA, 2000. [Google Scholar]
- Shen, L.; Stopher, P.R. Review of GPS travel survey and GPS data-processing methods. Transp. Rev. 2014, 34, 316–334. [Google Scholar] [CrossRef]
- Stopher, P.; Fitzgerald, C.; Zhang, J. Search for a global positioning system device to measure person travel. Transp. Res. Part C Emerg. Technol. 2008, 16, 350–369. [Google Scholar] [CrossRef]
- Zimowski, M.; Tourangeau, R.; Ghadialy, R.; Pedlow, S. Nonresponse in household travel surveys. Available online: http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.570.6554&rep=rep1&type=pdf (accessed on 9 November 2016).
- Gong, L.; Morikawa, T.; Yamamoto, T.; Sato, H. Deriving Personal Trip Data from GPS Data: A Literature Review on the Existing Methodologies. Procedia Soc. Behav. Sci. 2014, 138, 557–565. [Google Scholar] [CrossRef]
- Stopher, P.; Clifford, E.; Zhang, J.; Fitzgerald, C. Deducing Mode and Purpose from GPS Data. Available online: http://ws.econ.usyd.edu.au/itls/wp-archive/itls-wp-08-06.pdf (accessed on 9 November 2016).
- Beijing Municipal Committee of Transportation. The Fourth Beijing Comprehensive Transportation Survey Report (2012, July); Beijing Transport Research Centre, Beijing Municipal Committee of Transportation: Beijing, China, 2012.
- Bohte, W.; Maat, K. Deriving and validating trip purposes and travel modes for multi-day GPS-based travel surveys: A large-scale application in the Netherlands. Transp. Res. Part C Emerg. Technol. 2009, 17, 285–297. [Google Scholar] [CrossRef]
- Itsubo, S.; Hato, E. A Study of the Effectiveness of a Household Travel Survey Using GPS-equipped Cell Phones and a WEB Diary through a Comparative Study with a Paper based Travel Survey. In Proceedings of the Transportation Research Board 85th Annual Meeting, Washington, DC, USA, 22–26 January 2006.
- Kelly, P.; Krenn, P.; Titze, S.; Stopher, P.; Foster, C. Quantifying the Difference Between Self-Reported and Global Positioning Systems-Measured Journey Durations: A Systematic Review. Transp. Rev. 2013, 33, 443–459. [Google Scholar] [CrossRef]
- Krygsman, S.; Nel, J.H. The Use of Global Positioning Devices in Travel Surveys—A Developing Country Application. In Proceedings of the 28th Southern African Transport Conference (SATC 2009), Pretoria, South Africa, 6–9 July 2009.
- Marchal, P.; Roux, S.; Yuan, S.; Hubert, J.P.; Armoogum, J.; Madre, J.L.; Lee-Gosselin, M.E.H. A study of non-response in the GPS sub-sample of the French national travel survey 2007–08. In Proceedings of the 8th International Conference on Survey Methods in Transport, France, 25–31 May 2008.
- Oliveira, M.; Vovsha, P.; Wolf, J.; Birotker, Y.; Givon, D.; Paasche, J. Global positioning system-assisted prompted recall household travel survey to support development of advanced travel model in Jerusalem, Israel. Transp. Res. Record J. Transp. Res. Board 2011, 2246, 16–23. [Google Scholar] [CrossRef]
- Papinski, D.; Scott, D.M.; Doherty, S.T. Exploring the route choice decision-making process: A comparison of planned and observed routes obtained using person-based GPS. Transp. Res. Part F Traffic Psychol. Behav. 2009, 12, 347–358. [Google Scholar] [CrossRef]
- Rasmussen, T.K.; Ingvardson, J.B.; Halldórsdóttir, K.; Nielsen, O.A. Using Wearable GPS Devices in Travel Surveys: A Case Study in the Greater Copenhagen Area. In Proceedings of the Annual Transport Conference at Aalborg University, Aalborg, Denmark, 26–27 August 2013.
- Schnfelder, S.; Axhausen, K.W.; Antille, N.; Bierlaire, M. Exploring the potentials of automatically collected GPS data for travel behavior analysis. GI-Technol. Verk. Logist. 2002, 66, 455–456. [Google Scholar]
- Schuessler, N.; Axhausen, K.W. Processing Raw Data from Global Positioning Systems without Additional Information. Transp. Res. Rec. J. Transp. Res. Board 2009, 2105, 28–36. [Google Scholar] [CrossRef]
- Stopher, P.R.; Moutou, C.J.; Liu, W. Sustainability of Voluntary Travel Behavior Change Initiatives: A 5-Year Study. In Proceedings of the Australasian Transport Research Forum, Brisbane, Australia, 2–4 October 2013.
- Stopher, P.; Wargelin, L. Conducting a Household Travel Survey with GPS: Reports on a Pilot Study. In Proceedings of the 12th World Conference on Transport Research, Lisbon, Portugal, 11–15 July 2010.
- Vij, A.; Shankari, K. When is big data big enough? Implications of using GPS-based surveys for travel demand analysis. Transp. Res. Part C Emerg. Technol. 2015, 56, 446–462. [Google Scholar] [CrossRef]
- Wolf, J.; Hallmark, S.; Oliveira, M.; Guensler, R.; Sarasua, W. Accuracy issues with route choice data collection by using global positioning system. Transp. Res. Rec. J. Transp. Res. Board 1999, 1660, 66–74. [Google Scholar] [CrossRef]
- Zhou, J.; Golledge, R. Real-time tracking of activity scheduling/schedule execution within a unified data collection framework. Transp. Res. Part A Policy Pract. 2007, 41, 444–463. [Google Scholar] [CrossRef]
- Gong, H.; Chen, C.; Bialostozky, E.; Lawson, C.T. A GPS/GIS method for travel mode detection in New York City. Comput. Environ. Urban Syst. 2012, 36, 131–139. [Google Scholar] [CrossRef]
- Chen, C.; Gong, H.; Lawson, C.; Bialostozky, E. Evaluating the feasibility of a passive travel survey collection in a complex urban environment: Lessons learned from the New York City case study. Transp. Res. Part A Policy Pract. 2010, 44, 830–840. [Google Scholar] [CrossRef]
- Patterson, D.J.; Liao, L.; Fox, D.; Kautz, H. Inferring High-Level Behavior from Low-Level Sensors. In Proceedings of the Fifth International Conference on Ubiquitous Computing, Seattle, WA, USA, 12–15 October 2003.
- Feng, T.; Timmermans, H.J. Transportation mode recognition using GPS and accelerometer data. Transp. Res. Part C Emerg. Technol. 2013, 37, 118–130. [Google Scholar] [CrossRef]
- Rudloff, C.; Ray, M. Detecting Travel Modes and Profiling Commuter Habits Solely Based on GPS Data. In Proceedings of the Transportation Research Board 89th Annual Meeting, Washington, DC, USA, 10–14 January 2010.
- Bolbol, A.; Cheng, T.; Tsapakis, I.; Haworth, J. Inferring hybrid transportation modes from sparse GPS data using a moving window SVM classification. Comput. Environ. Urban Syst. 2012, 36, 526–537. [Google Scholar] [CrossRef]
- Byon, Y.; Liang, S. Real-time transportation mode detection using smartphones and artificial neural networks: Performance comparisons between smartphones and conventional global positioning system sensors. J. Intell. Transp. Syst. 2014, 18, 264–272. [Google Scholar] [CrossRef]
- Byon, Y.; Abdulhai, B.; Shalaby, A. Real-time transportation mode detection via tracking global positioning system mobile devices. J. Intell. Transp. Syst. 2009, 13, 161–170. [Google Scholar] [CrossRef]
- Maruyama, T.; Mizokami, S.; Hato, E. A Smartphone-Based Travel Survey Trial Conducted in Kumamoto, Japan: An Examination of Voluntary Participants’ Attributes. In Proceedings of the Transportation Research Board 93rd Annual Meeting, Washington, DC, USA, 12–16 January 2014.
- Gilani, H. Automatically Determine Route and Mode of Transport Using a GPS Enabled Phone. Graduate Thesis and Dissertations, University of South Florida, Tampa, FL, USA, 2005. [Google Scholar]
- Bierlaire, M.; Chen, J.; Newman, J. A probabilistic map matching method for smartphone GPS data. Transp. Res. Part C Emerg. Technol. 2013, 26, 78–98. [Google Scholar] [CrossRef]
- Xiao, Y.; Low, D.; Bandara, T.; Pathak, P. In Transportation Activity Analysis Using Smartphones. In Proceedings of the Consumer Communications and Networking Conference (CCNC), Las Vegas, NV, USA, 14–17 January 2012.
- Moher, D.; Liberati, A.; Tetzlaff, J.; Altman, D.G.; Group, T.P. Preferred Reporting Items for Systematic Reviews and Meta-Analyses: The PRISMA Statement. Int. J. Surg. 2010, 8, 336–341. [Google Scholar] [CrossRef] [PubMed]
- Xiao, G.; Juan, Z.; Gao, J. Travel Mode Detection Based on Neural Networks and Particle Swarm Optimization. Information 2015, 6, 522–535. [Google Scholar] [CrossRef]
- Xiao, G.; Juan, Z.; Gao, J. Inferring Trip Ends from GPS Data Based on Smartphones in Shanghai. In Proceedings of the Transportation Research Board 94th Annual Meeting, Washington, DC, USA, 11–15 January 2015.
- Lari, Z.A.; Golroo, A. Automated Transportation Mode Detection Using Smart Phone Applications via Machine Learning: Case Study Mega City of Tehran. In Proceedings of the Transportation Research Board 94th Annual Meeting, Washington, DC, USA, 11–15 January 2015.
- Yang, F.; Yao, Z.; Jin, P.J. GPS and Acceleration Data in Multimode Trip Data Recognition Based on Wavelet Transform Modulus Maximum Algorithm. Transp. Res. Rec. J. Transp. Res. Board 2015, 2526, 90–98. [Google Scholar] [CrossRef]
- Nitsche, P.; Widhalm, P.; Breuss, S.; Brändle, N.; Maurer, P. Supporting large-scale travel surveys with smartphones—A practical approach. Transp. Res. Part C Emerg. Technol. 2014, 43, 212–221. [Google Scholar] [CrossRef]
- Stenneth, L.; Wolfson, O.; Yu, P.S.; Xu, B. In Transportation Mode Detection Using Mobile Phones and GIS Information. In Proceedings of the 19th ACM SIGSPATIAL International Conference on Advances in Geographic Information Systems, Chicago, IL, USA, 1–4 November 2011.
- Zhang, L.; Dalyot, S.; Eggert, D.; Sester, M. Multi-Stage Approach to Travel-Mode Segmentation and Classification of GPS Traces. In Proceedings of the ISPRS Guilin 2011 Workshop on International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences, Guilin, China, 20–21 October 2011.
- Gonzalez, P.A.; Weinstein, J.S.; Barbeau, S.J.; Labrador, M.A.; Winters, P.L.; Georggi, N.L.; Perez, R. Automating mode detection for travel behaviour analysis by using global positioning systems-enabled mobile phones and neural networks. IET Intell. Transp. Syst. 2010, 4, 37–49. [Google Scholar] [CrossRef]
- Reddy, S.; Mun, M.; Burke, J.; Estrin, D.; Hansen, M.; Srivastava, M. Using mobile phones to determine transportation modes. ACM Trans. Sens. Netw. 2010, 6, 662–701. [Google Scholar] [CrossRef]
- Zheng, Y.; Liu, L.; Wang, L.; Xie, X. Learning Transportation Mode from Raw GPS Data for Geographic Applications on the Web. In Proceedings of the International Conference on World Wide Web, Beijing, China, 21–25 April 2008.
- Zhan, G.; Yan, X.; Zhu, S.; Wang, Y. Using hierarchical tree-based regression model to examine university student travel frequency and mode choice patterns in China. Transp. Policy 2016, 45, 55–65. [Google Scholar] [CrossRef]
- Asad, K.; Wang, X.; Son, S.; Paul, A. Travel by University Students in Virginia: Is This Travel Different from Travel by the General Population? Transp. Res. Rec. 2011, 2255, 137–145. [Google Scholar]
|Assessing data collecting methodological quality||2–7|
|Study design||Travel survey||2|
|Adequate sample size selection||Included||1|
|Survey duration||More than 2 weeks||2|
|less than 2 weeks||1|
|Overcoming drawbacks of measures||Included (e.g., the short battery lives, the signal loss)||1|
|Ground truth||Included (e.g., prompted recall survey)||1|
|Assessing data processing methodological quality||0–5|
|Data preprocessing methods||0–1|
|Data error recognition||Included||1|
|Trip/segment identification methods||0–2|
|Independent variables selection reliability testing||Testing||1|
|Parameter selection reliability testing||Testing||1|
|Travel mode detection methods||0–2|
|Independent variables selection reliability testing||Testing||1|
|Comparison of experimental results||Included||1|
|Lead Author (Year)||Location||Journal||Sample Size||Collection Period||Device||Technical Details||Processing Involved a|
|Xiao, G. 1 (2015)||Shanghai, China||Information||-||Mid-October 2013 to mid-July 2014||GPS-enabled smartphone||Random sampling; GPS-only survey;||TI, MD|
|Xiao, G. 2 (2015)||Shanghai, China||Computers, Environment and Urban Systems||-||Mid-October 2013 to mid-July 2014||GPS-enabled smartphone||Random sampling; GPS-only survey;||TI, MD|
|Xiao, G. (2015)||Shanghai, China||Transportation Research Board 94th Annual Meeting||-||Mid-October 2013 to late-May 2014||GPS-enabled smartphone||Random sampling; GPS-only survey; Every participant is required to attend the survey at last five days.||TI|
|Lari, Z.A. (2015)||Tehran, Iran||Transportation Research Board 94th Annual Meeting||35 participants (25 males and 10 females)||2 weeks||Smartphone equipped with GPS and accelerometer sensors;||Random sampling: running the application from 6 a.m. to 9 p.m.||MD|
|Yang, F. (2015)||Chengdu, China||Transportation Research Record: Journal of the Transportation Research Board||20 persons||-||Mobile phone||Volunteers are required to collect data about special multimode trips.||TI, MD|
|Nitsche, P. (2014)||Vienna, Austria||Transportation Research Part C: Emerging Technologies||15 volunteers||2 months||Android-based smartphone||Random sampling;||TI, MD|
|Byon, Y. (2014)||Toronto, Canada||Journal of Intelligent Transportation Systems||5 persons||100 Weekday (50 h data from conventional GPS data loggers and 50 h of GPS)||Smartphone||Random sampling in selected routes.||MD|
|Stenneth, L. (2011)||USA||Proceedings of the 19th ACM SIGSPATIAL International Conference on Advances in Geographic Information Systems||6 individuals (3 males and 3 females)||3 weeks||GPS-enabled mobile phone||Random sampling; GPS-only survey;||MD|
|Zhang, L. (2011)||Hanover City, Germany||Remote Sensing and Spatial Information Sciences||197sub-traces||-||Android-based smartphone||Random sampling; Tracer Android App has a travel-mode selection function.||SI, MD|
|Gonzalez, P.A. (2010)||USA||Intelligent Transport Systems||114 trips||-||GPS-enabled cell phone||Random sampling; GPS-only survey;||MD|
|Reddy, S. (2010)||USA||ACM Transactions on Sensor Networks (TOSN)||16 individuals (8 males and 8 females)||75 min||Nokia N95 equipped with GPS and acceleration sensors;||Random sampling; GPS-only survey; fifteen minutes of data for each of the five transportation modes;||MD|
|Zheng, Y. (2008)||Beijing, China||Proceedings of the 17th international conference on World Wide Web||45 persons||6 months||GPS phone and handheld GPS receivers||Random sampling; GPS-only survey;||SI, MD|
|Year||Lead Author||GPS Devices||Records’ Features Used for Error Recognition||Methods or Steps of Data Transforming|
|2015||Xiao, G.||GPS-enabled smartphone||Number of satellites, HDOP value, altitude value||Using three steps to implement data transforming|
|2015||Lari, Z.A.||Smartphone equipped with GPS and accelerometer sensors;||Maximum speed values of different modes||-|
|2014||Nitsche, P.||Android-based smartphone||-||Using the Kalman filter to preprocess the track data, transforming the data of tri-axial accelerometer|
|2011||Stenneth, L.||GPS-enabled mobile phone||The GPS accuracy, the change in speed||-|
|2011||Zhang, L.||Android-based smartphone||The use of smoothing method||The values of speed and heading|
|2010||Reddy, S.||Nokia N95 equipped with GPS and acceleration sensors||The accuracy (vertical, horizontal, heading and speed), dilution of precision (time, vertical, horizontal), the changes in speed values of single and accelerometer sampling frequency considered||-|
|2015||Xiao, G.||Hybrid method 1||Critical length, critical distance, dwell time||96.02%|
|2015||Yang, F.||The wavelet transform modules maximum algorithm||Modulus maxima lines||95%|
|2014||Nitsche, P.||Hybrid method||Speed threshold, high amplitudes accelerometer signal,||-|
|2011||Zhang, L.||Hybrid method||Small speed values, small change in position, large magnitude in heading change||-|
|2008||Zheng, Y.||Hybrid method||Change point, uniform duration, and uniform length||-|
|Lead Author||Methods/Steps||Input Variables||Travel Modes||Sample Size 1||Training and Testing Sample||Accuracy||Ground Truth|
|Gonzalez, P. A. (2010)||Neural Networks||For all GPS points case: average speed, maximum speed, estimated horizontal accuracy uncertainty, percent Cell-ID fixes, standard deviation of distances between stop locations and average dwell time For only critical points case: average acceleration, maximum acceleration, average speed, maximum speed, ratio of the number of critical points over the total distance of the trip, ratio of the number of critical points over the total time of the trip, total distance, and average distance between critical points||Car, Bus, and Walking||114 trips||10-fold cross-validation.||88.6% (all GPS points) 91.23% (critical points-only dataset)||The travel modes were manually noted by research team.|
|Yang, F. (2015)||(WTMM algorithm used for the TI) Neural Network||Average speed, maximum speed, standard deviation of speed, and standard deviation of acceleration||Walking, Bicycle, Bus, and Car||20 persons||-||More than 86%||-|
|Byon, Y. (2014)||Neural Networks||Speed, accelerometer, magnetometer, and of satellites||Auto, Bus, streetcar, Bike, Walking||5 persons, 100 Weekdays||-||Accuracy of travel mode detection with the smartphone in different conditions is higher than the accuracy with the datalogger.||-|
|Xiao, G. (2015)||(hybrid method used for the TI) Neural Networks and Particle Swarm Optimization (PSO-NNs)||Low-speed point rate, travel distance, average speed, average absolute acceleration, median speed, and 95% percentile speed||Walking, Bike, Bus, and Car||-||1240 segments as training set; 414 segments as testing set.||95.81% (training set) 94.44% (test set)||Prompted recall survey.|
|Xiao, G. (2015)||(hybrid method used for the TI) Bayesian Networks||travel distance, average speed, average absolute acceleration, 95% percentile speed, low speed rate, and average heading change||Walking, Bike, E-bike, Bus, and Car||-||1240 segments as training set; 414 segments as testing set.||94.74% (training set) 92.74% (test set)||Prompted recall survey.|
|Lari, Z.A. (2015)||Random Forest||Speed, accuracy, delta bearing, delta speed, acceleration, and delta acceleration||Car, Bus, and Walking||35 participants, 2 weeks||30% and 70% of data for testing and training.||Almost 96%||Users attach to each GPS file.|
|Stenneth, L. (2011)||Bayesian Net (BN), Decision Tree (DT), Random Forest (RF), Naïve Bayesian (NB), and Multilayer Perceptron (ML)||Average bus location closeness, candidate bus location closeness, average rail line trajectory closeness, bus stop closeness rate, average accuracy of GPS coordinates, average speed, average heading change, and average acceleration||Car, Bus, Aboveground Train, Walking, Bike, and Stationary||6 persons, 3 weeks||10-fold cross-validation.||92.5% (BN), 92.2% (DT), 93.7% (RF), 91.6% (NB), 83.3% (ML)||Travel modes were labeled in sensor reports.|
|Zheng, Y. (2008)||(hybrid method used for the SI) Decision Tree(DT), Bayesian Net(BN), Support Vector Machine (SVM) and Conditional Random Field (CRF)||Length, mean velocity, expectation of velocity, top three velocity and top three accelerations from each segment||Walking, Car, Bus, and Bike||45 persons, 6 months||30% and 70% of data for testing and training.||74% (DT), 70% (BN), 59% (SVM), 47% (CRF)||Prompted recall survey|
|Nitsche, P. (2014)||(hybrid method used for the TI) An ensemble of probabilistic classifiers combined with a Discrete Hidden Markov Model (DHMM)||5th, 50th and 95th percentile of speed, accelerations, decelerations, direction change, standard deviation of the high-frequency accelerometer magnitudes, and power Spectrum of the accelerometer signal for frequencies i/128 Hz with i = 1, ..., 64 and the sampling frequency = 50 Hz||Walking, Bicycle, Motorcycle, Car, Bus, Electric Tramway, Metro, and Train||15 volunteers, 2 months||-||Range from 65% (train, subway) to 95% (bicycle)||The current transport modes were annotated by the volunteers during travel.|
|Zhang, L. (2011)||(hybrid method used for the SI) Two-stage approach, and Support Vector Machines (SVMs) used in second stage||In the first stage: mean speed, maximum speed, and heading related changes; In the second stage: mean and standard deviation of maximum speed, mean and standard deviation of average speed, mean and standard deviation of average acceleration, mean and standard deviation of travel time, mean and standard deviation of acceleration, and ratio of stop time in respect to travel time||Walking, Bicycle, Car, Bus, Tram, and Train||197 sub-traces,||83 sub-trace as training data; 54 sub-trace as testing data.||93%||User can pick and modify travel modes in the Tracer APP.|
|Reddy, S. (2010)||Decision Tree followed by a first-order discrete Hidden Markov Model (DT-DHMM)||GPS speed, accelerometer variance, accelerometer DFT components from 1–3 Hz calculated||Still, Walking, Run, Bike, and Motor||16 individuals, 75 min||10-fold cross-validation.||93%||Experiment (i.e., mode known).|
|Criteria||Description||Score||Twelve Selected Papers||Percentage|
|Xiao 1||Xiao 2||Xiao 3||Lari||Yang||Byon||Nitsche||Stenneth||Zhang||Gonzalez||Reddy||Zheng|
|Assessing data collecting methodological quality|
|Study design||Travel survey||2||√ 4||√||√||√||√||√||√||√||66.7%|
|Adequate sample size selection||Included||1||0%|
|Survey duration||More than 2 weeks||2||√||√||√||√||√||√||√||√||66.7%|
|Less than 2week||1||√||√||√||√||33.3%|
|Overcoming drawbacks of measures||Included(e.g., the short battery lives, the signal loss)||1||√||√||√||√||√||√||√||√||66.7%|
|Ground truth||Included (e.g., prompted recall survey)||1||√||√||√||√||√||√||√||√||√||√||83.3%|
|Assessing data processing methodological quality|
|Data preprocessing methods|
|Data error recognition||Included||1||√||√||√||√||√||√||√||58.3%|
|Trip/segment identification methods|
|Independent variables selection reliability testing||Testing||1||√||√||√||√||√||√||50%|
|Parameter selection reliability testing||Testing||1||0%|
|Travel mode detection methods|
|Independent variables selection reliability testing 1||Testing||1||√||/ 5||√||√||√||√||√||√||√||72.7%|
|Comparison of experimental results||Included||1||√||/||√||√||√||√||√||√||√||72.7%|
|The total scores of each paper||10||8||10||7||3||4||7||8||6||4||4||8|
© 2016 by the authors; licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC-BY) license (http://creativecommons.org/licenses/by/4.0/).