An Application Study on Road Surface Monitoring Using DTW Based Image Processing and Ultrasonic Sensors
Abstract
:Featured Application
Abstract
1. Introduction
2. Materials and Methods
2.1. Hardware Setup
2.2. Automated Assessment of Potholes and Speed Bumps Using Imaging Processing
- Image augmentation (Algorithm steps 3 and 4),
- Photo categorization (Algorithm step 5),
- Extraction of visual properties of articles (Algorithm steps 6 to 11),
- Recognition and classification of irregularities by decision logic (Algorithm step 12) and
- Quantification (Algorithm step 13).
- Categorized video frames are input;
- The initial frame is selected;
- A blue channel is selected for the conversion of the preset 24-bit depth format into 8-bit depth format;
- Median filtering is used for photo augmentation;
- Application of weighted mean based adaptive thresholding is made to covert processed photos into a binary image wherein black pixels signify objects of concentration;
- Morphological erosion is used to add black pixels to link the slits in the binary photo;
- Morphological dilation is used to eradicate secluded black pixels or their minor cluster;
- Morphological erosion is used again to add black pixels to the binary photo;
- Connected element classification and chain coding methods are used to quantify the number of substances or areas of notice and evaluate the area (A) and perimeter (P) of each of the object/substance;
- Filter out all the objects/substances whose A = 192cm2 in the primary photo;
- Calculate standard deviation, circularity and average width of every remaining substance/object (critical objects, i.e., objects whose A ≥ 192 cm2);
- Categorize every substance/object into three kinds using heuristically derivative decision logic: Type (object) =
- (a)
- Pothole, if STD ≥ 12 & CIRC ≥ 0.14 & W ≥ 70 mm;
- (b)
- Speed Bump, if Convexity ≥ 5;
- (c)
- Unimportant Irregularity (UI), if otherwise;
- Save the visual frame with the extracted and enumerated data in its respective category type folder;
- Repeat steps 2 to 13 for all outstanding visual database;
- Finish.
3. Experimental Design and Location
4. Results and Discussion
4.1. Proposed Novel Framework
4.2. Image Processing Result
4.3. Manual Inspection
4.4. Validation of Proposed System with Other Techniques
5. Conclusions
Author Contributions
Funding
Acknowledgments
Conflicts of Interest
List of Abbreviations
GPS | Global Positioning System | PSD | Power Spectral Density |
DTW | Dynamic Time Warping | UI | Unimportant Irregularity |
HUD | Head Up Display | STD | Standard deviation |
IFS | Irregularity Frame Selection | TPR | True Positive Rate |
CIRC | Circularity | FPR | False Positive Rate |
W | Average width | FNR | False Negative Rate |
References
- Malik, Y.S. Road Accidents in India 2016; Ministry of Road Transport & Highways: Delhi, India, 2016.
- Guan, L.; Zou, M.; Wan, X.; Li, Y. Nonlinear Lamb Wave Micro-Crack Direction Identification in Plates with Mixed-Frequency Technique. Appl. Sci. 2020, 10, 2135. [Google Scholar] [CrossRef] [Green Version]
- Schnebele, E.; Tanyu, B.F.; Cervone, G.; Waters, N. Review of remote sensing methodologies for pavement management and assessment. Eur. Trans. Res. Rev. 2015, 7, 7. [Google Scholar] [CrossRef] [Green Version]
- Vittorio, A.; Rosolino, V.; Teresa, I.; Vittoria, C.M.; Vincenzo, P.G.; Francesco, D.M. Automated Sensing System for Monitoring of Road Surface Quality by Mobile Devices. Procedia-Soc. Behav. Sci. 2014, 111, 242–251. [Google Scholar] [CrossRef] [Green Version]
- Praticò, F.G.; Fedele, R.; Naumov, V.; Sauer, T. Detection and Monitoring of Bottom-Up Cracks in Road Pavement Using a Machine-Learning Approach. Algorithms 2020, 13, 81. [Google Scholar] [CrossRef] [Green Version]
- Vupparaboina, K.K.; Tamboli, R.R.; Shenu, P.M.; Jana, S. Laser-based detection and depth estimation of dry and water-filled potholes: A geometric approach. In Proceedings of the 2015 Twenty First National Conference on Communications (NCC), Mumbai, India, 27 February–1 March 2015; pp. 1–6. [Google Scholar]
- Mohan, P.; Padmanabhan, V.N.; Ramjee, R. Nericell: Rich Monitoring of Road and Traffic Conditions using Mobile Smartphones. In Proceedings of the 6th ACM Conference on Embedded Network Sensor Systems, Raleigh, NC, USA, 5–7 November 2008; p. 323. [Google Scholar]
- Eriksson, J.; Girod, L.; Hull, B.; Newton, R.; Madden, S.; Balakrishnan, H. The Pothole Patrol: Using a Mobile Sensor Network for Road Surface Monitoring. In Proceedings of the 6th International Conference on Mobile Systems, Applications, and Services, Breckenridge, CO, USA, 17-20 June 2008; p. 29. [Google Scholar] [CrossRef]
- Kang, B.; Choi, S. Pothole detection system using 2D LiDAR and camera. In Proceedings of the 2017 Ninth International Conference on Ubiquitous and Future Networks (ICUFN), Milan, Italy, 4–7 July 2017; pp. 744–746. [Google Scholar]
- Forrest, M.M.; Chen, Z.; Hassan, S.; Raymond, I.O.; Alinani, K. Cost Effective Surface Disruption Detection System for Paved and Unpaved Roads. IEEE Access 2018, 6, 48634–48644. [Google Scholar] [CrossRef]
- Singh, G.; Kumar, R.; Kashtriya, P. Detection of Potholes and Speed Breaker on Road. In Proceedings of the 2018 First International Conference on Secure Cyber Computing and Communication (ICSCCC), Jalandhar, India, 15–17 December 2018; pp. 490–495. [Google Scholar]
- Madli, R.; Hebbar, S.; Pattar, P.; Golla, V. Automatic Detection and Notification of Potholes and Humps on Roads to Aid Drivers. IEEE Sens. J. 2015, 15, 4313–4318. [Google Scholar] [CrossRef]
- Shivaleelavathi, B.G.; Yatnalli, V.; Chinmayi, S.Y.V.; Thotad, S. Design and Development of an Intelligent System for Pothole and Hump Identification on Roads. Int. J. Recent Technol. Eng. 2019, 8, 5294–5300. [Google Scholar] [CrossRef]
- Devapriya, W.; Babu, C.N.K.; Srihari, T. Real time speed bump detection using Gaussian filtering and connected component approach. In Proceedings of the 2016 World Conference on Futuristic Trends in Research and Innovation for Social Welfare (Startup Conclave), Coimbatore, India, 29 February 2016–1 March 2016; pp. 1–5. [Google Scholar]
- Chen, K.; Lu, M.; Fan, X.; Wei, M.; Wu, J. Road condition monitoring using on-board Three-axis Accelerometer and GPS Sensor. In Proceedings of the 2011 6th International ICST Conference on Communications and Networking in China (CHINACOM), Harbin, China, 17–19 August 2011; pp. 1032–1037. [Google Scholar]
- Arroyo, C.; Bergasa, L.M.; Romera, E. Adaptive fuzzy classifier to detect driving events from the inertial sensors of a smartphone. In Proceedings of the 2016 IEEE 19th International Conference on Intelligent Transportation Systems (ITSC), Janeiro, Brazil, 1–4 November 2016; pp. 1896–1901. [Google Scholar]
- Aljaafreh, A.; Alawasa, K.; Alja’afreh, S.; Abadleh, A. Fuzzy inference system for speed bumps detection using smart phone accelerometer sensor. J. Telecommun. Electron. Comput. Eng. 2017, 9, 133–136. [Google Scholar]
- Astarita, V.; Caruso, M.V.; Danieli, G.; Festa, D.C.; Giofrè, V.P.; Iuele, T.; Vaiana, R. A Mobile Application for Road Surface Quality Control: UNIquALroad. Procedia-Soc. Behav. Sci. 2012, 54, 1135–1144. [Google Scholar] [CrossRef] [Green Version]
- Gonzalez, R.C.; Woods, R.E.; Masters, B.R. Digital Image Processing, 3rd ed.; Tata McGraw-Hill Education: New York, NY, USA, 2009; Volume 14, ISBN 0201180758. [Google Scholar]
- Yu, B.X.; Yu, X. Vibration-Based System for Pavement Condition Evaluation. Appl. Adv. Technol. Transp. 2019, 1, 183–189. [Google Scholar]
- Han, S.; Liu, M.; Shang, W.; Qi, X.; Zhang, Z.; Dong, S. Timely and Durable Polymer Modified Patching Materials for Pothole Repairs in Low Temperature and Wet Conditions. Appl. Sci. 2019, 9, 1949. [Google Scholar] [CrossRef] [Green Version]
- Koch, C.; Brilakis, I. Pothole detection in asphalt pavement images. Adv. Eng. Inform. 2011, 25, 507–515. [Google Scholar] [CrossRef]
- Mohamed, A.; Fouad, M.M.M.; Elhariri, E.; El-Bendary, N.; Zawbaa, H.M.; Tahoun, M.; Hassanien, A.E. RoadMonitor: An Intelligent Road Surface Condition Monitoring System; Springer: Cham, Switzerland, 2015; pp. 377–387. [Google Scholar]
Component | Hardware Description |
---|---|
Arduino | Microcontroller board based on ATmega328 using an open-source, prototyping platform, having 14 digital input/output pins (6 can be used as PWM outputs), 6 analog inputs, 16 MHz crystal oscillator, USB connection, power jack, ICSP header, and a reset button. |
HY-SRF05 | Narrow acceptance angle, 5-pin male header with 2.54 mm (0.1") spacing, 35 mA active, Range: approximately to 4 m (12- feet), Power requirements: +5 VDC; Communication: Dimensions: 22 × 46 × 16 mm (0.81 × 1.8 × 0.6 in), positive TTL pulse, Operating temperature range: 0 to + 70 °C (+32 to +158 °F). Figure 4a Arduino with HY-SRF05. |
GPS Receiver | It is a satellite navigation system and is used to capture geographic location and time, irrespective of the weather conditions (Figure 4b). |
HUD | A mobile device that used to display the information on the windshield as shown in Figure 4c. |
Filters | Several filters like Speed, Separating Z-Axis, Moving Average, Band Pass, and Virtual Orientation are used. |
Technique Used | DTW + Ultrasonic Sensor + Image Processing |
---|---|
Location of Experiment | Noida City, UP, North India. |
OS of the Data Collection System | Windows 10 Pro |
Equipment used for Data Collection | Dell XPS 15z, HP Pavilion DV9000 |
Vehicle(s) used | Honda Amaze (2016) |
Distance Covered (km)/Travel time (hours) | (150–170 km/4 h) |
Detection | Location of potholes, speed bumps on the roads |
Scenario No. | Obstacle Type | Depth/Height (cm) | Latitude (°N) | Longitudinal (°E) |
---|---|---|---|---|
1 | B | 3.0 | 28.4215 | 77.5261 |
2 | B | 3.0 | 28.4216 | 77.5261 |
3 | P | 2.6 | 28.4215 | 77.5265 |
4 | B | 5.2 | 28.4214 | 77.5266 |
5 | B | 4.8 | 28.4214 | 77.5268 |
Scenario No. | Obstacle Type | Depth/Height (cm) | Latitude (°N) | Longitudinal (°E) |
---|---|---|---|---|
1 | P | 9.27 | 28.6186 | 77.5544 |
2 | B | 2.20 | 28.6243 | 77.5661 |
3 | B | 8.63 | 28.6543 | 77.5667 |
4 | P | 13.12 | 28.6722 | 77.5643 |
5 | P | 8.20 | 28.6932 | 77.5634 |
6 | P | 8.19 | 28.7423 | 77.5612 |
7 | B | 3.20 | 28.7865 | 77.5755 |
8 | P | 1.77 | 28.7933 | 77.5765 |
9 | B | 3.72 | 28.8654 | 77.5659 |
10 | P | 12.16 | 28.9653 | 77.5760 |
Scenario No. | Obstacle Type | Depth/Height (cm) | Latitude (°N) | Longitudinal (°E) |
---|---|---|---|---|
1 | B | 3.14 | 27.1937 | 76.5725 |
2 | B | 3.11 | 27.5405 | 76.4872 |
3 | P | 2.57 | 28.0691 | 76.4799 |
4 | B | 5.40 | 28.8080 | 76.6505 |
5 | B | 4.60 | 27.9979 | 77.1004 |
Scenario No. | Obstacle Type | Depth/Height (cm) | Latitude (°N) | Longitudinal (°E) |
---|---|---|---|---|
1 | P | 8.8695 | 27.4166 | 76.2903 |
2 | B | 2.1318 | 27.4507 | 75.7666 |
3 | B | 8.3590 | 27.6514 | 77.3884 |
4 | P | 12.9271 | 28.3281 | 76.5420 |
5 | P | 8.2082 | 27.7750 | 74.2999 |
6 | P | 7.8984 | 28.2824 | 75.7508 |
7 | B | 3.0682 | 28.0668 | 74.5035 |
8 | P | 1.7144 | 28.0620 | 75.3345 |
9 | B | 3.6010 | 27.9302 | 75.0760 |
10 | P | 11.6006 | 28.2817 | 75.0858 |
Scenario No. | Obstacle Type | Depth/Height (cm) | Latitude (°N) | Longitudinal (°E) |
---|---|---|---|---|
1 | B | 3.11 | 27.41 | 78.65 |
2 | B | 3.13 | 27.25 | 76.50 |
3 | P | 2.50 | 27.53 | 76.49 |
4 | B | 5.30 | 28.00 | 76.61 |
5 | B | 4.58 | 28.45 | 77.17 |
Scenario No. | Obstacle Type | Depth/Height (cm) | Latitude (°N) | Longitudinal (°E) |
---|---|---|---|---|
1 | P | 8.8 | 28.15 | 75.72 |
2 | B | 2.15 | 27.96 | 76.33 |
3 | B | 8.29 | 28.20 | 75.63 |
4 | P | 12.93 | 27.96 | 75.60 |
5 | P | 7.88 | 28.35 | 75.75 |
6 | P | 7.95 | 28.06 | 74.49 |
7 | B | 3.15 | 28.32 | 75.33 |
8 | P | 1.70 | 28.45 | 76.31 |
9 | B | 3.59 | 27.94 | 75.77 |
10 | P | 11.87 | 28.03 | 77.40 |
Scenario No. | Obstacle Type | Depth/Height (cm) | ||||
---|---|---|---|---|---|---|
Proposed System | Image Processing | Manual Inspection | % Error (PS vs. IP) | % Error (PS vs. MI) | ||
1 | B | 3.0 | 3.14 | 3.11 | 4.67 | 3.67 |
2 | B | 3.0 | 3.11 | 3.13 | 3.67 | 4.33 |
3 | P | 2.6 | 2.57 | 2.50 | −1.15 | −3.85 |
4 | B | 5.2 | 5.40 | 5.30 | 3.85 | 1.92 |
5 | B | 4.8 | 4.60 | 4.58 | −4.17 | −4.58 |
Scenario No. | Obstacle Type | Latitude (°N) | ||||
---|---|---|---|---|---|---|
Proposed System | Image Processing | Manual Inspection | % Error (PS vs. IP) | % Error (PS vs. MI) | ||
1 | B | 28.4215 | 27.1937 | 27.41 | 4.32 | 3.56 |
2 | B | 28.4216 | 27.5405 | 27.25 | 3.1 | 4.12 |
3 | P | 28.4215 | 28.0691 | 27.53 | 1.24 | 3.14 |
4 | B | 28.4214 | 28.8080 | 28.00 | −1.36 | 1.47 |
5 | B | 28.4214 | 27.9979 | 28.45 | 1.49 | −0.1 |
Scenario No. | Obstacle Type | Longitude (°E) | ||||
---|---|---|---|---|---|---|
Proposed System | Image Processing | Manual Inspection | % Error (PS vs. IP) | % Error (PS vs. MI) | ||
1 | B | 77.5261 | 76.5725 | 78.65 | 1.23 | -1.45 |
2 | B | 77.5261 | 76.4872 | 76.50 | 1.34 | 1.32 |
3 | P | 77.5265 | 76.4799 | 76.49 | 1.35 | 1.34 |
4 | B | 77.5266 | 76.6505 | 76.61 | 1.13 | 1.18 |
5 | B | 77.5268 | 77.1004 | 77.17 | 0.55 | 0.46 |
Scenario No. | Obstacle Type | Depth/Height (cm) | ||||
---|---|---|---|---|---|---|
Proposed System | Image Processing | Manual Inspection | % Error (PS vs. IP) | % Error (PS vs. MI) | ||
1 | B | 9.27 | 8.8695 | 8.88 | 4.32 | 4.21 |
2 | B | 2.20 | 2.1318 | 2.15 | 3.10 | 2.34 |
3 | P | 8.63 | 8.3590 | 8.29 | 3.14 | 3.96 |
4 | B | 13.12 | 12.9271 | 12.93 | 1.47 | 1.47 |
5 | B | 8.20 | 8.2082 | 7.88 | -0.10 | 3.96 |
6 | P | 8.19 | 7.8984 | 7.95 | 3.56 | 2.89 |
7 | B | 3.20 | 3.0682 | 3.15 | 4.12 | 1.63 |
8 | P | 1.77 | 1.7144 | 1.70 | 3.14 | 4.10 |
9 | B | 3.72 | 3.6010 | 3.59 | 3.20 | 3.50 |
10 | P | 12.16 | 11.6006 | 11.87 | 4.60 | 2.36 |
Scenario No. | Obstacle Type | Latitude (°N) | ||||
---|---|---|---|---|---|---|
Proposed System | Image Processing | Manual Inspection | % Error (PS vs. IP) | % Error (PS vs. MI) | ||
1 | B | 28.6186 | 27.4166 | 28.15 | 4.20 | 1.63 |
2 | B | 28.6243 | 27.4507 | 27.96 | 4.10 | 2.32 |
3 | P | 28.6543 | 27.6514 | 28.20 | 3.50 | 1.60 |
4 | B | 28.6722 | 28.3281 | 27.96 | 1.20 | 2.50 |
5 | B | 28.6932 | 27.7750 | 28.35 | 3.20 | 1.20 |
6 | P | 28.7423 | 28.2824 | 28.06 | 1.60 | 2.36 |
7 | B | 28.7865 | 28.0668 | 28.32 | 2.50 | 1.63 |
8 | P | 28.7933 | 28.0620 | 28.45 | 2.54 | 1.20 |
9 | B | 28.8654 | 27.9302 | 27.94 | 3.24 | 3.20 |
10 | P | 28.9653 | 28.2817 | 28.03 | 2.36 | 3.24 |
Scenario No. | Obstacle Type | Longitude (°E) | ||||
---|---|---|---|---|---|---|
Proposed System | Image Processing | Manual Inspection | % Error (PS vs. IP) | % Error (PS vs. MI) | ||
1 | B | 77.5544 | 76.2903 | 75.72 | 1.63 | 2.36 |
2 | B | 77.5661 | 75.7666 | 76.33 | 2.32 | 1.60 |
3 | P | 77.5668 | 77.3884 | 75.63 | 0.23 | 2.50 |
4 | B | 77.5669 | 76.5430 | 75.60 | 1.32 | 2.54 |
5 | B | 77.5654 | 74.2999 | 75.75 | 4.21 | 2.34 |
6 | P | 77.5658 | 75.7508 | 74.49 | 2.34 | 3.96 |
7 | B | 77.5755 | 74.5035 | 75.33 | 3.96 | 2.89 |
8 | P | 77.5765 | 75.3345 | 76.31 | 2.89 | 1.63 |
9 | B | 77.5659 | 75.0760 | 75.77 | 3.21 | 2.32 |
10 | P | 77.5760 | 75.0858 | 77.40 | 3.21 | 0.23 |
Author | Detection Rate | Approach |
---|---|---|
Devapriya et al. [14] | 30–92% TPR | Computer Vision Accelerometer & GPS |
Eriksson et al. [8] | 0.2 FPR | |
Mohan et al. [7] | 11.1% FPR & 22% FNR | Accelerometer, Microphone, GPS & GSM |
Forrest et al. [10] | 62% Accuracy | Ultrasonic |
Singh et al. [11] | 59.23% Accuracy | Global positioning system receiver and ultrasonic |
Madli et al. [12] | 74% Accuracy | Ultrasonic and GPS |
Shivaleelavathi et al. [13] | 79% Accuracy | Raspberry Pi, GPS, ultrasonic |
Mohamed et al. [23] | 75.6–87.8% (Accuracy) | Accelerometer |
Astarita et al. [18] | 90% Accuracy, 35% FP | Accelerometer using Smart Phone |
Proposed system | 95.48% (Accuracy) | Ultrasonic Sensor, Image Processing and manual Inspection |
© 2020 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (http://creativecommons.org/licenses/by/4.0/).
Share and Cite
Sharma, S.K.; Phan, H.; Lee, J. An Application Study on Road Surface Monitoring Using DTW Based Image Processing and Ultrasonic Sensors. Appl. Sci. 2020, 10, 4490. https://doi.org/10.3390/app10134490
Sharma SK, Phan H, Lee J. An Application Study on Road Surface Monitoring Using DTW Based Image Processing and Ultrasonic Sensors. Applied Sciences. 2020; 10(13):4490. https://doi.org/10.3390/app10134490
Chicago/Turabian StyleSharma, Sunil Kumar, Haidang Phan, and Jaesun Lee. 2020. "An Application Study on Road Surface Monitoring Using DTW Based Image Processing and Ultrasonic Sensors" Applied Sciences 10, no. 13: 4490. https://doi.org/10.3390/app10134490
APA StyleSharma, S. K., Phan, H., & Lee, J. (2020). An Application Study on Road Surface Monitoring Using DTW Based Image Processing and Ultrasonic Sensors. Applied Sciences, 10(13), 4490. https://doi.org/10.3390/app10134490