Automated Lane Centering: An Off-the-Shelf Computer Vision Product vs. Infrastructure-Based Chip-Enabled Raised Pavement Markers
Abstract
:1. Introduction
- Implementing the initial proof-of-concept CERPMs on actual roads;
- Real-time data processing and vehicle integration for LC;
- Comparing V2I against traditional on-vehicle lane line detection methods for LC;
- On-road testing for vehicle control using CERPMs for LC.
2. Methodology
2.1. AV Subsystem: Perception
2.1.1. On-Vehicle Sensor: GNSS
2.1.2. On-Vehicle Sensor: Mobileye Camera
- Setup
- Data Routing for AV System
- Data Processing
- Preparation for Lane Centering (LC) Using Mobileye
2.2. Chip Enabled Raised Pavement Markers
- Component Selection and Communication Protocol
- CERPM Setup
- Data Routing for AV System
- Data Processing
- Pseudo Lane-Line Projection for Visual Verification
- Preparation for Lane Centering (LC) Using CERPMs
2.3. AV Subsystem: Controls
2.3.1. Fixed Longitudinal Controller
2.3.2. Lateral Controller
- LC Using Mobileye
- LC Using CERPM
3. Test Routes
3.1. Route 1: Oak Ridge National Laboratory
3.2. Route 2: WMU Campus Drive Loop Steep Curvature
3.3. Route 3: WMU Campus Drive Low Curvature
4. Results
4.1. Signal Strength Analysis
4.2. Steep Curvature
4.2.1. Mobileye
4.2.2. CEPRM
4.2.3. LC Performance
4.3. Low Curvature
4.3.1. Mobileye
4.3.2. CERPM
4.3.3. LC Performance
5. Discussion
6. Conclusions
Author Contributions
Funding
Data Availability Statement
Acknowledgments
Conflicts of Interest
Abbreviations
AV | Autonomous Vehicles |
ADAS | Advanced Driver Assistance Systems |
HD | High Definition |
RSU | Road Side Units |
IIS | Infrastructure Information Source |
CERPM | Chip-Enabled Raised Pavement Markers |
LPWAN | Low-Power Wide-Area Network |
LC | Lane Centering |
V2I | Vehicle-to-Infrastructure |
RADAR | Radio Detection and Ranging |
GNSS | Global Navigation Satellite System |
LiDAR | Light Detection and Ranging |
LKA | Lane Keeping Assist |
AEB | Automated Emergency Braking |
NHTSA | National Highway Traffic Safety Administration |
RPM | Raised Pavement Markers |
ISM | Industrial Scientific and Medical |
CDA | Cooperative Driving Automation |
CAN | Controller Area Network |
ROS | Robotic Operating System |
RTK | Real-Time Kinematics |
CMOS | Complementary Metal-Oxide-Semiconductor |
HDRC | High Dynamic Range Camera |
SDK | Software Development Kit |
LoRa | Long-Range |
Rx | Receiver |
Tx | Transmitter |
RSSI | Receiver Signal Strength Indicator |
WGS | World Geodetic System |
IoT | Internet of Things |
NED | North-East-Down |
PID | Proportional-Integral-Derivative |
SISO | Single-Input Single-Output |
DARPA | Defense Advanced Research Projects Agency |
MDPS | Motor-Driven Power Steering |
EEAV | Energy-Efficient Autonomous Vehicles |
WMU | Western Michigan University |
ORNL | Oak Ridge National Laboratory |
USDOT | US Department of Transportation |
dBm | Decibel Milliwatts |
MSE | Mean Squared Error |
References
- Pendleton, S.D.; Andersen, H.; Du, X.; Shen, X.; Meghjani, M.; Eng, Y.H.; Rus, D.; Ang, M.H. Perception, Planning, Control, and Coordination for Autonomous Vehicles. Machines 2017, 5, 6. [Google Scholar] [CrossRef]
- Nidamanuri, J.; Nibhanupudi, C.; Assfalg, R.; Venkataraman, H. A Progressive Review: Emerging Technologies for ADAS Driven Solutions. IEEE Trans. Intell. Veh. 2022, 7, 326–341. [Google Scholar] [CrossRef]
- Advanced Driver Assistance Systems: Global Revenue Growth 2020–2023. Available online: https://www.statista.com/statistics/442726/global-revenue-growth-trend-of-advanced-driver-assistance-systems/ (accessed on 24 April 2023).
- Roadmap for Safer Vehicles 2030—A Global Perspective Panel. Available online: https://www.nhtsa.gov/speeches-presentations/roadmap-safer-vehicles-2030-global-perspective-panel (accessed on 4 December 2023).
- J3016_202104: Taxonomy and Definitions for Terms Related to Driving Automation Systems for On-Road Motor Vehicles—SAE International. Available online: https://www.sae.org/standards/content/j3016_202104/ (accessed on 21 December 2023).
- Sudhakaran, A. Lateral Control Using a MobilEye Camera for Lane Keeping Assist. Available online: https://assets.w3.tue.nl/w/fileadmin/content/faculteiten/wtb/Onderzoek/Onderzoeksgroepen/Dynamics_and_Control/Internship_reports/Sudhakaran_DC_2017.003.pdf (accessed on 4 December 2023).
- Chang, T.H.; Hsu, C.S.; Wang, C.; Yang, L.K. Onboard Measurement and Warning Module for Irregular Vehicle Behavior. IEEE Trans. Intell. Transp. Syst. 2008, 9, 501–513. [Google Scholar] [CrossRef]
- Chen, L.; Xu, X.; Pan, L.; Cao, J.; Li, X. Real-time lane detection model based on non bottleneck skip residual connections and attention pyramids. PLoS ONE 2021, 16, e0252755. [Google Scholar] [CrossRef] [PubMed]
- Masello, L.; Castignani, G.; Sheehan, B.; Murphy, F.; McDonnell, K. On the road safety benefits of advanced driver assistance systems in different driving contexts. Transp. Res. Interdiscip. Perspect. 2022, 15, 100670. [Google Scholar] [CrossRef]
- Kadav, P.; Sharma, S.; Araghi, F.M.; Asher, Z.D. Development of Computer Vision Models for Drivable Region Detection in Snow Occluded Lane Lines. In Machine Learning and Optimization Techniques for Automotive Cyber-Physical Systems; Kukkala, V.K., Pasricha, S., Eds.; Springer International Publishing: Cham, Switzerland, 2023; pp. 591–623. [Google Scholar]
- Utah Connected Webinar Series—UDOT. 2023. Available online: https://transportationtechnology.utah.gov/utah-connected-webinar-series/ (accessed on 21 December 2023).
- Zang, S.; Ding, M.; Smith, D.; Tyler, P.; Rakotoarivelo, T.; Kaafar, M.A. The Impact of Adverse Weather Conditions on Autonomous Vehicles: How Rain, Snow, Fog, and Hail Affect the Performance of a Self-Driving Car. IEEE Veh. Technol. Mag. 2019, 14, 103–111. [Google Scholar] [CrossRef]
- Olariu, C.; Ortega, J.D.; Javier Yebes, J. The role of cloud-computing in the development and application of ADAS. In Proceedings of the 2018 26th European Signal Processing Conference (EUSIPCO), Rome, Italy, 3–7 September 2018; pp. 1037–1041. [Google Scholar]
- Sharma, S.; Rojas, J.F.; Ekti, A.R.; Wang, C.r.; Asher, Z.; Meyer, R. Vehicle Lateral Offset Estimation Using Infrastructure Information for Reduced Compute Load; Technical Report 2023-01-0800, SAE Technical Paper; SAE International: Warrendale, PA, USA, 2023. [Google Scholar]
- Stephens, D.; Schroeder, J.; Klein, R.; Battelle Memorial Institute. Vehicle-to-Infrastructure (V2I) Safety Applications: Performance Requirements, Volume 1, Introduction and Common Requirements; Technical Report FHWA-JPO-16-248; U.S. Department of Transportation: Washington, DC, USA, 2015.
- Nayak, A.; Pike, A.; Rathinam, S. Effect of Pavement Markings on Machine Vision Used in ADAS Functions; SAE Technical Report 2022-01-0154; Texas A & M University: College Station, TX, USA, 2022. [Google Scholar]
- Brown, N.E.; Motallebiaraghi, F.; Rojas, J.F.; Ayantayo, S.; Meyer, R.; Asher, Z.D.; Ekti, A.R.; Wang, C.R.; Goberville, N.A.; Feinberg, B. Evaluation of Autonomous Vehicle Sensing and Compute Load on a Chassis Dynamometer. In Proceedings of the 2023 IEEE 26th International Conference on Intelligent Transportation Systems (ITSC), Bilbao, Spain, 24–28 September 2023; pp. 1989–1995. [Google Scholar]
- Lee, E.S.; Vora, A.; Parchami, A.; Chakravarty, P.; Pandey, G.; Kumar, V. Infrastructure Node-based Vehicle Localization for Autonomous Driving. arXiv 2021, arXiv:2109.10457. [Google Scholar]
- Miucic, R.; Rajab, S. Computer Vision-Based V2X Collaborative Perception; Technical Report 2022-01-0073, SAE Technical Paper; SAE International: Warrendale, PA, USA, 2022. [Google Scholar]
- Guidelines for the Use of Raised Pavement Markers. Available online: https://www.fhwa.dot.gov/publications/research/safety/97152/ch02.cfm (accessed on 21 December 2023).
- Mobileye 6 Series Collision Avoidance System. Available online: https://ims.mobileye.com/fleets/au/products/mobileye-6-collision-avoidance-system/ (accessed on 10 January 2024).
- Sharma, S.; Ekti, A.R.; Rojas, J.F.; Brown, N.E.; Pesin, D.; Wang, C.R.; Huff, S.; LaClair, T.J.; Asher, Z.D.; Meyer, R.T. Development and Evaluation of Chip-Enabled Raised Pavement Markers for Lane Line Detection. In Proceedings of the 2022 IEEE Sensors, Dallas, TX, USA, 30 October–2 November 2022; pp. 1–4. [Google Scholar]
- Kvaser USBcan Light 2xHS. Available online: https://www.kvaser.com/product/kvaser-usbcan-light-2xhs/ (accessed on 6 December 2023).
- Whitley, J. Mobileye 560/660—AutonomouStuff ROS Wiki—AutonomouStuff Confluence. Available online: https://autonomoustuff.atlassian.net/wiki/spaces/RW/pages/17478861/Mobileye+560+660 (accessed on 6 December 2023).
- WiFi LoRa 32(V3). Available online: https://heltec.org/project/wifi-lora-32-v3/ (accessed on 7 December 2023).
- AEACAQ190012-S915. Available online: https://www.digikey.com/en/products/detail/abracon-llc/AEACAQ190012-S915/8593253 (accessed on 7 December 2023).
- OpenCV: Camera Calibration and 3D Reconstruction. Available online: https://docs.opencv.org/4.x/d9/d0c/group__calib3d.html (accessed on 8 December 2023).
- Carow, K.; Kadav, P.; Patil, P.; Rojas, J.F.; Asher, Z. Projecting Lane Lines from Proxy High-Definition Maps for Automated Vehicle Perception in Road Occlusion Scenarios; Technical Report 2023-01-0051, SAE Technical Paper; Western Michigan University: Kalamazoo, MI, USA, 2023. [Google Scholar]
- Thrun, S.; Montemerlo, M.; Dahlkamp, H.; Stavens, D.; Aron, A.; Diebel, J.; Fong, P.; Gale, J.; Halpenny, M.; Hoffmann, G.; et al. Stanley: The robot that won the DARPA Grand Challenge. J. Field Robot. 2006, 23, 661–692. [Google Scholar] [CrossRef]
- Peiris, S.; Newstead, S.; Berecki-Gisolf, J.; Chen, B.; Fildes, B. Quantifying the Lost Safety Benefits of ADAS Technologies Due to Inadequate Supporting Road Infrastructure. Sustain. Sci. Pract. Policy 2022, 14, 2234. [Google Scholar] [CrossRef]
- Pavement Markings Regulations/Standards. Available online: https://highways.dot.gov/safety/other/visibility/pavement-markings-regulations-standards (accessed on 7 December 2023).
- Babić, D.; Babić, D.; Fiolić, M.; Eichberger, A.; Magosi, Z.F. A Comparison of Lane Marking Detection Quality and View Range between Daytime and Night-Time Conditions by Machine Vision. Energies 2021, 14, 4666. [Google Scholar] [CrossRef]
- Stojanović, B.; Hofer-Schmitz, K. Formal Methods for Connected Vehicle Protocols. In Proceedings of the 2019 27th Telecommunications Forum (TELFOR), Belgrade, Serbia, 26–27 November 2019; pp. 1–4. [Google Scholar]
- Marcillo, P.; Tamayo-Urgilés, D.; Valdivieso Caraguay, Á.L.; Hernández-Álvarez, M. Security in V2I Communications: A Systematic Literature Review. Sensors 2022, 22, 9123. [Google Scholar] [CrossRef] [PubMed]
Type | CERPM | Mobileye |
---|---|---|
Test Route 2 MSE | 0.42 m | N/A 1 |
Test Route 3 MSE | 0.38 m | 0.41 m |
Disclaimer/Publisher’s Note: The statements, opinions and data contained in all publications are solely those of the individual author(s) and contributor(s) and not of MDPI and/or the editor(s). MDPI and/or the editor(s) disclaim responsibility for any injury to people or property resulting from any ideas, methods, instructions or products referred to in the content. |
© 2024 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https://creativecommons.org/licenses/by/4.0/).
Share and Cite
Kadav, P.; Sharma, S.; Fanas Rojas, J.; Patil, P.; Wang, C.; Ekti, A.R.; Meyer, R.T.; Asher, Z.D. Automated Lane Centering: An Off-the-Shelf Computer Vision Product vs. Infrastructure-Based Chip-Enabled Raised Pavement Markers. Sensors 2024, 24, 2327. https://doi.org/10.3390/s24072327
Kadav P, Sharma S, Fanas Rojas J, Patil P, Wang C, Ekti AR, Meyer RT, Asher ZD. Automated Lane Centering: An Off-the-Shelf Computer Vision Product vs. Infrastructure-Based Chip-Enabled Raised Pavement Markers. Sensors. 2024; 24(7):2327. https://doi.org/10.3390/s24072327
Chicago/Turabian StyleKadav, Parth, Sachin Sharma, Johan Fanas Rojas, Pritesh Patil, Chieh (Ross) Wang, Ali Riza Ekti, Richard T. Meyer, and Zachary D. Asher. 2024. "Automated Lane Centering: An Off-the-Shelf Computer Vision Product vs. Infrastructure-Based Chip-Enabled Raised Pavement Markers" Sensors 24, no. 7: 2327. https://doi.org/10.3390/s24072327