Transforming Petrochemical Safety Using a Multimodal AI Visual Analyzer †
Abstract
:1. Introduction
2. Methodology
2.1. Selection of Multimodal AI Systems
2.2. Data Acquisition and Preparation
- PPE incompliance;
- Confined space entry;
- Hot work;
- Working at heights;
- Handling hazardous chemicals;
- Slip, trip, and fall hazards (general).
2.3. Performance Evaluation
- Hazard Detection Accuracy (True Positives, False Positives): Did the model correctly identify hazards present in the ground truth? Did it incorrectly identify hazards that were not present?
- Hazard Miss Rate (False Negatives): Did the model fail to identify hazards that were present according to SMEs?
- Categorization Correctness: Was the identified hazard assigned to the appropriate category?
- Justification Quality: Was the model’s reasoning sound and did it align with safety standards?
- Overall Relevance: Was the output focused and free from irrelevant information?
3. Results and Discussion
- Gemini 1.5 Pro (95/100): Receives the highest score due to its exceptional accuracy, comprehensive hazard identification, nuanced understanding, and detailed analysis. It consistently demonstrated a strong ability to identify and articulate safety concerns effectively. The minor limitation of potential over-interpretation is often acceptable and even beneficial in a safety context as it errs on the side of caution.
- GPT-4 (ChatGPT) (90/100): Scores very high for accuracy and conciseness. It effectively identifies the primary safety concerns in a clear and actionable manner. While slightly less detailed than Gemini, its ability to provide focused and relevant safety information is highly valuable. It is a very strong performer in hazard identification.
- Copilot (70/100): Scores lower in direct comparison to Gemini and GPT-4 for image-based hazard identification in these specific examples. While it offers a useful and structured approach to thinking about safety through categories, it is less effective at autonomously identifying specific hazards directly from the images. Its strength lies more in providing general safety frameworks and advice, requiring more user interaction to pinpoint image-specific risks. If the goal is to achieve a quick, automated hazard assessment from an image, it is less efficient in its out-of-the-box behavior compared to the other two models based on these examples.
4. Practical Implications
- Reduce accidents and near-miss incidents through real-time detection.
- Ensure compliance with OSHA standards and enhance the implementation of DuPont’s process safety guidelines.
- Improve operational efficiency by minimizing downtime related to safety incidents, as early detection can lead to quicker mitigation efforts.
5. Conclusions
Author Contributions
Funding
Institutional Review Board Statement
Informed Consent Statement
Data Availability Statement
Acknowledgments
Conflicts of Interest
References
- Knegtering, B.; Pasman, H.J. Safety of the process industries in the 21st century: A changing need of process safety management for a changing industry. J. Loss Prev. Process Ind. 2009, 22, 162–168. [Google Scholar] [CrossRef]
- Saisandhiya, N.R.; Vijay Babu, K. Hazard identification and risk assessment in petrochemical industry. Int. J. Res. Appl. Sci. Eng. Technol. 2020, 8, 778–783. [Google Scholar] [CrossRef]
- Shukla, A.; Karki, H. Application of robotics in offshore oil and gas industry—A review Part II. Robot. Auton. Syst. 2016, 75, 508–524. [Google Scholar] [CrossRef]
- Shahriar, S.; Lund, B.D.; Mannuru, N.R.; Arshad, M.A.; Hayawi, K.; Bevara, R.V.K.; Mannuru, A.; Batool, L. Putting GPT-4o to the Sword: A Comprehensive Evaluation of Language, Vision, Speech, and Multimodal Proficiency. Appl. Sci. 2024, 14, 7782. [Google Scholar] [CrossRef]
- Ma, J. The Application of Artificial Intelligence Technology in the Safety Monitoring System of Oil and Gas Ground. Procedia Comput. Sci. 2023, 228, 486–493. [Google Scholar] [CrossRef]
- Glass, H. Construction continues to struggle badly with output at its ‘lowest level in 13 years’. This Is Money, 9 November 2012. Available online: https://www.thisismoney.co.uk/money/news/article-2230500/Construction-continues-struggle-badly-output-lowest-level-13-years.html (accessed on 18 September 2024).
Hazard Class | Number of Instances | Percentage (%) | Categorized Hazard Types Within Class |
---|---|---|---|
| 63 | 25.2 | Head Protection: No hard hat, damaged hard hat. Eye Protection: No safety glasses, improper eyewear. Hand Protection: No gloves, incorrect glove type. Respiratory Protection: No respirator, improperly worn respirator. |
| 38 | 15.2 | Procedural Hazards: No entry permit displayed, lack of lockout/tagout. Atmospheric Hazards: No ventilation setup, missing gas detectors, obstructed entry/exit. |
| 50 | 20 | Fall Protection: No safety harness, improperly worn harness, damaged lanyard/lifeline. Equipment Hazards: Unsafe scaffolding, lack of guardrails, unstable ladder. |
| 33 | 13.2 | Spillage/Leakage: Chemical spills, leaking containers, pooling liquids. Container Issues: Unlabeled containers, corroded drums, improper storage. Ventilation: Lack of ventilation in chemical areas. |
| 29 | 11.6 | Environmental Clutter: Cluttered walkways, tools/equipment obstructing paths. Surface Hazards: Spills on floors, wet/slippery surfaces, uneven flooring. Visibility: Poor lighting, obstructed view. |
| 37 | 14.8 | Ignition Sources: Open flames near flammables, sparks from welding/grinding, overheated equipment. Flammable Materials: Improper storage of flammables, combustible materials near heat sources. Emergency Access: Blocked fire exits, missing fire extinguishers. |
Safety Concern | Gemini 1.5 Pro | GPT-4 (ChatGPT) | Copilot |
---|---|---|---|
Eye Hazard | Sparks and Flying Debris | Inadequate Eye Protection | PPE (General) |
Respiratory Hazard | Metal Dust and Fumes | Metal Dust and Fumes | N/A |
Noise Hazard | Chop Saw Noise | N/A | N/A |
Fire Hazard | Sparks, Flammable Materials | Fire Hazard (Sparks) | Fire Safety Issues (General) |
Ergonomic Hazard | Awkward Posture | Unsafe Work Position | Ergonomic Concerns (General) |
Kickback Hazard | Kickback Hazard | N/A | N/A |
Hazard Type | Gemini 1.5 Pro | GPT-4 (ChatGPT) | Copilot | Justification |
---|---|---|---|---|
| 95 | 90 | 75 | Gemini: Excellent at identifying specific PPE deficiencies (hard hat, eye protection, gloves, respiratory, boots, hi-vis vest). GPT-4: Very good, but slightly less detailed in PPE specifics across all images compared to Gemini. Copilot: Acknowledges PPE importance in its framework, but less direct in pinpointing specific PPE non-compliance in each image’s analysis. Framework is more general, less image-specific in initial responses. |
| 98 | 95 | 80 | Gemini: Comprehensive understanding of confined space hazards. Identified harness, respirator (likely SAR), but also gaps like atmospheric monitoring, communication, permits, rescue plan. GPT-4: Strong on permit, ventilation, rescue, gas detection, harness—key confined space elements. Slightly less detail than Gemini. Copilot: Framework addresses “Working at Heights” more than “Confined Space” specifically in the initial output, but general safety categories are relevant. Less targeted hazard ID for confined spaces from the image directly. |
| 90 | 85 | 70 | Gemini: Good on ladder, mezzanine guardrails. Identified ladder instability, lack of fall protection on mezzanine. GPT-4: Focused on elevated platform, ladder use, overhead crane—relevant height aspects. Copilot: Category-based approach addresses “Working at Heights” as a general category, but less specific hazard identification from the warehouse image example provided. Less direct linkage to specific image elements in initial response. |
| 92 | 90 | 70 | Gemini: Excellent in a Hazmat suit scenario, identified unknown hazards, drum integrity, handling procedures, decontamination. Also, linked fumes/dust in metalworking to respiratory hazards (though not “chemicals in containers” literally). GPT-4: Good on hazmat drum handling risks (manual handling, spill, contamination). Also flagged chemical/fire hazard in blacksmith image (open containers). Copilot: Framework mentions “Chemical and Biological Hazards” as a category, but less detailed in image-specific chemical hazard analysis. More about general principles than direct image interpretation. |
| 88 | 85 | 75 | Gemini: Identified clutter, materials on floor, uneven surfaces (uncovered pit). GPT-4: Good on trip hazards (scattered objects, debris), poor housekeeping. Copilot: Addresses “Slip, Trip, and Fall Risks” as a category, but less granular hazard identification from images. Framework helpful, but less direct image-based analysis in initial outputs. |
| 95 | 90 | N/A | Gemini: Excellent on stacked boxes (haphazard, mezzanine edge), unsecured pallet jack, forklift operation. GPT-4: Identified unsafe material handling, forklift safety. Copilot: Framework does not have a direct “Material Handling and Storage” category as explicitly, making direct scoring less applicable. While general framework touches on workplace organization, it is not as focused on material handling specifics in its initial responses. |
Disclaimer/Publisher’s Note: The statements, opinions and data contained in all publications are solely those of the individual author(s) and contributor(s) and not of MDPI and/or the editor(s). MDPI and/or the editor(s) disclaim responsibility for any injury to people or property resulting from any ideas, methods, instructions or products referred to in the content. |
© 2025 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https://creativecommons.org/licenses/by/4.0/).
Share and Cite
Bhatti, U.; Jaleel, Q.; Aslam, U.; Riaz, A.b.; Saeed, N.; Kamal, K. Transforming Petrochemical Safety Using a Multimodal AI Visual Analyzer. Eng. Proc. 2024, 78, 12. https://doi.org/10.3390/engproc2024078012
Bhatti U, Jaleel Q, Aslam U, Riaz Ab, Saeed N, Kamal K. Transforming Petrochemical Safety Using a Multimodal AI Visual Analyzer. Engineering Proceedings. 2024; 78(1):12. https://doi.org/10.3390/engproc2024078012
Chicago/Turabian StyleBhatti, Uzair, Qamar Jaleel, Umair Aslam, Ahrad bin Riaz, Najam Saeed, and Khurram Kamal. 2024. "Transforming Petrochemical Safety Using a Multimodal AI Visual Analyzer" Engineering Proceedings 78, no. 1: 12. https://doi.org/10.3390/engproc2024078012
APA StyleBhatti, U., Jaleel, Q., Aslam, U., Riaz, A. b., Saeed, N., & Kamal, K. (2024). Transforming Petrochemical Safety Using a Multimodal AI Visual Analyzer. Engineering Proceedings, 78(1), 12. https://doi.org/10.3390/engproc2024078012