An Overview of Augmented Reality
Abstract
:1. Introduction
- Real Environment (RE): it is the environment in which we live and is governed by the laws of physics;
- Augmented Reality (AR): physical reality in which participants also see virtual elements;
- Augmented Virtuality (AV): it is a virtual reality in which the participants also see real elements;
- Virtual Reality (VR): represents a synthetic world in which the participant is completely immersed.
- marker-based AR;
- AR not based on markers;
- AR based on projections;
- AR based on overlaps.
2. Applications in AR
2.1. A Brief History of AR
2.2. Game Applications
2.3. Medical Applications
2.4. Other Applications
3. AR Systems
3.1. Hardware and Software
- Input devices: these devices allow users to interact with AR systems. The AR interface acts as a mediator between these devices and the AR system. A typical example may be the interface used in the VOMAR application. The user can rearrange the furniture in his home using gestures that subsequently translate into commands in the application. Another example of an input device is gloves with built-in sensors, which can be used in various AR applications for drawing, gaming, and many more. The inputs can be of various types and nature: from gesticulating to blinking, touching to speaking [23,24].
- Sensors: these devices, useful in the tracking mechanisms, allow determining of the generic user’s position or the generic object. This operation is essential for the visual recording of the physical environment and its digital information; in this way, it will allow a fusion of images of the physical world and the digital world. The composition of the scene is possible through the use of data traced by cameras or 3D models. The tracking devices and techniques are different (GPS, ultrasound, to name a few) and have different settings and ranges; they improve the tracking accuracy of the AR system.
- Display: these devices allow users to interact with the AR system. Examples are HMDs, monitors, and wearable devices (such as glasses, gloves, and clothing). HMD includes one or more cameras based on holography and optics (diffraction and reflection) techniques. These devices are typically placed on the user’s forehead during use in a specific application (from videogames to the medical sector to the engineering field. Another device slightly different from the one just mentioned is the HUD (Head-Up Display). It is a very compact and lightweight device that provides additional information during any user’s activity. Finally, Google Glass, Hololens, and smartphones also fall into this category [25,26].
3.2. Design Limits
- Interoperability: in AR, objects and devices should communicate with each other without problems, producing a significant result for users regardless of their heterogeneous nature; precisely for this reason, one of the leading design problems of an AR system is the interoperability between such devices. To overcome this problem, we should focus on semantic communication between objects and devices and between devices and devices so that they can interact without necessarily knowing their structure, thus working towards a common goal. This goal can be achieved using the semantic web, which can enrich the digital content and be viewed through the user interface [30,31]. The term “semantic web” was coined by Tim Bernes-Lee. With it, we mean the transformation of the World Wide Web into an environment where published documents, such as HTML pages, files, and images, are associated with metadata that specifies their semantic context suitable for automatic processing, such as querying and interpretation by search engines. It will thus be possible to carry out searches based on a specific keyword and the construction of networks of relationships and connections between the various documents, according to more elaborate mechanisms than the simple hypertext link currently used by the normal WWW [32].
- Security and trust: since each device is unknown to the other devices, it is necessary to implement mechanisms for verifying the object’s authenticity through, for instance, certificates [33]. Another crucial point is the security of communication and the guarantee of information delivery. It is inadmissible that an untrusted device, a virus, or the loss of packets containing information compromise the correct functioning of the entire AR system. For this purpose, various encryption techniques (with symmetric and asymmetric keys) can be implemented through security algorithms such as AES (Advanced Encryption Standard), RSA, Diffie-Hellman, RC4 Double, Triple DES (Data Encryption Standard), and many others [34,35,36].
- Context sensitivity: communication between devices should be in real time, up-to-date, and relevant to the context of user requests. Consequently, applications must be designed in such a way as to be sensitive to the context and to correctly process and deliver the information relating to the latter in order to avoid possible conflicts between the various intelligent devices on the network.
- Minimal user intervention: another problem of AR applications is their dependence on the user and their intervention. Each IoT device should be self-sufficient and responsive. Its work should be invisible to the user, thus allowing them to have a much more autonomous system, even in any failures [37].
- Hardware problems: an IoT system uses a wide range of intelligent devices, from the least powerful, such as 8-bit devices, to the most powerful, such as 64-bit devices, which work in different environments and platforms, such as Atmel, Cortext, and Arduino. Therefore, one of the main objectives is the reduction in the energy consumption of these devices to make them more efficient. In this regard, various techniques can be used, such as the use of recharging mechanisms using kinetic energy for moving devices, using sunlight, and much more [38,39,40]. Two other main problems found at the hardware level are the device’s failures and heaviness caused by its size. In particular, to overcome this last problem, thanks to technological advancement, we are already able to develop compact and low-weight devices, gradually guaranteeing greater portability.
- Software problems: given that AR applications can be run on hardware platforms with different computational characteristics, the software level’s main problem is interoperability and compatibility. The operating system used to be versatile must be optimized in terms of code, size, and power. Examples of operating systems that can be practiced in the AR context are TynyOS, FreeRTOS, and OpenWSN. There are also specific browsers for AR, such as “Firefox Reality” launched by Mozilla, which allows the display of content through the use of viewers (HTC Vive or Google Daydream), and various other devices that are still in the process of development. Therefore, it appears evident that it is necessary to develop appropriate toolkits capable of supporting different devices and applications running on multiple platforms and an interface capable of interacting as multiple interfaces where possible [41,42].
4. AR in Industry 4.0
5. Conclusions
Author Contributions
Funding
Institutional Review Board Statement
Informed Consent Statement
Data Availability Statement
Acknowledgments
Conflicts of Interest
References
- Benassi, A.; Carboni, A.; Colantonio, S.; Coscetti, S.; Germanese, D.; Jalil, B.; Leone, R.; Magnavacca, J.; Magrini, M.; Martinelli, M.; et al. Augmented reality and intelligent systems in Industry 4.0. 2020. Available online: https://hal.archives-ouvertes.fr/hal-03018976/ (accessed on 24 November 2021).
- Bogue, R. Robotic vision boosts automotive industry quality and productivity. Ind. Robot Int. J. 2013, 40, 415–419. [Google Scholar] [CrossRef]
- Wang, H.; Ding, H.X. Autobody taillight assembly modeling and fitting variation sensitivity analysis. Proc. Inst. Mech. Eng. B 2013, 227, 587–594. [Google Scholar] [CrossRef]
- Palmarini, R.; Erkoyuncu, J.A.; Roy, R.; Torabmostaedi, H. A systematic review of augmented reality applications in maintenance. Robot. Comput.-Integr. Manuf. 2018, 49, 215–228. [Google Scholar] [CrossRef] [Green Version]
- Abdul Halim, A.Z. Applications of augmented reality for inspection and maintenance process in automotive industry. J. Fund. Appl. Sci. 2018, 10, 3S. [Google Scholar]
- Schmalstieg, D.; Hollerer, T. Augmented Reality: Principes and Practices; Addison-Wesley Professional: Boston, MA, USA, 2016. [Google Scholar]
- Ismail, A. The Sword Damocles. Available online: http://www.dsource.in/sites/default/files/course/virtual-reality-introduction/evolution-vr/sword-damocles-head-mounted-display/images/17.jpg (accessed on 14 September 2021).
- Microsoft. HoloLens 2. Available online: https://www.microsoft.com/it-it/hololens/hardware (accessed on 16 September 2021).
- Aschauer, A.; Reisner-Kollmann, I.; Wolfartsberger, J. Creating an Open-Source Augmented Reality Remote Support Tool for Industry: Challenges and Learnings. Procedia Comput. Sci. 2021, 180, 269–279. [Google Scholar] [CrossRef]
- Wu, S.K.; Hu, S.J.; Wu, S.M. Optimal door fitting with systematic fixture adjustment. Int. J. Flex. Manuf. Syst. 1994, 6, 99–121. [Google Scholar] [CrossRef] [Green Version]
- Santi, G.M.; Ceruti, A.; Liverani, A.; Osti, F. Augmented Reality in Industry 4.0 and Future Innovation Programs. Technologies 2021, 9, 33. [Google Scholar] [CrossRef]
- Lavrentieva, O.O.; Arkhypov, I.O.; Krupskyi, O.P.; Velykodnyi, D.O.; Filatov, S.V. Methodology of using mobile apps with augmented reality in students’ vocational preparation process for transport industry. In Proceedings of the 3rd International Workshop on Augmented Reality in Education, AREdu 2020, Kryvyi Rih, Ukraine, 13 May 2020. [Google Scholar]
- Egger, J.; Masood, T. Augmented reality in support of intelligent manufacturing—A systematic literature review. Comput. Ind. Eng. 2020, 140, 106195. [Google Scholar] [CrossRef]
- Norman, D.; Draper, S. User Centered System Design: New Perspectives on Human-Computer Interaction; CRC Press Tylor and Francis Group: Boca Raton, FL, USA, 1986. [Google Scholar]
- Boscarol, M. Cos’è lo User-Centered Design. Available online: https://www.usabile.it/302007.htm (accessed on 22 September 2021).
- Nielsen, J. Designing Web Usability; Apogeo Editore: Milan, Italy, 1998. [Google Scholar]
- Quandt, M.; Beinke, T.; Freitag, M. User-Centered Evaluation of an Augmented Reality-based Assistance system for Maintenance. Procedia CIRP 2020, 93, 921–926. [Google Scholar] [CrossRef]
- Hu, F.; Xie, D.; Shen, S. On the application of the internet of things in the field of medical and health care. In Proceedings of the 2013 IEEE International Conference on Green Computing and Communications and IEEE Internet of Things and IEEE Cyber, Physical and Social Computing, Washington, DC, USA, 20–23 August 2013; pp. 2053–2058. [Google Scholar]
- Jo, D.; Kim, G.J. An enabled iot for a smart and interactive environment: A survey and future directions. Sensors 2019, 19, 4330. [Google Scholar] [CrossRef] [Green Version]
- Shinde, G.R.; Dhotre, P.S.; Mahalle, P.N.; Dey, N. Internet of Things Integrated Augmented Reality; Springer: Cham, Switzerland, 2020. [Google Scholar]
- tom Dieck, M.C.; Jung, T.; Han, D.-I. Mapping requirements for the wearable smart glasses augmented reality museum application. J. Hosp. Tour. Technol. 2016, 7, 230–253. [Google Scholar] [CrossRef]
- Noreikis, M.; Savela, N.; Kaakinen, M.; Xiao, Y.; Oksanen, A. Effects of Gamified Augmented Reality in Public Spaces. IEEE Access 2019, 7, 148108–148118. [Google Scholar] [CrossRef]
- Sauro, J. Measuring Usability with The System Usability Scale (SUS), MeasuringU. Available online: https://measuringu.com/sus/ (accessed on 16 October 2021).
- Dirin, A.; Laine, T.H. User Experience in Mobile Augmented Reality/Emotions, Challenges, Opportunities and Best Practices. Computers 2018, 7, 33. [Google Scholar] [CrossRef] [Green Version]
- Endsley, T.C.; Sprehn, K.A.; Brill, R.M.; Ryan, K.J.; Vincent, E.C.; Martin Draper, J.M. Augmented reality design heuristics: Designing for dynamic interactions. In Proceedings of the Human Factors and Ergonomics Society 2017 Annual Meeting, Austin, TX, USA, 9–13 October 2017. [Google Scholar]
- Schumann, M.; Fuchs, C.; Kollatsch, C.; Klimant, P. Evaluation of augmented reality supported approaches for product design and production processes. Procedia CIRP 2021, 97, 160–165. [Google Scholar] [CrossRef]
- Oufqir, A.; El Abderrahmani, A.; Satori, K. ARKit and ARCore in serve to augmented reality. In Proceedings of the 2020 International Conference on Intelligent Systems and Computer Vision (ISCV), Fez, Morocco, 9–11 June 2020; pp. 1–7. [Google Scholar]
- Dini, G.; Dalle Mura, M. Application of augmented reality techniques in through-life engineering services. Procedia CIRP 2015, 38, 14–23. [Google Scholar] [CrossRef]
- Baroroh, D.K.; Chu, C.H.; Wang, L. Systematic literature review on augmented reality in smart manufacturing: Collaboration between human and computational intelligence. J. Manuf. Syst. 2020, 61, 696–711. [Google Scholar] [CrossRef]
- Dalle Mura, M.; Dini, G.; Failli, F. An integrated environment based on augmented reality and sensing device for manual assembly workstations. Procedia CIRP 2016, 41, 340–345. [Google Scholar] [CrossRef] [Green Version]
- Carmigniani, J.; Furht, B.; Anisetti, M.; Ceravolo, P.; Damiani, E.; Ivkovic, M. Augmented reality technologies, systems and applications. Multimed. Tools Appl. 2011, 51, 341–377. [Google Scholar] [CrossRef]
- World Wide Web Consortium. Available online: https://www.w3.org/2000/Talks/0906-xmlweb-tbl/text.htm (accessed on 14 October 2021).
- Lai, Z.H.; Tao, W.; Leu, M.C.; Yin, Z. Smart augmented reality instructional system for mechanical assembly towards worker-centered intelligent manufacturing. J. Manuf. Syst. 2020, 55, 69–81. [Google Scholar] [CrossRef]
- Westerfield, G.; Mitrovic, A.; Billinghurst, M. Intelligent augmented reality training for motherboard assembly. Int. J. Artif. Intell. Educ. 2015, 25, 157–172. [Google Scholar] [CrossRef]
- Sendari, S.; Firmansah, A.; Aripriharta, A. Performance Analysis of Augmented Reality Based on Vuforia Using 3D Marker Detection. In Proceedings of the 2020 4th International Conference on Vocational Education and Training (ICOVET), Malang, Indonesia, 19 September 2020; pp. 294–298. [Google Scholar]
- Williams, R.; Erkoyuncu, J.A.; Masood, T.; Vrabic, R. Augmented reality assisted calibration of digital twins of mobile robots. IFAC-PapersOnLine 2020, 53, 203–208. [Google Scholar] [CrossRef]
- Aouam, D.; Benbelkacem, S.; Zenati, N.; Zakaria, S.; Meftah, Z. Voice-based augmented reality interactive system for car’s components assembly. In Proceedings of the 2018 3rd International Conference on Pattern Analysis and Intelligent Systems (PAIS), Tebessa, Algeria, 24–25 October 2018; pp. 1–5. [Google Scholar]
- Rezende, L.S.O.; Sá, P.H.M.; Macedo, M.C.F.; Apolinário, A.L.; Winkler, I.; Moret, M.A. Volume Rendering: An Analysis based on the HoloLens Augmented Reality Device. In Proceedings of the 2020 22nd Symposium on Virtual and Augmented Reality (SVR), Porto de Galinhas, Brazil, 7–10 November 2020; pp. 35–38. [Google Scholar]
- Dalle Mura, M.; Dini, G. An augmented reality approach for supporting panel alignment in car body assembly. J. Manuf. Syst. 2021, 59, 251–260. [Google Scholar] [CrossRef]
- Khan, F.A.; Muvva, V.V.R.M.K.R.; Wu, D.; Arefin, M.S.; Phillips, N.; Swan, J.E. A Method for Measuring the Perceived Location of Virtual Content in Optical See through Augmented Reality. In Proceedings of the 2021 IEEE Conference on Virtual Reality and 3D User Interfaces Abstracts and Workshops (VRW), Lisbon, Portugal, 27 March–1 April 2021; pp. 657–658. [Google Scholar]
- Singh, R.; Bailey, S.; Chang, P.; Olyaei, A.; Hekmat, M.; Winoto, R. 34.2 A 21pJ/frame/pixel Imager and 34pJ/frame/pixel Image Processor for a Low-Vision Augmented-Reality Smart Contact Lens. In Proceedings of the 2021 IEEE International Solid-State Circuits Conference (ISSCC), San Francisco, CA, USA, 13–22 February 2021; pp. 482–484.
- Huang, R.; Sun, M. Network algorithm real-time depth image 3D human recognition for augmented reality. J. Real-Time Image Proc. 2021, 18, 307–319. [Google Scholar] [CrossRef]
- Mourtzis, D.; Angelopoulos, J.; Panopoulos, N. A Framework for Automatic Generation of Augmented Reality Maintenance & Repair Instructions based on Convolutional Neural Networks. Procedia CIRP 2020, 93, 977–982. [Google Scholar]
- Sreekanta, M.H.; Sarode, A.; George, K. Error Detection using Augmented Reality in the Subtractive Manufacturing Process. In Proceedings of the 2020 10th Annual Computing and Communication Workshop and Conference (CCWC), Las Vegas, NV, USA, 6–8 January 2020; pp. 592–597. [Google Scholar]
- Kim, J.; Lorenz, M.; Knopp, S.; Klimant, P. Industrial Augmented Reality: Concepts and User Interface Designs for Augmented Reality Maintenance Worker Support Systems. In Proceedings of the 2020 IEEE International Symposium on Mixed and Augmented Reality Adjunct (ISMAR-Adjunct), Recife, Brazil, 9–13 November 2020; pp. 67–69. [Google Scholar]
- Marino, E.; Barbieri, L.; Colacino, B.; Fleri, A.K.; Brunom, F. An Augmented Reality inspection tool to support workers in Industry 4.0 environments. Comput. Ind. 2021, 127, 103412. [Google Scholar] [CrossRef]
- Mekni, M.; Lemieux, A. Augmented Reality: Applications, Challenges and Future Trends. Appl. Comput. Sci. 2014, 20, 205–214. [Google Scholar]
- Lorenz, M.; Knopp, S.; Kim, J.; Klimant, P. Industrial Augmented Reality: 3D-Content Editor for Augmented Reality Maintenance Worker Support System. In Proceedings of the 2020 IEEE International Symposium on Mixed and Augmented Reality Adjunct (ISMAR-Adjunct), Recife, Brazil, 9–13 November 2020; pp. 203–205. [Google Scholar]
- Hu, M.; Weng, D.; Chen, F.; Wang, Y. Object Detecting Augmented Reality System. In Proceedings of the 2020 IEEE 20th International Conference on Communication Technology (ICCT), Nanning, China, 28–31 October 2020; pp. 1432–1438. [Google Scholar]
- Niantic. Available online: https://nianticlabs.com (accessed on 3 November 2021).
Publisher’s Note: MDPI stays neutral with regard to jurisdictional claims in published maps and institutional affiliations. |
© 2022 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https://creativecommons.org/licenses/by/4.0/).
Share and Cite
Arena, F.; Collotta, M.; Pau, G.; Termine, F. An Overview of Augmented Reality. Computers 2022, 11, 28. https://doi.org/10.3390/computers11020028
Arena F, Collotta M, Pau G, Termine F. An Overview of Augmented Reality. Computers. 2022; 11(2):28. https://doi.org/10.3390/computers11020028
Chicago/Turabian StyleArena, Fabio, Mario Collotta, Giovanni Pau, and Francesco Termine. 2022. "An Overview of Augmented Reality" Computers 11, no. 2: 28. https://doi.org/10.3390/computers11020028
APA StyleArena, F., Collotta, M., Pau, G., & Termine, F. (2022). An Overview of Augmented Reality. Computers, 11(2), 28. https://doi.org/10.3390/computers11020028