Ontology-Based Knowledge Representation in Robotic Systems: A Survey Oriented toward Applications
Abstract
:1. Introduction
1.1. Contribution
- It facilitated novice researchers and experts to overcome the challenging task of determining and utilizing the most suitable ontology-based semantic knowledge representation system for the intended robotic application (Section 3).
- It provided an analysis of selected KB systems from the domain of robotics, delineated the advantages, summarized the current main research trends, discussed the limitations, and outlined the possible future directions.
- Compared to the earlier surveys, this study tended to be more concerned with the most recent work. Therefore, it provided the readers an important opportunity to advance their understanding of state-of-the-art methods.
1.2. Inclusion and Exclusion Criteria
1.3. Survey Structure
2. Related Works
3. Analysis of Knowledge Representation Systems
# | KR Name | Publication | Year | Ref. |
---|---|---|---|---|
1 | KnowROB | Know rob 2.0: a 2nd-generation knowledge processing framework for cognition-enabled robotic agents | 2018 | [68] |
2 | OROSU | Knowledge representation applied to robotic orthopedic surgery | 2015 | [69] |
3 | CARESSES | The CARESSES EU-Japan project: making assistive robots culturally competent | 2017 | [70] |
4 | PMK | PMK: A knowledge processing framework for autonomous robotics perception and manipulation | 2019 | [71] |
5 | SARbot | High-level smart decision making of a robot based on an ontology in a search and rescue scenario | 2019 | [72] |
6 | IEQ | A Humanoid social robot-based approach for indoor environment quality monitoring and well-being improvement | 2020 | [73] |
7 | Smart Rules | An integrated semantic framework for designing context-aware Internet of Robotic Things systems | 2018 | [74] |
8 | ARBI | Ontology-based knowledge model for human-robot interactive services | 2020 | [75] |
9 | Worker-cobot | An ontology-based approach to enable knowledge representation and reasoning in worker-cobot agile manufacturing | 2017 | [76] |
10 | APRS | Implementation of an ontology-based approach to enable agility in kit building applications | 2018 | [77] |
3.1. Application Domain Scope
3.2. Idea and Contribution
3.3. Development Tools
3.4. Architecture
3.5. Ontology Scope
3.5.1. Object
3.5.2. Map of Environment
3.5.3. Task and Action
3.6. Reasoning Scope
3.6.1. Interaction Based on Visual Recognition
3.6.2. Interaction Based on Voice Recognition
3.6.3. Task Execution and Action Planning
3.7. Limitations/Challenges
4. Summary
5. Discussion and Future Research Directions
- The researchers are more focused on the development of ontologies in the field of knowledge representation, while the development of the mechanism lags.
- In the future, more research is needed towards the standardization and efficient implementation of ontology-based knowledge representation systems.
- We believe that along with the ontologies, future studies should also aim at the development of efficient queries and reasoning mechanisms that might be applied to many distributed ontologies with limited resources. In this direction, future work is certainly required to achieve a sustainable solution with a sound understanding of resources and quality.
- Besides, future research can be more beneficial towards robot autonomy if the potential effects of ontology-based KR systems with context awareness are considered more carefully.
- Looking forward, future research should be continued in more realistic settings for the development of culturally competent KRSs, which will endow the robots the ability to perform various complex tasks in dynamic environments by understanding the culture-specific needs and preferences.
6. Conclusions
Author Contributions
Funding
Conflicts of Interest
References
- Olszewska, J.I.; Barreto, M.; Bermejo-Alonso, J.; Carbonera, J.; Chibani, A.; Fiorini, S.; Goncalves, P.; Habib, M.; Khamis, A.; Olivares, A.; et al. Ontology for autonomous robotics. In Proceedings of the 2017 26th IEEE International Symposium on Robot and Human Interactive Communication (RO-MAN), Lisbon, Portugal, 28 August 2017; pp. 189–194. [Google Scholar]
- Bayat, B.; Bermejo-Alonso, J.; Carbonera, J.; Facchinetti, T.; Fiorini, S.; Goncalves, P.; Jorge, V.A.; Habib, M.; Khamis, A.; Melo, K.; et al. Requirements for building an ontology for autonomous robots. Ind. Robot. Int. J. 2016, 43, 469–480. [Google Scholar] [CrossRef]
- Čaić, M.; Mahr, D.; Oderkerken-Schröder, G. Value of social robots in services: Social cognition perspective. J. Serv. Mark. 2019, 33, 463–478. [Google Scholar] [CrossRef]
- Lim, G.H.; Suh, I.H.; Suh, H. Ontology-based unified robot knowledge for service robots in indoor environments. IEEE Trans. Syst. Man Cybern. Part Syst. Hum. 2010, 41, 492–509. [Google Scholar] [CrossRef]
- Munir, K.; Anjum, M.S. The use of ontologies for effective knowledge modelling and information retrieval. Appl. Comput. Inform. 2018, 14, 116–126. [Google Scholar] [CrossRef]
- Olivares-Alarcos, A.; Beßler, D.; Khamis, A.; Goncalves, P.; Habib, M.K.; Bermejo-Alonso, J.; Barreto, M.; Diab, M.; Rosell, J.; Quintas, J.; et al. A review and comparison of ontology-based approaches to robot autonomy. Knowl. Eng. Rev. 2019, 34, e29. [Google Scholar] [CrossRef] [Green Version]
- Topp, E.A.; Stenmark, M.; Ganslandt, A.; Svensson, A.; Haage, M.; Malec, J. Ontology-Based Knowledge Representation for Increased Skill Reusability in Industrial Robots. In Proceedings of the 2018 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), Madrid, Spain, 1–5 October 2018; pp. 5672–5678. [Google Scholar]
- Azevedo, H.; Belo, J.P.R.; Romero, R.A. OntPercept: A Perception Ontology for Robotic Systems. In Proceedings of the 2018 IEEE Latin American Robotic Symposium, 2018 Brazilian Symposium on Robotics (SBR) and 2018 Workshop on Robotics in Education (WRE), Joao Pessoa, Brazil, 6–10 November 2018; pp. 469–475. [Google Scholar]
- Joo, S.H.; Manzoor, S.; Rocha, Y.G.; Bae, S.H.; Lee, K.H.; Kuc, T.Y.; Kim, M. Autonomous navigation framework for intelligent robots based on a semantic environment modeling. Appl. Sci. 2020, 10, 3219. [Google Scholar] [CrossRef]
- Manzoor, S.; Joo, S.H.; Rocha, Y.G.; Lee, H.U.; Kuc, T.Y. A Novel Semantic SLAM Framework for Humanlike High-Level Interaction and Planning in Global Environment. In Proceedings of the 1st International Workshop on the Semantic Descriptor, Semantic Modeling and Mapping for Humanlike Perception and Navigation of Mobile Robots toward Large Scale Long-Term Autonomy (SDMM1), Macau, China, 8 November 2019. [Google Scholar]
- Ersen, M.; Oztop, E.; Sariel, S. Cognition-enabled robot manipulation in human environments: Requirements, recent work, and open problems. IEEE Robot. Autom. Mag. 2017, 24, 108–122. [Google Scholar] [CrossRef]
- Perzylo, A.; Somani, N.; Profanter, S.; Kessler, I.; Rickert, M.; Knoll, A. Intuitive instruction of industrial robots: Semantic process descriptions for small lot production. In Proceedings of the 2016 IEEE/rsj International Conference on Intelligent Robots and Systems (IROS), Daejeon Convention Center, DaeJeon, Korea, 9–14 October 2016; pp. 2293–2300. [Google Scholar]
- Waibel, M.; Beetz, M.; Civera, J.; d’Andrea, R.; Elfring, J.; Galvez-Lopez, D.; Häussermann, K.; Janssen, R.; Montiel, J.; Perzylo, A.; et al. Roboearth. IEEE Robot. Autom. Mag. 2011, 18, 69–82. [Google Scholar] [CrossRef] [Green Version]
- Tenorth, M.; Beetz, M. KnowRob: A knowledge processing infrastructure for cognition-enabled robots. Int. J. Robot. Res. 2013, 32, 566–590. [Google Scholar] [CrossRef]
- Beetz, M.; Tenorth, M.; Winkler, J. Open-ease. In Proceedings of the 2015 IEEE International Conference on Robotics and Automation (ICRA), Washington, DC, USA, 25–30 May 2015; pp. 1983–1990. [Google Scholar]
- Saxena, A.; Jain, A.; Sener, O.; Jami, A.; Misra, D.K.; Koppula, H.S. Robobrain: Large-scale knowledge engine for robots. arXiv 2014, arXiv:1412.0691. [Google Scholar]
- Daruna, A.; Liu, W.; Kira, Z.; Chetnova, S. Robocse: Robot common sense embedding. In Proceedings of the 2019 International Conference on Robotics and Automation (ICRA), Montreal, QC, Canada, 20–24 May 2019; pp. 9777–9783. [Google Scholar]
- Poux, F.; Ponciano, J.J. Self-Learning Ontology For Instance Segmentation of 3d Indoor Point Cloud. Int. Arch. Photogramm. Remote Sens. Spat. Inf. Sci. 2020, 43, 309–316. [Google Scholar] [CrossRef]
- Kanjaruek, S.; Li, D.; Qiu, R.; Boonsim, N. Automated ontology framework for service robots. In Proceedings of the 2015 IEEE International Conference on Robotics and Biomimetics (ROBIO), Zhuhai, China, 6–9 December 2015; pp. 219–224. [Google Scholar]
- Kanjaruek, S.; Li, D. Tracking Objects Robot for healthcare environments. In Proceedings of the 2017 IEEE International Conference on Internet of Things (iThings) and IEEE Green Computing and Communications (GreenCom) and IEEE Cyber, Physical and Social Computing (CPSCom) and IEEE Smart Data (SmartData), Exeter, UK, 21–23 June 2017; pp. 894–899. [Google Scholar]
- Velardi, P.; Pazienza, M.T.; Fasolo, M. How to encode semantic knowledge: A method for meaning representation and computer-aided acquisition. Comput. Linguist. 1991, 17, 153–170. [Google Scholar]
- Gibaud, B.; Forestier, G.; Feldmann, C.; Ferrigno, G.; Gonçalves, P.; Haidegger, T.; Julliard, C.; Katić, D.; Kenngott, H.; Maier-Hein, L.; et al. Toward a standard ontology of surgical process models. Int. J. Comput. Assist. Radiol. Surg. 2018, 13, 1397–1408. [Google Scholar] [CrossRef] [Green Version]
- Björkelund, A.; Bruyninckx, H.; Malec, J.; Nilsson, K.; Nugues, P. Knowledge for Intelligent Industrial Robots. In Proceedings of the AAAI Spring Symposium on Designing Intelligent Robots: Reintegrating AI, Palo Alto, CA, USA, 26 March 2012; Volume 12, p. 2. [Google Scholar]
- Demir, K.A.; Döven, G.; Sezen, B. Industry 5.0 and human-robot co-working. Procedia Comput. Sci. 2019, 158, 688–695. [Google Scholar] [CrossRef]
- Skobelev, P.; Borovik, S.Y. On the way from Industry 4.0 to Industry 5.0: From digital manufacturing to digital society. Ind. 4.0 2017, 2, 307–311. [Google Scholar]
- Sun, X.; Zhang, Y. A Review of Domain Knowledge Representation for Robot Task Planning. In Proceedings of the 2019 4th International Conference on Mathematics and Artificial Intelligence, Chengdu, China, 12–15 April 2019; pp. 176–183. [Google Scholar]
- Thosar, M.; Zug, S.; Skaria, A.M.; Jain, A. A Review of Knowledge Bases for Service Robots in Household Environments. 2018, pp. 98–110. Available online: https://www.researchgate.net/publication/328249457_A_Review_of_Knowledge_Bases_for_Service_Robots_in_Household_Environments (accessed on 29 April 2021).
- Gouidis, F.; Vassiliades, A.; Patkos, T.; Argyros, A.; Bassiliades, N.; Plexousakis, D. A review on intelligent object perception methods combining knowledge-based reasoning and machine learning. arXiv 2019, arXiv:1912.11861. [Google Scholar]
- Stojanovic, L.; Schneider, J.; Maedche, A.; Libischer, S.; Studer, R.; Lumpp, T.; Abecker, A.; Breiter, G.; Dinger, J. The role of ontologies in autonomic computing systems. IBM Syst. J. 2004, 43, 598–616. [Google Scholar] [CrossRef]
- Bermejo-Alonso, J.; Sanz, R.; Rodríguez, M.; Hernández, C. Ontology-based engineering of autonomous systems. In Proceedings of the 2010 IEEE Sixth International Conference on Autonomic and Autonomous Systems, Cancun, Mexico, 7–13 March 2010; pp. 47–51. [Google Scholar]
- Bermejo-Alonso, J.; Hernández, C.; Sanz, R. Model-based engineering of autonomous systems using ontologies and metamodels. In Proceedings of the 2016 IEEE International Symposium on Systems Engineering (ISSE), Edinburgh, UK, 21–26 July 2016; pp. 1–8. [Google Scholar]
- Bermejo-Alonso, J.; Sanz, R.; Rodríguez, M.; Hernández, C. Ontology engineering for the autonomous systems domain. In International Joint Conference on Knowledge Discovery, Knowledge Engineering, and Knowledge Management; Springer: Berlin/Heidelberg, Germany, 2011; pp. 263–277. [Google Scholar]
- Bozcuoğlu, A.K.; Kazhoyan, G.; Furuta, Y.; Stelter, S.; Beetz, M.; Okada, K.; Inaba, M. The exchange of knowledge using cloud robotics. IEEE Robot. Autom. Lett. 2018, 3, 1072–1079. [Google Scholar] [CrossRef]
- de Freitas, E.P.; Olszewska, J.I.; Carbonera, J.L.; Fiorini, S.R.; Khamis, A.; Ragavan, S.V.; Barreto, M.E.; Prestes, E.; Habib, M.K.; Redfield, S.; et al. Ontological concepts for information sharing in cloud robotics. J. Ambient. Intell. Humaniz. Comput. 2020, 1–12. [Google Scholar] [CrossRef]
- Dogmus, Z.; Erdem, E.; Patoglu, V. RehabRobo-Onto: Design, development and maintenance of a rehabilitation robotics ontology on the cloud. Robot. Comput. Integr. Manuf. 2015, 33, 100–109. [Google Scholar] [CrossRef]
- Balakirsky, S.; Kootbally, Z.; Schlenoff, C.; Kramer, T.; Gupta, S. An industrial robotic knowledge representation for kit building applications. In Proceedings of the 2012 IEEE/RSJ International Conference on Intelligent Robots and Systems, Vilamoura-Algarve, Portugal, 7–12 October 2012; pp. 1365–1370. [Google Scholar]
- Schäfer, F.; Kriesten, R.; Chrenko, D.; Gechter, F. No need to learn from each other? Potentials of knowledge modeling in autonomous vehicle systems engineering towards new methods in multidisciplinary contexts. In Proceedings of the 2017 IEEE International Conference on Engineering, Technology and Innovation (ICE/ITMC), Madeira, Portugal, 27–29 June 2017; pp. 462–468. [Google Scholar]
- Jäger, G.; Mueller, C.A.; Thosar, M.; Zug, S.; Birk, A. Towards robot-centric conceptual knowledge acquisition. arXiv 2018, arXiv:1810.03583. [Google Scholar]
- Beßler, D.; Koralewski, S.; Beetz, M. Knowledge Representation for Cognition- and Learning-enabled Robot Manipulation. Available online: https://www.semanticscholar.org/paper/Knowledge-Representation-for-Cognition-and-Robot-Be%C3%9Fler-Koralewski/a912517f69db6dd78f80c249320d5a781a67a70d (accessed on 29 April 2021).
- Fischer, L.; Hasler, S.; Deigmöller, J.; Schnürer, T.; Redert, M.; Pluntke, U.; Nagel, K.; Senzel, C.; Ploennigs, J.; Richter, A.; et al. Which Tool to Use? Grounded Reasoning in Everyday Environments with Assistant Robots. CogRob@ KR. 2018, pp. 3–10. Available online: https://www.semanticscholar.org/paper/Which-tool-to-use-Grounded-reasoning-in-everyday-Fischer-Hasler/25c3841a905553a370f89d657f3376f63207dc3b (accessed on 29 April 2021).
- Pinacho, L.S.; Wich, A.; Yazdani, F.; Beetz, M. Acquiring knowledge of object arrangements from human examples for household robots. In Joint German/Austrian Conference on Artificial Intelligence (Künstliche Intelligenz); Springer: Berlin/Heidelberg, Germany, 2018; pp. 131–138. [Google Scholar]
- Yang, G.; Wang, S.; Yang, J. Desire-driven reasoning for personal care robots. IEEE Access 2019, 7, 75203–75212. [Google Scholar] [CrossRef]
- Vassiliades, A.; Bassiliades, N.; Gouidis, F.; Patkos, T. A Knowledge Retrieval Framework for Household Objects and Actions with External Knowledge. InInternational Conference on Semantic Systems; Springer: Cham, Switzerland, 2020; pp. 36–52. [Google Scholar]
- Gehrig, D.; Krauthausen, P.; Rybok, L.; Kuehne, H.; Hanebeck, U.D.; Schultz, T.; Stiefelhagen, R. Combined intention, activity, and motion recognition for a humanoid household robot. In Proceedings of the 2011 IEEE/RSJ International Conference on Intelligent Robots and Systems, San Francisco, CA, USA, 25–30 September 2011; pp. 4819–4825. [Google Scholar]
- Patterson, D.J.; Fox, D.; Kautz, H.; Philipose, M. Fine-grained activity recognition by aggregating abstract object usage. In Proceedings of the Ninth IEEE International Symposium on Wearable Computers (ISWC’05), Osaka, Japan, 18–21 October 2005; pp. 44–51. [Google Scholar]
- Ramirez-Amaro, K.; Beetz, M.; Cheng, G. Transferring skills to humanoid robots by extracting semantic representations from observations of human activities. Artif. Intell. 2017, 247, 95–118. [Google Scholar] [CrossRef]
- Lemaignan, S.; Warnier, M.; Sisbot, E.A.; Clodic, A.; Alami, R. Artificial cognition for social human–robot interaction: An implementation. Artif. Intell. 2017, 247, 45–69. [Google Scholar] [CrossRef] [Green Version]
- Agostini, A.; Torras, C.; Wörgötter, F. Learning weakly correlated cause–effects for gardening with a cognitive system. Eng. Appl. Artif. Intell. 2014, 36, 178–194. [Google Scholar] [CrossRef] [Green Version]
- Agostini, A.; Torras, C.; Wörgötter, F. Efficient interactive decision making framework for robotic applications. Artif. Intell. 2017, 247, 187–212. [Google Scholar] [CrossRef] [Green Version]
- Potaov, A. Enabling Cognitive Visual Question Answering. Available online: https://blog.singularitynet.io/enabling-cognitive-visual-question-answering-a93febd454a7 (accessed on 8 March 2021).
- He, B.; Xia, M.; Yu, X.; Jian, P.; Meng, H.; Chen, Z. An educational robot system of visual question answering for preschoolers. In Proceedings of the 2017 IEEE 2nd International Conference on Robotics and Automation Engineering (ICRAE), Shanghai, China, 29–31 December 2017; pp. 441–445. [Google Scholar]
- Keren, G.; Fridin, M. Kindergarten Social Assistive Robot (KindSAR) for children’s geometric thinking and metacognitive development in preschool education: A pilot study. Comput. Hum. Behav. 2014, 35, 400–412. [Google Scholar] [CrossRef]
- Ren, S.; He, K.; Girshick, R.; Sun, J. Faster r-cnn: Towards real-time object detection with region proposal networks. arXiv 2015, arXiv:1506.01497. [Google Scholar] [CrossRef] [Green Version]
- Li, G.; Su, H.; Zhu, W. Incorporating external knowledge to answer open-domain visual questions with dynamic memory networks. arXiv 2017, arXiv:1712.00733. [Google Scholar]
- Narasimhan, M.; Schwing, A.G. Straight to the facts: Learning knowledge base retrieval for factual visual question answering. In Proceedings of the European Conference on Computer Vision (ECCV), Munich, Germany, 8–14 September 2018; pp. 451–468. [Google Scholar]
- Wang, P.; Wu, Q.; Shen, C.; Hengel, A.v.d.; Dick, A. Explicit knowledge-based reasoning for visual question answering. arXiv 2015, arXiv:1511.02570. [Google Scholar]
- Wu, Q.; Wang, P.; Shen, C.; Dick, A.; Van Den Hengel, A. Ask me anything: Free-form visual question answering based on knowledge from external sources. In Proceedings of the IEEE Conference on Computer Vision and Pattern, Recognition, Las Vegas, NV, USA, 27–30 June 2016; pp. 4622–4630. [Google Scholar]
- Shah, S.; Mishra, A.; Yadati, N.; Talukdar, P.P. Kvqa: Knowledge-aware visual question answering. In Proceedings of the AAAI Conference on Artificial Intelligence, Honolulu, Hi, USA, 27 January–1 February 2019; Volume 33, pp. 8876–8884. [Google Scholar]
- Wu, Q.; Teney, D.; Wang, P.; Shen, C.; Dick, A.; van den Hengel, A. Visual question answering: A survey of methods and datasets. Comput. Vis. Image Underst. 2017, 163, 21–40. [Google Scholar] [CrossRef] [Green Version]
- Malinowski, M.; Rohrbach, M.; Fritz, M. Ask your neurons: A neural-based approach to answering questions about images. In Proceedings of the IEEE International Conference on Computer Vision, Santiago, Chile, 7–13 December 2015; pp. 1–9. [Google Scholar]
- Ma, L.; Lu, Z.; Li, H. Learning to answer questions from image using convolutional neural network. In Proceedings of the AAAI Conference on Artificial Intelligence, Phoenix, AZ, USA, 12–17 February 2016; Volume 30. [Google Scholar]
- Zhu, Y.; Groth, O.; Bernstein, M.; Fei-Fei, L. Visual7w: Grounded question answering in images. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Las Vegas, NV, USA, 27–30 June 2016; pp. 4995–5004. [Google Scholar]
- Xu, H.; Saenko, K. Ask, attend and answer: Exploring question-guided spatial attention for visual question answering. InEuropean Conference on Computer Vision; Springer: Berlin/Heidelberg, Germany, 2016; pp. 451–466. [Google Scholar]
- Chen, K.; Wang, J.; Chen, L.C.; Gao, H.; Xu, W.; Nevatia, R. Abc-cnn: An attention based convolutional neural network for visual question answering. arXiv 2015, arXiv:1511.05960. [Google Scholar]
- Yang, Z.; He, X.; Gao, J.; Deng, L.; Smola, A. Stacked attention networks for image question answering. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Las Vegas, NV, USA, 27–30 June 2016; pp. 21–29. [Google Scholar]
- Deng, J.; Dong, W.; Socher, R.; Li, L.J.; Li, K.; Fei-Fei, L. Imagenet: A large-scale hierarchical image database. In Proceedings of the 2009 IEEE Conference on Computer Vision and Pattern Recognition, Miami, FL, USA, 20–25 June 2009; pp. 248–255. [Google Scholar]
- Lin, T.Y.; Maire, M.; Belongie, S.; Hays, J.; Perona, P.; Ramanan, D.; Dollár, P.; Zitnick, C.L. Microsoft coco: Common objects in context. InEuropean Conference on Computer Vision; Springer: Berlin/Heidelberg, Germany, 2014; pp. 740–755. [Google Scholar]
- Beetz, M.; Beßler, D.; Haidu, A.; Pomarlan, M.; Bozcuoğlu, A.K.; Bartels, G. Know rob 2.0—A 2nd generation knowledge processing framework for cognition-enabled robotic agents. In Proceedings of the 2018 IEEE International Conference on Robotics and Automation (ICRA), Brisbane, Australia, 21–25 May 2018; pp. 512–519. [Google Scholar]
- Gonçalves, P.J.; Torres, P.M. Knowledge representation applied to robotic orthopedic surgery. Robot. Comput.-Integr. Manuf. 2015, 33, 90–99. [Google Scholar] [CrossRef]
- Bruno, B.; Chong, N.Y.; Kamide, H.; Kanoria, S.; Lee, J.; Lim, Y.; Pandey, A.K.; Papadopoulos, C.; Papadopoulos, I.; Pecora, F.; et al. The CARESSES EU-Japan project: Making assistive robots culturally competent. In Italian Forum of Ambient Assisted Living; Springer: Berlin/Heidelberg, Germany, 2017; pp. 151–169. [Google Scholar]
- Diab, M.; Akbari, A.; Ud Din, M.; Rosell, J. PMK—A Knowledge Processing Framework for Autonomous Robotics Perception and Manipulation. Sensors 2019, 19, 1166. [Google Scholar] [CrossRef] [PubMed] [Green Version]
- Sun, X.; Zhang, Y.; Chen, J. High-Level Smart Decision Making of a Robot Based on Ontology in a Search and Rescue Scenario. Future Internet 2019, 11, 230. [Google Scholar] [CrossRef] [Green Version]
- Ribino, P.; Bonomolo, M.; Lodato, C.; Vitale, G. A Humanoid Social Robot Based Approach for Indoor Environment Quality Monitoring and Well-Being Improvement. Int. J. Soc. Robot. 2020, 13, 277–296. [Google Scholar] [CrossRef]
- Sabri, L.; Bouznad, S.; Rama Fiorini, S.; Chibani, A.; Prestes, E.; Amirat, Y. An integrated semantic framework for designing context-aware Internet of Robotic Things systems. Integr. Comput.-Aided Eng. 2018, 25, 137–156. [Google Scholar] [CrossRef]
- Chang, D.S.; Cho, G.H.; Choi, Y.S. Ontology-based knowledge model for human-robot interactive services. In Proceedings of the 35th Annual ACM Symposium on Applied Computing, Brno, Czech Republic, 30 March–3 April 2020; pp. 2029–2038. [Google Scholar]
- Sadik, A.R.; Urban, B. An ontology-based approach to enable knowledge representation and reasoning in worker-cobot agile manufacturing. Future Internet 2017, 9, 90. [Google Scholar] [CrossRef] [Green Version]
- Kootbally, Z.; Kramer, T.R.; Schlenoff, C.; Gupta, S.K. Implementation of an ontology-based approach to enable agility in kit building applications. Int. J. Semant. Comput. 2018, 12, 5–24. [Google Scholar] [CrossRef]
- Gonçalves, P. Towards an ontology for orthopaedic surgery, application to hip resurfacing. In Proceedings of the Hamlyn Symposium on Medical Robotics, London, UK, 22–25 June 2013; pp. 61–62. [Google Scholar]
- Diab, M.; Akbari, A.; Rosell, J. An ontology framework for physics-based manipulation planning. In Iberian Robotics Conference; Springer: Berlin/Heidelberg, Germany, 2017; pp. 452–464. [Google Scholar]
- Zhao, J.; Gao, J.; Zhao, F.; Liu, Y. A search-and-rescue robot system for remotely sensing the underground coal mine environment. Sensors 2017, 17, 2426. [Google Scholar] [CrossRef] [Green Version]
- Bujari, A.; Calafate, C.T.; Cano, J.C.; Manzoni, P.; Palazzi, C.E.; Ronzani, D. A location-aware waypoint-based routing protocol for airborne DTNs in search and rescue scenarios. Sensors 2018, 18, 3758. [Google Scholar] [CrossRef] [Green Version]
- Matarić, M.J. Socially assistive robotics: Human augmentation versus automation. Sci. Robot. 2017, 2, eaam5410. [Google Scholar] [CrossRef] [PubMed]
- Rossi, S.; Staffa, M.; Tamburro, A. Socially assistive robot for providing recommendations: Comparing a humanoid robot with a mobile application. Int. J. Soc. Robot. 2018, 10, 265–278. [Google Scholar] [CrossRef]
- Choi, J.H.; Lee, K. Investigation of the feasibility of POE methodology for a modern commercial office building. Build. Environ. 2018, 143, 591–604. [Google Scholar] [CrossRef]
- McGuinness, D.L.; Van Harmelen, F. OWL web ontology language overview. W3C Recomm. 2004, 10, 2004. [Google Scholar]
- Wielemaker, J.; Schrijvers, T.; Triska, M.; Lager, T. Swi-prolog. arXiv 2010, arXiv:1011.5332. [Google Scholar] [CrossRef] [Green Version]
- Quigley, M.; Conley, K.; Gerkey, B.; Faust, J.; Foote, T.; Leibs, J.; Wheeler, R.; Ng, A.Y. ROS: An open-source Robot Operating System. In Proceedings of the ICRA Workshop on Open Source Software, Kobe, Japan, 12 May 2009; Volume 3, p. 5. [Google Scholar]
- Bordini, R.H.; Hübner, J.F. BDI agent programming in AgentSpeak using Jason. InInternational Workshop on Computational Logic in Multi-Agent Systems; Springer: Berlin/Heidelberg, Germany, 2005; pp. 143–164. [Google Scholar]
- AgentSpeak, A.R. AgentSpeak (L): BDI agents speak out in a logical computable language. Aust. Artif. Intell. Inst. 1996, 1, 42–55. [Google Scholar]
- Amarilli, F.; Amigoni, F.; Fugini, M.G.; Zarri, G.P. A semantic-rich approach to IoT using the generalized world entities paradigm. In Manag. Web Things; Elsevier: Amsterdam, The Netherlands, 2017; pp. 105–147. [Google Scholar]
- Bellifemine, F.; Bergenti, F.; Caire, G.; Poggi, A. JADE—A java agent development framework. In Multi-Agent Programming; Springer: Berlin/Heidelberg, Germany, 2005; pp. 125–147. [Google Scholar]
- Chaib-draa, B.; Dignum, F. Trends in agent communication language. Comput. Intell. 2002, 18, 89–101. [Google Scholar] [CrossRef]
- Prestes, E.; Carbonera, J.L.; Fiorini, S.R.; Jorge, V.A.; Abel, M.; Madhavan, R.; Locoro, A.; Goncalves, P.; Barreto, M.E.; Habib, M.; et al. Towards a core ontology for robotics and automation. Robot. Auton. Syst. 2013, 61, 1193–1204. [Google Scholar] [CrossRef]
- Ferro, E.; Girolami, M.; Salvi, D.; Mayer, C.; Gorman, J.; Grguric, A.; Ram, R.; Sadat, R.; Giannoutakis, K.M.; Stocklöw, C. The universaal platform for aal (ambient assisted living). J. Intell. Syst. 2015, 24, 301–319. [Google Scholar] [CrossRef]
- Hoffmann, J.; Nebel, B. The FF planning system: Fast plan generation through heuristic search. J. Artif. Intell. Res. 2001, 14, 253–302. [Google Scholar] [CrossRef]
- Akbari, A.; Rosell, J. Task planning using physics-based heuristics on manipulation actions. In Proceedings of the 2016 IEEE 21st International Conference on Emerging Technologies and Factory Automation (ETFA), Berlin, Germany, 6–9 September 2016; pp. 1–8. [Google Scholar]
- Fox, M.; Long, D. PDDL2. 1: An extension to PDDL for expressing temporal planning domains. J. Artif. Intell. Res. 2003, 20, 61–124. [Google Scholar] [CrossRef]
- Wang, A.Y.; Sable, J.H.; Spackman, K.A. The SNOMED clinical terms development process: Refinement and analysis of content. In Proceedings of the AMIA Symposium, San Antonio, TX, USA, 9–13 November 2002; p. 845. [Google Scholar]
- Tenorth, M.; Beetz, M. KnowRob—Knowledge processing for autonomous personal robots. In Proceedings of the 2009 IEEE/RSJ International Conference on Intelligent Robots and Systems, St. Louis, MO, USA, 11–15 October 2009; pp. 4261–4266. [Google Scholar]
- Barattini, P.; Vicentini, F.; Virk, G.S.; Haidegger, T. Human-Robot Interaction: Safety, Standardization, and Benchmarking; CRC Press: Boca Raton, FL, USA, 2019. [Google Scholar]
- Tenorth, M.; Beetz, M. Representations for robot knowledge in the KnowRob framework. Artif. Intell. 2017, 247, 151–169. [Google Scholar] [CrossRef]
- Team, J.D. Drools Expert User Guide. Available online: https://docs.jboss.org/drools/release/5.2.0.CR1/drools-expert-docs/html_single/ (accessed on 21 December 2020).
- Sottara, D.; Mello, P.; Proctor, M. A configurable rete-oo engine for reasoning with different types of imperfect information. IEEE Trans. Knowl. Data Eng. 2010, 22, 1535–1548. [Google Scholar] [CrossRef]
- Tenorth, M.; Kunze, L.; Jain, D.; Beetz, M. Knowrob-map-knowledge-linked semantic object maps. In Proceedings of the 2010 IEEE 10th IEEE-RAS International Conference on Humanoid Robots, Nashville, TN, USA, 6–8 December 2010; pp. 430–435. [Google Scholar]
- Sadik, A.R.; Urban, B.; Adel, O. Using hand gestures to interact with an industrial robot in a cooperative flexible manufacturing scenario. In Proceedings of the 3rd International Conference on Mechatronics and Robotics Engineering, Paris, France, 8–12 February 2017; pp. 11–16. [Google Scholar]
- Gleeson, B.; MacLean, K.; Haddadi, A.; Croft, E.; Alcazar, J. Gestures for industry intuitive human-robot communication from human observation. In Proceedings of the 2013 IEEE 8th ACM/IEEE International Conference on Human-Robot Interaction (HRI), Tokyo, Japan, 3–6 March 2013; pp. 349–356. [Google Scholar]
- Bruno, B.; Menicatti, R.; Recchiuto, C.T.; Lagrue, E.; Pandey, A.K.; Sgorbissa, A. Culturally-competent human-robot verbal interaction. In Proceedings of the 2018 IEEE 15th International Conference on Ubiquitous Robots (UR), Honolulu, HI, USA, 26–30 June 2018; pp. 388–395. [Google Scholar]
- Bruno, B.; Recchiuto, C.T.; Papadopoulos, I.; Saffiotti, A.; Koulouglioti, C.; Menicatti, R.; Mastrogiovanni, F.; Zaccaria, R.; Sgorbissa, A. Knowledge representation for culturally competent personal robots: Requirements, design principles, implementation, and assessment. Int. J. Soc. Robot. 2019, 11, 515–538. [Google Scholar] [CrossRef] [Green Version]
- Mansouri, M.; Pecora, F. A robot sets a table: A case for hybrid reasoning with different types of knowledge. J. Exp. Theor. Artif. Intell. 2016, 28, 801–821. [Google Scholar] [CrossRef]
- Köckemann, U.; Pecora, F.; Karlsson, L. Grandpa Hates Robots-Interaction Constraints for Planning in Inhabited Environments. In Proceedings of the 28th National Conference on Artifical Intelligence AAAI, Quebec, QC, Canada, 27–31 July 2014; pp. 2293–2299. [Google Scholar]
- Khaliq, A.A.; Köckemann, U.; Pecora, F.; Saffiotti, A.; Bruno, B.; Recchiuto, C.T.; Sgorbissa, A.; Bui, H.D.; Chong, N.Y. Culturally aware planning and execution of robot actions. In Proceedings of the 2018 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), Madrid, Spain, 1–5 October 2018; pp. 326–332. [Google Scholar]
- Fuller, J. Transcultural Health and Social Care: Development of Culturally Competent Practitioners; Elsevier Health Science: Amsterdam, The Netherlands, 2007. [Google Scholar]
- Laboratories, S.N. JESS. Available online: http://alvarestech.com/temp/fuzzyjess/Jess60/Jess70b7/docs/index.html (accessed on 18 April 2021).
- Browne, P. JBoss Drools Business Rules; Packt Publishing Ltd.: Birmingham, UK, 2009. [Google Scholar]
# | Research Question | Sections |
---|---|---|
1 | What is the application domain? | Section 3.1 |
2 | What is the basic idea and main contribution? | Section 3.2 |
3 | Which development tools have been used? | Section 3.3 |
4 | What is the architecture? | Section 3.4 |
5 | What is the ontology scope? | Section 3.5 |
6 | What is the reasoning scope? | Section 3.6 |
7 | What are the limitations? | Section 3.7 |
KRS | Domain | Application |
---|---|---|
KnowRob | Domestic | Household manipulation task in the kitchen |
OROSU | Medical/hospital | Performs surgical procedures |
CARESSES | Domestic | Culturally competent assistive robot for elderly people |
PMK | Domestic | Indoor manipulation and motion planning to perform tasks such as serving a cup |
SARbot | Domestic | Disaster search and rescue operations |
IEQ | Domestic | An interactive humanoid social robot that provides suggestions |
SmartRules | Domestic | Monitoring and assisting elderly people |
ARBI | Medical/hospital | Performs the duty of robotic receptionist in the hospital |
Worker-cobot | Industrial/manufacturing | Establishes collaboration between human workers and industrial robots |
APRS | Industrial/manufacturing | Kit building |
KRS | Solution/Contribution |
---|---|
KnowRob | Goes beyond the local knowledge bases. Builds knowledge-enabled cloud-based systems. Relies on ontologies and semantic web technologies. Integrates physics simulation-based reasoning and game engine-based rendering techniques. |
OROSU | Integrates ontologies from the health care and robotics fields. Develops KRS for human body surgeries using robots. Tracks robotic actions and maintains pose information in drilling tasks. |
CARESSES | Endows the robot with communication skills through speech, gesture, recognition, and culturally aware capabilities. Enables the robot to change its behavior by adopting an individual’s culture. |
PMK | Formalizes ontological representation for semantic perception and manipulation. Performs TAMP by including sensing information, semantic, geometric, and spatial reasoning with ontological concepts. |
SARbot | Enables robotic search and rescue operations in an unknown environment by: Endowing the robot with high-level control and supporting decision making using the ontology. |
IEQ | Develops the IEQ ontology for monitoring indoor environment quality. Integrates IEQ with post-occupancy evaluation (POE). Enables the robot to make appropriate suggestions based on the individual’s preferences to control the indoor temperature. |
Smart Rules | Overcomes the limitations of standalone robots. Develops a context-aware IoRT knowledge representation system. Allows human-robot interaction in both the physical and cyber world. Deploys rules based on an environment ontology. |
ARBI | Uses symbolic representation for a better understanding of the environment. Develops an integrated model based on the ontology. Endows the robot to perform human-robot interactive services. |
Worker-cobot | Enables the robot to work in collaboration with human workers while sharing the same manufacturing unit. Achieves agile manufacturing through ontology-based MAS and BRMS. |
APRS | Introduces an ontology-based model for kitting process. Empowers the robot with agility. |
KRS | Development Tools |
---|---|
KnowRob | OWL, SWI-Prolog |
OROSU | OWL |
CARESSES | OWL, Bayesian Networks |
PMK | OWL, SWI-Prolog |
SARbot | OWL, SWRL, JESS |
IEQ | Normative |
SmartRules | SmartRules sub-language, -Concept |
ARBI | OWL, Prolog, SPARQL |
Worker-cobot | JADE, ACL |
APRS | XSDL |
KRS | Major Elements of Architecture | |
---|---|---|
KnowRob | Three components | Interface shell, logic-based language, hybrid reasoning shell |
OROSU | Multiple ontologies | Robotic and medical ontologies |
CARESSES | Three modules | Cultural knowledge base (CKB), culturally sensitive planning and execution, culture-aware human-robot interaction |
PMK | Four major components | Perception module, PM framework, TAMP planning, execution module |
SARbot | Three-level control | Low-, middle-, and high-level controls |
IEQ | Seven components | Knowledge base, dialog module, speech recognition module, light_Sound, and Therm_Hygrometricdata acquisition modules, normative reasoner module, suggestion module |
SmartRules | Two software layers | Lower abstract layer, top reasoning layer |
ARBI | Three components | Knowledge manager, task planner, context reasoner |
Worker-cobot | Three steps | Holonic Control Architecture (HCA), knowledge exchange and reasoning step |
APRS | Three models | Kitting workstation, action model, robot capability model |
KRS | Ontologies | Concepts/Classes |
---|---|---|
KnowRob | Inner world, virtual, logical, episodic memories, and DUL | Temporal, spatial, and mathematical things |
OROSU | CORA, human anatomical, PARTS, and SUMO | Medical sensing and manipulation action |
CARESSES | Modular structure | Not defined in [70] |
PMK | Meta ontology | Feature, WSobject, WSpace, actor, sensor, context reasoning, action |
SARbot | Entity, environment, and task | SLAM, object, task, and environment |
IEQ | IEQ | Occupant, environment, and recommendation |
SmartRules | Micro, DUL, | Object, person, and robot |
ARBI | ISRO, user, robot, action, perception, and environment | Person, SocialConcept, object, robot, and event |
Worker-cobot | Agent, agent administrative | Not defined in [76] |
APRS | Three | PointType, PartType |
Kno-wRob | OROUS | CARE-SSES | PMK | SARBot | IEQ | Smart-Rules | ARBI | Worke-rcoBot | APRS | |||
---|---|---|---|---|---|---|---|---|---|---|---|---|
Onto-logy Compo-nents | Object | Definition | D | *D | - | *D | - | - | *D | *D | - | - |
Concept | C | C | *C | C | *C | *C | C | C | *C | *C | ||
Entity | P | P | P | P | P | N | P, V | P | P | P | ||
Type | R,S | H | S | T, G | K | O | M | T, G | S | S | ||
Enviro-nment | Definition | -E | -E | -E | -E | -E | -E | -E | -E | -E | -E | |
Type | S | S | - | S | S, G | - | - | S | - | - | ||
Concept | PI | PI | -PI | PI | PI | -PI | PI | PI | -PI | -PI | ||
Action | Definition | *A | - | -A | - | - | -A | - | - | - | - | |
Concept | E | *E | -E | *E | *E | -E | *E | *E | *E | *E | ||
Cogn-itive Capabi-lities | Inter-action Based on Visual Recogn-ition | 1 | * | - | - | o | o | - | - | - | - | o |
2 | o | - | - | - | - | o | - | |||||
3 | o | - | - | - | o | - | - | |||||
4 | - | o | - | - | - | - | - | |||||
5 | - | - | - | o | - | - | - | |||||
6 | - | - | - | o | o | - | - | |||||
7 | - | - | - | - | o | - | - | |||||
8 | - | - | - | - | o | - | - | |||||
Interaction Based on Voice Recognition | - | - | o | - | - | o | - | o | - | - | ||
Task Execution and Task Planning | o | o | o | o | o | o | o | o | o | o |
KRS | Reasoning Scope |
---|---|
KnowROB | Hybrid reasoning, simulation-based reasoning, motion control reasoning |
OROSU | Action reasoning using HermiT and Pellet reasoners |
CARESSES | Cultural knowledge-based reasoning |
PMK | Perceptual reasoning, reasoning for object features, situation analysis, and planning |
SARbot | Task reasoning |
IEQ | Normative reasoning |
SmartRules | Reactive reasoning |
ARBI | Logical reasoning |
Worker-cobot | Reasoning for interaction |
APRS | Reasoning based on environmental knowledge |
KRS | Research Gaps/ Limitations |
---|---|
KnowRob | Most of the works, however, focused on manipulation tasks only |
OROSU | There is still a need for improvement for aligning medical and robotic ontologies due to the use of different upper ontologies |
CARESSES | The proposed usage of CARESSESS cultural knowledge at a large scale might be challenging in robots with a strong bias; its cultural KB is built by hand with the help of experts |
PMK | Although it provides general knowledge, some concepts are not well defined such as context-aware temporal and spatial relations, sensors’ knowledge, and task representation |
SARbot | In a disaster SAR scenario, there is still a need to study the task planning for multiple heterogeneous robots in an uncertain environment |
IEQ | To implement the IEQ, the environment should be robot-friendly, and the the user should have the correct pronunciation for speech interaction with the robot |
Smart Rules | Some limitations of SmartRules include: the assumption of IoT objects known in advance; it does not deal with novelty autonomously; it needs a better method to bridge the semantic gap between entity descriptions and their representation; it does not support reasoning in the unfamiliar model using the current states of SmartRules and -Concept |
ARBI | It requires extending the knowledge model to support a more socially relatable user experience |
Worker-cobot | Its case study addresses only a few operation resources |
APRS | The APRS project shows that despite significant efforts to improve agility in manufacturing kitting, more research is needed to deal with action failures. |
# | KRS | Applications | Idea and Contribution | Development Tools | Architecture | Ontology Scope | Reasoning Scope | Limitation(s) |
---|---|---|---|---|---|---|---|---|
1 | KnowRob | Table 3: KRS: Applications and Domain Scope | Table 4: KRS: Idea and Contribution | Table 5: KRS: Development Tools | Table 6: KRS: Architectural Components | Table 7: KRS: Ontology Scope Table 8: Ontological Components | Table 9: KRS: Reasoning Scope Table 8: Reasoning -based Cognitive Capabilities | Table 10: KRS: Limitations |
2 | OROSU | |||||||
3 | CARESSES | |||||||
4 | PMK | |||||||
5 | SARbot | |||||||
6 | IEQ | |||||||
7 | SmartRules | |||||||
8 | ARBI | |||||||
9 | Worker-cobot | |||||||
10 | ARPS |
Publisher’s Note: MDPI stays neutral with regard to jurisdictional claims in published maps and institutional affiliations. |
© 2021 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https://creativecommons.org/licenses/by/4.0/).
Share and Cite
Manzoor, S.; Rocha, Y.G.; Joo, S.-H.; Bae, S.-H.; Kim, E.-J.; Joo, K.-J.; Kuc, T.-Y. Ontology-Based Knowledge Representation in Robotic Systems: A Survey Oriented toward Applications. Appl. Sci. 2021, 11, 4324. https://doi.org/10.3390/app11104324
Manzoor S, Rocha YG, Joo S-H, Bae S-H, Kim E-J, Joo K-J, Kuc T-Y. Ontology-Based Knowledge Representation in Robotic Systems: A Survey Oriented toward Applications. Applied Sciences. 2021; 11(10):4324. https://doi.org/10.3390/app11104324
Chicago/Turabian StyleManzoor, Sumaira, Yuri Goncalves Rocha, Sung-Hyeon Joo, Sang-Hyeon Bae, Eun-Jin Kim, Kyeong-Jin Joo, and Tae-Yong Kuc. 2021. "Ontology-Based Knowledge Representation in Robotic Systems: A Survey Oriented toward Applications" Applied Sciences 11, no. 10: 4324. https://doi.org/10.3390/app11104324
APA StyleManzoor, S., Rocha, Y. G., Joo, S.-H., Bae, S.-H., Kim, E.-J., Joo, K.-J., & Kuc, T.-Y. (2021). Ontology-Based Knowledge Representation in Robotic Systems: A Survey Oriented toward Applications. Applied Sciences, 11(10), 4324. https://doi.org/10.3390/app11104324