Next Article in Journal / Special Issue
Leveraging Qualitative Reasoning to Learning Manipulation Tasks
Previous Article in Journal
Deliberation on Design Strategies of Automatic Harvesting Systems: A Survey
Previous Article in Special Issue
How? Why? What? Where? When? Who? Grounding Ontology in the Actions of a Situated Social Agent
Article Menu

Export Article

Open AccessArticle
Robotics 2015, 4(2), 223-252;

Learning Task Knowledge from Dialog and Web Access

School of Computer Science, Carnegie Mellon University, 5000 Forbes Avenue, Pittsburgh, PA 15213, USA
Department of Mechanical Engineering, Eindhoven University of Technology, Den Dolech 2, Eindhoven
State Key Laboratory of Industrial Control Technology, Zhejiang University, 38 Zheda Road, Hangzhou 456555, China
Department of Computer, Control, and Management Engineering "Antonio Ruberti", "Sapienza" University of Rome Via Ariosto 25, Rome 00185, Italy
Author to whom correspondence should be addressed.
Academic Editors: Nicola Bellotto, Nick Hawes and Mohan Sridharan
Received: 20 March 2015 / Revised: 29 May 2015 / Accepted: 5 June 2015 / Published: 17 June 2015
(This article belongs to the Special Issue Representations and Reasoning for Robotics)
Full-Text   |   PDF [10336 KB, uploaded 17 June 2015]   |  


We present KnoWDiaL, an approach for Learning and using task-relevant Knowledge from human-robot Dialog and access to the Web. KnoWDiaL assumes that there is an autonomous agent that performs tasks, as requested by humans through speech. The agent needs to “understand” the request, (i.e., to fully ground the task until it can proceed to plan for and execute it). KnoWDiaL contributes such understanding by using and updating a Knowledge Base, by dialoguing with the user, and by accessing the web. We believe that KnoWDiaL, as we present it, can be applied to general autonomous agents. However, we focus on our work with our autonomous collaborative robot, CoBot, which executes service tasks in a building, moving around and transporting objects between locations. Hence, the knowledge acquired and accessed consists of groundings of language to robot actions, and building locations, persons, and objects. KnoWDiaL handles the interpretation of voice commands, is robust regarding speech recognition errors, and is able to learn commands involving referring expressions in an open domain, (i.e., without requiring a lexicon). We present in detail the multiple components of KnoWDiaL, namely a frame-semantic parser, a probabilistic grounding model, a web-based predicate evaluator, a dialog manager, and the weighted predicate-based Knowledge Base. We illustrate the knowledge access and updates from the dialog and Web access, through detailed and complete examples. We further evaluate the correctness of the predicate instances learned into the Knowledge Base, and show the increase in dialog efficiency as a function of the number of interactions. We have extensively and successfully used KnoWDiaL in CoBot dialoguing and accessing the Web, and extract a few corresponding example sequences from captured videos. View Full-Text
Keywords: knowledge acquisition; knowledge based systems; knowledge transfer; robots; intelligent robots; service robots; mobile robots; human robot interaction; speech; speech recognition knowledge acquisition; knowledge based systems; knowledge transfer; robots; intelligent robots; service robots; mobile robots; human robot interaction; speech; speech recognition

Figure 1

This is an open access article distributed under the Creative Commons Attribution License which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited (CC BY 4.0).

Supplementary material


Share & Cite This Article

MDPI and ACS Style

Perera, V.; Soetens, R.; Kollar, T.; Samadi, M.; Sun, Y.; Nardi, D.; van de Molengraft, R.; Veloso, M. Learning Task Knowledge from Dialog and Web Access. Robotics 2015, 4, 223-252.

Show more citation formats Show less citations formats

Related Articles

Article Metrics

Article Access Statistics



[Return to top]
Robotics EISSN 2218-6581 Published by MDPI AG, Basel, Switzerland RSS E-Mail Table of Contents Alert
Back to Top