Developing a Simulation Model for Autonomous Driving Education in the Robobo SmartCity Framework ^†

Abstract

This paper focuses on long-term education in Artificial Intelligence (AI) applied to robotics. Specifically, it presents the Robobo SmartCity educational framework. It is based on two main elements: the smartphone-based robot Robobo and a real model of a smart city. We describe the development of a simulation model of Robobo SmartCity in the CoppeliaSim 3D simulator, implementing both the real mock-up and the model of Robobo. In addition, a set of Python libraries that allow teachers and students to use state-of-the-art algorithms in their education projects is described too.

1. Introduction

This work is focused on an educational framework for teaching intelligent robotics in secondary school or university, developed at the University of Coruña: the Robobo SmartCity. This framework is based on two main elements: (1) the smartphone-based robot Robobo [1], and a (2) model of a smart city, considering both simulation-based and real formats. In this framework, many different challenges and lessons on intelligent robotics can be carried out, mainly focused on the field of self-driving vehicles. It allows teachers to propose challenges dealing with basic problems in robotics, such as control navigation and obstacle avoidance, but also more complex ones that require computer vision, such as traffic sign detection and object identification. The idea of using autonomous driving and smart cities as environments for robotics teaching is becoming quite popular. For example, the authors of [2] proposed a modular and integrated approach towards teaching autonomous driving. Another relevant approach is the autoauto platform [3], which utilizes the concept of self-driving cars for teaching robotics and AI to young students. Costa et al. [4] presented an autonomous driving simulator to gain the attention of and prepare the students to compete in the Portuguese National Robotic Festival (PNRT), especially in the Autonomous Driving Competition (ADC). A very similar approach to the one proposed here is Duckietown [5], an online MOOC for teaching AI and robotics based on self-driving cars, with which the authors have created a whole educational environment, with different city layouts, traffic signals, etc.

2. City Model and Libraries

The smart city used in this project is a scaled model of a city neighborhood (Figure 1 left) represented by a rectangular city layout of 3.5 m × 4 m. This layout is made up of an external two-way road, surrounding a central part that contains a roundabout, where four two-way road sections intersect. There are four areas in the layout containing two buildings, one parking area and one green zone. The main elements related to robotics teaching are the traffic signs (Figure 1 right) that must be detected to control the movement along the city, the lanes that must be followed to avoid running out of the road, and other elements that can be in the city, such as other Robobos or small figures that simulate humans or animals at scale.

As the school’s budget is limited and they may encounter several difficulties both in acquiring and in mounting a real city model like the one presented above, a simulation of it has been created. The model replicates the real smart city mock-up and the real Robobo robot, providing the possibility to transfer the simulation result to the real robot directly. Developing and testing initial solutions in the simulated models and then switching them to real ones using the real Robobo is a highly efficient methodology for most schools.

The model runs under the CoppeliaSim simulator [6]. It is realistic and powerful simulator, especially in terms of the possibilities that the user has to control the scene, the physics, and the dynamics of the simulation. This simulation model is suitable for intermediate and advanced students with digital skills in terms of 3D design and programming. Only Python is supported, and the bridge for using the CoppeliaSim model and the real Robobo is straightforward, which is documented on the Robobo wiki [7].

To allow Robobo to move, sense, or interact with the environment in Robobo SmartCity, a series of libraries with different functionalities have been developed in the scope of this work. The aim of these libraries is to help and guide students in performing realistic programs while learning robotics and intelligence systems through autonomous driving. These libraries are [8]:

• Lane detection. It allows Robobo to detect the lanes of the city model, both the continuous and discontinuous ones. Its implementation is based on the Canny algorithm for edge detection and uses the Hough transformation line detection.
• Object recognition. It allows Robobo to identify the different objects that it may encounter during navigation. These objects can be other Robobos or objects found on the road, such as human-like dolls crossing in a pedestrian crossing, or pet-like dolls. The object recognition system employs a pre-configured artificial neural network (ANN) based on Mobilenet [9] as a recognition algorithm. Although it is possible to download it already pre-trained with a series of objects, it was trained from scratch using the machine learning framework Tensorflow with a set of specific objects relevant for this framework.
• ArUco detection. This library allows ArUco fiducial markers detection, which can be used as artificial landmarks for robot location, or in this case, to identify traffic signs.
• Traffic sign recognition. It allows detecting the vertical traffic signs using the camera, without relying upon ArUco or QR tags. This library is based on a multilayer perceptron ANN and a dataset of real traffic signs.

3. Education Project Example

This section is dedicated to presenting one specific project carried out by a student in his final undergraduate project at the Industrial Engineering school of the University of Coruña, to show the potential of the Robobo SmartCity for teaching advanced concepts in intelligent robotics.

The main objective was to improve the object detection library commented above. First, he had to test the real-time response of the library while taking images from the python stream. This functionality was tested in various situations within the city. For example, the student verified how the library allowed Robobo to avoid collision with other Robobos in the scene, as displayed in the left image of Figure 2. Moreover, to go deeper into his training about deep learning techniques, the student was assigned the task of training the original MobileNet model with new objects. Some of them were images of traffic lights (green, yellow, and red), used to improve the realism of the autonomous circulation of the robot in the city (Figure 2 right).

4. Conclusions

A simulated model of the Robobo SmartCity education framework has been presented, together with some libraries developed to support intelligent robotics teaching at different levels. The model’s abilities have been presented through a specific project carried out by a university student, showing the potentiality of the framework for teaching different concepts of robotics and AI that have impacts in real life.