IBotAcademy, a Learning Environment to Learn Skills and Competences in Mobile and Manipulator Robotics Using a Problem-Based Learning Methodology in the Industry 4.0 Education

Abstract

Smart industries base their development, maintenance, and operation on technologies compatible with what is known as Industry 4.0. In this context, the education sector needs the implementation of new learning schemes and environments for conceptual and practical learning, thereby supporting students’ training to help in the industrial digital transformation. The main contributions of this work are a learning environment for robotics that integrates software applications that quantitatively measure the student’s learning process; a competence framework focused on learning robotics, ROS background, and developing pickup/delivery tasks very common in the Industry 4.0 context; and a learning path implemented in step-by-step guided documentation that considers the competence framework proposed and progressively teaches ROS concepts. This work also describes the development of the learning environment and the software tools that support the student’s learning process. Two pilot tests were performed on iBotAcademy considering twenty students of the Universidad del Valle. The iBotAcademy mobile robotics and the manipulator robotics learning paths achieved an average of 82% positive answers, and 99% positive answers, respectively. The average coefficients of variation of these tests were 16.4% and 8.3%, showing high data consistency. Then, the iBotAcademy learning environment helps to train students for the industrial digital transformation.

1. Introduction

Currently, Industry 4.0 represents the spearhead of many digital transformation processes in companies at different levels of the automation pyramid, aiming to transform traditional industries into smart industries [1]. In this context, cyber-physical systems, information and communication technologies, artificial intelligence technologies, robotics, mobile devices, and big data are key tools for the development of smart industrial systems. The current global industry investments in IOT developments are US$ 149.2 BN for 2025, and considering a CAGR of 24%, a forecast market size of US$ 1.2TN in 2035 [2]. Then, workers in the industries must learn or update their skills and job competences in accordance with the advance of Industry 4.0; new professionals must learn these skills and competences from their basic training [3]; and these new professionals also should have experience with intelligent robotic systems in Industry 4.0, like IoT platforms, sensors, and smart robots [4].

According to the National Observatory of Information and Communication Technologies (ONTIC) [5], it reports a domestic Digital Gap Index of 0.384 for 2024; this index has been decreasing by 2.2% on average over the last 4 years. Ideally, this index should be close to zero. However, this index has different values depending on the region where it is measured. This index is obtained considering 4 components: motivation of people to include modern technologies in their daily living (component 1), connectivity and device availability (component 2), digital skills of people (component 3), and effective use of modern technologies to get personal benefits (component 4). Considering the analysis presented in [5], between 2023 and 2024, components 2 and 3 were decreasing, and components 1 and 4 were slightly increasing. So, components 2 and 3 are the main aspects to improve to reduce the index. Then, in this context, one of the goals of this work is focused on increasing people’s digital skills.

There is significant global interest in Industry 4.0 not only for its economic impacts but also for its social and educational ones, the cross-cutting nature of robotics as a multidisciplinary field of knowledge, and the growing interest in Industry 4.0 technologies. Higher education institutions require learning tools that foster the skills and competences required by Industry 4.0, such as creative thinking, computational thinking, and teamwork for the integration of technological solutions.

In this context, the research question that this work will try to answer can be stated as follows: how to develop a learning environment where students can be trained using a competence framework focused on Industry 4.0 technologies such as mobile and manipulator robotics? Considering this research question, specific research goals can be stated as follows: first, to design and implement a competence framework that quantitatively measure the student’s learning process; second, to design and implement a learning path inspired in the competence framework such that students progressively learn mobile and manipulator robotics, and ROS concepts using step-by-step guided documentation and software tools; and third, to design and implement a set of software tools to support the student’s learning process.

Then, as shown in Figure 1, the aim of this work is to present a learning environment called iBotAcademy. It is composed by a set of mobile and manipulator robots which run ROS noetic (Turtlebot3 and the Openmanipulator-X), they are used to learn the skills and competences for mobile and manipulator robotics; a hardware setup where the robots are deployed to perform different tasks (warehouse-like environment) which supports the learning process, a learning path (detailed in Section 3.2) with guided lessons to learn the basic knowledge of mobile and manipulator robotics (stored in the PC server connected to the OpenManipulator-X robots), and a software suite which tracks the evolution of the students skills and competences, check the proper operation of robots and allow users program the learning activities remotely.

Figure 1. iBotAcademy learning environment.

Considering the background review, the main contributions of this work are as follows:

• A learning environment for mobile and manipulator robotics that integrates software applications that quantitatively measure the student’s learning progress.
• A competence framework that is used to measure the learning process quantitatively and includes learning objectives and learning activity objectives for mobile and manipulator robotics focused on implementing a common job in Industry 4.0 as pickup/delivery tasks.
• A learning path for mobile and manipulator robotics that considers the competence framework proposed, and where students also progressively learn ROS concepts. Here, it is assumed that students have no previous knowledge about ROS, nor mobile or manipulator robotics.

This paper is organized as follows: Section 2 describes the related works; Section 3 and Section 4 describe the design and implementation of the learning environment; Section 5 shows the pilot tests and the results achieved; and Section 6 presents the conclusions.

2. Related Work

In this section, Table 1 shows a comparison of learning environments related to Industry 4.0 and robotics. These works were reviewed and analyzed considering scientific databases such as IEEE, Scopus, and ScienceDirect. An example of the search equation is shown as follows: TAK((learning environment OR robotics education) AND competence-based AND mobile robot AND manipulator robot AND Industry 4.0). Mainly, the publication period selected ranges from 2018 to 2025, which initially retrieved approximately 400 publications. Then, in this work, the following inclusion criteria were applied: the framework used (FW), programming languages employed (L. Prog.), robotic platforms, whether simulation is supported, access of students to the learning environment, sensors used by the robots, learning method assumed (L. Met.), and the learning context (L. Cont.). The most important contributions of this work are related to measuring quantitatively the competence framework proposed and developing software tools to measure and support the student’s learning process. Then, the works presented in Table 1 consider this subject.

Table 1. Related work (Ref. stands for reference; L. Prog. stands for language programming; Sim. stands for simulation; L. Met. stands for learning methodology; L. Cont. stands for learning context).

Ref.	Year	FW	L. Prog.	Robots	Sim.	Access	Sensors	L. Met.	L. Cont.
[1]	2022	ROS, JADE	Python, C++	Custom mobile robot	Yes	Both	2D lidar, camera, gas sensors	Competences	Formal and not formal
[3]	2022	None	Python	ArduinoBot, Educational Arm	No	In person	Camera, proximity	Constructive	Formal
[6]	2021	Custom	Python, C++, Java	Multiple AGV, Industrial arm	Yes	In person	2D lidar, prox., cameras, IMU,	Competences, problem-based	Formal
[7]	2020	OMIlab	PetriNet, UML, EER, BPMN, EPC	Industrial arm	No	In person	Ceiling camera	Design thinking	Formal
[8]	2019	ROS	Python	Docker, JupyterLab	Yes	Both	Virtual	Problem-based	Formal
[9]	2019	ROS	Python, C++	Lego NXT, UR10, Roboie-X	Yes	In person	Lidar, RGBD, Pos.	Problem-based	Formal
[10]	2019	ROS	C++	Multiple WSBots	Yes	In person	Ceiling camera	Constructive	Formal
[11]	2020	Many	Many	15 Industrial arms manuf. Cells	Yes	In person	2D/3D lidar, cameras	Constructive	Formal
[12]	2021	ROS, FlexBe	Python, C++	Franka robot arms	Yes	In person	Torque, position, RGB	Constructive	Formal
[13]	2023	Many	Python, C++, Java	AGV, AMR, Industrial arm	Yes	In person	2D lidar, pos., camera, torque	Competences, problem-based	Formal
[14]	2022	ROS, FlexBe	Python	Ufactory, Xarm 6, DashGo B1	Yes	In person	2D/3D lidar, cameras	Constructive	Formal
[15]	2022	ROS, SimTwo	Pascal	Custom mobile robots	Yes	In person	2D lidar, cameras	Constructive	Formal.
[16]	2023	SPOT SDK	Python	SPOT Boston Dyn.	Yes	In person	Cameras, 3D lidar	Problem-based	Formal
[17]	2023	ROS	C++	TI-RSLK	No	In person	2D lidar, prox.	Constructive	Not formal
[18]	2023	ROS	Python	Many	Yes	Both	Many	Problem-based	Not formal
[19]	2024	None	Blockly	EV3 Lego	No	Remote	Position, light, heading, angle	Problem-based	Not formal
[20]	2025	Robot SDK	Python, Blockly	Wlkata Mirobots, manuf. Cell.	No	In person	Position, angle, cameras	Competences	Not formal
[21]	2025	None	Python	Many	No	In person	Many	Problem-based	Formal
[22]	2025	Robot SDK	Block	UR5	No	In person	Angle, camera	Problem-based	Formal
[23]	2025	ROS	Python	Custom mobile robot	No	In person	IMU, Lidar, odometry	Problem-based	Not formal
[24]	2025	ROS	Python	Custom mobile robot	No	In person	IMU, Lidr, odometry	None	Not formal
[25]	2025	ROS	Python, C++	Many	Yes	In person	Many	Problem-based	Not formal

Looking at Table 1, it is noticeable that the presence of a framework such as ROS (Robotic Operating System) [26] facilitates portability and the simulation of robot behavior. In [25], ROS plays a crucial role in integrating different tasks across heterogeneous robotic platforms. Observing works such as [8,10,12,13,14,18,26] they focus on solving specific application problems using ROS, but they do not introduce the basic ROS concepts; they assume students know the ROS background. This could be challenging, since it is well known that the ROS learning curve is not easy to handle. In cases such [3,6,16,19,21], which have proprietary frameworks or no framework at all, the simulation of robot behavior is not available, or the developments performed by students cannot be shared with the community.

Another crucial factor is the programming language. According to Table 1, popular languages are C++ and Python, supported by frameworks such as ROS, OmiLab [7], and SimTwo [15]. Another programming language is Java, although not as widely used as Python and C++. To include a broader range of students’ ages, works such as [19,20] use Blockly, which is understood by students without previous knowledge of programming.

It is important that a learning environment for Industry 4.0 using robotics has real-world robots to solve tasks related to smart industries. Table 1 shows the use of various robotic platforms: simulated robots as described in [8]; combinations of mobile robots and industrial robotic arms as in [3,6,9,13,16,25]; multiple mobile robots as in [1,6,10,15,17]; customized prototype robots as in [1,15,17,21,23,24]; and industrial robotic arms as in [6,7,11,12,14,20,22]. The number of learning environments using manipulator robots is greater compared to mobile robots, because the former are essential in manufacturing cells, a well-known concept in automation pyramids. However, students involved in a learning process need to test the proper operation of the robotics platforms; the works reviewed do not show clear evidence of developing software tools to support this aspect.

Table 1 shows that the reviewed works can be divided into the following categories considering the information processing: information processing and access at the user interface level [6,7,8,11,17]; information processing and access using a user interface and directly on the robots [1,3,9,12,14,15,18,22]; and information processing and access at the robot level only [10,20,21,23,24] Then, processing and accessing the information through GUI and robots is the better option, since users can test the proper operation of robotic platforms before starting the learning process. Considering Table 1, the works reviewed gave little attention to this aspect.

Regarding the shift in learning methodologies following the COVID-19 pandemic, remote learning environments are important. According to Table 1, most of the environments consulted must be accessed in person, which is understandable since most have hardware/software deployment that requires an important level of user interaction. However, this interaction can also occur as described in [1,8,18].

As shown in Table 1, most of the consulted studies use RGB, RGBD, 2D LiDAR, or 3D LiDAR cameras to perceive the environment. In [3,9,13,20], position and torque sensors for the robot arm are available to provide feedback on the state of the robots involved.

The learning methodology is crucial to ensure students retain their knowledge as expected. Traditionally, the constructivist method has been used in engineering, and studies such as [3,10,11,12,14,15,17] implement this methodology, which has shown good results over the years. The current trend of the Ministry of National Education in Colombia is focused on competency-based assessment, as well as in Europe with the European Qualifications Framework. In this context, refs. [1,6,13,20] incorporate competency-based assessment, and in some cases, such as [6,13], this is combined with problem-based learning (PBL) and FabLab. In [7], it involves strategies such as Design Thinking. Also, the problem-based methodology is used in works such as [8,16,18,21]. In [18,21,22,25], the PBL methodology is needed since those works are focused on the development of a final solution; it is assumed that students already have important skills in ROS-based development; however, those works do not introduce the ROS background into the learning process.

In [1], the organization of competences and learning outcomes proposed by the authors is highly technical. They focus on the introduction to mobile robotics and ROS, on proposing a solution to a specific problem, and on ensuring the connectivity of this solution using IIoT technologies. In [6,13], the authors propose an organization of competences and learning outcomes focused on the creative process of new products. In [20], the authors propose relevant competences of workers for Industry 4.0 and 5.0, such as training operators in a risk-free environment, analyzing material flow, human–robot collaboration, and introducing Lean Manufacturing and Just-In-Time. Therefore, the learning processes and technological tools must be guaranteed [1], and then these results should be scaled to the curricular level, as the authors have done in [6,13]. It is worth noting that works such as [1,6,13,20] propose competence-based frameworks for the learning processes; however, those works do not measure the competence frameworks quantitatively, nor do they show a software tool to do that.

An interesting characteristic that could be part of the social outreach strategies of universities is not formal learning. This latter case is supported by studies such [1,17,18,20,23,24,25].

Considering the related work presented, it is worth noting that there is not clear evidence of quantitatively measuring the student’s learning process specifically when a competence-based framework is assumed; in addition, in works such as [6,13,20] a test bed, a competence framework and guided documentation is presented, however, they are not integrated into a learning environment, nor measuring quantitively the competence framework, nor including ROS background into the learning process and neither support the student’s learning process with software tools to test the mobile robotics platforms. Then, the aim of this work is to address these issues proposing the iBotAcademy learning environment answering the research question proposed in Section 1, which includes several properties observed in the literature such as: adopting a framework as ROS; enabling a high-level programming language with object-oriented programming support, such as Python or C++; a robotic setup that combines mobile and manipulator robots; a data processing system with user interface; support for remote and on-site access; the availability of RGB sensors, 2D LiDAR, and position sensors for manipulator robots; software support for the progressive monitoring of learning outcomes; and a problem-based [27] and competency-based learning methodology which is able of measuring quantitatively the student’s learning process. It is worth noting that, observing Table 1, few works combine these two learning methodologies [6,13]; then, this work has found the combination of these methodologies to be very advantageous. Additionally, this work differs from those presented in Table 1 in the following ways: two robotic platforms work together with the aim of developing pickup/delivery tasks, a competence framework is proposed to learn mobile and manipulator robotics concepts, a quantitative method of evaluation of these competences is also proposed; and iBotAcademy can be used in formal and informal education.

3. Design of the Learning Environment

The iBotAcademy is a learning environment for mobile and manipulator robots. Considering the related works, the learning environment was designed bearing in mind the following needs: first, undertake a robotics learning path with an aim in mind in the context of the Industry 4.0; second, measuring quantitatively the learning progress using the competences framework; and, a testbed environment for experimenting simulated and real robots running a widely accepted framework like ROS.

Figure 1 shows the iBotAcademy learning environment structure, which is composed of the following parts:

• User interfaces: Students interact with iBotAcademy through two main interfaces, first the iBotAcademy curriculum management tool to get access to the learning path and it measures quantitatively the competences-based design proposed in Section 3.1; it is worth noting that this software tool will not be detailed in this work; but, Section 4.3 describes the how the competences-based design is quantitatively measured. Second, students use their laptops to program mobile and manipulator robots in Python using ROS Noetic. Additionally, using the iBotAcademy curriculum management, teachers can perform curricular management tasks as well as monitor students’ performance thanks to the ROS grading packages developed in this work.
• Learning path: Considering the related work, the learning process in robotics is improved if there is a problem to solve. iBotAcademy was initially developed in the context of pickup and delivery tasks, which are common tasks in the Industry 4.0 context. Then, students must learn how mobile and manipulator robots can be programmed to do different jobs such as dealing with perception, map building, localization, path planning, image segmentation, and grasping. All these concepts are introduced while the ROS background is also presented in the guided documentation. It is assumed that students have no previous knowledge about ROS, nor mobile nor manipulator robotics.
• Mobile robot tools: The mobile robot concepts are learnt using real and simulated robots. The iBotAcademy uses TurtleBot3 mobile robots. These robots are monitored using a desktop software tool. This software tool is not explicitly found in related work, and it is needed to test the mobile robotics platforms properly to support the learning process.
• Manipulator robot tools: Similar to mobile robots, another desktop software tool was developed to monitor different OpenManipulator-X robots. Similarly, in this case, the manipulator robots need this software tool to test the robotic platform to support the learning process.
• Experimentation environment: iBotAcademy includes a warehouse-like setting which involves a limited area of work using walls and hallways, defined places for loading or unloading goods, and racks where to store objects.

3.1. Competence-Based Design

iBotAcademy considers the competency-based framework in the context of Industry 4.0, such that real-world applications such as pickup and delivery can be implemented using mobile and manipulator robots. It is worth noting that implementing a problem-based learning methodology brings context to the learning process. In this work, the main problem to solve is to develop a ROS solution using mobile and manipulator robots for a pickup/delivery task. The proposed competences are shown in detail in Table 2.

Table 2. Competences, learning outcomes, learning activities outcomes, exercises, grading and survey item (M for mobile robotics, and A for manipulator robotics).

Competences	Learning Outcomes	Learning Activities Outcomes	Exercise	Grading	Survey Item
C1—Apply basic navigation concepts using laser sensors, cameras and odometry using iBotAcademy learning environment to train students in computational thinking skills and in the context of perception tasks in robotics.	LO1.1 Apply odometry-based navigation concepts using iBotAcademy learning environment to develop computational solutions where mobile robots can reach specific positions in the work environment.	LAO1.1.1 Develop a software package using odometry-based navigation concepts.	To publish odometry data in ROS topics, draw a square, and drive the mobile robot to defined positions	ROS package existence; existence of folders, scripts and permissions; existence of user ROS topics; data on user ROS topics.	M1, M2, M3, M4, M5, M6
		LAO1.1.2 The mobile robot reaches 3 defined positions in its working environment.
	LO1.2 Apply laser sensor-based navigation concepts using iBotAcademy learning environment to develop computational solutions where mobile robots can build a map of the environment and avoid obstacles.	LAO1.2.1 Obtains a map of the environment using the mobile robot’s laser sensor.	Publishing relative polar coordinates of obstacles on a ROS topic and driving the mobile robot while publishing the obstacles found in a ROS topic	ROS package existence; existence of folders, scripts and permissions; existence of user ROS topics; data on user ROS topics; robot’s arrival to target position.	M1, M2, M3, M4, M5, M7, M8
		LAO1.2.2 The mobile robot avoids obstacles around it using the previously calculated map.
	LO1.3 Apply camera-based navigation concepts using iBotAcademy learning environment to develop computational solutions where robots can select and grasp objects	LAO1.3.1 The manipulator robot selects the object of interest.	Developing a solution to move the gripper and segment objects by color and publish these data in a ROS topic	ROS package existence; existence of folders, scripts and permissions; existence of user ROS topics; data on user ROS topics; robot’s grasping of objects of interest.	A1, A2, A3, A5, A6, A7, A8, A9, A10
		LAO1.3.2 The manipulator robot grasps the selected object of interest.
C2—Apply mobile robotics concepts to goods collection and dispatch tasks using the iBotAcademy learning environment to train students in computational thinking skills.	LO2.1—Apply concepts of automatic mapping of work environments and selecting the target point to develop computational solutions in collecting/dispatching tasks.	LAO2.1.1 Develop a software package that computes a map of the environment.	Developing a ROS package to build a map using GMapping and serving a map in ROS to be used by other nodes.	ROS package existence; existence of folders, scripts and permissions; existence of user ROS topics; data on user ROS topics; existence of map computed.	M1, M2, M3, M4, M5, M10
	LO2.2—Apply automatic route planning concepts in a static environment to develop computational solutions in goods collection and dispatch tasks.	LAO2.2.1 Develop a software package to capture the localization of the robot in at least 3 target positions using a map of the environment.	Using a previously built map to localize the mobile robot; then, using the global localization service of ROS; and developing a solution to implement a ROS service to save at least 3 robot positions captured from the AMCL node.	ROS package existence; existence of folders, scripts and permissions; existence of user ROS topics and services; data on user ROS topics and service; existence file with at least 3 positions in the map.	M1, M2, M3, M4, M5, M11
		LAO2.2.2 Develop a software package that performs point-to-point route planning in the work environment.	Developing a ROS package to plan robots’ path of the three previously stored locations; then, visit the target positions sequentially	ROS package existence; existence of folders, scripts and permissions; existence of user ROS topics; data on user ROS topics; robot’s arrival to target positions.	M1, M2, M3, M4, M5, M11
		LAO2.2.3 The mobile robot visits 3 target positions.
C3—Apply perception concepts in robotics to object grasping and storage tasks using the iBotAcademy learning environment to train students in computational thinking skills.	LO3.1—Apply concepts of manual navigation of the manipulator robot griper using the iBotAcademy learning environment to develop computational solutions in tasks of grasping, locating, and storing objects of interest.	LAO3.1.1 The manipulator robot is manually commanded, grasping an object of interest and placing it in a defined 2D position.	Developing a solution to move the gripper at least three positions in 3D using ROS services	ROS package existence; existence of folders, scripts and permissions; existence of user ROS service; data on user ROS service; robot’s arrival to target position.	M1, M2, M3, M4, M5
		LAO3.1.2 The manipulator robot is manually commanded, grasping an object of interest and storing it in a defined cell in a rack.
	LO3.2—Apply camera-assisted navigation concepts of the robot manipulator’s griper using the iBotAcademy learning environment to develop computational solutions in tasks of grasping, locating, and storing objects of interest.	LAO3.2.1 The manipulator robot is automatically commanded, grasping an object of interest and placing it in a defined 2D position using the griper and its camera.	Developing a ROS package to move two objects of interest segmented by color and shape to two distinct positions in 3D space	ROS package existence; existence of folders, scripts and permissions; existence of user ROS service and topics; data on user ROS service and topics; robot’s arrival to target position.	A1, A2, A3, A4 A5, A6, A7, A8, A9, A10
		LAO3.2.2 The manipulator robot is automatically commanded, grasping an object of interest and storing it in a defined cell in a rack using the griper and its camera
C4—Applies task integration considering perception, mapping, localization and planning of mobile robots and manipulators in goods pickup and delivery processes using the iBotAcademy learning environment to train students in computer service development skills	LO4.1—Applies task integration for perception, mapping, localization and planning in mobile robots and manipulators in goods pickup processes using the iBotAcademy learning environment to train students in software development skills.	LAO4.1.1 Integrates perception, mapping, localization, and planning tasks of mobile and manipulator robots into goods picking software application.	Picking task.	ROS package existence; existence of folders, scripts and permissions; existence of user ROS actions; data on user ROS actions; robot’s arrival to target positions, picking up the requested objects and returning to home position.	M1, M2, M3, M4, M5, A11, A12, A13
		LAO4.1.2 Mobile robots and manipulators perform the task of collecting goods which will be brought to the base station.
	LO4.2—Applies task integration for perception, mapping, localization and planning in mobile robots and manipulators in goods delivery processes using the iBotAcademy learning environment to train students in software development skills.	LAO4.2.1 Integrates perception, mapping, localization, and planning tasks of mobile and manipulator robots into a goods dispatch software application.	Delivery task.	ROS package existence; existence of folders, scripts and permissions; existence of user ROS actions; data on user ROS actions; robot’s arrival to target positions, delivering the requested objects and returning to home position.	M1, M2, M3, M4, M5, A11, A12, A13
		LAO4.2.2 Mobile robots and manipulators perform the task of dispatching goods which will be placed in the previously defined positions.

Pickup and delivery tasks in Industry 4.0 for mobile and manipulator robotics require C1, C2, C3 and C4 competences, which includes applying concepts such as mobile robot navigation using basic sensors and 2D lidars; manipulator robot concepts on motion and grasping of specific objects, map building, localization and path planning for mobile robots; and integrate all these concepts together using ROS to implement the pickup and delivery tasks.

To achieve competence C1, three learning outcomes must be satisfied (LO1.1, LO1.2 and LO1.3). These LOs are related to dealing with mobile and manipulator robotic sensors. To measure whether students achieve these LOs, learning activity outcomes are proposed in Table 2 (LAO1.1.1, LAO1.1.2, LAO1.2.1, and LAO1.2.2). These LAOs include developing packages using the ROS framework, driving the robot to specific positions in the environment, building a map with the 2D lidar, and avoiding obstacles. Also, managing the camera input data for the manipulator robot to recognize objects of specified color or shape.

Competence C2 is related to mobile robotics; students use the learnt concepts to build maps, locate a mobile robot in a map, and plan routes. To do so, LO2.1 and LO2.2 are proposed to finally achieve a mobile robot that safely navigates between two distinct locations in the robot’s environment. The LAOs proposed (LAO2.1.1, LAO2.2.1 and LAO2.2.2) allow measuring these LOs.

On the other hand, competence C3 is related to moving the manipulator robot manually or automatically to specific locations in the robot’s working space where objects of interest are placed. Two LOs are proposed in this competence (LO3.1 and LO3.2); they are focused on performing the grasping task manually (teleoperating the robot) and automatically (programming the robot). Four LAOs (LAO3.1.1, LAO3.1.2, LAO3.2.1 and LAO3.2.2) are proposed to measure the LOs of competence C3, where users will move the robot in 2D and 3D spaces to grasp the objects of interest.

Finally, all these concepts are integrated into competence C4, where the pickup and delivery solution is developed. Then, two LOs are proposed (LO4.1 and LO4.2) in which the pickup and delivery tasks are solved in two different solutions. Four LAOs are proposed (LAO 4.1.1, LAO4.1.2, LAO 4.2.1, and LAO4.2.2) to measure whether students complete the pickup and delivery tasks. LAO4.1.1 and LAO4.2.1 are related to mobile robotics tasks such as mapping, localization, planning, and obstacle avoidance. LAO4.1.2 and LAO4.2.2 are related to manipulator robotics concepts such as segmenting the objects, moving the robot, and grasping the object. The handshaking between the mobile and manipulator robots must be managed by these solutions using services and actions of ROS.

3.2. Learning Path Proposed

This section has two components: first, the mobile and manipulator robotics subjects; and second, the final problem that students should solve at the end of the learning path. As the learning path is focused on problem-based learning, students will face two types of problems to solve as follows: first, small ROS-based solutions presented at every step-by-step guide documentation, and second, the final ROS-based solution concerning pickup/delivery tasks integrating mobile and manipulator robotics.

iBotAcademy has an experimentation setting to test ROS-based solutions. However, learning robotics concepts and getting familiar with ROS is not easy. For this reason, a learning path is proposed as shown in Figure 2a, which considers the competence-based design described in Table 2. In the middle of Figure 2a are shown the target robots used; independently, the robotic platform and all subjects belonging to the learning path use the problem-based methodology to introduce the concepts, but in a learning context. It is assumed that students have no previous knowledge about ROS, neither mobile nor manipulator robotics. For the mobile robot TurtleBot3, students should gain knowledge about the following subjects:

Figure 2. (a) Learning path proposed. (b) Sequence diagram for the pickup and delivery task.

• Odometry-based navigation: In this lesson, the goal of students is to develop a ROS-based solution to publish odometry data in ROS topics, draw a square, and drive the mobile robot to defined positions. To solve these problems, students find concepts related to robot motion models, odometry sensors, and IMU sensors using ROS in the step-by-step guided documentation. This lesson is related to LO1.1, and it includes two types of LAOs (LAO1.1.1 and LAO1.1.2), guided activities where students check how the concepts are working, and hands-on activities where students should develop a solution to specific problems. Once students develop a ROS-based solution that solves the problems proposed, each solution is assessed using specific rubrics. These are measured by a grading program developed in ROS, which assigns a grade value for each hands-on activity. This grade is assigned to LAO1.1.1 and LAO1.1.2.
• Laser-based navigation: Here, students must solve the following robotics problems: publishing relative polar coordinates of obstacles on a ROS topic and driving the mobile robot while publishing the obstacles found on a ROS topic. To do so, students learn concepts related to using 2D lidar and avoiding obstacles using potential fields [28] and ROS. Guided learning activities include the development of ROS packages to use 2D lidar data to drive the mobile robot and avoid obstacles using potential fields. Two LAOs (LAO1.2.1 and LAO1.2.2) are related to this lesson. Once students develop a ROS-based solution that solves the problems proposed, each solution is assessed considering specific rubrics. They are measured by a grading program developed in ROS, and it assigns a grade value for each hands-on activity. This grade is assigned to LAO1.2.1 and LAO1.2.2.
• Map building: In this lesson, students have to solve the following robotics problems: developing a ROS package to build a map using GMapping [29] and serving a map in ROS to be used by other nodes. Then, this lesson teaches students concepts related to types of maps in mobile robotics, map building, sensors used to build them, and how they will be used using ROS. Once students develop a ROS-based solution that solves the problem proposed, which corresponds to LAO2.1.1, the solution is assessed using a rating program developed in ROS, and students must run it to get a measure of LAO2.1.1.
• Robot localization: Here, students must solve the following robotics problems: the first uses a previously built map to localize the mobile robot; the second uses the global localization service of ROS; and the third involves developing a solution to implement a ROS service to save at least 3 robot positions captured from the AMCL node. To do so, students gain knowledge about methods of mobile robot localization, especially particle filters [30] using ROS. Once students develop a ROS-based solution that solves the proposed problem, which corresponds to LAO2.2.1, the solution is assessed using a rating program developed in ROS to get a grade value for LAO2.2.1.
• Path planning: In this lesson, students must solve the following robotics problems: developing a ROS package to plan a robot’s path between the three previously stored locations; then, visit the target positions sequentially. Then, this lesson teaches students concepts of local and global path planning [30] using ROS. The guided activities describe how to configure ROS packages to use the move_base node, then how to guide the mobile robot to a specific goal point, and how the mobile robot responds to dynamic obstacles not previously mapped. Once students develop a ROS-based solution that solves the proposed problem, which corresponds to LAO2.2.2 and LAO2.2.3, the solution is assessed using a rating program developed in ROS, and students must run it to get a grade value for LAO2.2.2 and LAO2.2.3.

In the case of robot manipulators (Figure 2a), students should gain knowledge about the following subjects:

• Robot tele-operation: Here, students must solve the following robotics problem: developing a solution to move the gripper at least three positions in 3D using ROS services. To achieve this, students learn basic concepts of direct and inverse kinematic models. ROS packages such as moveit are used to discover these concepts. Also, students move the OpenManipulator-X robot using the original controllers and activate the gripper. Once students develop a ROS-based solution that solves the proposed problem, the solution is assessed when students run the rating program developed in ROS to get a grade value for LAO3.1.1 and LAO3.1.2.
• Object segmentation by color and shape: In this lesson, students must solve the following robotics problem: developing a solution to move the gripper and segment objects by color and publish these data in a ROS topic. Then, this lesson teaches students basic concepts of computer vision to segment objects depending on their color and shape using ROS. Once students develop a ROS-based solution that solves the proposed problem, which corresponds to LAO1.3.1 and LAO1.3.2, the solution is assessed using a rating program developed in ROS, and students must run it to get a measure of LAO1.3.1 and LAO1.3.2.
• Manual and automatic driving of the robot to grasp objects: In this lesson, students must solve the following robotics problem: developing a ROS package to move two objects of interest segmented by color and shape to two distinct positions in 3D space. Then, this lesson teaches students basic computer vision concepts to have an object of interest in the field of view of the camera and then grasp the object using ROS. This is done manually using teleoperation commands, then automatically using a program in Python. Once students develop a ROS-based solution that solves the proposed problem, which corresponds to LAO3.2.1 and LAO3.2.2, the solution is assessed using a rating program developed in ROS to get a grade value for LAO3.2.1 and LAO3.2.2.

The integration between the mobile and manipulator robotics occurs in the pickup/delivery solution, which students must implement at the end of this learning path. Figure 2b shows a sequence diagram of the proposed task. Here, two manipulator robots and one mobile robot are used, the sequence of events are as follows: the mobile robot runs a ROS action server to manage all the pickup (LAO4.1.1 and LAO4.1.2) or delivery (LAO4.2.1 and LAO4.2.2)task, the two manipulator robots runs a ROS action client to request pickup or delivery tasks; over all the execution of the action server, it sends feedback messages to the corresponding manipulator robots informing the position of the mobile robot; when the mobile robot arrives to the requested location, the mobile robot uses a ROS action client to request the load or unload task of specific objects to the manipulator robot; the manipulator robot runs a ROS action server to manage the object loading or unloading; at the end, the mobile robot drives to the next pickup or delivery location; finally, the mobile robot drives to the home location to finish the task.

3.3. Robot Software Tools Design

In iBotAcademy, students have access to two monitor software tools which are used to check if the TurtleBot3 mobile robot and the OpenManipulator-X robot are properly working. The iBotAcademy learning environment differs from other studies by providing these software tools to support students in the learning process, specifically when they are developing the ROS packages in the guided learning activities. These software tools were developed considering the Rational Unified Process methodology (RUP) [31]. In this case, functional, nonfunctional requirements, and conceptual diagrams are only included in this document due to space constraints.

In the case of the Turtlebot3 monitor software tool, the functional requirements are described as follows and plotted conceptually in Figure 3a: users can observe sensor data (2D lidar, odometry and IMU); users can remotely operate multiple mobile robot Turtlebot3; users can connect to real or simulated mobile robot; users can select between different mobile robots Turtlebot3 (real or simulated); and the software tool communicates to the mobile robot using WIFI.

Figure 3. (a) Conceptual diagram of Turtlebot3 monitor software tool. (b) Conceptual diagram of OpenManipulator-X monitor software tool.

In the case of the OpenManipulator-X monitor software tool (briefly called EduArm), the functional requirements are described as follows and plotted conceptually in Figure 3b: users can connect to the real or simulated OpenManipulator-X robot; users can get the angular or cartesian state of the robot; users can test color and shape segmentation of objects using the on-board camera; this software tool communicates to the robot wirelessly; and different positions in 3D can be stored and configured to move the robot to those positions.

The nonfunctional requirements for both software tools are as follows: the SO used was Ubuntu 20.04, the ROS framework used was Noetic, the programming language was Python 3.9, the QTdesigner package used was 6.0, and the simulation tool used was Gazebo 11.

4. Implementation of the Learning Environment

The iBotAcademy learning environment proposed in this work follows the learning process pipeline shown in Figure 2a. Current students who use iBotAcademy have two options: first, start the learning process concerning mobile robotics concepts; or, second, start the manipulator robotics concepts.

In general, the learning process has the following steps: first, students review the background concepts of each lesson which includes the robotics and ROS related concepts; second, students run and comprehend the guided learning activities using ROS in the simulated and real mobile or manipulator robot, and its surrounding environment (to test the robot hardware in steps 1 and 2); third, at the end of each lesson, students develop ROS hands-on activities, these learning activities are needed to pass to the next lesson, then, the grading ROS packages are run to measure these learning activities (to do this, the grading ROS packages measure the student’s performance); finally, a pickup/delivery solution must be developed by students, and the success of it is measured by the ROS grading packages.

It is important to understand the role of the iBotAcademy software tool to consolidate all the gradings to evaluate the student’s performance regarding the competences stated in Section 3.1.

4.1. Competence Evaluation in iBotAcademy

In Figure 2a, the learning process pipeline in iBotAcademy is shown. The grading packages were developed in ROS to evaluate the learning activities. The competence-based design proposed in Section 3.1 is evaluated quantitatively in the iBotAcademy learning environment, which differs from works described in the Section 2. To do that, Figure 4 shows the interface of these grading packages with the iBotAcademy curriculum management tool. The grading process is as follows: first, students carefully read the exercise description; second, students develop a solution in ROS to satisfy the exercise requirements; third, students run their solution and grading packages; finally, grading packages give students their grading value.

Figure 4. Grading package interface with iBotAcademy curriculum management tool.

In stage three, grading packages subscribe to the ROS topics required in the requested solution. This allows the grading packages to evaluate the learning activities. The grading packages consolidate all the evaluation aspects in a JSON file, which is sent to the iBotAcademy curriculum management tool, and then students can observe the grading result on the console screen.

To understand the grading process, Figure 5 shows the tree model of competences, LO, and LAO. It shows graphically what is detailed in Table 2. Given a competence C_i; it is composed by one or many LO₁ to LO_j; each LO is composed by one or several LAO₁ to LAO_k; in turn, to satisfy each LAO, one or several learning activities LA₁ to LA_m are performed by students; finally, each LA has one or many aspects to evaluate (P₁ to P_p).

Figure 5. Competence, learning outcomes, learning activities outcomes, and learning activities tree.

To quantitatively evaluate student progress in iBotAcademy, the following procedure is proposed: first, each LA is composed of one or many exercises that students must perform; each exercise contributes a percentage to the LA grade. This calculation is shown in (1).

$$E_m^{LA}={\sum }_p{s}_{(m,p)}^P\ast G$$

(1)

where G is the top grade (i.e., in the Universidad del Valle this value is 5.0), $s_{(m,p)}^P$ is the score obtained by the student for each exercise p expressed in percentage in LAm, and $E_m^{LA}$ is the quantitative evaluation of the LAm (which ranges from 0 to G). Then, one or several LA are needed to evaluate a LAO; this is shown in Equation (2).

$$E_k^{LAO}={\sum }_m{w}_{(k,m)}^{LA}{E}_m^{LA}$$

(2)

where $w_{(k, m)}^{LA}$ is the weight of each LA evaluation (which ranges from 0 to 1.0); in $E_m^{LA}$, m is the LA and $E_k^{LAO}$ is the quantitative evaluation of the LAO_k (which ranges from 0 to G). Afterwards, one or many LAOs are used to evaluate a LO as shown in Equation (3).

$$E_j^{LO}={\sum }_k{w}_{(k,j)}^{LAO}{E}_k^{LAO}$$

(3)

where $w_{(k, j)}^{LAO}$ is the weight of each LAO evaluations (which ranges from 0 to 1.0); in $E_k^{LAO}$, k is the LAO and $E_j^{LO}$ is the quantitative evaluation of the LOj (which ranges from 0 to G). Finally, one or more LOs are needed to evaluate a competence as depicted in Equation (4).

$$E_i^C={\sum }_j{w}_{(i,j)}^{LO}{E}_j^{LO}$$

(4)

where $w_{(i, j)}^{LO}$ is the weight of each LO evaluation (which ranges from 0 to 1.0); in $E_j^{LO}$, j is the LO, and $E_i^C$ is the quantitative evaluation of the competence i (which ranges from 0 to G). Putting together Equations (1)–(3) in Equation (4) yields Equation (5), which consolidates the quantitative evaluation of competence i.

$$E_i^C={\sum }_j{w}_{(i,j)}^{LO}{\sum }_k{w}_{(k,j)}^{LAO}{\sum }_m{w}_{(k,m)}^{LA}{\sum }_p{s}_{(m,p)}^P$$

(5)

Each LA proposed in the learning path in Section 3.2 has several exercises that students must accomplish. The grading ROS packages evaluate those exercises according to the pseudocode presented in Algorithm 1. It first initializes different variables required to check each exercise (exercise_params), the weights of each exercise (exercise_weights), which software entity is evaluated (exercise_parts), and initializes the grades set. Then, the token is obtained and validated from the iBotAcademy software tool; if this token is not valid, the evaluation is not performed. Afterwards, all the LAs are evaluated, and considering their weights, a grade is obtained. Finally, this grade is sent to the iBotAcademy software tool and shown in the console to students.

Algorithm 1. Grading ROS packages pseudocode

1: exercise_params = getExerciseParams() 2: exercise_weights = getExerciseWeights() 3: exercise_parts = getExerciseParts() 4: exercise_grades = {} 5: token = getTokenIBotAcademy() 6: if !valid(token) then 7:   displayErrorMessage() 8: return 9: i = 0 10: while |exercise_parts| < N do 11:   grade = evaluate_part(exercise_parts[i], exercise_weights[i], exercise_params) 12:   append(exercise_grades, grade) 13:   i = i + 1 14:   exercise_parts.next() 15: sendGrades2IBotAcademy(exercise_params, exercise_grades, token) 16: displayGrades2Students(exercise_grades, exercise_parts)

4.2. iBotAcademy Academic Contents

A crucial component of iBotAcademy is the conceptual content. It is assumed that students have no previous knowledge about ROS, neither mobile nor manipulator robotics. These contents are guided by the learning path defined in Section 3.2 and developed using the problem-based learning methodology. These contents include theory and practice; the theory considers robotic concepts, and the practice integrates these concepts into ROS packages. All the learning activities proposed in the learning path have textual and graphical orientations of what is expected that the students should develop using ROS packages. Some expected results of those learning activities in mobile robotics are shown in Figure 6. Similarly, for the manipulator robotics branch, expected results are shown in Figure 2a. Then, to guide the student to achieve the expected results, the learning activities are instructed using text and graphical supports. Some of these expected results are shown in Figure 7.

Figure 6. Results of some learning activities related to (a) Teleoperation (related to LO1.1). (b) ROS service for circles (related to LO1.1). (c) Wall approaching (related to LO1.2). (d) Obstacle avoidance based on potential fields (related to LO1.2). (e) Robot localization (related to LO2.1, LO4.1, LO4.2); the red arrows show the particle swarm, and the green dots the current laser scan reading. (f) Path planning (related to LO2.2, LO4.1, LO4.2).

Figure 7. Results of some learning activities related to (a) Robot model on MoveIt (related to LO1.3). (b) Driving the robot to different positions in space (related to LO3.1). (c,d) Color and shape segmentation (related to LO3.2). (e) Manual grasping (related to LO3.1, LO3.2, LO4.1). (f) Automatic grasping (related to LO3.1, LO3.2, LO4.2).

4.3. Hardware Setup

The hardware used in iBotAcademy is basically composed of, first, the warehouse-like environment where the mobile robots are moving, and the area is limited by a boundary made of non-woven fabric, which can be reconfigured to simulate different obstacles, and a rack where objects can be stored, as shown in Figure 8a. Second, three Turtlebot3 mobile robots (named Gonzo, Fozzie, and Kermit), as depicted in Figure 8b, are equipped with a differential motion system, IMU, monocular camera, 360° 2D lidar, and an additional plate where objects are placed. Finally, two OpenManipulator-X (Nebula and Gamora) robots, as shown in Figure 8c, are equipped with a gripper, an RGB camera, and a robot controller.

Figure 8. (a) Learning environment. (b) Mobile robot (Turtlebot3). (c) Manipulator robots (Openmanipulator-X).

4.4. Robot Software Tools

Figure 9a shows the GUI implementation considering the functional requirements described in Section 3.3 for the mobile robot software tool. In this GUI, students will find: in 1, 2D lidar data is plotted; in 2, 2D lidar distances at specific angles are shown; in 3, the IMU sensor data is shown; in 4, odometry data is depicted; in 5, an emergency button to stop the robot motion is available; in 6, a keypad to move the robot is depicted; in 7, the camera data stream is shown; and in 8, the robots namespace is selected, in case of working with multiple robots.

Figure 9. (a) Mobile robots’ software tool. (b) Configuration module of the manipulator robot’s software tool.

Similarly, Figure 9b and Figure 10 show the three modules of the manipulator robot software tool (EduArm) implemented considering the functional requirements described in Section 3.3. In this GUI, the first step is selecting whether students are working with a real robot or a simulated one (square 1, Figure 9b). Square No. 2 shows different functionalities such as enabling or disabling the robot actuators, opening the gripper, going to the reset or initial position, and closing the gripper. Then, students can control the manipulator robot using each joint (square No. 3) or using Cartesian coordinates of the robot’s gripper (square No. 4).

Figure 10. (a) Perception module and (b) Calibration module of the manipulator robot’s software tool (EduArm).

Figure 6a shows the perception module of the manipulator software tool. Here, students can test different HSV ranges of values to segment objects by color (square No. 1). The current camera view is shown in square No. 3, and the resulting image is shown in square No. 3. In square No. 2, students can experiment with segmentation using object shapes. Students can configure the 3D locations of the rack in iBotAcademy; this is shown in Figure 10b. Square No. 1 shows the list of these locations, and Square No. 2 shows the report of the current locations configured.

5. Pilot Test, Results and Limitations

In this section, the pilot tests, results, and the limitations of this work are presented. To show that the iBotAcademy learning environment proposed in this work answers the research question and the specific goals proposed, two quantitative pilot tests are proposed. They include students performing two learning paths (mobile and manipulator robotics), following the step-by-step guided instructions, doing the quantitative assessments of the competence framework for each step-by-step guide, and using the software tools that support the learning process. Afterwards, students were asked to fill in a survey once students have finished the full learning path selected. Then, this section shows the statistical results obtained. In addition, limitations of this work are also described, considering the analysis of the results presented. Additionally, it is worth mentioning that in the context of the institutional framework of the Universidad del Valle, specifically in the qualified registration and high-quality registration of each curriculum approved by the Education Ministry (see curricula at https://ingenieria.univalle.edu.co/, accessed on 19 February 2026), the student surveys are considered an important tool to measure the student’s experience in each course or learning activity at Universidad del Valle.

5.1. Pilot Tests and Results

Two quantitative pilot tests were performed in iBotAcademy: students conducted the learning path corresponding to the mobile (pilot test 1) or manipulator (pilot test 2) robotics using simulated and real robots. The experimental conditions for these tests are described as follows: first, all tests were developed in the Mobile Robotics lab at the Universidad del Valle, Cali, Colombia; second, these tests were performed over the first two weeks of June 2025; third, a total of 20 students were surveyed; fourth, previous knowledge of basic programming in Python was required; fifth, it is assumed that students have not previous knowledge about ROS, neither mobile nor manipulator robotics; and, students can use their laptops or desktop PCs in the lab to carry out the learning activities.

Both pilot tests were performed considering the following activities: first, students select the learning path of their preference; second, over an intensive week of work, students developed the guided and proposed exercises with the support of the iBotAcademy software tools and hardware resources; finally, students were asked to answer 10 or 13 questions depending on whether students selected the mobile or manipulator robot learning path. It is worth noting that the questions were designed considering the LO that students must progressively meet as they advance in each learning path. In this way, the students’ responses show whether they achieved the LO. These questions and the Likert-like answers related to the mobile robotics learning path are as follows:

• Are the instructions detailed enough? Ans.: Very detailed (5), Detailed (4), Neutral (3), Little detailed (2), Not detailed (1).
• Did the executed commands produce errors? Ans.: Never (5), Seldom (4), Sometimes (3), Often (2), Always
• Did the simulation show the expected results? Ans.: Always (5), Almost always (4), Sometimes (3), Seldom (2), Never (1).
• When working with the actual robot, was its behavior as expected? Ans.: Totally expected (5), Mostly expected (4), Neutral (3), Little expected (2), Not expected (1).
• Was it easy to move the robot using the keyboard? Ans.: Very easy (5), Easy (4), Neutral (3), Difficult (2), Very difficult (1).
• Was it easy to move the robot using a program? Ans.: Very easy (5), Easy (4), Neutral (3), Difficult (2), Very difficult (1).
• Was it easy to capture sensor data and perform a task with the actual robot? Ans.: Very easy (5), Easy (4), Neutral (3), Difficult (2), Very difficult (1).
• Was it easy to create a map? Ans.: Very easy (5), Easy (4), Neutral (3), Difficult (2), Very difficult (1).
• Was it easy to locate the robot on the map? Ans.: Very easy (5), Easy (4), Neutral (3), Difficult (2), Very difficult (1).
• Was it easy to plan a route with the robot? Ans.: Very easy (5), Easy (4), Neutral (3), Difficult (2), Very difficult (1).

Figure 11 shows the results of this survey. Considering answers to questions 1 and 3, users found the instructions detailed and very detailed, which in turn gave positive results when users were asked about the expected behavior of simulations. Regarding question 2, users felt satisfied with the commands issued in the instructions. When users were asked about their experience with the real robot (question 4), they expressed that the behavior was mostly and totally expected, which is a consequence of testing the guided exercises in simulations first. Questions range from 5 to 10 are related to the user experience with distinct stages over the learning path; those stages are organized with a level of difficulty that increases progressively. Specifically, observing the results in Figure 11, at the beginning of the learning path (question 5), 10% of the students found that it was difficult to move the robot with a keyboard; afterwards, there was no difficulty in moving the robot using a program (question 6). This shows that students achieved the LOs. In addition, considering the results of questions 7, 8, and 9, when students deal with more complex sensors, they achieved the LOs when applying the knowledge to a specific task like map building. It is worth noting that in question 9, there is no difficulty at all. Finally, observing the results for question 10, despite the fact that subjects like path planning involve localization, map building, and dealing with sensors, there is no difficulty mentioned by users regarding this subject.

Figure 11. Survey results related to the mobile robotics learning path. Here, Q1–Q10 stands for the number of questions mentioned before.

At the end, Figure 11 shows that users’ answers felt satisfied with the learning path tasks. However, questions 5 and 7 showed some difficulty for students; these events coincide in the learning path with two crucial moments: the first time users interact with the mobile robot actuators, and the first time users interact with the main sensor of the robot. Afterwards, students get used to dealing with these devices. Comparing these results with those reported in [1,6,13,20], it can be said that survey questions and answers assess whether students could use iBotAcademy to learn concepts on mobile or manipulator robotics, and ROS.

In addition, the users gave their personal comments regarding the experience; they are as follows: first, “I was a little confused by the relocation module; I didn’t quite understand the internal system it uses.”; second, “The step-by-step guide helped me better understand ROS and how TurtleBot3 works in practice, but I found the relocation process very complex.”; third, “The GUI is useful for quickly viewing LiDAR data, but I prefer the console for sending movement commands via the keyboard; it feels better to interpret them at a low level.”; and fourth, “The mapping part felt very comfortable; I love imagining how it would look in a real-world application, although it took me a long time because the robot I used was very slow.”. Considering these comments, the learning process in iBotAcademy has positive feedback; however, some improvements must be made to guided documentation, specifically in the subject of localization.

It is important to assess the data consistency of the results observed in Figure 11. It is known that the coefficient of variation (CV) can provide clues about data consistency [32]; in this context, researchers commonly interpret a CV below 10% as very low variability, 10–20% as good, 20–30% as acceptable, and values above 30% as problematic dispersion or highly heterogeneous data. Another aspect to consider is the type of user answers; in this work it was used the Likert scale of 5 levels. This Likert scale and the analysis of survey data were used to define data consistency in the evaluation system for public health emergencies [33] and in the medical endoscope service evaluation system, based on the analysis of the Delphi method [34]. In [33], the authors assume CV thresholds less than 25% to make decisions about the data analysis.

Then, to assess data consistency of survey results in Figure 11, the CV threshold assumed is 20%. This value was selected considering the 5 levels of Likert responses, since it is desirable to have $\overline{x}>4$. That is, the average value of the data must be such that the score of 5 does not deviate by more than 20%, as shown in (6) and the statistical values in Table 3.

$$Cv={\displaystyle \frac{\sigma }{\overline{x}}}\ast 100<20\mathrm{\%}$$

(6)

where σ is the standard deviation and $\overline{x}$ represents the arithmetic mean. It is expected that the result’s mode corresponds to the maximum value (5). Observing Table 3, the CV is less than 20% in most of the results; the mean is bigger than 4, and the mode is bigger than 4 in all these results. In Table 3, users had a satisfactory experience with the mobile robotics learning path. There are high values in the CV in questions 5, 7, and 10, which all belong to the users’ experience in different subjects in the learning path. This leads to improving the user experience specifically in these guidelines. It is worth noting that in [1,6,8,9,13,18,20,23,25] do not report statistical analysis of data consistency, they limit the analysis to present histograms of the user tests, or the data average as statistical indicator.

Table 3. Statistical survey results for the mobile robotics learning path.

Question	Mode	FMO (%)	σ	$\overline{\mathit{x}}$	CV
1	5.00	60.0%	0.5	4.60	10.6%
2	5.00	60.0%	0.7	4.50	14.9%
3	5.00	60.0%	0.7	4.56	15.0%
4	4.00	60.0%	0.5	4.40	11.1%
5	4.00	40.0%	0.9	4.10	23.0%
6	4.00	40.0%	0.7	4.20	17.8%
7	4.00	40.0%	0.9	4.10	23.0%
8	4.00	50.0%	0.7	4.10	17.1%
9	4.00	60.0%	0.5	4.40	11.1%
10	5.00	40.0%	0.8	4.10	20.3%

Concerning the robotics manipulator learning path, the questions asked to students involved in the pilot test are also designed considering the LOs that students must progressively meet as they advance in each learning path. These questions and the Likert-like answers are as follows (the answers are equal to all: Totally agree (5), Agree (4), Neutral (3), Disagree (2), Totally disagree (1)):

• The OpenManipulator-X robot was easy to use during field testing.
• The Edu-Arm interface and guided documentation allowed me to use the OpenManipulator-X robot within ROS.
• I feel I improved my Python programming skills applied to robotics.
• The work environment was suitable for performing storage, transport, loading, and unloading activities with the manipulator robot.
• The information provided in the guides is sufficient to understand the operation of the OpenManipulator-X robot.
• The guide documents were clear and easy to understand for completing the activities.
• I was able to complete the proposed exercises, and the verification packages helped me check my work.
• The verification packages helped me identify errors in my code.
• The messages from the verification packages were clear and easy to interpret.
• The Edu-Arm user interface is intuitive and easy to use.
• I believe that the Edu-Arm interface, guided documentation, and the use of the robotic manipulator were sufficient for conducting the goal of transport activities.
• I believe that the Edu-Arm interface, the guided documentation, and the use of the robotic manipulator were sufficient for conducting the goal of recognition activities by color and shape.
• I believe that the Edu-Arm interface, the guided documentation, and the use of the robotic manipulator were sufficient for conducting the goal of loading and unloading activities based on their color and shape.

Figure 12 shows the results of this survey. Questions 1 and 10 are related to the hardware and software tools used by users over the learning path. As a result, users’ experience was highly satisfactory. Answers to questions 2, 5, 6, 11, 12, and 13 check the user experience with the concepts and instructions in the documentation; they show that 80% of users totally agree and 20% agree that the documentation presented was sufficient to perform all learning activities. Answers to questions 3 and 7 allow us to observe that users felt their programming capabilities were improved, and to achieve this, the grading packages developed played a significant role. Also, answers to questions 4, 8, and 9 show that users felt the learning environment and the grading packages were helpful. Responses to questions 11, 12, and 13 show that students achieved the LOs regarding object recognition, transport, load, and unload activities described in the competence framework proposal of Section 3.1. In addition, it is worth noting that [1,6,8,9,13,16,18,19,20,21,22,23,25], authors assume ROS concepts are already known by students, and they are focused on developing ROS applications, and they are unaware of the complexity of the ROS learning curve. In contrast, the iBotAcademy learning environment has a learning path implemented in step-by-step guided documentation that introduces mobile and manipulator robotics and ROS concepts progressively.

Figure 12. Survey results related to the manipulator robotics learning path. Here, Q1–Q13 stands for the number of questions mentioned before.

Similarly to the mobile robotics learning path, all these answers were validated using the CV shown in (6) and the statistical values in Table 4. Table 4 shows that all CV values are less than 20%, the mean is greater than 4.4, and the mode is 5 in all results. Students had a highly positive experience with the robotics manipulator learning path. The biggest CV corresponds to questions 3 and 6, which are related to the Python programming skills and the clarity of the instructions described in the documentation. The user experience of these two aspects can be influenced by other factors which sadly could not be measured in this study.

Table 4. Statistical survey results for the manipulator robotics learning path.

Question	Mode	FMO (%)	σ	$\overline{\mathit{x}}$	CV
1	5.00	80.0%	0.4	4.80	8.3%
2	5.00	80.0%	0.4	4.80	8.3%
3	5.00	50.0%	0.7	4.40	15.1%
4	5.00	80.0%	0.3	4.89	6.4%
5	5.00	80.0%	0.4	4.80	8.3%
6	5.00	70.0%	0.5	4.70	9.8%
7	5.00	80.0%	0.4	4.80	8.3%
8	5.00	80.0%	0.4	4.80	8.3%
9	5.00	90.0%	0.3	4.90	6.1%
10	5.00	90.0%	0.3	4.90	6.1%
11	5.00	80.0%	0.4	4.80	8.3%
12	5.00	80.0%	0.4	4.80	8.3%
13	5.0	80.0%	0.3	4.90	6.1%

Since, at the end of each learning activity, the grading packages evaluate the student work, all these data are stored and related to the competence framework described in Section 3.1. Then, using this data, Figure 13 shows an example of the iBotAcademy software tool output, which can be accessed for each student. Each course in the iBotAcademy environment can be created to satisfy a specific number of competences. The iBotAcademy software considers the curriculum reform performed by the Engineering Faculty of Universidad del Valle, and the monitoring indicators of competencies in Engineering programs; one of those indicators is comparing the student grades with respect to the average grades in a given course [35]. Then, the left part of Figure 13a shows how the student’s quantitative evolution can be compared with an average student considering all the student grades so far. Also, the right part of Figure 13a shows how the student grades evolve over time. In the case shown in the right part of Figure 13a, it is shown how the student learning curve improves over time.

Figure 13. (a) Grades evolution over the learning process. (b) Preliminary report of student’s grades evolution. (c) Competence hierarchy and grading.

Figure 13b shows the preliminary student’s grades evolution up learning activity No. 3. In Figure 13b also shows that students can improve their grades at each learning activity. The iBotAcademy software tool can also show a heat map of the students’ grades for each learning activity, which is very useful to observe the students’ group behavior, identify the potentially problematic learning activities for each student or group of students, or the positive evolution of the student’s learning process.

Students can also access a separate way of displaying grades and their relationship with competences, LO and LAO. Figure 13c shows a sunburst diagram which allows displaying the student gradings hierarchically. Starting from the center, which is the name of the course, it is surrounded by the number of competences; then, each competence is surrounded by all the LO registered; finally, each LO is surrounded by all the LAO defined. If the student puts the mouse over each of these parts, the corresponding quantitative grade is shown.

In [13,20], the authors report nice software tools to track the industrial process over different manufacturing cells interconnected with mobile and manipulator robots. However, they do not report a software tool to show the student’s learning process over those manufacturing cells. The iBotAcademy learning environment follows a different approach, special developed software tools to support the student’s learning process and measure this process quantitatively, considering a competence framework.

Table 5 shows the overall grades obtained in the pilot test considering all the learning activities done; as mentioned at the beginning of Section 5, all these learning activities were performed over a period of 2 weeks. Each cell value in Table 5 has a color format as follows: red for grades between 0 and 1.9; orange for grades between 2.0 and 2.9; yellow for grades between 3.0 and 3.9; and green for grades between 4.0 and 5.0. In addition, the last four rows include a count of the number of grades between these ranges. Observing these counts, it can be observed that 93% of the students’ grades are higher than 3.0, which means in the University of Valle that the learning activity was approved. It is worth noting that by observing the first part of Table 5, specifically the average row (Av.), there is a performance difference in learning activity 2; other learning activities show a balanced performance. Finally, it is important to note that all students involved finished the total learning activities.

Table 5. Student grades and learning activities performed.

St.	Act.1	Act.2	Act.3	Act.4	Act.5	Act.6	Act.7	Act.8	Act.9	Act.10	Act.11	Act.12
1	3.5	4.9	4.7	3.7	3.7	5.0	4.8	5.0	3.6	4.6	4.9	4.9
2	4.3	3.0	4.5	3.9	4.7	4.9	4.2	3.1	5.0	4.3	5.0	4.8
3	3.3	5.0	4.6	4.0	4.7	4.9	3.2	3.3	3.3	4.1	4.7	3.2
4	4.4	3.4	4.9	4.0	3.9	3.4	4.0	3.7	4.0	3.9	3.5	4.2
5	2.3	3.8	3.9	4.2	4.8	3.3	3.7	3.2	4.8	4.2	3.4	4.0
6	4.7	4.7	4.7	3.7	3.0	4.8	3.9	4.0	3.2	4.6	4.6	3.6
7	4.9	1.2	3.5	4.5	3.5	4.2	3.7	4.1	3.0	4.7	3.1	3.3
8	4.9	3.6	4.2	3.7	4.9	3.1	3.2	4.7	3.2	4.4	4.8	4.8
9	4.9	4.8	4.6	4.3	4.5	4.3	3.5	4.3	3.1	4.8	3.8	4.4
10	3.8	3.5	1.4	4.6	4.4	4.4	4.2	4.8	4.1	4.5	3.4	4.8
11	4.8	4.2	4.5	4.8	3.7	3.2	4.8	4.9	4.9	4.6	3.1	3.4
12	4.5	3.9	2.1	3.8	4.4	4.9	4.8	4.5	3.4	3.7	4.7	3.6
13	3.9	4.3	3.0	3.1	5.0	5.0	4.1	3.1	4.7	4.3	3.6	3.5
14	4.5	4.9	4.6	4.7	5.0	4.5	4.4	3.0	4.1	3.6	5.0	4.3
15	4.4	3.0	3.9	3.4	4.3	3.1	3.4	4.3	3.1	4.4	3.8	3.5
16	4.0	4.3	4.8	3.1	3.1	5.0	4.6	4.5	4.9	4.1	3.6	3.5
17	3.1	4.1	4.2	4.6	3.3	4.0	3.5	4.2	4.4	3.3	3.2	4.1
18	3.8	3.2	3.5	3.6	4.5	4.7	4.3	3.6	4.2	3.2	4.1	4.7
19	4.4	3.9	3.7	4.1	3.6	4.4	3.7	4.3	3.1	3.1	3.9	3.2
20	5.0	4.6	4.7	3.9	4.7	4.7	4.8	3.0	4.2	3.9	4.6	4.9
Av.	4.2	3.8	4.0	4.1	4.0	4.0	4.0	4.1	4.1	4.3	4.0	4.0
Class	Act.1	Act.2	Act.3	Act.4	Act.5	Act.6	Act.7	Act.8	Act.9	Act.10	Act.11	Act.12
Red	0.0	1.0	1.0	0.0	0.0	0.0	0.0	0.0	0.0	0.0	0.0	0.0
Orang.	1.0	0.0	0.0	0.0	0.0	0.0	0.0	0.0	0.0	0.0	0.0	0.0
Yell.	11.0	12.0	5.0	10.0	12.0	8.0	8.0	6.0	9.0	6.0	11.0	10.0
Green	8.0	7.0	14.0	10.0	8.0	12.0	12.0	14.0	11.0	14.0	9.0	10.0

5.2. Final Remarks and Limitations

Comparing the results achieved in this work with those reported in [1,6,13,20] regarding the quantitative measurement of the student’s learning process using a competence framework, those works present in detail the learning platform or environment, and they have software tools to support the learning process. However, those software tools are designed to monitor the industrial process evolution rather than the student’s learning process; in addition, the statistical results reported are focused on satisfaction measurements instead of checking the student’s learning process.

Another learning methodology compatible with this work is problem-based learning; in this context, works such as [8,9,18,23] are focused on describing the learning platform or environment in great detail; they define the learning outcomes, some emphasize mobile robots and other in robot manipulators, or both. However, refs. [8,23,25] do not present results with users; in [9], the authors present results assessing student learning, but there is no quantitative measure of the student’s learning process; and in [18], there is no defined learning path, and the user results are focused on satisfaction surveys. In addition, all these works assume students have previous knowledge of ROS.

Other works that use problem-based learning, but do not use ROS, are [16,19,21,22]. They use the specific SDK to program the robots, which has the immediate disadvantage of not having shareable results. All these works emphasize having a detailed description of the robotics platform, but in [19,21,22] there are no user tests reported. A special case is shown in [19], where the authors propose a quantitative measure of the student’s learning process, but it is not tracked over time.

Despite the fact the promising results presented in Section 5.1, the iBotAcademy learning environment shows the following limitations: first, the size of the sample group of students that performed the learning paths, which is important to address as soon as possible to validate the impact of the learning environment. Second, it is worth noting that all the software developments were designed to work remotely; however, due to Internet access restrictions at the Universidad del Valle, this functionality could not be evaluated. Third, it is important to increase the number of mobile robots carrying objects, manipulator robots to implement different tasks (palletizing, selecting, and classifying objects), and a wider warehouse-like experimental environment to include more interactions between robots and humans. Fourth, review the step-by-step guides that students follow to implement the learning activities to clarify specifically the mobile robot localization subject, and increase the description of the command-line-based ROS interface. Finally, research projects take time to propose and develop; for this reason, this work was developed initially using ROS Noetic; then, an upgrade is underway to migrate to ROS2.

6. Conclusions

This work presented the iBotAcademy learning environment, which satisfies the research question and specific goals proposed as follows: students can be trained in the iBotAcademy learning environment developed using a competence framework and where quantitatively measures the student’s learning process on mobile and manipulator robotics; students can follow a learning path that teaches mobile and manipulator robotics, and ROS concepts progressively up implementing Industry 4.0 applications such as pickup/delivery tasks; and students can use software tools to support the learning process.

The iBotAcademy learning environment includes the following parts: first, student software tools to interact with mobile and manipulator robots, and presenting the learning path offered in this work; second, a learning path to understand the mobile and manipulator robotics concepts considering the problem-based methodology; third, an experimentation environment that includes a warehouse like setting, real mobile and manipulator robots, where pickup and delivery tasks can be developed.

The iBotAcademy learning environment considered a competency-based framework to measure the students’ learning process. A set of competences, learning outcomes, and learning activity outcomes were proposed considering the learning path offered for mobile and manipulator robotics. The learning path was proposed based on this set of competences, and at the end, students develop a pickup/delivery task.

Two software tools were developed to interact with the iBotAcademy robotic platforms, one to monitor the Turtlebot3 mobile robot and another to monitor the OpenManipulator-X robot. Students used these software tools to check the proper functionality of the mobile and manipulator robots when they are following the learning path documentation.

To monitor the evolution of the students’ learning process, this work proposed a set of ROS packages to evaluate the learning process quantitatively. The input data for this evaluation is provided by the grading ROS packages; they assign grades to the proposed exercises at each lesson in the learning path.

To test iBotAcademy, two quantitative pilot tests were performed, considering both proposed learning paths, one for mobile robotics and another for manipulator robotics. These pilot tests included a total of 20 students of the Universidad del Valle over two weeks in Jun 2025. These students performed the learning path offered; they developed the guided and proposed exercises, executed the grading ROS packages, and at the end answered two surveys to measure their experience. As a result, regarding the mobile robotics learning path, an average of 82% positive responses were received, and considering the statistical analysis of Table 3, the users responses achieve desirable values such as $\overline{x}>4$ (minimum of 4.1), and most of them with a CV less than 20% (in average 16.4%). Regarding the manipulator robotics learning path, an average of 99% of positive responses were received, and users’ responses achieved the desirable values such as $\overline{x}>4$ (minimum of 4.4), and a CV less than 20% (in average 8.3%), see Table 4. Observing the results of the pilot tests, students felt they achieved the LO proposed in the competence framework presented in Section 3.1. In addition, it can be observed from Figure 11 and Figure 12 a positive answer from students; this is a consequence of the big effort put into the guided documentation, which explains in detail the steps to perform and the concepts to learn. Considering the results presented in Table 5, it can be observed that 93% of the students approved the learning activities; therefore, students endorsed the learning paths. Also, these results show that the integration of the testbed environment, the software tools, the competence-based framework, and the quantitative measure of the learning process proposed in this work received a positive response.

Future work proposed is: first, develop a remote programming environment to access the student lessons, an IDE, remote consoles, and robotic simulations in Gazebo; and second, remotely connect the mobile and manipulator real robots to develop the students’ solutions with video feedback. This implies having hardware resources to guarantee scalability or transferability, which enables us to have broader impact and test the approach proposed in this paper in larger cohorts. The latter includes pre and post-measures to assess the impact of iBotAcademy. And, reviewing the step-by-step guide documentation that students follow to include a better description of the localization in mobile robots, and how students can leverage the command-line-based ROS interface.