Task Planner for Robotic Disassembly of Electric Vehicle Battery Pack

The rapidly growing deployment of Electric Vehicles (EV) put strong demands on the development of Lithium-Ion Batteries (LIBs) but also into its dismantling process, a necessary step for circular economy. The aim of this study is therefore to develop an autonomous task planner for the dismantling of EV Lithium-Ion Battery pack to a module level through the design and implementation of a computer vision system. This research contributes to moving closer towards fully automated EV battery robotic dismantling, an inevitable step for a sustainable world transition to an electric economy. For the proposed task planner the main functions consist in identifying LIB components and their locations, in creating a feasible dismantling plan, and lastly in moving the robot to the detected dismantling positions. Results show that the proposed method has measurement errors lower than 5 mm. In addition, the system is able to perform all the steps in the order and with a total average time of 34 s. The computer vision, robotics and battery disassembly have been successfully unified, resulting in a designed and tested task planner well suited for product with large variations and uncertainties.


Introduction
As the adoption rate for electric vehicles (EV) is now accelerating worldwide, EV Lithium-Ion Batteries (EVBs) repurposing or recycling volumes are expected to be larger in 5-10 years (120 GWh/year available by 2030 [1]) and legislation will likely demand higher collection and recycling rates as for example in the newly proposed regulation of the European parliament and of the council concerning batteries and waste in December 2020. Today, the automotive Lithium-Ion Batteries (LIBs) dismantling process is mainly carried out manually and the use of robotics in this process is limited to simple tasks or human assistance [2]. These manual processes are time consuming and must be done by highly skilled personnel. As a direct consequence, the manual total disassembly of Li-ion EVBs might not be profitable and would be stopped at an optimal level, i.e., partial disassembly, that achieves maximum profit while decreasing the environmental impact [3]. In comparison, automated systems are more robust, have a lower-cost, reduce injuries and/or sickness, make the workplace more attractive for those hard-to-recruit-and-retain skilled workers, and are best suited for up-scaling to high-volumes. Therefore, fully automated disassembly of EVBs is inevitable. The main challenges for the success of the automated systems in dismantling are the variations and uncertainties in used products [4]. These challenges in the robotic disassembly of Electrical and Electronic components in electrical vehicles have been presented in article [5] where the need for cognitive systems is identified to enhance the effectiveness of automated disassembly operations. In the case of automated disassembly of EV batteries, advances in Computer Vision (CV) and cognitive robotics offer promising tools but this topic remains an open research challenge [6]. The

Task Planner Design
The experimental setup is composed of IRB4400 robot (ABB, Zürich, Switzerland), IRBT4004 track (ABB, Zürich, Switzerland), and Zivid One 3D camera (Zivid, Oslo, Norway) mounted on the robot arm, all connected to a PC with Ubuntu and running the Robot Operating System (ROS), and a A3 Sportback e-tron hybrid Li-ion battery pack (Audi, Ingolstadt, Germany). More information about the connection setup and use of the ROS as a middleware can be found in article [19], whereas a complete description of the Audi battery pack and its disassembly sequences has been presented in article [3]. The task planner proposed in this paper and shown in Figure 1 analyses the information provided by the vision system based on Zivid camera, makes decisions regarding dismantling actions and sends a predefined path to the industrial robot's controller.   In order to set a sequence of the removal operations, only one image is used, which 105 is taken with lower camera angle with respect to a horizontal plane. The images with a  The main loop of the task planner is organised as the following: takes 2D and 3D images, detects and identifies components, finds the component's positions in the world reference frame, defines an order of operations, removes the components, and repeats this actions until the goal state is reached (refer to steps A-E respectively in Figure 2 Only Look Once) algorithm [21] which also provides the bounding box positions of all 99 constituting components. The output of the object detection procedure as shown in Fig.   100 3 is a file containing all detected objects, labels and corresponding coordinates. The  In order to set a sequence of the removal operations, only one image is used, which 105 is taken with lower camera angle with respect to a horizontal plane. The images with a The task planner moves the robot into several predefined poses to ensure that the system is able to observe different parts of the LIB pack. In order to reach better accuracy, up to eight pictures are required, especially when screws placed in the lateral sides of the battery are present.

Object Detection (2D Image) (B)
Different components in the image such as screws, battery modules, connecting plates, and Battery Management System (BMS) are detected using the YOLO (You Only Look Once) algorithm [20] which also provides the bounding box positions of all constituting components. The output of the object detection procedure as shown in Figure 3 is a file containing all detected objects, labels and corresponding coordinates.
The positions of the detected objects in different images are merged using a weighted mean of each positions. Object detection (YOLOv3) .txt File

Decision Making (C)
In order to set a sequence of the removal operations, only one image is used, which is taken with lower camera angle with respect to a horizontal plane. The images with a higher inclination are used for object detection only. A list of the component positions in the removal order is created by adding the detected screw positions and following by a computer vision analysis of each specific component. Based on the probability of being over the other components the remaining parts are added in the list in a correct disassembly order.

Position Calculation (D)
At this point the system defines the positions of the objects in 2D camera coordinates (pixels). In order to move the robot to those positions, they must be converted into 3D points. The 2D object coordinates in camera frame are transformed into 3D coordinates using the depth information of the 3D vision system. Then, the captured positions are known, so that the world reference position of the objects can be obtained. This action is done for all images. Once all the positions from the different points of view are found the nearby points (representing the same object or component) are merged.

Robot Communication and Removal Operation (E)
Once the order of the operations and all the positions are known, the last step is to be proceed in dismantling. The task planner calls the removal operation of every single component. Details on the design of the removal operation are not presented in this paper, since a scope is limited to moving the robot arm onto the calculated positions, where the selected removal operation based on the classification of the component is achieved.

Main Functions
The main functions and scripts used by the task planner connect the computed information by the mean of tests and loops as shown in the complete flow chart in Figure 4.  Fist of all, the main function main() whose flow chart is shown in Figure 4, declares and initialises all the variables used and transferred in and between the subsequent functions, as for example the screws, connecting plate, Battery Management System (BMS), or module positions and number. Then, it reads the file containing the classes names (classes.names), and saves them into a dictionary (dict_comp). The creation of this dictionary aims to allow the system to have access to the nomenclature in order to relate the detected classes with their name and characteristics necessary for further analysis.
Once the dictionary is created, the function starts its principal loop. Note that this loop keeps running until the dismantling is completed. The first action of the loop is to move the robot to predefined positions and take 3D images, i.e., XYZ + color (RGB) + quality (Q) for each pixel, of the battery pack. Next, once the algorithm has been trained, it runs the object detection to detect the components placed in these images, using the YOLOv3 algorithm.
After the detection, classification and pose estimation of the objects in the image frame are done. The function analyses each taken image, finds different components and their characteristics from Dict_comp, and converts the positions of the objects from the image base frame to the camera reference frame. Following this step, a sub-process establishes priorities to enable the decision-making operation on what component should be removed next. The respective functions are named what_component and has_comp_over and are further described in the next section. Thus, when the detected positions have been converted into the camera frame, they are further transformed into the world reference frame coordinates. At this stage, since several 3D pictures of the same components have been captured, the resulting multiple positions of the same components are merged, which increases the position accuracy. The output of this function is the positions of the different components in world reference frame. The components are then placed in order of removal preference and finally, the task planner's main function runs the removal operations for all the detected component before starting the main loop again if the goal state is not reached.

Cognitive Functions: what_component() and has_comp_over()
The function what_component() flow chart is shown in Figure 5 and the function is responsible for the decision making. The function analyses the detected objects, and decides the order of the removal operations using the computer vision based sub-function has_comp_over() . The sub-function has_comp_over() flow chart is shown in Figure 6, and the function establishes different probabilities of a specific component to have a component over.
The inputs for the function what_component() are: •    First of all, the function creates the variable compon_rem (array). The aim of this array is to contain the list of the components positions in the removal order. Given the nature of the disassembly process, the first components to be removed are screws, thus, their positions are the first to be added to the compon_rem array.
To add the rest of the components and decide the order of the operations, the function what_component runs over the list co_tot analysing each component using the function has_comp_over.
Essentially, has_comp_over realize a computer vision analysis of each specific component, and returns the probability of being over the other components. Thus, the components are ordered from more probability to be over to less probability.
The inputs for the sub-function has_comp_over are: • co_tot (array) • co_an (array): Array containing the image coordinates of a specific component (the component being analysed).
The function has_comp_over runs a nested for loop over the list containing all the components, analyzing the ones that are intersecting the component that is being inspected in the outer for loop.
To realise this comparison, the full image is converted into grey-scale and equalised. The output image contains a better distribution of the intensities maintaining the relevant image information [21].
After obtaining the equalised image, the function crops the image into the area of interest, in this case, the area of the analysed component. Then, segmentation is applied to the window, binarising it using an Otsu threshold. In the Otsu method, the threshold is determined by minimizing intra-class intensity variance [22].
When the window has been binarized, the mean of the pixel intensities in the intersection area is compared to the means of the intersecting components areas excluding the intersection. The component for which these two means (intersection alone and component area excluding intersection) are the most similar is the most probable to be on the top layer, i.e., over all the others, and hence to be removed first. An example with two overlapping components is shown in Figure 7. In this example, component 2 is over component 1 because: where f(A) is a function calculating the mean of pixel intensity in area A.
, in this case, the area of the analysed component. Then, segment indow, binarising it using an Otsu threshold. In the Otsu method mined by minimizing intra-class intensity variance [23]. hen the window has been binarized, the mean of the pixel intensi area is compared to the means of the intersecting components a rsection. The component for which these two means (intersec ent area excluding intersection) are the most similar is the most op layer, i.e. over all the others, and hence to be removed first. A erlapping components is shown in Fig.7. In this example, comp ent 1 because:

(A) is a function calculating the mean of pixel intensity in area A
. Component 2 with total area A 2 is overlapping component 1 with to tion area is I ter repeating this procedure for all the components, the function r alue, max(p_list), of the calculated probabilities of having anoth ping the analyzed one.  After repeating this procedure for all the components, the function returns the maximum value, max(p_list), of the calculated probabilities of having another component overlapping the analyzed one.

Merging the Pictures: merge_detection()
The function merge_detection aims to merge positions of the components (for this version only the screws) commonly detected in one or more images. The function flow chart is shown in Figure 8. In the next step the function merges the repeated positions to find the output list (rem_component_WR_filt). In order to filter the points, the function runs over the rem_component_WR array adding not-repeated components to the filtered list. The function considers two components as one, when the x and y distances are lower than 1cm, and defines the final position as the mean of both points.

Results
The aim of this section is to validate the order suggested by the task planner and to characterise its performance regarding time and accuracy. The Audi A3 Sportback e-tron Hybrid Li-ion Battery Pack serves as the case study. The description of the EVB pack and its components as well as the disassembly process of the battery are detailed in article [3] whereas Table 1 presents the composition, i.e., relative weight of each components and materials. For safety reason when testing the concept, all modules have been manually discharged separately. However, when integrated in the pilot plant, the EVB packs will be discharged prior complete disassembly at a certain state of charge depending if the modules are to be repaired, re-purposed, re-manufactured, or recycled. In addition, damaged EVBs that represent high risk for thermal runaway or gas emission will be sorted out. These last two steps are outside the scope of the present work.

Time Analysis
The timings of the operations realized by the task planner, i.e., image capture, object detection, decision making, and motion to estimated pose, have been recorded on 20 repetitions with the physical setup, resulting in the mean times summarized in Figure 10. During experiments, the speed of the industrial robot has been reduced to 25% speed for safety reasons. The expected timing at production speed (100% speed) are shown in black and red in Figure 10. repetitions with the physical setup, resulting in the mean times summarized in Fig.10.

246
During experiments, the speed of the industrial robot has been reduced to 25% speed 247 for safety reasons. The expected timing at production speed (100% speed) are shown in 248 black and red in Fig.10.    In this case, the image capture process refers to the robot movement into the image taking positions and image capturing. In order to implement the capturing, the process has been divided into seven different actions. The first action refers to the robot model loading, and the rest refer to the robot movements and image captures, see Figure 11.   In this stage the YOLO algorithm is applied to three taken images to detect and identify different components on 2D images. Mean time: 4.8 s. In this study 3 images are analysed but up to 8 images are required to increase an accuracy. The robot approaches the components to perform the removal operation. First, the robot rapidly moves to a safety position displaced thirty centimetres in the Z-axis above the object and then moves to the desired location. After that, the robot moves back to the safety position. The timings for this operation have been divided into six sub-processes.

Decision-Making: Optimal Path
The decision-making of the system (the order of the removal operations) affects directly onto the dismantling time. For this reason, the aim of this section is to analyse the order suggested by the task planner for the Audi A3 Sportback e-tron Hybrid Battery Pack.

Optimal Dismantling Plans
After manually dismantling and analysing the battery pack, the optimal dismantling plan is to first remove the screws; the second step is to remove the two connective components and the battery management system (the order of the removal operations for these three elements is not critical); and finally to remove the four battery modules in a arbitrary order.

Dismantling Plans Proposed by the System
A set of tests have been carried out under different conditions (i.e., different orientations, different ambient lights conditions, etc.), the system has given a good response.
It has been observed, within the proposed dismantling plans, that the system follows the guidelines defined in Section 3.3.1. Because the BMS and the two connective components (left and right) are not overlapping, the system is proposing two different plans that are equivalent. These are referred as the (A) and (B) plans and are illustrated in Figure 12.
In the (A) plan, the system begins removing screws. The screws are always the first components to be removed. Afterwards, the system removes one connective component (in some tests the left one in other tests the right one), the BMS, and the other connective component in that order. Finally, it removes the battery modules. See Figure 12. In the (B) plan, screws are still the first components to be removed. Then, the system proposes to remove the BMS and the connective components in that order. As in the previous plan, the battery modules are removed the last. Thus, the main difference observed between these two plans is that in the plan (B) the BMS is removed after the screws instead of connective plates. This has no impact on the final disassembly process.
plans that are equivalent. These are referred as the (A) and (B) plans and are illustrated 298 in Fig.12.

299
In the (A) plan, the system begins removing screws. The screws are always the first 300 components to be removed. Afterwards, the system removes one connective component It has been considered that the 3D printed tool is good enough for testing purposes

Accuracy
To analyse the system's accuracy, a 3D printed pointer has been used as Tool Center Point (TCP). The tool consists of a thin 25 cm long bar with a sharp end. With the tool mounted, the task planner has been run in debug mode. For safety reasons, the robot TCP has been moved 3 cm above in the Z-axis (word base frame) in order to avoid collisions with the battery pack in case of failure. Some of the tests are shown in Figure 13. In the majority of the cases, the system has an accuracy of (<5 mm). The accuracy has been measured by the mean of a laser beam attached to the red bar and pointing towards the target position, whereas the distance of the laser pointer to the target position is measured with a caliper.

Discussion
In this paper the proposed objectives have been achieved. Different research areas in computer vision, robotics, circular economy and electr(on)ic components (battery) disassembly have been successfully unified.
The assigned main hardware elements such as an industrial robot, 3D camera and PC have been interconnected to carry out the principal system tasks, such as object detection, pose estimation, decision-making, and robot displacement. Therefore, the system is able to recognise the dismantling object main components, to find their position, and to move the robot to the defined positions in a specific order. Lab tests have been used to validate the designed task planner. In this case, experiments were limited to Audi LIB pack, however, a similar procedure might be applied to any EV battery pack. Compared to other disassembly processes as reviewed by Zhou et al. [11], the proposed task planner relies on state-of-theart 3D camera system with high accuracy and does not require Computer Aided Design (CAD) models of the battery pack and its components. This presents a great advantage since EoL products are often different than their original CAD models, due to possible maintenance, deformations, or corrosion. Recognising the model and date of production of the EVB to be disassembled will help to determine a first disassembly sequence based on a self-updating database, but the system must also be flexible and robust enough to handle the above-mentioned variations or in the case of new or unrecognised model. Therefore, combined with reinforcement learning and machine reasoning algorithms the proposed disassembly framework will be able in future developments to learn how to disassemble new battery pack models, if not by itself, with only limited information from the human operator. The concept of cognitive robotics in disassembly has been developed and validated on End-of-Life treatment of LCD screen monitors [17]. However, some challenges remained as (1) the too high processing time making the process economically infeasible, (2) the remaining need for human assistance, and (3) the too high inaccuracy of the vision system leading to low success rate. This paper demonstrates that the recent advances in 3D vision system, fast object detection and localisation algorithms, as well as task planner design place EVBs with inherent uncertainties and large variations in design as a good candidate for achieving eventually autonomous and complete disassembly through the cognitive robotic concept.
The proposed task planner for disassembly of EVB pack into modules can also be extended in future work to a deeper level of disassembly, i.e., to battery cell level or even to the cell components (cell casing, electrodes, electrolyte, separator) which will increase the concentration of active materials in the subsequent steps for battery recycling and hence reduce the complexity and energy consumption of the pyrometallurgical and hydrometallurgical processes. Removing manual operations in the pre-processing stages will move the optimum disassembly level determined in the article [3] deeper toward complete disassembly when still considering techno-economic and environmental constraints.
The algorithm You Only Look Once (YOLO) is implemented to detect and find the components placed in the dismantling arena. The results show that the algorithm performs well, giving expected results and detects main components. For example the presence of screws can be distinguished from the presence of screw holes where the screw has been removed. The developed vision system can hence also be used to validate removal operations. The information extracted from the object detection was used in a pose estimation to find coordinates of components, where 2D images and the YOLO results have been matched with the 3D data sets. In future versions of the task planner, object detection and pose estimation might be realised directly in 3 dimensions based on the point cloud data with techniques such as complex-Yolo [23] or DeepGCNs [24]. However, higher processing time or computing resources are to be expected.
When validating the task planner and measuring the timings, the robot model has been loaded every time that the robot had to move. Thus, in future stages the robot model should be loaded just once, at the beginning making the disassembly sequence at least 9.6 s faster. Moreover, the ROS main has been run in manual mode at 25% of the maximum speed of the robot. In automatic mode, i.e., at 100% speed, the total time of the disassembly sequence, excluding the removal operations time, is expected to decrease from 53.6 s to 34 s.
Using an eye-in-hand configuration performs well, and has some advantages (i.e., only one camera is needed), but it presents some drawbacks too. In an industrial application, the continuous moves and removal operations could have negative consequences like unexpected collisions of the camera with the environment or causing miscalibrations.
The efforts in future stages of the research should be focused on instrumentation and tool design for the dismantling system. It is also essential to detect flexible bodies such as high-voltage wires, and wires transmitting data. Thus, the direction of the research on the object detection and pose estimation part should concern how to find a feasible solution for such the objects and remove them.

Conclusions
A new task planner has been designed for the disassembly of electric vehicle Li-ion battery packs, with as main objective to increase the flexibility and robustness of the system. Lab tests have been used to validate the designed task planner based on a Audi A3 Sportback e-tron hybrid Li-ion battery pack. The results obtained in the tests demonstrate that the obtained solution is able to recognise which component to remove first and the complete disassembly plan without a priori knowledge of the disassembly strategy and battery CAD models. This method is therefore well suited for product with large variations and hence increases the disassemblability. The achieved performances measured in term of accuracy, time to generate the disassembly plan and success rate validated the task planner concept and its ability to make autonomous decision. Further testing on a larger set of EV battery packs with other geometries and connections and addition of learning capabilities will be needed to further increase the robustness of the proposed method and the technological readiness level. However, the results already cast a new light on the use of automation in the EV LIB batteries disassembly process by bringing the technology one step closer to eventually fully automated operations and hence redefine the optimum level of disassembly for the batteries to enter the subsequent stages of recycling and metal recovery.
The experience in this field could also be adapted to be used for other dismantling processes and opens new doors and research challenges to other fields directly related to robotics.