Embedded Files
In this project we employ arm manipulators such as Franka Research 3 and Neuromeka IndyRP2_v2 equipped with different end effectors such as 2-finger grippers and electric screwdrivers to autonomously disassemble end of life e-mobility vehicles. Research entails imitation learning, visuo-servoing and impedence control.
AUTONOMOUS UNSCREWING SYSTEM
I developed an autonomous unscrewing system with Franka Emika FR3 using ROS. The unscrewing process consists of steps such as:
screw detection and localization - A YOLO model was trained with images of screw heads which was then used for detecting screws from images obtained from robot's mount camera feed. Then for localization of the screw, we used the associated point cloud data.
screwdriver alignment - The robot moves right above the screw to a distance of 3 mm in between the screw head and the screwdriver tip. The accuracy of screwdriver alignment is very important because this is the first step to a successful contact. In our case, the screw localization from the point cloud is accurate enough (~2mm) to proceed.
screwdriver latch - The screwdriver is lowered 1 mm at a time until a high force in the z-direction is observed from the FT sensor in the robot. A high force would indicate that there is a contact and furthermore indicate a successful latch.
unscrewing - An electric screwdriver is attached to the EE of the robot. This electric screwdriver is actuated electronically using a custom actuator design. When the screwdriver is turned on, the screw starts unscrewing and moves upwards applying a force on the robot. Once a certain force threshold is reached, we move the robot upwards by 1mm, maintaining the contact as well as reducing the force on the robot (safety for robot joints). This process is repeated until we observe a constant value in the force data which indicates that the screw has been unscrewed completely. At this point, we move the robot back to the home position.
IMITATION LEARNING
Data Collection
I developed a comprehensive dataset for imitation learning using the Franka Emika FR3 robotic arm, ensuring a rich and diverse data collection process for training autonomous manipulation models. The data acquisition was conducted through the Robot Operating System (ROS), where I recorded multi-modal sensor inputs, including transformation (TF) data, joint states, and force measurements, to capture the robot’s precise interactions with its environment. To enhance learning accuracy, I recorded camera data from three different angles—robot mount, front view, and side view—providing a holistic visual representation of the task execution. The demonstrations were performed using the Haply Inverse3 haptic device for tele-operation, enabling fine-grained control over the robot’s movements.
AI Models
The collected dataset was utilized to train and evaluate over 50 models, focusing on alignment and unscrewing tasks. I systematically analyzed model performance, refining learning strategies to improve generalization and efficiency in robotic manipulation.
ISAAC SIM SIMULATION OF DISASSEMBLY
I created a simulation in NVIDIA Isaac Sim where a Franka FR3 robot disassembles a box by first unscrewing 6 screws using an attached electric screwdriver and then detaches the top-lid of the box using a 2-finger gripper and places it on the side. The robot's movements were controlled using ROS and MoveIt!
Issues Debugged:
1. Simulating Nut-Bolt contact physics: Since the nut-bolt pair consists of object pairs with a lot of contact points along with their small size, it is crucial for their collision mesh to have an high resolution in order to simulate their contact well. In Isaac Sim this is achieved by using a SDF collision mesh. Also, the threading between the nut and bolt pairs were kept consistent to have a perfect fit.
ISAAC SIM SIMULATION OF ROBOT WORKCELL
I designed a simulation in NVIDIA Isaac Sim that automates battery pack disassembly using robotic manipulators and mobile robots. The system features dedicated pack and module disassembly stations where robotic arms systematically deconstruct battery packs, while mobile robots transport battery modules for further processing. A quadruped robot assists in module transport, enhancing automation. The simulation integrates human-expert data collection, autonomous human-robot collaboration, and efficient unmanufacturing strategies. A human tele-operator oversees the process remotely, ensuring seamless integration of robotics and AI for safe and efficient battery recycling or repurposing.
Page updated
Google Sites
Report abuse