Autonomous Manipulation Lab
The goal for this project is to develop a system that can autonomously detect blocks within the configuration space, and utilize the robotic arm to pick them up and place them at a given location. The project achieved success, with the system accurately identifying both the color and location of the blocks, and the robotic arm being able to de-stack and position them at the targeted locations.
Equipment
The equipment used for this project includes an RX200 robotic arm, featuring 7 servo joints and 5 degrees of freedom (DOF), accompanied by an Intel RealSense LiDAR camera.
Camera model
The project's initial phase involves correlating on-screen pixel positions with the robot's real-world workspace. This alignment, crucial for accurate block detection, is achieved through camera calibration using the camera's intrinsic matrix and an extrinsic matrix. This ensures on-screen pixels match real-world locations for precise robot manipulation through AprilTags.
Kinematics
The kinematics of a robotic arm enable us to manipulate and guide the arm to a desired location by actuating the joint motors. This control mechanism allows for precise movement and positioning of the robotic arm within its workspace.
Forward Kinematics: Using joint angles and link displacements and the Denavit-Hartenberg convention, we can determine the end-effector's position relative to its base.
Inverse Kinematics: This determines the joint angles needed to position the end-effector at a desired spot. Unlike forward kinematics, it works in reverse. We employed a geometric solution with kinematic decoupling
Block detector
Using OpenCV, we developed a block detection algorithm. This algorithm processes RGB images to identify blocks and determine their colors. By analyzing the depth image, the algorithm can distinguish whether a block is positioned on top of a stack or located on a flat surface. For every block detected, we drew a bounding box around it and labeled it with the color information. The position where the end-effector is designated to pick up the block corresponds to the block's centroid.
Results
The video presented below demonstrates the block detection algorithm and pick-and-place capabilities of the kinematic arm. The system is capable to stack and de-stack up to 5 blocks. During any pick-and-place, the arm operates collision aware and safety-critical.