Projects | Personal Website

Assembly Robot

Sun, 01 Sep 2024 00:00:00 +0000

Abstrct

Movement Primitives (MP) are promising methods for modeling robot motion from human demonstrations. Using learned parameters and a phase variable typically scaled to $\boldsymbol{[0,1]}$, MP can generate a trajectory as the phase transitions from 0 to 1. However, in assembly tasks that require high precision and online adjustments, typical MP methods do not work well, particularly when the trajectory is executed with limited controller gains, or when the task is hindered by some obstacles. Therefore, we propose Phase-Recognizing Movement Primitive (PMP), which can make a stable estimation of the task phase online, make suitable adjustments when the assembly task is hindered by external disturbances, and finally achieve precise assembly while using a low gain compliance controller. Specifically, given the robot state, we assume the phase is a random variable with a Gaussian distribution. Consequently, the phase velocity can be computed, enabling us to determine whether the task is hindered and to retry if the task is stuck. We test our method on a Peg-In-Hole assembly task in simulation and a Slide-In-The-Groove assembly task on real UR5. The experimental results show that PMP can make stable estimations of the phase and thus make adjustments to complete the hindered assembly tasks.

Experiment

BIM-based Robot

Fri, 01 Sep 2023 00:00:00 +0000

Overview

This research uses Building Information Modeling(BIM) information in 2D Simultaneous Localization and Mapping(SLAM). In this project, the robot uses the information from BIM to generate a 2D map as prior knowledge of the building. Then the mobile robot uses this global map for initial localization and path planning, which can be a coverage path for map updating or a path to a target position. Besides, the mobile robot can update this prior map based on its following exploration. In conclusion, the advantage of this method is that it can help the robot reach its desired target without generating a map first. Instead, the robot can reach the desired position based on prior knowledge and update the map while moving.

The SLAM approach used in this project is based on Cartographer, and the method is tested in simulation in Gazebo with some random obstacles. All the work was done by myself.

Abstract

Construction robots don’t have the global information of the building unless they are allowed to build a map by SLAM in advance, which is time-consuming and prevents construction robots from making global plan for the task. At the same time, Building Information Modeling (BIM) is a digitalization and standardization of the building information. With the existing of BIM, the building interior in construction scenes is actually semi-unknown instead of totally unknown. In this research, we proposed a pipeline to transform the BIM to a 2D ideal map. Then, we combined the 2D ideal map and SLAM together for robot navigation. By using this ideal 2D map as the initial global map of the robot, the robot can obtain global information about the interior of the building, thus saving the time and inhance the efficiency.

Piepline

This figure shows the pipline of data transformation.

Data Transform Pipeline

This figure shows the building in the form of obj.

Mesh

This figure shows the building in the form of Octree.

Octree

Simulation

This figure shows different map in SLAM.

Map Information

As shown in this figure, although our robot was new in the environment, it had the global information about the building according to the global costmap, which is generated from the 2D ideal map by the layered costmap structure. The largest box shows the global costmap, and the second large box shows the information from the sensor. And the smallest box shows the local costmap.

Right Global Path

This figure shows that the robot can generate a right path to the target although the sensor doesn’t have any information around the target point.

Right Local Path

This figureshows that the robot can change its local path according to the local cost map, to avoid the crush between itself and the obstacles.

Arrive Any Target

This figure shows that the robot can arrive any target point on this specific floor in the building.

Conclusion

In this paper, we proposed a pipeline to transform the BIM to a 2D ideal map, and then we used the 2D ideal to navigate the robot thus saving the time for building a map by SLAM in construction scenes. In the future, we will try to use more semantic information and lifecycle information in BIM to help robot finish its task.

Painting Robot

Fri, 01 Sep 2023 00:00:00 +0000

Overview

This project is the topic of my Master’s thesis.

In construction spraying operations, spray robots offer significant advantages over manual labor. However, to meet the high precision required, spray robots must accurately perceive large-scale objects. Additionally, the demands of the spraying process require that spray robots incorporate new procedural constraints into their path planning for wall coverage painting. This research mainly addresses the following two issues:

Full-coverage spray trajectory planning considering robot reachability constraints.
Easy-to-deploy and verifiable algorithms across systems and platforms. Integrated spraying software combined with specific hardware.

Robot Model

Perception

Initially, we used a region-growing based method for perception, projecting the wall surface onto a 2D plane. This approach effectively transitioned the problem from 3D coverage path planning to 2D coverage path planning.

However, we discovered that this method was inadequate for handling tasks involving corners and curved surfaces. Therefore, we are now exploring a slicing-based method to address these challenges.

Slicing-Based Painting Point Generation

The slicing-based painting point generation method uses a series of parallel planes to slice the area to be painted. This approach is commonly used in workpiece spraying. However, in construction spraying, the robot’s spraying range changes from known mesh files to perceived point cloud files, so the painting point generation method needs to be improved.

Painting Reachability Map Based Trajectory Optimization

This paper proposes a Compact Reach Map and a Painting Reach Map, and formulates the spray trajectory generation problem as an optimization problem based on these two maps.

Compact Reach Map

Painting Reach Map

Optimization

Combining the constraints of the Painting Reach Map, the trajectory generation can be formulated as a convex optimization problem for solution.

Project Status

This project is still in progress…

Experiments

Latex Paint Hanging Test

Wide Range

Racing Robot

Fri, 01 Sep 2023 00:00:00 +0000

Overview

This project aims to participate in a robot racing competition. In the competition, the robot is asked to finish the track and hit the target at the destination without getting into the black area as fast as possible. Robots are requested to use a humanoid robot with an omnidirectional chassis comprising three Mecanum wheels and a webcam on its backpack. All programs were running on a Raspberry Pi 4B in the backpack.

In this project, I was responsible for the simulation, vision, and control algorithm. I built a simulation environment in the V-REP simulator, tested an edge detection-based method for track detection, and used vision-based PD control for the chassis. Besides, we use Keras and TensorFlow to solve a classified problem and rotate the robot in the right direction since the robot is placed in a random direction at the starting point.

Simulation

We first built a simulation environment in V-REP to test our algorithm, we used a remote Python API for V-REP to get simulation data from the virtual camera and control the robot’s motors according to our algorithm. The field of view of the real webcam is tested to guarantee that the virtual camera has similar parameters to the real one.

Real Robot

The original position of the webcam is too high to get enough information on the ground in front of the robot. So we used a 3D printer to create an adjustable-angle camera mount, allowing it to tilt downwards to overlook the ground. To increase the friction of the wheels, we tried a new material for molding the tire and did some tests on the real robot.

Video of test

This video shows the whole process of the competition. As we can see, the robot starts with different orientations, then it classifies the command into three classes, namely left, right, and forward, according to the picture from the webcam. For the left or right command, the robot turns left or right with a specific angle. For the forward command, the robot starts to control its chassis with an edge detection-based PD control. The visual algorithm can be robust even when faced with camouflage plots. Finally, the robot uses an arc detection-based method to estimate the distance to the target and decelerate while adjusting its orientation.

Transport Robot

Fri, 01 Sep 2023 00:00:00 +0000

Overview

This project aims to make a lightweight transport robot for a material-carrying competition. Robots are asked to catch materials from several specific positions and carry them to some other positions. The more accurately the materials are placed, and the more materials are handled, the shorter the time used, resulting in a higher score. In this project, a mobile robot with a lightweight arm is designed, and two 2D cameras are used, which are mounted on the end effector of the arm and the side of the robot chassis, respectively. We use a visual serving strategy to adjust our arm to guarantee the accuracy of placing.

In this project, I was responsible for the control of the chassis, arm, and camera. I use a RANSAC-based method to detect the target and do a close-loop control for placing action. I also participated in the design of the robot, including the layout, the type of arm selected, and the carrying strategy.

Description of the Task

Several materials in three colors—red, green, and blue—are arranged in different positions on the shelf. Besides, there are the same number of QR codes on the shelf as the number of materials, each QR code represents the target position of the material. With all this information, the robot should first grasp these materials onto the robot, and carry them to the desired position.

Besides, black grids on the ground allow us to use grayscale sensors for the robot’s localization.

For the placement, we use a RANSAC-based method to detect the center of the target position according to the camera mounted on the end effector.

Final Version of Our Robot

This video shows the whole process of the competition, as we can see, the robot uses a visual serving strategy to adjust the end effector and achieve precise placement.

Group and Team

Team

Group