automated robot moving to collect data for camera in hand configuration and ROS based api #23

Ruthrash · 2023-04-05T15:57:18Z

usage:
python3 ROSCameraRobotCalibration.py --move_robot_automatically True --use_delta_poses False/True ....other args...

…collect data for calibration

n-yoshikawa · 2023-04-12T20:31:37Z

I tested the calibration script.

What I did

Start bash script

cd /root/git/frankapy
bash ./bash_scripts/start_control_pc.sh -i 172.16.0.4 -u sf3202inuc -d /root/git/franka-interface -a 172.16.0.2 -w 172.16.0.3

Start camera based on a previous comment at Test camera in environment calibration #22 (comment)

roslaunch realsense2_camera rs_camera.launch filters:=pointcloud enable_infra1:=false enable_infra2:=false

Start end-effctor tf (for customized gripper)

cd ~/git/frankapy/catkin_ws/src/franka_control
python3 ee_robotiq_transformation.py

Start calibration

python3 ROSCameraRobotCalibration.py --move_robot_automatically True --robot_base_frame_name /panda_link0 --ee_frame_name /virtual_eef_link --rgb_image_topic /camera/color/image_raw --rgb_camera_info_topic /camera/color/camera_info

Result

The robot moved automatically
The camera looked working because reprojection error was calculated
The automatic pose was not suitable for our custom gripper that is parallel to table
Problem 1: reprojection error was too high at some poses (maybe because I could not take enough distance between gripper and markers)

reprojection error 0.4223259860582057
#### Very high reprojection error ####
#### Ignoring this sample ####

Problem 2: ArUco tag was not detected in many poses because camera was not oriented to the markers.

No aruco tag detected in this sample
accepted pairs of pose, no. of frames processed 0 4

As a result, I could not get a calibration value and evaluate its correctness.

Setup

…oses for robot-camera calibraiton

n-yoshikawa · 2023-05-02T15:49:20Z

after truying the autoamtic calibration from the reset_arm confgituration, we had the following observations:

some times the robot ee was not pointing toward the borad, i.e., maybe those intermediate waypoints should be updated.
We tried two times and in both times the robot reached a joint limit and therefore stopped moving. here is a photo of the robot in joint limit:
it would be nice to print the following as a user feedback while running automatic trajectory: waypoint X/Y so that the user knows when to expect to finish the data collection.

images of two scenarios robot reacing joint limits:

n-yoshikawa · 2023-05-02T15:58:55Z

as we discussed we started the robot from another configuration, as shown below, and right after starting the robot goes to initial reset configuration and starts moving. So, regardless of initial configuration set by user, the robot always goes to reset configuration. However this is different from what we have discussed in the meeting and we were expecting, but we think it is safer to be this way, since the robot motion is quite wide. So, for different robot, maybe it is safer the user moves the robot ee.

initial configuration:

n-yoshikawa · 2023-05-02T15:59:48Z

There should be a photo with inidication of distances from the robot for the user, so that they know where to place the apriltag board.

kouroshD · 2023-05-02T19:59:10Z

calibration/docs/USAGE.md

+  2.a. If you are using relative poses, the robot will move relative to the initial end-effector configuration when you start the script, therefore, before starting frankapy, point the camera(camera in hand case) or the calibration tag(camera in environment case) to the calibration tag or the camera respectively. Ensure, the calibration tag's normal is close to perpendicular to the camera and there's around ~30cm between the tag and the camera.    



How can we enable relative poses? Mayebe we can say the option that should be True to use relative pose.

kouroshD · 2023-05-02T19:59:51Z

calibration/docs/USAGE.md

@@ -187,6 +190,9 @@ optional arguments:
                        is first manually moved to an initial pose and the script controls EE to
                        predefined relative poses. If false, the EE should be moved manually(white
                        status LED) and press enter to collect data
+ --use_delta_poses [USE_DELTA_POSES]:True(default)/False


is it the same as using relative poses?

kouroshD · 2023-05-02T20:01:44Z

calibration/docs/USAGE.md

+For the camera-in-environment case, the calibration tag needs to be **rigidly** attached to the robot's End-Effector. You could use the provided CAD files for the finger tips and gripping points in [models_4_3d_printing](../models_4_3d_printing). The [finger tips](../models_4_3d_printing/franka_custom_finger_tips.stl) are to be attached to Franka End-effector and the [gripping points](../models_4_3d_printing/finger_grasp_points.stl) or [handle plate](../models_4_3d_printing/finger_handle_plate.stl) are drilled/screwed onto the calibration tag. Now make the Franka End-effector with custom finger tips(figure 1) grasp the calibration tag(as show in figure 3) with the attached custom gripping points(figure 2), this ensures that the tag remains rigid with respect to the End-effector. 



Images can be helpful to show the configuration that you are refering to.

Ruthrash added 2 commits April 4, 2023 19:10

moving robot automatically with both abs poses and relative poses to …

a5c1367

…collect data for calibration

tested automated moving with both relative and absolute ee poses

0963bdb

kouroshD requested a review from n-yoshikawa April 12, 2023 16:05

updates documentation about automatic data collection with relative p…

0d51f73

…oses for robot-camera calibraiton

Ruthrash mentioned this pull request Apr 26, 2023

How to use the automated data collection for robot-camera calibation? #24

Closed

kouroshD reviewed May 2, 2023

View reviewed changes

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

automated robot moving to collect data for camera in hand configuration and ROS based api #23

automated robot moving to collect data for camera in hand configuration and ROS based api #23

Ruthrash commented Apr 5, 2023

n-yoshikawa commented Apr 12, 2023 •

edited

Loading

n-yoshikawa commented May 2, 2023 •

edited by kouroshD

Loading

n-yoshikawa commented May 2, 2023 •

edited by kouroshD

Loading

n-yoshikawa commented May 2, 2023

kouroshD May 2, 2023

kouroshD May 2, 2023

kouroshD May 2, 2023

		2.a. If you are using relative poses, the robot will move relative to the initial end-effector configuration when you start the script, therefore, before starting frankapy, point the camera(camera in hand case) or the calibration tag(camera in environment case) to the calibration tag or the camera respectively. Ensure, the calibration tag's normal is close to perpendicular to the camera and there's around ~30cm between the tag and the camera.

		For the camera-in-environment case, the calibration tag needs to be rigidly attached to the robot's End-Effector. You could use the provided CAD files for the finger tips and gripping points in [models_4_3d_printing](../models_4_3d_printing). The [finger tips](../models_4_3d_printing/franka_custom_finger_tips.stl) are to be attached to Franka End-effector and the [gripping points](../models_4_3d_printing/finger_grasp_points.stl) or [handle plate](../models_4_3d_printing/finger_handle_plate.stl) are drilled/screwed onto the calibration tag. Now make the Franka End-effector with custom finger tips(figure 1) grasp the calibration tag(as show in figure 3) with the attached custom gripping points(figure 2), this ensures that the tag remains rigid with respect to the End-effector.

automated robot moving to collect data for camera in hand configuration and ROS based api #23

Are you sure you want to change the base?

automated robot moving to collect data for camera in hand configuration and ROS based api #23

Conversation

Ruthrash commented Apr 5, 2023

n-yoshikawa commented Apr 12, 2023 • edited Loading

What I did

Result

Setup

n-yoshikawa commented May 2, 2023 • edited by kouroshD Loading

n-yoshikawa commented May 2, 2023 • edited by kouroshD Loading

n-yoshikawa commented May 2, 2023

kouroshD May 2, 2023

Choose a reason for hiding this comment

kouroshD May 2, 2023

Choose a reason for hiding this comment

kouroshD May 2, 2023

Choose a reason for hiding this comment

n-yoshikawa commented Apr 12, 2023 •

edited

Loading

n-yoshikawa commented May 2, 2023 •

edited by kouroshD

Loading

n-yoshikawa commented May 2, 2023 •

edited by kouroshD

Loading