A camera sees an object in its own coordinate system - from its lens. A robot moves in its own coordinate system - from its base. For the robot to pick exactly what the camera sees, you need to know precisely how these two systems relate to each other. That relationship is established by hand-eye calibration. Without it, even a very accurate camera and a very accurate robot will work inaccurately together.
What is calibrated in a robot vision system
- Camera calibration. The camera's intrinsic parameters: lens focal length, optical centre, distortion. Many 3D cameras and scanners come with factory calibration; for 2D cameras you often have to do it yourself.
- Robot tool and base calibration. An accurate TCP (tool centre point) and work base. See our article on industrial robot programming.
- Hand-eye calibration. The position and orientation of the camera relative to the robot. This step links vision and motion, so it has to be carried out separately in every cell, once the camera is mounted in its final position.
Eye-in-hand and eye-to-hand
There are two main ways to mount a camera, and each calibrates a different relationship.
- Eye-in-hand. The camera is attached to the robot flange or gripper and moves with it. The robot can look at the object from several sides, move closer, or scan a large object in sections. The calibration determines the camera's position relative to the flange.
- Eye-to-hand. The camera is fixed above the work area. An image can be captured while the robot is still finishing its previous move. The calibration determines the camera's position relative to the robot base.
In our log feeding cell the ToF camera hangs above the pile - that is eye-to-hand (more in bin picking). In the wagon inspection platform described below, the robot itself points the scanner and cameras - that is eye-in-hand.
How hand-eye calibration is done
- Calibration target. A plate with a known pattern (e.g. a checkerboard or a marker pattern), or another object whose geometry is known precisely and which the camera can find reliably.
- Many different poses. The robot is moved to many poses with different angles and distances. In eye-in-hand, the camera looks at a stationary plate; in eye-to-hand, the robot holds the plate.
- Data collection. In each pose, the robot flange position from the controller and the position of the calibration target in the camera image are recorded.
- Calculation. An algorithm finds the transformation that best fits all these pairs.
- Verification. The result is checked: residual error (RMS), control points, repeated measurement of the same pose.
Why accuracy depends on calibration
Calibration error adds to every pick and every measurement. If the camera sees the object to within a millimetre, but its position relative to the robot is known only to within several millimetres, the robot will still be off by several millimetres. Angular error is especially dangerous: a small rotational deviation turns into a large offset far from the camera.
The most common causes of inaccuracy:
- an inaccurate robot TCP or incorrectly entered tool data;
- too few poses, or poses that are too similar;
- a warped or poorly visible calibration plate;
- play in the camera mounting;
- the gripper, camera bracket or camera position changed after calibration.
Example: a wagon inspection platform
In a 2026 pilot project, an autonomous tracked platform carrying a Yaskawa Motoman MH5F robot drives alongside a freight train along an RTK GNSS route. The robot points a Photoneo 3D scanner and Basler cameras at the bogies: wheel profile, axle boxes, springs. Each finding is linked to a scan and a photo, and the decision is made by the inspector.
Here the camera moves with the robot, so hand-eye calibration determines whether scans taken in different poses line up with each other and with the robot's coordinates. After improving the calibration:
- the RMS error fell from 8.7 mm to 3.2 mm;
- repeatability of the same pose - 1.45 mm (based on few measurements so far).
For the scanners themselves, see Photoneo PhoXi 3D scanners; for the cameras, industrial cameras.
When to recalibrate
- if the camera or its bracket has been moved;
- after changing the gripper, camera bracket or lens;
- after a robot repair or a drive replacement;
- when pick or measurement error starts to grow for no obvious reason.
It is advisable to have a simple check procedure that the operator can carry out alone: a reference point or object that shows straight away whether the calibration is still valid.
We calibrate cameras and robots in new and existing cells, and we have our own KUKA and Yaskawa Motoman robots for testing. More under machine vision and robot integration services, in our projects and in our article on 3D machine vision. Is your robot with a camera "missing" its target? Get in touch - we will check the calibration.