A ToF (time-of-flight) camera is a 3D camera that measures depth from the time light takes to travel to an object and back. It captures a depth map of the whole scene in a single frame, which makes it well suited to large areas, piles and large objects. This article covers how a ToF camera works, its strengths and weaknesses, and how we use one with a robot.
How a time-of-flight camera works
The camera has its own light source, usually infrared. The light illuminates the scene, reflects off objects and returns to the sensor. Each sensor pixel determines how long the light took for the round trip and calculates the distance from that. The result is a depth map in which every pixel holds a distance value; converting it gives a point cloud.
Because the camera illuminates the scene itself, it does not need a pattern projector or a second camera, as structured light and stereo vision systems do. For a comparison of all the technologies, see 3D machine vision in industry.
Advantages
- Speed. The whole scene is captured at once, giving a continuous stream of depth images.
- The whole scene in one go. A single camera covers a large area - for example, an entire pile on a ramp.
- Robustness. Industrial models have rugged housings with dust and moisture protection.
- Simple installation. One compact device, with no need to align a projector and cameras relative to each other.
Limitations
- Lower resolution. ToF sensors have far fewer points than structured light scanners.
- Millimetre to centimetre accuracy. For fine measurements requiring tenths of a millimetre, structured light is the better choice - see Photoneo PhoXi and MotionCam-3D.
- Depth noise. A single frame has noticeable noise, especially on dark surfaces and at object edges.
- Reflections. Shiny surfaces can reflect light away from the camera or via several surfaces, producing missing or false points. More in 3D scanning of shiny surfaces.
Example: Basler blaze-101
One of the ToF cameras we work with is the Basler blaze-101. According to manufacturer and distributor data:
- depth resolution - VGA (640 × 480);
- 30 frames per second;
- 940 nm light;
- GigE interface, IP67 housing;
- field of view - 67° × 51°;
- millimetre-level accuracy (the distributor states about ±5 mm);
- working distance - up to approx. 10 m.
These specifications suit applications where a large area has to be monitored from a height of several metres and the objects to be found measure tens of centimetres or metres.
Our project: picking logs from a pile
We are currently commissioning a wood-processing cell in which a KUKA KR 60-3 robot with a KR C4 controller picks logs from a pile on a ramp and feeds them butt-end first into the feed rollers of a processing machine. The machine produces stakes and poles. The logs are 2600-3300 mm long and roughly 100-130 mm in diameter.
How the vision part works
- A Basler blaze-101 ToF camera mounted above the pile captures the entire ramp.
- A PC (Python) builds a 3D height map.
- Our own 3D detector finds the logs and estimates each one's axis, length and ends. In addition, the logs are segmented in photos using the SAM AI model.
- Pick rules decide which log to take and where to grip it.
- The coordinates are sent to the robot via EthernetKRL; the robot program is written as KRL modules.
The gripper is driven by two Delta ASDA-A2 servo drives over CANopen; both drives start within 0 ± 2 ms of each other, and clamping or unclamping takes approx. 0.22-0.25 s. The operator panel runs on the local network, and the operator can pause operation at any time. We follow the principle "AI suggests, safety permits": motion is always permitted by the safety circuit, never by a neural network.
How we reduced depth noise
A single ToF frame is too noisy to describe the surface of a log. So we fuse several frames: on the test bench, fusing 20 frames gave a depth noise of approx. 0.8-1.3 mm. Compared with a log diameter of roughly 100-130 mm, that noise is small. Frames can be fused because the pile stays still while it is being captured.
When to choose a ToF camera
- You need to cover a large area: a pile, a pallet, a container, large objects.
- The objects are large and millimetre or centimetre accuracy is enough.
- Speed and a simple, robust device matter.
If you need sub-millimetre accuracy or the shape of small parts, structured light scanners are the better fit.
Try it on your own objects
Our main focus is machine vision, especially 3D, and we work with both ToF cameras and Photoneo scanners. We start projects with a trial on your real objects, so that before any larger investment it is clear which technology to choose. More under machine vision services; you can reach us via our contact page.