Before a robot or vision system can do anything with an object, it first has to answer a question: where is the object in the image, and where do its edges end? When objects are identical, well lit and neatly arranged, classical algorithms handle this. When they are natural, varied, jumbled or dirty, artificial intelligence in manufacturing increasingly helps, in the form of AI segmentation.
What segmentation is
Neural networks for images solve several types of task:
- Classification. Answers what is in the image: a good part or a bad one.
- Detection. Finds an object and marks it with a bounding box.
- Segmentation. Marks the exact outline of an object: every pixel is assigned to a specific object or to the background. The result is called a mask.
A bounding box is often not enough for a robot: if two logs lie diagonally next to each other, their boxes overlap. A mask shows which pixel belongs to which object, so the axis, the ends and the pick point can be calculated.
SAM models
SAM (Segment Anything Model) is a family of general-purpose segmentation models. Their advantage is that you do not need to collect thousands of labelled images and train a model from scratch for every new part. The model already knows how to separate objects; we specify which objects we are interested in and adapt the result to the specific task.
A general-purpose model, however, knows nothing about production rules. It may mark a shadow, a piece of bark or a reflection as a separate object. That is why its output is always checked by other layers of the system.
In practice, AI segmentation is rarely the only element of a solution. Usually it is one of several steps: a camera or scanner provides the image, AI marks candidates, geometry and rules verify them, and only then is the result passed to the robot, the operator or a spreadsheet.
Example: a log pile and a 3D height map
In our log feeding cell, a KUKA robot picks a log from a pile and feeds it into a processing machine. The project is at the commissioning stage. A ToF 3D camera is mounted above the pile, and a PC (Python) combines several sources of information:
- SAM masks. AI marks individual logs in the photo.
- 3D height map. Shows which log is on top and where it is in space.
- Our own 3D detector. Estimates the log's axis, length and ends.
- Pick rules. Decide which log to pick and which to leave.
The mask says "what is what", while the 3D data says "where and how high". If the AI mask and the 3D geometry disagree, that object cannot be treated as a reliable target. For the whole process, see bin picking with 3D vision; for the cameras, ToF 3D cameras.
Example: detecting axle boxes
In a 2026 pilot project, an autonomous platform with a Yaskawa Motoman MH5F robot drives alongside a freight train, and the robot points a Photoneo 3D scanner and Basler cameras at the wagon bogies. On a recorded run, the SAM 3 model found 8 of 8 axle boxes with no false detections.
This is the result of a single run, not a statistic, but it shows that a general-purpose model can find a specific component in a complex image. Each finding is linked to a scan and a photo, and the inspector makes the decision.
Principle: AI proposes, safety permits
A neural network can be wrong, and this has to be accounted for in advance. That is why in our systems AI only proposes: which object to pick, where a component is, what might be a defect. Motion is always permitted by the safety circuit, not by a neural network. In practice this means:
- the AI result is checked against geometry and rules;
- an unclear case means stopping or asking the operator, not guessing;
- in inspection tasks, a person makes the final decision.
What an AI segmentation project needs
- Photos of real objects taken under the conditions the system will work in: the same lighting, distance and background.
- A clear description of what the system must find and what to do with the result next.
- Agreed behaviour for unclear cases: stop, skip or call the operator.
When AI segmentation is a good fit
- Natural objects whose shape differs every time: timber, logs, agricultural produce.
- Jumbled, overlapping objects in a pile or a bin.
- Complex backgrounds where classical thresholds and contour detection do not work.
- Inspection, when a specific component or zone has to be found in a large image.
When parts are identical and lighting is controlled, classical 3D machine vision is often enough, and AI is not needed. For defect detection, see quality inspection with machine vision.
We start AI segmentation projects with a trial on your real photos and parts. Find out more in our AI for manufacturing and machine vision services and projects. Send us a few photos of your objects: get in touch.