Vision-Based Robot Control System Uses Artificial Intelligence

ZF, a global technology company headquartered in Friedrichshafen, Germany, wanted to automate machine tending in a high-volume milling station where gears are manufactured. In this case, metal rings are picked from a crate and placed onto a conveyor belt. The ring is the raw part of what will become a gear during subsequent manufacturing steps. This task brings multiple challenges:

  • The rings in the crate shift during transportation, making their positions unpredictable.
  • The position of the crate, once delivered and facing the robot, can vary 20–30 mm in all directions.
  • The form of the crate can vary. Sometimes its sides are not straight. They can be pressed inward, with a deviation of 20–30 mm.
  • In the spring and autumn, the workspace gets direct sunlight. Although a crate’s inside walls are covered with carton paper, one-third of the rings are still directly exposed to sunlight.
  • Bubble foil on the inside of the crate covers some parts of the rings.
  • The surface of the rings can have oil and rust. This creates variance in their visual representation from the robot’s perspective.

The Solution

MIRAI is a vision-based robot control system from Micropsi Industries that, using artificial intelligence, enables robots to deal with complexity in production that would be otherwise impossible or extremely difficult to get around with hand-engineered solutions, prohibitively expensive as well. MIRAI attaches to and augments industrial robots. Once fitted with MIRAI, a robot can perceive its workspace and correct its movement where needed as it performs a task. MIRAI can be easily and quickly trained or retrained for myriad tasks by those with no background in engineering or artificial intelligence.

Compared with other available automation solutions, MIRAI makes it possible for:

  • Industrial robots to deal with all the variance that crops up in production.
  • Workers to use machine vision out of the box. They can set up the system and train a skill in only a few hours.
  • Workers without expert knowledge in robotics programming or artificial intelligence to train robots.

When deployed for a task, the MIRAI system kicks in when needed for a complex step or steps in an application process.

Classic automation solutions — with or without vision systems — would be either unable to deal with these complexities or very expensive to set up. Even then they would be tailored for this task alone and no others. This ring picking task for ZF was initially solved with classical robot programming, but MIRAI was markedly faster and more reliable for possible displacements.

The Solution Setup

ZF has deployed an automated machine tending solution that can deal with numerous challenges. The setup here essentially comprises the MIRAI kit (including the control box and the camera), a Universal Robots UR10e, an OnRobot force-torque sensor, and a Schunk gripper.

The metal rings in the crate arrive in layered beds, the workpieces laid closely together on their flat sides. Via its native controller, the UR robot has been programmed to move above individual rings in the crate. Once the robot is above a ring, the MIRAI system takes control: it moves the robot to the nearest ring and places the gripper in gripping position. After this position is reached, the robot’s native system reassumes control; the robot then picks up the ring, moves it to the conveyor belt, and places it on the belt.

Training the MIRAI-powered robot to carry out its part of the solution process took only four days.

For more information: