New Category of Machine Vision to Unlock a New Performance Dimension
Since their inception 150 years ago, all conventional video tools have represented motion by capturing a number of still frames each second. Displayed rapidly, such images create an illusion of continuous movement. From the flip book to the movie camera, the illusion became more convincing but its basic structure never changed.
For a computer, this representation of motion is of little use. The camera is blind between each frame, losing information on moving objects. Even when the camera is recording, each of its ‘snapshot’ images contains no information about the motion of elements in the scene. Worse still, within each image, the same irrelevant background objects are repeatedly recorded, generating excessive unhelpful data.
Evolution developed an elegant solution so that natural vision never encounters these problems. It doesn’t take frames. Cells in your eye report back to the brain when they detect a change in the scene – an event. If nothing changes, the cell doesn’t report anything. The more an object moves, the more your eye and brain sample it. This process allows human vision to collect all the information it needs, without wasting time and energy reprocessing images of the unchanging parts of the scene.
By only recording what changes, the eye and brain can gather useful information from things changing at up to 1000 times a second, without needing to engage enormous amounts of brain power. This is event-based vision whereby independent receptors collecting all the essential information, and nothing else.
New Category of Machine Vision to Unlock a New Performance Dimension
The new category of machine vision is expected to unlock a new performance dimension in speed, precision and endurance for ultra-high speed counting, vibration Measurement, kinematic monitoring and more.
Event Based Vision Systems Perceive Scene Vitality While Overlooking The Irrelevant
PROPHESEE creates both neuromorphic sensors and bio-inspired algorithms that function like the eye and brain. This holistic approach represents a fundamental shift in computer vision – the departure from frame-based sensors, to event-based vision systems.
Each pixel only reports when it senses movement. Whereas in a frame-based sensor all pixels record at the same time, in an event-based sensor each pixel is perfectly independent.
Producing up to 1000 times fewer data than a conventional sensor whilst achieving a higher equivalent temporal resolution of >10 000 fps. Industrial applications include processes monitoring, inspection, object identification, detection & tracking, high speed motion control, vibration analysis and robotics.
The Metavision sensor is a third generation 640 x 480 VGA event-based sensor. Prophesee has also recently partnered with Sony Semiconductor Solutions to integrate the Prophesee advantages with a new generation stacked HD event-based vision sensor co-developed with Sony. Prophesee has released two evaluation kits specifically for the architecture that provides developers early access to the sensor, which features 4.86µm pixel pitch and 1,280 x 720 pixel resolution. The sensors asynchronously detect luminance changes for each pixel and output the changed data. This is combined with information on pixel position (xy coordinates) and time, thereby enabling high-speed, low latency data output. Its sensors capture up to 1,000 times fewer data than a conventional sensor, while achieving a higher equivalent temporal resolution of 10,000 fps.
Prophesee was been named the winner of the prestigious VISION Award, announced at the recent VISION exhibition; the award recognizing innovation in machine vision. The judges cited Prophesee for making great strides with its event-based sensing, which advances machine vision capabilities over traditional frame-based approaches. Martin Wäny of the judging panel called event-based vision a ‘new paradigm’ in imaging technology, one that’s been worked on for 20 years, but is now coming to fruition through the efforts of Prophesee.
For more information: www.prophesee.ai
In the video below on the LEFT, a simulation of Frame-Based Vision running at 10fps. This approach leverages traditional cinema techniques and records a succession of static images to represent movement. Between these images, there is nothing, the system is blind by design.
On the RIGHT, the same scene recorded using Event-Based Vision. There is no gap between the frames, because there are no frames anymore. Instead, a continuous stream of essential information dynamically driven by movement, pixel by pixel.
