US researchers have developed a vision sensor which can visually track objects that it has been trained to ‘see’.

New color-sensitive vision sensor for tracking objects

US university Carnegie Mellon and Texan startup  Charmed Labs have been working together to create a small vision sensor, Pixy, which can detect objects and distinguish them by color. As robots are now being developed for use in the home, many researchers have been trying to furnish them with ‘human’ senses to enable them to integrate more effectively into our daily lives. The University of Tokyo for example has developed a patch which can give robots a finer sense of touch. This new sensor is equipped with visual sensitivity and can also memorize specific movements and actions, which will lead it to operate more autonomously.

Automated detection of colored objects

Pixy is essentially a tiny camera which can be programmed to recognize objects and their specific coloring. All you have to do is place an object of the sort you wish Pixy to track – a yellow ball for instance – in front of it for several seconds. The device will memorize the object and note its color and will subsequently be able to detect similar objects automatically without the aid of its user. It can detect and track up to seven different colors at once, plus a large range of objects, and can be re-programed at any time. Pixy works fast, processing an image frame every 20 milliseconds – which means it can track movement with a high degree of accuracy. Using a dedicated app, the pictures can then be sent automatically to the user’s computer.

Incorporating Pixy into bots

One of the basic objectives of Pixy’s creators is to make the vision sensor as easy as possible for a wide range of users to incorporate into their own bots. Our goal is “to bring advanced technologies to new audiences by making them easy to use and affordable,” explains Charmed Labs founder Rich LeGrand. Accordingly, users will be able to train the device themselves to recognize new objects, individual colors and color-codes and, in the longer term, teach it to perform ad hoc tasks. Up to now robots and robotic devices have been able to detect objects but without using ‘sight’. Pixy has been developed to be compatible with a range of micro-controllers such as Arduino, and so additional applications can be envisaged. One example would be to program Pixy to recognize a color-coded system on entry and exit doors so that it could serve to guide a motorized robot designed to assist people with physical disabilities.

By Thomas Meyer
Journalist, Business Analyst