Modelling three-dimensional objects on a computer screen using only your eyes may seem rather improbable but that is just what a team of US university researchers is doing. Their goal is to simplify the creation of objects which can then be printed in 3D.
Inventors and designers can now use eye-tracking to create structures on a computer screen using just their eyes. Eye-tracking technology has been developed first and foremost to allow instructions to be given to a computer or a robot. Recently we reported on Tobii, a firm specialising in eye-tracking, which is working on a process that can be used by the general public. Now a team of four researchers from Cornell University, New York and the University of Wyoming in the United States have just published a paper* on their eye-tracking project, the aim of which is to render the design task hands-free. The process they present could replace the traditional keyboard and computer mouse when designing three-dimensional objects. The process is geared to the design of objects to be manufactured using 3D printers without the user having to wear special glasses. However it also works for general object design and it has the potential to add value to creative brainstorming meetings.
Using neural networks to drive 3D design
The eye tracker operates by shining infrared light into the eye to create reflections that cause the pupil to appear as a bright, well-defined disc in the eye tracker camera. In order to respond to the user’s wishes their model uses evolutionary algorithms which mirror those of human beings in their design. The researchers use the CPPN-NEAT protocol for the encoding and development of the design of an object. CPPN (Compositional Pattern-Producing Network) is a mathematical model similar to a human neural network, and it uses several types of mathematical functions, enabling the encoding of designs in the same way as nature encodes its designs across our planet. CPPNs evolve according to the NEAT (Neuro-Evolution of Augmenting Topologies) algorithm, which is a genetic algorithm for the generation of evolving artificial neural networks.
Technique still requires refinement
The researchers’ work on interactive evolution through eye-tracking enables geometric forms to be altered on the basis of their position on the screen. Each eye movement is registered at each iteration of the algorithm and allows interactive evolution of the shape according to where the user’s look falls on the computer screen. The system is still at the project stage, but the researchers are already working on various scenarios. Other questions still need thinking through, such as the impact of a user’s attitude to the technology and the jerkiness caused by short fast eye movements around a fixed point. These drawbacks affecting the practical application of the technique have so far not been fully resolved.
* ‘Hands-free Evolution of 3D-printable Objects via Eye Tracking’, by Nick Cheney, Jeff Clune, Jason Yosinski and Hod Lipson, published 19 April 2013