Researchers at MIT have developed a camera which could be integrated into a TV set and, based on a face recognition system, measure the heart rate of a viewer who is looking at the screen.

TVs Soon Using Face Recognition to Measure Viewers’ Heart Rate?


Televisions of the future will not be there just for watching. They will be watching back, keeping an eye on the viewers…for their own good! Researchers at MIT's Computer Science and Artificial Intelligence Laboratory have developed a camera which can measure a person’s heart rate by observing movements of the face, and so check on his/her cardiac health. The algorithm, which is based on various techniques from the field of computer vision, could be adapted for use with advanced television sets and computers, explains, Ming-Zher Poh, one of the researchers.

System undergoing improvement

The procedure uses a facial recognition system which examines the face of the person whose heart rate is to be assessed. The system directs the camera to select 500 to 1,000 points around the mouth and nose in order to analyse head movement associated with blood flow. The researcher in charge of the project, Guha Balakrishnan, an MIT graduate student in the Electrical Engineering & Computer Science department, explains that the camera “avoids the eyes, because blinking is not to be taken into account.” Other non-significant movements such as those arising from respiration or changes in posture are also discounted so as to isolate those specific head movements that are associated with the pumping action of the heart. Although a lot of progress has been made – for instance Ming-Zher Poh explains that the researchers have managed to suppress the loud sound signal which was being produced when they first began the work – Guha Balakrishnan puts this progress in context, pointing out that the technique cannot yet be used with all camera-equipped devices.

Applications across various devices

Of course the idea is that the system can be used in the eHealth field, but Microsoft’s Kinect 1 has already engendered a number of applications in the medical sector. It is however likely that this approach will be used by the video game industry, but in a much less precise way and with other purposes in mind. This is in fact already the case with the Kinect 2, whose heart rate tracking capability is likely soon to be used to adjust the difficulty level of a game to gamers’ stress levels, rather than for genuine medical purposes. In fact the camera-based heart rate tracking concept already exists in the form of a smartphone app called Cardiio. However, while the app has proved very popular, Cardiio is first and foremost a tool to enable people who take physical exercise to obtain an estimate of the intensity of their efforts; the results it achieves are not accurate enough to be used for serious medical purposes. 


By Timothée Sicot
Chargé de veille