Beckman Institute Works to Bring new Dimension to Facial Recognition
Or imagine a similar computer program helping a police department detect probable lies by reading facial expressions during an interrogation.[an error occurred while processing this directive]These scenarios could be made possible by the work of researchers at the Beckman Institute for Advanced Science and Technology at the University of Illinois at Urbana-Champaign. They are using 3D scanning technology for two related projects that promise to increase a computer’s ability to recognize faces and react to facial expressions.
“Our two main projects require scans of real faces, with the variability that you find in the general population,” says Jesse Spencer-Smith, Beckman Fellow and principal investigator for both projects.
Creating greater face value
The Institute’s first project involves creating a large, multiracial database of scanned faces that will be used to develop face recognition programs. Face recognition has many interactive applications, from access control and credit card identification to law enforcement.
Most attempts at developing automatic face recognition systems involve the simple coding of video images of faces and their various features, which can then be “learned” by the computer. Such tactics don’t typically account for varying circumstances such as lighting conditions and positioning, however, which can easily stump a computer program.
Researchers at the Institute wanted scans in three dimensions to make them recognizable at different orientations by not just man, but also machine. They turned to Eyetronics (www.eyetronics.com) for 3D scanning technology used in feature films, television series, and character-intensive 3D games.
“Our looking to technology used in Hollywood is nothing new,” says Ben Grosser, director of the Imaging Technology Group. The Group acts as technical facilitator for researchers at the Institute and University, finding commercial technology, figuring out how to use it, and working with developers to apply it to a particular project. “We have often looked to the effects industry for potential new techniques in research.”
Beckman’s researchers take 10 photos of each subject’s face with Eyetronics’ ShapeCam, a portable scanning device based on off-the-shelf camera hardware. ShapeCam captures geometry and textures by taking a photo with two rapid flashes: the first captures geometry information by projecting a grid onto the subject through a fixed-focus lens. The second captures high-resolution textures with a simple picture.
“The fixed-focus lens allows us to use most of the 300 by 300 grid on the face,” Spencer-Smith says. The fine grid captures detailed facial geometry that is used to process the digital model.
Each subject is photographed both with and without a dome cap, which allows researchers to capture the general shape of the head and to recreate hair as a texture map.
“We get a complete scan using multiple views in half an hour,” Spencer-Smith says. “We can then use Eyetronics software to generate new faces by blending faces within the database.”
Data collected from the scans is processed using Eyetronics’ proprietary ShapeSnatcher software, which turns the 2D grid information into 3D models. ShapeSnatcher enables researchers to automatically stitch together the different 3D perspectives, blend them into a smooth model, and apply the face photos as high-resolution textures.
Completed models are loaded into Maya 3D animation software and mapped to an average face model in the Matlab computing architecture. This allows researchers to create new faces by combining the structure and texture maps of two or more faces. Rhino 3D modeling software and Curious Labs' Poser software are used to make blended models appear more realistic.
In the first year alone, the Institute expects to have a database of 400 faces, which can eventually be used to test recognition programs and contribute to the development of new generations of programs.
Capturing real expressions in 3D
A second related project at Beckman investigates dynamic emotional expressions. A computer’s ability to read a person’s facial expressions could improve the efficiency of applications ranging from computer-based tutorials to lie detection.
Research contributing to facial-expression recognition programs often involves prototypical expressions (happy, sad, fear, anger) to develop templates, breaking down facial expressions according to muscle movements. The expressions are typically studied using 2D photographs or video images that don’t always paint a complete picture.
“Using dynamic scanning, we are able to capture real, three-dimensional expressions,” Spencer-Smith says. “It allows researchers to examine the fine structural changes as well as the precise time course for universal and complex expressions.”
Dynamic scanning involves capturing the same physical image or set of images in rapid succession. Just as motion pictures show more than just a moment, dynamic scanning can reveal time-dependent changes that are often lost with static images.
Eyetronics’ 3D dynamic scanning system records facial expressions with a digital video camera and a fine grid that is projected on the face by a standard slide projector. The system’s software then processes the images frame by frame, creating a 3D model from the deformation of lines on the grid. The fixed-focus lens is again used to give researchers good depth-of-field. This allows the subject to move a bit without going out of focus.
The Eyetronics approach enables more attributes to be incorporated into templates for any given prototypical expression because the 3D image more closely matches a human face. In addition, the software’s mobility and inexpensive setup make it quick and easy for researchers to arrange capture sessions.
Spencer-Smith’s team uses the scans to evaluate and interpret expressions in detail, recording characteristics that go beyond the standard muscle movements already identified. They analyze the faces using an algorithm that determines eigenvector decomposition -- defined as the breakdown of a matrix into a product of three other matrices -- as well as a similar statistical technique developed at the Institute.
While primarily qualitative in nature for now, what’s learned from the analyses could eventually be translated into software code that extends the programs currently being developed for face recognition.
Getting underneath the skin
Facial databases and dynamic expression capture have the potential to spur a new realm of computer applications.
The facial database, for example, can provide a resource for psychologists to study how humans recognize known faces so reliably and quickly. Or conversely, how people make mistakes such as those frequently found in eyewitness testimonies.
Dynamic scanning of expressions can help researchers understand the rich signals present in dynamic expressions, and to study how expressions differ between cultures.
“Imagine a system that could help you get across a point you want to make in Russian,” Spencer-Smith says. “To express a sincere smile, you might need to smile less broadly and more gradually. An advanced facial recognition system could help improve communication and understanding across diverse cultures.”
Jill R. Aitoro (jilla@cramco.com) is a freelance writer specializing in computer graphics and other technology topics. She works at Cramblitt & Company in Cary, N.C.
Source: Eyetronics
[an error occurred while processing this directive]