Facial Recognition

Facial recognition only uses the center of the face.

Facial recognition only uses the center of the face.

Face Finding

When a computer looks at a scene to match faces, it first has to find the faces themselves since it doesn't see the scene as individual objects the way we do. This process is called face detection and this is also used with a modern cell phone when a small square box is drawn around a face on the phone camera screen. Face detection is a mature area and it can be done extremely fast. The basic process is that a computer algorithm is trained in things that are faces and things that are not faces and it learns to distinguish between them.

Face Matching

Once faces are found, the computer turns them into templates - mathematical representations of the face itself. A template is typically much smaller than the original image - anywhere from 1KB to 20KB; it just contains the key information needed to match the face. Interestingly, the region that the computer looks at is much smaller than the region humans look at when recognizing faces. The main reason for this is that this center region doesn't change compared to things like hair and beards. Face matching is also accomplished in gray scale, so the color information is gone by the time the matching starts. Another interesting thing is that a photograph can be turned into a template which will reliably match the same person, but a template cannot be turned back into an image.

For high-security applications, face recognition is rarely used by itself. Most systems suffer from a high failure-to-acquire rate (the sensors don’t capture usable images) because of lighting variations and the many different ways people can hold their head. And when a match is made, it is not as certain as a fingerprint or iris match primarily because of the "fuzziness" of the information used to match compared with those two modalities. Face matches are typically going to have a higher error rate than other modalities because of difficulties in the presentation.