Google's FaceNet, which has been described as having almost perfected the recognition of human faces, draws heavily upon work by Kilian Weinberger and, more generally, research by Thorsten Joachims. Associate Professor Kilian Weinberger was recently hired by Cornell and will move his lab to Ithaca this summer.
The system involves an adaptation of Weinberger's Large-Margin Nearest-Neighbor (LMNN) classifier to deep neural networks. It learns a new representation of facial images, in which two images of the same person are close together and two images of different people are far apart. This learning is facilitated through image triplets. Repeatedly (millions of times) two images of the same person are pulled together while simultaneously an image of a different person is pushed away. Joachims pioneered the use of such triples for learning.
The lead author of the Google CVPR 2015 paper on FaceNet, Florian Schroff, was formerly a postdoc with Serge Belongie.
Image below: Illustration of what LMNN does, represented as a cake made by Cornell CS alum John Blitzer ('02). the green dots represent faces of the same individual, which all move closer together. The blue and red dots represent faces of different persons, which move away from the green dots.
