Analysis of “Face Sketch Synthesis Algorithm Based on E-HMM and Selective Ensemble”
Summary:
Has a large collection of photos and sketches of those photos. To map the nonlinear relationship between photos and sketches, several models are generated by Embedded Hidden Markov Models (E-HMMs) which each produce a pseudo-sketch. The author then uses a strategy he calls “selective ensemble” to produce a finer pseudo-sketch from the others.
The E-HMMs have two states: super-states that represent the vertical macro-features (forehead, eyes, nose, etc.) and then these have embedded states that describe the local features.
Discussed earlier work that mapped the nonlinear relationship between photo and sketch by using patches. This involves divvying up the photos and sketches into small overlapping patches and, for each patch, finding the neighbors that similar to it, calculating a “reconstruction weight” for each neighbor, and then using these weights to sketch that patch. A pseudo-sketch is a combination of all the patches. However, there was a fine dance between the sizes of the patches and how much they overlapped that one would have to deal with to get details while avoiding artifacts.
Also discussed an approach using E-HMMs and Viterbi decoding that did not work.
Discussion:
It seems like a very solid technique, but I still don’t have a corpus of images. Not sure I could digest this paper anyway. The patch approach made more sense to me. He did mention a non-example-based face sketch synthesis approach that I’m looking into.
Is this similar to the technique that was first initiated at the Olympic Games in Atlanta, GA? Some of the facial recognition software used in the modern digital cameras allow for not only identification of facial structures, but movement (i.e. smile to snap). Refinement in this technology could be especially useful for those in the legal profession. It will be good to hear more about what is being done in this area.