Introduction

In this project, I produced a "morph" animation of my face into my roommate's, computed the mean of the faces of a population of Danish computer scientists, and extrapolated from this population mean to create a caricature of myself. So, if you keep scrolling you're guaranteed to see some cool stuff :)

Defining Correspondences

Before beginning to think of how to between two images, one must define pairs of corresponding points on the two images by hand (yes, this can unfortunately get quite tedious). After defining pairs of corresponding points between the two images, comes the "connecting" of the dots (a.k.a. drawing a bunch of triangles)!

There is a handy algorithm called the Delaunay triangulation which takes a bunch of coordinates and then produces a triangulation as seen above. However, it is important to note that one must use the same set of points to compute the triangulation for both images. This is because for the later steps, it must be ensured that the triangles in both images are comprised using the same numbered points (e.g. if the Delaunay triangulation returns a triangle with points (1, 5, 3), then in both images, the 1st, 5th, and 3rd points should form one triangle).

Thus, a weighted sum of two sets of points is passed into the Delaunay triangulation algorithm to produce the above ouput (in this case, alpha=0.5 which is just taking the average): $$\texttt{tri} = \texttt{Delaunay}(\texttt{mid_pts})\ \ \texttt{ where } \ \ \texttt{mid_pts} = (1-\alpha)\cdot\texttt{im1_pts} + \alpha\cdot\texttt{im2_pts}$$

Computing the "Mid-way Face"

Just to get a feel of how everything works, before computing the entire morph sequence, here I first compute the mid-way face. This is the image that should occur exactly halfway through the morph sequence.

  1. Compute the mid_pts and triangles using the above formula with alpha=0.5
  2. For every triangle, we fine the affine transformation matrix, A, between the triangle coordinates of each image to the triangle coordinates of the mid_pts.
  3. Say, the triangle of im1 is composed of the points (a, b, c) and the triangle of mid_pts is composed of the points (d, e, f), then we find A as follows: $$A = \begin{pmatrix} d_x & e_x & f_x \\ d_y & e_y & f_y \\ 1 & 1 & 1 \end{pmatrix} \begin{pmatrix} a_x & b_x & c_x \\ a_y & b_y & c_y \\ 1 & 1 & 1 \end{pmatrix}^{-1}$$
  4. Apply the inverse transformation (for im1 and im2) on points within the triangle in the mid_pts to find what pixel to fill in for the mid-way image
  5. Weight the pixel values by alpha when combining from the two images (alpha=0.5)
By executing the above steps, I was able to generate the mid-way face of me and my roommate (which was also seen at the very top of this page):

The Morph Sequence

Now to piece all of it together! The basic idea here is to just keep repeating the steps detailed above for computing the mid-way face. The only difference this time, is we do it for several different alpha values! For this assignment I produced 45 frames of animation. In other words, I repeated the above steps 45 times with 45 different alpha values evenly spaced on the range [0, 1]. I then stitched the 45 output images into the following .gif file:

The "Mean face" of a population

Now, using similar methods as previously mentioned, I aim to find the "mean" face of a population. I decide to use a dataset of male, Danish computer scientists. To compute the mean, or average, face of all these images, I basically do the same thing as I did in computing the mid-way face! The only difference now, is that I am transforming more than just two images to the mean points.

mean face of male Danish computer scientists

Now let's look at some images morphed into the shape of the mean face!

11-1m.jpg warped to mean

13-1m.jpg warped to mean

40-1m.jpg warped to mean
me warped to mean
mean warped to me

Caricatures: Extrapolating from the mean

Now, to take it one step further... instead of restricting ourselves to alpha values in [0,1], by taking values outside this range we can extrapolate (before we were interpolating) from the mean face found in the previous section. Here is the output that I was able to generate with alpha=-1.5 (please pardon the discoloration):

Bells and Whistles!

This part was particularly fun. I joined up with a bunch of students in the class and we were able to chain a bunch of morphs together to produce this marvelous gem: