Teaser Image

Project 1: Aligning and compositing the images of the Prokudin-Gorskii photo collection

You are allowed to work in groups of 1 or 2 or 3 for this project.

Useful Links

Background

Sergei Mikhailovich Prokudin-Gorskii (1863-1944) was a photographer ahead of his time. He saw color photography as the wave of the future and came up with a simple idea to produce color photos: record three exposures of every scene onto a glass plate using a red, a green, and a blue filter and then project the monochrome pictures with correctly coloured light to reproduce the color image; color printing of photos was very difficult at the time. Due to the fame he got from his color photos, including the only color portrait of Leo Tolstoy (a famous Russian author), he won the Tzar's permission and funding to travel across the Russian Empire and document it in 'color' photographs. His RGB glass plate negatives were purchased in 1948 by the Library of Congress. They are now digitized and available on-line.

Requirements

Take the digitized Prokudin-Gorskii glass plate images and automatically produce a color image with as few visual artifacts as possible. You will need to extract the three color channel images and align them so that they form a single RGB color image. The high-resolution images are quite large so you will need to have a fast and efficient aligning algorithm. You are required to implement a single-scale and multi-scale aligning algorithm that searches over a user-specified window of displacements. Also, you are required to try your algorithm on other images from the Prokudin-Gorskii collection.

Details

example negative

Important notes about the images:

MATLAB stencil code is available, as is python stencil code. You're free to do this project in whatever language you want.

Example digitized glass plate images, both hi-res and low-res versions are available in this zip file

Your program will take a glass plate image as input and produce a single color image as output. The program should divide the image into three equal parts and align the second and the third parts (G and R) to the first (B). For each image, you will need to record the displacement vector that was used to align the parts. Don't get your coordinate order mixed up -- Matlab matrices are accessed (y, x).

The easiest way to align the parts is to exhaustively search over a window of possible displacements (e.g. [-15,15] pixels), score each one using some image matching metric, and take the displacement with the best score. There are several possible metrics to measure how well images match:

Note that in this particular case, the images to be matched do not actually have the same brightness values (they are different color channels), so other metrics might work better.

Exhaustive search will become prohibitively expensive if the displacement search range or image resolution are too large (which will be the case for high-resolution glass plate scans). To avoid this, you will need to implement a better strategy. Some options:

  1. a coarse-to-fine search strategy using an image pyramid. This means shrinking your image (for example, you might shrink your original 3000 x 3000 pixel image to be 300 x 300 pixels), where you can solve the problem quickly because the image is small, and also because you may only have to slide the R,G,B channel a few pixels relative to each other. Once you have the answer for the 300 x 300 pixel version of the image, you can use that displacement estimate to search only the most likely displacements in the 600 x 600 pixel image, and so on, until you are optimizing on the original image but only have to search a little bit.
  2. There are other possible approaches to making a search faster. Can you think of any and test them?

Bells and Whistles

Although the color images resulting from this automatic procedure will often look strikingly real, they are still not nearly as good as the manually restored versions available on the LoC website and from other professional photographers. However, each photograph takes days of painstaking Photoshop work, adjusting the color levels, removing the blemishes, adding contrast, etc. Can you come up with ways to address these problems automatically? Feel free to come up with your own approaches or talk to the Professor or TAs about your ideas. There is no right answer here, just try out things and see what works.

Here are some ideas, but we will give credit for other clever ideas:

For all extra credit, be sure to demonstrate on your web page cases where your extra credit has improved image quality.

Web-Publishing Results

All the results for each project will be put on the course website so that the students can see each other's results. In class we will have presentations of the projects and the students will vote on who got the best results. If you do not want your results published to the web, you can choose to opt out. If you want to opt out, email the class TA saying so.

Write up

For this project, and all other projects, you must do a project writeup that will be shared as a webpage; either on the gwu blogging service (or any other blogging service that you like). In the report you will describe your algorithm and any decisions you made to write your algorithm a particular way. Show and discuss the results of your algorithm, including, if possible, examples where your algorithm works, where (and why!) it fails. Also discuss any bells and whistles that you did. Feel free to add any other information you feel is relevant. How much should you write about your project? The following are blog posts that are about the level of detail that I hope to see:

  1. Write up about edge detection
  2. write up about how to use an image warping function
  3. A write-up of how to transform between different color-spaces
These are not class projects so the format isn't exactly, but they should a little bit about how to use images to talk about image based algorithms and show the size and scope of the write-up I'm hoping for.

Rubric

Final Advice

Credits

Project derived from Alexei A. Efros' Computational Photography course, with permission.

Collaboration Policy

All work that you turn in must be your work, your code. But it is also incredibly useful to talk with classmates and explore. So I try to share here some explicit guidance. You are explictly ALLOWED to:
  1. Search the internet for answers to how to read in images, write out images, etc.

Good Luck!