Project2: CNNs and Transfer Learning (due 04/05 11:59pm)

In this project, you will build a classifer for detecting different breeds of dogs. The dataset we will be using has a number of interesting properties:

There are only about ~150 images per class (breed of dog). This is relatively small, given that:
Some dog breeds look very similar to one another, and therefore are difficult to distinguish
The images are of dogs in different poses and natural backgrounds, making classification potentially more difficult

Let's get started!

Part 1: Dataset download and extraction

This dataset comes from the Stanford Dogs Dataset, where you will find a description of the dataset. Use the link at the right to download the images.

As you can see from the accuracy graph on that webpage, building a classifier from so few images per class, especially when classes can look similar to one another, is very difficult. To make our lives easier, and to reduce training time, we're going to focus on just three classes from their datase:

n02109047-Great_Dane
n02094114-Norfolk_terrier
n02094258-Norwich_terrier

Once you've finished downloading the dataset, create a folder that has only the three subfolders of images above; you can delete all the other images.

Complete the following items. Note that we're not going to do CV here in the interest of compute time, but you should generally keep in mind its benefits in terms of training and measuring performance.

Also, feel free to train on just 30-50 images if you're using a CPU for 2-3 epochs -- otherwise it may take too long.

Please copy each line of the grading rubric (including number) into a markdown element in your jupyter notebook that matches the cell that completes it, so we don't miss anything during grading :-)

GRADING RUBRIC for Part 2:

1. Create train and holdout folders, where each folder has the three classes as subfolders. Discuss in a markdown cell how many images you chose to have in each class' holdout folder. (Note: you can move images into the folders by hand if you don't want to write a script, but make sure you don't have the same images in train and holdout). Make sure all the class names are the same across folders. Now repeat the process for train and validation, so that you have a train, validation, and holdout folders at the same directory level. Discuss how/why you chose what's in your holdout.	3 points
2. Define a list of image transformations to be used during training, passing them to `transforms.Compose()`. Discuss why you think these transformations might help.	3 points
3. Repeat the step above for test and validation transformations.	3 points
4. Correctly set up `DataLoader`s for the three folders (train, validation, holdout). Discuss what options you chose for these loaders, and why (including batch size, shuffling, and dropping last).	3 points

Part 3: Data model setup

Complete the following items: GRADING RUBRIC for Part 3:

5. Instantiate any pre-trained ImageNet model. Discuss why you chose it amongst the others.	3 points
6. Write code to freeze/unfreeze the pretrained model layers.	3 points
7. Replace the head of the model with sequential layer(s) to predict our three classes.	3 points
8. What activation function did you use in the step above? Why?	3 points
9. Did you use dropout in the step above? Why or why not?	3 points
10. Did you use batch normalization in the step above? Why or why not?	3 points
11. Choose and instantiate an optimizer. Discuss your choice.	3 points
12. Choose and instantiate a loss function. Discuss your choice.	3 points
13. Write code that places the model on the GPU, if it exists, otherwise using the CPU.	3 points

Part 4: Training your model

Complete the following items: GRADING RUBRIC for Part 4:

14. Correctly set up your model to train over 20+ epochs (if GPU) or 2+ epochs (if CPU).	3 points
15. Correctly set up your model to use your batches for training.	3 points
16. Correctly make predictions with your model (the predictions can be wrong).	3 points
17. Correctly call your loss function and back-propagate its results.	3 points
18. Use the optimizer correctly to update weights/gradients.	3 points
19. Correctly record training losses for each epoch.	3 points
20. Correctly set up validation at each epoch.	3 points
21. Correctly record validation losses for each epoch.	3 points
22. Correctly record training and validation accuracies for each epoch	3 points
23. Write code that places the model on the GPU, if it exists, otherwise using the CPU.	3 points

Part 5: Model evaluation

Complete the following items: GRADING RUBRIC for Part 5:

24. Graph training versus validation loss using `matplotlib.pyplot` (or other). Was your model overfitting, underfitting, or neither?	3 points
25. Make a list of reasons why your model may have under-performed.	3 points
26. Make a list of ways you could improve your model performance (you don't have to implement these unless you wan to).	3 points
27. Graph training versus validation accuracy using `matplotlib.pyplot` (or other). Score your model on its predictions on the holdout. Discuss why you think your results will or will not generalize.	3 points

All done! Where to go next?

Great work on this project! If you're interested in learning more about how researchers created this dataset, check out their paper.

Extra credit:

Discuss the paper in one paragraph.