INFO 290T Final Project

For your final project, you may work individually or as a group of no more than two students (group projects will be expected to produce higher-quality work than individual projects). You will build a custom image classifier using the tools you have learned in this class. The type of classification problem you solve will be up to you.

First, you will need to find an image dataset online. There are many popular datasets used for classification across many domains, including medicine, architecture, agriculture, satellite imagery, etc. Your dataset should be of at least moderate difficulty (more complicated than MNIST, for example), and should include any labels needed to answer your intended classification query (supervised learning only).

You will most likely need to perform a variety of pre-processing steps on your images to ensure that they are all the right size, resolution, number of channels, appropriate contrast, etc.

You will turn in a one-page proposal including information about your dataset and the intended classification question you hope to answer, along with a few other details described in the rubric below. Your proposal must be submitted for approval no later than Nov. 3rd. Next, you will design custom feature vectors using the filtering and image decomposition methods we covered in class. You must implement at least two non-learned features (e.g., multi-scale histogram of oriented gradients, as one example). Choose feature vectors that are well-suited to the type of images and classification question you are working on.

Finally, you will build and train at least two classifiers using two different classification methods that we have learned (for example, general least squares and support vector machines). You should use classification methods that are appropriate for your dataset and query. Consider working with a smaller subset of the data to test out your ideas for classifier design. During this step, you should divide your dataset into training, validation, and testing groups. For full credit, implement a hyperparameter search using subsets of the validation data. Avoid overfitting.

The final deliverable will be a notebook that includes well-commented code interspersed with written explanation describing your process and figures showing the results for each of these steps. We recommend that you start working on the project as early as possible.

Examples

1. distinguish between different types of pistachios
2. geo-locate based on landmarks
3. interpret American sign language
4. distinguish between different types of fruits and vegetables
5. from aerial imagery, classify vehicles

Rubric (60 pts)

Proposal (5 pts): Provide a link to the dataset, example image(s), a description of the variation in the dataset (e.g., categories, size/resolution, etc), a description of the intended classification problem (i.e., list of output categories), estimation of the approximate number of images expected to be in each category, and a guess of the types of image features that may be useful for this categorization (e.g., edges, histograms, etc.).

Feature Extraction (15 pts): This part should include the code to extract features, illustrations of the features extracted from example images, plots of the amount of variation in the dataset, as well as PCA decomposition and tSNE visualization of features. You must include at least two different features.

Classification (20 pts): This part should include code to perform classification using at least two methods learned in class, plots showing the results of classification per category, a discussion of possible reasons why the classifier might work better for some categories than others, and explanation of the limitations of the classifier.

Generalizability (10 pts): Your data should be split into train, validation, and test groups before training the classifier, and you should do a hyperparameter search using parts of the validation set, ideally in a way that avoids over-fitting and maximizes generalizability. Lastly, you will report performance on the test set, and include a discussion of whether you achieved generalizability and how your training process might be improved.

Quality of Explanation (10 pts): Overall quality of report, including readability of figures and code comments, quality of analysis, and discussion of limitations.

Contributions: If you are working as a team, you should provide a short description of which team member contributed to (and by how much) each part of the project (e.g., design, data collection, coding, data analysis, and writing).