Weekly

Due 1/22

(20 Points) Assignment 1: Exploratory Analysis
Perform exploratory analysis on the UCI MPG dataset. Generate reasonable statistics, search for outliers, and plots for each feature of the dataset. Use scatter plots with regression lines where reasonable. Make histograms for discrete data. Compare vehicles from each origin, are there any noticeable differences between the statistics for each origin? Which features are most associated with a good MPG?

Write a short report, a couple of paragraphs plus figures, to answer the above questions. Turn in your report on canvas and your code on Gitlab.
Submit your write-up to Canvas.

Due 1/29

(40 Points) Assignment 2: Regression
Apply linear regression to the data set from assignment 1 to predict MPG. Besides loading the data and fitting the model, make sure to do the following: feature engineering, cross validation, and error diagnostic plots.

Write a short report on your results. What features did you engineer? Did these engineered features help? Report MAE and MAPE for both the training and testing set, along with the size of train-test split. Include diagnostic plots as well. Speculate what important features are missing from the data.

Submit your report on Canvas and commit your code to GitLab.

Due 2/5

(40 Points) Assignment 3: Classification
Model people's salaries in the Census Income dataset with Logistic Regression. Predict individuals' income based on the other 14 features. Ignore rows with missing data. Be sure to do feature engineering, cross validation, and report precision, recall, and F1. Also include a clearly labeled confusion matrix in your report. Try removing features from the data and reevaluate the model. Does the removal of any features have no impact on the accuracy? Did removing a feature improve accuracy?

Submit your report on Canvas and commit your code to GitLab.

Due 2/19

(80 Points) Assignment 4: Gradient Descent for Logistic Regression
Implement the logistic regression model for Bernoulli (binary) predictions. Write a procedure to fit the model with stochastic gradient descent. Use the entropy loss function/gradient function in your implementation.

Use cross validation (a training/testing split) and fit the model to this dataset. Report the F1 scrores for both the training and testing splits in the README.md file.

Commit your code to GitLab.

Due 2/26

(60 Points) Assignment 5: Neural Networks

Apply a feed-forward neural network (MLP) to all the previous datasets:

• Census Income
• Banking dataset
• MPG

Due 3/10

(120 Points) Assignment 6: Decision Tree Implementation
Implement the decision tree model for classification and a procedure to fit the model based on Information Gain (entropy). Use the recursive partition training procedure to fit the tree model to data. Include natural stopping criteria of 100 percent homogeneity as well as hyperparameter to limit the maximum depth. You may include additional criteria like minimum data point also.

Second, implement one of the following ensemble methods: bagging, random forest, or ada boost.

Use cross validation (70-30 split) and fit a tree as well as an ensemble of trees to the Iris, MPG, the Census Income, and Bank datasets. Report precision, recall, & F1 for all classes per dataset in your report along with your hyperparameter choices.

Be sure to check the quality of your code with the Code Checker.

Submit your report on Canvas and commit your code to GitLab.

Due 3/19

(80 Points) Assignment 7: Ensembles
Fit the following models to the Iris, MPG, Bank, and Census Income datasets:

• Random Forests
• Adaboost + decision stumps
• Bagging + Neural Network
• Some combination of your choice!

Due 3/17

(10 Project Points) Pick a Project
For the final project for this course, you will do a project of your choice. However your project must fall into one of the following categories:

• Implement a model not previously covered by the homework e.g. a KD-Tree
• An interesting application of a machine learning model e.g. a spam filter
• Use a high-level tool set such Keras, Tensorflow, or Pytorch to implement a more complex model such as a GAN.

Due 4/2

(30 Project Points) Project Milestone
For this milestone you will be presenting your plan for your project as well as providing a description of the data you will be using. You will write up how you will take the problem you are working on and frame it as a machine learning task, precisely specifying how to represent the data (as vectors), and what the model will be predicting i.e. classification or regression.
You will also need to provide descriptive statistics about your data and some plots of various features of the data if it is reasonable.

You will present your milestone in class, plan for a 3 to 5 minute presentation. Present on:

1. The goal of your project
2. The dataset you are using
3. What kinds of models you plan on using or implementing

Due 4/21

(100 Project Points) Final Project
For your final project, you will need to write a short paper, about three pages, to describe what you did for your final project. Introduce the problem or the method you selected, describe how you approached or solved the problem, finally present the results e.g. accuracy etc.

Also, you will give a 12 minute presentation during finals week on the results of your project.

Submit your code for your final to your GitLab repository.

Submit your paper to Canvas.