(PA) Movies Part 2
Enhance Movies Part 1 by adding basic recommendation logic

IMPORTANT NOTE

NB It is important to read and heed this advice!

This will take several hours, so please don’t spend too much time trying to get a super accurate prediction algorithm. If this is a topic you are interested in, tthere are several published papers as wellas Brandeis courses on this subject. However, the goal for this assignment is to give you practice writing good Ruby code. Surprisingly it seems like returning a constant prediction of 4 does quite well compared to other more complicated algorithms. Interesting. Therefore, first complete this PA2 with a really simple predictor and get everything done. Then go back and refine the algorithm, but only if you don’t have more important projects to work on! This is a really fun kind of project to work on, especially once you start getting the hang of writing ruby code to implement sophisticated algorithms. We’ll discuss some very simple algorithms in class.

Introduction

Building on (PA) Movies Part 1, we are now going to include a prediction method that will predict the rating a user would give to a movie and you are to run some tests to determine how accurate your method is.

Take a look at the data files. In addition to the file u.data you will find a series of pairs of files, u?.base and u?.test. They have the exact same format as u.data, but partitioned into batches. For example, u1.test contains 2000 ratings and u1.base contains the other 8000 ratings. The same is true for all the other u?, except they contain different subsets.

What you will be designing for movies_2 is a very simplistic machine learning algorithm. The goal is to correctly guess what a certain user would rate a certain movie. Often in machine learning (and actually in all kinds of forecasting and modeling) we partition historical data (that is known to be real) into two batches. In our case we use the first batch to learn what we can about the patterns in users ratings, and create a prediction “model”. This is the training set of for us, the “base”.

Then we will use the second batch (the “test” set) to see how good our prediction model is. We can take each record in the test set, where we will find how user u rated movie m. We use our model to predict (guess) what the rating was, and we look at the real world result from the test set record to see what that user actually did. The difference between the two is a measure of how good our model is.

If we do this process across all the records in the test set, we can find an average and standard deviation (and whatever other statistics) of the error. This is what we are going to do.

Class design

I propose the following class design.

  • Class Ratings: contains one of the files of ratings, u.data, or u1.test or u1.base etc. Knows how to read the file while analyzing what it sees. Importantly it has a method predict(user, movie) which will generate a prediction, based on that file, of what rating a user would give to a movie.

  • Class Validator: Takes two instances of Ratings, one representing a base set and the other representing a test set. Has one important method, validate, which runs through all the entries in the test set and see what ratings would be predicted vs. which ones were given. Compute statistics about the prediction: how many predictions were exact, off by one, etc. Also the mean and stdev of the difference between the two.

  • Class Control: Has one key method, run, which instantiates and invokes the other two classes. It’s the top level of your program.

Final work product

  1. Install the reek gem and see if there are ways to improve your code.

  2. Do a systematic set of runs experimenting with the different u? files, and if you have more than one prediction algorithm, try those.

  3. Zip together your code plus a readme.md file containing a summary of your work. Include the following information about your solution. Submit the zip file to Latte.

    • The Algorithm. A description of your prediction algorithm and what you think are its advantages and drawbacks.
    • The Analysis. A description of the results of running some experiments to determine the accuracy of your method.
    • Benchmarking. An estimate of the time it takes run each prediction (determined by using Time.now to capture the current clock time before and after making k predictions) and an explanation of what you think will happen to that time if you increase the size of the training set by a factor of 10 or 100….
    • Reflection. What did you enjoy, what did you struggle with, did you think this assignment met its goal of giving you a real feel for programming Ruby? How would you improve or change it?