Project - Recommendation Systems: Movie Recommendations

Marks: 40


Context


Online streaming platforms like Netflix have plenty of movies in their repositories and if we can build a recommendation system to recommend relevant movies to users based on their historical interactions, this would improve customer satisfaction and hence improve revenue. The techniques that we will learn here will not only be limited to movies, it can be any item for which you want to build a recommendation system. For this case study, you can find the dataset here.


Objective


In this project we will be building various recommendation systems:

based on the ratings dataset


Dataset


The ratings dataset contains the following attributes:

Importing Libraries

Loading the data

Let's check the info of the data

Question 1: Exploring the dataset (7 Marks)

Q 1.1 Print the top 5 rows of the dataset. (1 Mark)

Q 1.2 Describe the distribution of ratings. (1 Mark)

This plot shows the distribution of all of the different ratings. This plot indicates that the ratings of 3 and greater are more common in the dataset with a rating of 4 being the most common rating.

Q 1.3 What is the total number of unique users and unique movies? (1 Mark)

There are 671 total unique users.

There are 9066 total unique movies.

Q 1.4 Is there any movie that has been interacted with more than once by the same user? (1 Mark)

No, there are no movies that have been interacted with more than once by the same user since the number of ratings by grouped by userId is equal to 1 for each movieId, which is proven by the sum equaling to 100004, which is the total amount of ratings in the data set.

Q 1.5 Which is the most interacted movie in the dataset? (1 Mark)

The movie that is most interacted with is the movie with movieId 356.

This plot shows the distribution of all interactions with movieid 356.

Q 1.6 Which user interacted the most with any movie in the dataset? (1 Mark)

User 547 has interacted the most with any movie in the dataset.

Q 1.7 What is the distribution of the user-movie interactions in this dataset? (1 Mark)

This plot depicts the distribution of the number of interactions by user. For interpretation, this plot essentially signifies that most users interact with movies around 0-50 times approximately. This means that the number of interactions a user has with a movie is most likely going to be less than 500 ( or even 100).

As we have now explored the data, let's start building Recommendation systems

Question 2: Create Rank-Based Recommendation System (3 Marks)

Model 1: Rank-Based Recommendation System

Rank-based recommendation systems provide recommendations based on the most popular items. This kind of recommendation system is useful when we have cold start problems. Cold start refers to the issue when we get a new user into the system and the machine is not able to recommend movies to the new user, as the user did not have any historical interactions in the dataset. In those cases, we can use rank-based recommendation system to recommend movies to the new user.

To build the rank-based recommendation system, we take average of all the ratings provided to each movie and then rank them based on their average rating.

Now, let's create a function to find the top n movies for a recommendation based on the average ratings of movies. We can also add a threshold for a minimum number of interactions for a movie to be considered for recommendation.

We can use this function with different n's and minimum interactions to get movies to recommend

Recommending top 5 movies with 50 minimum interactions based on popularity

Recommending top 5 movies with 100 minimum interactions based on popularity

Recommending top 5 movies with 200 minimum interactions based on popularity

Now that we have seen how to apply the Rank-Based Recommendation System, let's create Collaborative Filtering Based Recommendation Systems.

Model 2: User based Collaborative Filtering Recommendation System (7 Marks)

collaborative_filtering.PNG

In this above interactions matrix - out of the users B and C, which user is most likely to interact with the movie - The Terminal?

In this type of recommendation system, we do not need any information about the users or items. We only need user item interaction data to build a collaborative recommendation system. For example -

  1. Ratings provided by users. For example - ratings of books on goodread, movie ratings on imdb etc
  2. Likes of users on different facebook posts, likes on youtube videos
  3. Use/buying of a product by users. For example - buying different items on e-commerce sites
  4. Reading of articles by readers on various blogs

Types of Collaborative Filtering

Building Similarity/Neighborhood based Collaborative Filtering

test_image

Building a baseline similarity based recommendation system

Below we are building similarity based recommendation system using cosine similarity and using KNN to find similar users which are nearest neighbor to the given user.

We will be using a new library - surprise to build the remaining models, let's first import the necessary classes and functions from this library

Below we are loading the rating dataset, which is a pandas dataframe, into a different format called surprise.dataset.DatasetAutoFolds which is required by this library. To do this we will be using the classes Reader and Dataset

Making the dataset into surprise dataset and splitting it into train and test set

Build the first baseline similarity based recommendation system using cosine similarity and KNN

Q 3.1 What is the RMSE for baseline user based collaborative filtering recommendation system (1 Mark)

The RMSE is 0.9925.

Q 3.2 What is the Predicted rating for an user with userId =4 and for movieId= 10 and movieId=3? (1 Mark)

Let's us now predict rating for an user with userId=4 and for movieId=10

The predicted rating for a user with userId=4 and for movieId=10 is 3.62.

Let's predict the rating for the same userId=4 but for a movie which this user has not interacted before i.e. movieId=3

The predicted rating for a user with userId=4 and for movieId=3 is 3.20.

Improving user-user similarity based recommendation system by tuning its hyper-parameters

Below we will be tuning hyper-parmeters for the KNNBasic algorithms. Let's try to understand different hyperparameters of KNNBasic algorithm -

For more details please refer the official documentation https://surprise.readthedocs.io/en/stable/knn_inspired.html

Q 3.3 Perform hyperparameter tuning for the baseline user based collaborative filtering recommendation system and find the RMSE for tuned user based collaborative filtering recommendation system? (3 Marks)

Once the grid search is complete, we can get the optimal values for each of those hyperparameters as shown above

Below we are analysing evaluation metrics - RMSE and MAE at each and every split to analyze the impact of each value of hyperparameters

Now we will building final model by using tuned values of the hyperparameters which we received by using grid search cross validation

The RMSE is 0.9871.

Q 3.4 What is the Predicted rating for an user with userId =4 and for movieId= 10 and movieId=3 using tuned user based collaborative filtering? (1 Mark)

Let's us now predict rating for an user with userId=4 and for movieId=10 with the optimized model

The predicted rating for a user with userId=4 and for movieId=10 is 3.58.

Below we are predicting rating for the same userId=4 but for a movie which this user has not interacted before i.e. movieId=3, by using the optimized model as shown below -

The predicted rating for a user with userId=4 and for movieId=3 is 3.17.

Identifying similar users to a given user (nearest neighbors)

We can also find out the similar users to a given user or its nearest neighbors based on this KNNBasic algorithm. Below we are finding 5 most similar user to the userId=4 based on the msd distance metric

Implementing the recommendation algorithm based on optimized KNNBasic model

Below we will be implementing a function where the input parameters are -

Predicted top 5 movies for userId=4 with similarity based recommendation system

Q 3.5 Predict the top 5 movies for userId=4 with similarity based recommendation system (1 Mark)

Model 3 Item based Collaborative Filtering Recommendation System (7 Marks)

Q 4.1 What is the RMSE for baseline item based collaborative filtering recommendation system (1 Mark)

The RMSE is 0.9964.

Let's us now predict rating for an user with userId=4 and for movieId=10

Q 4.2 What is the Predicted rating for an user with userId =4 and for movieId= 10 and movieId=3? (1 Mark)

The predicted rating for a user with userId=4 and for movieId=10 is 4.42.

Let's predict the rating for the same userId=4 but for a movie which this user has not interacted before i.e. movieId=3

The predicted rating for a user with userId=4 and for movieId=3 is 4.06.

Q 4.3 Perform hyperparameter tuning for the baseline item based collaborative filtering recommendation system and find the RMSE for tuned item based collaborative filtering recommendation system? (3 Marks)

Once the grid search is complete, we can get the optimal values for each of those hyperparameters as shown above:

Below we are analysing evaluation metrics - RMSE and MAE at each and every split to analyze the impact of each value of hyperparameters

The RMSE is 0.9495.

Q 4.4 What is the Predicted rating for an item with userId =4 and for movieId= 10 and movieId=3 using tuned item based collaborative filtering? (1 Mark)

Let's us now predict rating for an user with userId=4 and for movieId=10 with the optimized model as shown below

The predicted rating for a user with userId=4 and for movieId=10 is 4.18.

Let's predict the rating for the same userId=4 but for a movie which this user has not interacted before i.e. movieId=3, by using the optimized model:

The predicted rating for a user with userId=4 and for movieId=3 is 4.36.

Identifying similar users to a given user (nearest neighbors)

We can also find out the similar users to a given user or its nearest neighbors based on this KNNBasic algorithm. Below we are finding 5 most similar user to the userId=4 based on the msd distance metric

Predicted top 5 movies for userId=4 with similarity based recommendation system

Q 4.5 Predict the top 5 movies for userId=4 with similarity based recommendation system (1 Mark)

Model 4 Collaborative Filtering - Matrix Factorization using SVD (7 marks)

Model-based Collaborative Filtering is a personalized recommendation system, the recommendations are based on the past behavior of the user and it is not dependent on any additional information. We use latent features to find recommendations for each user.

Latent Features: The features that are not present in the empirical data but can be inferred from the data. For example:

test_image

Now if we notice the above movies closely:

test_image

Here Action, Romance, Suspense and Comedy are latent features of the corresponding movies. Similarly, we can compute the latent features for users as shown below:

test_image

Singular Value Decomposition (SVD)

SVD is used to compute the latent features from the user-item matrix that we already learned earlier. But SVD does not work when we missing values in the user-item matrix.

First we need to convert the below movie-rating dataset:

test_image

into an user-item matrix as shown below:

test_image

We have already done this above while computing cosine similarities.

SVD decomposes this above matrix into three separate matrices:

U-matrix

test_image

the above matrix is a n x k matrix, where:

Sigma-matrix

test_image

the above matrix is a k x k matrix, where:

V-transpose matrix

test_image

the above matrix is a kxn matrix, where:

Build a baseline matrix factorization recommendation system

Q 5.1 What is the RMSE for baseline SVD based collaborative filtering recommendation system (1 Mark)

The RMSE is 0.9023.

Q 5.2 What is the Predicted rating for an user with userId =4 and for movieId= 10 and movieId=3? (1 Mark)

Let's us now predict rating for an user with userId=4 and for movieId=10

The predicted rating for a user with userId=4 and for movieId=10 is 4.09.

Let's predict the rating for the same userId=4 but for a movie which this user has not interacted before i.e. movieId=3:

The predicted rating for a user with userId=4 and for movieId=3 is 3.79.

Improving matrix factorization based recommendation system by tuning its hyper-parameters

In SVD, rating is predicted as -

$$\hat{r}_{u i}=\mu+b_{u}+b_{i}+q_{i}^{T} p_{u}$$

If user $u$ is unknown, then the bias $b_{u}$ and the factors $p_{u}$ are assumed to be zero. The same applies for item $i$ with $b_{i}$ and $q_{i}$.

To estimate all the unknown, we minimize the following regularized squared error:

$$\sum_{r_{u i} \in R_{\text {train }}}\left(r_{u i}-\hat{r}_{u i}\right)^{2}+\lambda\left(b_{i}^{2}+b_{u}^{2}+\left\|q_{i}\right\|^{2}+\left\|p_{u}\right\|^{2}\right)$$

The minimization is performed by a very straightforward stochastic gradient descent:

$$\begin{aligned} b_{u} & \leftarrow b_{u}+\gamma\left(e_{u i}-\lambda b_{u}\right) \\ b_{i} & \leftarrow b_{i}+\gamma\left(e_{u i}-\lambda b_{i}\right) \\ p_{u} & \leftarrow p_{u}+\gamma\left(e_{u i} \cdot q_{i}-\lambda p_{u}\right) \\ q_{i} & \leftarrow q_{i}+\gamma\left(e_{u i} \cdot p_{u}-\lambda q_{i}\right) \end{aligned}$$

There are many hyperparameters to tune in this algorithm, you can find a full list of hyperparameters here

Below we will be tuning only three hyperparameters -

Q 5.3 Perform hyperparameter tuning for the baseline SVD based collaborative filtering recommendation system and find the RMSE for tuned SVD based collaborative filtering recommendation system? (3 Marks)

Once the grid search is complete, we can get the optimal values for each of those hyperparameters as shown above

Below we are analysing evaluation metrics - RMSE and MAE at each and every split to analyze the impact of each value of hyperparameters

Now we will building final model by using tuned values of the hyperparameters which we received by using grid search cross validation

Q 5.4 What is the Predicted rating for an user with userId =4 and for movieId= 10 and movieId=3 using SVD based collaborative filtering? (1 Mark)

Let's us now predict rating for an user with userId=4 and for movieId=10 with the optimized model

The predicted rating for a user with userId=4 and for movieId=10 is 3.99.

Q 5.5 Predict the top 5 movies for userId=4 with SVD based recommendation system (1 Mark)

Predicting ratings for already interacted movies

Below we are comparing the rating predictions of users for those movies which has been already watched by an user. This will help us to understand how well are predictions are as compared to the actual ratings provided by users

Here we are comparing the predicted ratings by similarity based recommendation system against actual ratings for userId=7

Below we are comparing the predicted ratings by matrix factorization based recommendation system against actual ratings for userId=7

Precision and Recall @ k

RMSE is not the only metric we can use here. We can also examine two fundamental measures, precision and recall. We also add a parameter k which is helpful in understanding problems with multiple rating outputs.

See the Precision and Recall @ k section of your notebook and follow the instructions to compute various precision/recall values at various values of k.

To know more about precision recall in Recommendation systems refer to these links :

https://surprise.readthedocs.io/en/stable/FAQ.html

https://medium.com/@m_n_malaeb/recall-and-precision-at-k-for-recommender-systems-618483226c54

Question6: Compute the precision and recall, for each of the 6 models, at k = 5 and 10. This is 6 x 2 = 12 numerical values? (4 marks)

Question 7 ( 5 Marks)

7.1 Compare the results from the base line user-user and item-item based models.

7.2 How do these baseline models compare to each other with respect to the tuned user-user and item-item models?

7.3 The matrix factorization model is different from the collaborative filtering models. Briefly describe this difference. Also, compare the RMSE and precision recall for the models.

7.4 Does it improve? Can you offer any reasoning as to why that might be?

Write your Answer here:__

7.1) The base line user-user model "algo_knn_user" had a slightly lower RMSE than the base line item-item model "algo_knn_item", which were 0.9925 and 0.9964 respectively.

The prediction for userId = 4 and movieId = 10 were 3.62 and 4.42 respectively, so for this particular prediction, the user-user model predicted a rating closer to the actual rating of 4.

Also, the item-item model seems to over-predict while the user-user model under-predicts.

7.2) The baseline models compared to the tuned models have a greater RMSE. In addition, the baseline user-user model predicted a rating (3.62) which was closer to the actual rating than the tuned model which predicted a rating of 3.58. On the other hand, the item-item models behaved the opposite as the tuned model was closer to the actual rating than the baseline model.

7.3) The difference between the matrix faxtorization model and the collaborative filtering models is that the matrix factorization model focuses on the user's past behavior by converting the latent features, such as genre, of the movie the user rates, where as collaborative filtering models are developed based on related users or related items and not the user directly.

The respective RMSE values for the algorithms:

algo_knn_user: 0.9925 similarity_algo_optimized_user: 0.9871

algo_knn_item: 0.9964 similarity_algo_optimized_item: 0.9495

algo_svd: 0.9023 svd_algo_optimized: 0.8955

The precision and recall for all models is shown in Question 6. The precision is the highest for the baseline user-user model with k = 5. The precision is the lowest with the baseline item-item model with k=10. The recall is lowest with the baseline item-item model with k=5. The recall is highest with the baseline user-user model with k=10.

7.4) There was no improvement from the tuned or SVD models as the RMSE and precision continued to decrease with each new algorithm. Eventhough some of the tuned models and SVD models performed better for the specific userId=4, the baseline algorithms perform better on the entire dataset.

Conclusions

In this case study, we saw three different ways of building recommendation systems:

We also understood advantages/disadvantages of these recommendation systems and when to use which kind of recommendation systems. Once we build these recommendation systems, we can use A/B Testing to measure the effectiveness of these systems.

Here is an article explaining how Amazon use A/B Testing to measure effectiveness of its recommendation systems.