We will talk about how to create a recommendation system with very little difficulty, but before that, we will talk about the different Machine Learning-based techniques used in implementing these systems. There are two most common algorithms used for the recommendation system: Content-based filtering and Collaborative filtering.
A basic example of a recommendation system from real life can be online shopping through Amazon or other e-commerce. When you purchase something, let’s say a phone, you will get a recommendation to buy a headphone or a screen protector to go with your phone. Another example can be Netflix’s recommendation engine. If you watch a particular genre of movies on Netflix e.g. action movies, you will soon see the recommendations of the same or a genre similar to the movie that you previously watched. The reason Netflix is able to understand viewers’ preferences so well and is also able to predict the next movie or show that the viewers would like to watch is achieved through the magic of a recommendation engine. By the end of this tutorial, you will not only be able to create a recommendation system on your own, but you will also be able to understand the techniques and the logic behind these systems. So without further ado, let’s get started!
To explain what a recommendation system truly is, we will make an analogy between the behaviors of human beings and those of these recommendation systems. Imagine a friend who is familiar with your taste in movies and always recommends movies that you end up watching again and again. Recommendation systems work very much in the same manner. They are like your friends, who know your tastes, likes, and dislikes and make recommendations of things to you accordingly. The question is, how do these recommendation systems know so much about you? Well, recommendation systems basically filter data by using different algorithms and then recommend the most relevant items to you. They first understand the past behavior of a user, based on his/her activity, and on the basis of that, they recommend products that the user is likely to buy or use. The next item that you buy on Amazon, the next show that you watch on Netflix, the next video that you watch on YouTube, the next friend that you add on Facebook, and the next song that you listen to on Spotify, all work on the same principles of recommendation systems.
The most common types of recommendation systems are Content-Based and Collaborative Based Filtering recommendation systems.
Content-Based Filtering
In Content-Based Filtering, items are recommended to users on the basis of being similar to what the user likes, and that is known by the user’s previous actions or explicit feedback given by him/her. We will try to understand this technique through an example. Let’s say that we have a user by the name of user X. User X has watched a movie M1 which belonged to the genre of action. She gave this movie a rating of 5 (out of 5). She watched another movie M2, which belonged to the genre of action as well, and gave it a rating of 4. From this, we can understand that user X enjoys watching action movies and we came to this conclusion because of the high ratings that she has given to these movies. Let’s say the third movie, M3 has recently been released and is of the same genre i.e. action. It also has some similarities with the previous movies e.g. the lead actor, director, etc. Now, the recommendation system will suggest this movie to user X on the basis of User 1’s profile.
Main idea: Recommend items to customer X which are similar to previous items rated highly by X.
For each item, i.e. a movie, website, blog, etc., an item profile is created and this profile becomes a set of feature vectors. For a movie, such feature vectors would consist of the writer, actor, director, etc. Important features are extracted from these item profiles and are used to build the User Profile which is the weighted average of rated item profiles.
Collaborative Based Filtering
Collaborative Based Filtering utilizes the quality judgments of others who are similar to the target user. Consider an example where we have 2 users, X and Y. X watches a movie M1 and gives it a rating of 5. User Y watches the same movie and gives it a rating of 4. User X watches movie M2 which is coincidentally also seen by user Y and both give it a rating of 4. User X watches a third movie M3 and gives it a rating of 5. Now, since the ratings of the two movies, M1 and M2, given by X and Y are so similar to each other, it is highly likely that the movie M3 (rated as 5 by user X) will also be liked by the user Y. The recommendation system will therefore recommend the movie M3 for user Y to watch.
Main idea: Consider user X and find a set N of other users whose ratings are similar to X’s ratings. Then, estimate X’s ratings based on ratings of users in N.
Now that we have talked about the two main techniques used in recommendation systems, we will discuss a simple way of creating one on your own using Machine Learning on Google Compute Engine. You can make use of open source technologies to create a basic recommendation system on the cloud platform.
A recommendation system consists of 4 phases: 1) collection, 2) storage, 3) analysis, and 4) recommendation. Its architecture consists of a front-end, a storage component, and Machine Learning. In the front-end, you can see a page with top recommendations by deploying a simple application on Google App Engine. This will help you to build a scalable web platform. For storage, Google Cloud SQL integrates well with MLlib which is a Machine Learning library. Lastly, you can use Google Cloud Dataproc to deploy and run MLlib based scripts. For more on this, visit How to build your own recommendation engine using machine learning on Google Compute Engine
To provide the perfect recommendation, your recommendation system needs lots and lots of data with high quality. Most times, it is very difficult for an individual or small- or medium-sized companies to collect such large quantity data while maintaining high quality. Therefore, it is often more efficient to find another service that does laborious works for you. We could be your perfect solution!
Here at DATUMO, we crowdsource our tasks to diverse users located globally to ensure the quality and quantity on time. Moreover, our in-house managers double-check the quality of the collected or processed data. If you need data? If you need preprocessed data? Let us know!
To sum it all up, we started this tutorial by discussing what recommendation systems and what they are used for. We talked about some real-life examples where these systems are used e.g. by Facebook to suggest friends, by Netflix to recommend movies or shows, and by YouTube to recommend videos, etc. Then we discussed the techniques of Content-based filtering and Collaborative filtering that enable a recommendation system to do what it is supposed to do i.e. to make recommendations to users. We saw how the two techniques differed from each other, where one technique i.e. Content-based filtering, made suggestions on the basis of the user’s profile whereas the other technique, Collaborative filtering, harnessed the judgment of other similar users to make recommendations to the user in question. Lastly, we briefly touched upon how one can build his/her own recommendation system using Machine Learning on Google Compute Engine.
