In a paper at the recent RecSys 2010 conference, “The YouTube Video Recommendation System” (ACM), eleven Googlers describe the system behind YouTube’s recommendations and personalization in detail.
The most interesting disclosure in the paper is that YouTube has switched from their old recommendation algorithm based on random walks to a new one based on item-to-item collaborative filtering. Item-to-item collaborative filtering is the algorithm Amazon developedback in 1998. Over a decade later, it appears YouTube found a variation of Amazon’s algorithm to be the best for their video recommendations.
Other notable tidbits in the paper are what the Googlers have to do to deal with noisy information (noisy video metadata and user preference data), the importance of freshness on videos (much like news), that they primarily used online measures of user satisfaction (like CTR and session length) when competing different recommendation algorithms against each other and tuning each algorithms, and the overall improvement (about x3 better) that recommendations got over simple features that just showed popular content.
Some excerpts from the paper:
Recommending interesting and personally relevant videos to [YouTube] users [is] a unique challenge: Videos as they are uploaded by users often have no or very poor metadata. The video corpus size is roughly on the same order of magnitude as the number of active users. Furthermore, videos on YouTube are mostly short form (under 10 minutes in length). User interactions are thus relatively short and noisy … [unlike] Netflix or Amazon where renting a movie or purchasing an item are very clear declarations of intent. In addition, many of the interesting videos on YouTube have a short life cycle going from upload to viral in the order of days requiring constant freshness of recommendation.
To compute personalized recommendations we combine the related videos association rules with a user’s personal activity on the site: This can include both videos that were watched (potentially beyond a certain threshold), as well as videos that were explicitly favorited, “liked”, rated, or added to playlists … Recommendations … [are the] related videos … for each video … [the user has watched or liked after they are] ranked by … video quality … user’s unique taste and preferences … [and filtered] to further increase diversity.
To evaluate recommendation quality we use a combination of different metrics. The primary metrics we consider include click through rate (CTR), long CTR (only counting clicks that led to watches of a substantial fraction of the video), session length, time until first long watch, and recommendation coverage (the fraction of logged in users with recommendations). We use these metrics to both track performance of the system at an ongoing basis as well as for evaluating system changes on live traffic.
Recommendations account for about 60% of all video clicks from the home page … Co-visitation based recommendation performs at 207% of the baseline Most Viewed page … [and more than 207% better than] Top Favorited and Top Rated [videos].
For more on the general topic of recommendations and personalization on YouTube, please see my 2009 post, “YouTube needs to entertain“.
By the way, it would have been nice if the Googlers had cited theAmazon paper on item-to-item collaborative filtering. Seems like a rather silly mistake in an otherwise excellent paper.
Update: To be clear, this was not intended as an attack on Google in any way. Googlers built on previous work, as they should. What is notable here is that, despite another decade of research on recommender systems, despite all the work in the Netflix Prize, YouTube found that a variant of the old item-to-item collaborative filtering algorithm beat out all others for recommending YouTube videos. That is a very interesting result and one that validates the strengths of that old algorithm.
original:http://glinden.blogspot.com/2011/02/youtube-uses-amazons-recommendation.html