Artwork Personalization at Netflix
Netflix Technology BlogFollow
Dec 8, 2017 · 13 min read
By Ashok Chandrashekar, Fernando Amat, Justin Basilico and Tony Jebara
For many years, the main goal of the Netflix personalized recommendation system has been to get the right titles in front each of our members at the right time. With a catalog spanning thousands of titles and a diverse member base spanning over a hundred million accounts, recommending the titles that are just right for each member is crucial. But the job of recommendation does not end there. Why should you care about any particular title we recommend? What can we say about a new and unfamiliar title that will pique your interest? How do we convince you that a title is worth watching? Answering these questions is critical in helping our members discover great content, especially for unfamiliar titles. One avenue to address this challenge is to consider the artwork or imagery we use to portray the titles. If the artwork representing a title captures something compelling to you, then it acts as a gateway into that title and gives you some visual “evidence” for why the title might be good for you. The artwork may highlight an actor that you recognize, capture an exciting moment like a car chase, or contain a dramatic scene that conveys the essence of a movie or TV show. If we present that perfect image on your homepage (and as they say: an image is worth a thousand words), then maybe, just maybe, you will give it a try. =This is yet another way Netflix differs from traditional media offerings: we don’t have one product but over a 100 million different products with one for each of our members with ==personalized recommendations== and ==*personalized visuals*==.=
多年来，Netflix个性化推荐系统的主要目标，是为用户在合适的时间推荐合适的视频。Nteflix 网站上每个分类页面下有成千上万部影片，用户账号达数十亿，因此为每个成员推荐合适的视频至关重要。但推荐系统能做到的不仅是这些。怎样让用户对你推荐的视频感兴趣？怎样让一个陌生的视频激起用户的兴趣？什么样的视频值得关注？回答这些问题对于帮助用户发现好的内容至关重要，特别是对于不熟悉的视频。
视频的封面：为视频设计独立的海报或图像，是可以轻松地解决这个问题的方法之一。如果一张封面对用户有足够的吸引力，比如用户熟悉的演员、让人肾上腺激素飙升的汽车追逐场面，或者一部电影或电视节目精髓的戏剧性场景等信息（一张图片胜过千言万语），就会诱惑用户点开视频。这是 Netflix 与传统媒体产品不同的一点：我们提供的不是一个产品，而是一个千人千面的产品。就算一亿个用户进来，看到的也完全不同，我们为每个用户提供个性化推荐和个性化的视觉效果。
A Netflix homepage without artwork. This is how historically our recommendation algorithms viewed a page.
没有封面图的主页。过去，页面上推荐算法的效果
In previous work, we discussed an effort to find the single perfect artwork for each title across all our members. Through multi-armed bandit algorithms, we hunted for the best artwork for a title, say Stranger Things, that would earn the most plays from the largest fraction of our members. However, given the enormous diversity in taste and preferences, wouldn’t it be better if we could find the best artwork for each of our members to highlight the aspects of a title that are specifically relevant to them?
之前，我们讨论过如何做到为所有会员的视频匹配最合适的封面。通过多臂老虎机算法，我们可以为视频找到最合适的封面，以《怪奇物语》为例，这部影片获得了最高用户播放率。但是，鉴于用户的品味和偏好存在巨大差异，如果我们能够找到每个用户偏好的点，并在封面图中能呈现出他们最感兴趣的东西，效果不是更好吗？
Artwork for Stranger Things that each receive over 5% of impressions from our personalization algorithm. Different images cover a breadth of themes in the show to go beyond what any single image portrays.
增加海报后，新的视频通过个性化算法，得到了5%的提升。不同的图像涵盖了节目中的不同主题
As inspiration, let us explore scenarios where personalization of artwork would be meaningful. Consider the following examples where different members have different viewing histories. On the left are three titles a member watched in the past. To the right of the arrow is the artwork that a member would get for a particular movie that we recommend for them.
我们探讨一下封面个性化在哪些场景下具有重要意义。例如，每个用户有不同的观看历史，下图左是三个用户过去看过的视频，箭头右侧是我们为会员推荐的颇受欢迎的电影。
Let us consider trying to personalize the image we use to depict the movie Good Will Hunting. Here we might personalize this decision based on how much a member prefers different genres and themes. Someone who has watched many romantic movies may be interested in Good Will Hunting if we show the artwork containing Matt Damon and Minnie Driver, whereas, a member who has watched many comedies might be drawn to the movie if we use the artwork containing Robin Williams, a well-known comedian.
我们为电影《心灵捕手》设计个性化封面的根据是每个用户对不同类型和主题的偏好。对于看过许多浪漫爱情电影的人，如果他的推荐图片中包含马特·达蒙（Matt Damon）和米妮·司各德（Minnie Driver）的信息，可能他会对《心灵捕手》感兴趣，而如果是对于看过很多喜剧片的用户，我们在推荐图中包含知名喜剧演员罗宾·威廉斯（Robin Williams）的信息，吸引他的几率可能更大。
In another scenario, let’s imagine how the different preferences for cast members might influence the personalization of the artwork for the movie Pulp Fiction. A member who watches many movies featuring Uma Thurman would likely respond positively to the artwork for Pulp Fiction that contains Uma. Meanwhile, a fan of John Travolta may be more interested in watching Pulp Fiction if the artwork features John.
另外，个性化封面对喜欢不同演员的用户会产生什么影响呢？以《低俗小说》为例，一位观看过很多乌玛·瑟曼（Uma Thurman）出演电影的用户可能会对包含乌玛（Uma）信息的图片反应更为积极。同理，John Travolta 的粉丝更可能因为图像中包含 John 而被这部影片吸引。
Of course, not all the scenarios for personalizing artwork are this clear and obvious. So we don’t enumerate such hand-derived rules but instead rely on the data to tell us what signals to use. Overall, by personalizing artwork we help each title put its best foot forward for every member and thus improve our member experience.
当然，并不是所有的封面个性化场景都是这么明了的。所以我们并没有穷举这些规则，而是依靠数据来告诉我们应该使用什么图片。总体而言，通过封面个性化处理，我们可以帮助提高每个用户的体验。
Challenges 挑战
At Netflix, we embrace personalization and algorithmically adapt many aspects of our member experience, including the rows we select for the homepage, the titles we select for those rows, the galleries we display, the messages we send, and so forth. Each new aspect that we personalize has unique challenges; personalizing the artwork we display is no exception and presents different personalization challenges. One challenge of image personalization is that we can only select a single piece of artwork to represent each title in each place we present it. In contrast, typical recommendation settings let us present multiple selections to a member where we can subsequently learn about their preferences from the item a member selects. This means that image selection is a chicken-and-egg problem operating in a closed loop: if a member plays a title it can only come from the image that we decided to present to that member. What we seek to understand is when presenting a specific piece of artwork for a title influenced a member to play (or not to play) a title and when a member would have played a title (or not) regardless of which image we presented. Therefore artwork personalization sits on top of the traditional recommendation problem and the algorithms need to work in conjunction with each other. Of course, to properly learn how to personalize artwork we need to collect a lot of data to find signals that indicate when one piece of artwork is significantly better for a member.
Netflix 还通过算法对网站做了很多个性化处理，以提高会员体验，包括主页列表选择、列表的标题、展示的图片、发送的消息等等。对于我们来说，每一个方面的个性化处理都是独特的挑战，个性化封面也不例外。其中，图像个性化处理的挑战之一，是每个位置视频的封面只能有一张。相比之下，典型的推荐设置可以向会员提供多个选择，之后我们可以从会员的选择中了解他们的偏好。这就是个先有鸡还是先有蛋的问题。会员到底是因为个性化封面吸引他，点击的这个视频，还是因为本来就想看这个视频，和封面无关。因此，个性化封面推荐应该结合传统方法与算法才能奏效。当然，为了正确学习封面个性化，我们需要收集大量的数据，来找到能表明哪个封面对于用户更合适的信息。
Another challenge is to understand the impact of changing artwork that we show a member for a title between sessions. Does changing artwork reduce recognizability of the title and make it difficult to visually locate the title again, for example if the member thought was interested before but had not yet watched it? Or, does changing the artwork itself lead the member to reconsider it due to an improved selection? Clearly, if we find better artwork to present to a member we should probably use it; but continuous changes can also confuse people. Changing images also introduces an attribution problem as it becomes unclear which image led a member to be interested in a title.
另一个挑战，是要理解封面变化所产生的影响，是否会降低视频的可识别性，让视频在视觉上难以定位？例如，会员之前感兴趣但至今还没有注意到的视频，或者，封面改变是否会让用户改变想法。如果我们找到更好的图片呈现给会员并不断更换封面，会让会员感到迷惑。另外，改变封面也会引起一个问题，我们不清楚究竟是哪张封面引起了会员对视频的兴趣。
Next, there is the challenge of understanding how artwork performs in relation to other artwork we select in the same page or session. Maybe a bold close-up of the main character works for a title on a page because it stands out compared to the other artwork. But if every title had a similar image then the page as a whole may not seem as compelling. Looking at each piece of artwork in isolation may not be enough and we need to think about how to select a diverse set of images across titles on a page and across a session. Beyond the artwork for other titles, the effectiveness of the artwork for a title may depend on what other types of evidence and assets (e.g. synopses, trailers, etc.) we also display for that title. Thus, we may need a diverse selection where each can highlight complementary aspects of a title that may be compelling to a member.
接下来，是要理解封面如何与同一个页面或者阶段选择的其他封面进行合理关联。也许主角的大胆特写非常适用于页面上的视频封面，因为与其他作品相比，它显得非常突出。但是，如果整个页面的封面都是这一类型，那么它的效果反而会大打折扣。因此，孤立地看每一幅图片可能还不够，我们需要思考如何在整个页面使用多样化的图像。封面的效果可能还取决于图片之外其他的因素（例如简介、预告片等）。所以，我们的图片选择应该多样化，让每个视频之间都能形成互补。
To achieve effective personalization, we also need a good pool of artwork for each title. This means that we need several assets where each is engaging, informative and representative of a title to avoid “clickbait”. The set of images for a title also needs to be diverse enough to cover a wide potential audience interested in different aspects of the content. After all, how engaging and informative a piece of artwork is truly depends on the individual seeing it. Therefore, we need to have artwork that highlights not only different themes in a title but also different aesthetics. Our teams of artists and designers strive to create images that are diverse across many dimensions. They also take into consideration the personalization algorithms which will select the images during their creative process for generating artwork.
为了实现有效的个性化，我们还需要为每个视频提供优质的作品库。这意味着我们需要多个库存，并且每个库存的图片都是非常有吸引力、信息丰富且与视频契合，但要避免那种“诱导点击”式的图片。视频的图像集也需要足够多样化，以涵盖对内容的不同角度感兴趣的广大潜在观众。毕竟，一张封面的信息量取决于看到它的个体。因此，我们的封面不仅需要突出视频中的不同主题，还要突出不同的美学。
Finally, there are engineering challenges to personalize artwork at scale. One challenge is that our member experience is very visual and thus contains a lot of imagery. So using personalized selection for each asset means handling a peak of over 20 million requests per second with low latency. Such a system must be robust: failing to properly render the artwork in our UI brings a significantly degrades the experience. Our personalization algorithm also needs to respond quickly when a title launches, which means rapidly learning to personalize in a cold-start situation. Then, after launch, the algorithm must continuously adapt as the effectiveness of artwork may change over time as both the title evolves through its life cycle and member tastes evolve.
最后，是大规模个性化封面面临的工程挑战。由于我们的会员体验是视觉化的，包含大量的图像，因此，系统在峰值时需要每秒处理超过 2000 万个低延迟请求。这个系统必须足够强大，因为用户界面不能正确渲染图稿，用户体验会显著下降。而且，个性化算法还需要在视频上传时做出快速响应，这意味着要在冷启动的情况下快速个性化学习。启动后，该算法必须不断进行调试，因为封面的效果可能会随着时间的推移而变化，视频的生命周期不断演变，而且会员的品味也在不断变化。
Contextual bandits approach
Much of the Netflix recommendation engine is powered by machine learning algorithms. Traditionally, we collect a batch of data on how our members use the service. Then we run a new machine learning algorithm on this batch of data. Next we test this new algorithm against the current production system through an A/B test. An A/B test helps us see if the new algorithm is better than our current production system by trying it out on a random subset of members. Members in group A get the current production experience while members in group B get the new algorithm. If members in group B have higher engagement with Netflix, then we roll-out the new algorithm to the entire member population. Unfortunately, this batch approach incurs regret: many members over a long period of time did not benefit from the better experience. This is illustrated in the figure below.
Netflix 的大部分推荐引擎都采用机器学习算法。首先，我们会收集一批关于会员如何使用服务的数据，然后在这批数据上运行一个新的机器学习算法。接下来，我们对这种算法在现有生产系统上进行 A / B 测试。通过在随机子集上进行 A / B 测试，我们了解到新算法是否比现有的生产系统更好。A 组会员代表当前的产品体验，而 B 组代表新算法下的产品体验。如果 B 组中的会员对 Netflix 的参与度更高，那么我们将把这个新算法推广到整个会员群体。不幸的是，这种批处理方式也有缺憾（regret）：许多会员长期以来并没有更好的用户体验，如下图所示：
To reduce this regret, we move away from batch machine learning and consider online machine learning. For artwork personalization, the specific online learning framework we use is contextual bandits. Rather than waiting to collect a full batch of data, waiting to learn a model, and then waiting for an A/B test to conclude, contextual bandits rapidly figure out the optimal personalized artwork selection for a title for each member and context. Briefly, contextual bandits are a class of online learning algorithms that trade off the cost of gathering training data required for learning an unbiased model on an ongoing basis with the benefits of applying the learned model to each member context. In our previous unpersonalized image selection work, we used non-contextual bandits where we found the winning image regardless of the context. For personalization, the member is the context as we expect different members to respond differently to the images.
为了减小这个缺憾，我们放弃了批处理机器学习，而使用在线机器学习。对于图片个性化，我们使用的在线学习框架是contextual bandits。contextual bandits 并不是收集整批的数据，进行学习模型训练，直到 A / B 测试结束，而是可以迅速为每个会员找到最合适的个性化图片。简而言之，contextual bandits是一类在线学习算法，这种算法可以在学习无偏差模型所需的训练数据成本，和将学习模型应用于每个会员的好处之间进行权衡。在之前的工作中，我们使用非contextual bandits 方法进行封面选择，找到内容上最佳的图像。而对于个性化推荐，我们要考虑上下文，因为我们预计不同的会员会对图像做出不同的反应。
A key property of contextual bandits is that they are designed to minimize regret. At a high level, the training data for a contextual bandit is obtained through the injection of controlled randomization in the learned model’s predictions. The randomization schemes can vary in complexity from simple epsilon-greedy formulations with uniform randomness to closed loop schemes that adaptively vary the degree of randomization as a function of model uncertainty. We broadly refer to this process as data exploration. The number of candidate artworks that are available for a title along with the size of the overall population for which the system will be deployed informs the choice of the data exploration strategy. With such exploration, we need to log information about the randomization for each artwork selection. This logging allows us to correct for skewed selection propensities and thereby perform offline model evaluation in an unbiased fashion, as described later.
contextual bandits 的一个重要属性，是其是为尽量减小缺憾而设计的。在高层次上，我们通过在学习模型的预测中输入受控随机化来获得contextual bandits 的训练数据。随机化方案的复杂性可以从简单的具有均匀随机性的 epsilon-greedy 公式，到随着模型不确定性而自适应地改变随机化程度的闭环方案。我们将这个过程称为数据探索（data exploration）。进行这样的探索，我们需要记录每个封面选择的随机化信息。这种日志记录让我们可以纠正走偏的选择倾向，从而以稍后所述的不偏颇的方式执行离线模型评估。
Exploration in contextual bandits typically has a cost (or regret) due to the fact that our artwork selection in a member session may not use the predicted best image
