Essential Suite — Artwork Producer Assistant
译者
Gingo 白果
Netflix Technology BlogFollow
Feb 7, 2020 · 8 min read
By: Hamid Shahid & Syed Haq
Introduction 介绍
Netflix continues to invest in content for a global audience with a diverse range of unique tastes and interests. Correspondingly, the member experience must also evolve to connect this global audience to the content that most appeals to each of them. Images that represent titles on Netflix (what we at Netflix call “artwork”) have proven to be one of the most effective ways to help our members discover the content they love to watch. We thus need to have a rich and diverse set of artwork that is tailored for different parts of the Netflix experience (what we call product canvases). We also need to source multiple images for each title representing different themes so we can present an image that is relevant to each member’s taste.
Netflix继续投资于具有各种独特口味和兴趣的全球受众的内容。相应地，会员体验也必须不断发展，以将全球用户与最吸引他们的内容联系起来。代表Netflix标题的封面图（在Netflix我们称之为“艺术品”）已被证明是帮助我们的会员发现他们喜欢观看的内容的最有效方法之一。因此，我们需要拥有一组丰富多样的封面图，这些封面图是为Netflix不同部分（我们称为产品画布）的体验量身定制的。我们还需要为代表不同主题的每个标题获取多个封面图，以便我们呈现与每个成员的品味相关的图像。
Manual curation and review of these high quality images from scratch for a growing catalog of titles can be particularly challenging for our Product Creative Strategy Producers (referred to as producers in the rest of the article). Below, we discuss how we’ve built upon our previous work of harvesting static images directly from video source files and our computer vision algorithms to produce a set of artwork candidates that covers the major product canvases for the entire content catalog. The artwork generated by this pipeline is used to augment the artwork typically sourced from design agencies. We call this suite of assisted artwork “The Essential Suite”.
对于我们不断增长的标题目录，手动管理和查看这些高质量图像对于我们的产品创意策略生产者（在本文的其余部分中称为生产者）来说尤其具有挑战性。下面，我们讨论如何基于先前的工作直接从视频源文件中收获静态图像以及我们的计算机视觉算法，以生成一组涵盖整个内容目录主要“产品画布“的封面图候选。该方法将来自设计机构的图稿进行增强处理，生成封面图。我们称这套封面图辅助工具为“Essential Suite”。
Supplement, not replace 补充，而不是替代
Producers from our Creative Production team are the ultimate decision makers when it comes to the selection of artwork that gets published for each title. Our usage of computer vision to generate artwork candidates from video sources thus is focussed on alleviating the workload for our Creative Production team. The team would rather spend its time on creative and strategic tasks rather than sifting through thousands of frames of a show looking for the most compelling ones. With the “Essential Suite”, we are providing an additional tool in the producers toolkit. Through testing we have learned that with proper checks and human curation in place, assisted artwork candidates can perform on par with agency designed artwork.
我们创意制作团队的创意生产者是产品呈现最终的决策者，他们为每个影片标题选择合适的封面图候选。因此，我们使用计算机视觉从视频源生成封面图的重点是减轻我们的创意制作团队的工作量。团队宁愿将时间花在创造性和战略性任务上，而不是筛选数千帧图片以寻找最引人注目的那些图。通过“Essential Suite”，我们在生产者工具包中提供了额外的工具。通过测试，我们了解到，通过适当的检查和适当的人工策划，机器选出来的封面图可以和专业设计的封面图相媲美。
Design Agencies 设计机构
Netflix uses best-in-class design agencies to provide artwork that can be used to promote titles on and off the Netflix service. Netflix producers work closely with design agencies to request, review and approve artwork. All artwork is delivered through a web application made available to the design agencies.
Netflix使用一流的设计机构来提供可用于在Netflix服务内外推广的封面图。Netflix的创意生产者与设计机构紧密合作，对封面图进行请求，审查和批准。所有封面图均由设计机构通过Web应用程序提供。
The computer generated artwork can be considered as artwork provided by an “Internal agency”. The idea is to generate artwork candidates using video source files and “bubble it up” to the producers on the same artwork portal where they review all other artwork, ideally without knowing if it is an agency produced or internally curated artwork, thereby selecting what goes on product purely based on creative quality of the image.
可以将计算机生成的图稿视为“内部机构”提供的图稿。想法是使用视频源文件生成候选作品，然后将其“冒泡”，提供给创意生产者。创意生产者查看所有其他封面图，理想情况下，不知道该作品是由代理机构制作还是内部策划的封面图，从而选择处理方式完全基于图像的创意质量。
Assisted Artwork Generation Workflow 封面图辅助生成工作流程
The artwork generation process involves several steps, starting with the arrival of the video source files and culminating in generated artwork being made available to producers. We use an open source workflow engine Netflix Conductor to run the orchestration. The whole process can be divided into two parts.
封面图生成过程涉及几个步骤，从视频源文件的到达开始，最后达到将生成的封面图提供给创意生产者的目的。我们使用开源工作流引擎Netflix Conductor来运行业务流程。整个过程可以分为两部分
- Generation
- Review
1. Generation 生成
This article on AVA provides a good explanation on our technology to extract interesting images from video source files. The artwork generation workflow takes it a step further. For a given product canvas, it selects a handful of images from the hundreds of video stills most suitable for that particular product canvas. The workflow then crops and color-corrects the selected image, picks out the best spot to place the movie’s title based on negative space, selects and resizes the movie title and places it onto the image.
这篇关于AVA的文章很好地解释了我们从视频源文件中提取有趣图像的技术。封面图生成工作流程使它更进一步。对于给定的产品画布，它从数百个最适合该特定产品画布的视频静止图像中选择少量图像。然后，工作流程会裁剪并校正所选图像的颜色，选择最佳位置以根据负空间放置电影标题，选择并调整电影标题的大小并将其放置在图像上。
Here is an illustration of what it means if we had to do it manually
这说明了如果我们必须手动进行操作的流程
a. Image selection
b. Identify areas of interest
c. Cropped, color-corrected & title placed in the negative space
Image Selection / Analyze Image 图像选择/分析图像
Selection of the right still image is essential to generating good quality artwork. A lot of work has already been done in AVA to extract out a few hundreds of frames from hundreds of thousands of frames present in a typical video source. Broadly speaking, we use two methods to extract movie stills out of video source.
选择正确的静止图像对于生成高质量的艺术品至关重要。在AVA中已经完成了许多工作，以从典型视频源中存在的数十万个帧中提取几百个帧。广义上讲，我们使用两种方法从视频源中提取电影静止图像。
- AVA — Ava is primarily a character based algorithm. It picks up frames with a clear facial shot taking into account actors, facial expression and shot detection.
- Cinematics — Cinematics picks up aesthetically pleasing cinematic shots.
- AVA — Ava主要是基于演员的算法。它将演员，面部表情和镜头检测等因素考虑在内，以清晰的面部镜头拾取帧。
- Cinematics—Cinematics选择令人愉悦的镜头。
The combination of these two approaches produce a few hundred movie stills from a typical video source. For a season, this would be a few hundred shots for each episode. Our work here is to pick up the stills that best work for the desired canvas.
=Both of the above algorithms use a few heatmaps which define what kind of images have proven to be working best in different canvases.= The heatmaps are designed by internal artists who are experienced in designing promotional artwork/posters.
这两种方法的结合可以从一个典型的视频源中产生数百个电影静止图像。每一季，每个情节可以得到几百张截图。我们在这里的工作是挑选最适合所需画布的静止图像。
=以上两种算法都使用了一些热图，这些热图定义了已证明在不同画布上效果最佳的图像类型。= 热图由在设计促销性艺术品/海报方面经验丰富的内部艺术家设计
Heatmap for a BillboardWe make use of meta-information such as the size of desired canvas, the “unsafe regions” and the “regions of interest” to identify what image would serve best. “Unsafe regions” are areas in the image where badges such as Netflix logo, new episodes, etc are placed. “Regions of interest” are areas that are always displayed in multi-purpose canvases. These details are stored as metadata for each canvas type and passed to the algorithm by the workflow. Some of our canvases are cropped dynamically for different user interfaces. For such images, the “Regions of interest” will be the area that is always displayed in each crop.
我们利用诸如所需画布的大小，“不安全区域”和“感兴趣区域”之类的元信息来确定哪种图像最适合。“不安全区域”是图像中放置徽标（例如Netflix标志，新剧集等）的区域。“感兴趣的区域”是始终显示在多用途
