Search Federation Architecture at LinkedIn

March 14, 2018

author

Co-authors: Yi Shen, Claire Liu, and Ali Mohamed

Introduction: A brief history of Search Federation at LinkedIn 简史

Almost every part of LinkedIn contains data that needs to be discoverable by our members or customers. Use cases range from a member looking up a news article posted by someone in their network to a recruiter looking for candidates on the platform. One of the primary mechanisms for discovering this content is search.

Accordingly, at LinkedIn, we have many different search engines for a variety of use cases, such as People Search (members) and Job Search (posted job opportunities). The Search Federation team’s mission is to help our members and customers find resources by searching across these engines. We call these use case-focused engines “search backends.”

We also provide a family of mid-tier services designed to answer search and search typeahead queries. These services typically follow the federation pattern, where the user’s query is first expanded through query understanding, spell checking, and other techniques. This expansion is used to write queries specific for each search backend. The documents returned from the backends are then combined to be most useful to the user through a process called “blending.”

Collectively, this process makes up Search Federation at LinkedIn. Search Federation provides us with a way to personalize ambiguous queries. For instance, when a user searches for "machine learning" on LinkedIn, they could mean to search for people with machine learning skills, jobs requiring machine learning skills, or content about the topic.

LinkedIn的几乎每个部分都包含需要我们的会员或客户发现的数据。从用户查找其社交网络中某人发布的新闻文章 到 招聘人员在平台上寻找候选人。搜索是发现此内容的主要机制之一。

因此，在LinkedIn上，我们针对各种用例提供​​了许多不同的搜索引擎，例如“人物搜索”（成员）和“职位搜索”（发布的工作机会）。Search Federation团队的任务是通过搜索这些引擎来帮助我们的会员和客户找到资源。我们称这些针对用例的引擎为“搜索后端”。

我们还提供一系列中间层服务，旨在回答搜索和搜索提前查询。这些服务通常遵循联合身份验证模式，在该模式中，首先通过查询理解，拼写检查和其他技术来扩展用户的查询。此扩展用于编写特定于每个搜索后端的查询。然后，将从后端返回的文档进行合并，以通过称为“混合”的过程对用户最有用。

总的来说，此过程构成了LinkedIn上的Search Federation。Search Federation为我们提供了一种个性化歧义查询的方法。例如，当用户在LinkedIn上搜索“机器学习”时，他们可能意味着搜索具有机器学习技能的人，需要机器学习技能的工作或有关主题的内容。

Search Federation mid-tier building blocks Search Federation中层构建块

Historically, our Search Federation architecture has consisted of three mid-tier services.
从历史上看，我们的Search Federation体系结构由三个中层服务组成。

The federated-search-rest service was created in 2012 to provide a Rest.li API for the Search endpoint and served as a Search Federation mid-tier for the blended search experience. Additionally, federated-search-rest calls the search-decoration service to decorate the result with unindexed information.

federated-search-rest服务创建于2012年，旨在为Search端点提供Rest.li API，并充当Search Federation中间层的混合搜索体验。此外，联合搜索休息会调用搜索装饰服务，以使用未编制索引的信息来修饰结果。

The typeahead-rest service was created in Q1 2014 to provide a Rest.li API for the typeahead endpoint and to serve as a typeahead federation mid-tier for the blended typeahead experience.

typeahead-rest服务创建于2014年第一季度，以提供用于typeahead终结点的Rest.li API，并用作混合aatypehead体验的aatypeahead联合中间层。

The seas-federated-search service was created in Q2 2014 to be the new federator used by all external services to access search. During the Galene migrations, all calls from federated-search-rest to the legacy backends were moved over to seas-federated-search. In this blog post, we use "new federation mid-tier" to refer to seas-federated-search service.

seas-federated-search服务于2014年第二季度创建，是所有外部服务用来访问搜索的新联合身份验证程序。在Galene迁移期间，所有从联合搜索站点到传统后端的呼叫都移到了海联合搜索上。在此博客文章中，我们使用“新的联合中间层”来指代seas-federated-search服务。

Like with any service, over the years product requirements have changed, the number of callers has grown, and an increase in use cases has resulted in a good amount of legacy code. At the same time, there has been a tremendous increase in demand for ease-of-use and rapid iteration from developers modifying Search Federation architecture (110+ code contributors and 800+ commits, for instance, in 2017) for different verticals.

In early 2015, the team identified three main challenges in our legacy federation architecture that were impeding our ability to iterate quickly and threatened search stability as our scale increased:

与其他服务一样，多年来，产品要求已发生变化，调用方的数量有所增加，用例的增加导致大量的旧代码。同时，针对不同行业的开发人员修改Search Federation体系结构（例如，在2017年，提供了110多个代码贡献者和800多个提交），对易用性和快速迭代的需求已大大增加。

在2015年初，该团队发现了我们的旧联盟架构中的三个主要挑战，这些挑战阻碍了我们快速迭代的能力，并随着规模的扩大而威胁到搜索稳定性：

• Leverage between search and typeahead - Similar federation logic in these closely-related mid-tiers was written with different design patterns. This meant adding a new feature would always involve changing in stacks of multiple services and making separate deployments.
• Code isolation - The legacy design was based on inheritance so that verticals serving different use cases were inevitably coupled. On top of that, it wasn’t easy to extend for new vertical use cases in a clean way, and complicated if-else clauses were added in many shared components.
• Test and deployment - Low test coverage in 2015 and no Search Federation services supporting daily deployment made it difficult to reliably deploy changes without regressions.
• 在搜索和提前输入之间发挥杠杆作用-在这些密切相关的中间层中，类似的联合逻辑使用不同的设计模式编写。这意味着添加新功能将始终涉及更改多个服务的堆栈并进行单独的部署。
• 代码隔离-传统设计基于继承，因此服务于不同用例的垂直行业不可避免地会耦合在一起。最重要的是，要以干净的方式扩展新的垂直用例并不容易，并且在许多共享组件中添加了复杂的if-else子句。
• 测试和部署-2015年的测试覆盖率较低，并且没有支持每日部署的Search Federation服务使得难以可靠地部署更改而不进行回归。

Roadmap: Building a bridge to the future 路线图：搭建通往未来的桥梁

To address the above problems, the Search Federation team began implementing a redesign of the federation service with the following strategies: services consolidation, code isolation, and improving our overall test coverage.
为了解决上述问题，Search Federation团队开始采用以下策略对联合身份验证服务进行重新设计：服务合并，代码隔离和提高总体测试覆盖率。

Services consolidation:

• We merged the legacy search federation mid-tier and legacy typeahead federation mid-tier into a new federation mid-tier to ease leveraging between search and typeahead.
• Access to all backends now goes through this new federation mid-tier, the same place where query understanding, spell checking, and data fetching are happening.

服务整合：

• 我们将旧版搜索联合中间层和旧式提前键入联合中间层合并为新的联合中间层，以简化在搜索和预输入之间的平衡。
• 现在，对所有后端的访问都经过这个新的联合中间层，即查询理解，拼写检查和数据获取发生的同一位置。

Code isolation:

• We used a workflow framework to compose a use case specific workflow, which provides code isolation among verticals. No if-else clauses are needed, and new use cases can be onboarded easily by reusing common workflows.
• Modularize code per vertical use case.
  代码隔离：
• 我们使用了工作流框架来组成特定于用例的工作流，该工作流提供了垂直行业之间的代码隔离。不需要if-else子句，并且可以通过重用常见的工作流轻松地引入新的用例。
• 每个垂直用例模块化代码。

Overall test coverage improvement:

• Add more unit tests to increase code coverage.

• Add more integration tests to enhance PCx test.

• Add dark canary hosts.

  总体测试覆盖率的提高：

• 添加更多的单元测试以增加代码覆盖率。

• 添加更多的集成测试以增强PCx测试。

• 添加dark canary hosts。

Additionally, this had to be accomplished while accommodating the search needs of a rapidly-growing global membership, which was dramatically increasing the overall amount and number of types of content that needed to be searchable on LinkedIn. These issues needed to be addressed without disrupting the ~700 million searches that LinkedIn handles every single day.
此外，必须在满足快速增长的全球会员的搜索需求的同时实现这一点，这极大地增加了需要在LinkedIn上搜索的内容的总数和类型。必须解决这些问题，同时又不影响LinkedIn每天处理的约7亿次搜索。

Progress so far 到目前为止的进展

The Search Federation team kicked off the Search Federation Re-architecture Project by building the SeaS workflow framework, then we rolled-out the migration in stages: individual workflow migration, typeahead federation consolidation, and Search Federation consolidation.
Search Federation团队通过构建SeaS工作流框架启动了Search Federation重新架构项目，然后我们分阶段进行了迁移：单个工作流迁移，预输入联合合并和Search Federation合并。

Workflow migration (2016)
Starting in Q4 2015 and working through Q2 2016, we successfully migrated all query processing to new federation mid-tier and all 40+ search use cases to SeaS workflow. Access to all Galene backends was routed through this new federation mid-tier.

工作流程迁移（2016）
从2015年第四季度开始一直到2016年第二季度，我们成功地将所有查询处理迁移到了新的联合中间层，并将所有40多个搜索用例迁移到了SeaS工作流。对所有Galene后端的访问均通过此新的联合中间层进行。

Search Federation architecture at the end of 2016

Typeahead federation consolidation
Through the remainder of 2016, we made a lot of progress for typeahead consolidation, where we extracted typeahead-federator (core typeahead fanout and blending logic) out of the legacy typeahead federation mid-tier and integrated it into the new federation mid-tier. After that, typeahead-rest served solely as a proxy/adapter for API calls. Following ramping for federated and blended typeahead requests to 100% was done in Q1 2017 without any impact of relevance and operational metrics.
Typeahead联合合并
在2016年剩余时间里，我们在typeahead合并方面取得了很多进展，我们从传统的typeahead联合中间层提取了typeahead-federator（核心typeahead扇出和混合逻辑）并将其集成到新的联邦中间层。之后，typeahead-rest仅充当API调用的代理/适配器。在2017年第一季度，将联邦和混合预先输入的请求增加到100％之后，没有任何相关性和运营指标的影响。

Search Federation architecture at Q1/2017

The typeahead consolidation helped reduce the QPS of the new federation mid-tier by half, from 16K to 8K. There were 0 GCNs caused by this migration, and no noticeable latency change.
预先进行的合并有助于将新联盟中层的QPS降低一半，从16K降至8K。此迁移导致了0个GCN，并且没有明显的延迟变化。

**Search Federation consolidation
**Besides typeahead federation consolidation, another big missing piece of the re-architecture was to move search-federator (search fanout and blending logic) from the legacy search federation mid-tier to the new search federation mid-tier. There were a few challenges for this move, such as:

• A large number of deprecated Federated Search Rest.li API parameters and half-ramped features owned by different verticals. To clean them up, collaboration and coordination across multiple teams was critical.
• A good amount of legacy undocumented code was present in search-federat