Uber Labs 因果推断实践

资料仓库  收藏
0 / 534

Mediation Modeling at Uber: Understanding Why Product Changes Work (and Don’t Work)

Bonnie Li and Totte Harinen

April 18, 2018** 0

Mediation Modeling at Uber: Understanding Why Product Changes Work (and Don’t Work)

**Tweet23

25SHARES

At Uber Labs, our mission is to leverage insights and methodologies from behavioral science to help product and marketing teams improve the customer experience. Recently, we introduced mediation modeling, a statistical approach from academic research, to address user pain points.

Mediation modeling goes beyond simple cause and effect relationships in an attempt to understand what underlying mechanisms led to a result. Using this type of analysis, we can fine-tune product changes and develop new ones that focus on the underlying mechanisms behind successful features on the Uber platform.
在 Uber Labs，我们的任务是利用行为科学的洞察力和方法论来帮助产品和市场团队改善客户体验。 最近，我们引入了中介模型来解决用户的痛点，它是一种来自学术研究的统计方法。

为了理解导致结果的潜在机制，中介模型超越了简单的因果关系。 使用这种类型的分析，我们可以从微调或开发产品的变化中找到优步平台上功能成功背后的基本机制。

Knowing Whether vs. Understanding Why

At Uber, we have a strong culture of improvement, frequently conducting experiments to test whether one variable impacts another to ensure reliable, safe, and seamless user experiences.

Most of the time we have a hypothesis about why two variables are related. For example, suppose we believe that a promotion to give new riders trip discounts on their first few trips may improve new rider retention. While a standard analysis would tell us whether this promotion helped increase new rider retention, it would not tell us why. For example, do new riders return to the app after their first several trips specifically because of reduced trip fares? Or could it be that the promotion helped new riders familiarize themselves with the app, or something else entirely? If multiple mechanisms are present, which one plays a larger role and by how much? In a standard analysis, an underlying mechanism (i.e., why something happened) is often assumed to exist but not empirically tested with data.

While we may have some evidence that two variables are related, we may not have a clear understanding of why they are related, and when we do not understand why, we have to rely on trial and error. However, just like in academic research, knowing why is equally important for Uber because it helps us build better products for our users. For instance, in the example above, if we found that their familiarity with the app kept new riders on the platform, we should then prioritize product changes that encourage riders to use the app.

知其然 vs 知其所以然

在 Uber，我们有一个强大的改进文化，通过频繁开展实验来测试一个变量是否会影响另一个变量，以确保可靠、安全和无缝的用户体验。

大多数时候，我们对为什么两个变量是相关的有一个假设。 例如，假设我们相信在新乘客的前几次旅行中给予他们行程折扣的促销活动可以提高新乘客的留存率。虽然一个标准的分析会告诉我们这种促销是否有助于增加新的骑手留存，但是它不会告诉我们为什么。 例如，新乘客是否会在前几次行程后因为行程费用降低而重新使用该应用程序？ 或者是促销帮助新乘客熟悉了这个应用程序，或者完全是别的什么？ 如果存在多种机制，哪一种机制发挥更大的作用，作用大小如何？ 在一个标准的分析中，一个潜在的机制(例如，为什么会发生这样的事情)通常被认为是存在的，但是没有用数据进行实证检验。

虽然我们可能有一些证据表明两个变量是相关的，但我们可能不清楚为什么它们是相关的，当我们不明白为什么，我们不得不依赖于试验和错误。 然而，就像在学术研究中一样，知道为什么对 Uber 同样重要，因为它帮助我们为用户打造更好的产品。 例如，在上面的例子中，如果我们发现他们对应用程序的熟悉使新的乘客留在了平台上，那么我们应该优先考虑鼓励乘客使用应用程序的新的产品功能。

Mediation modeling: opening the black box

Mediation modeling opens the black box between a treatment and an outcome variable to reveal the underlying mechanisms, i.e. why something happened. Although widely used in academic research, this approach is under-utilized in business.1, 2, 3 When we have a causal assumption, instead of leaving it at that or relying on correlational evidence, mediation modeling lets us empirically test (vs. logically infer) the causal pathways between the two variables. More importantly, understanding these mechanisms enables us to develop better products quickly and more efficiently because it helps us pinpoint which features of these changes are responsible for making products successful.

So, what exactly can we do with mediation modeling to improve the user experience? We outline some sample hypothetical use cases in Figure 1, below:

First, we can use mediation modeling to test product assumptions. For example, we may believe that a new rider promotion could increase retention because of reduced trip fares (Figure 1a). With mediation modeling, we can empirically test this assumption. Our results of these test results can tell us whether our assumption is true, and if so, how much of the effect is because of reduced trip fares as opposed to other mechanisms (e.g., familiarity with the app).

Second, we can use mediation modeling to compare multiple mechanisms. In a hypothetical example, we may believe that a new design of Uber Eats menus can increase orders through more than one mechanism (Figure 1b). With mediation modeling, we are able to estimate how much of the treatment effect (i.e., increased orders) is accounted for by each of these mechanisms and which mechanism plays the largest role. The results help inform how we design our products and their future iterations.

Mediation modeling also allows us to connect intangible variables, such as consumer sentiment, with a specific feature to business metrics. As we know, customer feelings and satisfaction are critical to the success of a business. However, it is often difficult to quantify their business impact. Mediation modeling, however, enables us to test how these variables affect downstream business metrics (e.g., rider referrals, as depicted in Figure 1c).

Moreover, mediation modeling can be a creative way to break a long-term goal into smaller intermediate steps. For example, suppose our goal is to increase long-term rider satisfaction with Uber. How do we break this goal into smaller pieces that can be tied to our day-to-day work? If we have previously identified a key mediator behind rider satisfaction, we can then leverage this mediator as a short-term key performance indicator (KPI) (Figure 1d). If most of the effect of an intervention is mediated through a particular mechanism, then influencing that key mediator can be a necessary (although not sufficient) condition for the intervention to work.

Across various use cases, we identify upstream and downstream variables and test how they are connected with each other. Next, we take this explanation a step further and discuss the conceptual details behind mediation modeling.

中介模型: 打开黑匣子

中介模型打开了实验和结果变量之间的黑匣子，从而揭示了潜在的机制，即为什么会发生某些事情。当我们有一个因果假设，而不是让它停留在那里或依赖于相关证据，中介模型让我们经验性地检验(与逻辑推断)两个变量之间的因果路径。 更重要的是，理解这些机制使我们能够更快更有效地开发出更好的产品，因为它有助于我们确定这些变化的哪些特征是使产品成功的原因。

那么，我们究竟可以通过中介建模做些什么来改善用户体验呢？ 我们在下面的图1中概述了一些假设的用例:

首先，我们可以使用中介建模来测试产品假设。 例如，我们可能认为，一个新的附加促销可以增加保留，因为减少旅行费用(图1a)。 通过中介建模，我们可以对这一假设进行实证检验。 我们的这些测试结果可以告诉我们我们的假设是否正确，如果正确，有多少影响是由于旅行费用的减少而不是其他机制(例如，熟悉应用程序)。

其次，我们可以使用中介建模来比较多种机制。 在一个假设的例子中，我们可能认为 Uber Eats 菜单的新设计可以通过多种机制增加订单(图1b)。 通过中介建模，我们能够估计这些机制中的每一个对实验效果(即增加订单)的影响程度，以及哪种机制发挥了最大的作用。 这些结果有助于告知我们如何设计我们的产品及其未来的迭代。

中介建模还允许我们将无形变量(如消费者情绪)与特定特性联系到业务指标。 众所周知，顾客的感受和满意度对于企业的成功至关重要。 然而，通常很难量化它们对业务的影响。 然而，中介建模使我们能够测试这些变量如何影响下游业务指标(例如，如图1c 所示的附加引用)。

此外，中介建模可以是将长期目标分解为更小的中间步骤的一种创造性方法。 例如，假设我们的目标是提高长期乘客对 Uber 的满意度。 我们如何将这个目标分解成更小的部分，并与我们的日常工作联系起来？ 如果我们之前已经确定了骑手满意度背后的关键中介，那么我们就可以利用这个中介作为短期关键绩效指标(KPI)(图1 d)。 如果干预的大部分效果是通过某一特定机制引起的，那么影响该关键调解人可能是干预发挥作用的一个必要(虽然不是充分)条件。

在不同的用例中，我们识别上游或下游变量并测试它们之间的连接方式。 接下来，我们将进一步讨论这个解释，并讨论中介建模背后的概念细节。

Mediation modeling as causal inference

To successfully execute this technique, what exactly are the quantities we estimate and with what methodology? To answer these questions, we depict the simplest possible mediation model in Figure 2, below:

Figure 2. The conceptual framework of mediation modeling incorporates the intervention, mediator, and, finally, outcome.

From this type of model, our goal is to estimate three key quantities:

    • The average direct effect (ADE): c
    • The average causal mediated effect (ACME): ab
    • The average total effect (ATE): ab + c

In recent years, researchers have begun to understand mediation modeling in causal terms.4, 5, 6 This has allowed them to conceptualize mediation using formal frameworks developed for causal inference, such as the potential outcomes approach developed by Neyman, Rubin, and others.7

To better grasp how this framework functions, suppose we have an outcome Y and treatment assignment t, such that t is either 1 if an individual is in the treatment group or 0 if an individual is in the control group. Then, the outcome Y under treatment assignment t for individual i is Yi(t).

Now, we are often interested in estimating the difference in Y between the treatment and control, which for individual i is

Yi(1)−Yi(0)

However, most of the time we only observe one of these two outcomes because an individual is usually in just one experimental condition. So, for example if individual i is in the treatment condition (t=1), then outcome Yi(0) for that individual is just a potential outcome, i.e., an outcome that could have occurred but did not actually occur.

Since we usually cannot observe the treatment effects at an individual level, we estimate the group-level average treatment effect, which is defined as

E[Yi(1)−Yi(0)]

Here, the outcome Y under the treatment and the control assignment is estimated from the group-level quantities.

What does all this have to do with mediation modeling? Well, it turns out we can represent the three key mediation quantities using the above framework. Let M(t) correspond to the potential mediator value under treatment assignment t. Then, we can start defining the key mediation quantities as follows:

ATE=E[Yi(1,Mi(1))−Yi(0,Mi(0))]

In short, ATE is the difference between the potential outcomes under the treatment and the control assignment when the mediator changes as it actually does with the treatment assignment. This solves for the average effect of the treatment on the outcome.

As mentioned earlier, the goal of mediation analysis is to decompose the total treatment effect into two parts: the average direct effect (ADE) and the average causal mediated effect (ACME). In other words, ADE is the impact from the treatment on the outcome that does not go through the mediator. So, if you fix the value of the mediator while varying the value of the treatment status, then you will generate the direct effect. Using the potential outcomes notation, we have

ADE=E[Yi(1,Mi(t))−Yi(0,Mi(t))]

for t=0,1. So, the ADE can be understood as any additional contribution of the treatment assignment on the outcome once we prevent the mediator from changing with the treatment status.8 (This prevention does not happen literally. We use statistical models to estimate what the outcome would have been if the mediator were fixed.)

Once we have established the ADE, it is clear that the ACME is simply its complement:

ACME=E[Yi(t,Mi(1))−Yi(t,Mi(0))]

for t=0,1. ACME corresponds to the difference in potential outcomes that would occur if we were to flip the mediator into the value it would take under the treatment status while holding the treatment status itself fixed.

One of the great things about these definitions is that they do not make any reference to a particular model. Consequently, researchers like Imai et. al.8 have developed algorithms that estimate the key mediation quantities using any valid model. This means, among other things, that we are free to estimate the relationships in the mediation graph using nonparametric and nonlinear models, a considerable advancement compared to the past.9, 10, 11, 12 For example, in a traditional approach such as Hayes’ PROCESS method, the mediators cannot be categorical variables and the outcome variables are restricted to only those that can be properly modeled with ordinary least squares or logistic regression13. The limitations on mediators and outcome variables imposed by the PROCESS method do not let us model discrete data, which makes it unsuitable for our work at Uber.

Finally, the potential outcomes framework proves helpful in laying out the identification assumptions for mediation effects. In the context of randomized experiments, the major assumption is that the mediator should be statistically independent of the potential outcomes of Y conditional on the observed treatment status and the values of the pretreatment covariates included in the model.14, 15 The reason why the mediation quantities are not consistently estimated without this assumption is that it would be otherwise possible for some third variable to confound the mediator-outcome relationship.

Even though the potential outcomes framework helps us to define this assumption clearly, it is not possible to conclusively verify that the assumption holds. The best we can do is to (1) include any pretreatment covariates that theoretical considerations suggest could de-confound the relationship between the mediator and the outcome and (2) conduct sensitivity analyses to see how our estimates would change if our assumptions were not satisfied to different degrees.16 Luckily, as a data-driven technology company, we generally have a good set of pretreatment covariates that we can use in our models to mitigate the risk of confounding.

中介模型 VS 因果推断

为了成功地执行这项技术，我们究竟需要用几个什么样的方法呢？ 为了回答这些问题，我们在下面的图2中描述了最简单的中介模型:

图2. 中介建模的概念框架包含了干预、中介和最终的结果

根据这种类型的模型，我们的目标是估计三个关键数量:

  • 平均直接效应(ADE) :c
  • 平均因果调节效应(ACME) :ab
  • 平均总效应(ATE) : ab + c

近年来，研究人员已经开始理解中介模型的因果关系。这使他们能够使用为因果推断开发的正式框架将中介概念化，如由 Neyman，Rubin 和其他人开发的潜在结果方法。

为了更好地理解这个框架是如何运作的，假设我们有一个结果 y 和实验分配 t，如果一个人在实验组中 t 是1，如果一个人在对照组中 t 是0。 然后，实验分配下个体 i 的结果 y 是 Yi (t)。

现在，我们经常感兴趣的是估计实验和控制之间 y 的差异，对于个体 i

Yi(1)−Yi(0)Yi(1)−Yi(0)

然而，大多数时候我们只观察到这两种结果中的一种，因为一个人通常只处于一种实验状态。 例如，如果个体 i 处于治疗状态(t1) ，那么个体的结果 Yi (0)只是一个潜在的结果，也就是说，一个可能发生但实际上没有发生的结果。

由于我们通常不能观察治疗效果在个人水平，我们估计的群体水平的平均治疗效果，这是定义为

E[Y_i(1)−Y_i(0)]E[Yi​(1)−Yi​(0)]

在这里，处理和控制分配的结果 y 是估计从组水平的数量。

所有这些与中介建模有什么关系？ 事实证明，我们可以用上面的框架来表示三个关键的中介量。 设 m (t)对应于实验分配下的潜在介质值 t，那么，我们可以开始定义关键的介质量如下:

ATE=E[Y_i(1,M_i(1))−Y_i(0,M_i(0))]ATE=E[Yi​(1,Mi​(1))−Yi​(0,Mi​(0))]

简而言之，ATE 是实验下的潜在结果和控制分配之间的差异，当中介变化，因为它实际上与实验分配。 这解决了实验对结果的平均影响。

如前所述，中介建模的目标是将总体实验效应分解为两部分: 平均直接效应(ADE)和平均因果中介效应(ACME)。 换句话说，ADE 是实验对结果的影响，不需要经过中介变量。 因此，如果您在改变处理状态的值的同时确定中介的值，那么您将生成直接效果。 使用潜在的结果符号，我们有

ADE=E[Y_i(1,M_i(t))−Y_i(0,M_i(t)) ]ADE=E[Yi​(1,Mi​(t))−Yi​(0,Mi​(t))]

对于 t=0, 因此，ADE 可以理解为一旦我们阻止中介变量改变实验状态，实验分配对结果的任何额外贡献。 我们使用统计模型来估计，如果中介变量是固定的，结果会是什么。)

一旦我们建立了 ADE，很明显 ACME 只是它的补充:

ACME=E[Y_i(t,M_i(1))−Y_i(t,M_i(0))]ACME=E[Yi​(t,Mi​(1))−Yi​(t,Mi​(0))]

对于 t=0, Acme 对应于潜在结果的差异，如果我们将中介变量翻转为在实验状态下它将采取的值，同时保持实验状态本身固定，将会看到这种差异。

这些定义的优点之一是它们没有提到任何特定的模型。 因此，像 Imai 等研究人员。 已经开发了使用任何有效模型估计关键中介量的算法。 这意味着，除了其他事情之外，我们可以自由地使用非参数和非线性模型来估计中介图中的关系，与过去相比，这是一个相当大的进步。例如，在传统的方法中，如 Hayes’ PROCESS 方法，中介不能是分类变量，结果变量仅限于那些可以用一般最小平方法或回归方法适当建模的变量。 过程方法对中介变量和结果变量的限制不允许我们建立离散数据模型，这使得它不适合我们在 Uber 的工作。

最后，潜在结果框架证明有助于确定中介效应的识别假设。 在随机实验的背景下，主要的假设是中介变量应该在统计上独立于 y 的潜在结果，条件是观察到的实验状态和模型中包含的预处理协变量的值。

尽管潜在结果框架有助于我们清楚地定义这一假设，但不可能最终验证这一假设是否成立。 我们能做的最好的是(1)包括任何预处理协变量，理论上的考虑建议可以去混淆介质和结果之间的关系和(2)进行敏感性分析，看看我们的估计将如何变化，如果我们的假设不满足不同程度。幸运的是，作为一个数据驱动的技术公司，我们通常有一个良好的预处理协变量集，我们可以使用在我们的模型，以减轻混淆的风险。

Application example: customer support tickets

Now, let us take a look at a concrete example of how we applied mediation modeling at Uber through the lens of a recent analysis with our Customer Obsession team. We wanted to understand if adding a graph showing driver earnings led to fewer support tickets. To help users file less support tickets, we needed to understand why they were filing tickets in the first place.

In a previous experiment, the Customer Obsession team found that adding a graph displaying a driver-partner’s weekly earnings significantly increased their understanding of their earnings. Figure 3, below, shows the UI for the treatment and the control group:

应用示例: 用户工单处理

现在，让我们来看一个具体的例子，通过最近对我们的顾客至上团队的分析，我们如何在 Uber 中应用中介模型。 我们想知道，如果增加一个显示司机收入的图表，是否会导致更少的支持工单。 为了帮助用户提交更少的支持工单，我们需要了解他们为什么首先要提交工单。

在之前的一个实验中，顾客至上团队发现，添加一个显示司机-合伙人每周收入的图表，显著增加了他们对自己收入的理解。 图3，下面，显示了实验组和控制组的 UI:

Figure 3. The treatment and control earnings tab views offered slightly different interfaces for our users.
图3. 实验和控制盈利标签视图为我们的用户提供了略有不同的界面

In this analysis, earnings understanding was measured using a single-item survey that asked drivers how well they understood their earnings information on a five-point scale.

We wanted to know if earnings understanding was a significant behavioral mechanism behind earnings-related support tickets. In other words, we wanted to understand whether it was the case that the treatment improved earnings understanding, which in turn reduced support tickets. If that is the case, we would also like to estimate how much of the treatment effect was mediated through the path of earnings understanding (as opposed to other mechanisms).

To answer these questions, we used mediation modeling with the intervention as the independent variable, earnings understanding as the mediator, and earnings-related support tickets as the outcome variable (Figure 4). In addition, we included drivers’ pre-experiment support tickets and pre-experiment earnings experience (e.g., previous earnings, lifetime trips, and active days) as control variables. We finished the analysis by running a sensitivity analysis to determine how robust our results were against unmeasured mediator-outcome confounders.

在这个分析中，收益理解是通过一个单项调查来衡量的，这个调查询问了司机们对他们收益信息的理解程度，以5分为单位。

我们想知道收入理解是否是收入相关支持工单背后的一个重要的行为机制。 换句话说，我们想知道是否实验提高了对收入的理解，从而减少了支持票。 如果是这样的话，我们还想估计有多少实验效果是通过盈利理解的途径(而不是其他机制)来调节的。

为了回答这些问题，我们使用了以干预为独立变量的中介建模，以盈利理解为中介，以与盈利相关的支持票作为结果变量(图4)。 此外，我们还将驾驶员的实验前支持票和实验前收入经验(例如，以前的收入，终身旅行和活跃天数)作为控制变量。 我们通过运行一个敏感度分析以确定我们的结果对于未经测量的中介变量-结果混杂因素的强大程度。

Figure 4. Mediation modeling tests the underlying mechanism behind earnings-related support tickets.
图4. 中介建模测试与收益相关的支持工单背后的基本机制

The mediation modeling results showed that earnings understanding was indeed a significant mechanism behind earnings-related support tickets and it accounted for about 19 percent of the total treatment effect. Based on the results, we better understood how earnings understanding is an important path to reduce customer support tickets.

With these insights, we can better design products and communications that improve driver understanding of their earnings , thereby making the user experience more seamless and enjoyable. At the same time, we also recognized that there was a large opportunity for our team to further leverage models to uncover the mechanisms that mediate the remaining 81 percent of the effect. To accomplish this, we are partnering with the Customer Obsession team to test more of the remaining behavioral mechanisms. These results will inform the driver earnings product roadmap.

中介模型结果表明，收入理解确实是收入相关支持工单背后的一个重要机制，约占总实验效果的19% 。 基于这些结果，我们更好地理解了为什么盈利理解是减少客户支持工单的一个重要途径。

有了这些见解，我们可以更好地设计产品和沟通，提高司机对其收入的理解，从而使用户体验更加无缝和愉快。 与此同时，我们也意识到我们的团队有很大的机会进一步利用模型来揭示其余81% 影响的中介机制。 为了实现这一点，我们正在与顾客至上团队合作，测试更多剩余的行为机制。 这些结果将推进收益产品路线图的规划。

The future of mediation modeling

The future of mediation modeling is exciting: in recent years, academic researchers have started to investigate the prospect of conducting automatic mediator selection using high-dimensional data.17, 18 In these applications, we can specify potentially hundreds of mediators that are temporally between a treatment and an outcome. We then test whether there is a statistically reliable association between the treatment and each candidate mediator or between each candidate mediator and the outcome variable. With these approaches, we can potentially unpack the black box of the treatment-outcome relationship in unprecedented detail.

At Uber, we look forward to testing these data-driven algorithms and seeing how they compare with traditional theory-driven approaches through which researchers specifies the mediators before they conduct their research. We see mediation modeling as another interesting area where using machine learning methods to analyze experiments holds great potential.

If you are interested in leveraging applied behavioral science to power products and cities, learn about job opportunities with Uber Labs.

中介模型的未来

中介模型的未来是令人兴奋的: 近年来，学术研究人员已经开始研究使用高维数据进行自动中介变量选择。在这些应用中，我们可以指定潜在的数百个中介变量，它们在实验和结果之间是暂时的。 然后我们测试治疗和每个候选中介变量之间或者每个候选中介变量和结果变量之间是否存在统计学上可靠的联系。 有了这些方法，我们就有可能在前所未有的细节上揭开实验-结果关系的黑匣子。

在 Uber，我们期待测试这些数据驱动的算法，看看它们与传统的理论驱动方法相比如何。研究人员在进行研究之前，通过传统的理论驱动方法指定介质。 我们认为中介建模是另一个有趣的领域，在这里使用机器学习方法来分析实验具有很大的潜力。

如果你有兴趣利用应用行为科学为产品和城市提供动力，可以到 Uber Labs了解工作机会。

**References: **

  1. Aggarwal, P., & McGill, A. L. (2007). Is that car smiling at me? Schema congruity as a basis for evaluating anthropomorphized products. Journal of Consumer Research, 34(4), 468-479.
  2. Luo, X., & Bhattacharya, C. B. (2006). Corporate social responsibility, customer satisfaction, and market value. Journal of marketing, 70(4), 1-18.
  3. Usborne, E., & Taylor, D. M. (2010). The role of cultural identity clarity for self-concept clarity, self-esteem, and subjective well-being. Personality and Social Psychology Bulletin, 36(7), 883-897.
  4. Pearl, J. (2001). Direct and indirect effects. In Proceedings of the Seventeenth Conference on Uncertainty in Artificial Intelligence. Morgan Kaufmann, San Francisco, CA, 411–420.
  5. Imai, Kosuke, Luke Keele, and Dustin Tingley. “A general approach to causal mediation analysis.” *Psychological methods *15.4 (2010): 309-334.
  6. Pearl, Judea. “The causal mediation formula—a guide to the assessment of pathways and mechanisms.” Prevention Science 13.4 (2012): 426-436.
  7. Rubin, Donald B. “Estimating causal effects of treatments in randomized and nonrandomized studies.” Journal of educational Psychology 66.5 (1974): 688-701.
  8. Imai, Kosuke, Luke Keele, and Dustin Tingley. “A general approach to causal mediation analysis.” Psychological methods, 15.4 (2010): 309-334.
  9. Kenny, David A. “Mediation” (2018): http://davidakenny.net/cm/mediate.htm.
  10. Hayes, A. F. (2017). Introduction to mediation, moderation, and conditional process analysis: A regression-based approach. Guilford Publications.
  11. Imai, Kosuke, Luke Keele, and Dustin Tingley. “A general approach to causal mediation analysis.” Psychological methods, 15.4 (2010): 309-334.
  12. Tingley, D., Yamamoto, T., Hirose, K., Keele, L., & Imai, K. (2014) “Mediation: R package for causal mediation analysis.” Journal of Statistical Software, Volume 59, Issue 5.
  13. Hayes, A. F., Montoya, A. K., & Rockwood, N. J. (2017). The analysis of mechanisms and their contingencies: PROCESS versus structural equation modeling. Australasian Marketing Journal (AMJ), 25(1), 76-81.
  14. Imai, Kosuke, Luke Keele, and Dustin Tingley. “A general approach to causal mediation analysis.” Psychological methods, 15.4 (2010): 309-334.
  15. Pearl, Judea. “Interpretation and identification of causal mediation.” Psychological methods 19.4 (2014): 459-481.
  16. Imai, K., Keele, L., & Yamamoto, T. (2010). Identification, inference and sensitivity analysis for causal mediation effects. Statistical science, 51-71.
  17. Haixiang Zhang, Yinan Zheng, Zhou Zhang, Tao Gao, Brian Joyce, Grace Yoon, Wei Zhang, Joel Schwartz, Allan Just, Elena Colicino, Pantel Vokonas, Lihui Zhao, Jinchi Lv, Andrea Baccarelli, Lifang Hou, Lei Liu. Estimating and testing high-dimensional mediation effects in epigenetic studies, Bioinformatics, Volume 32, Issue 20, 15 October 2016, Pages 3150–3154.
  18. Yu, Qingzhao, and Bin Li. “mma: An R Package for Mediation Analysis with Multiple Mediators.” Journal of Open Research Software 5.1 (2017).

Comments

https://eng.uber.com/mediatio...