[论文翻译]分层用户兴趣电商推荐系统

Hierarchical User Profiling for E-commerce Recommender Systems

分层用户兴趣的电商推荐系统

Yulong Gu

顾玉龙

Data Science Lab,
JD.com guyulongcs@gmail.com

JD.com·guyulongcs@gmail.com数据科学实验室

Shuaiqiang
Wang

帅强王

Data Science
Lab, JD.com wangshuaiqiang1@jd.com

JD.com·wangshuaiqiang1@jd.com数据科学实验室

ABSTRACT

摘要

Hierarchical
user profiling that aims to model users' real-time in-terests in different
granularity is an essential issue for personal-ized recommendations in
E-commerce.On one hand, items (i.e. products) are usually organized
hierarchically in categories, and correspondingly users' interests are
naturally hierarchical on dif-ferent granularity of items and categories.On the
other hand, mul-tiple granularity oriented recommendations become very popular
in E-commerce sites, which require hierarchical user profiling in different
granularity as well.In this paper, we propose HUP, a Hierarchical User
Profiling framework to solve the hierarchical user profiling problem in
E-commerce recommender systems.In HUP, we provide a Pyramid Recurrent Neural
Networks, equipped with Behavior-LSTM to formulate users' hierarchical real-time
in-terests at multiple scales.Furthermore, instead of simply utilizing users'
item-level behaviors (e.g., ratings or clicks) in conventional methods, HUP
harvests the sequential information of users' tem-poral finely-granular
interactions (micro-behaviors, e.g., clicks on components of items like
pictures or comments, browses with nav-igation of the search engines or
recommendations) for modeling.Extensive experiments on two real-world
E-commerce datasets demonstrate the significant performance gains of the HUP
against state-of-the-art methods for the hierarchical user profiling and
recommendation problems.We release the codes and datasets at
https://github.com/guyulongcs/WSDM2020_HUP.

旨在以不同粒度对用户的实时兴趣进行建模的分层用户简档是电子商务中个性化推荐的一个重要问题。一方面，项目(即产品)通常是按类别分层组织的，相应地，用户的兴趣在项目和类别的不同粒度上自然是分层的。另一方面，面向多粒度的推荐在电子商务网站中变得非常流行，这些网站也需要不同粒度的分层用户配置。在本文中，我们提出了HUP，一个分层的用户剖析框架来解决电子商务推荐系统中的分层用户剖析问题。在HUP中，我们提供了一个配备了行为LSTM的金字塔递归神经网络，以在多个尺度上表达用户的分层实时兴趣。此外，代替在传统方法中简单地利用用户的项目级行为(例如，评级或点击)，HUP收集用户的临时精细粒度交互的顺序信息(微行为，例如，点击像图片或评论这样的项目的组成部分，浏览搜索引擎的导航或推荐)用于建模。在两个真实的电子商务数据集上进行的大量实验表明，对于分层用户剖析和推荐问题，HUP相对于最先进的方法有显著的性能提升。我们在https://github.com/guyulongcs/WSDM2020_HUP.发布代码和数据集

CCS CONCEPTS

•Information
systems→Personalization;Recommender sys-tems.

信息系统→个性化；推荐系统。

KEYWORDS

关键词

User profiling;Recommender
systems;Hierarchical user profiling;Pyramid Recurrent Neural
Networks;E-commerce

用户分析；推荐系统；分层用户分析；金字塔递归神经网络；电子商务

Permission to
make digital or hard copies of all or part of this work for personal or
classroom use is granted without fee provided that copies are not made or
distributed for profit or commercial advantage and that copies bear this notice
and the full citation on the first page.Copyrights for components of this work
owned by others than ACM must be honored.Abstracting with credit is
permitted.To copy otherwise, or republish, to post on servers or to
redistribute to lists, requires prior specific permission and/or a fee.Request
permissions from permissions@acm.org .WSDM '20, February 3-7, 2020, Houston,
TX, USA 2020 Association for Computing Machinery.ACM ISBN
978-1-4503-6822-3/20/02...$15.00 https://doi.org/10.1145/3336191.3371827

允许免费制作本作品的全部或部分数字或硬拷贝供个人或课堂使用，前提是拷贝的制作或分发不是为了盈利或商业利益，并且拷贝带有本通知和第一页的完整引文。必须尊重除ACM之外的其他人拥有的本作品组件的版权。允许用信用抽象。以其他方式复制或重新发布，在服务器上发布或重新发布到列表，需要事先获得特定许可和/或费用。向permissions@acm.org申请许可。WSDM '20，2020年2月3日至7日，美国德克萨斯州休斯顿2020计算机械协会。美国计算机学会ISBN 978-1-4503-6822-3/20/02。。。15.00美元https://doi.org/10.1145/3336191.3371827

1
INTRODUCTION

1导言

Figure 1:
Hierarchical recommendations in Amazon In the era of Internet, recommender
systems are playing crucial roles in various applications such as E-commerce
portals (e.g. Ama-zon, JD.com , Alibaba), social networking websites like
Facebook, video-sharing sites like Youtube, visual discovery sites like
Pinterest and so on.In practice, User Profiling [5, 11, 18, 24, 33, 38] is one
of the most important phases in recommender systems.It yields profile vectors,
which formally represent users' interests by deeply understanding their
historical interactions, can be used for candi-date generation [31, 42],
click-through rate prediction [4, 39, 40], conversion rate prediction [3, 16]
and long-term user engagement optimization [34-37, 44-46].

图1:亚马逊的分层推荐在互联网时代，推荐系统在各种应用中发挥着至关重要的作用，如电子商务门户网站(如Ama-zon、JD.com、阿里巴巴)、脸书等社交网站、Youtube等视频分享网站、Pinterest等视觉发现网站等。实际上，用户简档[5，11，18，24，33，38]是推荐系统中最重要的阶段之一。它通过深入了解用户的历史交互产生了形式上代表用户兴趣的简档向量，可用于candi-date生成[31，42]、点击率预测[4，39，40]、转化率预测[3，16]和长期用户参与度优化[34-37，44-46]。

Recently,
modeling users' hierarchical real-time interests is emerg-ing to be a crucial
issue in E-commerce recommender systems.Firstly, items (i.e. products) in
E-commerce sites are typically orga-nized in hierarchical
catalogue.Correspondingly, users' interests naturally lie hierarchically on
multiple granularity of items and categories.Secondly, different granularity of
recommendations (e.g. item, topic and category) become very popular in E-commerce
sites, and such scenarios require hierarchical user profiling in different

近年来，在电子商务推荐系统中，用户分层实时兴趣建模成为一个关键问题。首先，电子商务网站中的商品(即产品)通常以分级目录的形式组织。相应地，用户的兴趣自然分层地位于项目和类别的多个粒度上。其次，不同粒度的推荐(例如，项目、主题和类别)在电子商务网站中变得非常流行，并且这种场景需要在不同的

granularity
as well.For instance, Figure 1 illustrates a real example of hierarchical
recommendations in Amazon.The left side of the figure recommends some items (mobile
phones) to a user, while the right side shows a list of recommendations on the
categories of "phone accessories", "chargers" and so
on.Category recommen-dation can help the recommender systems quickly figure out
the main interest of the user and make better recommendations.

粒度也是如此。例如，图1展示了亚马逊分层推荐的一个真实例子。图的左侧向用户推荐了一些物品(手机)，右侧显示了“手机配件”、“充电器”等类别的推荐列表。类别推荐可以帮助推荐系统快速找出用户的主要兴趣，做出更好的推荐。

Existing user
profiling methods mainly focus on item recom-mendations, usually based on
users' item-level responses like rat-ings [20] or clicks [14].Among existing
methods, latent factor mod-eling is a popular branch, including matrix
factorization [13, 20, 38], neural embedding [8, 10], etc.Generally they learn
a unified embed-ding for the target user to represent her interests on the
items based on her historical behaviors.Recently, recurrent neural networks
(RNN) have achieved state-of-the-art performance in session-based
recommendations [14, 29].

现有的用户简档方法主要关注项目推荐，通常基于用户的项目级响应，如点击[20]或点击[14]。在现有的方法中，潜在因子模型是一个热门的分支，包括矩阵分解[13，20，38]，神经嵌入[8，10]等。一般来说，他们为目标用户学习一个统一的嵌入，以基于她的历史行为来表示她对项目的兴趣。最近，递归神经网络(RNN)在基于会话的建议中取得了最先进的性能[14，29]。

Existing
methods have the following limitations.First, when facing different granularity
of recommendation tasks, most of them usually need to run a similar algorithm
multiple times on different granularity of item organizations, where each run
builds users' certain level profile vectors for the corresponding
recommendation task, i.e., item-level profiles for item recommendations and
category-level profiles for category recommendations.Correspondingly, the
training process of each level's profile vectors is completely inde-pendent
from the others.However, users' multiple-level interests are closely
correlated.Figure 2 illustrates a user's hierarchical in-terests, including an
item level and two category levels, with her historical behaviors.Resulting
from the correlations between items and categories, improvement on one
recommendation task might benefit others.However, to the best of our knowledge,
such privi-lege has not been explored in existing methods.

现有方法有以下限制。首先，当面对不同粒度的推荐任务时，他们中的大多数通常需要在不同粒度的项目组织上多次运行类似的算法，其中每次运行都为相应的推荐任务构建用户的特定级别简档向量，即项目推荐的项目级简档和类别推荐的类别级简档。相应地，每个级别的轮廓向量的训练过程完全独立于其他级别。然而，用户的多层次兴趣密切相关。图2展示了用户的分层兴趣，包括一个项目级别和两个类别级别，以及她的历史行为。由于项目和类别之间的相关性，一项推荐任务的改进可能会使其他任务受益。然而，据我们所知，这种特权还没有在现有的方法中得到探索。

Second, only
harvesting the signals of users' item-level inter-actions like ratings and
clicks is insufficient.In most of the E-commerce portals, users provide
finely-granular responses such as clicking and browsing different modules
(e.g., comments and pictures) of items, adding to shopping carts and purchases,
which are referred to as "micro-behaviors" [30, 41].For example, the
bot-tom layer of Figure 2 presents a user's historical micro-behaviors

第二，仅仅收获用户的项目级交互信号如评分和点击是不够的。在大多数电子商务门户中，用户提供精细的响应，如点击和浏览不同模块(如评论和图片)的项目，添加到购物车和购买，这被称为“微行为”[30，41]。例如，图2的机器人层展示了用户的历史微观行为

in JD.com
(one of the largest e-commerce site in the world), includ-ing browsing a pair
of Nike shoes from the homepage, searching and reading specifications of iPhone
8, browsing Google Pixels 2 from the promoting page, searching iPhone X,
reading comments and adding it into the shopping cart for purchasing,
etc.Obviously, in comparison with users' item-level responses, micro-behaviors
provide more detailed information, and preliminary studies [30, 41] have
demonstrated the advantage of modeling such detailed behav-iors.However, to our
best knowledge, none of existing methods has leveraged such advantages to
improve the performance of multiple-level user profiling.

在JD.com(全球最大的电商网站之一)，包括从主页浏览一双耐克鞋、搜索阅读iPhone 8的规格、从推广页面浏览Google Pixels 2、搜索iPhone X、阅读评论并添加到购物车进行购买等。显然，与用户的项目级响应相比，微观行为提供了更详细的信息，初步研究[30，41]证明了对这种详细行为建模的优势。然而，据我们所知，现有的方法都没有利用这样的优势来提高多级用户概要分析的性能。

Third,
generally users' interests are dynamic and continuously shifting.Some
state-of-the-art methods like Time-LSTM [43] usu-ally incorporate time
intervals to track the interests shifting.How-ever, we argue that besides the
time intervals, the types of behaviors and their dwell time are also extremely
important.As shown in Figure 2, we know that iPhone X is preferable to others,
since vari-ous micro-behaviors are performed on iPhone X with long dwell
time.We also observe that triggered by making an order on iPhone X, the user's
interests on mobile phones drop sharply.Neglecting to model behavior types and
dwell times, Time-LSTM would be in trouble to capture users' detailed
preferences and interests shifting.

第三，一般来说，用户的兴趣是动态的，不断变化的。一些最先进的方法，如时间-LSTM [43]通常结合时间间隔来跟踪利益转移。然而，我们认为，除了时间间隔，行为类型和停留时间也极其重要。如图2所示，我们知道iPhone X比其他的更好，因为各种各样的微行为在长停留时间的iPhone X上执行。我们还观察到，在iPhone X上下单触发，用户对手机的兴趣急剧下降。如果忽略对行为类型和停留时间的建模，时代LSTM将很难捕捉到用户的详细偏好和兴趣转移。

To cope with
these challenges, we present HUP, a hierarchical user profiling framework to
precisely formulate users' real-time interests on multiple organizations of
items, targeting significant performance gains in recommendation accuracy.In
particular, it models users' multiple-level interests with a Pyramid Recurrent
Neural Networks, which typically consist of a micro layer, an item layer, and
multiple category recurrent neural network layers.The micro layer harvests the
detailed behavioral information and passes it to the higher layers, which could
abstract users' hierarchical inter-ests on the corresponding levels of the item
organizations simulta-neously.Furthermore, to sensitively track users'
real-time interests, we introduce Behavior-LSTM in each layer, where a behavior
gate is designed to model the types and dwell time of behaviors.Extensive
experiments for item recommendation and category recommenda-tion tasks have
been conducted on two large-scale real e-commerce datasets to demonstrate the
effectiveness of our proposed approach.

为了应对这些挑战，我们提出了HUP，这是一个分层的用户配置框架，可以精确地表达用户对多个项目组织的实时兴趣，目标是在推荐准确性方面获得显著的性能提升。具体来说，它使用金字塔递归神经网络对用户的多层次兴趣进行建模，该神经网络通常由微层、项目层和多类别递归神经网络层组成。微观层收集详细的行为信息并传递给更高层，可以同时抽象出用户在项目组织相应层次上的分层兴趣。此外，为了敏感地跟踪用户的实时兴趣，我们在每一层中引入了行为LSTM，其中设计了一个行为门来模拟行为的类型和停留时间。在两个大规模真实电子商务数据集上进行了项目推荐和类别推荐任务的大量实验，验证了所提方法的有效性。

224

Technical
Presentation

技术演示

WSDM '20, February
3-7, 2020, Houston, TX, USA

WSDM '20，2020年2月3日至7日，美国德克萨斯州休斯顿

To sum up,
our major contributions are listed as follows:

综上所述，我们的主要贡献如下:

• We
formulate a novel hierarchical user profiling problem, which aims to precisely
model users' multiple level interests simultaneously in E-commerce recommender
systems.

我们提出了一个新的分层用户简档问题，旨在精确地同时建模电子商务推荐系统中用户的多层次兴趣。

• We present
HUP, which exploits a Pyramid Recurrent Neu-ral Networks for hierarchical user
profiling based on users' historical micro-behaviors.

我们介绍HUP，它利用金字塔递归神经网络，根据用户的历史微观行为进行分层用户剖析。

• We propose
Behavior-LSTM, which utilizes a behavior gate to model the types and dwell time
of behaviors for effectively formulating users' real-time interests.

我们提出了行为-LSTM，它利用行为门来模拟行为的类型和停留时间，从而有效地表达用户的实时兴趣。

• We conduct
extensive experiments and prove that our method outperforms state-of-the-art
baselines greatly on both item recommendation and category recommendation
tasks.

我们进行了广泛的实验，证明我们的方法在项目推荐和类别推荐任务上都大大优于最先进的基线。

2 RELATED
WORK

2相关工作

2.1 User
Profiling for Recommendations

2.1推荐的用户概况

Recommender
systems [1] can recommend potentially interested items to users for tackling
the information overload problem.Ex-isting works mainly fall into either
content-based technology [26] or collaborative filtering [23].In both of them,
user profiling plays a critical role in formulating users' interests or
characteristics [5] based on their behaviors in the past [18, 24, 33, 35,
38].Classic col-laborative filtering techniques like matrix factorization [20]
learn users' static profiles from their rating preferences for estimation of
users' interests in the future [38].Furthermore, the evolutionary user
profiling can learn users' dynamic profiles along time based on the time
changing factor model [19], vector autoregression [24], dy-namic sparse topic
model [8], etc.However, these methods mainly focus on the item recommendation
problem, where neither the sequential information of users' behaviors nor the
hierarchy of the user profiles could be considered.

推荐系统[1]可以向用户推荐潜在感兴趣的项目，以解决信息过载问题。现有的作品主要分为基于内容的技术[26]和协同过滤[23]。在这两种情况下，用户简档在根据用户过去的行为[18，24，33，35，38]制定用户的兴趣或特征[5]方面起着至关重要的作用。像矩阵分解[20]这样的经典协作过滤技术从用户的评级偏好中学习用户的静态简档，以估计用户未来的兴趣[38]。此外，基于时变因子模型[19]、向量自回归[24]、动态稀疏主题模型[8]等，进化用户简档可以学习用户沿时间的动态简档。然而，这些方法主要集中在项目推荐问题上，既不能考虑用户行为的顺序信息，也不能考虑用户简档的层次结构。

2.2 RNN-based
User Profiling

2.2基于RNN的用户概况

In
recommender systems, recurrent neural networks (RNN) have shown impressive
advantages by modeling user's sequential behav-iors [14, 15, 17, 29].For
example, Hidasi et al. [14] introduced the concept of session-based
recommendations, and firstly proposed an RNN-based framework to process user's
click sequences on items in a session.Tan et al. [29] further improved its
performance by considering the data augmentation and temporal shift
issues.Hi-dasi et al. [15] integrated some content features extracted from
images and text into parallel RNN architectures, which demon-strated their
significant performance improvements over baselines.Li et al. [22] proposed a
neural attentive recommendation machine that can identify users' main purpose
of their current session tar-geting the performance gains.Beyond behaviors
within a session, Quadrana et al. [27] leveraged an additional GRU layer to
model users' cross-session activities for session-based recommendations.Recently,
it has been found that the temporal information and users' finely-granular
interactions are significantly helpful for recommen-dations.Wu et al. [32]
leveraged timestamps of behaviors with a long short-term memory (LSTM)
autoregressive method.Zhu et al.

在推荐系统中，递归神经网络(RNN)通过模拟用户的顺序行为表现出令人印象深刻的优势[14，15，17，29]。例如，Hidasi等人[14]引入了基于会话的推荐的概念，并首次提出了一个基于RNN的框架来处理用户在会话中对项目的点击序列。Tan等人[29]通过考虑数据增加和时间偏移问题，进一步提高了其性能。Hi-dasi等人[15]将从图像和文本中提取的一些内容特征集成到并行RNN体系结构中，这表明它们的性能比基线有显著提高。李等[22]提出了一种神经注意力推荐机，可以识别用户当前会话的主要目的，从而获得性能提升。除了会话内的行为，Quadrana等人[27]利用额外的GRU层来为基于会话的推荐建模用户的跨会话活动。最近，人们发现时间信息和用户的精细交互对推荐有很大的帮助。Wu等人[32]利用长短期记忆的行为时间戳()自回归方法。朱等。

[43] proposed
Time-LSTM, which used the time gates to model the time intervals between
behaviors.Wan and McAuley [30] ex-ploited the effectiveness of the relations
among users' different types of behaviors in recommendations.Zhou et al. [41]
trained a

[43]提出了时间LSTM，它使用时间门来模拟行为之间的时间间隔。万和麦考利[30]在推荐中探索了用户不同类型行为之间关系的有效性。周等[41]训练了一个

single layer
RNN model with the micro-behaviors for product rec-ommendation.However, this
method only models user's interests in items and just exploits micro behaviors
information as additional input, which might lead to inferior performance.Our
method uses multi-layer Behavior-LSTM cells and attentions to explicitly model
the micro-behaviors information, which can solve both the item recommendation
and the hierarchical categories recommendation problems.

具有产品推荐微观行为的单层RNN模型。然而，这种方法只模拟用户对项目的兴趣，只是利用微观行为信息作为额外的输入，这可能会导致性能下降。该方法利用多层行为LSTM单元和注意力对微观行为信息进行显式建模，既能解决项目推荐问题，又能解决分层类别推荐问题。

In a word,
most existing RNN-based methods fail to address the hierarchical user profiling
problem.In addition, to the best of our knowledge, there are no explorations
that could leverage the types, dwell time and time intervals of the behaviors
simultaneously in an RNN framework for user profiling.

总之，大多数现有的基于RNN的方法都无法解决分层用户配置问题。此外，据我们所知，没有任何探索能够在RNN框架中同时利用行为的类型、停留时间和时间间隔来进行用户剖析。

3 PROBLEM
FORMULATION

3问题表述

In this
section, we firstly introduce the background, notations and definitions in this
paper, and then formulate our problem formally.

在这一部分，我们首先介绍了本文的背景、符号和定义，然后正式阐述了我们的问题。

3.1
Background

3.1背景

Hierarchical
categories organize products of the E-commerce sites in different
granularity.The hierarchy is generally a tree struc-ture, where each lower
level category is an element of a higher level one, and products are usually
hung onto the finest categories as the leaf nodes of the tree.For example, the
first level category "Electronics" might include some second level
categories like "Tele-phone" and "Accessory", and
"Mobile Phone" is a category in the third and finest level belonging
to "Telephone".

分级类别以不同的粒度组织电子商务站点的产品。层次结构通常是一个树形结构，其中每个较低层次的类别都是较高层次的一个元素，产品通常挂在最细的类别上，作为树的叶节点。例如，第一级类别“电子产品”可能包括一些第二级类别，如“电话”和“附件”，“移动电话”是属于“电话”的第三级和最高级类别。

Micro-behaviors
are detailed unit interactions (e.g. reading the detail comments, carting) of
users with recommender systems.They can provide rich information for indicating
users' timely interests, including the type of behavior that a user conducts on
an item, how long a user dwells on an item and move to the next one [30, 41].In
this paper, we consider 10 types of micro behaviors, which are shown in Table
1.

微行为是用户与推荐系统之间的详细单元交互(例如阅读详细评论、搬运)。它们可以提供丰富的信息来指示用户的及时兴趣，包括用户对某个项目的行为类型、用户在某个项目上停留的时间以及移动到下一个项目的时间[30，41]。在本文中，我们考虑了10种微观行为，如表1所示。

Micro
behaviors Description

微观行为描述

Home2Product
Browse the product from the homepage ShopList2Product Browse the product from
the category page Sale2Product Browse the product from the sale page Cart2Product
Browse the product from the carted page SearchList2Product Browse the product
from the searched results Detail_comments Read the comments of the product
Detail_specification Read the specification of the product Detail_bottom Read
the bottom of page of the product Cart Add the product to the shopping cart
Order Make an order

主页2产品从主页浏览产品购物列表2产品从类别页面浏览产品销售2产品从销售页面浏览产品Cart 2产品从card页面浏览产品搜索列表2产品从搜索结果Detail_comments浏览产品阅读产品Detail_specification的注释阅读产品Detail_bottom的规格阅读产品Cart的页面底部将产品添加到购物车订单下订单

Table 1: List
of micro-behaviors

表1:微观行为列表

3.2
Hierarchical User Profiling

3.2分层用户分析

Definition
3.1 (Hierarchical User Profiling).Hierarchical user pro-filing aims to generate
the micro-level, item-level and hierarchical category-level profile vectors pmu
, piu and pcu = {p () cu , ..., p (K) cu } respectively based on her
micro-behaviors, which represent each target user u's interests in
corresponding granularity.

定义3.1(分层用户分析)。分级用户预归档旨在生成微观级别、项目级别和分级类别级别的配置文件向量pmu、piu和pcu = {p()、cu。。。，p (K) cu }分别基于她的微观行为，以相应的粒度表示每个目标用户u的兴趣。

225

Technical
Presentation

技术演示

WSDM '20,
February 3-7, 2020, Houston, TX, USA

WSDM '20，2020年2月3日至7日，美国德克萨斯州休斯顿

Definition
3.2 (Hierarchical Recommendations).Let U be a set of users, V be a set of
items, and C () ,C () , ...,C (K) be the K levels hierarchy of the
categories.The hierarchical recommendations task aims to recommend a set of
items Vˆ u and K set of categories Cˆ () u , ...,Cˆ (K) u to each target user u
by maximizing the relevance between u and her recommendations in different
granularity.

定义3.2(分级建议)。设U为一组用户，V为一组项目，C()，C()，。。。，C (K)是类别的K级层次结构。分层推荐任务旨在推荐一组项目和一组类别。。。，通过最大化u和她在不同粒度的推荐之间的相关性，将c(K)u分配给每个目标用户u。

4 HUP: A
HIERARCHICAL USER PROFILING FRAMEWORK

4 HUP:一个分层的用户配置框架

In this
section, we introduce HUP, a hierarchical user profiling framework for
hierarchical recommendations.As illustrated in Fig-ure 3, HUP utilizes a
Pyramid Recurrent Neural Networks to extract users' hierarchical interests from
micro-behaviors at multiple scales.

在本节中，我们将介绍HUP，一个用于分层推荐的分层用户分析框架。如图3所示，HUP利用金字塔递归神经网络从多尺度的微观行为中提取用户的层次兴趣。

4.1 The Input
and Embedding Layers

4.1输入层和嵌入层

Given a
target user u, the input of our model is a sequence of her micro-behaviors X = ⟨x1,
x2, ..., xN ⟩.The ith element xi = (ti ,vi ,ci , bi , di , дi) indicates that u
performs a micro-behavior of type bi on the item vi at the time ti , where vi
belongs to multiple-level categories ci = {c () i ,c () i , ...,c (K) i }, the
dwell time is di , and the time interval between xi and xi+1 is дi . Here both
dwell time and time interval are real numbers.As previous work did [41], we
discretize them into several buckets respectively for embed-ding.For each
micro-behavior xi , the embedding layer firstly uses embedding tables of items,
categories, behavior types, dwell time buckets and time intervals to transform
vi ,ci , bi , di , дi into low-dimensional dense vectors (i.e., evi ,eci ,ebi
,edi ,eдi ) respectively and then concatenates these vectors into a single
embedding vector ei .The embedding tables are initialized as random numbers.

给定一个目标用户u，我们模型的输入是她的一系列微观行为X = ⟨x1，x2，。。。⟩. xnith元素xi = (ti，vi，ci，bi，di，дi)表示u在时间ti对项目vi执行bi类型的微行为，其中vi属于多级类别ci = { c(I)，c(I)，...，c (K) i }，停留时间为di，xi到xi+1的时间间隔为дi，这里停留时间和时间间隔都是实数。正如前面的工作所做的[41]，我们将它们分别离散成几个桶用于嵌入。对于每个微行为xi，嵌入层首先使用项目、类别、行为类型、停留时间桶和时间间隔的嵌入表将vi、ci、bi、di、дi分别转换成低维密集向量(即evi、eci、ebi、edi、e7i)，然后将这些向量串联成单个嵌入向量ei。嵌入表被初始化为随机数。

4.2 Pyramid
Recurrent Neural Networks

4.2金字塔递归神经网络

Most of
previously recurrent neural networks (RNN)-based recom-mendation methods [5,
14, 15, 29, 41] use a single-layer RNN to generate user profile vectors, which
might not be capable of cap-turing user's hierarchical interests in different
levels.To solve this problem, inspired by the Spatial Pyramid Pooling-net
(SPP-net) [12], we propose a Pyramid Recurrent Neural Networks, which contains
a micro-level, an item-level and several category-level RNN layers to abstract
users' hierarchical interests at multiple scales simulta-neously.

大多数以前基于递归神经网络(RNN)的推荐方法[5，14，15，29，41]使用单层RNN来生成用户简档向量，这可能不能在不同级别上限制用户的分层兴趣。为了解决这个问题，受空间金字塔汇集网(SPP-net) [12]的启发，我们提出了一个金字塔递归神经网络，它包含一个微观层次、一个项目层次和几个类别层次的RNN层，以同时在多个尺度上抽象用户的分层兴趣。

The
micro-level RNN layer aims to model users' finest level in-terests.The input at
the time stamp i of this layer xMi comes from the embedding layer, and the
output of this layer YM is forwarded to the item-level RNN layer for further
calculations.The hidden state is updated after taking each micro-behavior as
input.The for-mulations of the Micro-level RNN layer are defined in Equation 1.

微观层面的RNN层旨在模拟用户最大的兴趣。该层xMi的时间戳I处的输入来自嵌入层，并且该层YM的输出被转发到项目级RNN层用于进一步计算。隐藏状态在将每个微行为作为输入后更新。微观层次RNN层的模拟在方程1中定义。

XM = [xMi ] =
[ei], i = 1, 2, ..., N

XM = [xMi ] =
[ei]，i = 1，2，...，N

YM = [yMi ] =
RN N M (XM ), i = 1, 2, ..., N (1)

YM = [yMi ] =
RN N M (XM)，i = 1，2，...，N (1)

The
item-level RNN layer models users' item-level interests.The input at the time
stamp i of this layer xIi is the concatenation of the item embedding evi and
the output of the micro-level layer.The hidden state is only updated after a
user have transferred her focuses from one item to another.Its output YI is
forwarded to the

项目级RNN层对用户的项目级兴趣进行建模。该层xIi的时间戳I处的输入是嵌入evi的项目和微观层的输出的连接。只有在用户将焦点从一个项目转移到另一个项目后，隐藏状态才会更新。其输出YI被转发到

category-level
RNN layers.The formulations of the Item-level RNN layer are defined in Equation
2.

类别级RNN图层。项目级RNN层的公式在等式2中定义。

XI = [xIi ] =
[evi ;yMi ]

Xi
=[XIi]=[EVI；yMi ]

YI = [yIi ] =
RN NI (XI ) (2)

The
category-level RNN layers formulate users' category-level interests.In the Kth
category layer (the finest granularity of cat-egories), the input at the time
stamp i is x (K) Ci , which is the con-catenation of the category embedding e
(K) ci and the output of the item-level RNN layer calculated on items under
this category.For other higher-level category layers, the input at the time
stamp i of the kth level category layer is X (k) C , which is the concatenation
of the category embedding e (k) ci in this layer and the output of the (k −
1)th level category layer.In each layer, the hidden state is only updated after
a user has moved her focuses from one category to another in this layer.The
formulations of the category-level RNN layers are defined in Equation 3.

类别级RNN层制定用户的类别级兴趣。在第Kth类别层(最细的类别粒度)中，时间戳I处的输入是x (K) Ci，这是嵌入e (K) ci的类别和对该类别下的项目计算的项目级RNN层的输出的组合。对于其他更高级别的类别层，在第k级类别层的时间戳I处的输入是X (k) C，这是该层中嵌入e (k) ci的类别和第(k-1)级类别层的输出的串联。在每一层中，隐藏状态只有在用户将她的焦点从该层中的一个类别移动到另一个类别后才会更新。类别级RNN层的公式在等式3中定义。

X (k) C = [x
(k) Ci ] = ( [e (k) ci ;yIi ]k = K [e (k) ci ;y (k−) Ci ], k = 1, ...,K − 1 Y
(k) C = [y (k) Ci ] = RN N(k) C X (k) C , k = , ...,K

x(k)C
=[x(k)Ci]=([e(k)Ci；yIi]K = K[e(K)ci；y(k)Ci]，k = 1，...，k1 Y(K)C =[Y(K)Ci]= RN N(K)C X(K)C，k =，...，K

(3)

4.3
Behavior-LSTM Cell

4.3行为-LSTM细胞

Generally
users' interests are dynamic and continuously shifting.Time-LSTM [43] is a
state-of-the-art method that incorporates time intervals between users'
sequential purchases to address the interest shifting problem.However, it
cannot model the behavior type and the dwell time information, which may lead
to inferior performance.We here propose Behavior-LSTM, a novel RNN layer that
provides an additional behavior gate to process the types and dwell time of the
behaviors, enabling HUP to track users' real-time interests more precisely.In
particular, it is described in Figure 4 and formulated in Equation 4:

一般来说，用户的兴趣是动态的，不断变化的。时间-LSTM [43]是一种最先进的方法，它结合了用户连续购买之间的时间间隔，以解决利益转移问题。但是，它无法对行为类型和停留时间信息进行建模，这可能会导致性能下降。我们在这里提出了行为-LSTM，一个新的RNN层，它提供了一个额外的行为门来处理行为的类型和停留时间，使HUP能够更精确地跟踪用户的实时兴趣。具体来说，它在图4中描述，并在等式4中表示:

It = σ
(WI[ht−1, xt ] + bI) Ft = σ (WF[ht−1, xt ] + bF) Tt = σ (WT[xt , ∆t ] + bT) At
= σ (WA[xt , at ] + bA) C˜ t = tanh(WC[ht−1, xt ] + bC) Ct = Ft ⊙ Ct−1 + It ⊙
Tt ⊙ At ⊙ C˜ t Ot = σ (WO[ht−1, xt ] + bO) ht = Ot ⊙ tanh(Ct )

It =σ(WI[ht 1，XT]+bI)Ft
=σ(WF[ht 1，xt ] + bF) Tt = σ (WT[xt，t ] + bT) At = σ (WA[xt，At]+Ba)c≘t = tanh(WC[ht 1，XT]+bC)Ct = Ft⊙Ct 1+It⊙TT⊙At⊙c≘t Ot =σ(WO[ht 1

(4)

where I, F,
T, A and O are the input, forget, time, behavior and output gates, C and h are
the cell state and hidden state vectors, WI,WF,WT,WA,WC andWO are weight
matrices, bI, bF, bT, bA, bC and bO are the biases, respectively.The input of
the Behavior-LSTM is a tuple (xt , at , ∆t ), where xt is the embedding vector
of the input at the time stamp t, at is the embedding vector of the behavior
type or dwell time information, and ∆t is the embedding vector of time interval
between current behavior and the next one.

其中I、F、T、A和O是输入、遗忘、时间、行为和输出门，C和h是单元状态和隐藏状态向量，WI、WF、WT、WA、WC和O分别是权重矩阵，bI、bF、bT、bA、bC和bO分别是偏差。行为-LSTM的输入是一个元组(xt，at，∏t)，其中xt是输入在时间戳t的嵌入向量，at是行为类型或停留时间信息的嵌入向量，而∏t是当前行为和下一个行为之间的时间间隔的嵌入向量。

In
Behavior-LSTM, the time gate T estimates how much informa-tion that should
maintain or pass to the next state, and the behavior gate A calculates the
importance of current behavior with the meta information of the behavior.In
particular, such meta information of the behaviors involves two aspects: their
types and users' dwell time.In particular, the behavior gate actually only
processes the types of micro-behaviors in the micro level RNN layer.It is
because

在行为-LSTM中，时间门T估计有多少信息应该保持或传递到下一个状态，行为门A用行为的元信息计算当前行为的重要性。特别地，这些行为的元信息涉及两个方面:它们的类型和用户的停留时间。特别是，行为门实际上只处理微观层次RNN层中的微观行为类型。这是因为

226

Technical
Presentation

技术演示

WSDM '20,
February 3-7, 2020, Houston, TX, USA

WSDM '20，2020年2月3日至7日，美国德克萨斯州休斯顿

Figure 3: The
architecture of the HUP.It uses a Pyramid Recurrent Neural Networks, which is
consisted of a micro layer, an item layer, and hierarchical category recurrent
neural networks layers, to extract users' hierarchical profile at multiple
scales.The profiles represent users' real-time interests in items and
hierarchical categories, based on which the most relevant categories and items
can be recommended to users.

图HUP的架构。它使用由微层、项目层和分层类别递归神经网络层组成的金字塔递归神经网络在多个尺度上提取用户的分层简档。简档表示用户对项目和分层类别的实时兴趣，基于此可以向用户推荐最相关的类别和项目。

Figure 4: The
architecture of the Behavior-LSTM.It has a behavior gate A and a time gate T,
where A models users' behavior information in micro behaviors, and T captures
the time intervals between users' micro behaviors.

图4:行为LSTM的架构。它有一个行为门A和一个时间门T，其中A在微观行为中建模用户的行为信息，T捕捉用户微观行为之间的时间间隔。

most of
micro-behaviors are instant responses and we could not get their dwell time,
but their types are extremely important for users' interest modeling.In the
item-level and hierarchical category-level RNN layers, this gate models the
dwell time on the items or categories.That is because the dwell time varies
significantly in items and categories and is very informative in presenting
users' interests.

大多数微行为都是即时响应，我们无法得到它们的停留时间，但是它们的类型对于用户兴趣建模是极其重要的。在项目级和分级类别级RNN图层中，该门对项目或类别的停留时间进行建模。这是因为停留时间在项目和类别上有很大的不同，并且在呈现用户兴趣方面非常有用。

4.4 The
Attention Layers

4.4关注层

The attention
mechanism [2] is a common technique in deep learn-ing.Usually, it is able to
mitigate long-term dependency issues as well as provide interpretations, which
is extremely important in real-world recommender systems.In particular, an
attention layer

注意机制[2]是深度学习中常见的技术。通常，它能够减轻长期依赖问题并提供解释，这在现实世界的推荐系统中非常重要。特别是关注层

takes the
output sequence Y = [y1,y2, ...,yT ] of an RNN as input and return a context
vector s. Let yi be a user's interests at time stamp i. The context vectors of
each attention layer is calculated as a weighted sum of the interests vectors
among all the time stamps, which is formulated formally in Equation 5.

取输出序列Y = [y1，y2，...假设yi是用户在时间戳I的兴趣。每个关注层的上下文向量被计算为所有时间戳中的兴趣向量的加权和，这在等式5中正式表述。

s = ÕTi=1 α i
y i ;α i = exp ( e i ) Í Tk = 1 exp ( e k ) ;e i = f ( y i y T a i )

s = Ti = 1αI
y I；αI = exp(e I)íTk = 1 exp(e k)；e i = f ( y i y T a i)

(5)

HUP has
multiple attention layers, where each is directly fol-lowed by its
corresponding RNN layer and therefore referred to as micro, item and category
level attention layers respectively.The context vectors from these attention
layers are denoted as sm, si and sc = {s () c ,s () c , ...,s (K) c }
respectively.The attention signal ai represents the type of micro-behaviors in
micro-level attention layer, and the dwell time in both the item and the
category level attention layers.f is an alignment model, which scores the
impor-tance of yi based on the hidden state yi , last hidden state yT and attention
signal ai .In order to achieve abundant expressive ability, we design the
alignment model f as two-layers feedforward neural networks, which is jointly
trained in the model.

HUP有多个关注层，每个关注层直接跟随其对应的RNN层，因此分别被称为微观、项目和类别级别的关注层。来自这些关注层的上下文向量被表示为sm、si和sc = { s(c)、s(c ),...，s (K) c }分别为。注意信号ai表示微观层次注意层中微观行为的类型，以及在项目和类别层次注意层中的停留时间。f是一个对齐模型，它根据隐藏状态yi、最后一个隐藏状态yT和注意信号ai对yi的重要性进行评分。为了获得丰富的表达能力，我们将对齐模型f设计为两层前馈神经网络，在模型中联合训练。

4.5 The Fully
Connected Layers

4.5完全连接的层

The fully
connected neural network layers transform users' con-text vectors from the
attention layers into hierarchical user pro-files.Specifically, they transform
users' micro-level, item-level and category-level context vectors sm, si and sc
= {s () c ,s () c , ...,s (K) c } into real-time user profile vectorspm,pi
andpc = {p () c , p () c , ..., p (K) c } in corresponding levels.

完全连接的神经网络层将用户的文本向量从注意力层转换成分层的用户文件。具体来说，它们转换用户的微观级别、项目级别和类别级别的上下文向量sm、si和sc = { s(c)、s(c ),...，s (K) c }转换为实时用户配置文件向量，pi和c = { p(c)，p(c)、...，p (K) c }在相应的级别。

227

Technical Presentation

技术演示

WSDM '20,
February 3-7, 2020, Houston, TX, USA

WSDM '20，2020年2月3日至7日，美国德克萨斯州休斯顿

4.6 Loss
Function

4.6损失函数

Deep learning
models like convolutional neural networks [21] and recurrent neural networks
[9] usually use softmax as the last layer for prediction.However, in real-world
recommendation scenarios, the possible items can be millions or billions, and
thus such calcu-lations on all items is prohibitively expensive.Given a user u
and her sequential activities Xu , we try to maximize the cosine simi-larity
between the user's real-time profile vectors (i.e. pm, pi and pc = {p () c ,
..., p (K) c }) and the embedding of the ground-truths, on which the target
user will act in the next time stamp N + 1(i.e., the next itemvN +1 for micro
and item level layers or the next hierarchi-cal categories cN +1 = {c () N +1 ,
...,c (K) N +1 } for category-level layers).Similar strategy has achieved
success in recommendation systems

像卷积神经网络[21]和递归神经网络[9]这样的深度学习模型通常使用softmax作为预测的最后一层。然而，在现实世界的推荐场景中，可能的项目可能是数百万或数十亿，因此对所有项目的这种计算是极其昂贵的。给定用户u和她的顺序活动Xu，我们试图最大化用户的实时简档向量(即pm、pi和PC = { p()} c，...p (K) c })和基础事实的嵌入，目标用户将在下一个时间戳N + 1(即，对于微观和项目级层的下一个项目vN +1或下一个分层类别cN+1 = { c(N+1，...，c (K) N +1 }用于类别级层)。类似的策略在推荐系统中取得了成功

[, , ].Let
LMu , LIu and LCu = {L () Cu ...L (K) Cu } be the losses of the micro-level,
item-level and category-level layers for the target user u. The loss of the
micro-level layers LMu can be calculated as Equation 6, where evN +1 is the
embedding of the ground-truth item vN +1.The losses of the item and category
levels can be calculated similarly.

[, , ].让LMu、LIu和LcU = { L()}
Cu。。。L (K) Cu }是目标用户u的微观层、项目层和类别层的损失。微观层LMu的损失可以计算为等式6，其中evN +1是基础事实项目vN +1的嵌入。项目和类别级别的损失可以类似地计算。

LMu =
cosine_proximity(pmevN +) = − pm evN +∥pm ∥ ∥evN +1 ∥ (6) The total loss L is the
weighted sum of losses in micro-level, item-level and category-level layers of
all users.Formally it is defined as follows:

LMu =余弦_邻近度(PME VN+)=
pm evN+∑pm∑evN+1∑(6)总损失L是所有用户的微观层、项目层和类别层损失的加权和。正式定义如下:

L = λlM Õ

L = λlM

u ∈U

LMu + λlI Õ

LMu + λlI

u ∈U

LIu + λlC Õ

LIu + λlC

u ∈U

k=1

L (k) Cu (7)

低钾铜(7)

where λlM ,
λlI and λlC are the coefficients of the losses in the micro-level, item-level
and multiple category-level layers respectively.

其中λlM、λlI和λlC分别是微观层次、项目层次和多类别层次的损失系数。

5
EXPERIMENTAL SETTINGS

5实验设置

5.1
Hierarchical Recommendations

5.1分级建议

We evaluate
our proposed HUP method on two tasks: item recom-mendations and category
recommendations.Given a target user u and a sequence of her micro-behaviors,
HUP generates a hierar-chical profile vectors for u, which represent the user's
interests in items and hierarchical categories respectively.At the same time,
the embedding vectors of the items and multiple-level categories can be learned
from HUP as well during the training stage.The item recommendation process is
as follows.At each recommendation stage, as previous work did [41], we first
retrieve a set of candidate items, which are similar to at least one of users'
browsed items in terms of cosine similarity on embeddings.We then calculate the
cosine similarity between each candidate item embedding and the user's
item-level profile vector pi as ranking score.Finally, we rank the candidate
items and select top items for recommendations.The category recommendations are
performed in a similar manner.

我们在两个任务上评估我们提出的HUP方法:项目推荐和类别推荐。给定一个目标用户u和她的一系列微行为，HUP为u生成一个hierar-chical简档向量，分别表示用户对项目和层次类别的兴趣。同时，在训练阶段，也可以从HUP中学习项目和多级类别的嵌入向量。物品推荐流程如下。在每个推荐阶段，正如之前的工作所做的那样[41]，我们首先检索一组候选项，这些候选项在嵌入的余弦相似性方面与至少一个用户浏览的项相似。然后，我们计算每个候选项目嵌入和用户的项目级配置文件向量pi之间的余弦相似度作为排名分数。最后，我们对候选项目进行排名，并选择推荐的顶级项目。类别推荐以类似的方式执行。

5.2 Dataset

5.2数据集

To evaluate
the effectiveness of HUP, we utilize the benchmark "JD Micro Behaviors
Datasets" [41], which are collected from a large e-commerce site JD.com
.The datasets contain users' micro-behaviors in two product categories
"Appliances" and "Computers", where each line is a sequence
of a user's micro behaviors in a session.

为了评估HUP的有效性，我们利用了从JD.com一个大型电子商务网站收集的基准“JD微行为数据集”[41]。数据集包含用户在两个产品类别“设备”和“计算机”中的微观行为，其中每一行都是用户在会话中的微观行为序列。

The statistics
of the datasets are shown in Table 2.In each dataset, we sort all the sessions
in chronological order, and use 70%, 10%, 20% sessions as the training,
validation and testing set respectively.As previous work did [41] , the last
item and the corresponding finest category in each session are used as ground
truth.

数据集的统计数据如表2所示。在每个数据集中，我们按照时间顺序对所有会话进行排序，并分别使用70%、10%、20%的会话作为训练集、验证集和测试集。正如之前的工作所做的[41]，每个会话中的最后一项和相应的最佳类别被用作基础事实。

Dataset
JD-Applicances JD-Computers Users 6,166,916 3,191,573 Products 169,856 419,388
Categories 103 93 Number of Micro behaviors 176,483,033 88,766,833 Table 2:
Statistics of the Datasets

数据集JD-应用JD-计算机用户6，166，916 3，191，573产品169，856 419，388类别103 93微观行为数量176，483，033 88，766，833表2:数据集统计

5.3 Baseline
Methods

5.3基线方法

We make a
comparative study of our approach HUP with the follow-ing methods, where the
last three are state-of-the-art RNN-based methods that have demonstrated
excellent performance recently.

我们对我们的HUP方法和后续方法进行了比较研究，其中后三种方法是基于RNN的最先进的方法，最近表现出了优异的性能。

• POP
recommends the most popular items to each user.This simple method is a common
mechanism in recommender sys-tems.This simple method has been proven to be
comparable to some sophisticated recommender algorithms [6].

POP向每个用户推荐最受欢迎的商品。这种简单的方法是推荐系统中常见的机制。这种简单的方法已经被证明可以与一些复杂的推荐算法相媲美[6]。

• BPR-MF
implements matrix factorization with the Bayesian personal ranking loss.It is
one the most popular methods for recommendations [13, 20, 28].

BPR-MF利用贝叶斯个人排名损失实现矩阵分解。这是最受欢迎的推荐方法之一[13，20，28]。

• Item-KNN is
a popular item-based recommender algorithm that uses similarities between items
for recommendations

项目-KNN是一种流行的基于项目的推荐算法，它使用项目之间的相似性进行推荐

[7].In
particular, the similarity is calculated with sim(i, j) =

[7].特别地，相似度是用sim(i，j) =计算的

F r eq(ij)

F r eq(i)×F r
eq(j) , where Freq(i) is the number of sequences that an item i shows up [7].

F r eq(i)×F r
eq(j)，其中F r eq(i)是一个项目I出现的序列数[7]。

• Word2vec
makes recommendations based on embedding of the last item in the sequence [10]
by Word2vec [25].It has been proved to be effective in recommendation [10].

Word2vec根据Word2vec
[25]在序列[10]中嵌入的最后一项提出建议。在建议[10]中已经证明是有效的。

•
Word2vec-avg makes recommendations based on the aver-age embedding of all items
in the sequence [41].• RIB [41] is a state-of-the-art method that uses RNN and
the

Word2vec-avg根据序列中所有项目的平均嵌入量提出建议[41]。RIB [41]是一种使用RNN和

attention
mechanism to model user's micro-behaviors for recommendation .

为推荐而模拟用户微观行为的注意机制。

• Time-LSTM
[43] integrates the time interval information between user's item-level
behaviors into LSTM.• S-HRNN [27] utilizes a hierarchical GRUs to model users'
interactions across sessions.

时间-LSTM [43]将用户项目级行为之间的时间间隔信息集成到LSTM。S-HRNN [27]利用一个分层的GRUs来模拟用户跨会话的交互。

5.4
Evaluation Metrics

5.4评估指标

We use two
widely used metrics Recall@K and MRR@K [14, 27, 29, 41] to compare our model
with the baselines.For the item recom-mendation problem, as previous work did
[41], we use Recall@20 and MRR@20 for evaluation.For category recommendation,
we use Recall@5 and MRR@5 instead because user's interests in categories are
relatively stable.We have implemented our framework HUP with Keras 2.2.The
embedding size of items behaviors, categories, dwell time and time intervals
are set to 30, 5, 8, 5 and 5 respectively, the batch size is 128 and the hidden
size of the PRNN layers is 100.

我们使用两个广泛使用的指标Recall@K和MRR@K [14，27，29，41]来比较我们的模型和基线。对于项目推荐问题，和以前的工作一样[41]，我们使用Recall@20和MRR@20进行评估。对于类别推荐，我们改用Recall@5和MRR@5，因为用户对类别的兴趣相对稳定。我们已经用Keras 2.2实现了我们的框架HUP。项目行为、类别、停留时间和时间间隔的嵌入大小分别设置为30、5、8、5和5，批量大小为128，PRNN层的隐藏大小为100。

228

Technical
WSDM '20, Presentation February 3-7, 2020, Houston, TX, USA

技术WSDM 20，演示2020年2月3日至7日，美国德克萨斯州休斯顿

Model

模型

Applicances
Computers

应用计算机

Item Rec
Category Rec Item Rec Category Rec

项目建议类别建议项目建议类别建议

Recall@20
MRR@20 Recall@5 MRR@5 Recall@20 MRR@20 Recall@5 MRR@5

回忆@20 MRR@20回忆@5 MRR@5回忆@20 MRR@20回忆@5 MRR@5

POP 3.1 0.5
45.0 24.0 3.4 1.0 44.0 28.6

持久性有机污染物3.1 0.5 45.0 24.0 3.4 1.0 44.0 28.6

BPR-MF 13.1
3.1 55.4 35.0 11.3 3.0 70.1 42.9

Item-KNN 42.9
9.6 87.0 43.1

项目-KNN

Word2vec 38.5
8.8 91.1 90.6 28.4 6.2 84.1 81.6

word 2 vec
38.5 8.8 91.1 90.6 28.4 6.2 84.1 81.6

Word2vec-avg
38.7 13.1 86.7 80.0 24.4 7.1 81.0 71.5

word 2
vec-avg 38.7 13.1 86.7 80.0 24.4 7.1 81.0 71.5

RIB 47.6 14.3
92.9 91.2 28.6 7.6 88.0 83.0

Time-LSTM
49.4 18.9 93.4 91.3 32.8 10.9 88.7 83.9

时间-LSTM 49.4 18.9 93.4 91.3 32.8 10.9 88.7 83.9

29.8

6.8 68.8 32.7

S-HRNN 49.8
19.2 92.6 90.4 33.0 11.0 88.2 82.9

南HRNN 49.8 19.2 92.6 90.4 33.0 11.0 88.2 82.9

HUP 51.520.593.891.635.012.089.284.4

HUP 51 . 520
. 593 . 891 . 635 . 012 . 089 . 284 . 4

HUP-NoMicro
49.6 19.3 93.1 91.1 32.6 10.6 88.2 83.6

HUP-LSTM 50.1
19.6 93.2 91.3 32.7 10.7 88.3 83.6

湖北-LSTM 50.1 19.6 93.2 91.3 32.7 10.7 88.3 83.6

HUP-TLSTM
50.9 20.0 93.5 91.3 33.8 11.3 88.8 84.0

HUP-NoAtt
50.3 19.7 93.6 91.5 33.8 11.4 89.1 84.2

HUP-Single
50.5 19.7 93.4 91.5 33.9 11.4 88.6 83.8

HUP-单身50.5 19.7
93.4 91.5 33.9 11.4 88.6 83.8

Table 3:
Performance of different methods for category recommendation and item
recommendation on two datasets."" indi-cates the statistically
significant improvements (i.e., two-sided t -test with p < 0.01) over both
the best baseline and all variants.

表3:两个数据集上类别推荐和项目推荐不同方法的性能。""表明在最佳基线和所有变量上的统计学显著改善(即，双侧t检验，p < 0.01)。

6
EXPERIMENTAL RESULTS 6.1 Comparison with Baselines

6实验结果6.1与基线的比较

Table 3 shows
the experimental results of different methods for the item and category
recommendation tasks on the Applicances and Computers datasets.We conducted
significance testing (t-test) on the improvements of our approaches over all
baselines."" denotes strong significant divergence with
p-value<0.01.From the table we can find that:

表3显示了应用程序和计算机数据集上项目和类别推荐任务的不同方法的实验结果。我们对我们的方法在所有基线上的改进进行了显著性测试。""表示p值< 0.01的强显著差异。从表中我们可以发现:

• HUP
significantly outperforms state-of-the-art methods for the two tasks on both
datasets.Specifically, for item rec-ommendation, HUP outperforms
state-of-the-art method by 3.4%, 6.1% in Recall@20 and 6.7%, 9.1% in MRR@20 for
the "Appliances" and "Computers" datasets respectively.For
cate-gory recommendation, our performance gains are relatively subtle, as this
problem is easier than item recommendation resulting from the denser dataset.

在这两个数据集上，HUP在这两项任务上的表现明显优于最先进的方法。具体来说，对于项目推荐，对于“设备”和“计算机”数据集，HUP在召回@20中的表现优于最先进的方法3.4%、6.1%，在MRR@20中的表现优于最先进的方法6.7%、9.1%。对于种类丰富的推荐，我们的性能提升相对微妙，因为这个问题比密集数据集产生的项目推荐更容易。

• The POP and
BPR-MF methods perform the worst.• Three RNN-based methods including RIB,
Time-LSTM and

持久性有机污染物方法和业务流程再造-多功能方法表现最差。三种基于RNN的方法，包括RIB、时代LSTM和

S-HRNN
significantly overcome conventional baselines.• By modeling the temporal
information, Time-LSTM achieves better performance than RIB.

HRNN大大超越了常规基线。通过对时间信息进行建模，时间LSTM获得了比RIB更好的性能。

6.2
Effectiveness of Components in HUP

6.2 HUP中组件的有效性

To
systematically validate the effectiveness of each component in HUP, we
implement the following variants of HUP, each eliminating a specific model
component.

为了系统地验证HUP中每个组件的有效性，我们实现了HUP的以下变体，每个变体都消除了一个特定的模型组件。

•
HUP-NoMicro.This variant does not use micro behaviors for modeling.It only uses
the item-level and category-level RNN layers based on users' interactions with
items and cat-egories.

HUP-NoMicro。此变体不使用微观行为进行建模。它只使用基于用户与物品和个人交互的物品级和类别级RNN层。

•
HUP-LSTM.This variant uses LSTM in the PRNN layers.The time-related mechanism
and type of micro-behaviors are absent in the method.

HUP-LSTM。这个变体在PRNN图层中使用了LSTM。该方法缺乏与时间相关的微观行为机制和类型。

•
HUP-TLSTM.This variant uses Time-LSTM [43] in the PRNN layers, where only the
time gates are used in the RNN layer to model the time interval among
behaviors.

HUP-TLSTM。该变体在PRNN层中使用时间LSTM
[43]，其中在RNN层中仅使用时间门来建模行为之间的时间间隔。

•
HUP-NoAtt.This variant removes the attention layers from HUP.

HUP-NoAtt。这个变体从HUP中移除了关注层。

•
HUP-Single.This variant solves each recommendation task independently, which
includes a single Behavior-LSTM layer and an attention layer in the framework.

HUP-单身。这个变体独立地解决每个推荐任务，它包括框架中的单个行为LSTM层和注意层。

The performance
of different variants are shown in the bottom part of Table 3.From the table,
we can see that the full version of HUP outperforms all of the variants and
find that:

表3的底部显示了不同变体的性能。从表中，我们可以看到完整版本的HUP优于所有变体，并发现:

(1) Pyramid
Recurrent Neural Networks.Comparing with the HUP-Single method, the performance
improvement demon-strate the effectiveness of PRNN.

(1)金字塔递归神经网络。通过与HUP-Single方法的比较，性能的提高证明了PRNN方法的有效性。

(2)
Micro-behaviors.The comparison with HUP-NoMicro demon-strates the importance of
micro-behaviors in HUP.We also notice that HUP-NoMicro obtains the worst
performance in all metrics.

(2)微观行为。与HUP-NoMicro的比较表明了微观行为在HUP中的重要性。我们还注意到HUP-NoMicro在所有指标中表现最差。

(3) Temporal
mechanisms.Evidenced by the performance loss of HUP-LSTM against HUP-TLSTM, and
HUP, temporal in-formation is necessary in modeling user interests.Furthermore,
sophisticated temporal mechanisms could receive improved perfor-mance.For
example, equipped with time gates, HUP-TLSTM can achieve better performance
than HUP-LSTM;HUP outperforms HUP-TLSTM by further using Behavior gates and
time-mechanisms in the attention layers.

(3)时间机制。从HUP-LSTM相对于HUP-TLSTM和HUP的性能损失可以看出，在建模用户兴趣时，时间信息是必要的。此外，复杂的时间机制可以获得更好的性能。例如，配备时间门，HUP-TLSTM可以实现比HUP-LSTM更好的性能；通过在注意层进一步使用行为门和时间机制，HUP优于HUP-TLSTM。

(4) Attention
layers.As demonstrated in our experiments, at-tention layers can significantly
improve the performance of HUP against the HUP-NoAtt variant.Meanwhile,
attention mechanisms also help interpret and visualize the recommendation
results as well.We will show that in the later case study section.

(4)注意层次。正如我们在实验中所证明的，张力层可以显著提高HUP对HUP-NoAtt变体的性能。同时，注意力机制也有助于解释和可视化推荐结果。我们将在后面的案例研究部分展示这一点。

6.3 Trade-off
in Loss Function

6.3损失函数的权衡

As formulated
in Equation (7), the loss function is composed of three components.According to
our experiments, HUP achieves

如公式(7)所示，损耗函数由三个部分组成。根据我们的实验，HUP实现了

229

Technical
Presentation

技术演示

WSDM '20,
February 3-7, 2020, Houston, TX, USA

WSDM '20，2020年2月3日至7日，美国德克萨斯州休斯顿

Figure 5: A
case study in an E-commerce Recommender Systems

图5:一个电子商务推荐系统的案例研究

Figure 6:
Performance of HUP with λlI which is the weight of the losses of item-level RNN
layer.

图6:带有λlI的HUP的性能，λlI是项目级RNN层损失的权重。

the best
performance when λlM = .Because the micro-level RNN layer is useful for
predicting the next micro-behaviors based on the historical ones, but does not
directly affect the predictions of the items and categories.However, this layer
can be used to inter-pret the real-time effectiveness of the historical
micro-behaviors for each user.Without loss of generality, we set λlC = − λlI
and ≤ λlI ≤ .Therefore we only need to tune λlI in the loss func-tion for a
trade-off between item and category recommendation tasks.Figure 6 shows the
performance of HUP with respect to dif-ferent values of λlI on the
"Applicances" dataset.The curves on the "Computers" dataset
are similar and thus absent from this paper for satisfying the requirement of
the page limit.From the figure we can see that the metrics for item
recommendation decline when λlI is getting lower while the trends are
completely opposite for category

λlM =时的最佳性能。因为微观层面的RNN层对于基于历史行为预测下一个微观行为是有用的，但是不直接影响项目和类别的预测。然而，该层可用于解释每个用户的历史微观行为的实时有效性。不失一般性，我们设置λLc =λLiI且≤λLiI≤。因此，我们只需要在损失函数中调整λlI，以便在项目和类别推荐任务之间进行权衡。图6显示了在“应用程序”数据集上，相对于λlI的不同值，HUP的性能。“计算机”数据集上的曲线是相似的，因此本文中没有满足页数限制要求的曲线。从图中我们可以看出，当λlI变低时，项目推荐的指标下降，而类别的趋势完全相反

recommendation.To
balance the performance of the user profiles in different levels, we set λlI
and λlC both to 0.5 in the experiments.

建议。为了平衡不同级别的用户配置文件的性能，我们在实验中将λlI和λlC都设置为0.5。

6.4 Case
Study

6.4案例研究

Figure 5
demonstrates a real case from the "Appliances" dataset to explain how
HUP works.The last 12 micro-behaviors on 7 items from 6 categories are listed
in the figure.The last item is the ground-truth, which spans 2
micro-behaviors.The right side of the figure visualizes the attention weights
of these micro-behaviors from our proposed HUP, HUP-TLSTM (a variant of HUP)
and RIB (a state-of-the-art baseline).From the figure we can see that:

图5展示了一个来自“设备”数据集的真实案例来解释HUP是如何工作的。图中列出了6类7个项目的最后12个微行为。最后一项是基础事实，它跨越了两个微观行为。图的右侧可视化了我们提出的HUP、HUP-TLSTM(HUP的变体)和RIB(最先进的基线)的这些微观行为的注意力权重。从图中我们可以看出:

(1) The
attention weights of the micro behaviors "Cart" and
"Search2Product" are higher than others for all methods, which means
these two micro behaviors are important for modeling user interests.

(1)微行为“Cart”和“Search2Product”的关注权重对于所有方法都高于其他方法，这意味着这两个微行为对于建模用户兴趣很重要。

(2) The time
interval between the browsing behaviors on the item "Bear Electric
Kettle" and the next one is 36 seconds.As the attention weights shown,
HUP-TLSTM and HUP pay much less attentions to the first 4 items than RIB
resulting from the time gates.It illustrates their ability of forgetting
history behaviors happened long time ago by modeling the time interval
information.

(2)“小熊电水壶”项目上的浏览行为与下一次浏览行为的时间间隔为36秒。如关注权重所示，HUP-TLSTM和HUP对前4个项目的关注度远远低于时间门导致的RIB。通过对时间间隔信息的建模，说明了他们遗忘很久以前发生的历史行为的能力。

(3) Time
interval between the browsing behaviors on "Yogurt Maker and Ice Cream
Machine" and "Bear Egg Cooker" is merely 2 seconds.This number
between "Bear Egg Cooker" and the next item (ground-truth) is also 2
seconds.Both are very short.HUP-TLSTM retains such history information and
still pays much attention to these two items resulting from the short time
interval.However, HUP can notice the fact that the user has already added these
two items to cart.It thus reduces the importance on these two items and their
categories, and then chooses an item from a related category (Yogurt Maker and
Toaster Bundle) for return.

(3)在“酸奶机和冰淇淋机”和“熊蛋煲”上浏览行为的时间间隔仅为2秒。“熊蛋煲”和下一个物品(地面-真相)之间的这个数字也是2秒。两者都很短。HUP-TLSTM保留了这样的历史信息，并且由于时间间隔短，仍然非常关注这两个项目。但是，HUP可以注意到用户已经将这两个项目添加到购物车中。因此，它降低了这两个项目及其类别的重要性，然后从相关类别(酸奶制造商和烤面包机捆绑包)中选择一个项目进行退货。

230

Technical
Presentation

技术演示

WSDM '20,
February 3-7, 2020, Houston, TX, USA

WSDM '20，2020年2月3日至7日，美国德克萨斯州休斯顿

7 CONCLUSIONS

7结论

In this
paper, we investigate the hierarchical user profiling problem, aiming to model
users' real-time interests in different granularity.It is crucial for
multiple-level recommendation tasks, such as item, category, topic, theme
recommendations and so on.We hence pro-pose HUP, a hierarchical user profiling
framework, which leverages a Pyramid Recurrent Neural Networks to abstract
users' interests in different granularity simultaneously from users'
micro-behaviors.To better model users' real-time interests, we design
Behavior-LSTM cells to integrate the meta information of behaviors (e.g. the
type, dwell time and time interval information) into HUP.Exten-sive experiments
on two real-world E-commerce datasets verify the effectiveness of our method
for both item and category recom-mendation tasks.

在本文中，我们研究了分层用户剖析问题，旨在对不同粒度的用户实时兴趣进行建模。对于项目、类别、主题、主题推荐等多层次推荐任务来说至关重要。因此，我们提出了HUP，一个分层的用户剖析框架，它利用金字塔递归神经网络从用户的微观行为中同时提取不同粒度的用户兴趣。为了更好地模拟用户的实时兴趣，我们设计了行为LSTM单元，将行为的元信息(例如类型、停留时间和时间间隔信息)集成到HUP中。在两个真实的电子商务数据集上的广泛实验验证了我们的方法对于项目和类别推荐任务的有效性。

Resulting
from its effectiveness and flexibility, our framework can be widely used to
recommend items (e.g. movies, music, news) and corresponding categories (e.g.
science fiction films, rock music, breaking news) in various web services (e.g.
videos or music sharing sites, social networks).

由于其有效性和灵活性，我们的框架可以广泛用于在各种网络服务(例如视频或音乐共享网站、社交网络)中推荐项目(例如电影、音乐、新闻)和相应的类别(例如科幻电影、摇滚乐、突发新闻)。