Hierarchical User Profiling for E-commerce Recommender Systems
分层用户兴趣的电商推荐系统
Yulong Gu
顾玉龙
Data Science Lab,
JD.com guyulongcs@gmail.com
JD.com·guyulongcs@gmail.com数据科学实验室
Shuaiqiang
Wang
帅强王
Data Science
Lab, JD.com wangshuaiqiang1@jd.com
JD.com·wangshuaiqiang1@jd.com数据科学实验室
ABSTRACT
摘要
Hierarchical
user profiling that aims to model users' real-time in-terests in different
granularity is an essential issue for personal-ized recommendations in
E-commerce.On one hand, items (i.e. products) are usually organized
hierarchically in categories, and correspondingly users' interests are
naturally hierarchical on dif-ferent granularity of items and categories.On the
other hand, mul-tiple granularity oriented recommendations become very popular
in E-commerce sites, which require hierarchical user profiling in different
granularity as well.In this paper, we propose HUP, a Hierarchical User
Profiling framework to solve the hierarchical user profiling problem in
E-commerce recommender systems.In HUP, we provide a Pyramid Recurrent Neural
Networks, equipped with Behavior-LSTM to formulate users' hierarchical real-time
in-terests at multiple scales.Furthermore, instead of simply utilizing users'
item-level behaviors (e.g., ratings or clicks) in conventional methods, HUP
harvests the sequential information of users' tem-poral finely-granular
interactions (micro-behaviors, e.g., clicks on components of items like
pictures or comments, browses with nav-igation of the search engines or
recommendations) for modeling.Extensive experiments on two real-world
E-commerce datasets demonstrate the significant performance gains of the HUP
against state-of-the-art methods for the hierarchical user profiling and
recommendation problems.We release the codes and datasets at
https://github.com/guyulongcs/WSDM2020_HUP.
旨在以不同粒度对用户的实时兴趣进行建模的分层用户简档是电子商务中个性化推荐的一个重要问题。一方面，项目(即产品)通常是按类别分层组织的，相应地，用户的兴趣在项目和类别的不同粒度上自然是分层的。另一方面，面向多粒度的推荐在电子商务网站中变得非常流行，这些网站也需要不同粒度的分层用户配置。在本文中，我们提出了HUP，一个分层的用户剖析框架来解决电子商务推荐系统中的分层用户剖析问题。在HUP中，我们提供了一个配备了行为LSTM的金字塔递归神经网络，以在多个尺度上表达用户的分层实时兴趣。此外，代替在传统方法中简单地利用用户的项目级行为(例如，评级或点击)，HUP收集用户的临时精细粒度交互的顺序信息(微行为，例如，点击像图片或评论这样的项目的组成部分，浏览搜索引擎的导航或推荐)用于建模。在两个真实的电子商务数据集上进行的大量实验表明，对于分层用户剖析和推荐问题，HUP相对于最先进的方法有显著的性能提升。我们在https://github.com/guyulongcs/WSDM2020_HUP.发布代码和数据集
CCS CONCEPTS
CCS CONCEPTS
•Information
systems→Personalization;Recommender sys-tems.
信息系统→个性化；推荐系统。
KEYWORDS
关键词
User profiling;Recommender
systems;Hierarchical user profiling;Pyramid Recurrent Neural
Networks;E-commerce
用户分析；推荐系统；分层用户分析；金字塔递归神经网络；电子商务
Permission to
make digital or hard copies of all or part of this work for personal or
classroom use is granted without fee provided that copies are not made or
distributed for profit or commercial advantage and that copies bear this notice
and the full citation on the first page.Copyrights for components of this work
owned by others than ACM must be honored.Abstracting with credit is
permitted.To copy otherwise, or republish, to post on servers or to
redistribute to lists, requires prior specific permission and/or a fee.Request
permissions from permissions@acm.org .WSDM '20, February 3-7, 2020, Houston,
TX, USA 2020 Association for Computing Machinery.ACM ISBN
978-1-4503-6822-3/20/02...$15.00 https://doi.org/10.1145/3336191.3371827
允许免费制作本作品的全部或部分数字或硬拷贝供个人或课堂使用，前提是拷贝的制作或分发不是为了盈利或商业利益，并且拷贝带有本通知和第一页的完整引文。必须尊重除ACM之外的其他人拥有的本作品组件的版权。允许用信用抽象。以其他方式复制或重新发布，在服务器上发布或重新发布到列表，需要事先获得特定许可和/或费用。向permissions@acm.org申请许可。WSDM '20，2020年2月3日至7日，美国德克萨斯州休斯顿2020计算机械协会。美国计算机学会ISBN 978-1-4503-6822-3/20/02。。。15.00美元https://doi.org/10.1145/3336191.3371827
1
INTRODUCTION
1导言
Figure 1:
Hierarchical recommendations in Amazon In the era of Internet, recommender
systems are playing crucial roles in various applications such as E-commerce
portals (e.g. Ama-zon, JD.com , Alibaba), social networking websites like
Facebook, video-sharing sites like Youtube, visual discovery sites like
Pinterest and so on.In practice, User Profiling [5, 11, 18, 24, 33, 38] is one
of the most important phases in recommender systems.It yields profile vectors,
which formally represent users' interests by deeply understanding their
historical interactions, can be used for candi-date generation [31, 42],
click-through rate prediction [4, 39, 40], conversion rate prediction [3, 16]
and long-term user engagement optimization [34-37, 44-46].
图1:亚马逊的分层推荐在互联网时代，推荐系统在各种应用中发挥着至关重要的作用，如电子商务门户网站(如Ama-zon、JD.com、阿里巴巴)、脸书等社交网站、Youtube等视频分享网站、Pinterest等视觉发现网站等。实际上，用户简档[5，11，18，24，33，38]是推荐系统中最重要的阶段之一。它通过深入了解用户的历史交互产生了形式上代表用户兴趣的简档向量，可用于candi-date生成[31，42]、点击率预测[4，39，40]、转化率预测[3，16]和长期用户参与度优化[34-37，44-46]。
Recently,
modeling users' hierarchical real-time interests is emerg-ing to be a crucial
issue in E-commerce recommender systems.Firstly, items (i.e. products) in
E-commerce sites are typically orga-nized in hierarchical
catalogue.Correspondingly, users' interests naturally lie hierarchically on
multiple granularity of items and categories.Secondly, different granularity of
recommendations (e.g. item, topic and category) become very popular in E-commerce
sites, and such scenarios require hierarchical user profiling in different
近年来，在电子商务推荐系统中，用户分层实时兴趣建模成为一个关键问题。首先，电子商务网站中的商品(即产品)通常以分级目录的形式组织。相应地，用户的兴趣自然分层地位于项目和类别的多个粒度上。其次，不同粒度的推荐(例如，项目、主题和类别)在电子商务网站中变得非常流行，并且这种场景需要在不同的
granularity
as well.For instance, Figure 1 illustrates a real example of hierarchical
recommendations in Amazon.The left side of the figure recommends some items (mobile
phones) to a user, while the right side shows a list of recommendations on the
categories of "phone accessories", "chargers" and so
on.Category recommen-dation can help the recommender systems quickly figure out
the main interest of the user and make better recommendations.
粒度也是如此。例如，图1展示了亚马逊分层推荐的一个真实例子。图的左侧向用户推荐了一些物品(手机)，右侧显示了“手机配件”、“充电器”等类别的推荐列表。类别推荐可以帮助推荐系统快速找出用户的主要兴趣，做出更好的推荐。
Existing user
profiling methods mainly focus on item recom-mendations, usually based on
users' item-level responses like rat-ings [20] or clicks [14].Among existing
methods, latent factor mod-eling is a popular branch, including matrix
factorization [13, 20, 38], neural embedding [8, 10], etc.Generally they learn
a unified embed-ding for the target user to represent her interests on the
items based on her historical behaviors.Recently, recurrent neural networks
(RNN) have achieved state-of-the-art performance in session-based
recommendations [14, 29].
现有的用户简档方法主要关注项目推荐，通常基于用户的项目级响应，如点击[20]或点击[14]。在现有的方法中，潜在因子模型是一个热门的分支，包括矩阵分解[13，20，38]，神经嵌入[8，10]等。一般来说，他们为目标用户学习一个统一的嵌入，以基于她的历史行为来表示她对项目的兴趣。最近，递归神经网络(RNN)在基于会话的建议中取得了最先进的性能[14，29]。
Existing
methods have the following limitations.First, when facing different granularity
of recommendation tasks, most of them usually need to run a similar algorithm
multiple times on different granularity of item organizations, where each run
builds users' certain level profile vectors for the corresponding
recommendation task, i.e., item-level profiles for item recommendations and
category-level profiles for category recommendations.Correspondingly, the
training process of each level's profile vectors is completely inde-pendent
from the others.However, users' multiple-level interests are closely
correlated.Figure 2 illustrates a user's hierarchical in-terests, including an
item level and two category levels, with her historical behaviors.Resulting
from the correlations between items and categories, improvement on one
recommendation task might benefit others.However, to the best of our knowledge,
such privi-lege has not been explored in existing methods.
现有方法有以下限制。首先，当面对不同粒度的推荐任务时，他们中的大多数通常需要在不同粒度的项目组织上多次运行类似的算法，其中每次运行都为相应的推荐任务构建用户的特定级别简档向量，即项目推荐的项目级简档和类别推荐的类别级简档。相应地，每个级别的轮廓向量的训练过程完全独立于其他级别。然而，用户的多层次兴趣密切相关。图2展示了用户的分层兴趣，包括一个项目级别和两个类别级别，以及她的历史行为。由于项目和类别之间的相关性，一项推荐任务的改进可能会使其他任务受益。然而，据我们所知，这种特权还没有在现有的方法中得到探索。
Second, only
harvesting the signals of users' item-level inter-actions like ratings and
clicks is insufficient.In most of the E-commerce portals, users provide
finely-granular responses such as clicking and browsing different modules
(e.g., comments and pictures) of items, adding to shopping carts and purchases,
which are referred to as "micro-behaviors" [30, 41].For example, the
bot-tom layer of Figure 2 presents a user's historical micro-behaviors
第二，仅仅收获用户的项目级交互信号如评分和点击是不够的。在大多数电子商务门户中，用户提供精细的响应，如点击和浏览不同模块(如评论和图片)的项目，添加到购物车和购买，这被称为“微行为”[30，41]。例如，图2的机器人层展示了用户的历史微观行为
in JD.com
(one of the largest e-commerce site in the world), includ-ing browsing a pair
of Nike shoes from the homepage, searching and reading specifications of iPhone
8, browsing Google Pixels 2 from the promoting page, searching iPhone X,
reading comments and adding it into the shopping cart for purchasing,
etc.Obviously, in comparison with users' item-level responses, micro-behaviors
provide more detailed information, and preliminary studies [30, 41] have
demonstrated the advantage of modeling such detailed behav-iors.However, to our
best knowledge, none of existing methods has leveraged such advantages to
improve the performance of multiple-level user profiling.
在JD.com(全球最大的电商网站之一)，包括从主页浏览一双耐克鞋、搜索阅读iPhone 8的规格、从推广页面浏览Google Pixels 2、搜索iPhone X、阅读评论并添加到购物车进行购买等。显然，与用户的项目级响应相比，微观行为提供了更详细的信息，初步研究[30，41]证明了对这种详细行为建模的优势。然而，据我们所知，现有的方法都没有利用这样的优势来提高多级用户概要分析的性能。
Third,
generally users' interests are dynamic and continuously shifting.Some
state-of-the-art methods like Time-LSTM [43] usu-ally incorporate time
intervals to track the interests shifting.How-ever, we argue that besides the
time intervals, the types of behaviors and their dwell time are also extremely
important.As shown in Figure 2, we know that iPhone X is preferable to others,
since vari-ous micro-behaviors are performed on iPhone X with long dwell
time.We also observe that triggered by making an order on iPhone X, the user's
interests on mobile phones drop sharply.Neglecting to model behavior types and
dwell times, Time-LSTM would be in trouble to capture users' detailed
preferences and interests shifting.
第三，一般来说，用户的兴趣是动态的，不断变化的。一些最先进的方法，如时间-LSTM [43]通常结合时间间隔来跟踪利益转移。然而，我们认为，除了时间间隔，行为类型和停留时间也极其重要。如图2所示，我们知道iPhone X比其他的更好，因为各种各样的微行为在长停留时间的iPhone X上执行。我们还观察到，在iPhone X上下单触发，用户对手机的兴趣急剧下降。如果忽略对行为类型和停留时间的建模，时代LSTM将很难捕捉到用户的详细偏好和兴趣转移。
To cope with
these challenges, we present HUP, a hierarchical user profiling framework to
precisely formulate users' real-time interests on multiple organizations of
items, targeting significant performance gains in recommendation accuracy.In
particular, it models users' multiple-level interests with a Pyramid Recurrent
Neural Networks, which typically consist of a micro layer, an item layer, and
multiple category recurrent neural network layers.The micro layer harvests the
detailed behavioral information and passes it to the higher layers, which could
abstract users' hierarchical inter-ests on the corresponding levels of the item
organizations simulta-neously.Furthermore, to sensitively track users'
real-time interests, we introduce Behavior-LSTM in each layer, where a behavior
gate is designed to model the types and dwell time of behaviors.Extensive
experiments for item recommendation and category recommenda-tion tasks have
been conducted on two large-scale real e-commerce datasets to demonstrate the
effectiveness of our proposed approach.
为了应对这些挑战，我们提出了HUP，这是一个分层的用户配置框架，可以精确地表达用户对多个项目组织的实时兴趣，目标是在推荐准确性方面获得显著的性能提升。具体来说，它使用金字塔递归神经网络对用户的多层次兴趣进行建模，该神经网络通常由微层、项目层和多类别递归神经网络层组成。微观层收集详细的行为信息并传递给更高层，可以同时抽象出用户在项目组织相应层次上的分层兴趣。此外，为了敏感地跟踪用户的实时兴趣，我们在每一层中引入了行为LSTM，其中设计了一个行为门来模拟行为的类型和停留时间。在两个大规模真实电子商务数据集上进行了项目推荐和类别推荐任务的大量实验，验证了所提方法的有效性。
224
224
Technical
Presentation
技术演示
WSDM '20, February
3-7, 2020, Houston, TX, USA
WSDM '20，2020年2月3日至7日，美国德克萨斯州休斯顿
To sum up,
our major contributions are listed as follows:
综上所述，我们的主要贡献如下:
• We
formulate a novel hierarchical user profiling problem, which aims to precisely
model users' multiple level interests simultaneously in E-commerce recommender
systems.
我们提出了一个新的分层用户简档问题，旨在精确地同时建模电子商务推荐系统中用户的多层次兴趣。
• We present
HUP, which exploits a Pyramid Recurrent Neu-ral Networks for hierarchical user
profiling based on users' historical micro-behaviors.
我们介绍HUP，它利用金字塔递归神经网络，根据用户的历史微观行为进行分层用户剖析。
• We propose
Behavior-LSTM, which utilizes a behavior gate to model the types and dwell time
of behaviors for effectively formulating users' real-time interests.
我们提出了行为-LSTM，它利用行为门来模拟行为的类型和停留时间，从而有效地表达用户的实时兴趣。
• We conduct
extensive experiments and prove that our method outperforms state-of-the-art
baselines greatly on both item recommendation and category recommendation
tasks.
我们进行了广泛的实验，证明我们的方法在项目推荐和类别推荐任务上都大大优于最先进的基线。
2 RELATED
WORK
2相关工作
2.1 User
Profiling for Recommendations
2.1推荐的用户概况
Recommender
systems [1] can recommend potentially interested items to users for tackling
the information overload problem.Ex-isting works mainly fall into either
content-based technology [26] or collaborative filtering [23].In both of them,
user profiling plays a critical role in formulating users' interests or
characteristics [5] based on their behaviors in the past [18, 24, 33, 35,
38].Classic col-laborative filtering techniques like matrix factorization [20]
learn users' static profiles from their rating preferences for estimation of
users' interests in the future [38].Furthermore, the evolutionary user
profiling can learn users' dynamic profiles along time based on the time
changing factor model [19], vector autoregression [24], dy-namic sparse topic
model [8], etc.However, these methods mainly focus on the item recommendation
problem, where neither the sequential information of users' behaviors nor the
hierarchy of the user profiles could be considered.
推荐系统[1]可以向用户推荐潜在感兴趣的项目，以解决信息过载问题。现有的作品主要分为基于内容的技术[26]和协同过滤[23]。在这两种情况下，用户简档在根据用户过去的行为[18，24，33，35，38]制定用户的兴趣或特征[5]方面起着至关重要的作用。像矩阵分解[20]这样的经典协作过滤技术从用户的评级偏好中学习用户的静态简档，以估计用户未来的兴趣[38]。此外，基于时变因子模型[19]、向量自回归[24]、动态稀疏主题模型[8]等，进化用户简档可以学习用户沿时间的动态简档。然而，这些方法主要集中在项目推荐问题上，既不能考虑用户行为的顺序信息，也不能考虑用户简档的层次结构。
2.2 RNN-based
User Profiling
2.2基于RNN的用户概况
In
recommender systems, recurrent neural networks (RNN) have shown impressive
advantages by modeling user's sequential behav-iors [14, 15, 17, 29].For
example, Hidasi et al. [14] introduced the concept of session-based
recommendations, and firstly proposed an RNN-based framework to process user's
click sequences on items in a session.Tan et al. [29] further improved its
performance by considering the data augmentation and temporal shift
issues.Hi-dasi et al. [15] integrated some content features extracted from
images and text into parallel RNN architectures, which demon-strated their
significant performance improvements over baselines.Li et al. [22] proposed a
neural attentive recommendation machine that can identify users' main purpose
of their current session tar-geting the performance gains.Beyond behaviors
within a session, Quadrana et al. [27] leveraged an additional GRU layer to
model users' cross-session activities for session-based recommendations.Recently,
it has been found that the temporal information and users' finely-granular
interactions are significantly helpful for recommen-dations.Wu et al. [32]
leveraged timestamps of behaviors with a long short-term memory (LSTM)
autoregressive method.Zhu et al.
在推荐系统中，递归神经网络(RNN)通过模拟用户的顺序行为表现出令人印象深刻的优势[14，15，17，29]。例如，Hidasi等人[14]引入了基于会话的推荐的概念，并首次提出了一个基于RNN的框架来处理用户在会话中对项目的点击序列。Tan等人[29]通过考虑数据增加和时间偏移问题，进一步提高了其性能。Hi-dasi等人[15]将从图像和文本中提取的一些内容特征集成到并行RNN体系结构中，这表明它们的性能比基线有显著提高。李等[22]提出了一种神经注意力推荐机，可以识别用户当前会话的主要目的，从而获得性能提升。除了会话内的行为，Quadrana等人[27]利用额外的GRU层来为基于会话的推荐建模用户的跨会话活动。最近，人们发现时间信息和用户的精细交互对推荐有很大的帮助。Wu等人[32]利用长短期记忆的行为时间戳()自回归方法。朱等。
[43] proposed
Time-LSTM, which used the time gates to model the time intervals between
behaviors.Wan and McAuley [30] ex-ploited the effectiveness of the relations
among users' different types of behaviors in recommendations.Zhou et al. [41]
trained a
[43]提出了时间LSTM，它使用时间门来模拟行为之间的时间间隔。万和麦考利[30]在推荐中探索了用户不同类型行为之间关系的有效性。周等[41]训练了一个
single layer
RNN model with the micro-behaviors for product rec-ommendation.However, this
method only models user's interests in items and just exploits micro behaviors
information as additional input, which might lead to inferior performance.Our
method uses multi-layer Behavior-LSTM cells and attentions to explicitly model
the micro-behaviors information, which can solve both the item recommendation
and the hierarchical categories recommendation problems.
具有产品推荐微观行为的单层RNN模型。然而，这种方法只模拟用户对项目的兴趣，只是利用微观行为信息作为额外的输入，这可能会导致性能下降。该方法利用多层行为LSTM单元和注意力对微观行为信息进行显式建模，既能解决项目推荐问题，又能解决分层类别推荐问题。
In a word,
most existing RNN-based methods fail to address the hierarchical user profiling
problem.In addition, to the best of our knowledge, there are no explorations
that could leverage the types, dwell time and time intervals of the behaviors
simultaneously in an RNN framework for user profiling.
总之，大多数现有的基于RNN的方法都无法解决分层用户配置问题。此外，据我们所知，没有任何探索能够在RNN框架中同时利用行为的类型、停留时间和时间间隔来进行用户剖析。
3 PROBLEM
FORMULATION
3问题表述
In this
section, we firstly introduce the background, notations and definitions in this
paper, and then formulate our problem formally.
在这一部分，我们首先介绍了本文的背景、符号和定义，然后正式阐述了我们的问题。
3.1
Background
3.1背景
Hierarchical
categories organize products of the E-commerce sites in different
granularity.The hierarchy is generally a tree struc-ture, where each lower
level category is an element of a higher level one, and products are usually
hung onto the finest categories as the leaf nodes of the tree.For example, the
first level category "Electronics" might include some second level
categories like "Tele-phone" and "Accessory", and
"Mobile Phone" is a category in the third and finest level belonging
to "Telephone".
分级类别以不同的粒度组织电子商务站点的产品。层次结构通常是一个树形结构，其中每个较低层次的类别都是较高层次的一个元素，产品通常挂在最细的类别上，作为树的叶节点。例如，第一级类别“电子产品”可能包括一些第二级类别，如“电话”和“附件”，“移动电话”是属于“电话”的第三级和最高级类别。
Micro-behaviors
are detailed unit interactions (e.g. reading the detail comments, carting) of
users with recommender systems.They can provide rich information for indicating
users' timely interests, including the type of behavior that a user conducts on
an item, how long a user dwells on an item and move to the next one [30, 41].In
this paper, we consider 10 types of micro behaviors, which are shown in Table
1.
微行为是用户与推荐系统之间的详细单元交互(例如阅读详细评论、搬运)。它们可以提供丰富的信息来指示用户的及时兴趣，包括用户对某个项目的行为类型、用户在某个项目上停留的时间以及移动到下一个项目的时间[30，41]。在本文中，我们考虑了10种微观行为，如表1所示。
Micro
behaviors Description
微观行为描述
Home2Product
Browse the product from the homepage ShopList2Product Browse the product from
the category page Sale2Product Browse the product from the sale page Cart2Product
Browse the product from the carted page SearchList2Product Browse the product
from the searched results Detail_comments Read the comments of the product
Detail_specification Read the specification of the product Detail_bottom Read
the bottom of page of the product Cart Add the product to the shopping cart
Order Make an order
主页2产品从主页浏览产品购物列表2产品从类别页面浏览产品销售2产品从销售页面浏览产品Cart 2产品从card页面浏览产品搜索列表2产品从搜索结果Detail_comments浏览产品阅读产品Detail_specification的注释阅读产品Detail_bottom的规格阅读产品Cart的页面底部将产品添加到购物车订单下订单
Table 1: List
of micro-behaviors
表1:微观行为列表
3.2
Hierarchical User Profiling
3.2分层用户分析
Definition
3.1 (Hierarchical User Profiling).Hierarchical user pro-filing aims to generate
the micro-level, item-level and hierarchical category-level profile vectors pmu
, piu and pcu = {p () cu , ..., p (K) cu } respectively based on her
micro-behaviors, which represent each target user u's interests in
corresponding granularity.
定义3.1(分层用户分析)。分级用户预归档旨在生成微观级别、项目级别和分级类别级别的配置文件向量pmu、piu和pcu = {p()、cu。。。，p (K) cu }分别基于她的微观行为，以相应的粒度表示每个目标用户u的兴趣。
225
225
Technical
Presentation
技术演示
WSDM '20,
February 3-7, 2020, Houston, TX, USA
WSDM '20，2020年2月3日至7日，美国德克萨斯州休斯顿
Definition
3.2 (Hierarchical Recommendations).Let U be a set of users, V be a set of
items, and C () ,C () , ...,C (K) be the K levels hierarchy of the
categories.The hierarchical recommendations task aims to recommend a set of
items Vˆ u and K set of categories Cˆ () u , ...,Cˆ (K) u to each target user u
by maximizing the relevance between u and her recommendations in different
granularity.
定义3.2(分级建议)。设U为一组用户，V为一组项目，C()，C()，。。。，C (K)是类别的K级层次结构。分层推荐任务旨在推荐一组项目和一组类别。。。，通过最大化u和她在不同粒度的推荐之间的相关性，将c(K)u分配给每个目标用户u。
4 HUP: A
HIERARCHICAL USER PROFILING FRAMEWORK
4 HUP:一个分层的用户配置框架
In this
section, we introduce HUP, a hierarchical user profiling framework for
hierarchical recommendations.As illustrated in Fig-ure 3, HUP utilizes a
Pyramid Recurrent Neural Networks to extract users' hierarchical interests from
micro-behaviors at multiple scales.
在本节中，我们将介绍HUP，一个用于分层推荐的分层用户分析框架。如图3所示，HUP利用金字塔递归神经网络从多尺度的微观行为中提取用户的层次兴趣。
4.1 The Input
and Embedding Layers
4.1输入层和嵌入层
Given a
target user u, the input of our model is a sequence of her micro-behaviors X = ⟨x1,
x2, ..., xN ⟩.The ith element xi = (ti ,vi ,ci , bi , di , дi) indicates that u
performs a micro-behavior of type bi on the item vi at the time ti , where vi
belongs to multiple-level categories ci = {c () i ,c () i , ...,c (K) i }, the
dwell time is di , and the time interval between xi and xi+1 is дi . Here both
dwell time and time interval are real numbers.As previous work did [41], we
discretize them into several buckets respectively for embed-ding.For each
micro-behavior xi , the embedding layer firstly uses embedding tables of items,
categories, behavior types, dwell time buckets and time intervals to transform
vi ,ci , bi , di , дi into low-dimensional dense vectors (i.e., evi ,eci ,ebi
,edi ,eдi ) respectively and then concatenates these vectors into a single
embedding vector ei .The embedding tables are initialized as random numbers.
给定一个目标用户u，我们模型的输入是她的一系列微观行为X = ⟨x1，x2，。。。⟩. xnith元素xi = (ti，vi，ci，bi，di，дi)表示u在时间ti对项目vi执行bi类型的微行为，其中vi属于多级类别ci = { c(I)，c(I)，...，c (K) i }，停留时间为di，xi到xi+1的时间间隔为дi，这里停留时间和时间间隔都是实数。正如前面的工作所做的[41]，我们将它们分别离散成几个桶用于嵌入。对于每个微行为xi，嵌入层首先使用项目、类别、行为类型、停留时间桶和时间间隔的嵌入表将vi、ci、bi、di、дi分别转换成低维密集向量(即evi、eci、ebi、edi、e7i)，然后将这些向量串联成单个嵌入向量ei。嵌入表被初始化为随机数。
4.2 Pyramid
Recurrent Neural Networks
4.2金字塔递归神经网络
Most of
previously recurrent neural networks (RNN)-based recom-mendation methods [5,
14, 15, 29, 41] use a single-layer RNN to generate user profile vectors, which
might not be capable of cap-turing user's hierarchical interests in different
levels.To solve this problem, inspired by the Spatial Pyramid Pooling-net
(SPP-net) [12], we propose a Pyramid Recurrent Neural Networks, which contains
a micro-level, an item-level and several category-level RNN layers to abstract
users' hierarchical interests at multiple scales simulta-neously.
大多数以前基于递归神经网络(RNN)的推荐方法[5，14，15，29，41]使用单层RNN来生成用户简档向量，这可能不能在不同级别上限制用户的分层兴趣。为了解决这个问题，受空间金字塔汇集网(SPP-net) [12]的启发，我们提出了一个金字塔递归神经网络，它包含一个微观层次、一个项目层次和几个类别层次的RNN层，以同时在多个尺度上抽象用户的分层兴趣。
The
micro-level RNN layer aims to model users' finest level in-terests.The input at
the time stamp i of this layer xMi comes from the embedding layer, and the
output of this layer YM is forwarded to the item-level RNN layer for further
calculations.The hidden state is updated after taking each micro-behavior as
input.The for-mulations of the Micro-level RNN layer are defined in Equation 1.
微观层面的RNN层旨在模拟用户最大的兴趣。该层xMi的时间戳I处的输入来自嵌入层，并且该层YM的输出被转发到项目级RNN层用于进一步计算。隐藏状态在将每个微行为作为输入后更新。微观层次RNN层的模拟在方程1中定义。
XM = [xMi ] =
[ei], i = 1, 2, ..., N
XM = [xMi ] =
[ei]，i = 1，2，...，N
YM = [yMi ] =
RN N M (XM ), i = 1, 2, ..., N (1)
YM = [yMi ] =
RN N M (XM)，i = 1，2，...，N (1)
The
item-level RNN layer models users' item-level interests.The input at the time
stamp i of this layer xIi is the concatenation of the item embedding evi and
the output of the micro-level layer.The hidden state is only updated after a
user have transferred her focuses from one item to another.Its output YI is
forwarded to the
项目级RNN层对用户的项目级兴趣进行建模。该层xIi的时间戳I处的输入是嵌入evi的项目和微观层的输出的连接。只有在用户将焦点从一个项目转移到另一个项目后，隐藏状态才会更新。其输出YI被转发到
category-level
RNN layers.The formulations of the Item-level RNN layer are defined in Equation
2.
类别级RNN图层。项目级RNN层的公式在等式2中定义。
XI = [xIi ] =
[evi ;yMi ]
Xi
=[XIi]=[EVI；yMi ]
YI = [yIi ] =
RN NI (XI ) (2)
YI = [yIi ] =
RN NI (XI ) (2)
The
category-level RNN layers formulate users' category-level interests.In the Kth
category layer (the finest granularity of cat-egories), the input at the time
stamp i is x (K) Ci , which is the con-catenation of the category embedding e
(K) ci and the output of the item-level RNN layer calculated on items under
this category.For other higher-level category layers, the input at the time
stamp i of the kth level category layer is X (k) C , which is the concatenation
of the category embedding e (k) ci in this layer and the output of the (k −
1)th level category layer.In each layer, the hidden state is only updated after
a user has moved her focuses from one category to another in this layer.The
formulations of the category-level RNN layers are defined in Equation 3.
类别级RNN层制定用户的类别级兴趣。在第Kth类别层(最细的类别粒度)中，时间戳I处的输入是x (K) Ci，这是嵌入e (K) ci的类别和对该类别下的项目计算的项目级RNN层的输出的组合。对于其他更高级别的类别层，在第k级类别层的时间戳I处的输入是X (k) C，这是该层中嵌入e (k) ci的类别和第(k-1)级类别层的输出的串联。在每一层中，隐藏状态只有在用户将她的焦点从该层中的一个类别移动到另一个类别后才会更新。类别级RNN层的公式在等式3中定义。
X (k) C = [x
(k) Ci ] = ( [e (k) ci ;yIi ]k = K [e (k) ci ;y (k−) Ci ], k = 1, ...,K − 1 Y
(k) C = [y (k) Ci ] = RN N(k) C X (k) C , k = , ...,K
x(k)C
=[x(k)Ci]=([e(k)Ci；yIi]K = K[e(K)ci；y(k)Ci]，k = 1，...，k1 Y(K)C =[Y(K)Ci]= RN N(K)C X(K)C，k =，...，K
(3)
(3)
4.3
Behavior-LSTM Cell
4.3行为-LSTM细胞
Generally
users' interests are dynamic and continuously shifting.Time-LSTM [43] is a
state-of-the-art method that incorporates time intervals between users'
sequential purchases to address the interest shifting problem.However, it
cannot model the behavior type and the dwell time information, which may lead
to inferior performance.We here propose Behavior-LSTM, a novel RNN layer that
provides an additional behavior gate to process the types and dwell time of the
behaviors, enabling HUP to track users' real-time interests more precisely.In
particular, it is described in Figure 4 and formulated in Equation 4:
一般来说，用户的兴趣是动态的，不断变化的。时间-LSTM [43]是一种最先进的方法，它结合了用户连续购买之间的时间间隔，以解决利益转移问题。但是，它无法对行为类型和停留时间信息进行建模，这可能会导致性能下降。我们在这里提出了行为-LSTM，一个新的RNN层，它提供了一个额外的行为门来处理行为的类型和停留时间，使HUP能够更精确地跟踪用户的实时兴趣。具体来说，它在图4中描述，并在等式4中表示:
It = σ
(WI[ht−1, xt ] + bI) Ft = σ (WF[ht−1, xt ] + bF) Tt = σ (WT[xt , ∆t ] + bT) At
= σ (WA[xt , at ] + bA) C˜ t = tanh(WC[ht−1, xt ] + bC) Ct = Ft ⊙ Ct−1 + It ⊙
Tt ⊙ At ⊙ C˜ t Ot = σ (WO[ht−1, xt ] + bO) ht = Ot ⊙ tanh(Ct )
It =σ(WI[ht 1，XT]+bI)Ft
=σ(WF[ht 1，xt ] + bF) Tt = σ (WT[xt，t ] + bT) At = σ (WA[xt，At]+Ba)c≘t = tanh(WC[ht 1，XT]+bC)Ct = Ft⊙Ct 1+It⊙TT⊙At⊙c≘t Ot =σ(WO[ht 1
(4)
(4)
where I, F,
T, A and O are the input, forget, time, behavior and output gates, C and h are
the cell state and hidden state vectors, WI,WF,WT,WA,WC andWO are weight
matrices, bI, bF, bT, bA, bC and bO are the biases, respectively.The input of
the Behavior-LSTM is a tuple (xt , at , ∆t ), where xt is the embedding vector
of the input at the time stamp t, at is the embedding vector of the behavior
type or dwell time information, and ∆t is the embedding vector of time interval
between current behavior and the next one.
其中I、F、T、A和O是输入、遗忘、时间、行为和输出门，C和h是单元状态和隐藏状态向量，WI、WF、WT、WA、WC和O分别是权重矩阵，bI、bF、bT、bA、bC和bO分别是偏差。行为-LSTM的输入是一个元组(xt，at，∏t)，其中xt是输入在时间戳t的嵌入向量，at是行为类型或停留时间信息的嵌入向量，而∏t是当前行为和下一个行为之间的时间间隔的嵌入向量。
In
Behavior-LSTM, the time gate T estimates how much informa-tion that should
maintain or pass to the next state, and the behavior gate A calculates the
importance of current behavior with the meta information of the behavior.In
particular, such meta information of the behaviors involves two aspects: their
types and users' dwell time.In particular, the behavior gate actually only
processes the types of micro-behaviors in the micro level RNN layer.It is
because
在行为-LSTM中，时间门T估计有多少信息应该保持或传递到下一个状态，行为门A用行为的元信息计算当前行为的重要性。特别地，这些行为的元信息涉及两个方面:它们的类型和用户的停留时间。特别是，行为门实际上只处理微观层次RNN层中的微观行为类型。这是因为
226
226
Technical
Presentation
技术演示
WSDM '20,
February 3-7, 2020, Houston, TX, USA
WSDM '20，2020年2月3日至7日，美国德克萨斯州休斯顿
Figure 3: The
architecture of the HUP.It uses a Pyramid Recurrent Neural Networks, which is
consisted of a micro layer, an item layer, and hierarchical category recurrent
neural networks layers, to extract users' hierarchical profile at multiple
scales.The profiles represent users' real-time interests in items and
hierarchical categories, based on which the most relevant categories and items
can be recommended to users.
图HUP的架构。它使用由微层、项目层和分层类别递归神经网络层组成的金字塔递归神经网络在多个尺度上提取用户的分层简档。简档表示用户对项目和分层类别的实时兴趣，基于此可以向用户推荐最相关的类别和项目。
Figure 4: The
architecture of the Behavior-LSTM.It has a behavior gate A and a time gate T,
where A models users' behavior information in micro behaviors, and T captures
the time intervals between users' micro behaviors.
图4:行为LSTM的架构。它有一个行为门A和一个时间门T，其中A在微观行为中建模用户的行为信息，T捕捉用户微观行为之间的时间间隔。
most of
micro-behaviors are instant responses and we could not get their dwell time,
but their types are extremely important for users' interest modeling.In the
item-level and hierarchical category-level RNN layers, this gate models the
dwell time on the items or categories.That is because the dwell time varies
significantly in items and categories and is very informative in presenting
users' interests.
大多数微行为都是即时响应，我们无法得到它们的停留时间，但是它们的类型对于用户兴趣建模是极其重要的。在项目级和分级类别级RNN图层中，该门对项目或类别的停留时间进行建模。这是因为停留时间在项目和类别上有很大的不同，并且在呈现用户兴趣方面非常有用。
4.4 The
Attention Layers
4.4关注层
The attention
mechanism [2] is a common technique in deep learn-ing.Usually, it is able to
mitigate long-term dependency issues as well as provide interpretations, which
is extremely important in real-world recommender systems.In particular, an
attention layer
注意机制[2]是深度学习中常见的技术。通常，它能够减轻长期依赖问题并提供解释，这在现实世界的推荐系统中非常重要。特别是关注层
takes the
output sequence Y = [y1,y2, ...,yT ] of an RNN as input and return a context
vector s. Let yi be a user's interests at time stamp i. The context vectors of
each attention layer is calculated as a weighted sum of the interests vectors
among all the time stamps, which is formulated formally in Equation 5.
取输出序列Y = [y1，y2，...假设yi是用户在时间戳I的兴趣。每个关注层的上下文向量被计算为所有时间戳中的兴趣向量的加权和，这在等式5中正式表述。
s = ÕTi=1 α i
y i ;α i = exp ( e i ) Í Tk = 1 exp ( e k ) ;e i = f ( y i y T a i )
s = Ti = 1αI
y I；αI = exp(e I)íTk = 1 exp(e k)；e i = f ( y i y T a i)
(5)
(5)
HUP has
multiple attention layers, where each is directly fol-lowed by its
corresponding RNN layer and therefore referred to as micro, item and category
level attention layers respectively.The context vectors from these attention
layers are denoted as sm, si and sc = {s () c ,s () c , ...,s (K) c }
respectively.The attention signal ai represents the type of micro-behaviors in
micro-level attention layer, and the dwell time in both the item and the
category level attention layers.f is an alignment model, which scores the
impor-tance of yi based on the hidden state yi , last hidden state yT and attention
signal ai .In order to achieve abundant expressive ability, we design the
alignment model f as two-layers feedforward neural networks, w
