Long-term Blood Pressure Prediction with Deep Recurrent Neural Networks

基于深度循环神经网络的长期血压预测

Abstract— Existing methods for arterial blood pressure (BP) estimation directly map the input physiological signals to output BP values without explicitly modeling the underlying temporal dependencies in BP dynamics. As a result, these models suffer from accuracy decay over a long time and thus require frequent calibration. In this work, we address this issue by formulating BP estimation as a sequence prediction problem in which both the input and target are temporal sequences. We propose a novel deep recurrent neural network (RNN) consisting of multi layered Long Short-Term Memory (LSTM) networks, which are incorporated with (1) a bidirectional structure to access larger-scale context information of input sequence, and (2) residual connections to allow gradients in deep RNN to propagate more effectively. The proposed deep RNN model was tested on a static BP dataset, and it achieved root mean square error (RMSE) of 3.90 and $\mathbf{2.66mmHg}$ for systolic BP (SBP) and diastolic BP (DBP) prediction respectively, surpassing the accuracy of traditional BP prediction models. On a multi-day BP dataset, the deep RNN achieved RMSE of 3.84, 5.25, 5.80 and ${\pmb5.81}{\bf m m H g}$ for the 1st day, 2nd day, 4th day and 6th month after the 1st day SBP prediction, and 1.80, 4.78, 5.0, 5.21 $\mathbf{mmHg}$ for corresponding DBP prediction, respectively, which outperforms all previous models with notable improvement. The experimental results suggest that modeling the temporal dependencies in BP dynamics significantly improves the longterm BP prediction accuracy.

摘要—现有的动脉血压(BP)估计方法直接将输入生理信号映射为输出BP值，而没有显式建模BP动态中的潜在时间依赖性。因此，这些模型会随着时间推移出现精度衰减，从而需要频繁校准。本研究通过将BP估计构建为输入和目标均为时间序列的序列预测问题来解决该问题。我们提出了一种由多层长短期记忆(LSTM)网络组成的新型深度循环神经网络(RNN)，该网络具有：(1)双向结构以获取输入序列更大尺度的上下文信息；(2)残差连接使深度RNN中的梯度能更有效传播。所提出的深度RNN模型在静态BP数据集上测试时，收缩压(SBP)和舒张压(DBP)预测的均方根误差(RMSE)分别达到3.90和2.66mmHg，超越了传统BP预测模型的精度。在多日BP数据集上，该深度RNN在第1天、第2天、第4天和第6个月后的SBP预测RMSE分别为3.84、5.25、5.80和5.81mmHg，相应DBP预测RMSE分别为1.80、4.78、5.0和5.21mmHg，以显著优势超越所有先前模型。实验结果表明，对BP动态中的时间依赖性建模能显著提升长期BP预测精度。

I. INTRODUCTION

I. 引言

Accordingly, such dependency is critical for continuous BP prediction and in particular, for long-term BP prediction.

因此，这种依赖性对于持续血压 (BP) 预测至关重要，尤其是长期血压预测。

Existing methods for cuffless and continuous BP estimation can be categorized into two groups, namely physiologi- cal model, i.e., pulse transit time model [3] [4], and regression model, such as decision tree, support vector regression and etc [5] [6]. These models suffers from accuracy decay over time, especially for multi-day continuous BP prediction. Such limitation has become the bottleneck that prevents the use of these models in practical applications. It is worth noting that the aforementioned models directly map present input to the target while ignoring the important temporal dependencies in BP dynamics. This could be the root of long-term inaccuracy.

现有无袖带连续血压估计方法可分为两类：生理模型（如脉搏波传导时间模型 [3][4]）和回归模型（如决策树、支持向量回归等 [5][6]）。这些模型存在随时间推移精度下降的问题，特别是在多日连续血压预测场景中。该局限性已成为阻碍这些模型实际应用的瓶颈。值得注意的是，上述模型直接将当前输入映射至目标值，而忽略了血压动态变化中重要的时间依赖性——这可能是长期预测失准的根本原因。

Compared with static BP prediction, the multi-day BP prediction is generally much more challenging. Due to the complex regulation mechanisms of human body, multi-day BP dynamics have more intricate temporal dependencies and a larger variation range. In this paper, we formulate the BP prediction as a sequence learning problem, and propose a novel deep RNN model, which is proved to be very effective for modeling long-range dependencies in BP dynamics and has achieved the state-of-the-art accuracy on multi-day continuous BP prediction.

与静态血压预测相比，多日血压预测通常更具挑战性。由于人体复杂的调节机制，多日血压动态具有更复杂的时间依赖性和更大的变化范围。本文将血压预测建模为序列学习问题，并提出一种新型深度RNN模型，该模型被证明能有效建模血压动态中的长程依赖关系，并在多日连续血压预测中达到了最先进的准确率。

As the leading risk factor of cardiovascular diseases (CVD) [1], high blood pressure (BP) has been commonly used as the critical criterion for diagnosing and preventing CVD. Therefore, accurate and continuous BP monitoring during people’s daily life is imperative for early detection and intervention of CVD. Traditional BP measurement devices, e.g., Omron products, are cuff-based and therefore bulky, discomfort to use, and only available for snapshot measurements. These disadvantages restrict the use of the cuff-based devices for long-term and continuous BP measurement, which are essential for nighttime monitoring and precise diagnosis of different CVD symptoms.

作为心血管疾病(CVD) [1]的首要风险因素，高血压(BP)通常被用作诊断和预防CVD的关键标准。因此，在人们日常生活中进行准确、持续的血压监测对CVD的早期发现和干预至关重要。传统血压测量设备(如欧姆龙产品)基于袖带，因而体积庞大、使用不适且仅能进行单次测量。这些缺点限制了基于袖带的设备在长期连续血压监测中的应用，而这对夜间监测和不同CVD症状的精确诊断至关重要。

A key feature of our cardiovascular system is its complex dynamic self-regulation that involves multiple feedback control loops in response to BP variation [2]. This mechanism gives the BP dynamics a temporal dependency nature.

我们心血管系统的一个关键特征是其复杂的动态自我调节机制，该机制涉及多个反馈控制回路以应对血压(BP)变化 [2]。这一机制使血压动态具有时间依赖性特征。

Peng Su, Xiao-Rong Ding, Yuan-Ting Zhang, Jing Liu and Ni Zhao are with the Department of Electronic Engineering, The Chinese University of Hong Kong, Hong Kong (psu@ee.cuhk.edu.hk; nzhao@ee.cuhk.edu.hk). Fen Miao is with the Key Laboratory for Health Informatics of the Chinese Academy of Sciences, Shenzhen Institutes of Advanced Technology, Shenzhen, China.

彭苏、丁晓荣、张元亭、刘晶和赵妮就职于香港中文大学电子工程系（psu@ee.cuhk.edu.hk; nzhao@ee.cuhk.edu.hk）。苗芬就职于中国科学院健康信息学重点实验室，深圳先进技术研究院，中国深圳。

Published as a conference paper at IEEE International Conference on Biomedical and Health Informatics (BHI) 2018.

以会议论文形式发表于2018年IEEE生物医学与健康信息学国际会议(BHI)。

II. THE MODEL

II. 模型

The goal of arterial BP prediction is to use multiple temporal physiological signals to predict BP sequence. Let $X_{T}=[x_{1},x_{2}\dotsb,x_{T}]$ be the input features extracted from electrocardiograph y (ECG) and photo ple thys m ogram (PPG) signals, and $Y_{T}=[y_{1},y_{2}\dots y_{T}]$ denote the target BP sequence. The conditional probability $p(Y_{T}\mid X_{T})$ is factorized as:

动脉血压预测的目标是利用多个时序生理信号来预测血压序列。设 $X_{T}=[x_{1},x_{2}\dotsb,x_{T}]$ 为从心电图(ECG)和光电容积图(PPG)信号中提取的输入特征，$Y_{T}=[y_{1},y_{2}\dots y_{T}]$ 表示目标血压序列。条件概率 $p(Y_{T}\mid X_{T})$ 分解为:

$$
p(Y_{T}\mid X_{T})=\prod_{t=1}^{T}p(y_{t}\mid h_{t}),
$$

$$
p(Y_{T}\mid X_{T})=\prod_{t=1}^{T}p(y_{t}\mid h_{t}),
$$

where $h_{t}$ can be interpreted as hidden state of BP dynamic system and it is generated from previous hidden state $h_{t-1}$ and current input $x_{t}$ as:

其中 $h_{t}$ 可解释为BP动态系统的隐藏状态，它由前一个隐藏状态 $h_{t-1}$ 和当前输入 $x_{t}$ 生成：

$$
h_{t}=f(h_{t-1},x_{t}).
$$

$$
h_{t}=f(h_{t-1},x_{t}).
$$

Figure 1 illustrates the overview of our proposed deep RNN model. The deep RNN consists of a bidirectional LSTM at the bottom layer, and a stack of multi layered Long ShortTerm Memory (LSTM) with residual connections. The full network was trained with back propagation through time [7] to miniaturize the difference between BP prediction and the ground truth.

图 1: 展示了我们提出的深度 RNN (Recurrent Neural Network) 模型概览。该深度 RNN 底层采用双向 LSTM (Long Short-Term Memory)，上层堆叠了带残差连接的多层 LSTM。整个网络通过随时间反向传播 [7] 进行训练，以最小化血压 (BP) 预测值与真实值之间的差异。

Fig. 1: DeepRNN architecture. Each rectangular box is an LSTM cell. The green dashed box at bottom is a bidirectional LSTM layer consisting of forward (orange) and backward (green) LSTM. The orange dashed box depicts the LSTM layer with residual connections.

图 1: DeepRNN架构。每个矩形框代表一个LSTM单元。底部的绿色虚线框是由前向(橙色)和后向(绿色)LSTM组成的双向LSTM层。橙色虚线框表示带有残差连接的LSTM层。

A. Bidirectional LSTM Structure

A. 双向 LSTM 结构

First, we introduce the basic block of our deep RNN model, a one-layer bidirectional Long short-term memory (LSTM). LSTM [8] was designed to address the vanishing gradient problem of conventional RNN by introducing a memory cell state $c_{t}$ and multiple gating mechanisms inside a standard RNN hidden state transition process.

首先，我们介绍深度RNN模型的基本模块——单层双向长短期记忆网络(LSTM)。LSTM [8]通过引入记忆细胞状态$c_{t}$和在标准RNN隐藏状态转换过程中加入多重门控机制，旨在解决传统RNN的梯度消失问题。

where $f,i$ and $o$ are respectively the forget gate, input gate, output gate that control how much information will be forgotten, accumulated, or be outputted. $W$ and $b$ terms denote weight matrices and bias vectors respectively. $\sigma$ and tanh stand for an element-wise application of the logistic sigmoid function and hyperbolic tangent function respectively, and $\odot$ denote element-wise multiplication.

其中 $f$、$i$ 和 $o$ 分别代表控制信息遗忘量、积累量和输出量的遗忘门、输入门和输出门。$W$ 和 $b$ 分别表示权重矩阵和偏置向量。$\sigma$ 和 tanh 分别代表逐元素应用的逻辑 sigmoid 函数和双曲正切函数，$\odot$ 表示逐元素乘法。

Conventional LSTMs use $h_{t}$ to capture information from the past history $x_{1},\dots,x_{t-1}$ , and the present input $x_{t}$ . To access larger-scale temporal context of input sequence, one can also incorporate nearby future information $x_{t+1},\dots,x_{T}$ to inform the downstream modeling process. Bidirectional RNN (BRNN) [9] can realize this function by processing the data in both forward and backward directions with two separate hidden layers, which then merge to the same output layer. As illustrated in the bottom of Figure1, a BRNN computes a forward hidden state $h_{t}^{f}$ , a backward hidden state $h_{t}^{b}$ and final output $h_{t}$ by following equations:

传统的 LSTM 使用 $h_{t}$ 来捕获过去历史 $x_{1},\dots,x_{t-1}$ 和当前输入 $x_{t}$ 的信息。为了获取输入序列更大范围的时间上下文，还可以整合附近未来信息 $x_{t+1},\dots,x_{T}$ 来指导下游建模过程。双向 RNN (BRNN) [9] 通过用两个独立的隐藏层分别向前和向后处理数据，然后将结果合并到同一个输出层来实现这一功能。如图 1 底部所示，BRNN 通过以下公式计算前向隐藏状态 $h_{t}^{f}$、后向隐藏状态 $h_{t}^{b}$ 和最终输出 $h_{t}$:

$$
\begin{array}{r l}{\boldsymbol{h}{t}^{f}=\mathcal{H}(\boldsymbol{W}{h h}^{f}\boldsymbol{h}{t-1}^{f}+\boldsymbol{W}{x h}^{f}\boldsymbol{x}{t}+\boldsymbol{b}{f})}\ {\boldsymbol{h}{t}^{b}=\mathcal{H}(\boldsymbol{W}{h h}^{b}\boldsymbol{h}{t+1}^{b}+\boldsymbol{W}{x h}^{b}\boldsymbol{x}{t}+\boldsymbol{b}{b})}\ {\boldsymbol{h}{t}=\boldsymbol{W}^{f}\boldsymbol{h}{t}^{f}+\boldsymbol{W}^{b}\boldsymbol{h}{t}^{b}+\boldsymbol{b}_{h}.}\end{array}
$$

$$
\begin{array}{r l}{\boldsymbol{h}{t}^{f}=\mathcal{H}(\boldsymbol{W}{h h}^{f}\boldsymbol{h}{t-1}^{f}+\boldsymbol{W}{x h}^{f}\boldsymbol{x}{t}+\boldsymbol{b}{f})}\ {\boldsymbol{h}{t}^{b}=\mathcal{H}(\boldsymbol{W}{h h}^{b}\boldsymbol{h}{t+1}^{b}+\boldsymbol{W}{x h}^{b}\boldsymbol{x}{t}+\boldsymbol{b}{b})}\ {\boldsymbol{h}{t}=\boldsymbol{W}^{f}\boldsymbol{h}{t}^{f}+\boldsymbol{W}^{b}\boldsymbol{h}{t}^{b}+\boldsymbol{b}_{h}.}\end{array}
$$

where $\mathcal{H}$ is implemented by Equations 3-7.

其中 $\mathcal{H}$ 由公式3-7实现。

B. Multi layered Architecture with Residual Connections

B. 带残差连接的多层架构

A variety of experimental results [10] [11] have suggested that RNNs with deep architecture can significantly outperform shallow RNNs. Simply by stacking multiple layers of RNN could readily gain expressive power. However, a full deep network could become difficult to train as it goes deeper, likely due to exploding and vanishing gradient problems [12].

多项实验结果 [10] [11] 表明，具有深层架构的循环神经网络 (RNN) 能显著优于浅层RNN。仅通过堆叠多层RNN就能轻松获得更强的表达能力。但随着网络深度增加，完整的深度网络会变得难以训练，这很可能源于梯度爆炸和梯度消失问题 [12]。

Inspired by the idea of attaching an identity skip connection between adjacent layers, which has shown good performance for training deep neural networks [13] [14] [15], we incorporate a residual connection from one LSTM layer to the next in our model, as shown in Figure 2. Let $x_{t}^{i},h_{t}^{i}$ , and

受到在相邻层之间添加恒等跳跃连接这一思路的启发（该技术在训练深度神经网络中已展现出良好性能 [13][14][15]），我们在模型中引入了从一个LSTM层到下一个LSTM层的残差连接，如图2所示。设$x_{t}^{i},h_{t}^{i}$，以及

Fig. 2: LSTM with residual connection.

图 2: 带残差连接的LSTM。

$\mathcal{H}{i}$ be the input, hidden state and LSTM function respectively associated with the $i$ -th LSTM layer $(i=1,2,\dots,L)$ , and $W^{i}$ is the corresponding weight of $\mathcal{H}{i}$ . The input to the $i$ -th LSTM layer $\boldsymbol{x}_{t}^{i}$ is element-wise added to this layer’s hidden state hi. This sum xi+ is then fed to the next LSTM layer. The LSTM block with residual connections can be implemented by:

$\mathcal{H}{i}$ 分别表示第 $i$ 个 LSTM 层 $(i=1,2,\dots,L)$ 的输入、隐藏状态和 LSTM 函数，$W^{i}$ 是 $\mathcal{H}{i}$ 的对应权重。第 $i$ 个 LSTM 层的输入 $\boldsymbol{x}_{t}^{i}$ 会与该层的隐藏状态 hi 进行逐元素相加，得到的和 xi+ 将输入到下一层 LSTM。带残差连接的 LSTM 模块可通过以下方式实现:

$$
\begin{array}{r l}{\quad h_{t}^{i}=\mathcal{H}{i}\left(x_{t}^{i},h_{t-1}^{i};W^{i}\right)}\ {\quad x_{t}^{i+1}=h_{t}^{i}+x_{t}^{i}}\ {\quad h_{t}^{i+1}=\mathcal{H}{i+1}\left(x_{t}^{i+1},h_{t-1}^{i+1};W^{i+1}\right).}\end{array}
$$

$$
\begin{array}{r l}{\quad h_{t}^{i}=\mathcal{H}{i}\left(x_{t}^{i},h_{t-1}^{i};W^{i}\right)}\ {\quad x_{t}^{i+1}=h_{t}^{i}+x_{t}^{i}}\ {\quad h_{t}^{i+1}=\mathcal{H}{i+1}\left(x_{t}^{i+1},h_{t-1}^{i+1};W^{i+1}\right).}\end{array}
$$

The deep RNN model can be created by stacking multiple such LSTM blocks on top of each other, with the output of previous block forming the input of the next. Once the toplayer hidden state is computed, the output $z_{t}$ can be obtained by:

深度RNN模型可以通过将多个这样的LSTM块相互堆叠来构建，前一个块的输出构成下一个块的输入。在计算完顶层隐藏状态后，输出$z_{t}$可通过以下方式获得：

$$
z_{t}=\sigma\big(W_{h z}^{L}h_{t}^{L}+W_{x z}^{L}x_{t}^{L}+b^{L}\big).
$$

$$
z_{t}=\sigma\big(W_{h z}^{L}h_{t}^{L}+W_{x z}^{L}x_{t}^{L}+b^{L}\big).
$$

C. Multi-task Training

C. 多任务训练

Given that we have multiple supervision signals like systolic BP (SBP), diastolic BP (DBP) and mean BP (MBP) which are closely related to each other, we adopt multi-task training strategy to train one single model to predict SBP,

考虑到我们拥有多个密切相关的监督信号，如收缩压 (SBP)、舒张压 (DBP) 和平均血压 (MBP)，我们采用多任务训练策略来训练单一模型以预测 SBP、

DBP and MBP in parallel. Accordingly, the training objective is to minimize the mean squared error (MSE) of total $N$ training samples as follow:

DBP和MBP并行处理。因此，训练目标是最小化总$N$个训练样本的均方误差 (MSE) ，如下所示:

$$
\mathcal{L}\big({x_{1:T},y_{1:T}}{N}\big)=\frac{1}{N}\sum_{i=1}^{N}\sum_{t=1}^{T}\left\lVert\boldsymbol{z}{t}-\boldsymbol{y}_{t}\right\rVert^{2}+\lambda\lVert\boldsymbol{\theta}\rVert^{2},
$$

$$
\mathcal{L}\big({x_{1:T},y_{1:T}}{N}\big)=\frac{1}{N}\sum_{i=1}^{N}\sum_{t=1}^{T}\left\lVert\boldsymbol{z}{t}-\boldsymbol{y}_{t}\right\rVert^{2}+\lambda\lVert\boldsymbol{\theta}\rVert^{2},
$$

where $y_{t}=[S B P,D B P,M B P]$ represents ground truth, $z_{t}$ is corresponding prediction. And $\bar{||\theta||}^{2}$ represents the $L_{2}$ regulation of model parameters and $\lambda$ is the corresponding penalty coefficient. One advantage of multi-task training is that learning to predict different BP values simultaneously could implicitly encode the quantitative constrains among SBP, DBP and MBP.

其中 $y_{t}=[S B P,D B P,M B P]$ 代表真实值，$z_{t}$ 是对应的预测值。$\bar{||\theta||}^{2}$ 表示模型参数的 $L_{2}$ 正则项，$\lambda$ 是对应的惩罚系数。多任务训练的一个优势是，同时学习预测不同的血压值可以隐式编码收缩压 (SBP)、舒张压 (DBP) 和平均动脉压 (MBP) 之间的定量约束关系。

III. ANALYSIS OF DEEP RNN ARCHITECTURE

III. 深度RNN架构分析

RNNs are inherently deep in time because of their hidden states transition. Despite the depth in time, the proposed Deep RNN model is also deep along layer structure. To simplify the analysis, here we mainly focus on the gradient flow along the depth of layers. Through recursively updating Equation 12, we will have:

RNN因其隐藏状态转换而具有时间上的深度。尽管在时间维度上具有深度，所提出的深度RNN模型在层结构上也是深度的。为简化分析，这里我们主要关注沿层深度的梯度流动。通过递归更新公式12，我们将得到：

$$
x_{t}^{L}=x_{t}^{l}+\sum_{i=l}^{L-1}\mathcal{H}_{i}\left(x_{t}^{i},h_{t-1}^{i};W^{i}\right),
$$

$$
x_{t}^{L}=x_{t}^{l}+\sum_{i=l}^{L-1}\mathcal{H}_{i}\left(x_{t}^{i},h_{t-1}^{i};W^{i}\right),
$$

for any deeper layer $L$ and shallower layer $l$ . Equation 16 leads to nice backward propagation properties. Denoting the loss function as $\mathcal{L}$ , by the chain rule of back propagation we have:

对于任意深层 $L$ 和浅层 $l$ ，公式16具有良好的反向传播特性。将损失函数记为 $\mathcal{L}$ ，根据反向传播的链式法则可得：

$$
\begin{array}{r l}&{\displaystyle\frac{\partial\mathcal{L}}{\partial x_{t}^{l}}=\frac{\partial\mathcal{L}}{\partial x_{t}^{L}}\frac{\partial x_{t}^{L}}{\partial x_{t}^{l}}}\ &{\quad\quad=\displaystyle\frac{\partial\mathcal{L}}{\partial x_{t}^{L}}\big(1+\frac{\partial}{\partial x_{t}^{l}}\sum_{i=l}^{L-1}\mathcal{H}{i}\big(x_{t}^{i},h_{t-1}^{i};W^{i}\big)\big).}\end{array}
$$

$$
\begin{array}{r l}&{\displaystyle\frac{\partial\mathcal{L}}{\partial x_{t}^{l}}=\frac{\partial\mathcal{L}}{\partial x_{t}^{L}}\frac{\partial x_{t}^{L}}{\partial x_{t}^{l}}}\ &{\quad\quad=\displaystyle\frac{\partial\mathcal{L}}{\partial x_{t}^{L}}\big(1+\frac{\partial}{\partial x_{t}^{l}}\sum_{i=l}^{L-1}\mathcal{H}{i}\big(x_{t}^{i},h_{t-1}^{i};W^{i}\big)\big).}\end{array}
$$

Equation 17 shows that the gradient $\frac{\partial\mathcal{L}}{\partial x_{t}^{l}}$ can be decomposed into two additive terms: a term of $\textstyle{\frac{\partial{\mathcal{L}}}{\partial x_{t}^{L}}}$ that propagates information directly without through any weight layers, and another term $\begin{array}{r}{\frac{\partial\mathcal{L}}{\partial x_{t}^{L}}\left(\overset{\cdot\partial}{\partial x_{t}^{l}}\sum_{i=l}^{L-1}\mathcal{H}{i}\right)}\end{array}$ that propagates through the weight layers. The first term of $\textstyle{\frac{\partial{\mathcal{L}}}{\partial x_{t}^{L}}}$ ensures that supervised information could directly back propagate to any shallower layer $x_{t}^{l}$ . In general the term $\frac{\dot{\partial}}{\partial x_{t}^{l}}\sum_{i=l}^{\star}\mathcal{H}{i}$ cannot always be $-1$ for all samples in a mini-batch, so the gradient $\frac{\partial\dot{\mathcal{L}}}{\partial x_{t}^{l}}$ is unlikely to be canceled out. This implies that the gradients of a layer does not vanish even when the intermediate weights are arbitrarily small. This nice back propagation property allows us to train deep RNN model that owns more expressive power without worrying about the gradient vanishing problem.

方程17表明，梯度$\frac{\partial\mathcal{L}}{\partial x_{t}^{l}}$可分解为两个相加项：一项是$\textstyle{\frac{\partial{\mathcal{L}}}{\partial x_{t}^{L}}}$（不经过任何权重层直接传播信息），另一项是$\begin{array}{r}{\frac{\partial\mathcal{L}}{\partial x_{t}^{L}}\left(\overset{\cdot\partial}{\partial x_{t}^{l}}\sum_{i=l}^{L-1}\mathcal{H}{i}\right)}\end{array}$（通过权重层传播）。第一项$\textstyle{\frac{\partial{\mathcal{L}}}{\partial x_{t}^{L}}}$确保监督信息可直接反向传播至任意浅层$x_{t}^{l}$。通常对于小批量中的所有样本，$\frac{\dot{\partial}}{\partial x_{t}^{l}}\sum_{i=l}^{\star}\mathcal{H}{i}$项不可能恒为$-1$，因此梯度$\frac{\partial\dot{\mathcal{L}}}{\partial x_{t}^{l}}$不太可能被抵消。这意味着即使中间权重任意小，该层的梯度也不会消失。这种优良的反向传播特性使我们能够训练具有更强表达能力的深度RNN模型，而无需担心梯度消失问题。

IV. EXPERIMENTS

IV. 实验

We evaluate the proposed model on both a static and multi-day continuous BP dataset. Root mean square error (RMSE) is used as the evaluation metric, which is defined as $R M S E=\sqrt{\frac{1}{N}\sum_{i=1}^{N}\sum_{t=1}^{T}\lVert\boldsymbol{z}_{t}-\boldsymbol{y}_{t}\rVert^{2}}$ . On both datasets we compare our model with the following reference models:

我们在静态和多日连续血压数据集上评估了所提出的模型。采用均方根误差 (RMSE) 作为评估指标，其定义为 $R M S E=\sqrt{\frac{1}{N}\sum_{i=1}^{N}\sum_{t=1}^{T}\lVert\boldsymbol{z}_{t}-\boldsymbol{y}_{t}\rVert^{2}}$ 。在两个数据集上，我们将模型与以下参考模型进行对比:

Fig. 3: Illustration of PPG feature.

图 3: PPG特征示意图

A. Dataset

A. 数据集

Static continuous BP dataset. The dataset, including ECG, PPG and BP were obtained from 84 healthy people including 51 males and 33 females. ECG and PPG signal were acquired with Biopac system and reference continuous BP was measured by Finapres system simultaneously in each experiment. The BP, ECG and PPG data of each subject were recorded at sampling frequency of $1000~\mathrm{Hz}$ for 10 minutes at the rest status.

静态连续血压数据集。该数据集包含从84名健康人（51名男性和33名女性）采集的心电图(ECG)、光电容积图(PPG)和血压(BP)数据。每次实验中，ECG和PPG信号通过Biopac系统采集，同时使用Finapres系统测量参考连续血压。每位受试者在静息状态下以$1000~\mathrm{Hz}$采样频率记录了10分钟的BP、ECG和PPG数据。

Multi-day continuous BP dataset. Similar dataset was obtained from 12 healthy subjects including 11 males and 1 female. The BP, ECG and PPG data of each subject were recorded for 8 minutes at the rest status in a multi-day period, namely 1st day, 2nd day, 4th day and 6 moth after the first day.

多日连续血压数据集。该数据集采集自12名健康受试者，包括11名男性和1名女性。每位受试者在静息状态下分别于第1天、第2天、第4天及第6个月进行多日监测，每次记录8分钟的血压(BP)、心电图(ECG)和光电容积图(PPG)数据。

B. Data Representation

B. 数据表示

Since the primary goal of this paper is to prove the importance of modeling temporal dependencies in BP dynamics for accurate BP prediction, we simply select 7 representative handcrafted features of ECG and PPG signals (shown in Fig 3) as follows:

本文的主要目标是证明建模血压动态中时间依赖关系对准确预测血压的重要性，因此我们仅选取了7个具有代表性的心电(ECG)和光电容积脉搏波(PPG)信号手工特征(如图3所示):

Now input $X_{T}$ becomes a $7\times T$ matrix, and each row of $X_{T}$ is normalized to have zero-mean and unit-variance. Further model performance gain could be expected by adding more informative features as model inputs.

现在输入 $X_{T}$ 变为一个 $7\times T$ 矩阵，且 $X_{T}$ 的每一行都经过零均值和单位方差归一化处理。通过添加更多信息丰富的特征作为模型输入，有望进一步提升模型性能。

1Chen’s PTT model only support SBP prediction, thus only its SBP prediction results were used for comparison with other models.

1 Chen的PTT模型仅支持收缩压(SBP)预测，因此仅使用其SBP预测结果与其他模型进行比较。

Fig. 4: Bland-Altman plots of the overall SBP and DBP predictions by a DeepRNN-4L model on the static continuous BP dataset.

图 4: DeepRNN-4L模型在静态连续血压数据集上对整体收缩压(SBP)和舒张压(DBP)预测的Bland-Altman图。

TABLE I: Detailed analysis of our Deep RNN models with comparison with different reference models. DeepRNN-xL represents a x layer RNN model. All the models are validated on the static continuous BP dataset. (unit: $\mathrm{mmHg}$ )

表 I: 深度 RNN 模型与不同参考模型的详细对比分析。DeepRNN-xL 表示 x 层 RNN 模型。所有模型均在静态连续血压数据集上验证。(单位: $\mathrm{mmHg}$ )

模型	RMSE(SBP)	RMSE(DBP)
PTT-Chen [3]	5.31	-
PTT-Poon [4]	5.75	3.50
BLR	7.45	6.20
SVR	6.54	6.28
DT	4.45	2.80
Kalman Filter	5.17	3.09
LSTM	6.31	4.58
BiLSTM	5.25	3.04
DeepRNN-2L	5.13	3.73
DeepRNN-3L	4.92	3.13
DeepRNN-4L	3.73	2.43

C. Implementation Details

C. 实现细节

All the RNN models were trained using mini-batches of size 64 and the Adam optimizer [16]. For each minibatch, we computed the norm of gradients $|g|$ . If $|g|>v$ , the gradients were scaled by $g v/\lVert g\rVert$ (v is set as 5 by default.). We run our model with different number of layers, with hidden state size as 128 at each layer. The sequence length $T$ of each training sample is set to 32, and it could be larger if deeper model is adopted. For saving computational cost, we only adopt bidirectional LSTM at first layer. Due to limited training samples of our BP prediction problem, the maximum depth of deep RNN model was set as 4 to avoid over fitting. Each training dataset was divided such that $70%$ of the data was used for training, $10%$ for validation and $20%$ for test. SBP, DBP and MBP were normalized to $(0,1]$ by their corresponding maximum, respectively. For evaluation on the multi-day continuous BP dataset, all deep RNN models were first pretrained on the static BP dataset then finetuned using part of the first-day data, and finally tested on the rest of the first-day data as well as the following days’ data.

所有RNN模型均采用64的小批量(mini-batch)和Adam优化器[16]进行训练。对于每个小批量，我们计算梯度范数$|g|$。若$|g|>v$，则按$g v/\lVert g\rVert$对梯度进行缩放（默认设置v=5）。我们测试了不同层数的模型，每层隐藏状态大小设为128。每个训练样本的序列长度$T$设为32，若采用更深层模型可适当增大。为节省计算成本，仅在第一层使用双向LSTM。由于血压预测问题的训练样本有限，深层RNN模型的最大深度设为4以防止过拟合。每个训练数据集按$70%$、$10%$和$20%$的比例划分为训练集、验证集和测试集。收缩压(SBP)、舒张压(DBP)和平均动脉压(MBP)分别按其最大值归一化至$(0,1]$区间。针对多日连续血压数据集的评估，所有深层RNN模型先在静态血压数据集上预训练，然后用首日部分数据微调，最终在首日剩余数据及后续天数数据上进行测试。

V. EXPERIMENTAL RESULTS

五、实验结果

Validation on static continuous BP dataset. As shown in Table I, the PTT models yield slightly better results than BLR and SVR models, but show poorer performance than DT, kalman filter, bidirectional LSTM and deep RNN (DeepRNN) models. The best accuracy was obtained by our 4-layer deep RNN (DeepRNN-4L) model which achieves a RMSE of 3.73 and 2.43 for SBP and DBP prediction respectively. The Bland-Altman plots (Figure 4) indicate that the DeepRNN4L predictions agreed well with the ground truth, with $95%$ of the differences lie within the agreement area. Figure 6 qualitatively shows the DeepRNN-4L prediction result on a representative subject from the static continuous BP dataset.

静态连续血压数据集的验证。如表1所示，PTT模型的结果略优于BLR和SVR模型，但表现不如DT、卡尔曼滤波器、双向LSTM和深度RNN (DeepRNN) 模型。最佳准确率由我们的4层深度RNN (DeepRNN-4L) 模型获得，其在SBP和DBP预测上的RMSE分别为3.73和2.43。Bland-Altman图 (图4) 表明DeepRNN4L的预测与真实值高度一致，95%的差异位于一致性区域内。图6定性展示了DeepRNN-4L模型在静态连续血压数据集上一个典型受试者的预测结果。

By incorporating a bidirectional structure in the model, i.e. by using the bidirectional LSTM (BiLSTM), the prediction accuracy is improved significantly as compared to the vanilla LSTM, with $17%$ decrease in the SBP RMSE and $34%$ decrease in DBP RMSE. Furthermore, it was observed that the improvement of prediction accuracy is enhanced with increasing depth of the DeepRNN network. For instance, replacing DeepRNN-2L with DeepRNN-4L results in $27%$ and $35%$ improvement on SBP and DBP prediction respectively. When we stack up to a 5-layer DeepRNN, the model tend to overfit and no clear benefits of depth can be observed any more.

通过在模型中引入双向结构（即使用双向LSTM (BiLSTM)），预测准确率相比普通LSTM显著提升，收缩压(SBP)均方根误差降低17%，舒张压(DBP)均方根误差降低34%。此外，研究发现DeepRNN网络深度增加会进一步提升预测精度。例如，将DeepRNN-2L替换为DeepRNN-4L后，SBP和DBP预测分别提升了27%和35%。但当网络层数增加到5层时，模型开始出现过拟合现象，继续增加深度已无法带来明显收益。

Validation on multi-day continuous BP dataset. Figure 5 compares the prediction performance of deep RNN against the reference models. It can be clearly seen that the DeepRNN models yield much better performance as compared to the PTT and regression models, likely due to the temporal dependencies modeling in the DeepRNN models. Kalman filter could model the time dependencies in sequence but dose not perform as well as DeepRNN models. It is likely because of the linearity assumption of kalman filter that both state transition and measurement functions are linear. This assumption may limit its capability to model the complex temporal dependencies in BP dynamics. The best accuracy was obtained by our DeepRNN-4L model which achieves a RMSE of 3.84, 5.25, 5.80 and $5.81\mathrm{mmHg}$ for the 1st day, 2nd day, 4th day and 6th month after the 1st day SBP prediction, and $\mathrm{1.80,4.78,5.0,5.21mmHg}$ for corresponding DBP prediction, respectively. As shown in Figure 5, all the PTT models, regression models and kalman filter exhibit pronounced accuracy decay from the second day. Although the prediction accuracy of the DeepRNN model also drops after the first day, it consistently provides the lowest RMSE values among all models. Figure 7 qualitatively shows the capability of DeepRNN to track long-term BP variation.

在多日连续血压数据集上的验证。图5比较了深度RNN与参考模型的预测性能。可以清楚地看到，DeepRNN模型相比PTT和回归模型表现出更优的性能，这可能是由于DeepRNN模型中对时间依赖关系的建模。卡尔曼滤波虽然能对序列中的时间依赖关系建模，但表现不如DeepRNN模型。这可能是由于卡尔曼滤波的线性假设，即状态转移和测量函数都是线性的。这一假设可能限制了其建模血压动态中复杂时间依赖关系的能力。我们的DeepRNN-4L模型获得了最佳准确度，在第1天、第2天、第4天和第6个月的收缩压预测中分别实现了3.84、5.25、5.80和$5.81\mathrm{mmHg}$的RMSE，在相应的舒张压预测中分别为$\mathrm{1.80,4.78,5.0,5.21mmHg}$。如图5所示，所有PTT模型、回归模型和卡尔曼滤波从第二天开始都表现出明显的准确度下降。尽管DeepRNN模型在第一天的预测准确度也有所下降，但它在所有模型中始终提供最低的RMSE值。图7定性地展示了DeepRNN跟踪长期血压变化的能力。

RMSE of SBP

图 1: SBP的均方根误差(RMSE)

RMSE of DBP Fig. 5: Overall RMSE comparison of different models on the multi-day continuous BP dataset.

图 5: 不同模型在多日连续血压数据集上的整体均方根误差 (RMSE) 对比。

TABLE II: Performance comparison of DeepRNN-4L model with residual connections vs. without residual connections. The result is obtained on the static continuous BP dataset. (unit: $\mathrm{mmHg}$ )

表 II: 带残差连接与不带残差连接的 DeepRNN-4L 模型性能对比。结果基于静态连续血压数据集获得。(单位: $\mathrm{mmHg}$)

	RMSE (收缩压)	RMSE (舒张压)
DeepRNN-4L (无残差)	5.31	3.13
DeepRNN-4L (带残差)	3.73	2.43

Importance of residual connections. To investigate the importance of residual connections, we conduct ablation study on the static continuous BP dataset. As shown in Table II, DeepRNN model incorporated with residual connections works considerably better than the counterpart. During training, we found residual connections significantly improve the gradient flow in the backward pass which make deep neural network easier to optimize. Accordingly, better performance could be obtained due to more expressive deep structure. The detailed reason for such computational benefit has been explained in section III.

残差连接的重要性。为了研究残差连接的重要性，我们在静态连续BP数据集上进行了消融实验。如表II所示，加入残差连接的DeepRNN模型表现明显优于未加入的版本。在训练过程中，我们发现残差连接显著改善了反向传播中的梯度流动，使深度神经网络更易于优化。因此，更具表现力的深层结构可获得更好的性能。该计算优势的具体原因已在第III节中说明。

Importance of multi-task training. Table III shows that multi-task training strategy can boost the prediction performance as compared with separate training of individual models. It can be explained by that the different training objectives involved in each task are strongly correlated and thus share a lot of data representations that capture the underlying factors, which can be learned by the same model structure. Hence, by learning the shared representations, it can crucially improve the model generalization ability.

多任务训练的重要性。表 III 表明，与单独训练单个模型相比，多任务训练策略可以提升预测性能。这可以解释为每个任务涉及的不同训练目标高度相关，因此共享了大量捕捉潜在因素的数据表征，这些表征可以通过相同的模型结构学习。因此，通过学习共享表征，可以显著提高模型的泛化能力。

TABLE III: Investigation of DeepRNN with different settings. Models trained using multi-task objective are marked with $\because\cdot\cdot$ . The result is obtained on the static continuous BP dataset. (unit: $\mathrm{mmHg}$ )

表 III: 不同设置下DeepRNN的探究。使用多任务目标训练的模型标记为 $\because\cdot\cdot$。结果在静态连续血压数据集上获得。(单位: $\mathrm{mmHg}$)

	RMSE (收缩压)	RMSE (舒张压)
DeepRNN-2L	6.24	4.55
DeepRNN-3L	5.05	3.30
DeepRNN-4L	4.27	3.02
DeepRNN-2L↑	5.13	3.73
DeepRNN-3L↑	4.92	3.13
DeepRNN-4L↑	3.73	2.43

VI. CONCLUSIONS

VI. 结论

In this work, we demonstrated that modeling the temporal dependency in BP dynamics can significantly improve longterm BP prediction accuracy, which is one of the most challenging problems in cuffless BP estimation. We proposed a novel deep RNN that incorporated with bidirectional LSTM and residual connections to tackle this challenge. The experimental results show that the deep RNN model achieves the state-of-the-art accuracy on both static and multi-day continuous BP datasets.

在这项工作中，我们证明了建模血压动态中的时间依赖性可以显著提高长期血压预测的准确性，这是无袖带血压估计中最具挑战性的问题之一。我们提出了一种结合双向LSTM和残差连接的新型深度RNN来解决这一挑战。实验结果表明，该深度RNN模型在静态和多日连续血压数据集上均达到了最先进的准确度。

REFERENCES

参考文献

Fig. 6: Comparison of the ground truth and DeepRNN prediction of two representative subjects from static continuous BP dataset. Each column represents the SBP and DBP predictions of one subject.

图 6: 静态连续血压数据集中两名代表性受试者的真实值与DeepRNN预测结果对比。每列展示一名受试者的收缩压(SBP)和舒张压(DBP)预测值。

Fig. 7: Comparison of the ground truth and DeepRNN prediction of one representative subject from multi-day continuous BP dataset. Figure (a), (b), (c) and (d) represent the results of 1st day, 2nd day, 4th day and 6th month after the 1st day, respectively.

图 7: 多日连续血压数据集中一名代表性受试者的真实值与DeepRNN预测结果对比。(a) 、(b) 、(c) 和 (d) 分别表示第1天、第2天、第4天及第1天后第6个月的测试结果。

machine translation system: Bridging the gap between human and machine translation,” arXiv preprint arXiv:1609.08144, 2016. [16] D. Kingma and J. Ba, “Adam: A method for stochastic optimization,” arXiv preprint arXiv:1412.6980, 2014.

机器翻译系统：弥合人类与机器翻译的差距，arXiv预印本arXiv:1609.08144，2016年。[16] D. Kingma和J. Ba， "Adam：一种随机优化方法"，arXiv预印本arXiv:1412.6980，2014年。