# CARN: Fast, Accurate, and Lightweight Super-Resolution with Cascading Residual Network

＞　译者语：一个轻量级好用的超分辨率算法
＞ 中英文对照地址 https://aiqianji.com/blog/article/7
＞ 论文中英文对照合集 ： https://aiqianji.com/blog/articles

## ABSTRACT

In recent years, deep learning methods have been successfully applied to single-image super-resolution tasks. Despite their great performances, deep learning methods cannot be easily applied to real-world applications due to the requirement of heavy computation. In this paper, we address this issue by proposing an accurate and lightweight deep learning model for image super-resolution. In detail, we design an architecture that implements a cascading mechanism upon a residual network. We also present a variant model of the proposed cascading residual network to further improve efficiency. Our extensive experiments show that even with much fewer parameters and operations, our models achieve performance comparable to that of state-of-the-art methods.

###### Keywords:

Super-Resolution, Deep Convolutional Neural Network

## I. 介绍　Introduction

Super-resolution (SR) is a computer vision task that reconstructs a high-resolution (HR) image from a low-resolution (LR) image. Specifically, we are concerned with single image super-resolution (SISR), which performs SR using a single LR image. SISR is generally difficult to achieve due to the fact that computing the HR image from an LR image is a many-to-one mapping. Despite such difficulty, SISR is a very active area because it can offer the promise of overcoming resolution limitations, and could be used in a variety applications such as video streaming or surveillance system.
Recently, convolutional neural network-based(CNN-based) methods have provided outstanding performance in SISR tasks[srcnn2014, vdsr2016, lapsrn2017]. From the SRCNN[srcnn2014] that has three convolutional layers to MDSR[mdsr2017] that has more than 160 layers, the depth of the network and the overall performance have dramatically grown over time. However, even though deep learning methods increase the quality of the SR images, they are not suitable for real-world scenarios. From this point of view, it is important to design lightweight deep learning models that are practical for real-world applications. One way to build a lean model is reducing the number of parameters. There are many ways to achieve this[han2015deep, squeezenet], but the most simple and effective approach is to use a recursive network. For example, DRCN[drcn2016] uses a recursive network to reduce redundant parameters, and DRRN[drnn2017] improves DRCN by adding a residual architecture to it. These models decrease the number of model parameters effectively when compared to the standard CNN and show good performance. However, there are two downsides to these models: 1) They first upsample the input image before feeding it to the CNN model, and 2) they increase the depth or the width of the network to compensate for the loss due to using a recursive network. These points enable the model to maintain the details of the image when reconstructed, but at the expense of increased number of operations and inference time.

Most of the works that aim to build a lean model focused primarily on reducing the number of parameters. However, as mentioned above, the number of operations is also an important factor to consider in real-world scenarios. Consider a situation where an SR system operates on a mobile device. Then, the execution speed of the system is also of crucial importance from a user-experience perspective. Especially the battery capacity, which is heavily dependent on the amount of computation performed, becomes a major problem. In this respect, reducing the number of operations in the deep learning architectures is a challenging and necessary step that has largely been ignored until now. Another scenario relates to applying SR methods on video streaming services. The demand for streaming media has skyrocketed, and hence requires large storage to store massive multimedia data. It is therefore imperative to compress data using lossy compression techniques before storing. Then, an SR technique can be applied to restore the data to the original resolution. However, because latency is the most critical factor in streaming services, the decompression process (i.e., super-resolution) has to be performed in real-time. To do so, it is essential to make the SR methods lightweight in terms of the number of operations.

To handle these requirements and improve the recent models, we propose a Cascading residual network (CARN) and its variant CARN-Mobile (CARN-M). We first build our CARN model to increase the performance, and extend it to CARN-M to optimize it for speed and the number of operations. Following the FSRCNN[fsrcnn2016], CARN and CARN-M take the LR images and compute the HR counterparts as the output of the network. The middle parts of our models are designed based on the ResNet[resnet]. The ResNet architecture has been widely used in deep learning-based SR methods[drnn2017, mdsr2017] because of the ease of training and superior performance. In addition to the ResNet architecture, CARN uses a cascading mechanism at both the local and the global level to incorporate the features from multiple layers. This has the effect of reflecting various levels of input representations in order to receive more information. In addition to the CARN model, we also provide the CARN-M model that allows the designer to tune the trade-off between the performance and the heaviness of the model. It does so by means of efficient residual block (residual-E) and recursive network architecture, which we describe in more detail in Section 3.

In summary, our main contributions are as follows: 1) We propose CARN, a neural network for SR based on the cascading modules, which achieves high performance. Our main modules, which we call the cascading modules, effectively boost the performance via multi-level representation and multiple shortcut connections. 2) We also propose CARN-M for efficient SR by combining the efficient residual block and the recursive network scheme. 3) We show through extensive experiments, that our model uses only a modest number of operations and parameters to achieve competitive results. Our CARN-M, which is the more lightweight SR model, shows comparable results to others with much fewer operations (Fig. 1).

1)提出了一种基于级联模块的神经网络 CARN，它在超分辨率任务上取得了较高的性能(图 1)。我们的级联模块通过多级表示和多条 shortcut 连接有效提升性能。
2)结合 residual-E 模块和递归网络方案，提出了有效的超分辨率算法：CARN-M。
3)我们通过广泛的实验表明，我们的模型仅使用适度数量的运算操作和参数来实现有竞争力的结果。我们的 CARN-M 是更轻量级的 SR 模型，运算操作更少，而效果显示出与其他模型相当的结果。

Since the success of AlexNet[alexnet] in image recognition task[imagenet2009], various deep learning approaches have been applied to many computer vision tasks[ssd, fasterrcnn, deconvnet, color]. The SISR task is one such task, and we present an overview of the deep learning-based SISR in section 2.1.

Another area we deal with in this paper is model compression. Recent deep learning models focus on squeezing model parameters and operations for application in low-power computing devices, which has many practical benefits in real-world applications. We briefly review this area in section 2.2.

### 2.1 基于深度学习的图像超分辨率 DEEP LEARNING BASED IMAGE SUPER-RESOLUTION

Recently, deep learning based models have shown dramatic improvements in the SISR task. Dong et al.[srcnn2014] first proposed a deep learning-based SR method, called SRCNN, which outperformed traditional algorithms. However, SRCNN has a large number of operations compared to its depth, as input images are upsampled before being fed into the network. Taking a different approach from SRCNN, FSRCNN[fsrcnn2016] and ESPCN[espcn2016] upsample images at the end of the networks. This latter approach leads to the reduction in the number of operations compared to the former. However, the overall performance could be degraded if there are not enough layers after the upsampling layer. Moreover, they cannot manage multi-scale training, as the input image size differs for each upsampling scale.

Despite the fact that the power of deep learning comes from deep layers, the aforementioned methods have settled for shallow layers because of the difficulty in training. To better harness the depth of deep learning models, Kim et al.[vdsr2016] proposed VDSR, which uses residual learning to map the LR images x to their residual images r. Then, VDSR produces the SR images y by adding the residual back into the original, i.e., y=x+r.

On the other hand, LapSRN[lapsrn2017] uses a Laplacian pyramid architecture to increase the image size gradually. By doing so, LapSRN effectively performs SR on extremely low-resolution cases with fewer number of operations compared to VDSR. The main difference is that VDSR upsamples the image at the beginning whereas LapSRN does so sequentially.

Another issue of DL-based SR is how to reduce the parameters and operation. For example, DRCN[drcn2016] uses a recursive network to reduce parameters by engaging in redundant usages of a small number of parameters. DRRN[drnn2017] improves upon DRCN by combining the recursive and residual network schemes to achieve better performance with less parameters. However, DRCN and DRRN use very deep networks to compensate for the loss of performance and hence these models require heavy computing resources. Hence, we aim to build a model that is lightweight in both size and computation. We will briefly discuss previous works that address such model efficiency issues in the following section.

### 2.2 高效的神经网络 EFFICIENT NEURAL NETWORK

Lately, there has been rising interest in building small and efficient neural networks[squeezenet, han2015deep, mobilenets]. These approaches can be categorized into two groups: 1) Compressing pretrained networks, and 2) designing small but efficient models. Han et al.[han2015deep] proposed deep compressing techniques, which consist of pruning, vector quantization, and Huffman coding to reduce the size of a pretrained network. In the latter category, SqueezeNet[squeezenet] builds an AlexNet-based architecture and achieves nearly the same performance level with 50× fewer parameters compared to AlexNet. MobileNet[mobilenets] builds an efficient neural network by applying depthwise separable convolution introduced in Sifre et al.[dwconv]. With depthwise separable convolution, it is easy to build lightweight, deep neural networks, which allow end-users to choose the appropriate network size based on the application constraints. Because of this simplicity, we also apply this technique in the residual block with some modification to achieve a lean neural network.

1)压缩预处理网络，
2)设计小而有效的模型。
Han et al.[11]提出了深度压缩技术，包括剪枝、矢量量化和霍夫曼编码，以减小预训练网络的大小。在后一类中，SheeNet[19]构建了一个基于 AlexNet 的架构，并以少 50 倍的参数实现了相当的性能水平。MobileNet [16]通过应用 Sifre 等人[34]中介绍的深度可分卷积建立了一个有效的网络。基于这种简化方法，我们在残差块中应用这种技术，并对其进行一些修改，以实现一个精简的神经网络。

## 3 提出的方法 Proposed Method

As mentioned in Section 1, we propose two main models: CARN and CARN-M. CARN is designed to make a high-performing SR model while suppressing the number of operations compared to the state-of-the-art methods. Based on CARN, we design CARN-M, which is a much more efficient SR model in terms of both parameters and operations.

### 3.1 级联残差网络 Cascading Residual Network

Our CARN model is based on ResNet[resnet2016]. The main difference between CARN and ResNet is the presence of local and global cascading modules. Fig. 2 (b) graphically depicts how the global cascading occurs. The outputs of intermediary layers are cascaded into the higher layers, and finally converge on a single 1x1 convolution layer. Note that the intermediary layers are implemented as cascading blocks, which host local cascading connections themselves. Such local cascading operations are shown in (c) and (d). Local cascading is almost identical to global one except that the unit blocks are plain residual blocks.

To express the implementation mathematically, let f be a convolution function and τ be an activation function. Then, we can define the i-th residual block Ri, which has two convolutions followed by a residual addition, as

Here, H^i is the output of the i-th residual block, W^i_R is the parameter set of the entire residual block, and W^{i,j}_Ris the parameter of the j-th convolution layer in the i-th block. With this formulation, we denote the output feature of the final residual block of ResNet asH^{u}, which becomes the input to the upsampling block.

Note that because our model has a single convolution layer before each residual block, the first residual block gets f(X;Wc) as input, where W_c is the parameter of the convolution layer.

In contrast to ResNet, our CARN model has a local cascading block illustrated in block (c) of Fig. 3 instead of a plain residual block. In here, we denote B^{i,j}as the output of the j-th residual block in the i-th cascading block, and W^i_c as the set of parameters of the i-th local cascading block. Then, the i-th local cascading block B^i_localis defined as

Finally, we define the output feature of the final cascading block H^u by combining both the local and global cascading.
H^0 is the output of the first convolution layer. In our settings, we set u=3 in CARN to match its depth with that of the corresponding ResNet.

The main difference between CARN and ResNet lies in the cascading mechanism. As shown in Fig. 2, CARN has global cascading connections represented as the blue arrows, each of which is followed by an 1×1 convolution layer. Cascading on both the local and global levels has two advantages: 1) The model incorporates features from multiple layers, which allows to learn multi-level representations. 2) Multi-level cascading connection behaves as multi-level shortcut connections that quickly propagate information from lower to higher layers.

Multi-level representation is used in many deep learning methods[lee2017multi, long2015fully] because of its successful performance with simple modifications. Our CARN follows such a scheme, but we apply this arrangement to a variety of feature levels to boost performance, as shown in equation 4. By doing so, our model reconstructs the LR image based on multi-level features. This facilitates the model to restore the details and contexts of the image simultaneously. As a result, our models effectively improve not only primitive objects such as stripes or lines, but also complex objects like hand or street lamps.

Another reason for adapting the cascading scheme is to leverage the effect of shortcut connection. The reason for the improved performance is two fold: First, the propagation of information follows multiple paths. The benefit of multi-path is well discussed in many deep learning models[densenet, unet]. Second, by adding extra convolution layers, our model can learn to choose the right pathway with the given input information flows. However, the strength of multiple shortcut is degraded when we use only one of local or global cascading, especially the local connection. We elaborate the details and present a case study on the effects of cascading mechanism in Section 4.4.

CARN 和 ResNet 的主要区别在于级联机制。如图 2 所示，CARN 具有用蓝色箭头表示的全局级联连接，每个连接后面是 1×1 卷积层。在局部和全局两个层次上进行级联有两个优点：
1)模型包含来自多个层次的特征，这允许学习多层次的表示。
2)多级级联连接表现为多级快捷连接，可将信息从较低层快速传播到较高层(反向传播时，反之亦然)。
CARN 采用了[25，28]中的多级表示方案，但我们如公式 4 所示将这种方案应用于不同特征级别，以提高性能。通过这样做，我们的模型基于多级特征重建低分辨率图像。这有助于模型同时恢复图像的细节和上下文。因此，我们的模型不仅在原始图像上效果很好，在复杂细节的图像上，也能有效地提高效果。

### 3.2 高效级联残差网络 EFFICIENT CASCADING RESIDUAL NETWORK

To improve the efficiency of CARN, we first propose an efficient residual (residual-E) block. We use an approach similar to that of MobileNet[mobilenets], but our formulation is more general. Our residual-E block consists of two 3×3 group convolutions and one 1×1 convolution, as shown in Fig. 3 (b). The latter convolution is the same as pointwise convolution, which is used in depthwise separable convolution[mobilenets]. The former convolution is a group extension of the depthwise convolution. The advantage of using group convolution over depthwise convolution is that it makes the efficiency of the model tunable. More precisely, the user can choose the group size appropriately because the group size and performance are usually in a trade-off relationship. The analysis on the cost efficiency of using the residual-E block is as follows.

Let K be the kernel size andC_inC_out be the number of input and output channels, respectively. Because we retain the feature resolution of the input and output by padding, we can denote F to be both the input and output feature size. The cost of a plain residual block is given as

Note that we only count the cost of convolution layers and ignore the addition or activation because both the plain and the efficient residual blocks have the same amount of cost in terms of addition and activation.

Let G be the group size. Then, the cost of a residual-E block, which consist of two group convolutions and one 1×1 convolution, is as given in equation 6.

By changing the plain residual block to our efficient residual block, we can reduce the computation by the ratio of

Because our model uses a kernel of size 3×3 for all group convolutions, and the number of channels is constantly 64, using an efficient residual block instead of a standard residual block can reduce the computation from 1.8 up to 14 times depending on the group size. To find the best trade-off between performance and computation, we performed an extensive case study in Section 4.4.

To further reduce the parameters, we apply a technique similar to the one used by the recursive network. That is, we make the parameters of the Cascading blocks shared, effectively making the blocks recursive. Fig. 3 (d) shows our block after applying the recursive scheme. This approach reduces the parameters by up to three times of their original number.

Despite the above measures, the upsampling block is another obstacle, as the number of channels has to be increased quadratically with respect to the upsampling ratio[espcn2016]. Moreover, we use multi-scale learning to boost the performance, so the parameters of the upsampling block are increased by up to 48% in CARN and 75% in CARN-M. To mitigate this problem, we replace the 3×3 convolution layer with a 1×1 convolution inside the upsampling block. This trick reduces the parameters by nine times but with little degradation in performance.

### 3.3 与最新技术的比较

SRDenseNet[37]使用 dense block 和 skip connection。与我们的模型的区别是：
1)我们使用全局级联，这比 skip connection 更通用。在 SRDenseNet 中，各级特征都在最后一个 dense block 的最后组合，但是我们的全局级联方案连接所有块，这样可以实现多级的 skip connection。
2)SRDenseNet 通过 contact 操作保留 dense block 的局部信息，而我们通过 1 × 1 卷积层逐步收集。使用额外的 1×1 卷积层可以有更高的表示能力。

MemNet [36]的动机和我们类似。然而，与我们的机制有两个主要区别。
1)在 MemNet 的存储块内部，每个递归单元的输出特征在网络的末端连接，然后用 1×1 卷积进行融合。而我们融合局部块中每个可能点的特征，这可以通过附加的卷积层和非线性来提高表示能力。总的来说，由于训练的困难，这种代表力往往得不到满足。然而，我们通过使用本地和全局级联机制克服了这个问题。我们将在第 4.4 节讨论细节。
2) MemNet 将向上采样的图像作为输入，因此加和乘的数量比我们的多。我们模型的输入是一个低分辨率图像，我们在网络末端对其进行上采样，因此我们计算效率更高。

## 4 实验结果 Experimental Results

### 4.1 数据集 DATASETS

There exist diverse single image super-resolution datasets, but the most widely used ones are the 291 image set by Yang et al.[yang2010] and the Berkeley Segmentation Dataset[b100]. However, because these two do not have sufficient images for training a deep neural network, we additionally use the DIV2K dataset[div2k]. The DIV2K dataset is a newly-proposed high-quality image dataset, which consists of 800 training images, 100 validation images, and 100 test images. Because of the richness of this dataset, recent SR models[mdsr2017, btsrn2017, cnf2017, selnet] use DIV2K as well. We use the standard benchmark datasets such as Set5[set5], Set14[yang2010], and B100[b100] for testing and benchmarking.

DIV2K 数据集是一个新提出的高质量图像数据集，由 800 幅训练图像、100 幅验证图像和 100 幅测试图像组成。由于这个数据集的丰富性，最近的 SR 模型[4，8，26，31]也使用了 DIV2K。 我们使用标准基准数据集，如集合 5 [3]、集合 14 [39]、B100 [29]和 Urban100 [18]进行测试和基准测试。

### 4.2 实施和训练细节 IMPLEMENTATION AND TRAINING DETAILS

We use the RGB input patches of size 48×48 from the LR images for training. We sample the LR patches randomly and augment them with random horizontal flips and 90 degree rotation. We train our models with the ADAM optimizer[adam] by setting $\beta_{1}=0.9$、$\beta_{2}=0.999$, and ϵ=10−8 in 5×105 steps. The minibatch size is 32, and the learning rate begins with 10−4 and is halved every 3×105 steps. All the weights and biases are initialized by equation 8. In here, we denote cin as the number of channels of input feature map.

The most well known and effective weight initialization methods are given by Glorot et al.[glorot2010] and He et al.[he2015delving]. However, such initialization routines tend to set the weights of our multiple narrow 1×1 convolution layers very high, resulting in an unstable training. Therefore, we sample the initial values from a uniform distribution to alleviate the initialization problem.

To train our model in a multi-scale manner, we first select the scaling factor as one of 2×, 3×, and 4× because our model can only process a single scale for each batch. Then, we can construct and argument our input batch, as described above. We use the L1 loss as our loss function instead of the L2. The L2 loss is widely used in the image restoration task due to its relationship with the peak signal-to-noise ratio (PSNR). However, in our experiments, L1 provides better convergence and performance. The downside of the L1 loss is that the convergence speed is relatively slower than that of L2 without the residual block. However, this drawback could be mitigated by using a ResNet style architecture such as that used by ours.

Glorot 等人[10]和何等人[12]给出了最著名和最有效的权重初始化方法。然而，这种初始化程序倾向于将我们的多个窄的 1×1 卷积层的权重设置得非常高，导致训练不稳定。因此，我们从均匀分布中采样初始值，以缓解初始化问题。

### 4.3 与最先进方法的比较 COMPARISON WITH STATE-OF-THE-ART METHODS

We compare the proposed CARN and CARN-M with state-of-the-art SR methods on two commonly-used image quality metrics: PSNR and the structural similarity index (SSIM)[ssim]. One thing to note here is that we represent the number of operations by Mult-Adds. Mult-Adds is the number of composite multiply-accumulate operations for a single image. In Fig. 4, we compare CARN(-M) against the benchmark algorithms using the number of operations (by Mult-Adds) and parameters. Our CARN model achieves result comparable with SelNet[selnet], both of which outperform all state-of-the-art models except MDSR[mdsr2017]. This is not surprising, because MDSR has nearly six times more parameters than that of either ours or SelNet’s. CARN and DRCN[drcn2016] have similar number of parameters, but CARN outperforms DRCN by using 100× fewer Mult-Adds. Our CARN-M model also outperforms most of the benchmark methods with 2,319M Mult-Adds, which is nearly half the amount of SRCNN[srcnn2014].

MDSR [26]的性能比我们的好，这并不奇怪，因为 MDSR 的 8M 参数比我们的参数多近 6 倍。CARN-M 模型也优于大多数基准方法，并显示出与重型模型相当的结果。

Table 1 shows the quantitative comparisons of the performances over the Set5, Set14, and B100 datasets. With only a few operations, our CARN outperforms almost all benchmark models on most of the datasets and shows similar scores with SelNet. In detail, the CARN has a similar parameters to DRCN, but the number of operations is on par with SRCNN[srcnn2014]. Our lightweight model, CARN-M has approximately the same number of parameters as DRRN (305K), but shows comparable results to others using approximately 2,000 times fewer operations. Note that MDSR is excluded from this table, because we only compare models that have roughly similar number of parameters as ours. In detail, MDSR has a parameter set whose size is 4.5 times larger than that of the second-largest model, which is DRCN.

As shown in Table 1, CARN model has a slightly larger number of parameters compared to the ones with similar complexity (i.e. between CARN and SelNet). However, this is because our models have to contain all possible upsampling layers in order to perform multi-scale learning, which could take up a large portion of parameters. On the other hand, VDSR and DRRN do not require this extra burden, even if multi-scale learning is performed, because they upsample the image by bicubic upsampling before processing it.

The benefit of using multi-scale learning is that it can process multiple scales using a single trained model. Only a few known architectures, such as VDSR, DRRN, and MDSR have this characteristic. This helps us alleviate the burden of heavy-weight model size when deploying the SR application on mobile devices; CARN(-M) only needs a single fixed model for multiple scales, whereas even the state-of-the-art algorithms require to train separate models for each supported scale. This property is well-suited for real-world products because the size of the applications has to be fixed while the scale of given LR images could vary.

In Fig. 5, we visually illustrate the qualitative comparisons over three datasets (Set14[yang2010], B100[b100] and Urban100[urban100]) for 4× scale. It can be seen that our model works better than others and accurately reconstructs not only stripes and line patterns, but also complex objects such as hand and street lamps.

### 4.4 模型分析 Model Analysis

To further investigate the performance behavior of the proposed methods, we analyze our models via ablation study. First, we show how local and global cascading modules affect the performance of CARN. Next, we analyze the trade-off between performance vs. parameters and operations. We train all cases for 3×105 steps with the same set of hyper-parameters for fair comparison.

Table 2 presents the ablation study on the effect of local and global cascading modules. In this table, the baseline is ResNet[resnet2016], CARN-NL is CARN without local cascading and CARN-NG is CARN without global cascading. The network topologies are all same, but because of the 1×1 convolution layer, the overall number of parameters is increased by up to 10%.

We see that the model with only global cascading (CARN-NL) shows better performance than the baseline because the global cascading mechanism effectively carries mid- to high-level frequency signals from shallow to deep layers. Furthermore, by gathering all features before the upsampling layers, the model can better leverage multi-level representations. By incorporating multi-level representations, the CARN model can consider a variety of information from many different receptive fields when reconstructing the image.

Somewhat surprisingly, using only local cascading blocks (CARN-NG) harms the performance. As discussed in He et al.[he2016identity], multiplicative manipulations such as 1×1 convolution on the shortcut connection can hamper information propagation, and thus lead to complications during optimization. Similarly, cascading connections in the local cascading blocks of CARN-NG behave as shortcut connections inside the residual blocks. Because these connections consist of concatenation and 1×1 convolutions, it is natural to expect performance degradation, as mentioned by He et al.[he2016identity]. That is, the advantage of multi-level representation is limited to the inside of each local cascading block. Therefore, there appears to be no benefit of using the cascading connection because of the increased number of multiplication operations in the cascading connection.

However, CARN uses both local and global cascading levels and outperforms all three models. This is because the global cascading mechanism eases the information propagation issues that CARN-NG suffers from. In detail, information propagates globally via global cascading, and information flows in the local cascading blocks are fused with the ones that come through global connections. By doing so, information is transmitted by multiple shortcuts and thus mitigates the vanishing gradient problem. In other words, the advantage of multi-level representation is leveraged by the global cascading connections, which help the information to propagate to higher layers.

Fig. 6 depicts the trade-off study of PSNR vs. parameters, and PSNR vs. operations (Mult-Adds) in relation to the efficient residual block and recursive network. In this experiment, we evaluate all possible group sizes of the group convolution in the efficient residual block for both the recursive and non-recursive cases. In both graphs, the blue line represents the model that does not use the recursive scheme and the orange line is the model that uses recursive cascading block.

Although all efficient models perform worse than the CARN model that shows 28.70 PSNR, the number of parameters and operations are decreased dramatically. For example, the G64 case shows a five-times reduction in both parameters and operations. However, unlike the comparable performance that is shown in Howard et al.[mobilenets], the degradation of performance is more pronounced in our experiment. There are two reasons for this: 1) We stack two consecutive group convolutions before the pointwise convolution in the efficient residual block. 2) Replacing the 3×3 convolution with a 1×1 in the upsampling layer harms the performance as well. However, despite the weakness, it is crucial to use the above two ideas to create fast, small, and accurate super-resolution models.

Next, we observe the case which uses the recursive scheme. As illustrated in Fig. 5(b), there is no change in Mult-Adds but the performance worsens, which seems reasonable given the decreased number of parameters in the recursive scheme. On the other hand, Fig. 5(a) shows that using the recursive scheme makes the model achieve better performance with less parameters. Based on these observations, we decide to choose the group size as four in the efficient residual block and use the recursive network scheme as our CARN-M model. By doing so, CARN-M reduces the parameters by five times and the operations by nearly four times with loss of 0.29 PSNR compared to CARN.

CARN(28.70 PSNR)比所有模型的表现都要好，但是参数越少，性能越差。，与 Howard 等人[16]中显示结果不同，我们的性能的下降更加明显。

### 5. 结论　Conclusion

In this work, we proposed a novel cascading network architecture that can perform SISR accurately and efficiently. The main idea behind our architecture is to add multiple cascading connections starting from each intermediary layer to the others. Such connections are made on both the local (block-wise) and global (layer-wise) levels, which allows for efficient flow of information and gradient. Our experiments show that employing both types of connections greatly outperforms those using only one or none at all.

We wish to further develop this work by improving and applying our technique on video data. Many streaming services require large storage to provide high-quality videos. In conjunction with our approach, one may devise a service that stores low-quality videos that go through our SR system to produce high-quality videos on-the-fly.

### 鸣谢

Namhyuk Ahn, Byungkon Kang, and Kyung-Ah Sohn
ECCV 2018
https://arxiv.org/pdf/1803.08664.pdf