Camouflaged Object Detection and Tracking: A Survey

 伪装对象检测和跟踪综述

  Abstract

Moving object detection and tracking have various applications, including surveillance, anomaly detection, vehicle navigation, etc. The literature on object detection and tracking is rich enough, and several essential survey papers exist. However, the research on camouflage object detection and tracking is limited due to its complexity. Existing work on this problem has been done based on either biological characteristics of the camouflaged objects or computer vision techniques. This article reviews the existing camouflaged object detection and tracking techniques using computer vision algorithms from the theoretical perspective. This article also addresses several issues of interest as well as future research direction in this area. We hope this review will help the reader learn the recent advances in camouflaged object detection and tracking. \keywords{Camouflaged object, detection, and tracking.

 摘要

移动对象检测和跟踪具有各种应用，包括监视，异常检测，车辆导航等。物体检测和跟踪的文献足够丰富，存在几个基本调查论文。然而，由于其复杂性，伪装对象检测和跟踪的研究受到限制。对该问题的现有工作是根据伪装物体或计算机视觉技术的任何生物学特性完成的。本文通过理论透视提供了使用计算机视觉算法的现有伪装对象检测和跟踪技术。本文还涉及该地区的若干利益问题以及未来的研究方向。我们希望本综述有助于读者了解伪装对象检测和跟踪最近的进步。 \关键字{伪装对象，检测和跟踪。

  Introduction

Object detection is a technique which deals with detecting instances of semantic objects of a specific class (such as human, buildings, cars, etc.) in digital images and videos. It has several computer vision applications, including image retrieval, video surveillance, and some other image and video analysis tasks. The considerable success is achieved for object detection problems in a controlled environment, but the issue remains unsolved in wild places.

Visual object detection and tracking become a very challenging problem due to several factors like (i) low-quality camera sensors (including low resolution, low bit depth, low frame rate and color distortion), (ii) challenging factors (like tracking non-rigid object, tracking small object, tracking multiple objects and tracking pose varying object), (iii) requirements for real-time tracking, (iv) multi-view object tracking and (v) variations in object appearance due to several complicated factors (such as illumination variation (Figurefigure_chellenges(a)), background clutter (Figurefigure_chellenges(b)), partial object occlusion (Figurefigure_chellenges(c)), full object occlusion (Figurefigure_chellenges(d)), large variation in object scale and orientation (Figuresfigure_chellenges(e) and (f)), partially camouflaged objects (Figurefigure_chellenges(g)), pose variation (Figurefigure_chellenges(h)), shape deformation (Figurefigure_chellenges(i)), rapid camera motion and noise. Detection and (or) tracking accuracy may be degraded and even failed due to these challenges. Numerous object tracking algorithms have been developed in the literature to handle these challenges. These invented algorithms with different properties and characteristics usually solve different visual object detection and tracking problems. Sometimes, objects hide their signatures into their surroundings and create camouflage. The occurrence of camouflage makes object detection a more complex problem. According to Copeland and Trivedi, camouflage is an attempt to conceal the signature of a target into the background". In other words, camouflage is the ability of prey to hide from predators by changing their body pattern, texture, and coloration as per the environment's texture. A camouflaged object cannot be adequately visible by human vision systems. In this context, computer vision-based approaches are proposed to analyze the camouflaged objects. Work-related to camouflage can be roughly divided into two major areas: (i) camouflage assessment and design, (ii) camouflage detection, or breakin. A camouflage detection system or de-camouflaging is used to extract a target from its background.

 简介

对象检测是一种处理数字图像和视频中特定类（如人，建筑物，汽车等）的语义对象的实例。它有几个计算机视觉应用程序，包括图像检索，视频监控和一些其他图像和视频分析任务。在受控环境中对目标检测问题实现了相当大的成功，但问题仍未解决在野外。

由于以下几个因素，视觉对象的检测和跟踪成为一个非常具有挑战性的问题：
（ⅰ ）低质量的相机传感器（包括低分辨率，低位深度，低帧速率和彩色失真），
（ii）具有挑战性的因素（如跟踪非刚性物体，跟踪小对象，跟踪多个对象和跟踪姿势变化对象），
（iii）实时跟踪的要求，
（iv）多视图对象跟踪
（v）对象外观的变化导致的几个复杂因素（例如照明变化（图线索_chellenges（a）），背景杂乱（图浮雕_chellenges（b） ），部分对象遮挡（图线索_Chellenges（c）），全对象遮挡（图线索_Chellenges（d）），对象比例的大变化和方向（图Figure_chellenges（e）和（f）），parti盟友伪装的物体（图线索_Chellenges（g）），姿势变化（图线索_chellenges（h）），形状变形（图线索（i）），快速相机运动和噪音。

由于这些挑战，检测和（或）跟踪精度可能会降低，甚至失败。文献中已经开发了许多对象跟踪算法以处理这些挑战。这些具有不同性质和特性的发明算法通常可以解决不同的视觉对象检测和跟踪问题。有时，物体将其签名隐藏到周围环境并创建伪装。伪装的发生使对象检测成为更复杂的问题。根据Copeland和Trivedi copeland1997models 的说法，“伪装是试图将目标的信息隐藏到背景中”。换句话说，伪装就像是猎物，通过根据环境改变他们的身体模式，纹理和着色，来躲避捕食者的能力。被伪装的物体不能被人类视觉系统充分看到。在这种情况下，我们提出了基于计算机视觉的方法来分析伪装的物体。与伪装相关的工作可以大致分为两个主要领域：（i）伪装评估和设计，（ii）伪装检测，或打破。伪装检测系统或去伪装主要用于从其背景中提取目标。

It discriminates foreground object from camouflaged image frames. Camouflage detection or breaking system has many potential applications, including (i) preserving wildlife, (ii) enemy detection in the battlefield, (iii) defect detection during manufacturing, (iv) identification of duplicate products during logistics, etc. Some animals have unique biological characteristics to make them camouflaged in the environment. More research work has been done for camouflage breaking based on the biological property of these animals. The vision features of a camouflaged object are very similar to the background. The color of a camouflaged object is the same as the surrounding environment, and the texture is destroyed to merge with the background. Such characteristics of camouflaged objects make detection and tracking tasks more difficult. Due to the complexity of the problem, less work has been done using computer vision-based techniques. In this article, we review existing computer vision-based approaches for detection and tracking of a camouflaged object. We also discuss the merits and demerits of each of the algorithms. We also discuss various issues of the existing algorithms and future direction on this particular topic. We hope this review will help the reader learn the recent advances in camouflaged object detection and tracking. The rest of the article organizes as follows. Sectiondetection_camouflaged_object discuss different existing detection and tracking algorithms for camouflaged object. Conclusive remark and future direction are presented in Sectionconclusions.

它识别从伪装的图像帧中的前景对象。伪装检测或破碎系统具有许多潜在的应用，包括（i）保存野生动物，（ii）在战场中的敌人检测，（iii）在制造期间的缺陷检测，（iv）在物流期间识别重复产品。一些动物有独特生物学特性使它们在环境中伪装。基于这些动物的生物学性质，已经为伪装破碎进行了更多的研究工作。伪装对象的视觉功能与背景非常相似。伪装对象的颜色与周围环境相同，纹理被破坏以与背景合并。伪装对象的这种特性使得检测和跟踪任务更加困难。由于问题的复杂性，使用基于计算机视觉的技术完成了更少的工作。在本文中，我们审查了用于检测和跟踪伪装对象的现有计算机视觉的方法。我们还讨论了每个算法的优点和缺点。我们还讨论了对该特定主题的现有算法和未来方向的各种问题。我们希望本综述有助于读者了解伪装对象检测和跟踪最近的进步。物品的其余部分组织如下。 SectionDetection_CamouFlaged_Object讨论伪装对象的不同现有检测和跟踪算法。结论性的言论和未来方向是在剖腹产中呈现的。

  Camouflaged Objects Detection and Tracking

Visual features of a camouflaged object are very much similar to the background --- (i) the intensity or color of the camouflaged object is close to their surrounding environment, (ii) the texture is destroyed to merge with the background, and (iii) the boundary of the camouflaged object is blurred. Such visual characteristics of camouflaged objects make detection and tracking tasks more difficult. Due to such complexity, less work has been done to attempt visual camouflage breaking in literature. However, the researchers have developed various algorithms using various visual features (e.g., intensity or color, texture, motion, gradient, etc.) to detect camouflaged objects from their surroundings. Here, we try to group the existing methods according to visual feature considering for detection and tracking of camouflaged objects. In the following subsection, the current techniques on consideration of each of the visual features are discussed.

 伪装对象检测和跟踪

伪装对象的视觉特征非常类似于背景---（i）伪装对象的强度或颜色靠近其周围环境，（ii）纹理被破坏与背景合并，（iii）模糊伪装对象的边界。伪装对象的这种视觉特征使得检测和跟踪任务更加困难。由于这种复杂性，已经完成了较少的工作来尝试在文献中突破视觉伪装。然而，研究人员使用各种视觉特征（例如，强度或颜色，纹理，运动，梯度等）开发了各种算法，以从周围环境中检测伪装的物体。在这里，我们尝试根据考虑检测和跟踪伪装对象的视觉特征来分组现有方法。在以下小节中，讨论了正在考虑每个可视特征的当前技术。

  Intensity/Color Features

The feature plays an essential role in the detection of camouflaged objects. Here, techniques developed based on the intensity/color values of the frames are mainly discussed. Boult et al. developed a background subtraction technique with two thresholds to detect the camouflaged target. Here, a higher threshold value is used to detect pixels that are certainly in the foreground. The lower one is considered to detect uncertain pixels (i.e., pixels that are either part of the background or camouflaged part of the object). Then, the quasi connected component is taken into consideration to get the camouflaged target. In this case, detection accuracy is also highly dependent on thresholds. The selection of proper threshold value itself is a problem. For slow-moving objects, this method fails to detect objects. On the other hand, Hung and Jiang devised a method to track a camouflaged object using sequential execution of weighted region consolidation and active contour. An iterative weighted region consolidation operator is used to fill the gaps introduced by camouflage. Then, an active contour model is built during tracking to capture the actual shape of the target. The performance of this method relies on the inter-frame difference. If the object has slow motion, it is challenging to localize the object using an iterative weighted region consolidation operator. Hence, tracking may fail for sequences containing slow-moving and uniform colored objects. In Boot discussed that we usually learn something general about target recognition, which allow us to guide our eyes to the target more efficiently and recognize it faster and farther from fixation. They described that the background contains regular patterns. Deviations from this regularity signify the presence of a camouflaged target. However, this does not happen for all kinds of camouflaged objects. If the background and object contain similar regular patterns, it becomes challenging to extract the object from its surrounding. To detect a camouflaged object, Chandesa et al. proposed an algorithm based on particle filter. Here, the Gaussian mixture model of particle distribution is considered to investigate camouflage's effect on the particle set representing the object. This method works well on the occluded object but not for the camouflaged object. Though this method works well, it needs object information (a priori) to execute the algorithm. In, Conte et al. proposed an algorithm to detect partially camouflaged people. Here, background subtraction is used to detect different parts of a person. Then grouping is performed based on a model of the shape of targets. This method is unable to provide satisfactory results for objects other than humans. In, a camouflaged model is proposed using a global model for the background and integration of global and local models for the foreground. Here, both the models helped to detect camouflaged objects. In general, intensity or color features are elementary and computationally efficient for fast, camouflaged object detection and tracking. The intensity or color feature can detect camouflaged objects where camouflage is occurred due to texture similarity with the background. In contrast, these features cannot detect camouflaged objects where camouflage occurs due to color similarity with the environment.

 强度/颜色特征

该功能在检测伪装对象中起重要作用。这里，主要讨论了基于帧的强度/颜色值开发的技术。 Boult *等人。*开发了一个有两个阈值的背景减法技术来检测伪装目标。这里，使用更高的阈值来检测肯定在前景中的像素。较低的被认为是检测不确定像素（即，作为对象的背景的一部分或伪装部分的像素）。然后，考虑到准连接的组件以获得伪装目标。在这种情况下，检测精度也高度依赖于阈值。选择适当的阈值本身是一个问题。对于慢动移动对象，此方法无法检测对象。另一方面，鸿和江设计了一种使用加权区域整合和活动轮廓的顺序执行来跟踪伪装对象的方法。迭代加权区域整合操作员用于填补伪装引入的间隙。然后，在跟踪期间构建有效轮廓模型以捕获目标的实际形状。该方法的性能依赖于帧间差异。如果对象具有慢动作，则使用迭代加权区域整合运算符本地化对象充满挑战。因此，跟踪可能导致包含缓慢和均匀的彩色物体的序列。在靴子中讨论了我们通常会学习关于目标识别的一般事物，这使我们能够更有效地引导我们的目光，并识别更快，远离固定。他们描述了背景包含常规模式。偏离这种规律性表示伪装目标的存在。但是，这不会发生各种伪装的物体。如果背景和对象包含类似的规则模式，则从其周围提取对象变得具有挑战性。要检测伪装对象，Chandesa *等。*提出了一种基于粒子滤波器的算法。这里，认为颗粒分布的高斯混合模型被认为调查伪装对代表物体的颗粒集的影响。此方法适用于遮挡对象，但不适用于伪装对象。虽然此方法运行良好，但它需要执行算法的对象信息（先验）。在，Conte *等。*提出了一种算法来检测部分伪装的人。这里，背景减法用于检测人的不同部分。然后基于目标形状的模型执行分组。这种方法无法为人类以外的物体提供令人满意的结果。在，使用全局模型来提出伪装模型，用于前景的全局和本地模型的背景和集成。在这里，两个模型都有助于检测伪装对象。通常，强度或颜色特征是基本的和计算上的快速，伪装对象检测和跟踪的计算。强度或颜色特征可以检测由于与背景的纹理相似性而发生伪装的伪装物体。相比之下，这些特征不能检测由于与环境颜色相似性而发生伪装的伪装物体。

  Motion Features

Motion is considered as an essential feature to detect an object. Several techniques have been developed based on the motion of the objects. McKee et al. concluded from their experiments that stereopsis is generally useful on breaking camouflage when both the observer and the scene are non-dynamic. Here, motion is a helpful feature for breaking camouflage on a static background. If the background is non-static, the motion feature fails to extract camouflaged objects from its surrounding. In this direction, Ternovskiy and Jannson have proposed a motion prediction approach to detect the target in the camouflage environment. This method is suitable for the sequences where changes occur due to the object and camera movement only. If changes occur due to illumination variation, this method considers changes occurring due to object movement. Hence, this method can not work correctly in such a situation. On the other hand, for breaking camouflage, Huimin et al. developed a computational model of visual moving image filtering in which Reichardt's elementary motion detectors are employed for detecting motion information. As this method relies on motion (due to object movement) information, motion due to other conditions like illumination variation, environmental condition changes, etc. produces more false alarms. All these techniques mentioned above are context-dependent and may not work well for various types of camouflaged objects. In, Yin et al. developed an algorithm to track a mobile object with a camouflage color based on the optical flow model. Here, the optical flow model is used to detect motion patterns of the object and the background. The motion patterns are clustered and detect the camouflaged object based on the optical flow's magnitude and location. After that, the Kalman filter is used to improve the detection accuracy. However, the accuracy of this model depends on the results of the optical flow. For slow-moving objects and objects with camera motion, this method fails to provide excellent results. motion plays a vital role in detecting camouflaged objects in the literature. Motion features help to detect camouflaged objects while camouflage occurs due to color/texture similarity with the background. However, motion features also fail to detect a camouflaged object with prolonged movement or stop/go motion.

 运动功能

运动被认为是检测对象的基本要素。已经基于对象的运动开发了几种技术。麦基*等。*从他们的实验中得出结论，即立体镜在观察者和场景都是非动态的情况下，立体镜通常有用。在这里，运动是在静态背景上破坏伪装的有用特征。如果背景是非静态的，则运动功能无法从其周围提取伪装对象。在这个方向上，Ternovskiy和Jannson提出了一种运动预测方法来检测伪装环境中的目标。该方法适用于仅由于对象和相机移动而发生变化的序列。如果由于照明变化而发生变化，则该方法认为由于对象移动而发生的变化。因此，这种方法无法在这种情况下正常工作。另一方面，为了破碎伪装，Huimin *等。*开发了一种可视运动图像滤波的计算模型，其中用于检测运动信息的Reichardt的基本运动检测器。由于该方法依赖于运动（由于对象运动）信息，因此由于照明变化等其他条件而导致的运动，环境条件变化等产生更多误报。上面提到的所有这些技术都是相关的上下文相关的，并且对于各种类型的伪装对象可能不起作用。在，Yin *等。*开发了一种算法，以基于光学流模型跟踪具有迷彩颜色的移动对象。这里，光学流模型用于检测物体和背景的运动模式。运动模式是基于光流量的幅度和位置检测伪装对象。之后，卡尔曼滤波器用于提高检测精度。然而，该模型的准确性取决于光流的结果。对于具有相机运动的缓慢移动对象和对象，此方法无法提供出色的结果。动作在检测文献中的伪装对象方面发挥着至关重要的作用。运动功能有助于检测伪装对象，而伪装是由于背景颜色/纹理相似性而发生的。然而，运动特征也无法检测到具有长时间运动或停止/去运动的伪装对象。

  Texture Features

Sometimes, the object's color is similar to the background, but they have different texture patterns. The texture is considered to discriminate against the object from its surrounding. Galun et al. developed a technique to detect camouflaged objects using a bottom-up aggregation framework that combines structural characteristics of texture elements with filter responses. It adaptively identifies the shape of texture elements and characterizes them by their size, aspect ratio, orientation, brightness, etc. Then various statistical measures of these properties are taken into account to distinguish between different textures. The said approach is applied to images containing various kinds of textures. This method works well for images containing different textures for objects and backgrounds. However, if the object and background contain a similar texture, this technique may fail to produce good results. In, Nagabhushan and Bhajantri proposed a technique for multiple camouflage breaking using co-occurrence matrix and Canny edge detector. The co-occurrence matrix is used to analyze the given image's texture, whereas the Canny edge detector is considered to detect the edges. A combination of both co-occurrence matrix and the Canny edge detector enhances the separability between objects containing different textures. Though this method provides good results for synthetic images, it is not applied to real-life data. Also, background information needs to be known before executing this method. Neider and Zelinsky discussed in the detection of camouflaged targets by looking through the distracters or by scrutinizing the target-similar background. In, Bhajantri and Nagabhushan proposed a technique to detect the camouflaged defect. Here, co-occurrence matrix-based texture features are computed within a small image region. The defective portion is detected by cluster analysis and watershed segmentation. The accuracy of this method depends on the texture feature. It may not work well for sequences where objects and background contain similar kind of texture.

 纹理特征

有时，对象的颜色类似于背景，但它们具有不同的纹理模式。纹理被认为是与周围的对象歧视。 Galun *等它自适应地识别纹理元素的形状，并通过其尺寸，纵横比，方向，亮度等表征它们。然后考虑这些属性的各种统计测量以区分不同的纹理。所述方法应用于含有各种纹理的图像。此方法适用于包含对象和背景的不同纹理的图像。但是，如果对象和背景包含相似的纹理，则该技术可能无法产生良好的结果。在，Nagabhushan和Bhajantri提出了一种使用共发生矩阵和罐头边缘检测器进行多次伪装的技术。共发生矩阵用于分析给定的图像的纹理，而罐头边缘检测器被认为是检测边缘。共发生矩阵和罐头边缘检测器的组合增强了包含不同纹理的物体之间的可分离性。虽然此方法为合成图像提供了良好的结果，但它不适用于现实生活数据。此外，在执行此方法之前需要知道背景信息。通过观察干扰因素或通过仔细检查目标类似的背景，在检测伪装目标时讨论了Neider和Zelinsky。在，Bhajantri和Nagabhushan提出了一种检测伪装缺陷的技术。这里，基于共同发生的基于矩阵的纹理特征在小图像区域内计算。通过聚类分析和流域分割来检测缺陷部分。此方法的准确性取决于纹理特征。对于物体和背景含有类似类型的纹理的序列，它可能无法正常工作。

Sengottuvelan et al. developed a technique to detect the camouflaged portion of the object and extract it from the environment in a given image. Here, the grey level co-occurrence matrix () based texture feature and dendrogram are used to detect the camouflaged object. This technique is very time-consuming due to the given image's division into several blocks or smaller regions. It does not work for images containing shading effects and object & background containing similar textures. Liming and Weidong proposed a technique based on weighted structural similarity () to design and evaluate camouflage texture. Here they used weighted structural similarity and original image feature to create a camouflage image. It can be used for breaking the camouflage. In, Owens introduced several background matching algorithms that attempt to make the object look like whatever is behind it. It is impossible to match the background from every possible viewpoint exactly. But the proposed models are forced to make trade-offs between different perceptual factors, such as conspicuousness of occlusion boundaries and the amount of texture distortion. In the same direction, Li proposed a texture guided weighted voting () method detect foreground object in camouflaged scenes. This method employed the stationary wavelet transform to decompose the image into frequency bands. This technique could effectively capture small and hardly noticeable differences between the foreground and background in the image domain in certain wavelet frequency bands. Finally, the foreground is detected using a weighted voting scheme based on all the wavelet bands' intensity and texture. Experimental results demonstrate that this method achieves superior performance compared to the current state-of-the-art results. Though texture feature extraction from the color or intensity is costly. However, it is more effective for detecting camouflaged objects. While camouflage occurs due to color similarity with background, texture features give promising results in such cases.

Sengottuvelan 等。开发了一种检测对象的伪装部分的技术，并从给定图像中从环境中提取它。这里，基于灰度的共发生矩阵（）基于纹理特征和树木图来检测伪装对象。由于给定的图像的划分为几个块或更小的区域，这种技术非常耗时。它不适用于包含包含类似纹理的阴影效果和对象\和背景的图像。 Limining和Weidong提出了一种基于加权结构相似性（）的技术来设计和评估伪装纹理。在这里，它们使用加权结构相似性和原始图像特征来创建迷彩图像。它可用于打破伪装。在，欧文斯介绍了几个背景匹配算法，该算法试图使物体看起来像它后面的任何东西。完全是不可能将背景从每个可能的观点匹配。但拟议的模型被迫在不同感知因素之间进行权衡，例如遮挡边界的显着性和纹理变形的量。在同一方向，李提出了一个纹理引导的加权投票（）方法检测伪装场景中的前景对象。该方法采用静止小波变换将图像分解成频带。该技术可以有效地捕获在某些小波频带中的图像域中的前景和背景之间的小且难以明显的差异。最后，使用基于所有小波频带的强度和纹理的加权投票方案来检测前景。实验结果表明，与目前最先进的结果相比，该方法实现了卓越的性能。虽然纹理特征提取颜色或强度昂贵。但是，检测伪装对象更有效。虽然由于与背景颜色相似而发生伪装，但纹理特征在这种情况下具有有希望的结果。

  Gradient Features

When the object has a similar color, texture as the background, it is challenging to detect objects using these features. For those sequences, gradient information is useful to extract the object from the background region. In this direction, various metho