[论文翻译]伪装的物体检测和跟踪综述


下载PDF:https://arxiv.org/pdf/2012.13581v1.pdf


  Camouflaged Object Detection and Tracking: A Survey

 伪装对象检测和跟踪综述

  Abstract

Moving object detection and tracking have various applications, including surveillance, anomaly detection, vehicle navigation, etc. The literature on object detection and tracking is rich enough, and several essential survey papers exist. However, the research on camouflage object detection and tracking is limited due to its complexity. Existing work on this problem has been done based on either biological characteristics of the camouflaged objects or computer vision techniques. This article reviews the existing camouflaged object detection and tracking techniques using computer vision algorithms from the theoretical perspective. This article also addresses several issues of interest as well as future research direction in this area. We hope this review will help the reader learn the recent advances in camouflaged object detection and tracking. \keywords{Camouflaged object, detection, and tracking.

 摘要

移动对象检测和跟踪具有各种应用,包括监视,异常检测,车辆导航等。物体检测和跟踪的文献足够丰富,存在几个基本调查论文。然而,由于其复杂性,伪装对象检测和跟踪的研究受到限制。对该问题的现有工作是根据伪装物体或计算机视觉技术的任何生物学特性完成的。本文通过理论透视提供了使用计算机视觉算法的现有伪装对象检测和跟踪技术。本文还涉及该地区的若干利益问题以及未来的研究方向。我们希望本综述有助于读者了解伪装对象检测和跟踪最近的进步。 \关键字{伪装对象,检测和跟踪。

  Introduction

Object detection is a technique which deals with detecting instances of semantic objects of a specific class (such as human, buildings, cars, etc.) in digital images and videos. It has several computer vision applications, including image retrieval, video surveillance, and some other image and video analysis tasks. The considerable success is achieved for object detection problems in a controlled environment, but the issue remains unsolved in wild places.

Visual object detection and tracking become a very challenging problem due to several factors like (i) low-quality camera sensors (including low resolution, low bit depth, low frame rate and color distortion), (ii) challenging factors (like tracking non-rigid object, tracking small object, tracking multiple objects and tracking pose varying object), (iii) requirements for real-time tracking, (iv) multi-view object tracking and (v) variations in object appearance due to several complicated factors (such as illumination variation (Figurefigure_chellenges(a)), background clutter (Figurefigure_chellenges(b)), partial object occlusion (Figurefigure_chellenges(c)), full object occlusion (Figurefigure_chellenges(d)), large variation in object scale and orientation (Figuresfigure_chellenges(e) and (f)), partially camouflaged objects (Figurefigure_chellenges(g)), pose variation (Figurefigure_chellenges(h)), shape deformation (Figurefigure_chellenges(i)), rapid camera motion and noise. Detection and (or) tracking accuracy may be degraded and even failed due to these challenges. Numerous object tracking algorithms have been developed in the literature to handle these challenges. These invented algorithms with different properties and characteristics usually solve different visual object detection and tracking problems. Sometimes, objects hide their signatures into their surroundings and create camouflage. The occurrence of camouflage makes object detection a more complex problem. According to Copeland and Trivedi, camouflage is an attempt to conceal the signature of a target into the background". In other words, camouflage is the ability of prey to hide from predators by changing their body pattern, texture, and coloration as per the environment's texture. A camouflaged object cannot be adequately visible by human vision systems. In this context, computer vision-based approaches are proposed to analyze the camouflaged objects. Work-related to camouflage can be roughly divided into two major areas: (i) camouflage assessment and design, (ii) camouflage detection, or breakin. A camouflage detection system or de-camouflaging is used to extract a target from its background.

 简介

对象检测是一种处理数字图像和视频中特定类(如人,建筑物,汽车等)的语义对象的实例。它有几个计算机视觉应用程序,包括图像检索,视频监控和一些其他图像和视频分析任务。在受控环境中对目标检测问题实现了相当大的成功,但问题仍未解决在野外。

由于以下几个因素,视觉对象的检测和跟踪成为一个非常具有挑战性的问题:
(ⅰ )低质量的相机传感器(包括低分辨率,低位深度,低帧速率和彩色失真),
(ii)具有挑战性的因素(如跟踪非刚性物体,跟踪小对象,跟踪多个对象和跟踪姿势变化对象),
(iii)实时跟踪的要求,
(iv)多视图对象跟踪
(v)对象外观的变化导致的几个复杂因素(例如照明变化(图线索_chellenges(a)),背景杂乱(图浮雕_chellenges(b) ),部分对象遮挡(图线索_Chellenges(c)),全对象遮挡(图线索_Chellenges(d)),对象比例的大变化和方向(图 Figure_chellenges(e)和(f)),parti 盟友伪装的物体(图线索_Chellenges(g)),姿势变化(图线索_chellenges(h)),形状变形(图线索(i)),快速相机运动和噪音。

由于这些挑战,检测和(或)跟踪精度可能会降低,甚至失败。文献中已经开发了许多对象跟踪算法以处理这些挑战。这些具有不同性质和特性的发明算法通常可以解决不同的视觉对象检测和跟踪问题。有时,物体将其签名隐藏到周围环境并创建伪装。伪装的发生使对象检测成为更复杂的问题。根据 Copeland 和 Trivedi copeland1997models 的说法,“伪装是试图将目标的信息隐藏到背景中”。换句话说,伪装就像是猎物,通过根据环境改变他们的身体模式,纹理和着色,来躲避捕食者的能力。被伪装的物体不能被人类视觉系统充分看到。在这种情况下,我们提出了基于计算机视觉的方法来分析伪装的物体。与伪装相关的工作可以大致分为两个主要领域:(i)伪装评估和设计,(ii)伪装检测,或打破。伪装检测系统或去伪装主要用于从其背景中提取目标。

It discriminates foreground object from camouflaged image frames. Camouflage detection or breaking system has many potential applications, including (i) preserving wildlife, (ii) enemy detection in the battlefield, (iii) defect detection during manufacturing, (iv) identification of duplicate products during logistics, etc. Some animals have unique biological characteristics to make them camouflaged in the environment. More research work has been done for camouflage breaking based on the biological property of these animals. The vision features of a camouflaged object are very similar to the background. The color of a camouflaged object is the same as the surrounding environment, and the texture is destroyed to merge with the background. Such characteristics of camouflaged objects make detection and tracking tasks more difficult. Due to the complexity of the problem, less work has been done using computer vision-based techniques. In this article, we review existing computer vision-based approaches for detection and tracking of a camouflaged object. We also discuss the merits and demerits of each of the algorithms. We also discuss various issues of the existing algorithms and future direction on this particular topic. We hope this review will help the reader learn the recent advances in camouflaged object detection and tracking. The rest of the article organizes as follows. Sectiondetection_camouflaged_object discuss different existing detection and tracking algorithms for camouflaged object. Conclusive remark and future direction are presented in Sectionconclusions.

它识别从伪装的图像帧中的前景对象。伪装检测或破碎系统具有许多潜在的应用,包括(i)保存野生动物,(ii)在战场中的敌人检测,(iii)在制造期间的缺陷检测,(iv)在物流期间识别重复产品。一些动物有独特生物学特性使它们在环境中伪装。基于这些动物的生物学性质,已经为伪装破碎进行了更多的研究工作。伪装对象的视觉功能与背景非常相似。伪装对象的颜色与周围环境相同,纹理被破坏以与背景合并。伪装对象的这种特性使得检测和跟踪任务更加困难。由于问题的复杂性,使用基于计算机视觉的技术完成了更少的工作。在本文中,我们审查了用于检测和跟踪伪装对象的现有计算机视觉的方法。我们还讨论了每个算法的优点和缺点。我们还讨论了对该特定主题的现有算法和未来方向的各种问题。我们希望本综述有助于读者了解伪装对象检测和跟踪最近的进步。物品的其余部分组织如下。 SectionDetection_CamouFlaged_Object 讨论伪装对象的不同现有检测和跟踪算法。结论性的言论和未来方向是在剖腹产中呈现的。

  Camouflaged Objects Detection and Tracking

Visual features of a camouflaged object are very much similar to the background --- (i) the intensity or color of the camouflaged object is close to their surrounding environment, (ii) the texture is destroyed to merge with the background, and (iii) the boundary of the camouflaged object is blurred. Such visual characteristics of camouflaged objects make detection and tracking tasks more difficult. Due to such complexity, less work has been done to attempt visual camouflage breaking in literature. However, the researchers have developed various algorithms using various visual features (e.g., intensity or color, texture, motion, gradient, etc.) to detect camouflaged objects from their surroundings. Here, we try to group the existing methods according to visual feature considering for detection and tracking of camouflaged objects. In the following subsection, the current techniques on consideration of each of the visual features are discussed.

 伪装对象检测和跟踪

伪装对象的视觉特征非常类似于背景---(i)伪装对象的强度或颜色靠近其周围环境,(ii)纹理被破坏与背景合并,(iii)模糊伪装对象的边界。伪装对象的这种视觉特征使得检测和跟踪任务更加困难。由于这种复杂性,已经完成了较少的工作来尝试在文献中突破视觉伪装。然而,研究人员使用各种视觉特征(例如,强度或颜色,纹理,运动,梯度等)开发了各种算法,以从周围环境中检测伪装的物体。在这里,我们尝试根据考虑检测和跟踪伪装对象的视觉特征来分组现有方法。在以下小节中,讨论了正在考虑每个可视特征的当前技术。

  Intensity/Color Features

The feature plays an essential role in the detection of camouflaged objects. Here, techniques developed based on the intensity/color values of the frames are mainly discussed. Boult et al. developed a background subtraction technique with two thresholds to detect the camouflaged target. Here, a higher threshold value is used to detect pixels that are certainly in the foreground. The lower one is considered to detect uncertain pixels (i.e., pixels that are either part of the background or camouflaged part of the object). Then, the quasi connected component is taken into consideration to get the camouflaged target. In this case, detection accuracy is also highly dependent on thresholds. The selection of proper threshold value itself is a problem. For slow-moving objects, this method fails to detect objects. On the other hand, Hung and Jiang devised a method to track a camouflaged object using sequential execution of weighted region consolidation and active contour. An iterative weighted region consolidation operator is used to fill the gaps introduced by camouflage. Then, an active contour model is built during tracking to capture the actual shape of the target. The performance of this method relies on the inter-frame difference. If the object has slow motion, it is challenging to localize the object using an iterative weighted region consolidation operator. Hence, tracking may fail for sequences containing slow-moving and uniform colored objects. In Boot discussed that we usually learn something general about target recognition, which allow us to guide our eyes to the target more efficiently and recognize it faster and farther from fixation. They described that the background contains regular patterns. Deviations from this regularity signify the presence of a camouflaged target. However, this does not happen for all kinds of camouflaged objects. If the background and object contain similar regular patterns, it becomes challenging to extract the object from its surrounding. To detect a camouflaged object, Chandesa et al. proposed an algorithm based on particle filter. Here, the Gaussian mixture model of particle distribution is considered to investigate camouflage's effect on the particle set representing the object. This method works well on the occluded object but not for the camouflaged object. Though this method works well, it needs object information (a priori) to execute the algorithm. In, Conte et al. proposed an algorithm to detect partially camouflaged people. Here, background subtraction is used to detect different parts of a person. Then grouping is performed based on a model of the shape of targets. This method is unable to provide satisfactory results for objects other than humans. In, a camouflaged model is proposed using a global model for the background and integration of global and local models for the foreground. Here, both the models helped to detect camouflaged objects. In general, intensity or color features are elementary and computationally efficient for fast, camouflaged object detection and tracking. The intensity or color feature can detect camouflaged objects where camouflage is occurred due to texture similarity with the background. In contrast, these features cannot detect camouflaged objects where camouflage occurs due to color similarity with the environment.

 强度/颜色特征

该功能在检测伪装对象中起重要作用。这里,主要讨论了基于帧的强度/颜色值开发的技术。 Boult *等人。*开发了一个有两个阈值的背景减法技术来检测伪装目标。这里,使用更高的阈值来检测肯定在前景中的像素。较低的被认为是检测不确定像素(即,作为对象的背景的一部分或伪装部分的像素)。然后,考虑到准连接的组件以获得伪装目标。在这种情况下,检测精度也高度依赖于阈值。选择适当的阈值本身是一个问题。对于慢动移动对象,此方法无法检测对象。另一方面,鸿和江设计了一种使用加权区域整合和活动轮廓的顺序执行来跟踪伪装对象的方法。迭代加权区域整合操作员用于填补伪装引入的间隙。然后,在跟踪期间构建有效轮廓模型以捕获目标的实际形状。该方法的性能依赖于帧间差异。如果对象具有慢动作,则使用迭代加权区域整合运算符本地化对象充满挑战。因此,跟踪可能导致包含缓慢和均匀的彩色物体的序列。在靴子中讨论了我们通常会学习关于目标识别的一般事物,这使我们能够更有效地引导我们的目光,并识别更快,远离固定。他们描述了背景包含常规模式。偏离这种规律性表示伪装目标的存在。但是,这不会发生各种伪装的物体。如果背景和对象包含类似的规则模式,则从其周围提取对象变得具有挑战性。要检测伪装对象,Chandesa *等。*提出了一种基于粒子滤波器的算法。这里,认为颗粒分布的高斯混合模型被认为调查伪装对代表物体的颗粒集的影响。此方法适用于遮挡对象,但不适用于伪装对象。虽然此方法运行良好,但它需要执行算法的对象信息(先验)。在,Conte *等。*提出了一种算法来检测部分伪装的人。这里,背景减法用于检测人的不同部分。然后基于目标形状的模型执行分组。这种方法无法为人类以外的物体提供令人满意的结果。在,使用全局模型来提出伪装模型,用于前景的全局和本地模型的背景和集成。在这里,两个模型都有助于检测伪装对象。通常,强度或颜色特征是基本的和计算上的快速,伪装对象检测和跟踪的计算。强度或颜色特征可以检测由于与背景的纹理相似性而发生伪装的伪装物体。相比之下,这些特征不能检测由于与环境颜色相似性而发生伪装的伪装物体。

  Motion Features

Motion is considered as an essential feature to detect an object. Several techniques have been developed based on the motion of the objects. McKee et al. concluded from their experiments that stereopsis is generally useful on breaking camouflage when both the observer and the scene are non-dynamic. Here, motion is a helpful feature for breaking camouflage on a static background. If the background is non-static, the motion feature fails to extract camouflaged objects from its surrounding. In this direction, Ternovskiy and Jannson have proposed a motion prediction approach to detect the target in the camouflage environment. This method is suitable for the sequences where changes occur due to the object and camera movement only. If changes occur due to illumination variation, this method considers changes occurring due to object movement. Hence, this method can not work correctly in such a situation. On the other hand, for breaking camouflage, Huimin et al. developed a computational model of visual moving image filtering in which Reichardt's elementary motion detectors are employed for detecting motion information. As this method relies on motion (due to object movement) information, motion due to other conditions like illumination variation, environmental condition changes, etc. produces more false alarms. All these techniques mentioned above are context-dependent and may not work well for various types of camouflaged objects. In, Yin et al. developed an algorithm to track a mobile object with a camouflage color based on the optical flow model. Here, the optical flow model is used to detect motion patterns of the object and the background. The motion patterns are clustered and detect the camouflaged object based on the optical flow's magnitude and location. After that, the Kalman filter is used to improve the detection accuracy. However, the accuracy of this model depends on the results of the optical flow. For slow-moving objects and objects with camera motion, this method fails to provide excellent results. motion plays a vital role in detecting camouflaged objects in the literature. Motion features help to detect camouflaged objects while camouflage occurs due to color/texture similarity with the background. However, motion features also fail to detect a camouflaged object with prolonged movement or stop/go motion.

 运动功能

运动被认为是检测对象的基本要素。已经基于对象的运动开发了几种技术。麦基*等。*从他们的实验中得出结论,即立体镜在观察者和场景都是非动态的情况下,立体镜通常有用。在这里,运动是在静态背景上破坏伪装的有用特征。如果背景是非静态的,则运动功能无法从其周围提取伪装对象。在这个方向上,Ternovskiy 和 Jannson 提出了一种运动预测方法来检测伪装环境中的目标。该方法适用于仅由于对象和相机移动而发生变化的序列。如果由于照明变化而发生变化,则该方法认为由于对象移动而发生的变化。因此,这种方法无法在这种情况下正常工作。另一方面,为了破碎伪装,Huimin *等。*开发了一种可视运动图像滤波的计算模型,其中用于检测运动信息的 Reichardt 的基本运动检测器。由于该方法依赖于运动(由于对象运动)信息,因此由于照明变化等其他条件而导致的运动,环境条件变化等产生更多误报。上面提到的所有这些技术都是相关的上下文相关的,并且对于各种类型的伪装对象可能不起作用。在,Yin *等。*开发了一种算法,以基于光学流模型跟踪具有迷彩颜色的移动对象。这里,光学流模型用于检测物体和背景的运动模式。运动模式是基于光流量的幅度和位置检测伪装对象。之后,卡尔曼滤波器用于提高检测精度。然而,该模型的准确性取决于光流的结果。对于具有相机运动的缓慢移动对象和对象,此方法无法提供出色的结果。动作在检测文献中的伪装对象方面发挥着至关重要的作用。运动功能有助于检测伪装对象,而伪装是由于背景颜色/纹理相似性而发生的。然而,运动特征也无法检测到具有长时间运动或停止/去运动的伪装对象。

  Texture Features

Sometimes, the object's color is similar to the background, but they have different texture patterns. The texture is considered to discriminate against the object from its surrounding. Galun et al. developed a technique to detect camouflaged objects using a bottom-up aggregation framework that combines structural characteristics of texture elements with filter responses. It adaptively identifies the shape of texture elements and characterizes them by their size, aspect ratio, orientation, brightness, etc. Then various statistical measures of these properties are taken into account to distinguish between different textures. The said approach is applied to images containing various kinds of textures. This method works well for images containing different textures for objects and backgrounds. However, if the object and background contain a similar texture, this technique may fail to produce good results. In, Nagabhushan and Bhajantri proposed a technique for multiple camouflage breaking using co-occurrence matrix and Canny edge detector. The co-occurrence matrix is used to analyze the given image's texture, whereas the Canny edge detector is considered to detect the edges. A combination of both co-occurrence matrix and the Canny edge detector enhances the separability between objects containing different textures. Though this method provides good results for synthetic images, it is not applied to real-life data. Also, background information needs to be known before executing this method. Neider and Zelinsky discussed in the detection of camouflaged targets by looking through the distracters or by scrutinizing the target-similar background. In, Bhajantri and Nagabhushan proposed a technique to detect the camouflaged defect. Here, co-occurrence matrix-based texture features are computed within a small image region. The defective portion is detected by cluster analysis and watershed segmentation. The accuracy of this method depends on the texture feature. It may not work well for sequences where objects and background contain similar kind of texture.

 纹理特征

有时,对象的颜色类似于背景,但它们具有不同的纹理模式。纹理被认为是与周围的对象歧视。 Galun *等它自适应地识别纹理元素的形状,并通过其尺寸,纵横比,方向,亮度等表征它们。然后考虑这些属性的各种统计测量以区分不同的纹理。所述方法应用于含有各种纹理的图像。此方法适用于包含对象和背景的不同纹理的图像。但是,如果对象和背景包含相似的纹理,则该技术可能无法产生良好的结果。在,Nagabhushan 和 Bhajantri 提出了一种使用共发生矩阵和罐头边缘检测器进行多次伪装的技术。共发生矩阵用于分析给定的图像的纹理,而罐头边缘检测器被认为是检测边缘。共发生矩阵和罐头边缘检测器的组合增强了包含不同纹理的物体之间的可分离性。虽然此方法为合成图像提供了良好的结果,但它不适用于现实生活数据。此外,在执行此方法之前需要知道背景信息。通过观察干扰因素或通过仔细检查目标类似的背景,在检测伪装目标时讨论了 Neider 和 Zelinsky。在,Bhajantri 和 Nagabhushan 提出了一种检测伪装缺陷的技术。这里,基于共同发生的基于矩阵的纹理特征在小图像区域内计算。通过聚类分析和流域分割来检测缺陷部分。此方法的准确性取决于纹理特征。对于物体和背景含有类似类型的纹理的序列,它可能无法正常工作。

Sengottuvelan et al. developed a technique to detect the camouflaged portion of the object and extract it from the environment in a given image. Here, the grey level co-occurrence matrix () based texture feature and dendrogram are used to detect the camouflaged object. This technique is very time-consuming due to the given image's division into several blocks or smaller regions. It does not work for images containing shading effects and object & background containing similar textures. Liming and Weidong proposed a technique based on weighted structural similarity () to design and evaluate camouflage texture. Here they used weighted structural similarity and original image feature to create a camouflage image. It can be used for breaking the camouflage. In, Owens introduced several background matching algorithms that attempt to make the object look like whatever is behind it. It is impossible to match the background from every possible viewpoint exactly. But the proposed models are forced to make trade-offs between different perceptual factors, such as conspicuousness of occlusion boundaries and the amount of texture distortion. In the same direction, Li proposed a texture guided weighted voting () method detect foreground object in camouflaged scenes. This method employed the stationary wavelet transform to decompose the image into frequency bands. This technique could effectively capture small and hardly noticeable differences between the foreground and background in the image domain in certain wavelet frequency bands. Finally, the foreground is detected using a weighted voting scheme based on all the wavelet bands' intensity and texture. Experimental results demonstrate that this method achieves superior performance compared to the current state-of-the-art results. Though texture feature extraction from the color or intensity is costly. However, it is more effective for detecting camouflaged objects. While camouflage occurs due to color similarity with background, texture features give promising results in such cases.

Sengottuvelan 等。开发了一种检测对象的伪装部分的技术,并从给定图像中从环境中提取它。这里,基于灰度的共发生矩阵()基于纹理特征和树木图来检测伪装对象。由于给定的图像的划分为几个块或更小的区域,这种技术非常耗时。它不适用于包含包含类似纹理的阴影效果和对象\和背景的图像。 Limining 和 Weidong 提出了一种基于加权结构相似性()的技术来设计和评估伪装纹理。在这里,它们使用加权结构相似性和原始图像特征来创建迷彩图像。它可用于打破伪装。在,欧文斯介绍了几个背景匹配算法,该算法试图使物体看起来像它后面的任何东西。完全是不可能将背景从每个可能的观点匹配。但拟议的模型被迫在不同感知因素之间进行权衡,例如遮挡边界的显着性和纹理变形的量。在同一方向,李提出了一个纹理引导的加权投票()方法检测伪装场景中的前景对象。该方法采用静止小波变换将图像分解成频带。该技术可以有效地捕获在某些小波频带中的图像域中的前景和背景之间的小且难以明显的差异。最后,使用基于所有小波频带的强度和纹理的加权投票方案来检测前景。实验结果表明,与目前最先进的结果相比,该方法实现了卓越的性能。虽然纹理特征提取颜色或强度昂贵。但是,检测伪装对象更有效。虽然由于与背景颜色相似而发生伪装,但纹理特征在这种情况下具有有希望的结果。

  Gradient Features

When the object has a similar color, texture as the background, it is challenging to detect objects using these features. For those sequences, gradient information is useful to extract the object from the background region. In this direction, various methods have been developed to detect the camouflaged object. In, Tankus and Yeshurum proposed a $ D_{arg} $ operator (context-free) to enhance an area whose shading corresponds to a convex (or concave) $ 3D $ object to separate such area from a flat background having similar features (like color and texture). $ D_{arg} $ is applied directly to the grey-level function of the image. It responds to smooth three-dimensional convex or concave patches in objects and is not limited by any particular light source or reflectance function. Results obtained using the $ D_{arg} $ operator are highly dependent on threshold values. The selection of a suitable threshold is a major issue. Further, this method does not work well for an environment containing concave background and dark-colored objects. Gradient features are useful in case of camouflage occur due to intensity/color and texture similarity with the background.

 渐变功能

当对象具有类似的颜色时,纹理作为背景,使用这些功能检测对象是具有挑战性的。对于那些序列,梯度信息可用于从背景区域中提取对象。在这种方向上,已经开发了各种方法来检测伪装的物体。在,坦克和 yeshurum 提出了一个$D_{arg} $运算符(无背景),以增强阴影对应于凸(或凹)$3D $对象的区域,以将这些区域与具有类似特征的平坦背景(如颜色和颜色)分开质地)。 $D_{arg} $直接应用于图像的灰度级功能。它响应对象中的平滑三维凸面或凹形贴片,并且不受任何特定光源或反射功能的限制。使用$D_{arg} $操作员获得的结果高度依赖于阈值。选择合适的阈值是一个主要问题。此外,该方法对包含凹形背景和深色对象的环境不起作用。梯度特征对于伪装而有用,因为由于与背景的强度/颜色和纹理相似性而发生。

  Combination of Various Features

Camouflage occurs when the visual characteristics of the objects are too similar to the background. The objective of a camouflage detection system is to separate the camouflaged object from the background. In such cases, features generated from single-cue (like color or texture or shape or motion) is not sufficient to extract camouflaged objects from the background because the underlying phenomenon for the occurrence of camouflage is not known. In this context, the integration of features generated from multi-cue (color, texture, shape, motion, etc.) may increase the separability between the camouflaged objects and background. Various algorithms have been proposed based on integrating different features like color, texture, motion, shape, etc. In this direction, to detect camouflaged objects, Harville et al. proposed a foreground segmentation technique using both color and depth information. Information loss occurs due to a $ 3 $ D scene's projection into a $ 2 $ D picture. Here, depth information is considered to reduce the information loss due to $ 3 $ D to $ 2 $ D projection. The use of depth information increases detection accuracy and also increases the computational cost. In, Pong and Bowdenb proposed the use of the stochastic process to handle camouflage. This algorithm assumes that camouflage occurs when the new observation can not be associated with an existing task. The performance of this method depends on the color and motion of the object. Here, color is used to distinguish the camouflaged object and its surrounding, whereas motion is considered to separate out moving objects and the static part of the frame. Hence, both this information helps to detect moving camouflaged objects. Sometimes, this method may fail for an object, which is very similar to the background, and it contains more static objects. In, Brady and Kersten proposed a computational approach to combine low-level features with high-level models to detect and recognize an object when camouflage is present. Here, bootstrapped learning is considered to build the shape of the object, and it depends on the result of object detection. If object detection is not perfect, then recognition may not always be correct, and it fails to track correctly. Furthermore, in, Losa proposed a particle filter based on structural similarity measure for tracking camouflaged objects. Here, the structural similarity measure reflects the distance between two frames by comparing their luminance, contrast, and spatial characteristics.

 各种特征的组合

伪装时发生在对象的视觉特性与背景中的视觉特性发生时。伪装检测系统的目的是将伪装对象与背景分开。在这种情况下,从单色(如颜色或纹理或形状或运动)产生的特征是不足以从背景中提取伪装的物体,因为伪装的底层现象是不知道的。在这种情况下,从多线索(颜色,纹理,形状,运动等)产生的要素的集成可以增加伪装对象和背景之间的可分离性。已经提出了各种算法,基于在此方向上整合不同的特征,如颜色,纹理,运动,形状等,以检测伪装的物体 Harville *等。*使用两种颜色和深度信息提出了前景分段技术。由于$3 $ D 场景进入$2 $ D 图片,发生信息丢失。这里,考虑深度信息以减少由于$3 $ D 至$2$ D 投影而导致的信息丢失。深度信息的使用增加了检测精度,并且还增加了计算成本。在,Pong 和 Bowdenb 提出使用随机过程来处理伪装。该算法假设当新观察不能与现有任务相关联时发生伪装。该方法的性能取决于对象的颜色和运动。这里,颜色用于区分伪装的物体及其周围,而动作被认为是分离出框架的移动物体和静态部分。因此,这两个信息都有助于检测移动的伪装对象。有时,此方法可能会失败,对象非常类似于背景,并且它包含更多静态对象。在,Brady 和 Kersten 提出了一种计算方法,将低级功能与高级模型相结合以在存在伪装时检测和识别对象。在这里,被认为是构建对象的形状的引导学习,它取决于对象检测的结果。如果对象检测不完美,则识别可能并不总是正确的,并且无法正确跟踪。此外,在,LOSA 提出了一种基于结构相似性测量的粒子滤波器,用于跟踪伪装对象。这里,结构相似度测量通过比较它们的亮度,对比度和空间特征来反映两个帧之间的距离。

Here, the used measure is sensitive to relative rather than absolute changes in the frame. However, this technique may not be robust to significant alteration of the tracked object. However, Jiang proposed a model to track the object in a video sequence by locating a list of object features that are ranked according to their ability to differentiate against the background. Here, a mixture of color, texture, and object motion is considered multidimensional to represent the object. Multi-features representation increases the separability between the objects and their surrounding. Experimental results show that this model works well for detecting the camouflaged object. Recently, Mondal developed a camouflaged object tracking algorithm using a probabilistic neural network and fuzzy energy-based active contour. Here, multi-features like color, texture, and shape are integrated to increase the discrimination between camouflaged objects and their background. The camouflaged object is detected by modifying the probabilistic neural network, and finally, it is tracked by the proposed fuzzy energy-based active contour model. The experiments showed that this method could provide good tracking results under a camouflaged environment. Color features work while camouflage occurs due to texture similarity with the background. Texture features are effective, while camouflage occurs due to color similarity. The motion of the camouflaged object can detect it while the object with good motion. These features, e.g., color, texture, and motion, can detect the camouflaged object in a specific environment. A combination of various features, e.g., color, texture, motion, etc. effectively work without constraints.

这里,使用的措施对相对而不是帧中的绝对变化敏感。然而,这种技术可能对跟踪物体的显着改变不稳定。然而,江提出了一种模型来通过定位根据它们对背景的能力排列的对象特征列表来跟踪视频序列中的对象。这里,颜色,纹理和对象运动的混合被认为是多维以表示对象。多特征表示增加了物体与周围之间的可分离性。实验结果表明,该模型适用于检测伪装对象。最近,Mondal 使用概率神经网络和基于模糊的基于能量的活动轮廓开发了伪装的对象跟踪算法。这里,像颜色,纹理和形状一样的多个功能,以增加伪装对象与其背景之间的歧视。通过修改概率神经网络来检测伪装对象,最后,由所提出的模糊能量基主动轮廓模型跟踪。实验表明,该方法可以在伪装环境下提供良好的跟踪结果。颜色特征在伪装时,由于与背景的纹理相似度而发生。纹理特征是有效的,而伪装是由于颜色相似性而发生的。伪装物体的运动可以在具有良好运动的物体时检测到它。这些功能,例如颜色,纹理和运动,可以检测特定环境中的伪装对象。各种特征的组合,例如颜色,纹理,运动等有效地工作而没有约束。

  Deep Features

As the color of the camouflaged object is similar to the background, detection of it is challenging. In, the authors considered the image enhancement technique to increase the discrimination between object and background. The region proposal network () is considered to estimate the required target's accurate positioning. The deep neural network is used to identify extracted RoI. Finally, the detection of a camouflaged object is done. In this direction, Fang proposed strong semantic dilation network to detect camouflaged people. The authors claimed that full use of semantic information in convolutional neural networks and dilated convolutions are also added to enlarge the receptive field to find camouflage people. The authors presented impressive results on a standard dataset. All the features, e.g., color, texture, motion, and gradient, are hand-crafted and may not use for all types of camouflaged objects. On the contrary, the in-depth feature is learned by the network from the extensive training images. The deep feature is more generic than the hand-crafted features. It works better than hand-crafted features.

 深度功能

作为伪装对象的颜色类似于背景,它的检测是具有挑战性的。在作者中,作者认为图像增强技术增加了对象和背景之间的辨别。该区域提议网络()被认为估计所需的目标准确定位。深神经网络用于识别提取的 ROI。最后,完成伪装对象的检测。朝着这个方向,方面提出了强烈的语义扩张网络来检测伪装的人。作者声称,还增加了充分利用卷积神经网络和扩张卷积的语义信息来扩大接受领域以找到伪装人。作者在标准数据集中呈现出令人印象深刻的结果。所有功能,例如颜色,纹理,运动和渐变是手工制作的,可能无法用于所有类型的伪装对象。相反,网络通过广泛的培训图像学习了深入的特征。深度特征比手工制作的功能更通用。它比手工制作功能更好。

  Other Techniques

Some other approaches that exist in the literature are discussed in this section. Marouani et al. developed aircraft recognition technique in the presence of camouflage. It works on low-level matching between segments of the projection of a $ 3D $ model of the objects of interest. In, Guilan and Shunqing discussed the use of spectral pattern recognition techniques to distinguish color camouflage from green vegetation background. Here, the considered spectral feature is the basic characteristic for recognizing the target. Experimentally, they concluded that the considered feature is optimum for recognition than other state-of-the-art features. However, it cannot provide good results for camouflaged objects in the background other than green vegetation. Beiderman presented a novel approach where the secondary speckle pattern is monitored for a time, to extract the temporal/spectral signature of the objects. Special image processing algorithms allow obtaining a unique signature of the object region that can be used to classify, recognize, and identify objects. This approach can detect and recognize camouflaged objects. Pan developed an algorithm to detect the camouflaged portion of the object in a complex background with the help of $ D_{arg} $ operator. The main drawback of this method is the selection of proper threshold value. Liu and Huang developed a novel foreground object detection scheme that integrates the top-down information based on the expectation-maximization () framework. Here, top-down information is incorporated into the object model in a generalized framework. A foreground model is constructed based on the object model and the state of each target. The author concluded that the method is giving good results for detecting camouflaged objects. Recently, Malathi and Bhuyan developed a background subtraction scheme to detect the camouflaged object. Here, pixels corresponding to the background are quantized into codebooks, sufficient to represent a background. Codebooks extracted for each of the cameras are combined to extract the foreground from the background. To detect the camouflaged target in complex background, Zhou and Cun-chao proposed a novel spectral-polarimetric image fusion algorithm based on Shearlet transform. Kernel fuzzy c-means clustering algorithm is applied to a fused spectral-polarimetric image to separate camouflaged object from its background. In, Kim proposed a fully autonomous feature selection and camouflaged object detection method based on the online analysis of spectral and spatial features for hyper-spectral images. Here, a statistical distance metric is considered to generate candidate feature bands, and entropy-based spatial grouping property is used to reduce useless feature bands. Camouflaged objects are detected by optical spectral-spatial feature analysis with less computational complexity. Mangale and Khambete fused Thermal infrared and visible spectrum imaging modality to detect camouflaged objects. The authors presented impressive results. Li proposed a camouflaged object detection model in wavelet transformed feature space. In this method, the likelihood of each wavelet coefficient being foreground is estimated by formulating foreground and background models for each wavelet band. This method effectively aggregates the likelihoods from different wavelet bands based on the wavelet transforms' characteristics detects camouflaged objects.

 其他技术

在本节中讨论了文献中存在的一些其他方法。 Marouani *等。*在伪装的情况下开发了飞机识别技术。它适用于感兴趣对象的$3D $模型的投影段之间的低级匹配。在,桂兰和顺清讨论了光谱模式识别技术的使用区分了绿色植被背景的彩色伪装。这里,考虑的光谱特征是识别目标的基本特征。在实验上,他们得出结论,考虑的特征是识别的最佳,而不是其他最先进的特征。但是,除了绿色植被之外的背景中,它无法为伪装物体提供良好的结果。 Beiderman 呈现了一种新的方法,其中监视了次要散斑图案的时间,以提取对象的时间/光谱特征。特殊图像处理算法允许获得可用于分类,识别和识别对象的对象区域的唯一签名。这种方法可以检测和识别伪装的物体。 PAN 开发了一种算法,在$D_{arg} $操作员的帮助下检测复杂背景中对象的伪装部分。该方法的主要缺点是选择适当的阈值。刘和黄制定了一种新颖的前景对象检测方案,基于期望 - 最大化()框架集成了自上而下信息。这里,自上而下信息被纳入广义框架中的对象模型。前景模型是基于对象模型和每个目标的状态构建的。作者得出结论,该方法对检测伪装对象提供了良好的效果。最近,Malathi 和 Bhuyan 开发了一种检测伪装对象的背景减法方案。这里,对应于背景的像素被量化为码本,足以表示背景。为每个摄像机提取的码本组合以从背景中提取前景。为了检测复杂背景中的伪装目标,Zhou 和 Cun-Chao 提出了一种基于 Shearlet 变换的新型光谱 - 偏振图像融合算法。内核模糊 C-Means 聚类算法应用于融合光谱 - 偏振图像,以将伪装对象与其背景分开。在,基于对超光谱图像的频谱和空间特征的在线分析,KIM 提出了完全自主特征选择和伪装对象检测方法。这里,认为统计距离度量被认为生成候选特征频带,并且基于熵的空间分组属性用于减少无用的特征频带。通过光谱 - 空间特征分析检测伪装对象,具有较少的计算复杂性。 MANGALE 和 KHAMBETE 融合的热红外和可见光谱成像模型检测伪装的物体。作者提出了令人印象深刻的结果。李提出了小波变换特征空间中的伪装对象检测模型。在该方法中,通过为每个小波频带制定前景和背景模型来估计每个小波系数是前景的可能性。该方法有效地聚集了基于小波变换的不同小波带的似然性检测伪装对象。

  Conclusion and Future Directions

In this article, a review of camouflaged object detection and tracking using computer vision-based approaches are presented. Due to the complexity of the problem, less research work has been done for breaking camouflage. Among them, very few works have been done for extracting camouflaged objects in an unsupervised way. It is crucial to detect the camouflaged object in the real scenario. As color, texture, and shape of the camouflaged object are very similar to its surroundings, techniques based on the single feature cannot extract camouflaged objects well. The integration of several features like color, texture, motion, shape, and some other features related to camouflaged objects' biological characteristics may be considered to improve the camouflage braking system. However, camouflage breaking has several crucial real-life applications, including soldier detection on the battlefield and wildlife preservation. In such cases, the unsupervised technique is necessary to detect such camouflaged objects.

 结论和未来方向

在本文中,介绍了使用计算机视觉的方法的伪装对象检测和跟踪的审查。由于问题的复杂性,缺少伪装已经完成了更少的研究工作。其中,已经以无监督方式提取伪装的物体很少有效。在真实场景中检测伪装对象是至关重要的。作为伪装对象的颜色,纹理和形状与周围环境非常相似,基于单个功能的技术无法良好地提取伪装的物体。可以认为颜色,纹理,动作,形状和与伪装物体的生物特性相关的若干特征的整合可以被认为是改善伪装制动系统。然而,伪装突破具有几种至关重要的现实生活应用,包括战场和野生动物保存的士兵检测。在这种情况下,无监督的技术是检测这种伪装物体的必要条件。