• Title/Summary/Keyword: Video object segmentation

Search Result 20, Processing Time 0.305 seconds

Higher-Order Conditional Random Field established with CNNs for Video Object Segmentation

  • Hao, Chuanyan;Wang, Yuqi;Jiang, Bo;Liu, Sijiang;Yang, Zhi-Xin
    • KSII Transactions on Internet and Information Systems (TIIS)
    • /
    • v.15 no.9
    • /
    • pp.3204-3220
    • /
    • 2021
  • We perform the task of video object segmentation by incorporating a conditional random field (CRF) and convolutional neural networks (CNNs). Most methods employ a CRF to refine a coarse output from fully convolutional networks. Others treat the inference process of the CRF as a recurrent neural network and then combine CNNs and the CRF into an end-to-end model for video object segmentation. In contrast to these methods, we propose a novel higher-order CRF model to solve the problem of video object segmentation. Specifically, we use CNNs to establish a higher-order dependence among pixels, and this dependence can provide critical global information for a segmentation model to enhance the global consistency of segmentation. In general, the optimization of the higher-order energy is extremely difficult. To make the problem tractable, we decompose the higher-order energy into two parts by utilizing auxiliary variables and then solve it by using an iterative process. We conduct quantitative and qualitative analyses on multiple datasets, and the proposed method achieves competitive results.

Video Object Segmentation with Weakly Temporal Information

  • Zhang, Yikun;Yao, Rui;Jiang, Qingnan;Zhang, Changbin;Wang, Shi
    • KSII Transactions on Internet and Information Systems (TIIS)
    • /
    • v.13 no.3
    • /
    • pp.1434-1449
    • /
    • 2019
  • Video object segmentation is a significant task in computer vision, but its performance is not very satisfactory. A method of video object segmentation using weakly temporal information is presented in this paper. Motivated by the phenomenon in reality that the motion of the object is a continuous and smooth process and the appearance of the object does not change much between adjacent frames in the video sequences, we use a feed-forward architecture with motion estimation to predict the mask of the current frame. We extend an additional mask channel for the previous frame segmentation result. The mask of the previous frame is treated as the input of the expanded channel after processing, and then we extract the temporal feature of the object and fuse it with other feature maps to generate the final mask. In addition, we introduce multi-mask guidance to improve the stability of the model. Moreover, we enhance segmentation performance by further training with the masks already obtained. Experiments show that our method achieves competitive results on DAVIS-2016 on single object segmentation compared to some state-of-the-art algorithms.

Video object segmentation and frame preprocessing for real-time and high compression MPEG-4 encoding (실시간 고압축 MPEG-4 부호화를 위한 비디오 객체 분할과 프레임 전처리)

  • 김준기;이호석
    • The Journal of Korean Institute of Communications and Information Sciences
    • /
    • v.28 no.2C
    • /
    • pp.147-161
    • /
    • 2003
  • Video object segmentation is one of the core technologies for content-based real-time MPEG-4 encoding system. For real-time requirement, the segmentation algorithm should be fast and accurate but almost all existing algorithms are computationally intensive and not suitable for real-time applications. The MPEG-4 VM(Verification Model) has provided basic algorithms for MPEG-4 encoding but it has many limitations in practical software development, real-time camera input system and compression efficiency. In this paper, we implemented the preprocessing system for real-time camera input and VOP extraction for content-based video coding and also implemented motion detection to achieve the 180 : 1 compression rate for real-time and high compression MPEG-4 encoding.

Implementation of Video Object Segmentation System for Interactive Personal Broadcasting Service (양방향 개인방송 서비스를 위한 동영상 객체분할 시스템의 구현)

  • Yu, Hong-Yeon;Jun, Do-Young;Kim, Min-Sung;Hong, Sung-Hoon
    • Proceedings of the KIEE Conference
    • /
    • /
    • pp.17-19
    • /
    • 2007
  • This paper describe an interactive video object segmentation tool which can be used to generate MPEG-4 video object planes for multimedia broadcasting and enables content based functionalities. In order to apply these functionalities, each frame of video sequence should be represented in terms of video objects. Semiautomatic segmentation can be thought of as a user-assisted segmentation technique. A user can initially mark objects of interest around the real object boundaries. Then the user-guided and selected objects are continuously separated from the unselected areas though time evolution in the image sequences. We proposed method shows very promising result and this encourages the development of object based video editing system.

  • PDF

A Study on Video Object Segmentation using Nonlinear Multiscale Filtering (비선형 다중스케일 필터링을 사용한 비디오 객체 분할에 관한 연구)

  • 이웅희;김태희;이규동;정동석
    • The Journal of Korean Institute of Communications and Information Sciences
    • /
    • v.28 no.10C
    • /
    • pp.1023-1032
    • /
    • 2003
  • Object-based coding, such as MPEG-4, enables various content-based functionalities for multimedia applications. In order to support such functionalities, as well as to improve coding efficiency, each frame of video sequences should be segmented into video objects. In this paper. we propose an effective video object segmentation method using nonlinear multiscale filtering and spatio-temporal information. Proposed method performs a spatial segmentation using a nonlinear multiscale filtering based on the stabilized inverse diffusion equation(SIDE). And, the segmented regions are merged using region adjacency graph(RAG). In this paper, we use a statistical significance test and a time-variant memory as temporal segmentation methods. By combining of extracted spatial and temporal segmentations, we can segment the video objects effectively. Proposed method is more robust to noise than the existing watershed algorithm. Experimental result shows that the proposed method improves a boundary accuracy ratio by 43% on "Akiyo" and by 29% on "Claire" than A. Neri's Method does.

Computation ally Efficient Video Object Segmentation using SOM-Based Hierarchical Clustering (SOM 기반의 계층적 군집 방법을 이용한 계산 효율적 비디오 객체 분할)

  • Jung Chan-Ho;Kim Gyeong-Hwan
    • Journal of the Institute of Electronics Engineers of Korea SP
    • /
    • v.43 no.4
    • /
    • pp.74-86
    • /
    • 2006
  • This paper proposes a robust and computationally efficient algorithm for automatic video object segmentation. For implementing the spatio-temporal segmentation, which aims for efficient combination of the motion segmentation and the color segmentation, an SOM-based hierarchical clustering method in which the segmentation process is regarded as clustering of feature vectors is employed. As results, problems of high computational complexity which required for obtaining exact segmentation results in conventional video object segmentation methods, and the performance degradation due to noise are significantly reduced. A measure of motion vector reliability which employs MRF-based MAP estimation scheme has been introduced to minimize the influence from the motion estimation error. In addition, a noise elimination scheme based on the motion reliability histogram and a clustering validity index for automatically identifying the number of objects in the scene have been applied. A cross projection method for effective object tracking and a dynamic memory to maintain temporal coherency have been introduced as well. A set of experiments has been conducted over several video sequences to evaluate the proposed algorithm, and the efficiency in terms of computational complexity, robustness from noise, and higher segmentation accuracy of the proposed algorithm have been proved.

Video Object Segmentation using Kernel Density Estimation and Spatio-temporal Coherence (커널 밀도 추정과 시공간 일치성을 이용한 동영상 객체 분할)

  • Ahn, Jae-Kyun;Kim, Chang-Su
    • Journal of IKEEE
    • /
    • v.13 no.4
    • /
    • pp.1-7
    • /
    • 2009
  • A video segmentation algorithm, which can extract objects even with non-stationary backgrounds, is proposed in this work. The proposed algorithm is composed of three steps. First, we perform an initial segmentation interactively to build the probability density functions of colors per each macro block via kernel density estimation. Then, for each subsequent frame, we construct a coherence strip, which is likely to contain the object contour, by exploiting spatio-temporal correlations. Finally, we perform the segmentation by minimizing an energy function composed of color, coherence, and smoothness terms. Experimental results on various test sequences show that the proposed algorithm provides accurate segmentation results.

  • PDF

A design of MPEG-4 video object segmentation using color/motion information (칼라/움직임 정보를 이용한 MPEG-4 비디오 객체 분할 설계)

  • 김준기;이호석
    • Proceedings of the Korean Information Science Society Conference
    • /
    • /
    • pp.206-208
    • /
    • 2000
  • 본 논문은 칼라 정보와 움직임 정보를 이용한 객체 분할 기법의 설계에 대하여 소개한다. 객체 분할 알고리즘은 L*u*v 공간의 칼라 특성과 움직임 특성을 결합하여 설계하였다. 즉 공간 분할은 mean shift 칼라 클러스터링 알고리즘(color clustering algorithm)을 사용하여 중심 칼라 영역에 따라 동일한 칼라 지역으로 통합한다. 시간 분할은 움직임 검출을 위하여 affine six parameter 움직임 모델과 optical flow equation를 이용하여 움직임이 발생한 부분을 검출한다. 다음에 공간 분할과 시간 분할에 따라 결과를 통합하고 MAD(mean absolute difference)를 사용하여 객체를 추출하는 알고리즘을 설계하였다.

  • PDF

딥러닝 기반 동영상 객체 분할 기술 동향

  • Go, Yeong-Jun
    • Broadcasting and Media Magazine
    • /
    • v.25 no.2
    • /
    • pp.44-51
    • /
    • 2020
  • 동영상 프레임 내 객체 영역들을 배경으로부터 분할하는 기술인 동영상 객체 분할(video object segmentation)은 다양한 컴퓨터 비전 분야에 활용 가능한 연구 분야이다. 최근, 동영상 객체 분할과 관련된 연구 내용으로 CVPR, ICCV, ECCV의 컴퓨터 비전 최우수 학회에 매년 20편 가까이 발표될 정도로 많은 관심을 받고 있다. 동영상 객체 분할은 사용자가 제공하는 정보에 따라 비지도(unsupervised) 동영상 객체 분할, 준지도(semi-supervised) 동영상 객체 분할, 인터렉티브(interactive) 동영상 객체 분할의 세 카테고리로 분류할 수 있다. 본 고에서는 최근 연구가 활발하게 수행되고 있는 비지도 동영상 객체 분할과 준지도 동영상 객체 분할 연구의 최신 동향에 대해 소개하고자 한다.

Automatic Video Object Segmentation Using Effective Thresholding (효과적인 임계값을 이용한 자동영상 분할 기법)

  • 이지호;유홍연;홍성훈
    • Proceedings of the IEEK Conference
    • /
    • /
    • pp.1976-1979
    • /
    • 2003
  • 본 논문에서는 연속영상에서 잡음과 객체가 잘 분할되지 않는 환경 내에 있는 객체를 자동으로 분할하는 차영상 기반 알고리즘을 제안하였다. 기존의 차영상 기반의 단일 임계간을 이용한 방식에는 잡음에 크게 영향을 받고 배경과 객체가 비슷한 밝기 값을 가지는 경우 잘 추출되지 않는 많은 문제점이 있다. 본 논문에서는 이러한 문제점을 해결하고자 임계값을 설정하는 영역을 축소하여 잡음간섭의 최소화를 구성하였고 축소된 영역 내의 윤곽선정보를 이용하여 배경 밝기 값의 유사함에서 나오는 간섭을 최소화함으로써 정밀한 객체를 추출할 수 있었다.

  • PDF