Dog Activities Recognition System using Dog-centered Cropped Images

반려견에 초점을 맞춰 추출하는 영상 기반의 행동 탐지 시스템

In recent years, the growing popularity of dogs due to the benefits they bring their owners has contributed to the increase of the number of dogs raised. For owners, it is their responsibility to ensure their dogs' health and safety. However, it is challenging for them to continuously monitor their dogs' activities, which are important to understand and guarantee their wellbeing. In this work, we introduce a camera-based monitoring system to help owners automatically monitor their dogs' activities. The system receives sequences of RGB images and uses YOLOv7 to detect the dog presence, and then applies post-processing to perform dog-centered image cropping on each input sequence. The optical flow is extracted from each sequence, and both sequences of RGB and flow are input to a two-stream EfficientNet to extract their respective features. Finally, the features are concatenated, and a bi-directional LSTM is utilized to retrieve temporal features and recognize the activity. The experiments prove that our system achieves a good performance with the F-1 score exceeding 0.90 for all activities and reaching 0.963 on average.