• Journal of Information Processing Systems
• /
• v.16 no.6
• /
• pp.1250-1260
• /
• 2020

This study proposes a novel deep neural network model that can accurately detect objects and their relationships in an image and represent them as a scene graph. The proposed model utilizes several multimodal features, including linguistic features and visual context features, to accurately detect objects and relationships. In addition, in the proposed model, context features are embedded using graph neural networks to depict the dependencies between two related objects in the context feature vector. This study demonstrates the effectiveness of the proposed model through comparative experiments using the Visual Genome benchmark dataset.

• KSII Transactions on Internet and Information Systems (TIIS)
• /
• v.15 no.12
• /
• pp.4456-4475
• /
• 2021

The system for re-identifying persons is used to find and verify the persons crossing through different spots using various cameras. Much research has been done to re-identify the person by utilising features with deep-learned or hand-crafted information. Deep learning techniques segregate and analyse the features of their layers in various forms, and the output is complex feature vectors. This paper proposes a distinctive framework called Integrated Level Feature Pattern (ILFP) framework, which integrates local and global features. A new deep learning architecture named modified XceptionNet (m-XceptionNet) is also proposed in this work, which extracts the global features effectively with lesser complexity. The proposed framework gives better performance in Rank1 metric for Market1501 (96.15%), CUHK03 (82.29%) and the newly created NEC01 (96.66%) datasets than the existing works. The mean Average Precision (mAP) calculated using the proposed framework gives 92%, 85% and 98%, respectively, for the same datasets.

• KIPS Transactions on Software and Data Engineering
• /
• v.8 no.8
• /
• pp.343-348
• /
• 2019

In recent years, extractive summarization systems based on end-to-end deep learning models have become popular. These systems do not require human-crafted features and adopt data-driven approaches. However, previous related studies have shown that linguistic analysis features such as part-of-speeches, named entities and word's frequencies are useful for extracting important sentences from a document to generate a summary. In this paper, we propose an extractive summarization system based on deep neural networks using conventional linguistic analysis features. In order to prove the usefulness of the linguistic analysis features, we compare the models with and without those features. The experimental results show that the model with the linguistic analysis features improves the Rouge-2 F1 score by 0.5 points compared to the model without those features.

• Journal of Korea Multimedia Society
• /
• v.22 no.10
• /
• pp.1133-1141
• /
• 2019

Most of the previous studies for bark recognition have focused on the extraction of LBP-like statistical features. Deep learning approach was not well studied because of the difficulty of acquiring large volume of bark image dataset. To overcome the bark dataset problem, this study utilizes the MobileNet which was trained with the ImageNet dataset. This study proposes two approaches. One is to extract features by the pixel-wise convolution and classify the features with SVM. The other is to tune the weights of the MobileNet by flexibly freezing layers. The experimental results with two public bark datasets, BarkTex and Trunk12, show that the proposed methods are effective in bark recognition. Especially the results of the flexible tunning method outperform state-of-the-art methods. In addition, it can be applied to mobile devices because the MobileNet is compact compared to other deep learning models.

• KSII Transactions on Internet and Information Systems (TIIS)
• /
• v.11 no.2
• /
• pp.1118-1133
• /
• 2017

Computer vision-based human activity recognition (HAR) has become very famous these days due to its applications in various fields such as smart home healthcare for elderly people. A video-based activity recognition system basically has many goals such as to react based on people's behavior that allows the systems to proactively assist them with their tasks. A novel approach is proposed in this work for depth video based human activity recognition using joint-based motion features of depth body shapes and Deep Belief Network (DBN). From depth video, different body parts of human activities are segmented first by means of a trained random forest. The motion features representing the magnitude and direction of each joint in next frame are extracted. Finally, the features are applied for training a DBN to be used for recognition later. The proposed HAR approach showed superior performance over conventional approaches on private and public datasets, indicating a prominent approach for practical applications in smartly controlled environments.

• Journal of Information Processing Systems
• /
• v.14 no.6
• /
• pp.1318-1330
• /
• 2018

Video captioning refers to the process of extracting features from a video and generating video captions using the extracted features. This paper introduces a deep neural network model and its learning method for effective video captioning. In this study, visual features as well as semantic features, which effectively express the video, are also used. The visual features of the video are extracted using convolutional neural networks, such as C3D and ResNet, while the semantic features are extracted using a semantic feature extraction network proposed in this paper. Further, an attention-based caption generation network is proposed for effective generation of video captions using the extracted features. The performance and effectiveness of the proposed model is verified through various experiments using two large-scale video benchmarks such as the Microsoft Video Description (MSVD) and the Microsoft Research Video-To-Text (MSR-VTT).

• Journal of the Korea Society of Computer and Information
• /
• v.24 no.9
• /
• pp.21-27
• /
• 2019

In this paper, we propose a deep auto-encoder-based pipe leak detection (PLD) technique from time-series acoustic data collected by microphone sensor nodes. The key idea of the proposed technique is to learn representative features of the leak-free state using leak-free time-series acoustic data and the deep auto-encoder. The proposed technique can be used to create a PLD model that detects leaks in the pipeline in an unsupervised learning manner. This means that we only use leak-free data without labeling while training the deep auto-encoder. In addition, when compared to the previous supervised learning-based PLD method that uses image features, this technique does not require complex preprocessing of time-series acoustic data owing to the unsupervised feature extraction scheme. The experimental results show that the proposed PLD method using the deep auto-encoder can provide reliable PLD accuracy even considering unsupervised learning-based feature extraction.

• KSII Transactions on Internet and Information Systems (TIIS)
• /
• v.13 no.4
• /
• pp.2180-2197
• /
• 2019

With the proliferation of the Android malicious applications, malware becomes more capable of hiding or confusing its malicious intent through the use of code obfuscation, which has significantly weaken the effectiveness of the conventional defense mechanisms. Therefore, in order to effectively detect unknown malicious applications on the Android platform, we propose DroidVecDeep, an Android malware detection method using deep learning technique. First, we extract various features and rank them using Mean Decrease Impurity. Second, we transform the features into compact vectors based on word2vec. Finally, we train the classifier based on deep learning model. A comprehensive experimental study on a real sample collection was performed to compare various malware detection approaches. Experimental results demonstrate that the proposed method outperforms other Android malware detection techniques.

• The Journal of the Acoustical Society of Korea
• /
• v.41 no.2
• /
• pp.115-121
• /
• 2022

In this paper, we propose a music classification system according to user emotions using Electroencephalogram (EEG) features that appear when listening to music. In the proposed system, the relationship between the emotional EEG features extracted from EEG signals and the auditory features extracted from music signals is learned through a deep regression neural network. The proposed system based on the regression model automatically generates EEG features mapped to the auditory characteristics of the input music, and automatically classifies music by applying these features to an attention-based deep neural network. The experimental results suggest the music classification accuracy of the proposed automatic music classification framework.

• KSII Transactions on Internet and Information Systems (TIIS)
• /
• v.12 no.1
• /
• pp.454-475
• /
• 2018

This paper presents a novel Android malware classification model planned to classify and categorize Android malicious code at Drebin dataset. The amount of malicious mobile application targeting Android based smartphones has increased rapidly. In this paper, Restricted Boltzmann Machine and Deep Belief Network are used to classify malware into families of Android application. A texture-fingerprint based approach is proposed to extract or detect the feature of malware content. A malware has a unique "image texture" in feature spatial relations. The method uses information on texture image extracted from malicious or benign code, which are mapped to uncompressed gray-scale according to the texture image-based approach. By studying and extracting the implicit features of the API call from a large number of training samples, we get the original dynamic activity features sets. In order to improve the accuracy of classification algorithm on the features selection, on the basis of which, it combines the implicit features of the texture image and API call in malicious code, to train Restricted Boltzmann Machine and Back Propagation. In an evaluation with different malware and benign samples, the experimental results suggest that the usability of this method---using Deep Belief Network to classify Android malware by their texture images and API calls, it detects more than 94% of the malware with few false alarms. Which is higher than shallow machine learning algorithm clearly.