• The Journal of Society for e-Business Studies
• /
• v.27 no.1
• /
• pp.15-28
• /
• 2022

Due to recent expansion of online market such as clothing, utilizing customer review has become a major marketing measure. User review has been used as a tool of analyzing sentiment of customers. Sentiment analysis can be largely classified with machine learning-based and lexicon-based method. Machine learning-based method is a learning classification model referring review and labels. As research of sentiment analysis has been developed, multi-modal models learned by images and video data in reviews has been studied. Characteristics of words in reviews are differentiated depending on products' and customers' categories. In this paper, sentiment is analyzed via considering review data and metadata of products and users. Gated Recurrent Unit (GRU), Long Short-Term Memory (LSTM), Self Attention-based Multi-head Attention models and Bidirectional Encoder Representation from Transformer (BERT) are used in this study. Same Multi-Layer Perceptron (MLP) model is used upon every products information. This paper suggests a multi-modal sentiment analysis model that simultaneously considers user reviews and product meta-information.

• ETRI Journal
• /
• v.44 no.2
• /
• pp.194-207
• /
• 2022

In the last decade, substantial progress has been achieved in intelligent traffic control technologies to overcome consistent difficulties of traffic congestion and its adverse effect on smart cities. Edge computing is one such advanced progress facilitating real-time data transmission among vehicles and roadside units to mitigate congestion. An edge computing-based deep reinforcement learning system is demonstrated in this study that appropriately designs a multiobjective reward function for optimizing different objectives. The system seeks to overcome the challenge of evaluating actions with a simple numerical reward. The selection of reward functions has a significant impact on agents' ability to acquire the ideal behavior for managing multiple traffic signals in a large-scale road network. To ascertain effective reward functions, the agent is trained withusing the proximal policy optimization method in several deep neural network models, including the state-of-the-art transformer network. The system is verified using both hypothetical scenarios and real-world traffic maps. The comprehensive simulation outcomes demonstrate the potency of the suggested reward functions.

• Proceedings of the Korea Information Processing Society Conference
• /
• 2022.11a
• /
• pp.510-512
• /
• 2022

컴퓨터 비전 분야에서 오랫동안 사용되었던 CNN(Convolution Neural Network)은 오분류를 일으키기 위해 악의적으로 추가된 섭동에 매우 취약하다. ViT(Vision Transformer)는 입력 이미지의 전체적인 특징을 탐색하는 어텐션 구조를 적용함으로 CNN의 국소적 특징 탐색보다 특성 픽셀에 섭동을 추가하는 적대적 공격에 강건한 특성을 보이지만 최근 어텐션 구조에 대한 강건성 분석과 다양한 공격 기법의 발달로 보안 취약성 문제가 제기되고 있다. 본 논문은 ViT가 CNN 대비 강건성을 가지는 구조적인 특징을 분석하는 연구와 어텐션 구조에 대한 최신 공격기법을 소개함으로 향후 등장할 ViT 파생 모델의 강건성을 유지하기 위해 중점적으로 다루어야 할 부분이 무엇인지 소개한다.

• Smart Media Journal
• /
• v.12 no.9
• /
• pp.45-59
• /
• 2023

Weakly supervised object localization (WSOL) is a task of localizing an object in an image using only image-level labels. Previous studies have followed the conventional class activation mapping (CAM) pipeline. However, we reveal the current CAM approach suffers from problems which cause original CAM could not capture the complete defects features. This work utilizes a convolutional neural network (CNN) pretrained on image-level labels to generate class activation maps in a multi-scale manner to highlight discriminative regions. Additionally, a vision transformer (ViT) pretrained was treated to produce multi-head attention maps as an auxiliary detector. By integrating the CNN-based CAMs and attention maps, our approach localizes defective regions without requiring bounding box or pixel-level supervision during training. We evaluate our approach on a dataset of apple images with only image-level labels of defect categories. Experiments demonstrate our proposed method aligns with several Object Detection models performance, hold a promise for improving localization.

• The Journal of the Institute of Internet, Broadcasting and Communication
• /
• v.23 no.4
• /
• pp.115-119
• /
• 2023

The natural language processing models used in current artificial intelligence are huge, causing various difficulties in processing and analyzing data in real time. In order to solve these difficulties, we proposed a method to improve the efficiency of processing by using less memory and checked the performance of the proposed model. The technique applied in this paper to evaluate the performance of the proposed model is to divide the large corpus by adjusting the number of attention heads and embedding size of the BERT[1] model to be small, and the results are calculated by averaging the output values of each forward. In this process, a random offset was assigned to the sentences at every epoch to provide diversity in the input data. The model was then fine-tuned for classification. We found that the split processing model was about 12% less accurate than the unsplit model, but the number of parameters in the model was reduced by 56%.

• Journal of the Korea Society of Computer and Information
• /
• v.28 no.4
• /
• pp.31-39
• /
• 2023

In this paper, we propose an Internal/External Knowledge Distillation (IEKD), which utilizes both external correlations between feature maps of heterogeneous models and internal correlations between feature maps of the same model for transferring knowledge from a teacher model to a student model. To achieve this, we transform feature maps into a sequence format and extract new feature maps suitable for knowledge distillation by considering internal and external correlations through a transformer. We can learn both internal and external correlations by distilling the extracted feature maps and improve the accuracy of the student model by utilizing the extracted feature maps with feature matching. To demonstrate the effectiveness of our proposed knowledge distillation method, we achieved 76.23% Top-1 image classification accuracy on the CIFAR-100 dataset with the "ResNet-32×4/VGG-8" teacher and student combination and outperformed the state-of-the-art KD methods.

• International Journal of Computer Science & Network Security
• /
• v.21 no.9
• /
• pp.281-291
• /
• 2021

The Islamic jurisprudential legal system represents an essential component of the Islamic religion, that governs many aspects of Muslims' daily lives. This creates many questions that require interpretations by qualified specialists, or Muftis according to the main sources of legislation in Islam. The Islamic jurisprudence is usually classified into branches, according to which the questions can be categorized and classified. Such categorization has many applications in automated question-answering systems, and in manual systems in routing the questions to a specialized Mufti to answer specific topics. In this work we tackle the problem of automatic categorisation of Islamic jurisprudential legal questions using deep learning techniques. In this paper, we build a hierarchical deep learning model that first extracts the question text features at two levels: word and sentence representation, followed by a text classifier that acts upon the question representation. To evaluate our model, we build and release the largest publicly available dataset of Islamic questions and answers, along with their topics, for 52 topic categories. We evaluate different state-of-the art deep learning models, both for word and sentence embeddings, comparing recurrent and transformer-based techniques, and performing extensive ablation studies to show the effect of each model choice. Our hierarchical model is based on pre-trained models, taking advantage of the recent advancement of transfer learning techniques, focused on Arabic language.

• KIPS Transactions on Software and Data Engineering
• /
• v.13 no.1
• /
• pp.35-49
• /
• 2024

Video captioning technology, as a significant outcome of the integration between computer vision and natural language processing, has emerged as a key research direction in the field of artificial intelligence. This technology aims to achieve automatic understanding and language expression of video content, enabling computers to transform visual information in videos into textual form. This paper provides an initial analysis of the research trends in deep learning-based video captioning and categorizes them into four main groups: CNN-RNN-based Model, RNN-RNN-based Model, Multimodal-based Model, and Transformer-based Model, and explain the concept of each video captioning model. The features, pros and cons were discussed. This paper lists commonly used datasets and performance evaluation methods in the video captioning field. The dataset encompasses diverse domains and scenarios, offering extensive resources for the training and validation of video captioning models. The model performance evaluation method mentions major evaluation indicators and provides practical references for researchers to evaluate model performance from various angles. Finally, as future research tasks for video captioning, there are major challenges that need to be continuously improved, such as maintaining temporal consistency and accurate description of dynamic scenes, which increase the complexity in real-world applications, and new tasks that need to be studied are presented such as temporal relationship modeling and multimodal data integration.

• The Journal of the Acoustical Society of Korea
• /
• v.40 no.5
• /
• pp.530-536
• /
• 2021

Recently, due to the development of deep learning, end-to-end speech recognition, which directly maps graphemes to speech signals, shows good performance. Especially, among the end-to-end models, conformer shows the best performance. However end-to-end models only focuses on the probability of which grapheme will appear at the time. The decoding process uses a greedy search or beam search. This decoding method is easily affected by the final probability output by the model. In addition, the end-to-end models cannot use external pronunciation and language information due to structual problem. Therefore, in this paper conformer with lexicon transducer is proposed. We compare phoneme-based model with lexicon transducer and grapheme-based model with beam search. Test set is consist of words that do not appear in training data. The grapheme-based conformer with beam search shows 3.8 % of CER. The phoneme-based conformer with lexicon transducer shows 3.4 % of CER.

• Proceedings of the Korean Institute of Information and Commucation Sciences Conference
• /
• 2016.05a
• /
• pp.505-508
• /
• 2016

In city, tracing of power transmission lines is difficult due to compound installation of overhead and underground lines, transposition, bad view caused by trees or big buildings. It is hard problem for electrical technician on site to trace power transformers or power lines to serve customers in 3 phase -4 wires power distribution systems. It is necessary that the correct and fast tracing method is required for load balancing among distribution lines. Old technology use to trace lines with high power impulse injection. Our proposed method uses to trace lines with very small power high frequency signal injection. Simulation models for 3-phase power transformers, 3-phase wire lines, and customer loads are described to investigate the transmission characteristics of high frequency power line carrier. Distribution lines have only a limited ability to carry higher frequencies. Typically power transformers in the distribution system prevent propagating the higher frequency carrier signal. The proposed method uses the limited propagation ability to identify the power transformer to serve customers. The system consists of a transmitter and a receiver with power-line communication module. Some experiments are conducted to verify the theoretical concepts in a big commercial building. Also some simulations are done to help and understand the concepts by using MATLAB Simulink simulator.