• Title/Summary/Keyword: Large Dataset

Search Result 541, Processing Time 0.031 seconds

A Re-configuration Scheme for Social Network Based Large-scale SMS Spam (소셜 네트워크 기반 대량의 SMS 스팸 데이터 재구성 기법)

  • Jeong, Sihyun;Noh, Giseop;Oh, Hayoung;Kim, Chong-Kwon
    • Journal of KIISE
    • /
    • v.42 no.6
    • /
    • pp.801-806
    • /
    • 2015
  • The Short Message Service (SMS) is one of the most popular communication tools in the world. As the cost of SMS decreases, SMS spam has been growing largely. Even though there are many existing studies on SMS spam detection, researchers commonly have limitation collecting users' private SMS contents. They need to gather the information related to social network as well as personal SMS due to the intelligent spammers being aware of the social networks. Therefore, this paper proposes the Social network Building Scheme for SMS spam detection (SBSS) algorithm that builds synthetic social network dataset realistically, without the collection of private information. Also, we analyze and categorize the attack types of SMS spam to build more complete and realistic social network dataset including SMS spam.

Feature Selection for Anomaly Detection Based on Genetic Algorithm (유전 알고리즘 기반의 비정상 행위 탐지를 위한 특징선택)

  • Seo, Jae-Hyun
    • Journal of the Korea Convergence Society
    • /
    • v.9 no.7
    • /
    • pp.1-7
    • /
    • 2018
  • Feature selection, one of data preprocessing techniques, is one of major research areas in many applications dealing with large dataset. It has been used in pattern recognition, machine learning and data mining, and is now widely applied in a variety of fields such as text classification, image retrieval, intrusion detection and genome analysis. The proposed method is based on a genetic algorithm which is one of meta-heuristic algorithms. There are two methods of finding feature subsets: a filter method and a wrapper method. In this study, we use a wrapper method, which evaluates feature subsets using a real classifier, to find an optimal feature subset. The training dataset used in the experiment has a severe class imbalance and it is difficult to improve classification performance for rare classes. After preprocessing the training dataset with SMOTE, we select features and evaluate them with various machine learning algorithms.

A Design of SOA-based Data Integration Framework for Effective Spatial Data Mining (효과적인 공간 데이터 마이닝을 위한 SOA 기반 데이터 통합 프레임워크 설계)

  • Moon, Il-Hwan;Hur, Hwan;Kim, Sam-Keun
    • The KIPS Transactions:PartD
    • /
    • v.18D no.5
    • /
    • pp.385-392
    • /
    • 2011
  • Recently, the concern of IT-in-Agriculture convergence technology that combines information technology and agriculture is increasing rapidly. Especially, the crop cultivation related prediction services by spatial data mining (SDM) can play an important role in reducing the damage of natural disaster and enhancing crop productivity. However, the data conversion and integration procedure to acquire the learning dataset of SDM for the prediction service need a lot of effort and time, because of their heterogeneity between distributed data. In addition, calculating spatial neighborhood relationships between spatial and non-spatial data necessitates requires the complicated calculation procedure for large dataset. In this paper, we suggest a SOA-based data integration framework that can effectively integrate distributed heterogeneous data by treating each data source as a service unit and support to find the optimal prediction service by improving productivity of learning dataset for SDM. In our experiment, we confirmed that our framework can be effectively applied to find the optimal prediction service for the frost damage area, by considering the case of peach crop cultivation in Icheon in Korea.

The development of food image detection and recognition model of Korean food for mobile dietary management

  • Park, Seon-Joo;Palvanov, Akmaljon;Lee, Chang-Ho;Jeong, Nanoom;Cho, Young-Im;Lee, Hae-Jeung
    • Nutrition Research and Practice
    • /
    • v.13 no.6
    • /
    • pp.521-528
    • /
    • 2019
  • BACKGROUND/OBJECTIVES: The aim of this study was to develop Korean food image detection and recognition model for use in mobile devices for accurate estimation of dietary intake. MATERIALS/METHODS: We collected food images by taking pictures or by searching web images and built an image dataset for use in training a complex recognition model for Korean food. Augmentation techniques were performed in order to increase the dataset size. The dataset for training contained more than 92,000 images categorized into 23 groups of Korean food. All images were down-sampled to a fixed resolution of $150{\times}150$ and then randomly divided into training and testing groups at a ratio of 3:1, resulting in 69,000 training images and 23,000 test images. We used a Deep Convolutional Neural Network (DCNN) for the complex recognition model and compared the results with those of other networks: AlexNet, GoogLeNet, Very Deep Convolutional Neural Network, VGG and ResNet, for large-scale image recognition. RESULTS: Our complex food recognition model, K-foodNet, had higher test accuracy (91.3%) and faster recognition time (0.4 ms) than those of the other networks. CONCLUSION: The results showed that K-foodNet achieved better performance in detecting and recognizing Korean food compared to other state-of-the-art models.

Survey on Out-Of-Domain Detection for Dialog Systems (대화시스템 미지원 도메인 검출에 관한 조사)

  • Jeong, Young-Seob;Kim, Young-Min
    • Journal of Convergence for Information Technology
    • /
    • v.9 no.9
    • /
    • pp.1-12
    • /
    • 2019
  • A dialog system becomes a new way of communication between human and computer. The dialog system takes human voice as an input, and gives a proper response in voice or perform an action. Although there are several well-known products of dialog system (e.g., Amazon Echo, Naver Wave), they commonly suffer from a problem of out-of-domain utterances. If it poorly detects out-of-domain utterances, then it will significantly harm the user satisfactory. There have been some studies aimed at solving this problem, but it is still necessary to study about this intensively. In this paper, we give an overview of the previous studies of out-of-domain detection in terms of three point of view: dataset, feature, and method. As there were relatively smaller studies of this topic due to the lack of datasets, we believe that the most important next research step is to construct and share a large dataset for dialog system, and thereafter try state-of-the-art techniques upon the dataset.

Investigating Non-Laboratory Variables to Predict Diabetic and Prediabetic Patients from Electronic Medical Records Using Machine Learning

  • Mukhtar, Hamid;Al Azwari, Sana
    • International Journal of Computer Science & Network Security
    • /
    • v.21 no.9
    • /
    • pp.19-30
    • /
    • 2021
  • Diabetes Mellitus (DM) is one of common chronic diseases leading to severe health complications that may cause death. The disease influences individuals, community, and the government due to the continuous monitoring, lifelong commitment, and the cost of treatment. The World Health Organization (WHO) considers Saudi Arabia as one of the top 10 countries in diabetes prevalence across the world. Since most of the medical services are provided by the government, the cost of the treatment in terms of hospitals and clinical visits and lab tests represents a real burden due to the large scale of the disease. The ability to predict the diabetic status of a patient without the laboratory tests by performing screening based on some personal features can lessen the health and economic burden caused by diabetes alone. The goal of this paper is to investigate the prediction of diabetic and prediabetic patients by considering factors other than the laboratory tests, as required by physicians in general. With the data obtained from local hospitals, medical records were processed to obtain a dataset that classified patients into three classes: diabetic, prediabetic, and non-diabetic. After applying three machine learning algorithms, we established good performance for accuracy, precision, and recall of the models on the dataset. Further analysis was performed on the data to identify important non-laboratory variables related to the patients for diabetes classification. The importance of five variables (gender, physical activity level, hypertension, BMI, and age) from the person's basic health data were investigated to find their contribution to the state of a patient being diabetic, prediabetic or normal. Our analysis presented great agreement with the risk factors of diabetes and prediabetes stated by the American Diabetes Association (ADA) and other health institutions worldwide. We conclude that by performing class-specific analysis of the disease, important factors specific to Saudi population can be identified, whose management can result in controlling the disease. We also provide some recommendations learnt from this research.

A DCT Learning Combined RRU-Net for the Image Splicing Forgery Detection (DCT 학습을 융합한 RRU-Net 기반 이미지 스플라이싱 위조 영역 탐지 모델)

  • Young-min Seo;Jung-woo Han;Hee-jung Kwon;Su-bin Lee;Joongjin Kook
    • Journal of the Semiconductor & Display Technology
    • /
    • v.22 no.1
    • /
    • pp.11-17
    • /
    • 2023
  • This paper proposes a lightweight deep learning network for detecting an image splicing forgery. The research on image forgery detection using CNN, a deep learning network, and research on detecting and localizing forgery in pixel units are in progress. Among them, CAT-Net, which learns the discrete cosine transform coefficients of images together with images, was released in 2022. The DCT coefficients presented by CAT-Net are combined with the JPEG artifact learning module and the backbone model as pre-learning, and the weights are fixed. The dataset used for pre-training is not included in the public dataset, and the backbone model has a relatively large number of network parameters, which causes overfitting in a small dataset, hindering generalization performance. In this paper, this learning module is designed to learn the characterization depending on the DCT domain in real-time during network training without pre-training. The DCT RRU-Net proposed in this paper is a network that combines RRU-Net which detects forgery by learning only images and JPEG artifact learning module. It is confirmed that the network parameters are less than those of CAT-Net, the detection performance of forgery is better than that of RRU-Net, and the generalization performance for various datasets improves through the network architecture and training method of DCT RRU-Net.

  • PDF

A Comparative Study on Artificial in Intelligence Model Performance between Image and Video Recognition in the Fire Detection Area (화재 탐지 영역의 이미지와 동영상 인식 사이 인공지능 모델 성능 비교 연구)

  • Jeong Rok Lee;Dae Woong Lee;Sae Hyun Jeong;Sang Jeong
    • Journal of the Society of Disaster Information
    • /
    • v.19 no.4
    • /
    • pp.968-975
    • /
    • 2023
  • Purpose: We would like to confirm that the false positive rate of flames/smoke is high when detecting fires. Propose a method and dataset to recognize and classify fire situations to reduce the false detection rate. Method: Using the video as learning data, the characteristics of the fire situation were extracted and applied to the classification model. For evaluation, the model performance of Yolov8 and Slowfast were compared and analyzed using the fire dataset conducted by the National Information Society Agency (NIA). Result: YOLO's detection performance varies sensitively depending on the influence of the background, and it was unable to properly detect fires even when the fire scale was too large or too small. Since SlowFast learns the time axis of the video, we confirmed that detects fire excellently even in situations where the shape of an atypical object cannot be clearly inferred because the surrounding area is blurry or bright. Conclusion: It was confirmed that the fire detection rate was more appropriate when using a video-based artificial intelligence detection model rather than using image data.

Training Data Sets Construction from Large Data Set for PCB Character Recognition

  • NDAYISHIMIYE, Fabrice;Gang, Sumyung;Lee, Joon Jae
    • Journal of Multimedia Information System
    • /
    • v.6 no.4
    • /
    • pp.225-234
    • /
    • 2019
  • Deep learning has become increasingly popular in both academic and industrial areas nowadays. Various domains including pattern recognition, Computer vision have witnessed the great power of deep neural networks. However, current studies on deep learning mainly focus on quality data sets with balanced class labels, while training on bad and imbalanced data set have been providing great challenges for classification tasks. We propose in this paper a method of data analysis-based data reduction techniques for selecting good and diversity data samples from a large dataset for a deep learning model. Furthermore, data sampling techniques could be applied to decrease the large size of raw data by retrieving its useful knowledge as representatives. Therefore, instead of dealing with large size of raw data, we can use some data reduction techniques to sample data without losing important information. We group PCB characters in classes and train deep learning on the ResNet56 v2 and SENet model in order to improve the classification performance of optical character recognition (OCR) character classifier.

Changed Relationship between Snowfall over the Yeongdong region of the Korean Peninsula and Large-scale Factors

  • Cho, Keon-Hee;Chang, Eun-Chul
    • Journal of the Korean earth science society
    • /
    • v.38 no.3
    • /
    • pp.182-193
    • /
    • 2017
  • A typical snowfall pattern occurs over the east coastal region of the Korean Peninsula, known as the Yeongdong region. The precipitation over the Yeongdong region is influenced by the cold and dry northeasterly wind which advects over warm and moist sea surface of the East Sea of Korea. This study reveals the influence of large-scale factors, affecting local to remote areas, on the mesoscale snowfall system over the Yeongdong region. The National Centers for Environmental Prediction-Department of Energy reanalysis dataset, Extended Reconstructed sea surface temperature, and observed snowfall data are analyzed to reveal the relationship between February snowfall and large-scale factors from 1981 to 2014. The Yeongdong snowfall is associated with the sea level pressure patterns over the Gaema Plateau and North Pacific near the Bering Sea, which is remotely associated to the sea surface temperature (SST) variability over the North Pacific. It is presented that the relationship between the Yeongdong snowfall and large-scale factors is strengthened after 1999 when the central north Pacific has warm anomalous SST. These enhanced relationships explain the atmospheric patterns of recent strong snowfall years (2010, 2011, and 2014). It is suggested that the newly defined index in this study based on related SST variability can be used for a seasonal predictor of the Yeongdong snowfall with 2-month leading.