• Title/Summary/Keyword: Learning data set

Search Result 1,114, Processing Time 0.026 seconds

Predictive Analysis of Financial Fraud Detection using Azure and Spark ML

  • Priyanka Purushu;Niklas Melcher;Bhagyashree Bhagwat;Jongwook Woo
    • Asia pacific journal of information systems
    • /
    • v.28 no.4
    • /
    • pp.308-319
    • /
    • 2018
  • This paper aims at providing valuable insights on Financial Fraud Detection on a mobile money transactional activity. We have predicted and classified the transaction as normal or fraud with a small sample and massive data set using Azure and Spark ML, which are traditional systems and Big Data respectively. Experimenting with sample dataset in Azure, we found that the Decision Forest model is the most accurate to proceed in terms of the recall value. For the massive data set using Spark ML, it is found that the Random Forest classifier algorithm of the classification model proves to be the best algorithm. It is presented that the Spark cluster gets much faster to build and evaluate models as adding more servers to the cluster with the same accuracy, which proves that the large scale data set can be predictable using Big Data platform. Finally, we reached a recall score with 0.73, which implies a satisfying prediction quality in predicting fraudulent transactions.

Ensemble Learning of Region Based Classifiers (지역 기반 분류기의 앙상블 학습)

  • Choi, Sung-Ha;Lee, Byung-Woo;Yang, Ji-Hoon
    • The KIPS Transactions:PartB
    • /
    • v.14B no.4
    • /
    • pp.303-310
    • /
    • 2007
  • In machine learning, the ensemble classifier that is a set of classifiers have been introduced for higher accuracy than individual classifiers. We propose a new ensemble learning method that employs a set of region based classifiers. To show the performance of the proposed method. we compared its performance with that of bagging and boosting, which ard existing ensemble methods. Since the distribution of data can be different in different regions in the feature space, we split the data and generate classifiers based on each region and apply a weighted voting among the classifiers. We used 11 data sets from the UCI Machine Learning Repository to compare the performance of our new ensemble method with that of individual classifiers as well as existing ensemble methods such as bagging and boosting. As a result, we found that our method produced improved performance, particularly when the base learner is Naive Bayes or SVM.

PubMiner: Machine Learning-based Text Mining for Biomedical Information Analysis

  • Eom, Jae-Hong;Zhang, Byoung-Tak
    • Genomics & Informatics
    • /
    • v.2 no.2
    • /
    • pp.99-106
    • /
    • 2004
  • In this paper we introduce PubMiner, an intelligent machine learning based text mining system for mining biological information from the literature. PubMiner employs natural language processing techniques and machine learning based data mining techniques for mining useful biological information such as protein­protein interaction from the massive literature. The system recognizes biological terms such as gene, protein, and enzymes and extracts their interactions described in the document through natural language processing. The extracted interactions are further analyzed with a set of features of each entity that were collected from the related public databases to infer more interactions from the original interactions. An inferred interaction from the interaction analysis and native interaction are provided to the user with the link of literature sources. The performance of entity and interaction extraction was tested with selected MEDLINE abstracts. The evaluation of inference proceeded using the protein interaction data of S. cerevisiae (bakers yeast) from MIPS and SGD.

A Ship Intelligent Anti-Collision Decision-Making Supporting System Based On Trial Manoeuvre

  • Zhuo, Yongqiang;Yao, Jie
    • Proceedings of the Korean Institute of Navigation and Port Research Conference
    • /
    • 2006.10a
    • /
    • pp.176-183
    • /
    • 2006
  • A novel intelligent anti-collision decision-making supporting system is addressed in this paper. To obtain precise anti-collision information capability, an innovative neurofuzzy network is proposed and applied. A fuzzy set interpretation is incorporated into the network design to handle imprecise information. A neural network architecture is used to train the parameters of the Fuzzy Inference System (FIS). The learning process is based on a hybrid learning algorithm and off-line training data. The training data are obtained by trial manoeuvre. This neurofuzzy network can be considered to be a self-learning system with the ability to learn new information adaptively without forgetting old knowledge. This supporting system can decrease ship operators' burden to deal with bridge data and help them to make a precise anti-collision decision.

  • PDF

Effective Analsis of GAN based Fake Date for the Deep Learning Model (딥러닝 훈련을 위한 GAN 기반 거짓 영상 분석효과에 대한 연구)

  • Seungmin, Jang;Seungwoo, Son;Bongsuck, Kim
    • KEPCO Journal on Electric Power and Energy
    • /
    • v.8 no.2
    • /
    • pp.137-141
    • /
    • 2022
  • To inspect the power facility faults using artificial intelligence, it need that improve the accuracy of the diagnostic model are required. Data augmentation skill using generative adversarial network (GAN) is one of the best ways to improve deep learning performance. GAN model can create realistic-looking fake images using two competitive learning networks such as discriminator and generator. In this study, we intend to verify the effectiveness of virtual data generation technology by including the fake image of power facility generated through GAN in the deep learning training set. The GAN-based fake image was created for damage of LP insulator, and ResNet based normal and defect classification model was developed to verify the effect. Through this, we analyzed the model accuracy according to the ratio of normal and defective training data.

Semi-Supervised Recursive Learning of Discriminative Mixture Models for Time-Series Classification

  • Kim, Minyoung
    • International Journal of Fuzzy Logic and Intelligent Systems
    • /
    • v.13 no.3
    • /
    • pp.186-199
    • /
    • 2013
  • We pose pattern classification as a density estimation problem where we consider mixtures of generative models under partially labeled data setups. Unlike traditional approaches that estimate density everywhere in data space, we focus on the density along the decision boundary that can yield more discriminative models with superior classification performance. We extend our earlier work on the recursive estimation method for discriminative mixture models to semi-supervised learning setups where some of the data points lack class labels. Our model exploits the mixture structure in the functional gradient framework: it searches for the base mixture component model in a greedy fashion, maximizing the conditional class likelihoods for the labeled data and at the same time minimizing the uncertainty of class label prediction for unlabeled data points. The objective can be effectively imposed as individual mixture component learning on weighted data, hence our mixture learning typically becomes highly efficient for popular base generative models like Gaussians or hidden Markov models. Moreover, apart from the expectation-maximization algorithm, the proposed recursive estimation has several advantages including the lack of need for a pre-determined mixture order and robustness to the choice of initial parameters. We demonstrate the benefits of the proposed approach on a comprehensive set of evaluations consisting of diverse time-series classification problems in semi-supervised scenarios.

A Spatial Analysis of Seismic Vulnerability of Buildings Using Statistical and Machine Learning Techniques Comparative Analysis (통계분석 기법과 머신러닝 기법의 비교분석을 통한 건물의 지진취약도 공간분석)

  • Seong H. Kim;Sang-Bin Kim;Dae-Hyeon Kim
    • Journal of Industrial Convergence
    • /
    • v.21 no.1
    • /
    • pp.159-165
    • /
    • 2023
  • While the frequency of seismic occurrence has been increasing recently, the domestic seismic response system is weak, the objective of this research is to compare and analyze the seismic vulnerability of buildings using statistical analysis and machine learning techniques. As the result of using statistical technique, the prediction accuracy of the developed model through the optimal scaling method showed about 87%. As the result of using machine learning technique, because the accuracy of Random Forest method is 94% in case of Train Set, 76.7% in case of Test Set, which is the highest accuracy among the 4 analyzed methods, Random Forest method was finally chosen. Therefore, Random Forest method was derived as the final machine learning technique. Accordingly, the statistical analysis technique showed higher accuracy of about 87%, whereas the machine learning technique showed the accuracy of about 76.7%. As the final result, among the 22,296 analyzed building data, the seismic vulnerabilities of 1,627(0.1%) buildings are expected as more dangerous when the statistical analysis technique is used, 10,146(49%) buildings showed the same rate, and the remaining 10,523(50%) buildings are expected as more dangerous when the machine learning technique is used. As the comparison of the results of using advanced machine learning techniques in addition to the existing statistical analysis techniques, in spatial analysis decisions, it is hoped that this research results help to prepare more reliable seismic countermeasures.

Structural Relations of Learning Orientation, Self-Efficacy, Learning Transfer and Job Performance of Farmers who Participated in the Strong and Small Farms Education (강소농교육 참여 농업인의 직무성과와 학습지향성, 자기효능감, 학습전이의 구조적 관계)

  • Kim, Sa-Gyun;Yang, Suk-Joon
    • Journal of Agricultural Extension & Community Development
    • /
    • v.22 no.4
    • /
    • pp.455-464
    • /
    • 2015
  • The purposes of this study are to explain and identify the frame of structural relations of learning orientation, self-efficacy, learning transfer and job performance of farmers who participated in the strong and small farms education. This is an experimental research with the data collected from 495 farmers who have taken the farm education. Based on the collected data, the study conducted a structural equation modeling(SEM) to confirm the validity and analyze the structural relations of the suggested model. Using measured and latent variables drew from the analyses, the study set a structural equation model and tested the model by analysis of the structural equation modeling with AMOS 18.0. The results found from the empirical analysis can be summarized as follows. 1) Learning orientation and self-efficacy positively influenced job performance through learning transfer. 2) The hypothesis that learning orientation would have direct impact on job performance was not supported. 3) The strong and small farms education is useful to expand learning transfer and to enhance job performance. So, government policy support has to reinforce learning support on farmers in order to achieve high performance of learning and job management through farm educations.

Performance Comparison of Reinforcement Learning Algorithms for Futures Scalping (해외선물 스캘핑을 위한 강화학습 알고리즘의 성능비교)

  • Jung, Deuk-Kyo;Lee, Se-Hun;Kang, Jae-Mo
    • The Journal of the Convergence on Culture Technology
    • /
    • v.8 no.5
    • /
    • pp.697-703
    • /
    • 2022
  • Due to the recent economic downturn caused by Covid-19 and the unstable international situation, many investors are choosing the derivatives market as a means of investment. However, the derivatives market has a greater risk than the stock market, and research on the market of market participants is insufficient. Recently, with the development of artificial intelligence, machine learning has been widely used in the derivatives market. In this paper, reinforcement learning, one of the machine learning techniques, is applied to analyze the scalping technique that trades futures in minutes. The data set consists of 21 attributes using the closing price, moving average line, and Bollinger band indicators of 1 minute and 3 minute data for 6 months by selecting 4 products among futures products traded at trading firm. In the experiment, DNN artificial neural network model and three reinforcement learning algorithms, namely, DQN (Deep Q-Network), A2C (Advantage Actor Critic), and A3C (Asynchronous A2C) were used, and they were trained and verified through learning data set and test data set. For scalping, the agent chooses one of the actions of buying and selling, and the ratio of the portfolio value according to the action result is rewarded. Experiment results show that the energy sector products such as Heating Oil and Crude Oil yield relatively high cumulative returns compared to the index sector products such as Mini Russell 2000 and Hang Seng Index.

Finding the best suited autoencoder for reducing model complexity

  • Ngoc, Kien Mai;Hwang, Myunggwon
    • Smart Media Journal
    • /
    • v.10 no.3
    • /
    • pp.9-22
    • /
    • 2021
  • Basically, machine learning models use input data to produce results. Sometimes, the input data is too complicated for the models to learn useful patterns. Therefore, feature engineering is a crucial data preprocessing step for constructing a proper feature set to improve the performance of such models. One of the most efficient methods for automating feature engineering is the autoencoder, which transforms the data from its original space into a latent space. However certain factors, including the datasets, the machine learning models, and the number of dimensions of the latent space (denoted by k), should be carefully considered when using the autoencoder. In this study, we design a framework to compare two data preprocessing approaches: with and without autoencoder and to observe the impact of these factors on autoencoder. We then conduct experiments using autoencoders with classifiers on popular datasets. The empirical results provide a perspective regarding the best suited autoencoder for these factors.