• Title/Summary/Keyword: Tabular Data Learning

Search Result 15, Processing Time 0.02 seconds

Multi-Variate Tabular Data Processing and Visualization Scheme for Machine Learning based Analysis: A Case Study using Titanic Dataset (기계 학습 기반 분석을 위한 다변량 정형 데이터 처리 및 시각화 방법: Titanic 데이터셋 적용 사례 연구)

  • Juhyoung Sung;Kiwon Kwon;Kyoungwon Park;Byoungchul Song
    • Journal of Internet Computing and Services
    • /
    • v.25 no.4
    • /
    • pp.121-130
    • /
    • 2024
  • As internet and communication technology (ICT) is improved exponentially, types and amount of available data also increase. Even though data analysis including statistics is significant to utilize this large amount of data, there are inevitable limits to process various and complex data in general way. Meanwhile, there are many attempts to apply machine learning (ML) in various fields to solve the problems according to the enhancement in computational performance and increase in demands for autonomous systems. Especially, data processing for the model input and designing the model to solve the objective function are critical to achieve the model performance. Data processing methods according to the type and property have been presented through many studies and the performance of ML highly varies depending on the methods. Nevertheless, there are difficulties in deciding which data processing method for data analysis since the types and characteristics of data have become more diverse. Specifically, multi-variate data processing is essential for solving non-linear problem based on ML. In this paper, we present a multi-variate tabular data processing scheme for ML-aided data analysis by using Titanic dataset from Kaggle including various kinds of data. We present the methods like input variable filtering applying statistical analysis and normalization according to the data property. In addition, we analyze the data structure using visualization. Lastly, we design an ML model and train the model by applying the proposed multi-variate data process. After that, we analyze the passenger's survival prediction performance of the trained model. We expect that the proposed multi-variate data processing and visualization can be extended to various environments for ML based analysis.

A novel window strategy for concept drift detection in seasonal time series (계절성 시계열 자료의 concept drift 탐지를 위한 새로운 창 전략)

  • Do Woon Lee;Sumin Bae;Kangsub Kim;Soonhong An
    • Annual Conference of KIPS
    • /
    • 2023.05a
    • /
    • pp.377-379
    • /
    • 2023
  • Concept drift detection on data stream is the major issue to maintain the performance of the machine learning model. Since the online stream is to be a function of time, the classical statistic methods are hard to apply. In particular case of seasonal time series, a novel window strategy with Fourier analysis however, gives a chance to adapt the classical methods on the series. We explore the KS-test for an adaptation of the periodic time series and show that this strategy handles a complicate time series as an ordinary tabular dataset. We verify that the detection with the strategy takes the second place in time delay and shows the best performance in false alarm rate and detection accuracy comparing to that of arbitrary window sizes.

Deep Interpretable Learning for a Rapid Response System (긴급대응 시스템을 위한 심층 해석 가능 학습)

  • Nguyen, Trong-Nghia;Vo, Thanh-Hung;Kho, Bo-Gun;Lee, Guee-Sang;Yang, Hyung-Jeong;Kim, Soo-Hyung
    • Annual Conference of KIPS
    • /
    • 2021.11a
    • /
    • pp.805-807
    • /
    • 2021
  • In-hospital cardiac arrest is a significant problem for medical systems. Although the traditional early warning systems have been widely applied, they still contain many drawbacks, such as the high false warning rate and low sensitivity. This paper proposed a strategy that involves a deep learning approach based on a novel interpretable deep tabular data learning architecture, named TabNet, for the Rapid Response System. This study has been processed and validated on a dataset collected from two hospitals of Chonnam National University, Korea, in over 10 years. The learning metrics used for the experiment are the area under the receiver operating characteristic curve score (AUROC) and the area under the precision-recall curve score (AUPRC). The experiment on a large real-time dataset shows that our method improves compared to other machine learning-based approaches.

A Comparative Analysis of Ensemble Learning-Based Classification Models for Explainable Term Deposit Subscription Forecasting (설명 가능한 정기예금 가입 여부 예측을 위한 앙상블 학습 기반 분류 모델들의 비교 분석)

  • Shin, Zian;Moon, Jihoon;Rho, Seungmin
    • The Journal of Society for e-Business Studies
    • /
    • v.26 no.3
    • /
    • pp.97-117
    • /
    • 2021
  • Predicting term deposit subscriptions is one of representative financial marketing in banks, and banks can build a prediction model using various customer information. In order to improve the classification accuracy for term deposit subscriptions, many studies have been conducted based on machine learning techniques. However, even if these models can achieve satisfactory performance, utilizing them is not an easy task in the industry when their decision-making process is not adequately explained. To address this issue, this paper proposes an explainable scheme for term deposit subscription forecasting. For this, we first construct several classification models using decision tree-based ensemble learning methods, which yield excellent performance in tabular data, such as random forest, gradient boosting machine (GBM), extreme gradient boosting (XGB), and light gradient boosting machine (LightGBM). We then analyze their classification performance in depth through 10-fold cross-validation. After that, we provide the rationale for interpreting the influence of customer information and the decision-making process by applying Shapley additive explanation (SHAP), an explainable artificial intelligence technique, to the best classification model. To verify the practicality and validity of our scheme, experiments were conducted with the bank marketing dataset provided by Kaggle; we applied the SHAP to the GBM and LightGBM models, respectively, according to different dataset configurations and then performed their analysis and visualization for explainable term deposit subscriptions.

Design and Implementation of IoT based Low cost, Effective Learning Mechanism for Empowering STEM Education in India

  • Simmi Chawla;Parul Tomar;Sapna Gambhir
    • International Journal of Computer Science & Network Security
    • /
    • v.24 no.4
    • /
    • pp.163-169
    • /
    • 2024
  • India is a developing nation and has come with comprehensive way in modernizing its reducing poverty, economy and rising living standards for an outsized fragment of its residents. The STEM (Science, Technology, Engineering, and Mathematics) education plays an important role in it. STEM is an educational curriculum that emphasis on the subjects of "science, technology, engineering, and mathematics". In traditional education scenario, these subjects are taught independently, but according to the educational philosophy of STEM that teaches these subjects together in project-based lessons. STEM helps the students in his holistic development. Youth unemployment is the biggest concern due to lack of adequate skills. There is a huge skill gap behind jobless engineers and the question arises how we can prepare engineers for a better tomorrow? Now a day's Industry 4.0 is a new fourth industrial revolution which is an intelligent networking of machines and processes for industry through ICT. It is based upon the usage of cyber-physical systems and Internet of Things (IoT). Industrial revolution does not influence only production but also educational system as well. IoT in academics is a new revolution to the Internet technology, which introduced "Smartness" in the entire IT infrastructure. To improve socio-economic status of the India students must equipped with 21st century digital skills and Universities, colleges must provide individual learning kits to their students which can help them in enhancing their productivity and learning outcomes. The major goal of this paper is to present a low cost, effective learning mechanism for STEM implementation using Raspberry Pi 3+ model (Single board computer) and Node Red open source visual programming tool which is developed by IBM for wiring hardware devices together. These tools are broadly used to provide hands on experience on IoT fundamentals during teaching and learning. This paper elaborates the appropriateness and the practicality of these concepts via an example by implementing a user interface (UI) and Dashboard in Node-RED where dashboard palette is used for demonstration with switch, slider, gauge and Raspberry pi palette is used to connect with GPIO pins present on Raspberry pi board. An LED light is connected with a GPIO pin as an output pin. In this experiment, it is shown that the Node-Red dashboard is accessing on Raspberry pi and via Smartphone as well. In the final step results are shown in an elaborate manner. Conversely, inadequate Programming skills in students are the biggest challenge because without good programming skills there would be no pioneers in engineering, robotics and other areas. Coding plays an important role to increase the level of knowledge on a wide scale and to encourage the interest of students in coding. Today Python language which is Open source and most demanding languages in the industry in order to know data science and algorithms, understanding computer science would not be possible without science, technology, engineering and math. In this paper a small experiment is also done with an LED light via writing source code in python. These tiny experiments are really helpful to encourage the students and give play way to learn these advance technologies. The cost estimation is presented in tabular form for per learning kit provided to the students for Hands on experiments. Some Popular In addition, some Open source tools for experimenting with IoT Technology are described. Students can enrich their knowledge by doing lots of experiments with these freely available software's and this low cost hardware in labs or learning kits provided to them.