• Title/Summary/Keyword: 비정형로그

Search Result 15, Processing Time 0.017 seconds

Investigations on Techniques and Applications of Text Analytics (텍스트 분석 기술 및 활용 동향)

  • Kim, Namgyu;Lee, Donghoon;Choi, Hochang;Wong, William Xiu Shun
    • The Journal of Korean Institute of Communications and Information Sciences
    • /
    • v.42 no.2
    • /
    • pp.471-492
    • /
    • 2017
  • The demand and interest in big data analytics are increasing rapidly. The concepts around big data include not only existing structured data, but also various kinds of unstructured data such as text, images, videos, and logs. Among the various types of unstructured data, text data have gained particular attention because it is the most representative method to describe and deliver information. Text analysis is generally performed in the following order: document collection, parsing and filtering, structuring, frequency analysis, and similarity analysis. The results of the analysis can be displayed through word cloud, word network, topic modeling, document classification, and semantic analysis. Notably, there is an increasing demand to identify trending topics from the rapidly increasing text data generated through various social media. Thus, research on and applications of topic modeling have been actively carried out in various fields since topic modeling is able to extract the core topics from a huge amount of unstructured text documents and provide the document groups for each different topic. In this paper, we review the major techniques and research trends of text analysis. Further, we also introduce some cases of applications that solve the problems in various fields by using topic modeling.

A Study on the Data Collection Methods based Hadoop Distributed Environment (하둡 분산 환경 기반의 데이터 수집 기법 연구)

  • Jin, Go-Whan
    • Journal of the Korea Convergence Society
    • /
    • v.7 no.5
    • /
    • pp.1-6
    • /
    • 2016
  • Many studies have been carried out for the development of big data utilization and analysis technology recently. There is a tendency that government agencies and companies to introduce a Hadoop of a processing platform for analyzing big data is increasing gradually. Increased interest with respect to the processing and analysis of these big data collection technology of data has become a major issue in parallel to it. However, study of the collection technology as compared to the study of data analysis techniques, it is insignificant situation. Therefore, in this paper, to build on the Hadoop cluster is a big data analysis platform, through the Apache sqoop, stylized from relational databases, to collect the data. In addition, to provide a sensor through the Apache flume, a system to collect on the basis of the data file of the Web application, the non-structured data such as log files to stream. The collection of data through these convergence would be able to utilize as a basic material of big data analysis.

A Study on Big Data Anti-Money Laundering Systems Design through A Bank's Case Analysis (A 은행 사례 분석을 통한 빅데이터 기반 자금세탁방지 시스템 설계)

  • Kim, Sang-Wan;Hahm, Yu-Kun
    • The Journal of Bigdata
    • /
    • v.1 no.1
    • /
    • pp.85-94
    • /
    • 2016
  • Traditional Anti-Money Laundering (AML) software applications monitor bank customer transactions on a daily basis using customer historical information and account profile data to provide a "whole picture" to bank management. With the advent of Big Data, these applications could be benefited from size, variety, and speed of unstructured data, which have not been used in AML applications before. This study analyses the weaknesses of a bank's current AML systems and proposes an AML systems taking advantage of Big Data. For example, early warning of AML risk can be improved by exposing identities and uncovering hidden relationships through predictive and entity analytics on real-time and outside data such as SNS data.

  • PDF

A Tombstone Filtered LSM-Tree for Stable Performance of KVS (키밸류 저장소 성능 제어를 위한 삭제 키 분리 LSM-Tree)

  • Lee, Eunji
    • The Journal of the Institute of Internet, Broadcasting and Communication
    • /
    • v.22 no.4
    • /
    • pp.17-22
    • /
    • 2022
  • With the spread of web services, data types are becoming more diversified. In addition to the form of storing data such as images, videos, and texts, the number and form of properties and metadata expressing the data are different for each data. In order to efficiently process such unstructured data, a key-value store is widely used for state-of-the-art applications. LSM-Tree (Log Structured Merge Tree) is the core data structure of various commercial key-value stores. LSM-Tree is optimized to provide high performance for small writes by recording all write and delete operations in a log manner. However, there is a problem in that the delay time and processing speed of user requests are lowered as batches of deletion operations for expired data are inserted into the LSM-Tree as special key-value data. This paper presents a Filtered LSM-Tree (FLSM-Tree) that solves the above problem by separating the deleted key from the main tree structure while maintaining all the advantages of the existing LSM-Tree. The proposed method is implemented in LevelDB, a commercial key-value store and it shows that the read performance is improved by up to 47% in performance evaluation.

The Metabolic Effects of FGF21: From Physiology to Pharmacology (생리, 약학적 관점에서 fibroblast growth factor 21 (FGF21)의 대사 효과 고찰)

  • Song, Parkyong
    • Journal of Life Science
    • /
    • v.30 no.7
    • /
    • pp.640-650
    • /
    • 2020
  • Fibroblast growth factor 21 (FGF21) is an atypical member of the FGF protein family which is highly synthesized in the liver, pancreas, and adipose tissue. Depending on the expression tissue, FGF21 uses endo- or paracrine features to regulate several metabolic pathways including glucose metabolism and energy homeostasis. Different physiologically stressful conditions such as starvation, a ketogenic diet, extreme cold, and mitochondrial dysfunction are known to induce FGF21 synthesis in various tissues to exert either adaptive or defensive mechanisms. More specifically, peroxisome proliferator-activated receptor gamma and peroxisome proliferator-activated receptor alpha control FGF21 expression in adipose tissue and liver, respectively. In addition, the pharmacologic administration of FGF21 has been reported to decrease the body weight and improve the insulin sensitivity and lipoprotein profiles of obese mice and type 2 diabetes patients meaning that FGF21 has attracted huge interest as a therapeutic agent for type 2 diabetes, obesity, and non-alcoholic fatty liver disease. However, understanding FGF21 remains complicated due to the paradoxical condition of its tissue-dependent expression. For example, nutrient deprivation largely increases hepatic FGF21 levels whereas adipose tissue-derived FGF21 is increased under feeding condition. This review discusses the issues of interest that have arisen from existing publications, including the tissue-specific function of FGF21 and its action mechanism. We also summarize the current stage of a clinical trial using several FGF21 analogs.