• Title/Summary/Keyword: 비정형분석

Search Result 484, Processing Time 0.029 seconds

A Study on the Trends of Construction Safety Accident in Unstructured Text Using Topic Modeling (비정형 텍스트 기반의 토픽 모델링을 이용한 건설 안전사고 동향 분석)

  • Lee, Sang-Gyu
    • Journal of the Korea Academia-Industrial cooperation Society
    • /
    • v.19 no.10
    • /
    • pp.176-182
    • /
    • 2018
  • In order to understand and track the trends of construction safety accident, this study shows the topic trends in the construction safety accident with LDA(Latent Dirichlet Allocation)-based topic modeling method for data analytics. Especially, it performs to figure out the main issue of construction safety accident with unstructured data analysis based on the topic modeling rather than a variety of structured data analysis for preventing to safety accident in construction industry. To apply this methodology, I randomly collected to 540 news article data about construction accident from January 2017 to February 2018. Based on the unstructured data with the LDA-based topic modeling, I found the 10 topics and identified key issues through 10 keyword in each 10 topics. I forecasted the topic issue related to construction safety accident based on analysis of time-series trends about the news data from January 2017 to February 2018. With this method, this research gives a hint about ways of using unstructured news article data to anticipate safety policy and research field and to respond to construction accident safety issues in the future.

Spark-based Network Log Analysis Aystem for Detecting Network Attack Pattern Using Snort (Snort를 이용한 비정형 네트워크 공격패턴 탐지를 수행하는 Spark 기반 네트워크 로그 분석 시스템)

  • Baek, Na-Eun;Shin, Jae-Hwan;Chang, Jin-Su;Chang, Jae-Woo
    • The Journal of the Korea Contents Association
    • /
    • v.18 no.4
    • /
    • pp.48-59
    • /
    • 2018
  • Recently, network technology has been used in various fields due to development of network technology. However, there has been an increase in the number of attacks targeting public institutions and companies by exploiting the evolving network technology. Meanwhile, the existing network intrusion detection system takes much time to process logs as the amount of network log increases. Therefore, in this paper, we propose a Spark-based network log analysis system that detects unstructured network attack pattern. by using Snort. The proposed system extracts and analyzes the elements required for network attack pattern detection from large amount of network log data. For the analysis, we propose a rule to detect network attack patterns for Port Scanning, Host Scanning, DDoS, and worm activity, and can detect real attack pattern well by applying it to real log data. Finally, we show from our performance evaluation that the proposed Spark-based log analysis system is more than two times better on log data processing performance than the Hadoop-based system.

Analysis of the ESG Research Trend : Focusing on SCOPUS DB (ESG 주요 연구 동향 분석: SCOPUS DB를 중심으로)

  • Kyoo-Sung Noh
    • Journal of Digital Convergence
    • /
    • v.21 no.2
    • /
    • pp.9-16
    • /
    • 2023
  • The purpose of this study is to analyze research trends on ESG (Environmental, Social, and Governance), and to present a direction for companies and investors to use ESG information. To this end, text mining, one of the atypical data mining techniques, was used for analysis. Thesis abstracts from January 2014 to February 2023 were collected from the SCOPUS database, and Economics, Econometrics and Finance were the most common. The United States and China published the most ESG papers, and Korea published the 6th most papers in the world. This study is meaningful in that it analyzed the main research trends of ESG using text mining techniques such as LDA and topic modeling. It was confirmed that ESG is being conducted in various fields, not in a specific field, and it is differentiated from previous studies in that it analyzed various influencing factors and ripple effects of ESG.

State-of-the-Art of Diagrid Structural Systems (Diagrid 구조시스템의 기술 현황 분석)

  • Han, Kyung-Soo;Jung, In-Yong;Ju, Young-Kyu;Kim, Sang-Dae
    • 한국방재학회:학술대회논문집
    • /
    • 2010.02a
    • /
    • pp.36-36
    • /
    • 2010
  • 본 연구에서는 다이아그리드 구조시스템의 기술 현황 동향을 파악, 분석하고 향후발전과제를 제시하였다. 비정형 건물을 표현하기에 적합한 구조 시스템인 다이아그리드의 개념 및 원리를 설명하고 높이, 유형, 지역별로 건물 적용사례를 살펴보았다. 또한 국내의 연구 동향을 변수별로 정리하여 추가적인 연구개발 사항을 파악하고 이를 위한 구체적인 방안을 제시하였다.

  • PDF

The Design and Implementation of OWL Ontology Construction System through Information Extraction of Unstructured Documents (비정형 문서의 정보추출을 통한 OWL 온톨로지 구축 시스템의 설계 및 구현)

  • Jo, Dae Woong;Choi, Ji Woong;Kim, Myung Ho
    • Journal of the Korea Society of Computer and Information
    • /
    • v.19 no.10
    • /
    • pp.23-33
    • /
    • 2014
  • The development of the information retrieval field is evolving to the research field searching accurately for the information from thing finding rapidly a large amount of information. Personalization and the semantic web technology is a key technology. The automatic indexing technology about the web document and throughput go beyond the research stage and show up as the practical service. However, there is a lack of research on the document information retrieval field about the attached document type of except the web document. In this paper, we illustrate about the method in which it analyzed the text content of the unstructured documents prepared in the text, word, hwp form and it how to construction OWL ontology. To build TBox of the document ontology and the resources which can be obtained from the document is selected, and we implement with the system in order to utilize as the instant of the constructed document ontology. It is effectually usable in the information retrieval and document management system using the semantic technology of the correspondence document as the ontology automatic construction of this kind of the unstructured documents.

Topic Automatic Extraction Model based on Unstructured Security Intelligence Report (비정형 보안 인텔리전스 보고서 기반 토픽 자동 추출 모델)

  • Hur, YunA;Lee, Chanhee;Kim, Gyeongmin;Lim, HeuiSeok
    • Journal of the Korea Convergence Society
    • /
    • v.10 no.6
    • /
    • pp.33-39
    • /
    • 2019
  • As cyber attack methods are becoming more intelligent, incidents such as security breaches and international crimes are increasing. In order to predict and respond to these cyber attacks, the characteristics, methods, and types of attack techniques should be identified. To this end, many security companies are publishing security intelligence reports to quickly identify various attack patterns and prevent further damage. However, the reports that each company distributes are not structured, yet, the number of published intelligence reports are ever-increasing. In this paper, we propose a method to extract structured data from unstructured security intelligence reports. We also propose an automatic intelligence report analysis system that divides a large volume of reports into sub-groups based on their topics, making the report analysis process more effective and efficient.

Big Data Platform for Public Library Users: Focusing on the Cultural Programs and Community Service (이용자를 위한 공공도서관 빅데이터 플랫폼 구축 방안 연구 - 문화프로그램 및 커뮤니티 서비스 정보를 중심으로 -)

  • Yoon, SoYoung
    • Journal of the Korean BIBLIA Society for library and Information Science
    • /
    • v.33 no.3
    • /
    • pp.347-370
    • /
    • 2022
  • Most public library websites provide unstructured cultural program data, which cannot be produced and utilized systematically as bibliographic information. It is not sufficiently used in existing library big data research or cases, and there is a risk of disappearing when the website is reorganized or the person in charge is changed. This study developed a data schema that can be used in conjunction with bibliographic data by collecting and analyzing cultural programs and community service data produced in an unstructured manner and proposed to share and utilize public library cultural programs and community service data, and establish a library big data platform that can serve as an information channel between librarians who are cultural program planners. Library program data posted on the library website can be integrated and managed through the platform, securing continuity of work, and systematically managing and preserving the specialized service history of individual libraries.

Comparative Analysis of Forecasting Accuracy and Model Performance for Development of Coastal Wave Forecasting System Based on Unstructured Grid (비정형격자 기반 국지연안 파랑예측시스템 구축을 위한 예측정확도 및 모델성능 비교분석)

  • Min, Roh;Sang Myeong, Oh;Pil-Hun, Chang;Hyun-Suk, Kang;Hyung Suk, Kim
    • Journal of Korean Society of Coastal and Ocean Engineers
    • /
    • v.34 no.6
    • /
    • pp.188-197
    • /
    • 2022
  • We develop a coastal wave forecasting system by using the unstructured grid based on sea wind data of Global Data Assimilation and Prediction System. The verification is performed to examine the performance and accuracy of the wave model. Since the conventional grid has limited wave forecasting on complex coastlines and bathymetry, the unstructured grid system is applied for precise numerical simulation, and applicability for operational support is evaluated. Both grid systems show similar prediction trends in offshore and coastal areas, and the difference in prediction errors according to the grid system is not large. In addition, the applicability of the operational wave forecasting system is confirmed by dramatically reducing the model execution time of the unstructured grid under the same conditions.

Security Operation Implementation through Big Data Analysis by Using Open Source ELK Stack (오픈소스 ELK Stack 활용 정보보호 빅데이터 분석을 통한 보안관제 구현)

  • Hyun, Jeong-Hoon;Kim, Hyoung-Joong
    • Journal of Digital Contents Society
    • /
    • v.19 no.1
    • /
    • pp.181-191
    • /
    • 2018
  • With the development of IT, hacking crimes are becoming intelligent and refined. In Emergency response, Big data analysis in information security is to derive problems such as abnormal behavior through collecting, storing, analyzing and visualizing whole log including normal log generated from various information protection system. By using the full log data, including data we have been overlooked, we seek to detect and respond to the abnormal signs of the cyber attack from the early stage of the cyber attack. We used open-source ELK Stack technology to analyze big data like unstructured data that occur in information protection system, terminal and server. By using this technology, we can make it possible to build an information security control system that is optimized for the business environment with its own staff and technology. It is not necessary to rely on high-cost data analysis solution, and it is possible to accumulate technologies to defend from cyber attacks by implementing protection control system directly with its own manpower.

A Comparative Analysis of Cognitive Change about Big Data Using Social Media Data Analysis (소셜 미디어 데이터 분석을 활용한 빅데이터에 대한 인식 변화 비교 분석)

  • Yun, Youdong;Jo, Jaechoon;Hur, Yuna;Lim, Heuiseok
    • KIPS Transactions on Software and Data Engineering
    • /
    • v.6 no.7
    • /
    • pp.371-378
    • /
    • 2017
  • Recently, with the spread of smart device and the introduction of web services, the data is rapidly increasing online, and it is utilized in various fields. In particular, the emergence of social media in the big data field has led to a rapid increase in the amount of unstructured data. In order to extract meaningful information from such unstructured data, interest in big data technology has increased in various fields. Big data is becoming a key resource in many areas. Big data's prospects for the future are positive, but concerns about data breaches and privacy are constantly being addressed. On this subject of big data, where positive and negative views coexist, the research of analyzing people's opinions currently lack. In this study, we compared the changes in peoples perception on big data based on unstructured data collected from the social media using a text mining. As a results, yearly keywords for domestic big data, declining positive opinions, and increasing negative opinions were observed. Based on these results, we could predict the flow of domestic big data.