• Title/Summary/Keyword: 비정형분석

Search Result 484, Processing Time 0.032 seconds

A MVC Framework for Visualizing Text Data (텍스트 데이터 시각화를 위한 MVC 프레임워크)

  • Choi, Kwang Sun;Jeong, Kyo Sung;Kim, Soo Dong
    • Journal of Intelligence and Information Systems
    • /
    • v.20 no.2
    • /
    • pp.39-58
    • /
    • 2014
  • As the importance of big data and related technologies continues to grow in the industry, it has become highlighted to visualize results of processing and analyzing big data. Visualization of data delivers people effectiveness and clarity for understanding the result of analyzing. By the way, visualization has a role as the GUI (Graphical User Interface) that supports communications between people and analysis systems. Usually to make development and maintenance easier, these GUI parts should be loosely coupled from the parts of processing and analyzing data. And also to implement a loosely coupled architecture, it is necessary to adopt design patterns such as MVC (Model-View-Controller) which is designed for minimizing coupling between UI part and data processing part. On the other hand, big data can be classified as structured data and unstructured data. The visualization of structured data is relatively easy to unstructured data. For all that, as it has been spread out that the people utilize and analyze unstructured data, they usually develop the visualization system only for each project to overcome the limitation traditional visualization system for structured data. Furthermore, for text data which covers a huge part of unstructured data, visualization of data is more difficult. It results from the complexity of technology for analyzing text data as like linguistic analysis, text mining, social network analysis, and so on. And also those technologies are not standardized. This situation makes it more difficult to reuse the visualization system of a project to other projects. We assume that the reason is lack of commonality design of visualization system considering to expanse it to other system. In our research, we suggest a common information model for visualizing text data and propose a comprehensive and reusable framework, TexVizu, for visualizing text data. At first, we survey representative researches in text visualization era. And also we identify common elements for text visualization and common patterns among various cases of its. And then we review and analyze elements and patterns with three different viewpoints as structural viewpoint, interactive viewpoint, and semantic viewpoint. And then we design an integrated model of text data which represent elements for visualization. The structural viewpoint is for identifying structural element from various text documents as like title, author, body, and so on. The interactive viewpoint is for identifying the types of relations and interactions between text documents as like post, comment, reply and so on. The semantic viewpoint is for identifying semantic elements which extracted from analyzing text data linguistically and are represented as tags for classifying types of entity as like people, place or location, time, event and so on. After then we extract and choose common requirements for visualizing text data. The requirements are categorized as four types which are structure information, content information, relation information, trend information. Each type of requirements comprised with required visualization techniques, data and goal (what to know). These requirements are common and key requirement for design a framework which keep that a visualization system are loosely coupled from data processing or analyzing system. Finally we designed a common text visualization framework, TexVizu which is reusable and expansible for various visualization projects by collaborating with various Text Data Loader and Analytical Text Data Visualizer via common interfaces as like ITextDataLoader and IATDProvider. And also TexVisu is comprised with Analytical Text Data Model, Analytical Text Data Storage and Analytical Text Data Controller. In this framework, external components are the specifications of required interfaces for collaborating with this framework. As an experiment, we also adopt this framework into two text visualization systems as like a social opinion mining system and an online news analysis system.

Structuring of unstructured big data and visual interpretation (부산지역 교통관련 기사를 이용한 비정형 빅데이터의 정형화와 시각적 해석)

  • Lee, Kyeongjun;Noh, Yunhwan;Yoon, Sanggyeong;Cho, Youngseuk
    • Journal of the Korean Data and Information Science Society
    • /
    • v.25 no.6
    • /
    • pp.1431-1438
    • /
    • 2014
  • We analyzed the articles from "Kukje Shinmun" and "Busan Ilbo", which are two local newpapers of Busan Metropolitan City. The articles cover from January 1, 2013 to December 31, 2013. Meaningful pattern inherent in 2889 articles of which the title includes "Busan" and "Traffic" and related data was analyzed. Textmining method, which is a part of datamining, was used for the social network analysis (SNA). HDFS and MapReduce (from Hadoop ecosystem), which is open-source framework based on JAVA, were used with Linux environment (Uubntu-12.04LTS) for the construction of unstructured data and the storage, process and the analysis of big data. We implemented new algorithm that shows better visualization compared with the default one from R package, by providing the color and thickness based on the weight from each node and line connecting the nodes.

Differences and Multi-dimensionality of the Perception of Career Success among Korean Employees: A Topic Modeling Approach (기업근로자 경력성공 인식의 다차원성과 차이: 토픽모델링의 적용)

  • Lee, Jaeeun;Chae, Chungil
    • The Journal of the Korea Contents Association
    • /
    • v.19 no.6
    • /
    • pp.58-71
    • /
    • 2019
  • The purpose of this study is to explore the multi-dimensionality and the differences of the career success that is revealed by the employee's perception. In order to fulfill the research purpose, LDA topic modeling has applied to extract latent topics of career success from 126 Korean employees' open-end survey questionnaires. The extracted latent topics are social recognition, continuing service within an organization, expertise, financial rewards, and pursuing personal meaning. The occurrence probability of each topic was different by individual characteristics such as gender, education, position. Study findings showed there is multi-dimensionality in career success, and there are differences of topic occurrence probability by demographic characteristics. Additionally, this study showed how to apply the recently developed machine learning approach in order to reduce the researcher's bias by adapting the LDA topic modeling to the qualitative open-ended survey data.

An Automatically Extracting Formal Information from Unstructured Security Intelligence Report (비정형 Security Intelligence Report의 정형 정보 자동 추출)

  • Hur, Yuna;Lee, Chanhee;Kim, Gyeongmin;Jo, Jaechoon;Lim, Heuiseok
    • Journal of Digital Convergence
    • /
    • v.17 no.11
    • /
    • pp.233-240
    • /
    • 2019
  • In order to predict and respond to cyber attacks, a number of security companies quickly identify the methods, types and characteristics of attack techniques and are publishing Security Intelligence Reports(SIRs) on them. However, the SIRs distributed by each company are huge and unstructured. In this paper, we propose a framework that uses five analytic techniques to formulate a report and extract key information in order to reduce the time required to extract information on large unstructured SIRs efficiently. Since the SIRs data do not have the correct answer label, we propose four analysis techniques, Keyword Extraction, Topic Modeling, Summarization, and Document Similarity, through Unsupervised Learning. Finally, has built the data to extract threat information from SIRs, analysis applies to the Named Entity Recognition (NER) technology to recognize the words belonging to the IP, Domain/URL, Hash, Malware and determine if the word belongs to which type We propose a framework that applies a total of five analysis techniques, including technology.

A Study on Improvement of Pension Operation and Management using Big Data Analysis Techniques (빅데이터 분석기법을 활용한 숙박업체 운영 개선 방안에 대한 연구)

  • Yoon, Sunhee
    • The Journal of the Convergence on Culture Technology
    • /
    • v.7 no.4
    • /
    • pp.815-821
    • /
    • 2021
  • The advantage of big data is to collect a large amount of data on the Internet and refine and use valuable data. That is, the unstructured data is processed so that the user can analyze and utilize it from a necessary point of view. This paper is a relatively small project and is based on unstructured data that can be closely applied to real life and used for marketing. The subjects of the experiment were modeled on lodging companies in the Seoul metropolitan area an hour away from Seoul, and analyzed for the increase in lodging rates before and after marketing using big data. As an experiment that shows the effects of increasing sales, reducing costs, and increasing returns by users, we propose a system to determine and filter whether data input in the process of analyzing big data such as social networks can be used as accommodation-related information.

Detection and spread of agricultural drought warning based on news data (뉴스 데이터 기반 농업 가뭄 전조 감지 및 확산 분석)

  • Min-Jin Kim;Won-Ho Nam;Mi-Hye Yang;Ji-Wan Lee;Seong-Joon Kim
    • Proceedings of the Korea Water Resources Association Conference
    • /
    • 2023.05a
    • /
    • pp.339-339
    • /
    • 2023
  • 2018년도 연강수량은 1,386.9 mm로 평년 수준의 비가 내렸으나, 7-8월에 발생한 폭염 및 가뭄으로 인해 강원, 전남, 경북, 충남 지역에서 밭작물 중심으로 22,767 ha 피해가 발생하였다. 2019년도 연강수량은 1,171.9 mm로 평년보다 약 10% 적은 수준의 비가 내렸으며, 동일시기에 인천, 강원, 충남 지역에서 논·밭작물 중심으로 3,112 ha 피해가 발생하였다. 기후변화로 인해 강수량이 지역별로 편중되어 국지적 가뭄이 빈발하여 지역별로 가뭄의 영향, 피해, 체감 정도가 상이한 양상을 보이고 있다. 가뭄을 사전에 방지하기 위해서는 가뭄 모니터링과 선제적 대응이 중요하며, 현재 가뭄의 특징, 상황 등의 목적에 따라 다양한 형태의 가뭄 모니터링 시스템이 개발되고 있다. 가뭄의 피해와 영향 정도는 지역 간의 차이가 나타나 지역별로 가뭄을 체감하는 정도가 다르기 때문에 하나의 결과로 나타내기에는 어려움이 있다. 따라서, 가뭄의 발생 시기와 확산 정도를 분석하여 특보 발효 시점과 가뭄 대응 방안에 대한 기준 마련이 중요하다. 본 연구는 현대 사회에서 가장 중요한 요소로써 활용되고 있는 빅데이터 중 비정형 데이터로 분류되는 뉴스데이터와 논·밭의 가뭄을 나타내는 농업 가뭄과 관련된 데이터를 활용하여 농업 가뭄의 전조를 파악해보고자 한다. 2018년부터 2019년까지 전국을 대상으로 농업 가뭄에 관한 키워드 선정 후, 웹 크롤링을 통해 뉴스데이터를 수집해 논 가뭄과 밭 가뭄을 구별하여 각 지역에서 최초로 가뭄 기사가 발행된 시기와 지역별 기사의 빈도를 산출하였다. 뉴스데이터의 농업 가뭄 전조 감지 가능성을 파악하기 위해 국가가뭄정보통계집에서 논 물마름 시기와 밭 시듦 시기 자료를 활용하였다. 가뭄 피해 발생시기와 관련 기사 발행 시기 비교, 농업용수 대상 비상용수 지원 자료를 활용하여 실제 농업 가뭄확산 감지 여부를 분석하여 농업 가뭄 관련 뉴스데이터 간의 상관성을 분석하였다.

  • PDF

Correlation of Experimental ana Analytical Inelastic Responses of 1:12 Scale Irregular High-Rise RC Buildings (1:12축소 비정형 고층 RC 건물의 비선형거동에 대한 실험과 해석의 상관성)

  • Ko, Dong-Woo;Lee, Han-Seon
    • Journal of the Earthquake Engineering Society of Korea
    • /
    • v.11 no.2 s.54
    • /
    • pp.95-104
    • /
    • 2007
  • Three types of high-rise RC building structures having irregularity in the lower two stories were selected as prototypes and were performed nonlinear static analysis by using OpenSees to verify the analysis technique and to investigate the seismic capacity of those buildings. The first one has a symmetrical moment resisting frame (Model 1), the second has an infilled shear wall in the central frame (Model 2), and the third has an infilled shear wall only in one of exterior frames (Model 3). Fiber model, which consists of concrete and reinforcing bar represented from stress-strain relationship, is adapted used for simulate the nonlinearity of members, and MVLEM(Multi vertical linear element model) is used for simulate the behavior of wall. The analytical results are simulate the behavior of piloti stories well, for example, the stiffness and yield farce of piloti stories, the up-lift of wall and the variation of lateral stiffness of column due to the variation of axial forces. Overstrength of Model 2 and Model 3 are about 2 times larger than that of Model 1. The reason of the high oversttrength and ductility of Model 2 and Model 3 is that the conservative design of Model 2 and Model 3, whose beam and column sections are the same as those of Model 1. The ductilities of Model 1 and Model 3 are slightly larger than that of Model 1 and Model 3. Model 1 and Model 3 reached mechanism condition, whereas Model 2 failed to the shear failure of shear wall and the large axial forces in columns due to large overturning moment.

Implementation of CNN-based Classification Training Model for Unstructured Fashion Image Retrieval using Preprocessing with MASK R-CNN (비정형 패션 이미지 검색을 위한 MASK R-CNN 선형처리 기반 CNN 분류 학습모델 구현)

  • Seunga, Cho;Hayoung, Lee;Hyelim, Jang;Kyuri, Kim;Hyeon-Ji, Lee;Bong-Ki, Son;Jaeho, Lee
    • Journal of Korea Society of Industrial Information Systems
    • /
    • v.27 no.6
    • /
    • pp.13-23
    • /
    • 2022
  • In this paper, we propose a detailed component image classification algorithm by fashion item for unstructured data retrieval in the fashion field. Due to the COVID-19 environment, AI-based online shopping malls are increasing recently. However, there is a limit to accurate unstructured data search with existing keyword search and personalized style recommendations based on user surfing behavior. In this study, pre-processing using Mask R-CNN was conducted using images crawled from online shopping sites and then classified components for each fashion item through CNN. We obtain the accuaracy for collar of the shirt's as 93.28%, the pattern of the shirt as 98.10%, the 3 classese fit of the jeans as 91.73%, And, we further obtained one for the 4 classes fit of jeans as 81.59% and the color of the jeans as 93.91%. At the results for the decorated items, we also obtained the accuract of the washing of the jeans as 91.20% and the demage of jeans accuaracy as 92.96%.

A Study on Characteristics of the Visual Perception of Atypical and typical Space in Houses (비정형과 정형 주거공간의 시지각적 특성에 관한 연구)

  • Lee, Sang-Hwa
    • Journal of the Korean housing association
    • /
    • v.18 no.3
    • /
    • pp.81-89
    • /
    • 2007
  • This study is aimed at understanding objectively on the visual perception of atypical space and form in contemporary architecture. It is accomplished to compose the various form and the atypical space in contemporary architecture, which by means of applying digital technology to architecture is able to materialize complicated form and space. Therefore this the sis is needed to understand user's senses and affects in atypical space and form. Atypical space is able to interpret geometrically and analyze quantitatively into visual perception. This analysis is applied to Isovist used as quantitative analysis recently, and physical analysis of space used as quantitative analysis defined to vertical elements, which is analysed comparatively to visual perception in observer's point and characteristics in spatial elements. Examples are selected to four examples represented to typical and atypical space in contemporary houses. As examples are analysed to visual perception of spatial character, this study is intended to understand to spatial character in aspects of spatial composition and observer's visual perception. In results of perceptive analysis on four spatial examples, Atypical spaces are composed of changeable spatiality, which are composed changeably of the relational method with adjacent spaces. Also, typical spaces are composed monotonously of spatiality and relational method on control of the length in boundary plane and openness.

A Study on Inelastic Behavior of an Asymmetric Tall Building (비대칭 초고층건물의 비탄성거동에 관한 연구)

  • 윤태호;김진구;정명채
    • Journal of the Earthquake Engineering Society of Korea
    • /
    • v.1 no.3
    • /
    • pp.37-44
    • /
    • 1997
  • In this paper, the inelastic behavior of an asymmetric tall building is investigated. The asymmetry in rigidity caused by the structural asymmetry induces torsional as well as lateral deformation. The inelastic analysis of such an asymmetric structure is difficult to carry out with a planar model and thus requires a full three dimensional model. In this paper a 102 story unsymmetric tall building is analized by static push-over procedure and its behavior is investigated. The analysis are performed with and without floor rotation to compare the results. According to the results the static behavior of the model building, as expected, turned out to be dependent heavily an the asymmetry of the plan shapes of the building.

  • PDF