• Title/Summary/Keyword: 비정형데이터분석

Search Result 405, Processing Time 0.032 seconds

A Design on a Streaming Big Data Processing System (스트리밍 빅데이터 처리 시스템 설계)

  • Kim, Sungsook;Kim, GyungTae;Park, Kiejin
    • Proceedings of the Korea Information Processing Society Conference
    • /
    • 2015.10a
    • /
    • pp.99-101
    • /
    • 2015
  • 현재 다양한 센서 기기에서 쏟아지는 대용량의 정형/비정형의 스트림 데이터의 경우 기존의 단일 스트리밍 처리 시스템 만으로 처리하기에는 한계가 있다. 클러스터의 디스크가 아닌 메모리들을 사용하여 대용량 데이터 처리를 할 수 있는 Spark 는 분산 처리 임에도 불구하고 강력한 데이터 일관성과 실시간성을 확보할 수 있는 플랫폼이다. 본 연구에서는 대용량 스트림 데이터 분석 시 발생하는 메모리 공간 부족과 실시간 병렬 처리 문제를 해결하고자, 클러스터의 메모리를 이용하여 대용량 데이터의 분산 처리와 스트림 실시간 처리를 동시에 할 수 있도록 구성하였다. 실험을 통하여, 기존 배치 처리 방식과 제안 시스템의 성능 차이를 확인 할 수 있었다.

Study of Trust Bigdata Platform (신뢰성 빅데이터 플렛폼의 연구)

  • Kim, Jeong-Joon;Kwak, Kwang-Jin;Lee, Don-Hee;Lee, Yong-Soo
    • The Journal of the Institute of Internet, Broadcasting and Communication
    • /
    • v.16 no.6
    • /
    • pp.225-230
    • /
    • 2016
  • Recently, Web has arisen large amount of data that to the development of the network and the Internet. In order to process it appeared that Big Data technology. Big Data technologies have been studied aiming a multifaceted and accurate analysis using existing regular data and a variety of data social data. But social data does not have the expertise and objectivity. And such manipulation and concealment and distortion of information have been raised troubling. Thus, this paper proposes for trust big data platform and will be described in detail. The big data platform proposed in this paper consists of data refiner, Data Analyzer, co-truster, visualizer, searcher, etc.

Application Method of Big-Data for Improvement for Construction Project Management System (빅 데이터 기반 건설사업정보시스템 기능 개선 방안 연구)

  • Kim, Jin-Uk;Kim, Young-Jin;Ok, Hyun;Yang, Sung-Hoon
    • Proceedings of the Korean Society of Computer Information Conference
    • /
    • 2015.07a
    • /
    • pp.301-303
    • /
    • 2015
  • 국내 건설행정 투명화 및 경쟁력 향상 목적으로 개발된 건설사업정보시스템에 정부와 운영주체는 다양한 기능개선 방안과 관련 연구를 수행하며 시스템 성능을 개선시켜왔다. 그러나 기 추진된 성능향상 방안이 공공업무 처리에 중점 되어 대국민 사용자를 위한 콘텐츠 및 기능 등의 서비스가 미흡한 상황이다. 이에 본 논문에서는 건설사업정보 건설인허가시스템의 도로점용장소별 허가현황 기능을 중심으로 빅 데이터를 이용한 허가현황 정보 제공 방안을 제안하였다. 제안한 기능개선 방안은 기 구축된 비정형 데이터를 빅 데이터 기반으로 재분석하여 구글 맵에 가시화함으로써 공공업무 데이터 처리 뿐만 아니라 대국민 서비스를 위한 콘텐츠 제공이 가능하도록 하였다. 뿐만 아니라 그동안 축적된 15TB이상의 건설관련 데이터의 재활용 가능성을 시사함으로써 시스템 활용성 증대 및 개편 방향에 도움이 될 것으로 판단된다.

  • PDF

A Study on the Polarity of Apartment Price News Using Big Data Analysis Method (빅데이터 분석기법을 활용한 아파트 가격 관련 뉴스 기사의 극성 분석)

  • Cho, Sang-Yeon;Hong, Eun-Pyo
    • Journal of Digital Convergence
    • /
    • v.17 no.9
    • /
    • pp.47-54
    • /
    • 2019
  • This study confirms the polarity of news articles on apartment prices using Opinion Mining which has widely been used for a big data analysis. The analyses were carried out utilizing internet news articles posted on the Naver for two years: 2012 and 2018. We proposed a sentiment analysis model and modeled a topic-oriented sentiment dictionary construction methods. As a result of analyzing the proposed sentiment analysis model, it was confirmed that there was a difference according to the tendency of the media companies in selecting social issues at the time of rising apartment prices. At the same time, we were able to find more affirmative articles in the media companies which share similar sentiment with the government in charge. In this paper, we proposed a sentiment analysis model that can be used in real estate field and analyzed the polarity of unformatted data related to real estate. In order to integrate them into various fields in the future, it is necessary to build the sentiment dictionaries by themes, as well as to collect various unformatted data over extended periods.

A Study on the Procedure of Using Big Data to Solve Smart City Problems Based on Citizens' Needs and Participation (시민 니즈와 참여 기반의 스마트시티 문제해결을 위한 빅 데이터 활용 절차에 관한 연구)

  • Chang, Hye-Jung
    • The Journal of Korea Institute of Information, Electronics, and Communication Technology
    • /
    • v.13 no.2
    • /
    • pp.102-112
    • /
    • 2020
  • Smart City's goal is to solve urban problems through smart city's component technology, thereby developing eco-friendly and sustainable economies and improving citizens' quality of life. Until now, smart cities have evolved into component technologies, but it is time to focus attention on the needs and participation of citizens in smart cities. In this paper, we present a big data procedure for solving smart city problems based on citizens' needs and participation. To this end, we examine the smart city project market by region and major industry. We also examine the development stages of the smart city market area by sector. Additionally it understands the definition and necessity of each sector for citizen participation, and proposes a method to solve the problem through big data in the seven-step big data problem solving process. The seven-step big data process for solving problems is a method of deriving tasks after analyzing structured and unstructured data in each sector of smart cities and deriving policy programs accordingly. To attract citizen participation in these procedures, the empathy stage of the design thinking methodology is used in the unstructured data collection process. Also, as a method of identifying citizens' needs to solve urban problems in smart cities, the problem definition stage of the design sinking methodology was incorporated into the unstructured data analysis process.

A Study on Recognition of Artificial Intelligence Utilizing Big Data Analysis (빅데이터 분석을 활용한 인공지능 인식에 관한 연구)

  • Nam, Soo-Tai;Kim, Do-Goan;Jin, Chan-Yong
    • Proceedings of the Korean Institute of Information and Commucation Sciences Conference
    • /
    • 2018.05a
    • /
    • pp.129-130
    • /
    • 2018
  • Big data analysis is a technique for effectively analyzing unstructured data such as the Internet, social network services, web documents generated in the mobile environment, e-mail, and social data, as well as well formed structured data in a database. The most big data analysis techniques are data mining, machine learning, natural language processing, and pattern recognition, which were used in existing statistics and computer science. Global research institutes have identified analysis of big data as the most noteworthy new technology since 2011. Therefore, companies in most industries are making efforts to create new value through the application of big data. In this study, we analyzed using the Social Matrics which a big data analysis tool of Daum communications. We analyzed public perceptions of "Artificial Intelligence" keyword, one month as of May 19, 2018. The results of the big data analysis are as follows. First, the 1st related search keyword of the keyword of the "Artificial Intelligence" has been found to be technology (4,122). This study suggests theoretical implications based on the results.

  • PDF

Big Data using Artificial Intelligence CNN on Unstructured Financial Data (비정형 금융 데이터에 관한 인공지능 CNN 활용 빅데이터 연구)

  • Ko, Young-Bong;Park, Dea-Woo
    • Proceedings of the Korean Institute of Information and Commucation Sciences Conference
    • /
    • 2022.05a
    • /
    • pp.232-234
    • /
    • 2022
  • Big data is widely used in customer relationship management, relationship marketing, financial business improvement, credit information and risk management. Moreover, as non-face-to-face financial transactions have become more active recently due to the COVID-19 virus, the use of financial big data is more demanded in terms of relationships with customers. In terms of customer relationship, financial big data has arrived at a time that requires an emotional rather than a technical approach. In relational marketing, it was necessary to emphasize the emotional aspect rather than the cognitive, rational, and rational aspects. Existing traditional financial data was collected and utilized through text-type customer transaction data, corporate financial information, and questionnaires. In this study, the customer's emotional image data, that is, atypical data based on the customer's cultural and leisure activities, is acquired through SNS and the customer's activity image is analyzed with an artificial intelligence CNN algorithm. Activity analysis is again applied to the annotated AI, and the AI big data model is designed to analyze the behavior model shown in the annotation.

  • PDF

An Insight Study on Keyword of IoT Utilizing Big Data Analysis (빅데이터 분석을 활용한 사물인터넷 키워드에 관한 조망)

  • Nam, Soo-Tai;Kim, Do-Goan;Jin, Chan-Yong
    • Proceedings of the Korean Institute of Information and Commucation Sciences Conference
    • /
    • 2017.10a
    • /
    • pp.146-147
    • /
    • 2017
  • Big data analysis is a technique for effectively analyzing unstructured data such as the Internet, social network services, web documents generated in the mobile environment, e-mail, and social data, as well as well formed structured data in a database. The most big data analysis techniques are data mining, machine learning, natural language processing, and pattern recognition, which were used in existing statistics and computer science. Global research institutes have identified analysis of big data as the most noteworthy new technology since 2011. Therefore, companies in most industries are making efforts to create new value through the application of big data. In this study, we analyzed using the Social Matrics which a big data analysis tool of Daum communications. We analyzed public perceptions of "Internet of things" keyword, one month as of october 8, 2017. The results of the big data analysis are as follows. First, the 1st related search keyword of the keyword of the "Internet of things" has been found to be technology (995). This study suggests theoretical implications based on the results.

  • PDF

A Meta Analysis of Innovation Diffusion Theory based on Behavioral Intention of Consumer (혁신확산이론 기반 소비자 행위의도에 관한 메타분석)

  • Nam, Soo-Tai;Kim, Do-Goan;Jin, Chan-Yong
    • Proceedings of the Korean Institute of Information and Commucation Sciences Conference
    • /
    • 2017.10a
    • /
    • pp.140-141
    • /
    • 2017
  • Big data analysis, in the large amount of data stored as the data warehouse which it refers the process of discovering meaningful new correlations, patterns, trends and creating new values. Thus, Big data analysis is an effective analysis of various big data that exist all over the world such as social big data, machine to machine (M2M) sensor data, and corporate customer relationship management data. In the big data era, it has become more important to effectively analyze not only structured data that is well organized in the database, but also unstructured big data such as the internet, social network services, and explosively generated web documents, e-mails, and social data in mobile environments. By the way, a meta analysis refers to a statistical literature synthesis method from the quantitative results of many known empirical studies. We reviewed a total of 750 samples among 50 studies published on the topic related as IDT between 2000 and 2017 in Korea.

  • PDF

Implications Deduction through Analysis of Reverse Engineering Process and Case Study for Prefabrication and Construction of Freeform Envelop Panels (비정형 건축물의 외장 패널의 선제작과 시공을 위한 역설계 프로세스와 사례 분석을 통한 시사점 도출)

  • Ryu, Han-Guk;Kim, Sung-Jin
    • Journal of the Korea Institute of Building Construction
    • /
    • v.16 no.6
    • /
    • pp.579-585
    • /
    • 2016
  • 3D laser scanning can be used for scanning the freeform surface and building a model from which the measurements could be taken, in order to solve the difficulty with getting access to the exact freeform shape and position data of the complex building envelope. The shape making process using 3D scanning is as follows: point cloud, mesh surface segmentation, NURBS(Non-Uniform Rational B-spline) surface generation, and parametric solid model generation. In this research, we review previous studies, reverse engineering notion, importance of reverse engineering usage for freeform envelope, and previous cases in order to identify the detail reverse engineering process for prefabrication and construction of freeform panels using 3D laser scanning technology. Therefore, the purpose of this research is to present a basic information which should be considered during design and construction phase and improve quality and constructibility of freeform building by analyzing the reverse engineering process and case study for prefabrication and construction of freeform panels using 3D laser scanning. The research results will enable 3D shape engineering and design parameterization using reverse engineering to be used in various construction projects.