• 제목/요약/키워드: Big data Problem

검색결과 577건 처리시간 0.03초

빅 데이터 기반의 상권 서비스 확장을 위한 설문조사시스템 설계 및 구현 (Design and Implementation of a Survey System for Expanding Big Data-Based Commercial District Service)

  • 이원철;강만수;김진호
    • 한국빅데이터학회지
    • /
    • 제5권2호
    • /
    • pp.171-186
    • /
    • 2020
  • 우리나라의 영세 소상공인과 자영업자의 비중이 주요 선진국에 비해 과도하게 높고 빈번한 창업과 폐업이 반복되어 국가 경제에 막대한 피해를 초래하고 있다. 이러한 문제를 해결하기 위해 소상공인을 위한 다양한 연구가 진행 중이며, 정부는 소상공인을 위해 빅 데이터를 이용한 상권정보 분석 서비스를 제공하고 있다. 상권정보 분석 서비스 중 서울시에서 운영하는 우리마을가게 상권분석서비스는 소상공인 관련 빅 데이터 분석 서비스를 제공하기 위해 지속적인 서비스 개선을 진행하고 있다. 그러나 다양한 기관에서 제공받은 빅 데이터를 통합하여 서비스를 구축하였기 때문에 데이터 신뢰성의 한계, 데이터 분석의 한계, 서비스 구성의 한계가 존재한다. 이러한 한계를 극복하기 위해 본 논문에서는 빅 데이터 기반의 상권 서비스와 연계 분석이 가능한 위치기반 설문조사시스템을 제안한다. 제안된 설문조사시스템은 설문정보와 상권정보를 연계하여 빅 데이터 상권 분석 서비스를 확장할 수 기반을 마련하였다.

금융 상품 추천에 관련된 빅 데이터 활용을 위한 개발 방법 (A study on development method for practical use of Big Data related to recommendation to financial item)

  • 김석수
    • 한국컴퓨터정보학회논문지
    • /
    • 제19권8호
    • /
    • pp.73-81
    • /
    • 2014
  • 본 연구에서는 활용 기술로 데이터 저장 레이어, 데이터 처리 레이어, 데이터 분석 레이어, 시각화 레이어 등의 빅 데이터 기술을 활용한 개발 방법을 제안한다, 각 단계에서 저장, 처리, 분석된 데이터는 시각화를 통하여 볼 수 있게 하였다. Hadoop을 통하여 데이터를 처리한 후 처리된 데이터를 Mahout으로 실행하여 분석 결과를 시각화 하였다. 이 과정을 통해서 금융 상품에 가입된 고객의 여러 특성을 파악하였고, 각 고객에 따른 금융 상품의 추천을 적시에 수행할 수 있었다. 본 연구에서는 빅 데이터의 배경 및 문제점을 소개하고, 빅 데이터가 새로운 비즈니스 기회를 어떻게 창출하는지 금융상품 추천 사례를 중심으로 개발 방법과 사례 연구를 논의한다.

도시 빅데이터를 활용한 스마트시티의 교통 예측 모델 - 환경 데이터와의 상관관계 기계 학습을 통한 예측 모델의 구축 및 검증 - (Big Data Based Urban Transportation Analysis for Smart Cities - Machine Learning Based Traffic Prediction by Using Urban Environment Data -)

  • 장선영;신동윤
    • 한국BIM학회 논문집
    • /
    • 제8권3호
    • /
    • pp.12-19
    • /
    • 2018
  • The research aims to find implications of machine learning and urban big data as a way to construct the flexible transportation network system of smart city by responding the urban context changes. This research deals with a problem that existing a bus headway model is difficult to respond urban situations in real-time. Therefore, utilizing the urban big data and machine learning prototyping tool in weathers, traffics, and bus statues, this research presents a flexible headway model to predict bus delay and analyze the result. The prototyping model is composed by real-time data of buses. The data is gathered through public data portals and real time Application Program Interface (API) by the government. These data are fundamental resources to organize interval pattern models of bus operations as traffic environment factors (road speeds, station conditions, weathers, and bus information of operating in real-time). The prototyping model is implemented by the machine learning tool (RapidMiner Studio) and conducted several tests for bus delays prediction according to specific circumstances. As a result, possibilities of transportation system are discussed for promoting the urban efficiency and the citizens' convenience by responding to urban conditions.

Feature Selection Using Submodular Approach for Financial Big Data

  • Attigeri, Girija;Manohara Pai, M.M.;Pai, Radhika M.
    • Journal of Information Processing Systems
    • /
    • 제15권6호
    • /
    • pp.1306-1325
    • /
    • 2019
  • As the world is moving towards digitization, data is generated from various sources at a faster rate. It is getting humungous and is termed as big data. The financial sector is one domain which needs to leverage the big data being generated to identify financial risks, fraudulent activities, and so on. The design of predictive models for such financial big data is imperative for maintaining the health of the country's economics. Financial data has many features such as transaction history, repayment data, purchase data, investment data, and so on. The main problem in predictive algorithm is finding the right subset of representative features from which the predictive model can be constructed for a particular task. This paper proposes a correlation-based method using submodular optimization for selecting the optimum number of features and thereby, reducing the dimensions of the data for faster and better prediction. The important proposition is that the optimal feature subset should contain features having high correlation with the class label, but should not correlate with each other in the subset. Experiments are conducted to understand the effect of the various subsets on different classification algorithms for loan data. The IBM Bluemix BigData platform is used for experimentation along with the Spark notebook. The results indicate that the proposed approach achieves considerable accuracy with optimal subsets in significantly less execution time. The algorithm is also compared with the existing feature selection and extraction algorithms.

R을 이용한 성경 데이터의 빈도와 소셜 네트워크 분석 (Frequency and Social Network Analysis of the Bible Data using Big Data Analytics Tools R)

  • 반재훈;하종수
    • 한국정보통신학회:학술대회논문집
    • /
    • 한국정보통신학회 2018년도 추계학술대회
    • /
    • pp.93-96
    • /
    • 2018
  • 데이터를 저장하고 분석하여 새로운 지식을 얻을 수 있는 빅데이터 처리기술은 사회의 여러 분야에서 중요성이 강조되고 있으며 정보통신기술 분야의 핵심 이슈로 부각되면서 관련 기술에 대한 관심이 증가하고 있다. 이러한 빅데이터를 분석할 수 있는 도구인 R은 통계 기반의 정보 분석을 가능하게 하는 언어와 환경이다. 본 논문에서는 이를 이용하여 성경데이터를 분석한다. R을 이용하여 어떠한 텍스트가 분포되어 있는지를 빈도 조사를 수행하며 소셜 네트워크 분석을 통해 성경을 분석한다.

  • PDF

공공의 빅데이터 활용을 위한 전자정부 역할 연구 (A research paper for e-government's role for public Big Data application)

  • 배용근;조영주;정영철
    • 한국정보통신학회논문지
    • /
    • 제21권11호
    • /
    • pp.2176-2183
    • /
    • 2017
  • 4차 산업혁명의 주요 요소가 되는 빅데이터 가치는 민간부분에서 산업 생산성을 높이고, 공공부분에서 대국민 및 기업에 대한 행정 서비스를 제공해 줄 수 있는 부분이기도 하다. ICT 선진국들은 공공부분의 빅데이터 활용 방안을 빠르게 제시하고 있다. 특히 사회 위기관리 차원에 있어 재난의 사전 예측시스템을 잘 갖추고 있다. 우리나라 정부의 입장에서도 사회 위기관리 차원의 빅데이터 공공 활용 방안에 많은 관심을 기울이고 있다. 하지만 빅데이터의 전반적인 인프라 부분에 취약성을 드러내고 있는 현실은 앞으로 사회현안 문제해결 차원의 준비와 실천이 요구되는 사항이다. 따라서 우리는 빅데이터 활용 현상의 문제를 분석하고, 각국의 선도적 빅데이터 공공 활용이 선행되는 사례를 검토해 앞으로 나아가야 할 정책의 다양성을 제시하여야 한다. 이에 본 논문은 빅데이터 활용에 있어 나타나고 있는 문제점을 분석하여 전자정부의 역할과 정책을 제언하였다. 제시한 정책 사항은 정보개방과 법 제도 개선의 문제, 빅데이터 환경에서의 개인정보 침해 위협을 관리하는 빅데이터 서비스 고려 사항 문제, 기술적 측면에서 공공의 빅데이터 활용 관련 기술개발 및 빅데이터 운영 분석 기술개발 필요성 문제 등을 제시하였다.

Travel Route Recommendation Utilizing Social Big Data

  • Yu, Yang Woo;Kim, Seong Hyuck;Kim, Hyeon Gyu
    • 한국컴퓨터정보학회논문지
    • /
    • 제27권5호
    • /
    • pp.117-125
    • /
    • 2022
  • 최근 여행에 대한 관심이 높아지면서, 번거로운 여행 일정을 대신 수립해주는 여행 일정 추천 서비스에 대한 연구가 활발히 진행되고 있다. 여행 일정 추천에 있어 가장 중요하면서도 공통적으로 제시되는 목표는 여행 목적지 근처의 인기 관광지를 포함한 최단 거리 여행 경로를 제공하는 것이다. 다수의 기존 연구에서는 개인 맞춤형 스케줄 제공에 초점을 맞추었으며, 사용자의 여행 이동 경로 이력이나 SNS 리뷰가 존재하지 않을 경우 설문 조사가 필요한 문제점이 있었다. 또한 최단 거리를 계산할 때 발생할 수 있는 현실적인 문제점도 명확히 지적되지 않았다. 이와 관련하여, 본 논문에서는 소셜 빅데이터를 활용하여 인기 관광지를 알아내기 위한 정량화된 방법을 소개하고, 최단 거리 알고리즘 적용시 발생할 수 있는 문제점과 이를 해결하기 위한 휴리스틱 알고리즘을 함께 제시한다. 제안 방법을 검증하기 위해, 경상남도를 대상으로 63,000여 개의 플레이스 정보를 수집하고 빅데이터 분석을 수행했으며, 실험을 통해 제안한 휴리스틱 스케줄링 알고리즘이 실제 데이터 상에서 실시간 처리가 가능함을 확인하였다.

빅데이터 도입의도에 미치는 영향요인에 관한 연구: 전략적 가치인식과 TOE(Technology Organizational Environment) Framework을 중심으로 (An Empirical Study on the Influencing Factors for Big Data Intented Adoption: Focusing on the Strategic Value Recognition and TOE Framework)

  • 가회광;김진수
    • Asia pacific journal of information systems
    • /
    • 제24권4호
    • /
    • pp.443-472
    • /
    • 2014
  • To survive in the global competitive environment, enterprise should be able to solve various problems and find the optimal solution effectively. The big-data is being perceived as a tool for solving enterprise problems effectively and improve competitiveness with its' various problem solving and advanced predictive capabilities. Due to its remarkable performance, the implementation of big data systems has been increased through many enterprises around the world. Currently the big-data is called the 'crude oil' of the 21st century and is expected to provide competitive superiority. The reason why the big data is in the limelight is because while the conventional IT technology has been falling behind much in its possibility level, the big data has gone beyond the technological possibility and has the advantage of being utilized to create new values such as business optimization and new business creation through analysis of big data. Since the big data has been introduced too hastily without considering the strategic value deduction and achievement obtained through the big data, however, there are difficulties in the strategic value deduction and data utilization that can be gained through big data. According to the survey result of 1,800 IT professionals from 18 countries world wide, the percentage of the corporation where the big data is being utilized well was only 28%, and many of them responded that they are having difficulties in strategic value deduction and operation through big data. The strategic value should be deducted and environment phases like corporate internal and external related regulations and systems should be considered in order to introduce big data, but these factors were not well being reflected. The cause of the failure turned out to be that the big data was introduced by way of the IT trend and surrounding environment, but it was introduced hastily in the situation where the introduction condition was not well arranged. The strategic value which can be obtained through big data should be clearly comprehended and systematic environment analysis is very important about applicability in order to introduce successful big data, but since the corporations are considering only partial achievements and technological phases that can be obtained through big data, the successful introduction is not being made. Previous study shows that most of big data researches are focused on big data concept, cases, and practical suggestions without empirical study. The purpose of this study is provide the theoretically and practically useful implementation framework and strategies of big data systems with conducting comprehensive literature review, finding influencing factors for successful big data systems implementation, and analysing empirical models. To do this, the elements which can affect the introduction intention of big data were deducted by reviewing the information system's successful factors, strategic value perception factors, considering factors for the information system introduction environment and big data related literature in order to comprehend the effect factors when the corporations introduce big data and structured questionnaire was developed. After that, the questionnaire and the statistical analysis were performed with the people in charge of the big data inside the corporations as objects. According to the statistical analysis, it was shown that the strategic value perception factor and the inside-industry environmental factors affected positively the introduction intention of big data. The theoretical, practical and political implications deducted from the study result is as follows. The frist theoretical implication is that this study has proposed theoretically effect factors which affect the introduction intention of big data by reviewing the strategic value perception and environmental factors and big data related precedent studies and proposed the variables and measurement items which were analyzed empirically and verified. This study has meaning in that it has measured the influence of each variable on the introduction intention by verifying the relationship between the independent variables and the dependent variables through structural equation model. Second, this study has defined the independent variable(strategic value perception, environment), dependent variable(introduction intention) and regulatory variable(type of business and corporate size) about big data introduction intention and has arranged theoretical base in studying big data related field empirically afterwards by developing measurement items which has obtained credibility and validity. Third, by verifying the strategic value perception factors and the significance about environmental factors proposed in the conventional precedent studies, this study will be able to give aid to the afterwards empirical study about effect factors on big data introduction. The operational implications are as follows. First, this study has arranged the empirical study base about big data field by investigating the cause and effect relationship about the influence of the strategic value perception factor and environmental factor on the introduction intention and proposing the measurement items which has obtained the justice, credibility and validity etc. Second, this study has proposed the study result that the strategic value perception factor affects positively the big data introduction intention and it has meaning in that the importance of the strategic value perception has been presented. Third, the study has proposed that the corporation which introduces big data should consider the big data introduction through precise analysis about industry's internal environment. Fourth, this study has proposed the point that the size and type of business of the corresponding corporation should be considered in introducing the big data by presenting the difference of the effect factors of big data introduction depending on the size and type of business of the corporation. The political implications are as follows. First, variety of utilization of big data is needed. The strategic value that big data has can be accessed in various ways in the product, service field, productivity field, decision making field etc and can be utilized in all the business fields based on that, but the parts that main domestic corporations are considering are limited to some parts of the products and service fields. Accordingly, in introducing big data, reviewing the phase about utilization in detail and design the big data system in a form which can maximize the utilization rate will be necessary. Second, the study is proposing the burden of the cost of the system introduction, difficulty in utilization in the system and lack of credibility in the supply corporations etc in the big data introduction phase by corporations. Since the world IT corporations are predominating the big data market, the big data introduction of domestic corporations can not but to be dependent on the foreign corporations. When considering that fact, that our country does not have global IT corporations even though it is world powerful IT country, the big data can be thought to be the chance to rear world level corporations. Accordingly, the government shall need to rear star corporations through active political support. Third, the corporations' internal and external professional manpower for the big data introduction and operation lacks. Big data is a system where how valuable data can be deducted utilizing data is more important than the system construction itself. For this, talent who are equipped with academic knowledge and experience in various fields like IT, statistics, strategy and management etc and manpower training should be implemented through systematic education for these talents. This study has arranged theoretical base for empirical studies about big data related fields by comprehending the main variables which affect the big data introduction intention and verifying them and is expected to be able to propose useful guidelines for the corporations and policy developers who are considering big data implementationby analyzing empirically that theoretical base.

빅데이터 기반의 실시간 네트워크 트래픽 분석 플랫폼 설계 (On the Design of a Big Data based Real-Time Network Traffic Analysis Platform)

  • 이동환;박정찬;유찬곤;윤호상
    • 정보보호학회논문지
    • /
    • 제23권4호
    • /
    • pp.721-728
    • /
    • 2013
  • 빅데이터는 오늘날 가장 각광받고 있는 데이터 수집 및 분석기술의 경향으로, 대량의 비정형 데이터 분석을 요구하는 다양한 분야에 접목되어 효용성을 인정받고 있다. 네트워크 트래픽 분석 역시 대량의 비정형 데이터를 다루는 분야로, 빅데이터 접목시 그 효과가 극대화될 수 있다. 따라서 본 논문에서는 고도의 보안이 요구되는 군 C4I망과 같은 내부망 환경의 침해사고 및 이상행위를 실시간으로 탐지하기 위한 빅데이터 기반의 네트워크 트래픽 분석 플랫폼(RENTAP)을 소개한다. 빅데이터 분석 지원을 위해 최근 각광받고 있는 오픈소스 솔루션들을 대상으로 비교 분석을 수행하였으며, 선정된 솔루션을 기반으로 고안된 최종 설계에 대해서 설명한다.

A Public Perception Study on the new word "Corona Blue":Focusing on Social Media Big Data Analysis

  • Ann, Myung Suk
    • International Journal of Advanced Culture Technology
    • /
    • 제8권3호
    • /
    • pp.133-139
    • /
    • 2020
  • The purpose of this study is to contribute to the provision of basic data for psychological quarantine policy and counseling by examining the public perception of the "corona blue" phenomenon through analysis of social media big data. To do this, key words related to the word 'Corona Blue' were derived and analyzed using the big data analysis program 'Textom'. As a result of the analysis, words such as 'Corona 19', 'depression', 'problem' and 'overcome' were derived as key words. For the analysis results,"pride and awarenes as the public perception of Corona 19", "depression and anxiety as a group trauma as the corona blue phenomenon", "spreading a psychological quarantine culture and demanding social healing as the perception of overcoming corona Blue," and "hope for return to daily life and changes in daily life as the perception of post corona" were discussed. In conclusion, we have identified the need for active psychological support from the community By revealing that Corona Blue is a depression as a group trauma. At this time, it is confirmed that it is necessary to prioritize social healing and psychological quarantine for the main risk groups such as youth or the vulnerable, who are the socially weak.