• Title/Summary/Keyword: big data system

Search Result 2,053, Processing Time 0.026 seconds

Study on Educational Utilization Methods of Big Data (빅데이터의 교육적 활용 방안 연구)

  • Lee, Youngseok;Cho, Jungwon
    • Journal of the Korea Academia-Industrial cooperation Society
    • /
    • v.17 no.12
    • /
    • pp.716-722
    • /
    • 2016
  • In the recent rapidly changing IT environment, the amount of smart digital data is growing exponentially. As a result, in many areas, utilizing big data research and development services and related technologies is becoming more popular. In SMART learning, big data is used by students, teachers, parents, etc., from a perspective of the potential for many. In this paper, we describe big data and can utilize it to identify scenarios. Big data, obtained through customized learning services that can take advantage of the scheme, is proposed. To analyze educational big data processing technology for this purpose, we designed a system for big data processing. Education services offer the measures necessary to take advantage of educational big data. These measures were implemented on a test platform that operates in a cloud-based operations section for a pilot training program that can be applied properly. Teachers try using it directly, and in the interest of business and education, a survey was conducted based on enjoyment, the tools, and users' feelings (e.g., tense, worried, confident). We analyzed the results to lay the groundwork for educational use of big data.

Big data-based piping material analysis framework in offshore structure for contract design

  • Oh, Min-Jae;Roh, Myung-Il;Park, Sung-Woo;Chun, Do-Hyun;Myung, Sehyun
    • Ocean Systems Engineering
    • /
    • v.9 no.1
    • /
    • pp.79-95
    • /
    • 2019
  • The material analysis of an offshore structure is generally conducted in the contract design phase for the price quotation of a new offshore project. This analysis is conducted manually by an engineer, which is time-consuming and can lead to inaccurate results, because the data size from previous projects is too large, and there are so many materials to consider. In this study, the piping materials in an offshore structure are analyzed for contract design using a big data framework. The big data technologies used include HDFS (Hadoop Distributed File System) for data saving, Hive and HBase for the database to handle the saved data, Spark and Kylin for data processing, and Zeppelin for user interface and visualization. The analyzed results show that the proposed big data framework can reduce the efforts put toward contract design in the estimation of the piping material cost.

Big Data based Epidemic Investigation Support System using Mobile Network Data (이동통신 데이터를 활용한 빅데이터 기반 역학조사지원 시스템)

  • Lee, Min-woo;Kim, Ye-ji;Yi, Jae-jin;Moon, Kyu-hwan;Hwang, SeonBae;Jun, Yong-joo;Hahm, Yu-Kun
    • The Journal of Bigdata
    • /
    • v.5 no.2
    • /
    • pp.187-199
    • /
    • 2020
  • The World Health Organization declared COVID-19 a pandemic on March 11. South Korea recorded 27,000 cases of the coronavirus illness, and more than 50 million coronavirus cases were confirmed all over the world. An epidemiological investigation becomes important once again due to the spread of COVID-19 infections. However, there were a number of confirmed coronavirus cases from Deagu and Gyeongbuk. Limitations of the epidemiological investigation methods were recognized. The Korea Disease Control and Prevention Agency developed the Epidemiological Investigation Support System(EISS) to utilize the smart city data hub technology and utilized the system in the epidemiological investigation. As a part of EISS, The proposed system is big-data bsed epidemiological investigation support system processing mobile network data. The established system is the epidemiological investigation support system based on big data to process mobile carriers' big data. Processing abnormal values of mobile carriers' data which was impossible with existing staff or creating hotspot regions where more than two people were in contact with an infected person were realized. As a result, our system processes outlier of mobile network data in 30 seconds, while processes hotspot around in 10 minutes. as a first time to adapt and support bigdata system into epidemiological investigation, our system proposes the practical utilizability of big-data system into epidemiological investigation.

Big data platform for health monitoring systems of multiple bridges

  • Wang, Manya;Ding, Youliang;Wan, Chunfeng;Zhao, Hanwei
    • Structural Monitoring and Maintenance
    • /
    • v.7 no.4
    • /
    • pp.345-365
    • /
    • 2020
  • At present, many machine leaning and data mining methods are used for analyzing and predicting structural response characteristics. However, the platform that combines big data analysis methods with online and offline analysis modules has not been used in actual projects. This work is dedicated to developing a multifunctional Hadoop-Spark big data platform for bridges to monitor and evaluate the serviceability based on structural health monitoring system. It realizes rapid processing, analysis and storage of collected health monitoring data. The platform contains offline computing and online analysis modules, using Hadoop-Spark environment. Hadoop provides the overall framework and storage subsystem for big data platform, while Spark is used for online computing. Finally, the big data Hadoop-Spark platform computational performance is verified through several actual analysis tasks. Experiments show the Hadoop-Spark big data platform has good fault tolerance, scalability and online analysis performance. It can meet the daily analysis requirements of 5s/time for one bridge and 40s/time for 100 bridges.

Evaluation of the Relationship between Meteorological, Agricultural and In-situ Big Data Droughts (기상학적 가뭄, 농업 가뭄 및 빅데이터 현장가뭄간의 상관성 평가)

  • LEE, Ji-Wan;JANG, Sun-Sook;AHN, So-Ra;PARK, Ki-Wook;KIM, Seong-Joon
    • Journal of the Korean Association of Geographic Information Studies
    • /
    • v.19 no.1
    • /
    • pp.64-79
    • /
    • 2016
  • The purpose of this study is to find the relationship between precipitation deficit, SPI(standardized precipitation index)-12 month, agricultural reservoir water storage deficit and agricultural drought-related big data, and to evaluate the usefulness of agricultural risk management through big data. For the long term drought (from January 2014 to September 2015), each data was collected and analysed with monthly and Provincial base. The minimum SPI-12 and maximum reservoir water storage deficit compared to normal year were occurred at the same time of July 2014, and August and September 2015. The maximum frequency of big data was occurred at June and July of 2014, and March and June to September of 2015. The maximum big data was occurred 1 month advanced in 2014 and 2 months advanced in 2015 than the maximum reservoir water storage deficit. The occurrence of big data was sensitive to spring drought from March, late Jangma of June, dry Jangma of July and the rainfall deficit of September 2015. The big data was closely related with the meteorological drought and agricultural drought. Because the big data is the in situ feeling drought, it is proved as a useful indicator for agricultural risk management.

Analyzing Smart Grid Energy Data using Hadoop Based Big Data System (하둡기반 빅데이터 시스템을 이용한 스마트그리드 전력데이터 분석)

  • Cho, YoungTak;Lee, WonJin;Lee, Ingyu;On, Byung-Won;Choi, Jung-In
    • The Transactions of the Korean Institute of Electrical Engineers P
    • /
    • v.64 no.2
    • /
    • pp.85-91
    • /
    • 2015
  • With the increasing popularity of Smart Grid infrastructure, it is much easier to collect energy usage data using AMI (Advanced Measuring Instrument) from residential housing, buildings and factories. Several researches have been done to improve an energy efficiency by analyzing the collected energy usage data. However, it is not easy to store and analyze the energy data using a traditional relational database management system since the data size grows exponentially with an increasing popularity of Smart grid infrastructure. In this paper, we are proposing a Hadoop based Big data system to store and analyze energy usage data. Based on our limited experiments, Hadoop based energy data analysis is three times faster than that of a relational database management system based approach with the current system.

A Context-Awareness Modeling User Profile Construction Method for Personalized Information Retrieval System

  • Kim, Jee Hyun;Gao, Qian;Cho, Young Im
    • International Journal of Fuzzy Logic and Intelligent Systems
    • /
    • v.14 no.2
    • /
    • pp.122-129
    • /
    • 2014
  • Effective information gathering and retrieval of the most relevant web documents on the topic of interest is difficult due to the large amount of information that exists in various formats. Current information gathering and retrieval techniques are unable to exploit semantic knowledge within documents in the "big data" environment; therefore, they cannot provide precise answers to specific questions. Existing commercial big data analytic platforms are restricted to a single data type; moreover, different big data analytic platforms are effective at processing different data types. Therefore, the development of a common big data platform that is suitable for efficiently processing various data types is needed. Furthermore, users often possess more than one intelligent device. It is therefore important to find an efficient preference profile construction approach to record the user context and personalized applications. In this way, user needs can be tailored according to the user's dynamic interests by tracking all devices owned by the user.

A Study on implementation model for security log analysis system using Big Data platform (빅데이터 플랫폼을 이용한 보안로그 분석 시스템 구현 모델 연구)

  • Han, Ki-Hyoung;Jeong, Hyung-Jong;Lee, Doog-Sik;Chae, Myung-Hui;Yoon, Cheol-Hee;Noh, Kyoo-Sung
    • Journal of Digital Convergence
    • /
    • v.12 no.8
    • /
    • pp.351-359
    • /
    • 2014
  • The log data generated by security equipment have been synthetically analyzed on the ESM(Enterprise Security Management) base so far, but due to its limitations of the capacity and processing performance, it is not suited for big data processing. Therefore the another way of technology on the big data platform is necessary. Big Data platform can achieve a large amount of data collection, storage, processing, retrieval, analysis, and visualization by using Hadoop Ecosystem. Currently ESM technology has developed in the way of SIEM (Security Information & Event Management) technology, and to implement security technology in SIEM way, Big Data platform technology is essential that can handle large log data which occurs in the current security devices. In this paper, we have a big data platform Hadoop Ecosystem technology for analyzing the security log for sure how to implement the system model is studied.

A Basic Study of Construction Procurement Management using Block Chain & Big Data Technology (블록체인과 빅데이터 기술을 이용한 건설 조달 관리 기초 연구)

  • Kim, Ki-Ho;Son, Seung-Hyun;Kim, Sun-Kuk
    • Proceedings of the Korean Institute of Building Construction Conference
    • /
    • 2019.05a
    • /
    • pp.127-128
    • /
    • 2019
  • The cost of procurement in construction projects varies depending on the type of project, but the proportion of the cost component is very large. Therefore, efficient procurement system operation affects the success of the project. However, difficulties arise in procurement management due to problems such as availability of data due to limited information, inaccurate scheduling and integration of costs. Therefore, the purpose of this study is to develop a procurement management system to help efficient communication decision by combining Big Data which can analyze a lot of information and Block Chain technology which can secure information and record reliability to satisfy the above requirements. The results of this study are used to develop a system to develop academically improved procurement management system and practically to develop a system to secure business competitiveness and to facilitate rapid communication among project participants.

  • PDF

A Study on the Ferry Sewol Disaster Cause and Marine Disaster Prevention Informatization with Big Data : In terms of ICT Administrative Spatial Informatization and Maritime Disaster Prevention System development (세월호사고원인과 빅데이터 해양방재정보화연구 -ICT행정공간정보화와 해양방재시스템개발 측면에서-)

  • Lee, Sang-Yun;Yoon, Hong-Joo
    • The Journal of the Korea institute of electronic communication sciences
    • /
    • v.11 no.6
    • /
    • pp.567-580
    • /
    • 2016
  • In recent years, our society, because of the arrival of a new paradigm according to the rapid changes in ICT has entered into future smart society and the ubiquitous era. So it can be a notable turning point in the marine disaster prevention system with big data, aspects of the era change. Therefore, this study was to derive a desirable vision for the big data marine disaster prevention informatization in terms of ICT maritime disaster prevention system development as preparedness for the maritime disaster by applying 'scenario planning' as a foresight method. Soon this study derived a successful marine disaster prevention informatization strategy as preparedness for the maritime disaster like Ferry Sewol Disaster. It proposed the big data marine disaster prevention informatization system with the use of the administrative aspects of information with spatial informatization as big data information. Also this study explored the future leadership strategy of the big data marine disaster prevention informatization in smart society. Eventually in 2030 to around, In order to still remain our marine disaster prevention informatization as a leading ICT nation, this study suggested the following strategy. It is important to ready the advanced Big Data administrative spatial informatization system In terms of prevention of incidents like Ferry Sewol Disaster.