• Journal of Digital Contents Society
• /
• v.18 no.1
• /
• pp.149-158
• /
• 2017

Big data have been presenting us with exciting opportunities and challenges in economic development. For instance, in the agriculture sector, mixing up of various agricultural data (e.g., weather data, soil data, etc.), and subsequently analyzing these data deliver valuable and helpful information to farmers and agribusinesses. However, massive data in agriculture are generated in every minute through multiple kinds of devices and services such as sensors and agricultural web markets. It leads to the challenges of big data problem including data collection, data storage, and data analysis. Although some systems have been proposed to address this problem, they are still restricted either in the type of data, the type of storage, or the size of data they can handle. In this paper, we propose a novel design of a platform for collecting and analyzing agricultural big data. The proposed platform supports (1) multiple methods of collecting data from various data sources using Flume and MapReduce; (2) multiple choices of data storage including HDFS, HBase, and Hive; and (3) big data analysis modules with Spark and Hadoop.

• The Journal of the Institute of Internet, Broadcasting and Communication
• /
• v.16 no.6
• /
• pp.225-230
• /
• 2016

Recently, Web has arisen large amount of data that to the development of the network and the Internet. In order to process it appeared that Big Data technology. Big Data technologies have been studied aiming a multifaceted and accurate analysis using existing regular data and a variety of data social data. But social data does not have the expertise and objectivity. And such manipulation and concealment and distortion of information have been raised troubling. Thus, this paper proposes for trust big data platform and will be described in detail. The big data platform proposed in this paper consists of data refiner, Data Analyzer, co-truster, visualizer, searcher, etc.

• KSII Transactions on Internet and Information Systems (TIIS)
• /
• v.15 no.3
• /
• pp.974-992
• /
• 2021

Recently, the healthcare field has undergone rapid changes owing to the accumulation of health big data and the development of machine learning. Data mining research in the field of healthcare has different characteristics from those of other data analyses, such as the structural complexity of the medical data, requirement for medical expertise, and security of personal medical information. Various methods have been implemented to address these issues, including the machine learning model and cloud platform. However, the machine learning model presents the problem of opaque result interpretation, and the cloud platform requires more in-depth research on security and efficiency. To address these issues, this paper presents a recent technology for Internet-of-Things-based (IoT-based) health big data processing. We present a cloud-based IoT health platform and health big data processing technology that reduces the medical data management costs and enhances safety. We also present a data mining technology for health-risk prediction, which is the core of healthcare. Finally, we propose a study using explainable artificial intelligence that enhances the reliability and transparency of the decision-making system, which is called the black box model owing to its lack of transparency.

• Journal of Korean Institute of Industrial Engineers
• /
• v.42 no.2
• /
• pp.112-121
• /
• 2016

This paper focuses on competencies of data scientists and behavioral intention that affect big data analysis performance. This experiment examined nine core factors required by data scientists. In order to investigate this, we conducted a survey to gather data from 103 data scientists who participated in big data competition at Kaggle platform and used factor analysis and PLS-SEM for the analysis methods. The results show that some key competency factors have influential effect on the big data analysis performance. This study is to provide a new theoretical basis needed for relevant research by analyzing the structural relationship between the individual competencies and performance, and practically to identify the priorities of the core competencies that data scientists must have.

• Journal of information and communication convergence engineering
• /
• v.21 no.3
• /
• pp.216-224
• /
• 2023

Smart lighting systems have become increasingly popular in several public sectors because of trends toward urbanization and intelligent technologies. In this study, we designed and implemented a web application platform to explore and monitor data acquired from lighting devices at Thammasat University (Rangsit Campus, Thailand). The platform provides a convenient interface for administrative and operative staff to monitor, control, and collect data from sensors installed on campus in real time for creating geographically specific big data. Platform development focuses on both back- and front-end applications to allow a seamless process for recording and displaying data from interconnected devices. Responsible persons can interact with devices and acquire data effortlessly, minimizing workforce and human error. The collected data were analyzed using an exploratory data analysis process. Missing data behavior caused by system outages was also investigated.

• International journal of advanced smart convergence
• /
• v.8 no.1
• /
• pp.82-86
• /
• 2019

We aim to use public data different from the remote BEMS energy diagnostics technology and already established and then switch the conventional operation environment to a big-data-based integrated management environment to operate and build a building energy management environment of maximized efficiency. In Step 1, various network management environments of the system integrated with a big data platform and the BEMS management system are used to collect logs created in various types of data by means of the big data platform. In Step 2, the collected data are stored in the HDFS (Hadoop Distributed File System) to manage the data in real time about internal and external changes on the basis of integration analysis, for example, relations and interrelation for automatic efficient management.

• Spatial Information Research
• /
• v.21 no.5
• /
• pp.43-51
• /
• 2013

In this study, the concept of spatial big data and effective ways to build a spatial big data system are presented. Big Data is defined as 3V(volume, variety, velocity). Spatial big data is the basis for evolution from 3V's big data to 6V's big data(volume, variety, velocity, value, veracity, visualization). In order to build an effective spatial big data, spatial big data system building should be promoted. In addition, spatial big data system should be performed a national spatial information base, convergence platform, service providers, and providers as a factor of production. The spatial big data system is made up of infrastructure(hardware), technology (software), spatial big data(data), human resources, law etc. The goals for the spatial big data system build are spatial-based policy support, spatial big data platform based industries enable, spatial big data fusion-based composition, spatial active in social issues. Strategies for achieving the objectives are build the government-wide cooperation, new industry creation and activation, and spatial big data platform built, technologies competitiveness of spatial big data.

• Proceedings of the Korea Information Processing Society Conference
• /
• 2017.04a
• /
• pp.661-664
• /
• 2017

Recently, during disasters occurrence, dealing with emergencies has been handled well by the early transmission of disaster relating notifications on social media networks (e.g., Twitter or Facebook). Intuitively, with their characteristics (e.g., real-time, mobility) and big communities whose users could be regarded as volunteers, social networks are proved to be a crucial role for disasters response. However, the amount of data transmitted during disasters is an obstacle for filtering informative messages; because the messages are diversity, large and very noise. This large volume of data could be seen as Social Big Data (SBD). In this paper, we proposed a big data platform for collecting and analyzing disasters' data from SBD. Firstly, we designed a collecting module; which could rapidly extract disasters' information from the Twitter; by big data frameworks supporting streaming data on distributed system; such as Kafka and Spark. Secondly, we developed an analyzing module which learned from SBD to distinguish the useful information from the irrelevant one. Finally, we also designed a real-time visualization on the web interface for displaying the results of analysis phase. To show the viability of our platform, we conducted experiments of the collecting and analyzing phases in 10 days for both real-time and historical tweets, which were about disasters happened in South Korea. The results prove that our big data platform could be applied to disaster information based systems, by providing a huge relevant data; which can be used for inferring affected regions and victims in disaster situations, from 21.000 collected tweets.

• The Journal of Bigdata
• /
• v.8 no.2
• /
• pp.83-95
• /
• 2023

Big data analytics and AI play a critical role in the development of smart plants. This study presents a big data platform for plant data and an 'AutoML platform' for AI-based plant O&M(Operation and Maintenance). The big data platform collects, processes and stores large volumes of data generated in plants using Hadoop, Spark, and Kafka. The AutoML platform is a machine learning automation system aimed at constructing predictive models for equipment prognostics and process optimization in plants. The developed platforms configures a data pipeline considering compatibility with existing plant OISs(Operation Information Systems) and employs a web-based GUI to enhance both accessibility and convenience for users. Also, it has functions to load user-customizable modules into data processing and learning algorithms, which increases process flexibility. This paper demonstrates the operation of the platforms for a specific process of an oil company in Korea and presents an example of an effective data utilization platform for smart plants.

• Journal of Korea Society of Digital Industry and Information Management
• /
• v.14 no.4
• /
• pp.57-68
• /
• 2018

This paper presents a smart firefighting appliances monitoring system based on big data analytics platform using GS1 for Smart City. Typical firefighting appliances are fire hydrant, fire extinguisher, fire alarm, sprinkler, fire engine, etc. for the fire of classes A/B/C/D/E. Among them, the dry chemical fire extinguisher have been widely supplied and 6 millions ones were replaced for the aging ones over 10 years in the past year. However, only 5% of them have been collected for recycling of chemical materials included the heavy metals of environment pollution. Therefore, we considered the trace of firefighting appliances from production to disposal for the public open service. In the paper, we suggest 1) a smart firefighting appliances system using GS1, 2) a big data analytics platform and 3) a public open service and visualization with the analyzed information, for fire extinguishers from production to disposal. It can give the information and the visualized diagrams with the analyzed data through the public open service and the free Apps.