• Journal of Digital Contents Society
• /
• v.15 no.4
• /
• pp.533-545
• /
• 2014

According to the evolving of IT technologies, the amount of data we are facing increasing exponentially. Thus, the technique for managing and analyzing these vast data that has emerged is a distributed processing system of big data. A quality evaluation for the existing distributed processing systems has been proceeded by the structured data environment. Thus, if we apply this to the evaluation of distributed processing systems of big data which has to focus on the analysis of the unstructured data, a precise quality assessment cannot be made. Therefore, a study of the quality evaluation model for the distributed processing systems is needed, which considers the environment of the analysis of big data. In this paper, we propose a new quality evaluation model by deriving the quality evaluation elements based on the ISO/IEC9126 which is the international standard on software quality, and defining metrics for validating the elements.

• Structural Monitoring and Maintenance
• /
• v.7 no.4
• /
• pp.345-365
• /
• 2020

At present, many machine leaning and data mining methods are used for analyzing and predicting structural response characteristics. However, the platform that combines big data analysis methods with online and offline analysis modules has not been used in actual projects. This work is dedicated to developing a multifunctional Hadoop-Spark big data platform for bridges to monitor and evaluate the serviceability based on structural health monitoring system. It realizes rapid processing, analysis and storage of collected health monitoring data. The platform contains offline computing and online analysis modules, using Hadoop-Spark environment. Hadoop provides the overall framework and storage subsystem for big data platform, while Spark is used for online computing. Finally, the big data Hadoop-Spark platform computational performance is verified through several actual analysis tasks. Experiments show the Hadoop-Spark big data platform has good fault tolerance, scalability and online analysis performance. It can meet the daily analysis requirements of 5s/time for one bridge and 40s/time for 100 bridges.

• Journal of Digital Convergence
• /
• v.14 no.4
• /
• pp.251-258
• /
• 2016

Big data processing is a broad term for processing data sets so large or complex that traditional data processing applications are inadequate. Widespread use of smart devices results in a huge impact on the way we process data. Many organizations are contemplating how to incorporate or integrate those devices into their enterprise data systems. We have proposed a way to process big data by way of integrating Raspberry Pi into a Hadoop cluster as a computational grid. We have then shown the efficiency through several experiments and the ease of scaling of the proposed system.

• The Journal of the Korea Contents Association
• /
• v.22 no.6
• /
• pp.125-134
• /
• 2022

This study aims to critically discuss the problems that need to be solved from a humanities perspective in order to utilize big data. It identifies and discusses three research problems that may arise from collecting, processing, and using big data. First, it looks at the fake information circulating with regard to problems with the data itself, specifically looking at article-type advertisements and fake news related to politics. Second, discrimination by the algorithm was cited as a problem with big data processing and its results. This discrimination was seen while searching for engineers on the portal site. Finally, problems related to the invasion of personal related information were seen in three categories: the right to privacy, the right to self-determination of information, and the right to be forgotten. This study is meaningful in that it points out the problems facing in the aspect of big data utilization from the humanities perspective in the era of big data and discusses possible problems in the collection, processing, and use of big data, respectively.

• Journal of Digital Convergence
• /
• v.11 no.4
• /
• pp.267-271
• /
• 2013

Our society has two prospective properties because of IT technology. Firstly, it is accelerated a degree of convergence. And convergence regions are expanded. For example, smart healthcare region was created by IT technology and medical industry. The efforts to convergence will be continued. Because of these properties, A number of data are made in our life. Through many devices such as smart phone, camera, game machine, tablet pc, various data types are produced. In this paper, we described utilization of Big Data. And we analysed Big Data processing process.

• Journal of the Korea Academia-Industrial cooperation Society
• /
• v.17 no.12
• /
• pp.716-722
• /
• 2016

In the recent rapidly changing IT environment, the amount of smart digital data is growing exponentially. As a result, in many areas, utilizing big data research and development services and related technologies is becoming more popular. In SMART learning, big data is used by students, teachers, parents, etc., from a perspective of the potential for many. In this paper, we describe big data and can utilize it to identify scenarios. Big data, obtained through customized learning services that can take advantage of the scheme, is proposed. To analyze educational big data processing technology for this purpose, we designed a system for big data processing. Education services offer the measures necessary to take advantage of educational big data. These measures were implemented on a test platform that operates in a cloud-based operations section for a pilot training program that can be applied properly. Teachers try using it directly, and in the interest of business and education, a survey was conducted based on enjoyment, the tools, and users' feelings (e.g., tense, worried, confident). We analyzed the results to lay the groundwork for educational use of big data.

• Annual Conference of KIPS
• /
• 2024.05a
• /
• pp.552-553
• /
• 2024

협력적 필터링은 추천시스템을 구축하는 알고리즘으로 고객별 선호도를 예측하는데 사용되고 있다. 이에 본 연구는 행복감에 영향을 주는 요인인 자존감과 생활여건을 사용하여, 협력적 필터링을 기반으로 한 예측정확도가 높은 모형을 연구하고자 한다. 이를 위해, 자존감과 생활여건에 대한 응답자 간의 유사도 가중치를 각각 계산한 후, 자존감 유사도 가중치를 적용한 모형으로 행복감을 예측하고, 자존감 유사도 가중치에 생활여건 유사도 가중치를 부여한 유사도 가중치를 적용한 모형으로 행복감을 예측하였다. 그 결과 전자의 모형이 후자의 모형보다 예측정확도가 높게 나타났다.

• The Journal of the Korea institute of electronic communication sciences
• /
• v.11 no.4
• /
• pp.433-440
• /
• 2016

In order to construct a system for big data processing, one needs to configure the node by using network equipments to connect multiple computers or establish cloud environments through virtual hosts on a single computer. However, there are many restrictions on constructing the big data analysis system including complex system configuration and cost. These constraints are becoming a major obstacle to professional manpower training for big data areas which is emerging as one of the most important national competitiveness. As a result, for professional manpower training of big data areas, this paper proposes a Raspberry Pi Board based educational big data processing system which is capable of practical training at an affordable price.

• Journal of the Korean Society of Surveying, Geodesy, Photogrammetry and Cartography
• /
• v.35 no.5
• /
• pp.405-414
• /
• 2017

Spatial data processing often poses challenges due to the unique characteristics of spatial data and this becomes more complex in spatial big data processing. Some tools have been developed and provided to users; however, they are not common for a regular user. This paper presents a benchmark test between two notable tools of spatial big data processing: GIS Tools for Hadoop and SpatialHadoop. At the same time, a MapReduce application is introduced to be used as a baseline to evaluate the effectiveness of two tools and to derive the impact of number of maps/reduces on the performance. By using these tools and New York taxi trajectory data, we perform a spatial data processing related to filtering the drop-off locations within Manhattan area. Thereby, the performance of these tools is observed with respect to increasing of data size and changing number of worker nodes. The results of this study are as follows 1) GIS Tools for Hadoop automatically creates a Quadtree index in each spatial processing. Therefore, the performance is improved significantly. However, users should be familiar with Java to handle this tool conveniently. 2) SpatialHadoop does not automatically create a spatial index for the data. As a result, its performance is much lower than GIS Tool for Hadoop on a same spatial processing. However, SpatialHadoop achieved the best result in terms of performing a range query. 3) The performance of our MapReduce application has increased four times after changing the number of reduces from 1 to 12.

• Journal of Korea Multimedia Society
• /
• v.22 no.2
• /
• pp.194-202
• /
• 2019

In the information society represented by the Fourth Industrial Revolution, various types of data and information that are difficult to see are produced, processed, and processed and circulated to enhance the value of existing goods. The IoT(Internet of Things) paradigm will change the appearance of individual life, industry, disaster, safety and public service fields. In order to implement the IoT paradigm, several elements of technology are required. It is necessary that these various elements are efficiently connected to constitute one system as a whole. It is also necessary to collect, provide, transmit, store and analyze IoT data for implementation of IoT platform. We designed and implemented a big data processing IoT platform for IoT service implementation. Proposed platform system is consist of IoT sensing/control device, IoT message protocol, unstructured data server and big data analysis components. For platform testing, fixed IoT devices were implemented as solar power generation modules and mobile IoT devices as modules for table tennis stroke data measurement. The transmission part uses the HTTP and the CoAP, which are based on the Internet. The data server is composed of Hadoop and the big data is analyzed using R. Through the emprical test using fixed and mobile IoT devices we confirmed that proposed IoT platform system normally process and operate big data.