• Title/Summary/Keyword: big data tasks

Search Result 98, Processing Time 0.021 seconds

A Study on Big Data Processing Technology Based on Open Source for Expansion of LIMS (실험실정보관리시스템의 확장을 위한 오픈 소스 기반의 빅데이터 처리 기술에 관한 연구)

  • Kim, Soon-Gohn
    • The Journal of Korea Institute of Information, Electronics, and Communication Technology
    • /
    • v.14 no.2
    • /
    • pp.161-167
    • /
    • 2021
  • Laboratory Information Management System(LIMS) is a centralized database for storing, processing, retrieving, and analyzing laboratory data, and refers to a computer system or system specially designed for laboratories performing inspection, analysis, and testing tasks. In particular, LIMS is equipped with a function to support the operation of the laboratory, and it requires workflow management or data tracking support. In this paper, we collect data on websites and various channels using crawling technology, one of the automated big data collection technologies for the operation of the laboratory. Among the collected test methods and contents, useful test methods and contents useful that the tester can utilize are recommended. In addition, we implement a complementary LIMS platform capable of verifying the collection channel by managing the feedback.

Locality-Sensitive Hashing Techniques for Nearest Neighbor Search

  • Lee, Keon Myung
    • International Journal of Fuzzy Logic and Intelligent Systems
    • /
    • v.12 no.4
    • /
    • pp.300-307
    • /
    • 2012
  • When the volume of data grows big, some simple tasks could become a significant concern. Nearest neighbor search is such a task which finds from a data set the k nearest data points to queries. Locality-sensitive hashing techniques have been developed for approximate but fast nearest neighbor search. This paper introduces the notion of locality-sensitive hashing and surveys the locality-sensitive hashing techniques. It categories them based on several criteria, presents their characteristics, and compares their performance.

Proposal of Big Data Analysis and Visualization Technique Curriculum for Non-Technical Majors in Business Management Analysis (경영분석 업무에 종사하는 비 기술기반 전공자를 위한 빅데이터 분석 및 시각화 기법 교육과정 제안)

  • Hong, Pil-Tae;Yu, Jong-Pil
    • Journal of Practical Engineering Education
    • /
    • v.12 no.1
    • /
    • pp.31-39
    • /
    • 2020
  • Big data analysis is analyzed and used in a variety of management and industrial sites, and plays an important role in management decision making. The job competency of big data analysis personnel engaged in management analysis work does not necessarily require the acquisition of microscopic IT skills, but requires a variety of experiences and humanities knowledge and analytical skills as a Data Scientist. However, big data education by state-run and state-run educational institutions and job education institutions based on the National Competency Standards (NCS) is proceeding in terms of software engineering, and this teaching methodology can have difficult and inefficient consequences for non-technical majors. Therefore, we analyzed the current Big Data platform and its related technologies and defined which of them are the requisite job competency requirements for field personnel. Based on this, the education courses for big data analysis and visualization techniques were organized for non-technical-based majors. This specialized curriculum was conducted by working-level officials of financial institutions engaged in management analysis at the management site and was able to achieve better educational effects The education methods presented in this study will effectively carry out big data tasks across industries and encourage visualization of big data analysis for non-technical professionals.

Big data distributed processing system using RHadoop (RHadoop을 이용한 빅데이터 분산처리 시스템)

  • Shin, Ji Eun;Jung, Byung Ho;Lim, Dong Hoon
    • Journal of the Korean Data and Information Science Society
    • /
    • v.26 no.5
    • /
    • pp.1155-1166
    • /
    • 2015
  • It is almost impossible to store or analyze big data increasing exponentially with traditional technologies, so Hadoop is a new technology to make that possible. In recent R is using as an engine for big data analysis based on distributed processing with Hadoop technology. With RHadoop that integrates R and Hadoop environment, we implemented parallel multiple regression analysis with various data sizes of actual data and simulated data. Experimental results showed our RHadoop system was faster as the number of data nodes increases. We also compared the performance of our RHadoop with lm function and biglm packages available on bigmemory. The results showed that our RHadoop was faster than other packages owing to paralleling processing with increasing the number of map tasks as the size of data increases.

A Study on the Current Situations about the Use of Big Data for Cost Estimating Tasks in CM Companies (CM사 견적업무의 빅데이터 활용 현황에 관한 연구)

  • Kim, Hyeon Jin;Kim, Han Soo
    • Korean Journal of Construction Engineering and Management
    • /
    • v.22 no.6
    • /
    • pp.24-33
    • /
    • 2021
  • Cost management is a major function of CM (construction management) companies for clients and cost estimating is a critical task in that it provides a baseline for cost management and a foundation for decision making in construction projects. For this purpose, CM companies need to obtain and use good quality data, which leads to more accurate and efficient cost estimating. As the use of big data becomes increasingly important in the construction industry, researches related to the theme have become the active areas of studies. However, literature review shows that the current situations in relation to the use of big for cost estimating of CM companies are under-researched. The objective of the study is to identify key characteristics and implications in the use of big data for cost estimating of CM companies, which can contribute to develop strategies for such purposes.

The Smart Port Management System Based on Big-data (빅데이터 기반 스마트 항만 운용시스템)

  • Lee, Woo;Kim, Sang-Hyun;Oh, Seung-Hong;Kim, Won-Jung
    • The Journal of the Korea institute of electronic communication sciences
    • /
    • v.17 no.1
    • /
    • pp.167-172
    • /
    • 2022
  • Currently, ship control, tug, and pilot work in import/export ports including Gwangyang Port are operated according to factors such as the order of arrival and departure regardless of the shipping company. Also, even this is done very inefficiently by hand. Therefore, there is an urgent need to develop a system to increase the efficiency of port and ship operation through standardization and digitalization of tasks related to Berthing and unberthing of ships. In this study, we propose a method to increase the efficiency of port and vessel operation by designing a smart port operation system based on big data such as vessel location information, pilotage and tug schedule, arrival/departure operation information, and weather information.

Range Segmentation of Dynamic Offloading (RSDO) Algorithm by Correlation for Edge Computing

  • Kang, Jieun;Kim, Svetlana;Kim, Jae-Ho;Sung, Nak-Myoung;Yoon, Yong-Ik
    • Journal of Information Processing Systems
    • /
    • v.17 no.5
    • /
    • pp.905-917
    • /
    • 2021
  • In recent years, edge computing technology consists of several Internet of Things (IoT) devices with embedded sensors that have improved significantly for monitoring, detection, and management in an environment where big data is commercialized. The main focus of edge computing is data optimization or task offloading due to data and task-intensive application development. However, existing offloading approaches do not consider correlations and associations between data and tasks involving edge computing. The extent of collaborative offloading segmented without considering the interaction between data and task can lead to data loss and delays when moving from edge to edge. This article proposes a range segmentation of dynamic offloading (RSDO) algorithm that isolates the offload range and collaborative edge node around the edge node function to address the offloading issue.The RSDO algorithm groups highly correlated data and tasks according to the cause of the overload and dynamically distributes offloading ranges according to the state of cooperating nodes. The segmentation improves the overall performance of edge nodes, balances edge computing, and solves data loss and average latency.

Ergonomic Problems and Their Improving Measures in Office Environment of General Hospitals

  • Kee, Dohyung
    • Journal of the Ergonomics Society of Korea
    • /
    • v.34 no.2
    • /
    • pp.135-143
    • /
    • 2015
  • Objective: The purposes of this study are to investigate ergonomic problems and to propose their improving measures in office environment of big general hospitals. Background: Office tasks have varying risk factors of work-related musculoskeletal disorders (WMSDs). The first symptom resulted from the office work was recognized as an occupational disease in Korea in 1986. Although the symptoms have increased since its first recognition, there has been few study on the effect of office work environment settings. Method: First, the author took pictures of working scenes performed in three big university hospitals. Next, the pictures were analyzed in view point of ergonomics. Based on the analysis, their improving measures were proposed for reducing work stress. Results: The results showed that most physical office environment settings such as dimensions of tables/desks and chairs, leg room, thigh, knee and foot clearances, and chairs used did not satisfy the ergonomically recommended design guidelines. In addition, some clerks placed personal belongings under their desks, put monitors in high position and did not lean against the backrest of chairs in seated tasks, which resulted in poor working postures of leg, back, neck etc. It is recommended that the hospital management should provide their clerks with ergonomically designed office furniture and continuously perform ergonomics training program for raising clerks' recognition for office ergonomics. Conclusion: Most office environment settings investigated in this study were not in good condition in view point of ergonomic design for the settings. Application: It would be useful as basic data for establishing ergonomically good office environment in hospitals.

Investigating the Influence of a Food-themed TV Program on Delivery Food Order Amount Using Big Data with Difference-in-Differences Method (빅 데이터를 이용한 음식방송의 효과 확인: 이중차이분석을 적용하여)

  • Park, Jihye;Park, Jaehong
    • Information Systems Review
    • /
    • v.18 no.1
    • /
    • pp.25-39
    • /
    • 2016
  • This study suggests a case for people who are unfamiliar with data analysis to equip them in using big data easily without complex programming tasks. Consequently, we investigate whether a food-themed TV program influences the number of delivery food orders with the use of the difference-in-differences method. Results show that the number of delivery food orders significantly increased after broadcasting four of five food-themed TV program episodes, each of which focuses on a particular delivery food. This study contributes to the existing literature by presenting the possibility that food-themed TV programs can positively affect not only the broadcast delivery food stores but also the entire delivery food business. In addition, this study provides practical contributions by recommending a big data analysis methodology that can be easily employed by many people.

Managerial Factors Influencing Dose Reduction of the Nozzle Dam Installation and Removal Tasks Inside a Steam Generator Water Chamber (증기발생기 수실 노즐댐 설치 및 제거작업의 피폭선량 저감에 영향을 주는 관리요인에 관한 연구)

  • Lee, Dhong Ha
    • Journal of the Ergonomics Society of Korea
    • /
    • v.36 no.5
    • /
    • pp.559-568
    • /
    • 2017
  • Objective: The aim of this study is to investigate the effective managerial factors influencing dose reduction of the nozzle dam installation and removal tasks ranking within top 3 in viewpoint of average collective dose of nuclear power plant maintenance job. Background: International Commission on Radiation Protection (ICRP) recommended to reduce unnecessary dose and to minimize the necessary dose on the participants of maintenance job in radiation fields. Method: Seven sessions of nozzle dam installation and removal task logs yielded a multiple regression model with collective dose as a dependent variable and work time, number of participants, space doses before and after shield as independent variables. From the sessions in which a significant reduction in collective dose occurred, the effective managerial factors were elicited. Results: Work time was the most important factor contributing to collective dose reduction of nozzle dam installation and removal task. Introduction of new technology in nozzle dam design or maintenance job is the most important factor for work time reduction. Conclusion: With extended task logs and big data processing technique, the more accurate prediction model illustrating the relationship between collective dose reduction and effective managerial factors would be developed. Application: The effective managerial factors will be useful to reduce collective dose of decommissioning tasks as well as regular preventive maintenance tasks for a nuclear power plant.