• Title/Summary/Keyword: Big data Processing

Search Result 1,063, Processing Time 0.029 seconds

Social Network Analysis by Utilizing Disaster Risk Big Data (재난 위험신고 빅데이터를 활용한 사회연결망 분석)

  • Han, Ji-Ah;Jeong, Duk-Hoon
    • The Journal of Bigdata
    • /
    • v.1 no.2
    • /
    • pp.45-63
    • /
    • 2016
  • According to changes of recent climate social structures, frequency of occurrence new or complex disasters are increasing. So the importance of disaster prevention is increasing. To provide useful information of disaster prevention activities, We use the "Safety Sinmungo" main processing practices included Facility safety management in Ministry of Public Safety and Security. Facility safety management is the most and common disaster prevention activities. We identified the keywords in the risk report and facilities to residents report and analyzed the seasonal and inter-regional facilities report distribution process. We also utilized social network analysis techniques to configure a 1-mode, 2-mode facilities around the keyword for differences.

  • PDF

Multi-Parameter Operation Method for Robust Disparity Plane (강건한 시차 평면을 위한 다중 파라미터 연산 기법)

  • Kim, Hyun-Jung;Weon, Il-Yong;Lee, Chang-Hun
    • KIPS Transactions on Software and Data Engineering
    • /
    • v.4 no.5
    • /
    • pp.241-246
    • /
    • 2015
  • Although many different methods have been used to solve stereo correspondent problems, the deviation of accuracy is too big. Among those many methods, the one that uses segmentation information of input image has received high attention in academic field since it is very close to vision recognition. In this thesis, the existing method of acquiring a single value by using the segment information and initial disparity value was viewed in NP-hard problem to propose a new method. In order to verify the validity of the proposed method, well-known data were used for experiment and the resulted data was analyzed. Although there were some disadvantages in the time aspect, it showed somewhat useful results in the accuracy aspect.

A New File System for Multimedia Data Stream (멀티미디어 데이터 스트림을 위한 파일 시스템의 설계 및 구현)

  • Lee, Minsuk;Song, Jin-Seok
    • IEMEK Journal of Embedded Systems and Applications
    • /
    • v.1 no.2
    • /
    • pp.90-103
    • /
    • 2006
  • There are many file systems in various operating systems. Those are usually designed for server environments, where the common cases are usually 'multiple active users', 'great many small files' And they assume a big main memory to be used as buffer cache. So the existing file systems are not suitable for resource hungry embedded systems that process multimedia data streams. In this study, we designed and implemented a new file system which efficiently stores and retrieves multimedia data steams. The proposed file system has a very simple disk layout, which guarantees a quick disk initialization and file system recovery. And we introduced a new indexing-scheme, called the time-based indexing scheme, with the file system. With the indexing scheme, the file system maintains the relation between time and the location for all the multimedia streams. The scheme is useful in searching and playing the compressed multimedia streams by locating exact frame position with given time, resulting in reduction of CPU processing and power consumption. The proposed file system and its APIs utilizing the time-based indexing schemes were implemented firstly on a Linux environment, though it is operating system independent. In the performance evaluation on a real DVR system, which measured the execution time of multi-threaded reading and writing, we found the proposed file system is maximum 38.7% faster than EXT2 file system.

  • PDF

Underwater Acoustic Research Trends with Machine Learning: General Background

  • Yang, Haesang;Lee, Keunhwa;Choo, Youngmin;Kim, Kookhyun
    • Journal of Ocean Engineering and Technology
    • /
    • v.34 no.2
    • /
    • pp.147-154
    • /
    • 2020
  • Underwater acoustics that is the study of the phenomenon of underwater wave propagation and its interaction with boundaries, has mainly been applied to the fields of underwater communication, target detection, marine resources, marine environment, and underwater sound sources. Based on the scientific and engineering understanding of acoustic signals/data, recent studies combining traditional and data-driven machine learning methods have shown continuous progress. Machine learning, represented by deep learning, has shown unprecedented success in a variety of fields, owing to big data, graphical processor unit computing, and advances in algorithms. Although machine learning has not yet been implemented in every single field of underwater acoustics, it will be used more actively in the future in line with the ongoing development and overwhelming achievements of this method. To understand the research trends of machine learning applications in underwater acoustics, the general theoretical background of several related machine learning techniques is introduced in this paper.

A Database Design Method for Wind Power Plant SCADA System based on IEC61400-25 (IEC61400-25 국제표준기반 풍력 SCADA시스템을 위한 데이터베이스 설계방안)

  • Chae, Chang Hun;Choi, Hyo Yul;Choi, Jun Suk
    • KIPS Transactions on Computer and Communication Systems
    • /
    • v.1 no.3
    • /
    • pp.151-160
    • /
    • 2012
  • In this paper, a database method for Wind Power Plant SCADA system based on IEC61400-25 was designed. To manage big data, which is produced by the introduction of international standards and large/grouping of wind power plant, database should be systematically designed. As identify the characteristics of the wind power data and reflect the requirements of a user, it would be decreasing the waste of data space and managing efficiently the system. As a result, it is expected to reduce cost and effort in development and maintenance of Wind Power Plant.

Blockchain for the Trustworthy Decentralized Web Architecture

  • Kim, Geun-Hyung
    • International Journal of Internet, Broadcasting and Communication
    • /
    • v.13 no.1
    • /
    • pp.26-36
    • /
    • 2021
  • The Internet was created as a decentralized and autonomous system of interconnected computer networks used for data exchange across mutually trusted participants. The element technologies on the Internet, such as inter-domain and intra-domain routing and DNS, operated in a distributed manner. With the development of the Web, the Web has become indispensable in daily life. The existing web applications allow us to form online communities, generate private information, access big data, shop online, pay bills, post photos or videos, and even order groceries. This is what has led to centralization of the Web. This centralization is now controlled by the giant social media platforms that provide it as a service, but the original Internet was not like this. These giant companies realized that the decentralized network's huge value involves gathering, organizing, and monetizing information through centralized web applications. The centralized Web applications have heralded some major issues, which will likely worsen shortly. This study focuses on these problems and investigates blockchain's potentials for decentralized web architecture capable of improving conventional web services' critical features, including autonomous, robust, and secure decentralized processing and traceable trustworthiness in tamper-proof transactions. Finally, we review the decentralized web architecture that circumvents the main Internet gatekeepers and controls our data back from the giant social media companies.

Association Rule Mining and Collaborative Filtering-Based Recommendation for Improving University Graduate Attributes

  • Sheta, Osama E.
    • International Journal of Computer Science & Network Security
    • /
    • v.22 no.6
    • /
    • pp.339-345
    • /
    • 2022
  • Outcome-based education (OBE) is a tried-and-true teaching technique based on a set of predetermined goals. Program Educational Objectives (PEOs), Program Outcomes (POs), and Course Outcomes (COs) are the components of OBE. At the end of each year, the Program Outcomes are evaluated, and faculty members can submit many recommended measures which dependent on the relationship between the program outcomes and its courses outcomes to improve the quality of program and hence the overall educational program. When a vast number of courses are considered, bad actions may be proposed, resulting in unwanted and incorrect decisions. In this paper, a recommender system, using collaborative filtering and association rules algorithms, is proposed for predicting the best relationship between the program outcomes and its courses in order to improve the attributes of the graduates. First, a parallel algorithm is used for Collaborative Filtering on Data Model, which is designed to increase the efficiency of processing big data. Then, a parallel similar learning outcomes discovery method based on matrix correlation is proposed by mining association rules. As a case study, the proposed recommender system is applied to the Computer Information Systems program, College of Computer Sciences and Information Technology, Al-Baha University, Saudi Arabia for helping Program Quality Administration improving the quality of program outcomes. The obtained results revealed that the suggested recommender system provides more actions for boosting Graduate Attributes quality.

A Novel Node Management in Hadoop Cluster by using DNA

  • Balaraju. J;PVRD. Prasada Rao
    • International Journal of Computer Science & Network Security
    • /
    • v.23 no.9
    • /
    • pp.134-140
    • /
    • 2023
  • The distributed system is playing a vital role in storing and processing big data and data generation is speedily increasing from various sources every second. Hadoop has a scalable, and efficient distributed system supporting commodity hardware by combining different networks in the topographical locality. Node support in the Hadoop cluster is rapidly increasing in different versions which are facing difficulty to manage clusters. Hadoop does not provide Node management, adding and deletion node futures. Node identification in a cluster completely depends on DHCP servers which managing IP addresses, hostname based on the physical address (MAC) address of each Node. There is a scope to the hacker to theft the data using IP or Hostname and creating a disturbance in a distributed system by adding a malicious node, assigning duplicate IP. This paper proposing novel node management for the distributed system using DNA hiding and generating a unique key using a unique physical address (MAC) of each node and hostname. The proposed mechanism is providing better node management for the Hadoop cluster providing adding and deletion node mechanism by using limited computations and providing better node security from hackers. The main target of this paper is to propose an algorithm to implement Node information hiding in DNA sequences to increase and provide security to the node from hackers.

A Study of Ginseng Culture within 'Joseonwangjosilok' through Textual Frequency Analysis

  • Mi-Hye Kim
    • CELLMED
    • /
    • v.14 no.2
    • /
    • pp.2.1-2.10
    • /
    • 2024
  • Through big data analysis of the 'Joseonwangjosilok', this study examines the perception of ginseng among the ruling class and its utilization during the Joseon era. It aims to provide foundational data for the development of ginseng into a high-value cultural commodity. The focus of this research, the Joseonwangjosilok, comprises 1,968 volumes in 948 books, spanning a record of 518 years. Data was collected through web crawling on the website of the National Institute of Korean History, followed by frequency analysis of significant words. To assess the interest in ginseng across the reigns of 27 kings during the Joseon era, ginseng frequency records were adjusted based on years in power and the number of articles, creating an interest index for comparative rankings across reigns. Analysis revealed higher interest in ginseng during the reigns of King Jeongjo and King Yeongjo in the 18th century, King Sunjo in the 19th century, King Sejong in the 15th century, King Sukjong in the 17th century, and King Gojong in the 19th century. Examining the temporal emergence and changes in ginseng during the Joseon era, general ginseng types like insam and sansam had the highest frequency in the 15th century. It appears that Korea adeptly utilized ceremonial goods in diplomatic relations with China and Japan, meeting the demand for ginseng from their royal and aristocratic societies. Processed ginseng varieties such as hongsam and posam, along with traded and taxed ginseng, showed peak frequency in the 18th century. This coincided with increased cultivation, allowing a higher supply and fostering the development of ginseng processing technologies like hongsam.

Sentimental Analysis of Twitter Data Using Machine Learning and Deep Learning: Nickel Ore Export Restrictions to Europe Under Jokowi's Administration 2022

  • Sophiana Widiastutie;Dairatul Maarif;Adinda Aulia Hafizha
    • Asia pacific journal of information systems
    • /
    • v.34 no.2
    • /
    • pp.400-420
    • /
    • 2024
  • Nowadays, social media has evolved into a powerful networked ecosystem in which governments and citizens publicly debate economic and political issues. This holds true for the pros and cons of Indonesia's ore nickel export restriction to Europe, which we aim to investigate further in this paper. Using Twitter as a dependable channel for conducting sentiment analysis, we have gathered 7070 tweets data for further processing using two sentiment analysis approaches, namely Support Vector Machine (SVM) and Long Short Term Memory (LSTM). Model construction stage has shown that Bidirectional LSTM performed better than LSTM and SVM kernels, with accuracy of 91%. The LSTM comes second and The SVM Radial Basis Function comes third in terms of best model, with 88% and 83% accuracies, respectively. In terms of sentiments, most Indonesians believe that the nickel ore provision will have a positive impact on the mining industry in Indonesia. However, a small number of Indonesian citizens contradict this policy due to fears of a trade dispute that could potentially harm Indonesia's bilateral relations with the EU. Hence, this study contributes to the advancement of measuring public opinions through big data tools by identifying Bidirectional LSTM as the optimal model for the dataset.