• Title/Summary/Keyword: 대용량의 점데이터

Search Result 130, Processing Time 0.026 seconds

An Efficient MapReduce-based Skyline Query Processing Method with Two-level Grid Blocks (2-계층 그리드 블록을 이용한 효과적인 맵리듀스 기반 스카이라인 질의 처리 기법)

  • Ryu, Hyeongcheol;Jung, Sungwon
    • Journal of KIISE
    • /
    • v.44 no.6
    • /
    • pp.613-620
    • /
    • 2017
  • Skyline queries are used extensively to solve various problems, such as in decision-making, because they find data that meet a variety of user criteria. Recent research has focused on skyline queries by using the MapReduce framework for large database processing, mainly in terms of applying existing index structures to MapReduce. In a skyline, data closer to the origin dominate more area. However, the existing index structure does not reflect such characteristics of the skyline. In this paper, we propose a grid-block structure that groups grid cells to match the characteristics of a skyline, and a two-level grid-block structure that can be used even when there are no data close to the origin. We also propose an efficient skyline-query algorithm that uses the two-level grid-block structure.

A study on the security policy improvement using the big data (빅데이터를 이용한 보안정책 개선에 관한 연구)

  • Kim, Song-Young;Kim, Joseph;Lim, Jong-In;Lee, Kyung-Ho
    • Journal of the Korea Institute of Information Security & Cryptology
    • /
    • v.23 no.5
    • /
    • pp.969-976
    • /
    • 2013
  • The information protection systems of company are intended to detect all weak points, intrusion, document drain. All actions of people in company are recorded and can check persistently. On the other hand, what analyze security log generated by these systems becomes more difficult. Most staff who manages the security systems, and analyze log is more incomprehensible than a user or a person of drain for an information distribution process of the work-site operations and the management procedure of the critical information. Such a reality say the serious nature of the internal information leakage that can be brought up more. While the research on the big data proceeds actively recently, the successful cases are being announced in the various areas. This research is going to present the improved big data processing technology and case of the security field.

Analysis of NWP GRIB Data for LEO Satellite Mission Planning (저궤도 관측위성 임무계획(Mission Planning)을 위한 기상수치예보 GRIB Data 분석)

  • Seo Jeong-Soo;Seo Seok-Bae;Bae Hee-Jin;Kim Eun-Kyou
    • Proceedings of the KSRS Conference
    • /
    • 2006.03a
    • /
    • pp.178-186
    • /
    • 2006
  • 기상 수치예보는 (Numerical Weather Pridiction, NWP)는 바람, 기온, 등과 같은 기상요소의 시간 변화를 나타내는 물리방정식을 컴퓨터로 풀어 미래의 대기 상태를 예상하는 과학적인 방법으로 지구를 상세한 격자 2진부호(GRIdded Binary, 이하 GRIB)로 나누어 그 격자점에서의 값으로 대기 상태를 나타낸다. 지구 각지에서의 각종 관측자료를 기초로 격자점상의 현재값을 구한다. 대용량의 격자데이터는 이진형태이어서 컴퓨터, 서버 저장장치에서 동일형태 데이터로 존재한다. 우리나라 최초의 저궤도 관측 위성인 다목적 실용위성 KOMPSAT-1호(이하, 아리랑 위성1호)는 전자광학카메라(Electro Optical Camera, EOC)를 탑재하여 1999년 12월 21일에 발사된 이후 2006년 1월 현재까지 6여년간 성공적으로 임무를 수행, 7049여회의 영상을 획득하여 국가적으로 귀중한 자료로 활용하고 있다. 아리랑 위성1호는 일일 2-3회 EOC영상을 획득하고 있으며, 임무계획(Mission Planning)은 MP(Mission Planner)가 사용자로부터 자료를 수집하여 임무분석 및 계획 서브시스템(MAPS)에 의해 계산되어진 위성의 제도예측 데이터에 촬영하고자하는 목표지점 좌표를 입력하여 자동명령생성기(KSCG)에 의해 계산된 촬영 경사각도(Tilt)값을 위성에 전송하여 목표지역의 영상을 획득하게 된다. 위성영상 획득에 있어 고가의 위성을 운영하면서 기상의 상태를 정확히 예측하여 실패없이 유효한 영상을 획득하는 것이 무엇보다 중요하다. 본 논문에서는 효율적인 위성임무계획을 위한 기상수치예보 자료를 분석하여 앞으로 발사하게 될 고해상 카메라 탑제위성인 아리랑 위성2호와 3호에 적용하고자 한다. the sufficient excess reactivity to override this poisoning must be inserted, or its concentration is decreased sufficiently when its temporary shutdown is required. As ratter of fact, these have an important influence not only on reactor safety but also on economic aspect in operation. Considering these points in this study, the shutdown process was cptimized using the Pontryagin's maximum principle so that the shutdown mirth[d was improved as to restart the reactor to its fulpower at any time, but the xenon concentration did not excess the constrained allowable value during and after shutdown, at the same time all the control actions were completed within minimum time from beginning of the shutdown.및 12.36%, $101{\sim}200$일의 경우 12.78% 및 12.44%, 201일 이상의 경우 13.17% 및 11.30%로 201일 이상의 유기의 경우에만 대조구와 삭제 구간에 유의적인(p<0.05) 차이를 나타내었다.는 담수(淡水)에서 10%o의 해수(海水)

  • PDF

Block Allocation Method for Efficiently Managing Temporary Files of Hash Joins on SSDs (SSD상에서 해시조인 임시 파일의 효과적인 관리를 위한 블록 할당 방법)

  • Joontae, Kim;Sangwon, Lee
    • KIPS Transactions on Computer and Communication Systems
    • /
    • v.11 no.12
    • /
    • pp.429-436
    • /
    • 2022
  • Temporary files are generated when the Hash Join is performed on tables larger than the memory. During the join process, each temporary file is deleted sequentially after it completes the I/O operations. This paper reveals for that the fallocate system call and file deletion-related trim options significantly impact the hash join performance when temporary files are managed on SSDs rather than hard disks. The experiment was conducted on various commercial and research SSDs using PostgreSQL, a representative open-source database. We find that it is possible to improve the join performance up to 3 to 5 times compared to the default combination depending on whether fallocate and trim options are used for temporary files. In addition, we investigate the write amplification and trim command overhead in the SSD according to the combination of the two options for temporary files.

Explanation-based Data Mining in Data Warehouse (데이타 웨어하우스 환경에서의 설명기반 데이타 마이닝)

  • 김현수;이창호
    • Proceedings of the Korea Database Society Conference
    • /
    • 1999.06a
    • /
    • pp.115-123
    • /
    • 1999
  • 산업계 전반에 걸친 오랜 정보시스템 운용의 결과로 대용량의 데이타들이 축적되고 있다. 이러한 데이타로부터 유용한 지식을 추출하기 위해 여러 가지 데이타 마이닝 기법들이 연구되어왔다. 특히 데이타 웨어하우스의 등장은 이러한 데이타 마이닝에 있어 필요한 데이타 제공 환경을 제공해 주고 있다. 그러나 전문가의 적절한 판단과 해석을 거치지 않은 데이타 마이닝의 결과는 당연한 사실이거나, 사실과 다른 가짜이거나 또는 관련성이 없는(trivial, spurious and irrelevant) 내용만 무수히 쏟아낼 수 있다. 그러므로 데이타 마이닝의 결과가 비록 통계적 유의성을 가진다 하더라고 그 정당성과 유용성에 대한 검증과정과 방법론의 정립이 필요하다. 데이타 마이닝의 가장 어려운 점은 귀납적 오류를 없애기 위해 사람이 직접 그 결과를 해석하고 판단하며 아울러 새로운 탐색 방향을 제시해야 한다는 것이다. 본 논문의 목적은 이러한 데이타 마이닝에서 추출된 결과를 검증하고 아울러 새로운 지식 탐색 방향을 제시하는 방법론을 정립하는데 있다. 본 논문에서는 데이타 마이닝 기법 중 연관규칙탐사로 얻어진 결과를 설명가능성 여부의 판단을 통해 검증하는 기법을 제안하며, 이를 통해 얻어진 검증된 지식을 토대로 일반화를 통한 새로운 가설을 생성하여 데이타 웨어하우스로부터 연관규칙을 검증하는 일련의 아키텍쳐(architecture)를 제시하고자 한다. 먼저 데이타 마이닝 결과에 대한 설명의 필요성을 제시하고, 데이타 웨어하우스와 데이타 마이닝 기법들에 대한 간략한 설명과 연관규칙탐사에 대한 정의 및 방법을 보이고, 대상 영역에 대한 데이타 웨어하우스의 스키마를 보였다. 다음으로 도메인 지식(domain knowledge)과 연관규칙탐사를 통해 얻어진 결과를 표현하기 위한 지식표현 방법으로 Relational predicate Logic을 제안하였다. 연관규칙탐사로 얻어진 결과를 설명하기 위한 방법으로는 연관규칙탐사로 얻어진 연관규칙에 대해 Relational Predicate Logic으로 표현된 도메인 지식으로서 설명됨을 보이게 한다. 또한 이러한 설명(explanation)을 토대로 검증된 지식을 일반화하여 새로운 가설을 연역적으로 생성하고 이를 연관규칙탐사론 통해 검증한 후 새로운 지식을 얻는 반복적인 Explanation-based Data Mining Architecture를 제시하였다. 본 연구의 의의로는 데이타 마이닝을 통한 귀납적 지식생성에 있어 귀납적 오류의 발생을 고메인 지식을 통해 설명가능 함을 보임으로 검증하고 아울러 이러한 설명을 통해 연역적으로 새로운 가설지식을 생성시켜 이를 가설검증방식으로 검증함으로써 귀납적 접근과 연역적 접근의 통합 데이타 마이닝 접근을 제시하였다는데 있다.

  • PDF

Compression of Medical Images Using DWT (DWT을 이용한 의료영상 압축)

  • Lim, Jae-Dong;Lee, Sang-Bock
    • Journal of the Korean Society of Radiology
    • /
    • v.2 no.2
    • /
    • pp.11-16
    • /
    • 2008
  • The most difficult of implementation PACS is large amount of data. Therefore, PACS needs mass storage, as well as rapid transmission time. Consequently, medical images needs compression when stored in PACS. WT(wavelet transform) was announced by Ingrid Daubechies and Stephane Mallat, WT was methods of signal analysis by a base functions set same as Fourie transform. This paper estimated an efficiency, that experimental medical images compressed by DWT. The result of estimated, we are knows effectiveness that display to remained signal in low frequency region after 4-level DWT form $512{\times}512{\times}2^8$ input images. Compression ratio of images by 4-level DWT was 1:16. It is a high compression ratio, the other side has a problem appears on staircase phenomenon.

  • PDF

Geometry Transformation in Spatial Domain Using Coefficient Changes in Frequency Domain toward Lightweight Image Encryption (주파수 영역에서의 계수 값 변환에 의한 공간 영역에서의 기하학적 변환과 이를 이용한 이미지 경량 암호화)

  • Joo, Jeong Hyun;Yi, Kang
    • Journal of KIISE
    • /
    • v.42 no.2
    • /
    • pp.162-168
    • /
    • 2015
  • Image data is mostly stored in compressed form because of its huge size. Therefore, a series of cumbersome procedures is required to apply a transformation to image data: decompression, extraction of spatial data, transformation and recompression. In this paper, we employ DCT(Discrete Cosine Transform) coefficients to change the spatial presentation of images. DCT is commonly used in still image compression standards such as JPEG and moving picture compression standards such as MPEG-2, MPEG-4, and H.264. In this paper, we derived mathematically the relationship between the geometry transformation in the spatial domain and coefficient changes in the DCT domain and verified it with images in the JPEG file format. Because of the efficiency of transformation in the frequency domain, our findings can be utilized for light-weight partial image encryption for privacy data protection or entertainment contents protection.

An Address Translation Technique Large NAND Flash Memory using Page Level Mapping (페이지 단위 매핑 기반 대용량 NAND플래시를 위한 주소변환기법)

  • Seo, Hyun-Min;Kwon, Oh-Hoon;Park, Jun-Seok;Koh, Kern
    • Journal of KIISE:Computing Practices and Letters
    • /
    • v.16 no.3
    • /
    • pp.371-375
    • /
    • 2010
  • SSD is a storage medium based on NAND Flash memory. Because of its short latency, low power consumption, and resistance to shock, it's not only used in PC but also in server computers. Most SSDs use FTL to overcome the erase-before-overwrite characteristic of NAND flash. There are several types of FTL, but page mapped FTL shows better performance than others. But its usefulness is limited because of its large memory footprint for the mapping table. For example, 64MB memory space is required only for the mapping table for a 64GB MLC SSD. In this paper, we propose a novel caching scheme for the mapping table. By using the mapping-table-meta-data we construct a fully associative cache, and translate the address within O(1) time. The simulation results show more than 80 hit ratio with 32KB cache and 90% with 512KB cache. The overall memory footprint was only 1.9% of 64MB. The time overhead of cache miss was measured lower than 2% for most workload.

Research for the Element to Analyze the Performance of Modern-Web-Browser Based Applications (모던 웹 브라우저(Modern-Web-Browser) 기반 애플리케이션 성능분석을 위한 요소 연구)

  • Park, Jin-tae;Kim, Hyun-gook;Moon, Il-young
    • Proceedings of the Korean Institute of Information and Commucation Sciences Conference
    • /
    • 2018.10a
    • /
    • pp.278-281
    • /
    • 2018
  • The early Web technology was to show text information through a browser. However, as web technology advances, it is possible to show large amounts of multimedia data through browsers. Web technologies are being applied in a variety of fields such as sensor network, hardware control, and data collection and analysis for big data and AI services. As a result, the standard has been prepared for the Internet of Things, which typically controls a sensor via HTTP communication and provides information to users, by installing a web browser on the interface of the Internet of Things. In addition, the recent development of web-assembly enabled 3D objects, virtual/enhancing real-world content that could not be run in web browsers through a native language of C-class. Factors that evaluate the performance of existing Web applications include performance, network resources, and security. However, since there are many areas in which web applications are applied, it is time to revisit and review these factors. In this thesis, we will conduct an analysis of the factors that assess the performance of a web application. We intend to establish an indicator of the development of web-based applications by reviewing the analysis of each element, its main points, and its needs to be supplemented.

  • PDF

Risk based policy at big data era: Case study of privacy invasion (빅 데이터 시대 위험기반의 정책 - 개인정보침해 사례를 중심으로 -)

  • Moon, Hyejung;Cho, Hyun Suk
    • Informatization Policy
    • /
    • v.19 no.4
    • /
    • pp.63-82
    • /
    • 2012
  • The world's best level of ICT(Information, Communication and Technology) infrastructure has experienced the world's worst level of ICT accident in Korea. The number of major accidents of privacy invasion has been three times larger than the total number of Internet user of Korea. The cause of the severe accident was due to big data environment. As a result, big data environment has become an important policy agenda. This paper has conducted analyzing the accident case of data spill to study policy issues for ICT security from a social science perspective focusing on risk. The results from case analysis are as follows. First, ICT risk can be categorized 'severe, strong, intensive and individual'from the level of both probability and impact. Second, strategy of risk management can be designated 'avoid, transfer, mitigate, accept' by understanding their own culture type of relative group such as 'hierarchy, egalitarianism, fatalism and individualism'. Third, personal data has contained characteristics of big data such like 'volume, velocity, variety' for each risk situation. Therefore, government needs to establish a standing organization responsible for ICT risk policy and management in a new big data era. And the policy for ICT risk management needs to balance in considering 'technology, norms, laws, and market'in big data era.

  • PDF