• 제목/요약/키워드: huge data

검색결과 1,411건 처리시간 0.024초

3-Dimensional Tiling Technique to Process Huge Size High Resolution Satellite Image Seamlessly and Rapidly

  • Kim, Jun-Chul;Jung, Chan-Gyu;Kim, Moon-Gyu
    • 대한원격탐사학회지
    • /
    • 제23권5호
    • /
    • pp.375-383
    • /
    • 2007
  • This paper presents the method to provide a fast service for user in image manipulation such as zooming and panning of huge size high resolution satellite image(e.g. Giga bytes per scene). The proposed technique is based on the hierarchical structure that has 3D-Tiling in horizontal and vertical direction to provide the image service more effectively than 2D-Tiling technique in the past does. The essence of the proposed technique is to create tiles of optimum level in real time on the basis of current displaying area, which change as user manipulates huge image. Consequently, this technique provides seamless service, and will be very powerful and useful for manipulation of images of huge size without data conversion.

유전적 프로그래밍과 SOM을 결합한 개선된 선박 설계용 데이터 마이닝 시스템 개발 (Development of Data Mining System for Ship Design using Combined Genetic Programming with Self Organizing Map)

  • 이경호;박종훈;한영수;최시영
    • 한국CDE학회논문집
    • /
    • 제14권6호
    • /
    • pp.382-389
    • /
    • 2009
  • Recently, knowledge management has been required in companies as a tool of competitiveness. Companies have constructed Enterprise Resource Planning(ERP) system in order to manage huge knowledge. But, it is not easy to formalize knowledge in organization. We focused on data mining system by genetic programming(GP). Data mining system by genetic programming can be useful tools to derive and extract the necessary information and knowledge from the huge accumulated data. However when we don't have enough amounts of data to perform the learning process of genetic programming, we have to reduce input parameter(s) or increase number of learning or training data. In this study, an enhanced data mining method combining Genetic Programming with Self organizing map, that reduces the number of input parameters, is suggested. Experiment results through a prototype implementation are also discussed.

데이터 스트림 환경에서 데이터 완전도 보장을 위한 과부하 예측 부하 분산 기법 (Load balancing method of overload prediction for guaranteeing the data completeness in data stream)

  • 김영기;신숭선;백성하;이동욱;김경배;배해영
    • 한국멀티미디어학회논문지
    • /
    • 제12권9호
    • /
    • pp.1242-1251
    • /
    • 2009
  • 유비쿼터스 환경에서 데이터 스트림 관리 시스템(Data Stream Management System: DSMS)은 수많은 센서로부터 생성되는 대량의 데이터 스트림을 처리한다. 기존의 시스템은 처리 능력 이상의 데이터 스트림이 입력되면 데이터의 일부를 제거하여 적정 부하를 유지하는 부하 제한 기법(Load Shedding)을 사용한다. 부하 제한 기법은 입력되는 데이터의 일부를 의도적으로 손실하여 데이터 완전도(Data Completeness)가 감소하기 때문에 처리 결과의 신뢰도 또한 감소한다. 따라서 본 논문에서는 시스템 처리 능력 이상의 데이터 스트림 입력 시 데이터 완전도 보장을 위한 과부하 예측 부하 분산 기법을 제안한다. 제안 기법은 데이터 손실이 예상되는 부하 시점을 미리 예측하고 예측된 부하 시점에 도달 시 부하를 분산하여 데이터 손실을 감소시킨다. 본 논문에서는 기존의 부하 제한 기법과의 비교 실험을 통해 제안 기법의 성능을 평가한다.

  • PDF

모바일매핑시스템을 이용한 도로표지판 자동 추출에 관한 연구 (Automatic Identification of Road Sign in Mobile Mapping System)

  • 정재승;정동훈;김병국;성정곤
    • 한국측량학회:학술대회논문집
    • /
    • 한국측량학회 2007년도 춘계학술발표회 논문집
    • /
    • pp.221-224
    • /
    • 2007
  • MMS(Mobile Mapping System) generates a efficient image data for mapping and facility management. However, this image data of MMS has many difficulties in a practical use because of huge data volume. Therefore the important information likes road sign post must be extracted from huge MMS image data. In Korea, there is the HMS(Highway Management System) to manage a national road that acquire the line and condition of road from the MMS images. In the HMS each road sign information is manually inputted by the keyboard from moving MMS. This manually passive input way generate the error like inaccurate position, mistaking input in this research we developed the automatic road sign identifying technique using the image processing and the direct geo-referencing by GPS/INS data. This development brings not only good flexibility for field operations, also efficient data processing in HMS.

  • PDF

GIS에서 대용량 파일을 위한 대용량 공유 디스크 파일시스템의 메타데이터 구조 (Metadata Structrues of Huge Shared Disk File System for Large Files in GIS)

  • 김경배;이용주;박춘서;신범주
    • Spatial Information Research
    • /
    • 제10권1호
    • /
    • pp.93-106
    • /
    • 2002
  • 기존의 과일시스템은 소형의 과일을 효과적으로 저장하고 관리하기 위해서 설계되었다. 따라서 기존의 유닉스나 리눅스와 같은 과일 시스템은 지리정보시스템에서 발생되는 대용량의 지리정보 데이터를 효과적으로 처리하는 것은 어렵다. 본 논문에서는 지리정보시스템에서 발생되는 기가에서 테라바이트의 대용량 데이터 파일을 저장하기 위한 효과적인 메타데이터 구조 및 관리 기법을 제안한다. 제안된 기법에서는 대용량 파일을 저장하기 위해 동적 다단계 기법을 사용하고 있으며, 대용량의 파일 시스템을 제공하기 위하여 동적 비트맵 기법을 사용한다 본 논문에서 제안된 기법은 SAN 환경에서의 대용량 공유 디스크 파일시스템인 SANtopia에서 구현되었다.

  • PDF

스키마 변화에 따른 EPCIS의 효과 분석 및 검증에 관한 연구 (A Study on Analyzing the Effects and Verification of EPCIS by Influences of Shema)

  • 이종석
    • 대한안전경영과학회지
    • /
    • 제15권1호
    • /
    • pp.209-216
    • /
    • 2013
  • RFID/USN, are recognized as the new growth engines for the future, regardless of the advanced and developing countries. RFID, in particular, already entered practical stage by global companies. EPCIS, which is the one of the EPC golabal network components, makes a huge load on the system due to the large amount of entering data by following events. In this study, a data model developed based on ER-WIN by collecting four kind information that occurred in the EPC global networks. This model supports both in processing of high performance and huge capacity of data by a considerable storage capacity and input speed. A simulation was developed in order to verify the performance. Each model tested several times and results were compared.

Is Hepatectomy for Huge Hepatocellular Carcinoma (≥10cm in Diameter) Safe and Effective? A Single-center Experience

  • Yang, Jian;Li, Chuan;Wen, Tian-Fu;Yan, Lu-Nan;Li, Bo;Wang, Wen-Tao;Yang, Jia-Yin;Xu, Ming-Qing
    • Asian Pacific Journal of Cancer Prevention
    • /
    • 제15권17호
    • /
    • pp.7069-7077
    • /
    • 2014
  • Background: This retrospective study aimed to validate the safety and effectiveness of hepatectomy for huge hepatocellular carcinoma (HCC). Materials and Methods: Data of patients who underwent hepatectomy for HCC between January 2006 and December 2012 were reviewed. The patients were divided into three groups: huge HCC(${\geq}10cm$ in diameter), large HCC(${\geq}5$ but<10 cm in diameter) and small HCC(<5cm in diameter). Results: Characteristics of pre-operative patients in all three groups were homogeneously distributed except for alpha fetal protein (AFP)(p<0.001).The 30, 60, 90-day post-operative mortality rates were not different among the three groups (p=0.785, p=0.560, and p=0.549). Laboratory data at 1, 3, and 7 days after surgery also did not vary. The 5-year overall survival (OS) and 5-year disease-free survival (DFS) rates in the huge and large HCC groups were lower than that of the small HCC group (OS: 32.5% vs 36.3% vs 71.2%, p=0.000; DFS: 20.0% vs 24.8% vs 40.7%, p=0.039), but there was no difference between the huge and large HCC groups (OS: 32.5% vs 36.3%, p=0.667; DFS: 20.0% vs 24.8%, p=0.540). In multivariate analysis, five independent poor prognostic factors that affected OS were significantly associated with worse survival (p<0.05), namely, AFP level, macrovascular invasion, Edmondsone Steiner grade, surgical margin and Ishak score. AFP level, macrovascular invasion, microvascular invasion, and surgical margin influenced disease-free survival independently (p<0.05). Conclusions: The safety of hepatectomy for huge HCC is similar to that for large and small HCC; and this approach for huge HCC may achieve similar long-term survival and disease-free survival as for large HCC.

Forecasting Symbolic Candle Chart-Valued Time Series

  • Park, Heewon;Sakaori, Fumitake
    • Communications for Statistical Applications and Methods
    • /
    • 제21권6호
    • /
    • pp.471-486
    • /
    • 2014
  • This study introduces a new type of symbolic data, a candle chart-valued time series. We aggregate four stock indices (i.e., open, close, highest and lowest) as a one data point to summarize a huge amount of data. In other words, we consider a candle chart, which is constructed by open, close, highest and lowest stock indices, as a type of symbolic data for a long period. The proposed candle chart-valued time series effectively summarize and visualize a huge data set of stock indices to easily understand a change in stock indices. We also propose novel approaches for the candle chart-valued time series modeling based on a combination of two midpoints and two half ranges between the highest and the lowest indices, and between the open and the close indices. Furthermore, we propose three types of sum of square for estimation of the candle chart valued-time series model. The proposed methods take into account of information from not only ordinary data, but also from interval of object, and thus can effectively perform for time series modeling (e.g., forecasting future stock index). To evaluate the proposed methods, we describe real data analysis consisting of the stock market indices of five major Asian countries'. We can see thorough the results that the proposed approaches outperform for forecasting future stock indices compared with classical data analysis.

분산형 데이터마이닝 구현을 위한 의사결정나무 모델 전송 기술 (The Transfer Technique among Decision Tree Models for Distributed Data Mining)

  • 김충곤;우정근;백성욱
    • 디지털콘텐츠학회 논문지
    • /
    • 제8권3호
    • /
    • pp.309-314
    • /
    • 2007
  • 분산형 데이터마이닝을 위해 의사결정나무 알고리즘은 분산형 협업 환경에 적합하도록 변환되어야 한다. 본 논문에서 제시된 분산형 데이터마이닝 시스템은 각각의 사이트에서 부분적인 데이터를 위한 데이터마이닝 작업을 수행할 수 있는 에이전트와 여러 에이전트들의 협업을 통해 최종적인 의사결정나무 모델을 완성할 수 있도록 에이전트들 간의 통신을 중재하는 미디에이터로 구성되어 있다. 분산형 데이터마이닝의 장점 중에 하나는 여러 사이트에 분산되어 있는 대량의 데이터를 분산 처리하므로 데이터마이닝의 소요시간을 현저하게 줄일 수 있다는 점이다. 그러나 각 사이트들에 존재하고 있는 에이전트들 간의 통신에 부하가 과도하게 걸린다면, 효율적인 시스템으로의 활용도가 낮아질 것 이다. 본 논문은 에이전트들 간에 의사결정나무 모델의 전송량을 최소로 할 수 있는 방법론에 초점을 맞추었다.

  • PDF