• Title/Summary/Keyword: data processing technique

Search Result 1,981, Processing Time 0.03 seconds

A study on Korean language processing using TF-IDF (TF-IDF를 활용한 한글 자연어 처리 연구)

  • Lee, Jong-Hwa;Lee, MoonBong;Kim, Jong-Weon
    • The Journal of Information Systems
    • /
    • v.28 no.3
    • /
    • pp.105-121
    • /
    • 2019
  • Purpose One of the reasons for the expansion of information systems in the enterprise is the increased efficiency of data analysis. In particular, the rapidly increasing data types which are complex and unstructured such as video, voice, images, and conversations in and out of social networks. The purpose of this study is the customer needs analysis from customer voices, ie, text data, in the web environment.. Design/methodology/approach As previous study results, the word frequency of the sentence is extracted as a word that interprets the sentence has better affects than frequency analysis. In this study, we applied the TF-IDF method, which extracts important keywords in real sentences, not the TF method, which is a word extraction technique that expresses sentences with simple frequency only, in Korean language research. We visualized the two techniques by cluster analysis and describe the difference. Findings TF technique and TF-IDF technique are applied for Korean natural language processing, the research showed the value from frequency analysis technique to semantic analysis and it is expected to change the technique by Korean language processing researcher.

Measurement of Size Distributions of Submicron Electrosprays Using a Freezing Method and an Image Processing Technique (냉각법 및 영상 처리기법을 이용한 서브마이크론 정전분무 액적의 크기분포 측정)

  • Ku, Bon-Ki;Kim, Sang-Soo;Kim, Yu-Dong
    • Proceedings of the KSME Conference
    • /
    • 2001.06e
    • /
    • pp.100-106
    • /
    • 2001
  • The size distributions of electrospray droplets from the Taylor cone in cone-jet mode are directly measured by using a freezing method and a transmission electron microscope (TEM) image processing technique. These results are compared with the data obtained by an aerodynamic size spectrometer (TSI Aerosizer DSP). The use of glycerol seeded with NaI and a freezing method make it possible to sample droplets with their original sizes preserved. Since pictures of droplets are taken with TEM with very low vapor pressure of the solution, evaporation is suppressed by freezing. For liquid flow rates below 1 nl/sec, the measured droplet diameters by the TEM image processing technique and the aerosizer are in the range of 0.25 to $0.32{\mu}m$ and 0.30 to $0.40{\mu}m$, respectively. Comparing the TEM data with the aerosizer measurements, it has been revealed that the TEM image processing technique can afford more accurate values of droplet size distributions in the submicron range of 0.1 to $0.4{\mu}m$.

  • PDF

LOSSLESS DATA COMPRESSION ON SAR DISPLAY IMAGES (SAR 디스플레이 영상을 위한 무손실 압축)

  • Lee, Tae-hee;Song, Woo-jin;Do, Dae-won;Kwon, Jun-chan;Yoon, Byung-woo
    • Proceedings of the IEEK Conference
    • /
    • 2001.09a
    • /
    • pp.117-120
    • /
    • 2001
  • Synthetic aperture radar (SAR) is a promising active remote sensing technique to obtain large terrain information of the earth in all-weather conditions. SAR is useful in many applications, including terrain mapping and geographic information system (GIS), which use SAR display images. Usually, these applications need the enormous data storage because they deal with wide terrain images with high resolution. So, compression technique is a useful approach to deal with SAR display images with limited storage. Because there is some indispensable data loss through the conversion of a complex SAR image to a display image, some applications, which need high-resolution images, cannot tolerate more data loss during compression. Therefore, lossless compression is appropriate to these applications. In this paper, we propose a novel lossless compression technique for a SAR display image using one-step predictor and block arithmetic coding.

  • PDF

A Data Structuring Technique for Performance Enhancement of Query Processing in the Data Warehouses (DW에서의 질의어처리 성능향상을 위한 데이터 구조화 방법)

  • Lee Deok Heun;Oh Mi Hwa;Cho Jae Hun;Choi In Soo
    • Journal of the Korea Society of Computer and Information
    • /
    • v.10 no.1 s.33
    • /
    • pp.7-14
    • /
    • 2005
  • An OLAP(On-Line Analytical Processing) system is the decision support tool with which a user can analyze the information interactively in the various aspects. However, the traditional existing construction of an OLAP system has the inefficiency Problem of increasing the processing time and cost caused by the use of complex MDX(Multidimensional Expressions) queries. In an attempt to solve this problem, a new concept of data structuring technique, where a unit column whose elements are all 1 is added to the fact table, was suggested. With the data structuring technique, we can reduce the processing time and cost in OLAP systems.

  • PDF

Image Processing Technique for Measuring the Static Displacement of Bridges from General Inspection Photograph (일반 점검사진에서 교량의 정적 변위 추출을 위한 영상처리기법)

  • Cho, Jun Sang;Huh, Young
    • KSCE Journal of Civil and Environmental Engineering Research
    • /
    • v.31 no.3A
    • /
    • pp.173-180
    • /
    • 2011
  • This paper aims to propose an image processing technique for measuring the static displacement of bridges from general inspection photograph; the color, shape, and spatial transformations of an arbitrary image stored in bridge management system database are used. This study is verified by using numerical analyses with experiments; the results demonstrate that the static displacement of bridges are measured by proposed technique. Moreover, this technique is able to obtain the static structural response of the bridge with changes in temperatures.

Designing of Dynamic Sensor Networks based on Meter-range Swarming Flight Type Air Nodes

  • Kang, Chul-Gyu;Kim, Dae-Hwan
    • Journal of information and communication convergence engineering
    • /
    • v.9 no.6
    • /
    • pp.625-628
    • /
    • 2011
  • Dynamic sensor network(DSN) technology which is based on swarming flight type air node offers analyzed and acquired information on target data gathered by air nodes in rotation flight or 3 dimension array flight. Efficient operation of dynamic sensor network based on air node is possible when problems of processing time, data transmission reliability, power consumption and intermittent connectivity are solved. Delay tolerant network (DTN) can be a desirable alternative to solve those problems. DTN using store-and-forward message switching technology is a solution to intermittent network connectivity, long and variable delay time, asymmetric data rates, and high error rates. However, all processes are performed at the bundle layer, so high power consumption, long processing time, and repeated reliability technique occur. DSN based on swarming flight type air node need to adopt store-and-forward message switching technique of DTN, the cancelation scheme of repeated reliability technique, fast processing time with simplified layer composition.

Data processing technique for data measured in MO image measurement system

  • Lee, Wongi;Lee, Hyoyeon;Yoo, Jaeun;Youm, Dojun
    • Progress in Superconductivity and Cryogenics
    • /
    • v.15 no.1
    • /
    • pp.25-28
    • /
    • 2013
  • We report processing technique in the MO image measurement system. Calibration procedure is not only considered to perpendicular field but also in-plane field. Current density and field profiles are obtained by Biot-savart law and inversion method. We show example of $(Gd,Y)_1Ba_2Cu_3O_{7-{\delta}}-BaZrO_3$ film that have tilted nano rod pinning centers about $13^{\circ}$ from the c-axis.

An Adequacy Based Test Data Generation Technique Using Genetic Algorithms

  • Malhotra, Ruchika;Garg, Mohit
    • Journal of Information Processing Systems
    • /
    • v.7 no.2
    • /
    • pp.363-384
    • /
    • 2011
  • As the complexity of software is increasing, generating an effective test data has become a necessity. This necessity has increased the demand for techniques that can generate test data effectively. This paper proposes a test data generation technique based on adequacy based testing criteria. Adequacy based testing criteria uses the concept of mutation analysis to check the adequacy of test data. In general, mutation analysis is applied after the test data is generated. But, in this work, we propose a technique that applies mutation analysis at the time of test data generation only, rather than applying it after the test data has been generated. This saves significant amount of time (required to generate adequate test cases) as compared to the latter case as the total time in the latter case is the sum of the time to generate test data and the time to apply mutation analysis to the generated test data. We also use genetic algorithms that explore the complete domain of the program to provide near-global optimum solution. In this paper, we first define and explain the proposed technique. Then we validate the proposed technique using ten real time programs. The proposed technique is compared with path testing technique (that use reliability based testing criteria) for these ten programs. The results show that the adequacy based proposed technique is better than the reliability based path testing technique and there is a significant reduce in number of generated test cases and time taken to generate test cases.

A study on the efficient early warning method using complex event processing (CEP) technique (복합 이벤트 처리기술을 적용한 효율적 재해경보 전파에 관한 연구)

  • Kim, Hyung-Woo;Kim, Goo-Soo;Chang, Sung-Bong
    • 한국정보통신설비학회:학술대회논문집
    • /
    • 2009.08a
    • /
    • pp.157-161
    • /
    • 2009
  • In recent years, there is a remarkable progress in ICTs (Information and Communication Technologies), and then many attempts to apply ICTs to other industries are being made. In the field of disaster managements, ICTs such as RFID (Radio Frequency IDentification) and USN (Ubiquitous Sensor Network) are used to provide safe environments. Actually, various types of early warning systems using USN are now widely used to monitor natural disasters such as floods, landslides and earthquakes, and also to detect human-caused disasters such as fires, explosions and collapses. These early warning systems issue alarms rapidly when a disaster is detected or an event exceeds prescribed thresholds, and furthermore deliver alarm messages to disaster managers and citizens. In general, these systems consist of a number of various sensors and measure real-time stream data, which requires an efficient and rapid data processing technique. In this study, an event-driven architecture (EDA) is presented to collect event effectively and to provide an alert rapidly. A publish/subscribe event processing method to process simple event is introduced. Additionally, a complex event processing (CEP) technique is introduced to process complex data from various sensors and to provide prompt and reasonable decision supports when many disasters happen simultaneously. A basic concept of CEP technique is presented and the advantages of the technique in disaster management are also discussed. Then, how the main processing methods of CEP such as aggregation, correlation, and filtering can be applied to disaster management is considered. Finally, an example of flood forecasting and early alarm system in which CEP is incorporated is presented It is found that the CEP based on the EDA will provide an efficient early warning method when disaster happens.

  • PDF

An Efficient Technique for Processing of Spatial Data Using GPU (GPU를 사용한 효율적인 공간 데이터 처리)

  • Lee, Jae-Il;Oh, Byoung-Woo
    • Spatial Information Research
    • /
    • v.17 no.3
    • /
    • pp.371-379
    • /
    • 2009
  • Recently, GPU (Graphics Processing Unit) has been improved rapidly on the need of speed for gaming. As a result, GPU contains multiple ALU (Arithmetic Logic Unit) for parallel processing of a lot of graphics data, such as transform, ray tracing, etc. Therefore, this paper proposed a technique for parallel processing of spatial data using GPU. Spatial data consists of multiple coordinates, and each coordinate contains value of x and y axis. To display spatial data graphics operations have to be processed to large amount of coordinates. Because the graphics operation is identical and coordinates are multiple data, SIMD (Single Instruction Multiple Data) parallel processing of GPU can be used for processing of spatial data to improve performance. This paper implemented SIMD parallel processing of spatial data using two kinds of SDK (Software Development Kit). CUDA and ATI Stream are used for NVIDIA and ATI GPU respectively. Experiments that measure time of calculation for graphics operations are carried out to observe enhancement of performance. Experimental result is reported that proposed method can enhance performance up to 1,162% for graphics operations. The proposed method that uses parallel processing with GPU for spatial data can be generally used to enhance performance for applications which deal with large amount of spatial data.

  • PDF