• Title/Summary/Keyword: in-memory computing

Search Result 766, Processing Time 0.029 seconds

A Study on Optimum Coding Method for Correlation Processing of Radio Astronomy (전파천문 상관처리를 위한 최적 코딩 방법에 관한 연구)

  • Shin, Jae-Sik;Oh, Se-Jin;Yeom, Jae-Hwan;Roh, Duk-Gyoo;Chung, Dong-Kyu;Oh, Chung-Sik;Hwang, Ju-Yeon;So, Yo-Hwan
    • Journal of the Institute of Convergence Signal Processing
    • /
    • v.16 no.4
    • /
    • pp.139-148
    • /
    • 2015
  • In this paper, the optimum coding method is proposed by using open library in order to improve the performance of a software correlator developed for Korea-Japan Joint VLBI Correlator(KJJVC). The correlation system for VLBI observing system is generally implemented with hardware using ASIC or FPGA because the computational quantity is increased geometrically according to the participated observatory number. However, the software correlation system is recently constructed at a massive server such as a cluster using software according to the development of computing power. Since VLBI correlator implemented with hardware is able to conduct data processing with real-time or quasi real-time compared with mostly observational time, software correlation has to perform optimal data processing in coding work so as to have the same performance as that of the hardware. Therefore, in this paper, the experimental comparison was conducted by open-source based fftw library released in FFT processing stage, which is the most important part of the correlator system for performing optimum coding work in software development phase, such as general method using fftw library or methods using SSE(Streaming SIMD Extensions), shared memory, or OpenMP, and method using merged techniques listed above. Through the experimental results, the proposed optimum coding method for improving the performance of developed software correlator using fftw library, shared memory and OpenMP is effectively confirmed by reducing correlation time compared with conventional method.

Design and Implementation of Distributed In-Memory DBMS-based Parallel K-Means as In-database Analytics Function (분산 인 메모리 DBMS 기반 병렬 K-Means의 In-database 분석 함수로의 설계와 구현)

  • Kou, Heymo;Nam, Changmin;Lee, Woohyun;Lee, Yongjae;Kim, HyoungJoo
    • KIISE Transactions on Computing Practices
    • /
    • v.24 no.3
    • /
    • pp.105-112
    • /
    • 2018
  • As data size increase, a single database is not enough to serve current volume of tasks. Since data is partitioned and stored into multiple databases, analysis should also support parallelism in order to increase efficiency. However, traditional analysis requires data to be transferred out of database into nodes where analytic service is performed and user is required to know both database and analytic framework. In this paper, we propose an efficient way to perform K-means clustering algorithm inside the distributed column-based database and relational database. We also suggest an efficient way to optimize K-means algorithm within relational database.

Performance Comparison of Python and Scala APIs in Spark Distributed Cluster Computing System (Spark 기반에서 Python과 Scala API의 성능 비교 분석)

  • Ji, Keung-yeup;Kwon, Youngmi
    • Journal of Korea Multimedia Society
    • /
    • v.23 no.2
    • /
    • pp.241-246
    • /
    • 2020
  • Hadoop is a framework to process large data sets in a distributed way across clusters of nodes. It has been a popular platform to process big data, but in recent years, other platforms became competitive ones depending on the characteristics of the application. Spark is one of distributed platforms to enable real-time data processing and improve overall processing performance over Hadoop by introducing in-memory processing instead of disk I/O. Whereas Hadoop is designed to work on Java and data analysis is processed using Java API, Spark provides a variety of APIs with Scala, Python, Java and R. In this paper, the goal is to find out whether the APIs of different programming languages af ect the performances in Spark. We chose two popular APIs: Python and Scala. Python is easy to learn and is used in AI domain in a wide range. Scala is a programming language with advantages of parallelism. Our experiment shows much faster processing with Scala API than Python API. For the performance issues on AI-based analysis, further study is needed.

A Study on Performance Enhancement of RFID Anti-Collision Protocols (RFID 충돌방지 프로토콜의 성능 개선에 관한 연구)

  • Kim, Young-Beom
    • Journal of the Institute of Convergence Signal Processing
    • /
    • v.12 no.4
    • /
    • pp.281-285
    • /
    • 2011
  • One of the key issues in implementing RFID systems is to design anti-collision protocols for identifying all the tags in the interrogation zone of a RFID reader with the minimum identification delay. In this paper, Furthermore, in designing such protocols, the limited resources in tags and readers in terms of memory and computing capability should be fully taken into consideration. we first investigate two typical RFID anti-collision algorithms, namely RFID Gen2 Q algorithm (accepted as the worldwide standard in industrial domain) and FAFQ algorithm including their drawbacks and propose a new RFID anti-collision algorithm, which can improve the performance of RFID systems in terms of tag identification time considerably. Further, we compared performance of the proposed algorithm with Q algorithm and FAFQ algorithm through computer simulation.

A Development of mobile broadcasting monitor for improving reliability on IP-TV Platform based on TIT (TIT 기반에 IP-TV 플랫폼의 신뢰성 향상을 위한 방송 모니터 개발)

  • Sso, Sang-Jin;Jin, Hyun-Joon;Park, Noh-Kyung
    • Journal of Internet Computing and Services
    • /
    • v.8 no.5
    • /
    • pp.59-66
    • /
    • 2007
  • In korea, TIT(Transport Information Technology) based IP-TV services have been provided in Saemaeul trains and some sections of subway trains, But the software systems for the service performed in alternated fashions and suffered from many problems such as suspension, memory leaking and overflow, These problems increased playback loss time and resulted in bad reliabilities, In this paper, a software TIT monitor is designed and implemented for Monitoring module and Reset module in physically poor environments, The designed system formalized monitoring time intervals for effective monitoring, Through the real experiments, playback time is improved in 7.2% comparing to existing system.

  • PDF

A Fast Algorithm for Korean Text Extraction and Segmentation from Subway Signboard Images Utilizing Smartphone Sensors

  • Milevskiy, Igor;Ha, Jin-Young
    • Journal of Computing Science and Engineering
    • /
    • v.5 no.3
    • /
    • pp.161-166
    • /
    • 2011
  • We present a fast algorithm for Korean text extraction and segmentation from subway signboards using smart phone sensors in order to minimize computational time and memory usage. The algorithm can be used as preprocessing steps for optical character recognition (OCR): binarization, text location, and segmentation. An image of a signboard captured by smart phone camera while holding smart phone by an arbitrary angle is rotated by the detected angle, as if the image was taken by holding a smart phone horizontally. Binarization is only performed once on the subset of connected components instead of the whole image area, resulting in a large reduction in computational time. Text location is guided by user's marker-line placed over the region of interest in binarized image via smart phone touch screen. Then, text segmentation utilizes the data of connected components received in the binarization step, and cuts the string into individual images for designated characters. The resulting data could be used as OCR input, hence solving the most difficult part of OCR on text area included in natural scene images. The experimental results showed that the binarization algorithm of our method is 3.5 and 3.7 times faster than Niblack and Sauvola adaptive-thresholding algorithms, respectively. In addition, our method achieved better quality than other methods.

Shortest Path Calculation Using Parallel Processor System (병력구조 전산기를 이용한 최단 경로 계산)

  • 서창진;이장규
    • The Transactions of the Korean Institute of Electrical Engineers
    • /
    • v.34 no.6
    • /
    • pp.230-237
    • /
    • 1985
  • Shortest path calculations for a large-scale network have to be performed using a decomposition techniqre, since the calculations require large memory size which increases by the square of the number of vertices in the network. Also, the calculation time increases by the cube of the number of vertices in the network. In the decomposition technique,the network is broken into a number of smaller size subnetworks for each of which shortest paths are computed. A union of the solutions provides the solution of the original network. In all of the decomposition algirithms developed up to now, boundary vertices which divide all the subnetworks have to be included in computing shortest paths for each subnetwork. In this paper, an improved algorithm is developed to reduce the number of boundary vertices to be engaged. In the algorithm, only those boundary vertices that are directly connected to the subnetwork are engaged. The algorithm is suitable for an application to real time computation using a parallel processor system which consists of a number of micro-computers or prcessors. The algorithm has been applied to a 39- vertex network and a 232-vertex network. The results show that it is efficient and has better performance than any other algorithms. A parallel processor system has been built employing an MZ-80 micro-computer and two Z-80 microprocessor kits. The former is used as a master processor and the latter as slave processors. The algorithm is embedded into the system and proven effective for real-time shortest path computations.

  • PDF

Assessment of computational performance for a vector parallel implementation: 3D probabilistic model discrete cracking in concrete

  • Paz, Carmen N.M.;Alves, Jose L.D.;Ebecken, Nelson F.F.
    • Computers and Concrete
    • /
    • v.2 no.5
    • /
    • pp.345-366
    • /
    • 2005
  • This work presents an assessment of the computational performance of a vector-parallel implementation of probabilistic model for concrete cracking in 3D. This paper shows the continuing efforts towards code optimization as reported in earlier works Paz, et al. (2002a,b and 2003). The probabilistic crack approach is based on the direct Monte Carlo method. Cracking is accounted by means of 3D interface elements. This approach considers that all nonlinearities are restricted to interface elements modeling cracks. The heterogeneity governs the overall cracking behavior and related size effects on concrete fracture. Computational kernels in the implementation are the inexact Newton iterative driver to solve the non-linear problem and a preconditioned conjugate gradient (PCG) driver to solve linearized equations, using an element by element (EBE) strategy to compute matrix-vector products. In particular the paper analyzes code behavior using OpenMP directives in parallel vector processors (PVP), such as the CRAY SV1 and CRAY T94. The impact of the memory architecture on code performance, and also some strategies devised to circumvent this issue are addressed by numerical experiment.

Data Processing Architecture for Cloud and Big Data Services in Terms of Cost Saving (비용절감 측면에서 클라우드, 빅데이터 서비스를 위한 대용량 데이터 처리 아키텍쳐)

  • Lee, Byoung-Yup;Park, Jae-Yeol;Yoo, Jae-Soo
    • The Journal of the Korea Contents Association
    • /
    • v.15 no.5
    • /
    • pp.570-581
    • /
    • 2015
  • In recent years, many institutions predict that cloud services and big data will be popular IT trends in the near future. A number of leading IT vendors are focusing on practical solutions and services for cloud and big data. In addition, cloud has the advantage of unrestricted in selecting resources for business model based on a variety of internet-based technologies which is the reason that provisioning and virtualization technologies for active resource expansion has been attracting attention as a leading technology above all the other technologies. Big data took data prediction model to another level by providing the base for the analysis of unstructured data that could not have been analyzed in the past. Since what cloud services and big data have in common is the services and analysis based on mass amount of data, efficient operation and designing of mass data has become a critical issue from the early stage of development. Thus, in this paper, I would like to establish data processing architecture based on technological requirements of mass data for cloud and big data services. Particularly, I would like to introduce requirements that must be met in order for distributed file system to engage in cloud computing, and efficient compression technology requirements of mass data for big data and cloud computing in terms of cost-saving, as well as technological requirements of open-source-based system such as Hadoop eco system distributed file system and memory database that are available in cloud computing.

XML Fragmentation for Resource-Efficient Query Processing over XML Fragment Stream (자원 효율적인 XML 조각 스트림 질의 처리를 위한 XML 분할)

  • Kim, Jin;Kang, Hyun-Chul
    • The KIPS Transactions:PartD
    • /
    • v.16D no.1
    • /
    • pp.27-42
    • /
    • 2009
  • In realizing ubiquitous computing, techniques of efficiently using the limited resource at client such as mobile devices are required. With a mobile device with limited amount of memory, the techniques of XML stream query processing should be employed to process queries over a large volume of XML data. Recently, several techniques were proposed which fragment XML documents into XML fragments and stream them for query processing at client. During query processing, there could be great difference in resource usage (query processing time and memory usage) depending on how the source XML documents are fragmented. As such, an efficient fragmentation technique is needed. In this paper, we propose an XML fragmentation technique whereby resource efficiency in query processing at client could be enhanced. For this, we first present a cost model of query processing over XML fragment stream. Then, we propose an algorithm for resource-efficient XML fragmentation. Through implementation and experiments, we showed that our fragmentation technique outperformed previous techniques both in processing time and memory usage. The contribution of this paper is to have made the techniques of query processing over XML fragment stream more feasible for practical use.