• Title/Summary/Keyword: Journal repository

Search Result 987, Processing Time 0.025 seconds

Apache NiFi-based ETL Process for Building Data Lakes (데이터 레이크 구축을 위한 Apache NiFi기반 ETL 프로세스)

  • Lee, Kyoung Min;Lee, Kyung-Hee;Cho, Wan-Sup
    • The Journal of Bigdata
    • /
    • v.6 no.1
    • /
    • pp.145-151
    • /
    • 2021
  • In recent years, digital data has been generated in all areas of human activity, and there are many attempts to safely store and process the data to develop useful services. A data lake refers to a data repository that is independent of the source of the data and the analytical framework that leverages the data. In this paper, we designed a tool to safely store various big data generated by smart cities in a data lake and ETL it so that it can be used in services, and a web-based tool necessary to use it effectively. Implement. A series of processes (ETLs) that quality-check and refine source data, store it safely in a data lake, and manage it according to data life cycle policies are often significant for costly infrastructure and development and maintenance. It is a labor-intensive technology. The mounting technology makes it possible to set and execute ETL work monitoring and data life cycle management visually and efficiently without specialized knowledge in the IT field. Separately, a data quality checklist guide is needed to store and use reliable data in the data lake. In addition, it is necessary to set and reserve data migration and deletion cycles using the data life cycle management tool to reduce data management costs.

A reuse recommendation framework of artifacts based on task similarity to improve R&D performance (연구개발 생산성 향상을 위한 태스크 유사도 기반 산출물 재사용 추천 프레임워크)

  • Nam, Seungwoo;Daneth, Horn;Hong, Jang-Eui
    • Journal of Convergence for Information Technology
    • /
    • v.9 no.2
    • /
    • pp.23-33
    • /
    • 2019
  • Research and development(R&D) activities consist of analytical survey and state-of-the-art report writing for technical information. As R & D activities become more concrete, it often happens that they refer to related technical documents that were created in previous steps or created in previous similar projects. This paper proposes a research-task based reuse recommendation framework(RTRF), which is a reuse recommendation system that enables researchers to efficiently reuse the existing artifacts. In addition to the existing keyword-based retrieval and reuse, the proposed framework also provides reusable information that researchers may need by recommending reusable artifacts based on task similarity; other developers who have a similar task to the researcher's work can recommend reusable documents. A case study was performed to show the researchers' efficiency in the process of writing the technology trend report by reusing existing documents. When reuse is performed using RTRF, it can be seen that documents of different stages or other research fields are reused more frequently than when RTRF is not used. The RTRF may contribute to the efficient reuse of the desired artifacts among huge amount of R&D documents stored in the repository.

A proposal on a proactive crawling approach with analysis of state-of-the-art web crawling algorithms (최신 웹 크롤링 알고리즘 분석 및 선제적인 크롤링 기법 제안)

  • Na, Chul-Won;On, Byung-Won
    • Journal of Internet Computing and Services
    • /
    • v.20 no.3
    • /
    • pp.43-59
    • /
    • 2019
  • Today, with the spread of smartphones and the development of social networking services, structured and unstructured big data have stored exponentially. If we analyze them well, we will get useful information to be able to predict data for the future. Large amounts of data need to be collected first in order to analyze big data. The web is repository where these data are most stored. However, because the data size is large, there are also many data that have information that is not needed as much as there are data that have useful information. This has made it important to collect data efficiently, where data with unnecessary information is filtered and only collected data with useful information. Web crawlers cannot download all pages due to some constraints such as network bandwidth, operational time, and data storage. This is why we should avoid visiting many pages that are not relevant to what we want and download only important pages as soon as possible. This paper seeks to help resolve the above issues. First, We introduce basic web-crawling algorithms. For each algorithm, the time-complexity and pros and cons are described, and compared and analyzed. Next, we introduce the state-of-the-art web crawling algorithms that have improved the shortcomings of the basic web crawling algorithms. In addition, recent research trends show that the web crawling algorithms with special purposes such as collecting sentiment words are actively studied. We will one of the introduce Sentiment-aware web crawling techniques that is a proactive web crawling technique as a study of web crawling algorithms with special purpose. The result showed that the larger the data are, the higher the performance is and the more space is saved.

A Study on the Conceptual Design of Integrated Management System for Public SW Project Information (공공 소프트웨어(SW) 사업정보 통합 관리체계의 개념적 설계에 관한 연구)

  • Shin, Kitae;Park, Chankwon
    • The Journal of Society for e-Business Studies
    • /
    • v.24 no.2
    • /
    • pp.199-216
    • /
    • 2019
  • The public SW market is 3 trillion won, which is less than 10% of the total SW market. However, due to the nature of the domestic market, it is an important market with a relatively large impact on small and medium-sized software companies. In this market, government is operating the Public SW Project Demand Forecasting System in order to support the marketing activities of small and medium sized SW companies and establish a fair market order. The current system has limitations such as lack of user convenience, insufficient analysis capability and less business connection. This study was conducted to identify the problems of these systems and to propose a new system for improving the convenience of users and expanding the information utilization of SMEs. To this end, we analyzed the requirements of each stakeholder. We proposed the 2-phased forecasting cycle, the management cycle, and the system life cycle of public SW projects and created a unified identifier (UID) so that the information of those projects can be identified and linked among them. As a result, an integrated reference model of project information management based on system life cycle was developed, which can explain the demand forecasting and project information, and the improved processes was also designed to implement them. Through the result of this study, it is expected that integrated management of public SW projects will be possible.

A study on configuring deployment of digital repositories for the archives management systems (대량기록물 처리를 위한 영구기록물관리시스템의 디지털저장소 배치형상 연구)

  • Yim, Jin-Hee;Lee, Dae-Wook
    • The Korean Journal of Archival Studies
    • /
    • no.32
    • /
    • pp.177-217
    • /
    • 2012
  • The National Archives of Korea(NAK) has a mission to ingest large-scaled digital records and information from a number of different government agencies annually from 2015. There are important issues related to the digital records and information transfer between NAK and agencies, and one of them is how to configure deployment of digital repositories for the archives management systems. The purpose of this paper is to offer the way to design it by examining the checkpoints through the whole life cycle of digital records and information in the archives management systems and calculating the amount of ingested digital records and information to the systems in 2015 and deploying the digital repositories configured according to the amount the records and information. Firstly, this paper suggests that the archives management systems in NAK should be considered and examined into at least three different parts called Ingest tier, Preservation tier and Access tier in aspects to the characteristics of the flow and process of the digital records and information. Secondly, as a results of the calculation the amount of the digital records and information ingested to the archives management systems in 2015 is sum up to around 2.5 Tera bytes. This research draws several requirements related to the large-scaled data and bulk operations which should be satisfied by the database or database management system implemented on to the archives management systems. Thirdly, this paper configures digital repositories deployment according to the characteristics of the three tiers respectively. This research triggers discussion in depth and gives specific clues about how to design the digital repositories in the archives management systems for preparing the year of 2015.

Revisiting Archical Appraisal Theories for their Application to Community Archives (공동체 아카이브를 위한 기록평가론의 재조명)

  • Seol, Moon-Won;Kim, Young
    • The Korean Journal of Archival Studies
    • /
    • no.48
    • /
    • pp.210-252
    • /
    • 2016
  • Community creates, receives and preserves the records, which made the community members and the entire society remember their history. As for community archives, archival appraisal is very political activity because appraisal of community records means whose memory remain alive in history. This study aims to analyze archival appraisal theories from the perspective of community and community archives, and suggest appraisal model for community archives. This study begins with examining the meaning of community archives and appraisal related issues including; i) community identity and independence of archives, ii) struggle of memory and multiple narratives, iii) uniqueness of each community and its archives, and iv) community archives as memory process and social inclusion. At the next stage, it deals with the archival theories from Schellenberg's archival values theory to macro-appraisal, to investigate possible application of theories for community records appraisal. It finds that the societal approach of macro-appraisal have advantage to appraise the community records. This study finally suggests the appraisal model for community archives by modifying the macro-appraisal components as well as by complying the principles of community archives. The model consists of the purpose and object of appraisal, principle and basis of valuation, and cooperation model between mainstream repository and community.

Development of Chemical and Biological Decontamination Technology for Radioactive Liquid Wastes and Feasibility Study for Application to Liquid Waste Management System in APR1400 (액체방사성폐기물에 대한 화학적, 생물학적 제염기술 개발 및 APR1400 액체폐기물관리계통 적용을 위한 타당성 연구)

  • Son, YoungJu;Lee, Seung Yeop;Jung, JaeYeon;Kim, Chang-Lak
    • Journal of Nuclear Fuel Cycle and Waste Technology(JNFCWT)
    • /
    • v.17 no.1
    • /
    • pp.59-73
    • /
    • 2019
  • A decontamination technology for radioactive liquid wastes was newly developed and hypothetically applied to the liquid waste management system (LWMS) of the nuclear power plant (NPP) to evaluate its decontamination efficacy for the purpose of the fundamental reduction of spent resins. The basic principle of the developed technology is to convert major radionuclide ions in the liquid wastes into inorganic crystal minerals via chemical or biological techniques. In a laboratory batch experiment, the biological method selectively removed more than 80% of cesium within 24 hours, and the chemical method removed more than 95% of cesium. Other major nuclides (Co, Ni, Fe, Cr, Mn, Eu), which are commonly present in nuclear radioactive liquid wastes, were effectively scavenged by more than 99%. We have designed a module including the new technology that could be hypothetically installed between the reverse osmosis (R/O) package and the organic ion-exchange resin in the LWMS of the APR1400 reactor. From a technical evaluation for the virtual installation, we found that more than 90% of major radionuclides in the radioactive liquid wastes were selectively removed, resulting in a large volume reduction of spent resins. This means that if the new technology is commercialized in the future, it could possibly provide drastic cost reduction and significant extension of the life of resins in the management of spent resins, consequently leading to delay the saturation time of the Wolsong repository.

Evaluation of Soil-Water Characteristic Curve for Domestic Bentonite Buffer (국내 벤토나이트 완충재의 함수특성곡선 평가)

  • Yoon, Seok;Jeon, Jun-Seo;Lee, Changsoo;Cho, Won-Jin;Lee, Seung-Rae;Kim, Geon-Young
    • Journal of Nuclear Fuel Cycle and Waste Technology(JNFCWT)
    • /
    • v.17 no.1
    • /
    • pp.29-36
    • /
    • 2019
  • High-level radioactive waste (HLW) such as spent fuel is inevitably produced when nuclear power plants are operated. A geological repository has been considered as one of the most adequate options for the disposal of HLW, and it will be constructed in host rock at a depth of 500~1,000 meters below ground level with the concept of an engineered barrier system (EBS) and a natural barrier system. The compacted bentonite buffer is one of the most important components of the EBS. As the compacted bentonite buffer is located between disposal canisters with spent fuel and the host rock, it can restrain the release of radionuclides and protect canisters from the inflow of groundwater. Because of inflow of groundwater into the compacted bentonite buffer, it is essential to investigate soil-water characteristic curves (SWCC) of the compacted bentonite buffer in order to evaluate the entire safety performance of the EBS. Therefore, this paper conducted laboratory experiments to analyze the SWCC for a Korean Ca-type compacted bentonite buffer considering dry density, confined or unconfined condition, and drying or wetting path. There was no significant difference of SWCC considering dry density under unconfined condition. Furthermore, it was found that there was higher water suction in unconfined condition that in confined condition, and higher water suction during drying path than during wetting path.

Nonlinear Structural Analysis of the Spent Nuclear Fuel Disposal Canister Subjected to an Accidental Drop and Ground Impact Event (추락낙하 사고 시 지면과 충돌하는 고준위폐기물 처분용기의 비선형구조해석)

  • Kwon, Young-Joo
    • Journal of the Computational Structural Engineering Institute of Korea
    • /
    • v.32 no.2
    • /
    • pp.75-86
    • /
    • 2019
  • The biggest obstacle in the nuclear power generation is the high level radioactive waste such as the spent nuclear fuel. High level radioactivities and generated heat make the safe treatment of the spent nuclear fuel very difficult. Nowadays, the only treatment method is a deep geological disposal technology. This paper treats the structural safe design problem of the spent nuclear fuel disposal canister which is one of the core technologies of the deep geological disposal technology. Especially, this paper executed the nonlinear structural analysis for the stresses and deformations occurring in the canister due to the impulsive force applied to the spent nuclear fuel disposal canister in the case of an accidental drop and ground impact event from the transportation vehicle in the repository. The main content of the analysis is about that the impulsive force is obtained using the commercial rigid body dynamic analysis computer code, RecurDyn, and the stress and deformation caused by this impulsive force are obtained using the commercial finite element static structural analysis computer code, NISA. The analysis results show that large stresses and deformations may occur in the canister, especially in the rid or the bottom of the canister, due to the impulsive force occurring during the collision impact period.

An Influence Analysis on the Gap Space of an Engineered Barrier for an HLW Repository (고준위폐기물처분장 공학적방벽의 갭 공간이 미치는 영향 분석)

  • Yoon, Seok;Lee, Changsoo;Kim, Min-Jun
    • Journal of the Korean Geotechnical Society
    • /
    • v.37 no.4
    • /
    • pp.19-26
    • /
    • 2021
  • The high-level radioactive waste (HLW) produced from nuclear power plants is disposed in a rock-mass at a depth of hundreds meters below the ground level. Since HLW is very dangerous to human being, it must be disposed of safely by the engineered barrier system (EBS). The EBS consists of a disposal canister, backfill material, buffer material, and so on. When the components of EBS are installed, gaps inevitably exist not only between the rock-mass and buffer material but also between the canister and buffer material. The gap can reduce water-retarding capacity and heat release efficiency of the buffer material, so it is necessary to investigate properties of gap-filling materials and to analyze gap spacing effect. Furthermore, there has been few researches considering domestic disposal system compared to overseas researches. In this reason, this research derived the peak temperature of the bentonite buffer material considering domestic disposal system based on the numerical analysis. The gap between the canister and buffer material had a minor effect on the peak temperature of the bentonite buffer material, but there was 40% difference of the peak temperature of the bentonite buffer material because of the gap existence between the buffer material and rock mass.