• Title/Summary/Keyword: 데이터베이스 성능

Search Result 1,856, Processing Time 0.034 seconds

A proposal on a proactive crawling approach with analysis of state-of-the-art web crawling algorithms (최신 웹 크롤링 알고리즘 분석 및 선제적인 크롤링 기법 제안)

  • Na, Chul-Won;On, Byung-Won
    • Journal of Internet Computing and Services
    • /
    • v.20 no.3
    • /
    • pp.43-59
    • /
    • 2019
  • Today, with the spread of smartphones and the development of social networking services, structured and unstructured big data have stored exponentially. If we analyze them well, we will get useful information to be able to predict data for the future. Large amounts of data need to be collected first in order to analyze big data. The web is repository where these data are most stored. However, because the data size is large, there are also many data that have information that is not needed as much as there are data that have useful information. This has made it important to collect data efficiently, where data with unnecessary information is filtered and only collected data with useful information. Web crawlers cannot download all pages due to some constraints such as network bandwidth, operational time, and data storage. This is why we should avoid visiting many pages that are not relevant to what we want and download only important pages as soon as possible. This paper seeks to help resolve the above issues. First, We introduce basic web-crawling algorithms. For each algorithm, the time-complexity and pros and cons are described, and compared and analyzed. Next, we introduce the state-of-the-art web crawling algorithms that have improved the shortcomings of the basic web crawling algorithms. In addition, recent research trends show that the web crawling algorithms with special purposes such as collecting sentiment words are actively studied. We will one of the introduce Sentiment-aware web crawling techniques that is a proactive web crawling technique as a study of web crawling algorithms with special purpose. The result showed that the larger the data are, the higher the performance is and the more space is saved.

Semi-automatic Construction of Learning Set and Integration of Automatic Classification for Academic Literature in Technical Sciences (기술과학 분야 학술문헌에 대한 학습집합 반자동 구축 및 자동 분류 통합 연구)

  • Kim, Seon-Wu;Ko, Gun-Woo;Choi, Won-Jun;Jeong, Hee-Seok;Yoon, Hwa-Mook;Choi, Sung-Pil
    • Journal of the Korean Society for information Management
    • /
    • v.35 no.4
    • /
    • pp.141-164
    • /
    • 2018
  • Recently, as the amount of academic literature has increased rapidly and complex researches have been actively conducted, researchers have difficulty in analyzing trends in previous research. In order to solve this problem, it is necessary to classify information in units of academic papers. However, in Korea, there is no academic database in which such information is provided. In this paper, we propose an automatic classification system that can classify domestic academic literature into multiple classes. To this end, first, academic documents in the technical science field described in Korean were collected and mapped according to class 600 of the DDC by using K-Means clustering technique to construct a learning set capable of multiple classification. As a result of the construction of the training set, 63,915 documents in the Korean technical science field were established except for the values in which metadata does not exist. Using this training set, we implemented and learned the automatic classification engine of academic documents based on deep learning. Experimental results obtained by hand-built experimental set-up showed 78.32% accuracy and 72.45% F1 performance for multiple classification.

Deep Learning Structure Suitable for Embedded System for Flame Detection (불꽃 감지를 위한 임베디드 시스템에 적합한 딥러닝 구조)

  • Ra, Seung-Tak;Lee, Seung-Ho
    • Journal of IKEEE
    • /
    • v.23 no.1
    • /
    • pp.112-119
    • /
    • 2019
  • In this paper, we propose a deep learning structure suitable for embedded system. The flame detection process of the proposed deep learning structure consists of four steps : flame area detection using flame color model, flame image classification using deep learning structure for flame color specialization, $N{\times}N$ cell separation in detected flame area, flame image classification using deep learning structure for flame shape specialization. First, only the color of the flame is extracted from the input image and then labeled to detect the flame area. Second, area of flame detected is the input of a deep learning structure specialized in flame color and is classified as flame image only if the probability of flame class at the output is greater than 75%. Third, divide the detected flame region of the images classified as flame images less than 75% in the preceding section into $N{\times}N$ units. Fourthly, small cells divided into $N{\times}N$ units are inserted into the input of a deep learning structure specialized to the shape of the flame and each cell is judged to be flame proof and classified as flame images if more than 50% of cells are classified as flame images. To verify the effectiveness of the proposed deep learning structure, we experimented with a flame database of ImageNet. Experimental results show that the proposed deep learning structure has an average resource occupancy rate of 29.86% and an 8 second fast flame detection time. The flame detection rate averaged 0.95% lower compared to the existing deep learning structure, but this was the result of light construction of the deep learning structure for application to embedded systems. Therefore, the deep learning structure for flame detection proposed in this paper has been proved suitable for the application of embedded system.

Construction of a Sub-catchment Connected Nakdong-gang Flood Analysis System Using Distributed Model (분포형 모형을 이용한 소유역 연계 낙동강 홍수해석시스템 구축)

  • Choi, Yun-Seok;Won, Young-Jin;Kim, Kyung-Tak
    • Proceedings of the Korea Water Resources Association Conference
    • /
    • 2018.05a
    • /
    • pp.202-202
    • /
    • 2018
  • 본 논문에서는 분포형 강우-유출 모형인 GRM(Grid based Rainfall-runoff Model)(최윤석, 김경탁, 2017)을 이용해서 낙동강 유역을 대상으로 대유역 홍수해석시스템을 구축하고, 유출해석을 위한 실행시간을 평가하였다. 유출모형은 낙동강의 주요 지류와 본류를 소유역으로 구분하여 모형을 구축하고, 각 소유역의 유출해석 결과를 실시간으로 연계할 수 있도록 하여 낙동강 전체 유역의 유출모형을 구축하였다. 이와 같이 하나의 대유역을 다수의 소유역시스템으로 분할하여 모형을 구축할 경우, 유출해석시스템 구성이 복잡해지는 단점이 있으나, 소유역별로 각기 다른 자료를 이용하여 다양한 해상도로 유출해석을 할 수 있으므로, 소유역별 특성에 맞는 유출모형 구축이 가능한 장점이 있다. 또한 각 소유역시스템은 별도의 프로세스로 계산이 진행되므로, 대유역을 고해상도로 해석하는 경우에도 계산시간을 단축할 수 있다. 본 연구에서는 낙동강 유역을 20개(본류 구간 3개, 1차 지류 13개, 댐상류 4개)의 소유역으로 분할하여 계산 시간을 검토하였으며, 최종적으로 21개(본류 구간 3개, 1차 지류 13개, 댐상류 5개)의 소유역으로 분할하여 유출해석시스템을 구축하였다. 댐 상류 유역은 댐하류와 유량전달이 없이 독립적으로 모의되고, 댐과 연결된 하류 유역은 관측 방류량을 상류단 하천의 경계조건으로 적용한다. 지류 유역은 본류 구간과 연결되고, 지류의 계산 유량은 본류와의 연결지점에 유량조건으로 실시간으로 입력된다. 이때 본류와 지류의 유량 연계는 데이터베이스를 매개로 하였다. 유출해석시스템의 성능을 평가하기 위해서 Microsoft 클라우드 서비스인 Azure를 이용하였다. 낙동강 유역을 20개 소유역으로 구성한 경우에서의 유출해석시스템의 속도 평가 결과 Azure virtual machine instance DS15 v2(OS : Windows Server 2012 R2, CPU : 2.4 GHz Intel $Xeon^{(R)}$ E5-2673 v3 20 cores)에서 1.5분이 소요 되었다. 계산시간 평가시 GRM은 'IsParallel=false' 옵션을 적용하였으며, 모의 기간은 24시간을 기준으로 하였다. 연구결과 분포형 모형을 이용한 대유역 유출해석시스템 구축이 가능했으며, 계산시간도 충분히 단축할 수 있었다. 또한 추가적인 CPU와 병렬계산을 적용할 경우, 계산시간은 더 단축될 수 있으며, 이러한 기법들은 분포형 모형을 이용한 대유역 유출해석시스템 구축시 유용하게 활용될 수 있을 것으로 판단된다.

  • PDF

Recombinant Production and Antimicrobial Activity of an Antimicrobial Model Peptide (Uu-ilys-CF) Derived from Spoon Worm Lysozyme, Uu-ilys (개불 라이소자임 유래 항균성 모델 펩타이드(Uu-ilys-CF)의 재조합 단백질 생산 및 항균 활성)

  • Oh, Hye Young;Go, Hye-Jin;Park, Nam Gyu
    • Journal of Life Science
    • /
    • v.31 no.1
    • /
    • pp.83-89
    • /
    • 2021
  • Uu-ilys, an i-type lysozyme from spoon worm (Urechis unicinctus), is an innate immune factor that plays an important role in the defense against pathogens. It also possesses non-enzymatic antibacterial activity. Thus, there is a possibility to develop an antimicrobial model peptide from Uu-ilys. In this study, we report the design, production, and antibacterial activity of an Uu-ilys analog that exhibits antibacterial activity. The Uu-ilys structure was fragmented according to its secondary structures to predict the regions with antimicrobial activity using antimicrobial peptide (AMP) prediction tools from different AMP databases. A peptide containing the C-terminal fragment was predicted to exert antimicrobial activity. The chosen fragment was designated as an Uu-ilys analog containing the C-terminal fragment, Uu-ilys-CF. To examine the possibility of developing an AMP using the sequence of Uu-ilys-CF, recombinant fusion protein (TrxA-Uu-ilys-CF) was produced in an expression system that was heterologous. The produced fusion protein was cleaved after methionine leaving Uu-ilys-CF free from the fusion protein. This was then isolated through high performance liquid chromatography and reverse phase column, CapCell-Pak C18. The antibacterial activity of Uu-ilys-CF against different microbial strains (four gram-positive, six gram-negative, and one fungal strain) were assessed through the ultrasensitive radial diffusion assay (URDA). Among the bacterial strains tested, Salmonella enterica was the most susceptible. While the fungal strain tested was not susceptible to Uu-ilys-CF, broad spectrum antibacterial activity was observed.

Abnormal Crowd Behavior Detection via H.264 Compression and SVDD in Video Surveillance System (H.264 압축과 SVDD를 이용한 영상 감시 시스템에서의 비정상 집단행동 탐지)

  • Oh, Seung-Geun;Lee, Jong-Uk;Chung, Yongw-Ha;Park, Dai-Hee
    • Journal of the Korea Institute of Information Security & Cryptology
    • /
    • v.21 no.6
    • /
    • pp.183-190
    • /
    • 2011
  • In this paper, we propose a prototype system for abnormal sound detection and identification which detects and recognizes the abnormal situations by means of analyzing audio information coming in real time from CCTV cameras under surveillance environment. The proposed system is composed of two layers: The first layer is an one-class support vector machine, i.e., support vector data description (SVDD) that performs rapid detection of abnormal situations and alerts to the manager. The second layer classifies the detected abnormal sound into predefined class such as 'gun', 'scream', 'siren', 'crash', 'bomb' via a sparse representation classifier (SRC) to cope with emergency situations. The proposed system is designed in a hierarchical manner via a mixture of SVDD and SRC, which has desired characteristics as follows: 1) By fast detecting abnormal sound using SVDD trained with only normal sound, it does not perform the unnecessary classification for normal sound. 2) It ensures a reliable system performance via a SRC that has been successfully applied in the field of face recognition. 3) With the intrinsic incremental learning capability of SRC, it can actively adapt itself to the change of a sound database. The experimental results with the qualitative analysis illustrate the efficiency of the proposed method.

A Study on Reducing Learning Time of Deep-Learning using Network Separation (망 분리를 이용한 딥러닝 학습시간 단축에 대한 연구)

  • Lee, Hee-Yeol;Lee, Seung-Ho
    • Journal of IKEEE
    • /
    • v.25 no.2
    • /
    • pp.273-279
    • /
    • 2021
  • In this paper, we propose an algorithm that shortens the learning time by performing individual learning using partitioning the deep learning structure. The proposed algorithm consists of four processes: network classification origin setting process, feature vector extraction process, feature noise removal process, and class classification process. First, in the process of setting the network classification starting point, the division starting point of the network structure for effective feature vector extraction is set. Second, in the feature vector extraction process, feature vectors are extracted without additional learning using the weights previously learned. Third, in the feature noise removal process, the extracted feature vector is received and the output value of each class is learned to remove noise from the data. Fourth, in the class classification process, the noise-removed feature vector is input to the multi-layer perceptron structure, and the result is output and learned. To evaluate the performance of the proposed algorithm, we experimented with the Extended Yale B face database. As a result of the experiment, in the case of the time required for one-time learning, the proposed algorithm reduced 40.7% based on the existing algorithm. In addition, the number of learning up to the target recognition rate was shortened compared with the existing algorithm. Through the experimental results, it was confirmed that the one-time learning time and the total learning time were reduced and improved over the existing algorithm.

Building a Korean conversational speech database in the emergency medical domain (응급의료 영역 한국어 음성대화 데이터베이스 구축)

  • Kim, Sunhee;Lee, Jooyoung;Choi, Seo Gyeong;Ji, Seunghun;Kang, Jeemin;Kim, Jongin;Kim, Dohee;Kim, Boryong;Cho, Eungi;Kim, Hojeong;Jang, Jeongmin;Kim, Jun Hyung;Ku, Bon Hyeok;Park, Hyung-Min;Chung, Minhwa
    • Phonetics and Speech Sciences
    • /
    • v.12 no.4
    • /
    • pp.81-90
    • /
    • 2020
  • This paper describes a method of building Korean conversational speech data in the emergency medical domain and proposes an annotation method for the collected data in order to improve speech recognition performance. To suggest future research directions, baseline speech recognition experiments were conducted by using partial data that were collected and annotated. All voices were recorded at 16-bit resolution at 16 kHz sampling rate. A total of 166 conversations were collected, amounting to 8 hours and 35 minutes. Various information was manually transcribed such as orthography, pronunciation, dialect, noise, and medical information using Praat. Baseline speech recognition experiments were used to depict problems related to speech recognition in the emergency medical domain. The Korean conversational speech data presented in this paper are first-stage data in the emergency medical domain and are expected to be used as training data for developing conversational systems for emergency medical applications.

Implementation of Responsive Web-based Vessel Auxiliary Equipment and Pipe Condition Diagnosis Monitoring System (반응형 웹 기반 선박 보조기기 및 배관 상태 진단 모니터링 시스템 구현)

  • Sun-Ho, Park;Woo-Geun, Choi;Kyung-Yeol, Choi;Sang-Hyuk, Kwon
    • Journal of Navigation and Port Research
    • /
    • v.46 no.6
    • /
    • pp.562-569
    • /
    • 2022
  • The alarm monitoring technology applied to existing operating ships manages data items such as temperature and pressure with AMS (Alarm Monitoring System) and provides an alarm to the crew should these sensing data exceed the normal level range. In addition, the maintenance of existing ships follows the Planned Maintenance System (PMS). whereby the sensing data measured from the equipment is monitored and if it surpasses the set range, maintenance is performed through an alarm, or the corresponding part is replaced in advance after being used for a certain period of time regardless of whether the target device has a malfunction or not. To secure the reliability and operational safety of ship engine operation, it is necessary to enable advanced diagnosis and prediction based on real-time condition monitoring data. To do so, comprehensive measurement of actual ship data, creation of a database, and implementation of a condition diagnosis monitoring system for condition-based predictive maintenance of auxiliary equipment and piping must take place. Furthermore, the system should enable management of auxiliary equipment and piping status information based on a responsive web, and be optimized for screen and resolution so that it can be accessed and used by various mobile devices such as smartphones as well as for viewing on a PC on board. This update cost is low, and the management method is easy. In this paper, we propose CBM (Condition Based Management) technology, for autonomous ships. This core technology is used to identify abnormal phenomena through state diagnosis and monitoring of pumps and purifiers among ship auxiliary equipment, and seawater and steam pipes among pipes. It is intended to provide performance diagnosis and failure prediction of ship auxiliary equipment and piping for convergence analysis, and to support preventive maintenance decision-making.

Suitable clothing recommendation system by size and skin color (의류 사이즈별 및 피부톤에 기반을 둔 의류 추천 시스템)

  • Park, Chang-Young;Lim, Byeong-Chan;Lee, Won-Joon;Lee, Chang-Su;Kim, Min-Su;Lee, Sang-Yong
    • Journal of Digital Convergence
    • /
    • v.20 no.3
    • /
    • pp.407-413
    • /
    • 2022
  • Existing clothing recommendation systems remain at the level of showing appropriate photos when a user selects a type of clothing he or she likes after entering his or her own body size or body size. When a user purchases clothing using such recommendation systems, there are many cases in which it does not fit or does not fit the user's body size. In this study, to solve these problems of existing clothing recommendation systems, a system was implemented in which the user receives not only size but also skin tone and recommends clothing suitable for the user's body size as well as skin tone. In this system, clothing size information obtained through web crawling was periodically stored in a database for eight male tops to recommend clothing, and the entire pixel of the clothing image was analyzed to extract color text values. In order to confirm the performance of this system, a survey was conducted on 100 male college students, and the satisfaction level was 70%. Most of the reasons for not being satisfied are that the recommended clothing is limited, so it is judged that it is necessary to expand the target clothing in the future.