• Title/Summary/Keyword: cluster method

Search Result 2,498, Processing Time 0.035 seconds

Development of Big-data Management Platform Considering Docker Based Real Time Data Connecting and Processing Environments (도커 기반의 실시간 데이터 연계 및 처리 환경을 고려한 빅데이터 관리 플랫폼 개발)

  • Kim, Dong Gil;Park, Yong-Soon;Chung, Tae-Yun
    • IEMEK Journal of Embedded Systems and Applications
    • /
    • v.16 no.4
    • /
    • pp.153-161
    • /
    • 2021
  • Real-time access is required to handle continuous and unstructured data and should be flexible in management under dynamic state. Platform can be built to allow data collection, storage, and processing from local-server or multi-server. Although the former centralize method is easy to control, it creates an overload problem because it proceeds all the processing in one unit, and the latter distributed method performs parallel processing, so it is fast to respond and can easily scale system capacity, but the design is complex. This paper provides data collection and processing on one platform to derive significant insights from various data held by an enterprise or agency in the latter manner, which is intuitively available on dashboards and utilizes Spark to improve distributed processing performance. All service utilize dockers to distribute and management. The data used in this study was 100% collected from Kafka, showing that when the file size is 4.4 gigabytes, the data processing speed in spark cluster mode is 2 minute 15 seconds, about 3 minutes 19 seconds faster than the local mode.

A Study on the Electronic Structures of Li Intercalated Vanadium Sulfide and Oxide (Li의 첨가에 따른 Vanadium의 유화물과 산화물의 전자상태계산에 관한 연구)

  • Jung, Hyun-Chul;Kim, Hui-Jin;Won, Dae-Hee;Yoon, Dong-Joo;Kim, Yang-Soo;Kim, Byung-Il
    • Korean Journal of Metals and Materials
    • /
    • v.46 no.9
    • /
    • pp.604-608
    • /
    • 2008
  • The layered compounds vanadium disulfide($VS_2$) and vanadium dioxide($VO_2$) intercalated with Li are investigated for using the Discrete Variational $(DV)-X{\alpha}$ molecular orbital method. The chemical bonding properties of the atoms were examined by bond overlap population of electronic states. The plot of density of states supports the covalent bonding properties by showing the overlap between the atoms. There is a strong tendency of covalent bonding between V-S and V-O. The intensity of covalent bonding of $VS_2$ is stronger than $VO_2$. The net charge of $LiVO_2$ is higher than that of $LiVS_2$. This results of the calculation of $VO_2$ and $VS_2$ indicate that $(DV)-X{\alpha}$ method can be widely applied in the new practical materials.

Genetic Differences in Natural and Cultured River Pufferfish Populations by PCR Analysis

  • Yoon, Jong-Man
    • Development and Reproduction
    • /
    • v.24 no.4
    • /
    • pp.327-335
    • /
    • 2020
  • Genomic DNA (gDNA) extracted from two populations of natural and cultured river pufferfish (Takifugu obscurus) was amplified by polymerase chain reaction (PCR). The complexity of the fragments derived from the two locations varied dramatically. The genetic distances (GDs) between individuals numbered 15 and 12 in the cultured population was 0.053, which was the lowest acknowledged. The oligonucleotide primer OPC-11 identified 88 unique loci shared within each population reflecting the natural population. The OPC-05 primer identified 44 loci shared by the two populations. The average band-sharing (BS) values of individuals in the natural population (0.683±0.014) were lower than in those derived from the cultured population (0.759±0.009) (p<0.05). The shortest GD demonstrating a significant molecular difference was found between the cultured individuals # 15 and # 12 (GD=0.053). Individual # 02 of the natural population was most distantly related to cultured individual # 22 (GD=0.827). A cluster tree was built using the unweighted pair group method with arithmetic mean (UPGMA) Euclidean GD analysis based on a total of 578 various fragments derived from five primers in the two populations. Obvious markers identified in this study represent the genetic structure, species security, and proliferation of river pufferfish in the rivers of the Korean peninsula.

Developing efficient model updating approaches for different structural complexity - an ensemble learning and uncertainty quantifications

  • Lin, Guangwei;Zhang, Yi;Liao, Qinzhuo
    • Smart Structures and Systems
    • /
    • v.29 no.2
    • /
    • pp.321-336
    • /
    • 2022
  • Model uncertainty is a key factor that could influence the accuracy and reliability of numerical model-based analysis. It is necessary to acquire an appropriate updating approach which could search and determine the realistic model parameter values from measurements. In this paper, the Bayesian model updating theory combined with the transitional Markov chain Monte Carlo (TMCMC) method and K-means cluster analysis is utilized in the updating of the structural model parameters. Kriging and polynomial chaos expansion (PCE) are employed to generate surrogate models to reduce the computational burden in TMCMC. The selected updating approaches are applied to three structural examples with different complexity, including a two-storey frame, a ten-storey frame, and the national stadium model. These models stand for the low-dimensional linear model, the high-dimensional linear model, and the nonlinear model, respectively. The performances of updating in these three models are assessed in terms of the prediction uncertainty, numerical efforts, and prior information. This study also investigates the updating scenarios using the analytical approach and surrogate models. The uncertainty quantification in the Bayesian approach is further discussed to verify the validity and accuracy of the surrogate models. Finally, the advantages and limitations of the surrogate model-based updating approaches are discussed for different structural complexity. The possibility of utilizing the boosting algorithm as an ensemble learning method for improving the surrogate models is also presented.

Ecotoxicity Assessment of Potassium Hydrogen Phthalate and Verification of Standard Reference Toxicity Test Method Using Potassium Hydrogen Phthalate

  • Dong Jin Choi
    • Proceedings of the National Institute of Ecology of the Republic of Korea
    • /
    • v.4 no.1
    • /
    • pp.49-62
    • /
    • 2023
  • Phthalates are animal carcinogens. Potassium hydrogen phthalate (KHP), which has the least complicated structure among phthalates, is used for the analysis of total organic carbon and formaldehyde. However, its toxicity has not been confirmed. A 24-hour acute toxicity test was performed using Daphnia magna, a water flea used to evaluate aquatic toxicity owing to its high sensitivity. The lowest observed effect concentration of KHP was found to be 240 mg/L. The effects of phosphorus, nitrogen, and Cr(6+), which are able to be discharged along with KHP, were also confirmed using tests. At 240 mg/L KHP, toxicity increased as phosphorus, nitrogen, and Cr(6+) increased. In addition, tests were performed to confirm the half maximal effective concentration of KHP. Through 10 test repetitions, the average ecotoxicity value was found to be 0.3, the average half maximal effective concentration was 327.75 mg/L, and the coefficient of variation (%) was 3.16%; because the latter value is lower than 25%, which is what is generally suggested for the water pollution standard method, the reproducibility of the tests is sufficient to replace the existing standard reference toxicity test that uses potassium dichromate. In addition, the half maximum effective concentration of potassium hydrogen phthalate is approximately 218 times more than that of potassium dichromate; therefore, toxicity is relatively low. In conclusion, KHP is a feasible alternative to the highly toxic potassium dichromate for performing the standard reference toxicity test.

Cluster-Based Similarity Calculation of IT Assets: Method of Attacker's Next Targets Detection

  • Dongsung Kim;Seon-Gyoung Shon;Dan Dongseong Kim;Huy-Kang Kim
    • Journal of the Korea Society of Computer and Information
    • /
    • v.29 no.5
    • /
    • pp.1-10
    • /
    • 2024
  • Attackers tend to use similar vulnerabilities when finding their next target IT assets. They also continuously search for new attack targets. Therefore, it is essential to find the potential targets of attackers in advance. Our method proposes a novel approach for efficient vulnerable asset management and zero-day response. In this paper, we propose the ability to detect the IT assets that are potentially infected by the recently discovered vulnerability based on clustering and similarity results. As the experiment results, 86% of all collected assets are clustered within the same clustering. In addition, as a result of conducting a similarity calculation experiment by randomly selecting vulnerable assets, assets using the same OS and service were listed.

Clustering-based Cooperative Routing using OFDM for Supporting Transmission Efficiency in Mobile Wireless Sensor Networks (모바일 무선 센서네트워크에서 전송 효율 향상을 지원하기 위한 OFDM을 사용한 클러스터링 기반의 협력도움 라우팅)

  • Lee, Joo-Sang;An, Beong-Ku
    • The Journal of the Institute of Internet, Broadcasting and Communication
    • /
    • v.10 no.6
    • /
    • pp.85-92
    • /
    • 2010
  • In this paper, we propose a Clustering-based Cooperative Routing using OFDM (CCRO) for supporting transmission efficiency in mobile wireless sensor networks. The main features and contributions of the proposed method are as follows. First, the clustering method which uses the location information of nodes as underlying infrastructure for supporting stable transmission services efficiently is used. Second, cluster-based cooperative data transmission method is used for improving data transmission and reliability services. Third, OFDM based data transmission method is used for improving data transmission ratio with channel efficiency. Fourth, we consider realistic approach in the view points of the mobile ad-hoc wireless sensor networks while conventional methods just consider fixed sensor network environments. The performance evaluation of the proposed method is performed via simulation using OPNET and theoretical analysis. The results of performance evaluation show improvement of transmission efficiency.

Effects of Vine Induction Method on the Growth and Fruit Yield in Korean Schisandra (오미자 덩굴 유인방법이 생육 및 과실 수량에 미치는 영향)

  • Kim, Ju Ho;Lee, Beom Gyun;Choi, Eun Young
    • Korean Journal of Medicinal Crop Science
    • /
    • v.25 no.2
    • /
    • pp.83-88
    • /
    • 2017
  • Background: This study was aimed to determine the optimal vine induction method for growing of Korean schisandra (Schisandra chinensis), by comparing plant growth and fruit yields between plants grown with either fence-type (U-type) or A-type induction. Methods and Results: Plants were transplanted on August 17, 2014, and the plant height, stem node number and weight were measured every two weeks, six times from June 17, 2016. The plant height, stem node number, and leaf length and width were higher with the A-type than with the U-type induction, by approximately 37.0%, 49.1%, 27.6%, and 12.7%, respectively. Although there was no significant difference between the photosynthesis rates of plants grown with the two vine induction method, the leaf area and leaf number per plant were higher in the plants grown with the A-type than the U-type, by approximately 23.7% and 46.0%, respectively. The number of green-color pixels, in a defined area of digital camera images of creeper leaves from the inducted vines, was significantly higher in the plants grown with the A-type than the U-type. The number of fruit clusters per plant was approximately 26 and 36, under the U-type and A-type, respectively. A two fold higher total fruit weight per plant was observed in the plants grown under the A-type (250 g/plant) than the U-type (120 g/plant). Conclusions: The A-type vine induction method is optimal for cultivation of Korean schisandra.

Risk assessment of water inrush in karst tunnels based on a modified grey evaluation model: Sample as Shangjiawan Tunnel

  • Yuan, Yong-cai;Li, Shu-cai;Zhang, Qian-qing;Li, Li-ping;Shi, Shao-shuai;Zhou, Zong-qing
    • Geomechanics and Engineering
    • /
    • v.11 no.4
    • /
    • pp.493-513
    • /
    • 2016
  • A modified grey clustering method is presented to systematically evaluate the risk of water inrush in karst tunnels. Based on the center triangle whitenization weight function and upper and lower limit measure whitenization weight function, the modified grey evaluation model doesn't have the crossing properties of grey cluster and meets the standard well. By adsorbing and integrating the previous research results, seven influence factors are selected as evaluation indexes. A couple of evaluation indexes are modified and quantitatively graded according to four risk grades through expert evaluation method. The weights of evaluation indexes are rationally distributed by the comprehensive assignment method. It is integrated by the subjective factors and the objective factors. Subjective weight is given based on analytical hierarchy process, and objective weight obtained from simple dependent function. The modified grey evaluation model is validated by Jigongling Tunnel. Finally, the water inrush risk of Shangjiawan Tunnel is evaluated by using the established model, and the evaluation result obtained from the proposed method is agrees well with practical situation. This risk assessment methodology provides a powerful tool with which planners and engineers can systematically assess the risk of water inrush in karst tunnels.

A Scalable Clustering Method for Categorical Sequences (범주형 시퀀스들에 대한 확장성 있는 클러스터링 방법)

  • Oh, Seung-Joon;Kim, Jae-Yearn
    • Journal of the Korean Institute of Intelligent Systems
    • /
    • v.14 no.2
    • /
    • pp.136-141
    • /
    • 2004
  • There has been enormous growth in the amount of commercial and scientific data, such as retail transactions, protein sequences, and web-logs. Such datasets consist of sequence data that have an inherent sequential nature. However, few clustering algorithms consider sequentiality. In this paper, we study how to cluster sequence datasets. We propose a new similarity measure to compute the similarity between two sequences. We also present an efficient method for determining the similarity measure and develop a clustering algorithm. Due to the high computational complexity of hierarchical clustering algorithms for clustering large datasets, a new clustering method is required. Therefore, we propose a new scalable clustering method using sampling and a k-nearest-neighbor method. Using a real dataset and a synthetic dataset, we show that the quality of clusters generated by our proposed approach is better than that of clusters produced by traditional algorithms.