• Title/Summary/Keyword: Evaluation metrics

Search Result 512, Processing Time 0.025 seconds

Software Quality Prediction based on Defect Severity (결함 심각도에 기반한 소프트웨어 품질 예측)

  • Hong, Euy-Seok
    • Journal of the Korea Society of Computer and Information
    • /
    • v.20 no.5
    • /
    • pp.73-81
    • /
    • 2015
  • Most of the software fault prediction studies focused on the binary classification model that predicts whether an input entity has faults or not. However the ability to predict entity fault-proneness in various severity categories is more useful because not all faults have the same severity. In this paper, we propose fault prediction models at different severity levels of faults using traditional size and complexity metrics. They are ternary classification models and use four machine learning algorithms for their training. Empirical analysis is performed using two NASA public data sets and a performance measure, accuracy. The evaluation results show that backpropagation neural network model outperforms other models on both data sets, with about 81% and 88% in terms of accuracy score respectively.

A Study on BSR Noise and Sound Quality Property for Vehicle Interior Module (자동차 인테리어 모듈의 BSR 소음과 음질 특성 연구)

  • Shin, Su-Hyun;Cheong, Cheol-Ung;Jung, Sung-Soo;Kang, Dae-Hwan
    • Transactions of the Korean Society for Noise and Vibration Engineering
    • /
    • v.22 no.6
    • /
    • pp.550-555
    • /
    • 2012
  • Among the various elements affecting a customer's evaluation of automobile quality, buzz, squeak and rattle(BSR) have been considered to be major factors. In most vehicle manufacturers, the BSR problems are solved by find-fix method with the vehicle road test, mainly due to various excitation sources, complex generation mechanism and subjective response. To systematically tackle the BSR problems in early stage of the vehicle development cycle, these difficulties should be resolved. The aim of the present paper is to characterize the sound quality property of BSR noise that can be used to assess the subjective responses to BSR. The four sound metrics from Zwicker's sound quality parameter are computed for the signals recorded for eight BSR noise source regions localized by using the acoustic-field visualized results. Then, the jury test of BSR noise are performed. On the basis of the computed sound metrics and jury test result is evaluated to represent the harshness of BSR noise. It is expected that the developed BSR measuring system and sound quality properties can be used to reduce the automotive interior BSR noise in terms of subjective levels as well as objective levels.

Flow based Sequential Grouping System for Malicious Traffic Detection

  • Park, Jee-Tae;Baek, Ui-Jun;Lee, Min-Seong;Goo, Young-Hoon;Lee, Sung-Ho;Kim, Myung-Sup
    • KSII Transactions on Internet and Information Systems (TIIS)
    • /
    • v.15 no.10
    • /
    • pp.3771-3792
    • /
    • 2021
  • With the rapid development of science and technology, several high-performance networks have emerged with various new applications. Consequently, financially or socially motivated attacks on specific networks have also steadily become more complicated and sophisticated. To reduce the damage caused by such attacks, administration of network traffic flow in real-time and precise analysis of past attack traffic have become imperative. Although various traffic analysis methods have been studied recently, they continue to suffer from performance limitations and are generally too complicated to apply in existing systems. To address this problem, we propose a method to calculate the correlation between the malicious and normal flows and classify attack traffics based on the corresponding correlation values. In order to evaluate the performance of the proposed method, we conducted several experiments using examples of real malicious traffic and normal traffic. The evaluation was performed with respect to three metrics: recall, precision, and f-measure. The experimental results verified high performance of the proposed method with respect to first two metrics.

Similarity Analysis Between SAR Target Images Based on Siamese Network (Siamese 네트워크 기반 SAR 표적영상 간 유사도 분석)

  • Park, Ji-Hoon
    • Journal of the Korea Institute of Military Science and Technology
    • /
    • v.25 no.5
    • /
    • pp.462-475
    • /
    • 2022
  • Different from the field of electro-optical(EO) image analysis, there has been less interest in similarity metrics between synthetic aperture radar(SAR) target images. A reliable and objective similarity analysis for SAR target images is expected to enable the verification of the SAR measurement process or provide the guidelines of target CAD modeling that can be used for simulating realistic SAR target images. For this purpose, this paper presents a similarity analysis method based on the siamese network that quantifies the subjective assessment through the distance learning of similar and dissimilar SAR target image pairs. The proposed method is applied to MSTAR SAR target images of slightly different depression angles and the resultant metrics are compared and analyzed with qualitative evaluation. Since the image similarity is somewhat related to recognition performance, the capacity of the proposed method for target recognition is further checked experimentally with the confusion matrix.

Application of a comparative analysis of random forest programming to predict the strength of environmentally-friendly geopolymer concrete

  • Ying Bi;Yeng Yi
    • Steel and Composite Structures
    • /
    • v.50 no.4
    • /
    • pp.443-458
    • /
    • 2024
  • The construction industry, one of the biggest producers of greenhouse emissions, is under a lot of pressure as a result of growing worries about how climate change may affect local communities. Geopolymer concrete (GPC) has emerged as a feasible choice for construction materials as a result of the environmental issues connected to the manufacture of cement. The findings of this study contribute to the development of machine learning methods for estimating the properties of eco-friendly concrete, which might be used in lieu of traditional concrete to reduce CO2 emissions in the building industry. In the present work, the compressive strength (fc) of GPC is calculated using random forests regression (RFR) methodology where natural zeolite (NZ) and silica fume (SF) replace ground granulated blast-furnace slag (GGBFS). From the literature, a thorough set of experimental experiments on GPC samples were compiled, totaling 254 data rows. The considered RFR integrated with artificial hummingbird optimization (AHA), black widow optimization algorithm (BWOA), and chimp optimization algorithm (ChOA), abbreviated as ARFR, BRFR, and CRFR. The outcomes obtained for RFR models demonstrated satisfactory performance across all evaluation metrics in the prediction procedure. For R2 metric, the CRFR model gained 0.9988 and 0.9981 in the train and test data set higher than those for BRFR (0.9982 and 0.9969), followed by ARFR (0.9971 and 0.9956). Some other error and distribution metrics depicted a roughly 50% improvement for CRFR respect to ARFR.

Analyzing Planning Performance of Road Construction Projects Using Preliminary Feasibility Analysis Data (예비타당성조사 결과를 활용한 도로건설사업의 계획단계 성과 분석 연구)

  • Mun, Junbu;Yun, Sungmin
    • Korean Journal of Construction Engineering and Management
    • /
    • v.24 no.1
    • /
    • pp.3-11
    • /
    • 2023
  • According to the post evaluation scheme in Korea of a public construction project which is more than 30 Billion KRW, project performance is evaluated by investigating outcomes and effects of the construction after the completion of the project. The current post evaluation results can be used for planning and estimating a construction project in the future. However, it is not easy to utilized for an on-going project because the system does not provide the phase-based performance of a project. Although project planning performance is important for project initiation, few attempt has been made to evaluate planning performance in Korea. The purpose of this study is to provide a conceptual performance evaluation of planning performance using preliminary feasibility study conducted by Korea Development Institute. This study developed a planning performance database using data extracted from preliminary feasibility study reports of the completed 354 road construction projects. This study analyzed the performance of the planning stage of road projects by developing absolute metrics such as standard construction cost and standard construction schedule based on a Lane-Km. Using the standard construction cost and schedule metrics, the planning performance was analyzed by project characteristics. The results of this study can be used for phase-based performance evaluation from planning phase to construction phase.

Sound Quality Evaluation of the Level D Noise for the vehicle using Mahalanobis Distance (Mahalanobis Distance 를 이용한 차량 D 단 소음의 음질 평가)

  • Park, Sang-Gil;Park, Won-Sik;Sim, Hyoun-Jin;Lee, Jung-Youn;Oh, Jae-Eung
    • Proceedings of the Korean Society for Noise and Vibration Engineering Conference
    • /
    • 2007.11a
    • /
    • pp.311-317
    • /
    • 2007
  • The reduction of the Vehicle interior noise has been the main interest of NVH engineers. The driver's perception on the vehicle noise is affected largely by psychoacoustic characteristic of the noise as well as the SPL. The previous methods to evaluation of the SQ about vehicle interior noise are linear regression analysis of subjective SQ metrics by statistics and the estimation of the subjective SQ values by neural network. But these are so depended on jury test very much that they result in many difficulties. So, to reduce jury test weight, we suggested a new method using Mahalanobis distance for SQ evaluation. And, optimal characteristic values influenced on the result of the SQ evaluation were derived by signal to noise ratio(SN ratio) of the Taguchi method. Finally, the new method to evaluate SQ is constructed using Mahalanobis-Taguchi system(MTS). Furthermore, the MTS method for SQ evaluation was compared by the result of SQ grade table at the previous study and their virtues and faults introduced.

  • PDF

Sound Quality evaluation of the interior noise for the driving vehicle using Mahalanobis Distance (Mahalanobis Distance 를 이용한 주행중 차량 실내소음의 음질평가)

  • Park, Sang-Gil;Kim, Ho-San;Bae, Chul-Yong;Lee, Bong-Hyun;Oh, Jae-Eung
    • Proceedings of the Korean Society for Noise and Vibration Engineering Conference
    • /
    • 2007.11a
    • /
    • pp.318-321
    • /
    • 2007
  • Since human listening is very sensitive to sound, a subjective index of a sound quality is required. Therefore, in the analysis for each situation, the sound evaluation is composed with sound quality factor. Many researchers spends their effort to make a more reliable and more accurate of sound in term of sound quality index for various system noise. The previous methods to evaluation of the SQ about vehicle interior noise are linear regression analysis of subjective SQ metrics by statistics and the estimation of the subjective SQ values by neural network. But these are so depended on jury test very much that they result in many difficulties. So, to reduce jury test weight, we suggested a new method using Mahalanobis distance for SQ evaluation. Threrefore, in this study Mahalanobis distance for the vehicle interior noise was derived using the objective SQ except jury test. Finnaly, the results of the SQ evaluation was analyzed discrimination between reference and abnormal group.

  • PDF

Performance Evaluation Methodology in Virtual Environments (가상화 시스템의 성능 평가 방법)

  • Jang, Ji-Yong;Han, Sae-Young;Kim, Jin-Seok;Park, Sung-Yong
    • The KIPS Transactions:PartA
    • /
    • v.15A no.3
    • /
    • pp.167-180
    • /
    • 2008
  • Consolidating servers into a virtualized system increases entire system utilization, while suffers from performance degradation due to the additional virtualization layer. In this paper, we proposed a performance evaluation methodology for comparing virtualized systems with native non-virtualized systems. We defined a system waste rate per consolidated throughput as a metric, and described the method for calculating system waste rate and consolidated throughput for both of virtualized systems and non-virtualized systems. Using the proposing performance evaluation methodology, we established testbeds, evaluated their performance, and compared the metrics of both systems. As a result of the evaluation, we could show the appropriateness of our methodology and analyze the effect of the application characteristics.

Sound Quality Evaluation and Grade Construction of the Level D Noise for the Vehicle Using MTS (MTS기법을 이용한 차량 D단 소음의 음질 평가 및 음질 등급화 구축)

  • Park, Sang-Gil;Park, Won-Sik;Sim, Hyoun-Jin;Lee, Jung-Youn;Oh, Jae-Eung
    • Transactions of the Korean Society for Noise and Vibration Engineering
    • /
    • v.18 no.4
    • /
    • pp.393-399
    • /
    • 2008
  • The reduction of the Vehicle interior noise has been the main interest of NVH engineers. The driver's perception on the vehicle noise is affected largely by psychoacoustic characteristic of the noise as well as the SPL. The previous methods to evaluation of the SQ about vehicle interior noise are linear regression analysis of subjective SQ metrics by statistics and the estimation of the subjective SQ values by neural network. But these are so depended on jury test very much that they result in many difficulties. So, to reduce jury test weight, we suggested a new method using Mahalanobis distance for SQ evaluation. And, optimal characteristic values influenced on the result of the SQ evaluation were derived by signal to noise ratio(SN ratio) of the Taguchi method. Finally, the new method to evaluate SQ is constructed using Mahalanobis-Taguchi system(MTS). Furthermore, the MTS method for SQ evaluation was compared by the result of SQ grade table at the previous study and their virtues and faults introduced.