• Title/Summary/Keyword: Validation Metrics

Search Result 69, Processing Time 0.026 seconds

Analytical Validation of the SDL Metrics Set (SDL 메트릭 집합의 분석적 검증)

  • Hong, Ui-Seok;Jeong, Myeong-Hui
    • The Transactions of the Korea Information Processing Society
    • /
    • v.7 no.4
    • /
    • pp.1112-1121
    • /
    • 2000
  • Design metrics that quantify the design phase play an important role in reducing system development cost because the problems in early phases of software development seriously affected the quality of the late products. Real-time systems such as telecommunication systems are so large that design quantification is more important in real-time system design. Although many metrics have been proposed, few of them are correctly validated. This paper revises the SDL metrics set proposed in earlier study [9] and perform an analytical validation o the metrics set. Axiomatic approach and dimensional analysis are used for metrics validation and the old metrics are revised ot satisfy the validation criteria.

  • PDF

A Metrics Set for Measuring Software Module Severity (소프트웨어 모듈 심각도 측정을 위한 메트릭 집합)

  • Hong, Euy-Seok
    • Journal of the Korea Society of Computer and Information
    • /
    • v.20 no.1
    • /
    • pp.197-206
    • /
    • 2015
  • Defect severity that is a measure of the impact caused by the defect plays an important role in software quality activities because not all software defects are equal. Earlier studies have concentrated on defining defect severity levels, but there have almost never been trials of measuring module severity. In this paper, first, we define a defect severity metric in the form of an exponential function using the characteristics that defect severity values increase much faster than severity levels. Then we define a new metrics set for software module severity using the number of defects in a module and their defect severity metric values. In order to show the applicability of the proposed metrics, we performed an analytical validation using Weyuker's properties and experimental validation using NASA open data sets. The results show that ms is very useful for measuring the module severity and msd can be used to compare different systems in terms of module severity.

Fault Prediction Using Statistical and Machine Learning Methods for Improving Software Quality

  • Malhotra, Ruchika;Jain, Ankita
    • Journal of Information Processing Systems
    • /
    • v.8 no.2
    • /
    • pp.241-262
    • /
    • 2012
  • An understanding of quality attributes is relevant for the software organization to deliver high software reliability. An empirical assessment of metrics to predict the quality attributes is essential in order to gain insight about the quality of software in the early phases of software development and to ensure corrective actions. In this paper, we predict a model to estimate fault proneness using Object Oriented CK metrics and QMOOD metrics. We apply one statistical method and six machine learning methods to predict the models. The proposed models are validated using dataset collected from Open Source software. The results are analyzed using Area Under the Curve (AUC) obtained from Receiver Operating Characteristics (ROC) analysis. The results show that the model predicted using the random forest and bagging methods outperformed all the other models. Hence, based on these results it is reasonable to claim that quality models have a significant relevance with Object Oriented metrics and that machine learning methods have a comparable performance with statistical methods.

A Pragmatic Framework for Predicting Change Prone Files Using Machine Learning Techniques with Java-based Software

  • Loveleen Kaur;Ashutosh Mishra
    • Asia pacific journal of information systems
    • /
    • v.30 no.3
    • /
    • pp.457-496
    • /
    • 2020
  • This study aims to extensively analyze the performance of various Machine Learning (ML) techniques for predicting version to version change-proneness of source code Java files. 17 object-oriented metrics have been utilized in this work for predicting change-prone files using 31 ML techniques and the framework proposed has been implemented on various consecutive releases of two Java-based software projects available as plug-ins. 10-fold and inter-release validation methods have been employed to validate the models and statistical tests provide supplementary information regarding the reliability and significance of the results. The results of experiments conducted in this article indicate that the ML techniques perform differently under the different validation settings. The results also confirm the proficiency of the selected ML techniques in lieu of developing change-proneness prediction models which could aid the software engineers in the initial stages of software development for classifying change-prone Java files of a software, in turn aiding in the trend estimation of change-proneness over future versions.

Theoretical Validation of Inheritance Metric in QMOOD against Weyuker's Properties

  • Alharthi, Mariam;Aljedaibi, Wajdi
    • International Journal of Computer Science & Network Security
    • /
    • v.21 no.7
    • /
    • pp.284-296
    • /
    • 2021
  • Quality Models are important element of the software industry to develop and implement the best quality product in the market. This type of model provides aid in describing quality measures, which directly enhance the user satisfaction and software quality. In software development, the inheritance technique is an important mechanism used in object-oriented programming that allows the developers to define new classes having all the properties of super class. This technique supports the hierarchy design for classes and makes an "is-a" association among the super and subclasses. This paper describes a standard procedure for validating the inheritance metric in Quality Model for Object-Oriented Design (QMOOD) by using a set of nine properties established by Weyuker. These properties commonly using for investigating the effectiveness of the metric. The integration of two measuring methods (i.e. QMOOD and Weyuker) will provide new way for evaluating the software quality based on the inheritance context. The output of this research shows the extent of satisfaction of the inheritance metric in QMOOD against Weyuker nine properties. Further results proved that Weyker's property number nine could not fulfilled by any inheritance metrics. This research introduces a way for measuring software that developed using object-oriented approach. The theoretical validation of the inheritance metric presented in this paper is a small step taken towards producing quality software and in providing assistance to the software industry.

Development and Validation of a Deep Learning System for Segmentation of Abdominal Muscle and Fat on Computed Tomography

  • Hyo Jung Park;Yongbin Shin;Jisuk Park;Hyosang Kim;In Seob Lee;Dong-Woo Seo;Jimi Huh;Tae Young Lee;TaeYong Park;Jeongjin Lee;Kyung Won Kim
    • Korean Journal of Radiology
    • /
    • v.21 no.1
    • /
    • pp.88-100
    • /
    • 2020
  • Objective: We aimed to develop and validate a deep learning system for fully automated segmentation of abdominal muscle and fat areas on computed tomography (CT) images. Materials and Methods: A fully convolutional network-based segmentation system was developed using a training dataset of 883 CT scans from 467 subjects. Axial CT images obtained at the inferior endplate level of the 3rd lumbar vertebra were used for the analysis. Manually drawn segmentation maps of the skeletal muscle, visceral fat, and subcutaneous fat were created to serve as ground truth data. The performance of the fully convolutional network-based segmentation system was evaluated using the Dice similarity coefficient and cross-sectional area error, for both a separate internal validation dataset (426 CT scans from 308 subjects) and an external validation dataset (171 CT scans from 171 subjects from two outside hospitals). Results: The mean Dice similarity coefficients for muscle, subcutaneous fat, and visceral fat were high for both the internal (0.96, 0.97, and 0.97, respectively) and external (0.97, 0.97, and 0.97, respectively) validation datasets, while the mean cross-sectional area errors for muscle, subcutaneous fat, and visceral fat were low for both internal (2.1%, 3.8%, and 1.8%, respectively) and external (2.7%, 4.6%, and 2.3%, respectively) validation datasets. Conclusion: The fully convolutional network-based segmentation system exhibited high performance and accuracy in the automatic segmentation of abdominal muscle and fat on CT images.

Metrics Measurement System Supporting Quality Evaluation of Java Program (Java 프로그램의 품질평가를 지원하는 메트릭 측정 시스템)

  • Park, Ok-Cha;Yoo, Cheol-Jung;Chang, Ok-Bae
    • Journal of KIISE:Computing Practices and Letters
    • /
    • v.7 no.2
    • /
    • pp.151-164
    • /
    • 2001
  • Java, used as the most representative object-oriented language, isil becoming the popular language for Internet/Intranet based program development. Moreover, it is used for development language in a variety of areas such as component based development language. In the view of reuse and maintenance of developed program, quality evaluation of program is becoming a more important issue. So, metrics measurement for quality evaluation of program that is developed at present including existing Java application is necessary. However, it is necessary that whether existing object-oriented software metrics is suitable on Java program is to be validated So, in this paper, we build an automated metrics measurement system that needs to validate on object-oriented software metrics and wish to support metrics measurement that is to determine it. The purpose of this system is to support a precise quality evaluation tool. In this system, we apply the metrics classified by Briand. Briand classified the metrics by formalizing mathematically them to verify feasibility of existing object-oriented software metrics. Using the proposed system, we can make comparison and analysis of validation on existing object-oriented metrics by calculating quantitative information more rapidly from Java source program. If there is any problem in feasibility of the metrics, we can establish a suitable metrics on Java program by considering reiJ,1forcement of the existing metrics or proposing new metrics.

  • PDF

Development of Evaluation Metrics for Pedestrian Flow Optimization in a Complex Service Environment Based on Behavior Observation Method

  • Bahn, Sang-Woo;Lee, Chai-Woo;Kwon, Sang-Hyun;Yun, Myung-Hwan
    • Journal of the Ergonomics Society of Korea
    • /
    • v.29 no.4
    • /
    • pp.647-654
    • /
    • 2010
  • In a service environment, the spatial layout is an important factor that has a great impact on customers' behavioral characteristics including wayfinding and purchasing. Previous studies have shown a gap between marketing, focusing solely on profitability and satisfaction, and architecture, looking only into efficiency of pedestrian flow. To balance such disparity, this study suggests an integrated approach for assessing behavioral patterns in complex service environments. With the objective that complex service environments should aim to increase its profitability and efficiency while guaranteeing customer satisfaction, quantitative metrics was developed for evaluation. The metrics was defined to use data from behavior observation including path tracking, population counting, and gaze analysis, while previous studies have relied on abstract survey methods that were prone to sampling errors and loss of data. For validation of the metrics in a real world setting, a case study was conducted at 4 train stations in Korea. In the case study, experiments were conducted to gather the required data in all 4 train stations, while their physical layouts were also analyzed. With the results from the case study, comparative evaluation of the 4 train stations in terms of behavioral efficiency was possible, together with a discussion on the effect of their physical settings.

Validation of Mid Air Collision Detection Model using Aviation Safety Data (항공안전 데이터를 이용한 항공기 공중충돌위험식별 모형 검증 및 고도화)

  • Paek, Hyunjin;Park, Bae-seon;Kim, Hyewook
    • Journal of the Korean Society for Aviation and Aeronautics
    • /
    • v.29 no.4
    • /
    • pp.37-44
    • /
    • 2021
  • In case of South Korea, the airspace which airlines can operate is extremely limited due to the military operational area located within the Incheon flight information region. As a result, safety problems such as mid-air collision between aircraft or Traffic alert and Collision Avoidance System Resolution Advisory (TCAS RA) may occur with higher probability than in wider airspace. In order to prevent such safety problems, an mid-air collision risk detection model based on Detect-And-Avoid (DAA) well clear metrics is investigated. The model calculates the risk of mid-air collision between aircraft using aircraft trajectory data. In this paper, the practical use of DAA well clear metrics based model has been validated. Aviation safety data such as aviation safety mandatory report and Automatic Dependent Surveillance Broadcast is used to measure the performance of the model. The attributes of individual aircraft track data is analyzed to correct the threshold of each parameter of the model.

Analyzing Machine Learning Techniques for Fault Prediction Using Web Applications

  • Malhotra, Ruchika;Sharma, Anjali
    • Journal of Information Processing Systems
    • /
    • v.14 no.3
    • /
    • pp.751-770
    • /
    • 2018
  • Web applications are indispensable in the software industry and continuously evolve either meeting a newer criteria and/or including new functionalities. However, despite assuring quality via testing, what hinders a straightforward development is the presence of defects. Several factors contribute to defects and are often minimized at high expense in terms of man-hours. Thus, detection of fault proneness in early phases of software development is important. Therefore, a fault prediction model for identifying fault-prone classes in a web application is highly desired. In this work, we compare 14 machine learning techniques to analyse the relationship between object oriented metrics and fault prediction in web applications. The study is carried out using various releases of Apache Click and Apache Rave datasets. En-route to the predictive analysis, the input basis set for each release is first optimized using filter based correlation feature selection (CFS) method. It is found that the LCOM3, WMC, NPM and DAM metrics are the most significant predictors. The statistical analysis of these metrics also finds good conformity with the CFS evaluation and affirms the role of these metrics in the defect prediction of web applications. The overall predictive ability of different fault prediction models is first ranked using Friedman technique and then statistically compared using Nemenyi post-hoc analysis. The results not only upholds the predictive capability of machine learning models for faulty classes using web applications, but also finds that ensemble algorithms are most appropriate for defect prediction in Apache datasets. Further, we also derive a consensus between the metrics selected by the CFS technique and the statistical analysis of the datasets.