March 2000
Volume 2 • Number 2

Contents

Risk Identification Techniques for Defect Reduction and Quality Improvement

This article surveys and compares risk identification techniques that can be used for defect reduction and quality improvement. Each technique is briefly described and illustrated with practical examples from industrial or governmental projects. The techniques are compared using several criteria, including simplicity, accuracy, and stability of results; ease of result interpretation; and utility in guiding defect reduction and quality improvement activities. The author also recommends an integrated life-cycle approach so these techniques can be used throughout the software development process for quality assessment and improvement.

Key words: defects and software metrics, learning algorithms, pattern matching, quality improvement, risk identification, statistical analysis techniques

by Jeff Tian, Department of Computer Science and Engineering, Southern Methodist University

INTRODUCTION

Software must be as defect-free as practical. A defect generally refers to a problem in the software, which may lead to undesirable consequences for both the software development organization and the software users. The potential for such undesirable consequences, including schedule delays, cost overruns, and highly defective software products, is usually referred to as risk. Various statistical and learning algorithm based techniques have been developed to identify and reduce such risks.

In software, a defect generally results in fixes to several modules or product components, and those fixes may also be propagated to related product releases. The author uses the term defect fix, denoted as DF, to refer to the action of identifying and correcting a localized problem. It is also used to measure product quality in this article. DF can be identified with specific modules; therefore, it can be used to analyze software quality at both the module level and the product level.

Defect distribution is highly uneven for most software products, regardless of their size, functionality, implementation language, and other characteristics. A lot of empirical evidence has accumulated over the years to support the so-called 80:20 rule, which states that 20 percent of the software components are responsible for 80 percent of the problems. Figure 1 shows a typical defect distribution for a commercial software product studied in (Tian and Troster 1998), where only 19.2 percent (248) of the modules have more than 2 DF (DF > 2), but they represent 84 percent (1989) of the total DF.

Because of the uneven defect distribution among software modules, there is a great need for risk identification techniques so actions can be focused on those potentially high-defect modules for effective quality improvement. To measure and characterize these high-defect modules, various software metrics can be used to capture information about software design, code, size, change, and so on (Fenton and Pfleeger 1996). Once the measurement data are collected from existing project databases or calculated using measurement tools, various techniques can be employed to analyze the data to identify high-defect modules. Like other statistical techniques, these risk identification techniques cannot establish proof of a causal relationship. They can, however, provide strong evidence that there may be a causal relationship in an observed effect. By extracting the characteristics of existing high-defect modules, these analyses can help software professionals identify new modules demonstrating similar measurement characteristics and take early actions to reduce risks or prevent potential problems.

The author next briefly surveys risk identification techniques and illustrates their use through practical examples from industrial and governmental projects. Data, models, and analysis results presented in this article are extracted from several commercial (IBM) software products studied in (Khoshgoftaar and Szabo 1996; Tian 1995; Tian and Troster 1998), governmental (NASA) projects studied in (Porter and Selby 1990; Briand, Basili, and Hetmanski 1993), as well as software systems used in aerospace, medical, and telecommunication industries studied in (Munson and Khoshgoftaar 1992; Khoshgoftaar et al. 1996). The article concludes with a comparison of these techniques and the author’s recommendation for an integrated life-cycle approach where selected techniques can be used effectively through software development for defect reduction and quality improvement.