Process Implementation
Once the critical metrics have been defined and the critical risks have been identified, one can identify the areas where process improvements are needed. Process improvements will tend to address two areas:
More reliable calculation of the critical metrics.
Improvements in the critical metrics.
There are many techniques that can lower risk and improve defect levels. The challenge is to choose the techniques that best address the risk profile and critical metrics defined in the earlier steps. As much as possible, this step should build upon existing methodology and procedures. The figure below illustrates this concept by cross referencing various techniques with the defect management objectives they are most effective in addressing. Some techniques tend to be very effective across a broad array of objectives and should receive high priority in the Defect Management Plan.
Technique Vs. Objective Matrix
Technique/
Objective
|
Prevention
|
Discovery
|
Resolution
|
Management
Reporting
|
Process
Improvement
|
|
Reduce
Probability
|
Reduce
Impact
|
Find
|
Report
|
Acknow-ledge
|
Prior-itize
|
Schedule
|
Fix
|
Report
|
|
|
Inspections
|
X
|
|
X
|
X
|
X
|
|
|
|
|
|
|
Testing
|
|
|
X
|
X
|
X
|
|
|
|
|
|
|
Early Test
Planning
|
X
|
|
X
|
X
|
X
|
|
|
|
|
|
|
Defensive
Design
|
X
|
|
X
|
X
|
X
|
|
|
|
|
|
|
Assertions
|
X
|
|
X
|
X
|
X
|
|
|
|
|
|
|
Help Desk
|
|
|
X
|
X
|
|
|
|
|
|
|
|
Contingency Planning
|
|
X
|
|
|
|
|
|
|
|
|
|
Defect
Reporting
|
|
|
|
X
|
|
X
|
X
|
|
X
|
X
|
X
|
Defect Classification
|
|
|
|
|
|
|
|
|
|
|
X
|
Disaster
Recovery
Planning
|
|
X
|
|
|
|
|
|
|
|
|
|
The following areas should be viewed as prerequisites to process improvement:
Documented Process: The process used to develop and maintain software should be documented.
Deliverable Definitions: The process must define the required content of deliverables, the point in the development process where each deliverable is produced, and the point in the process where each deliverable is baselined. This step is required to define and identify defects.
Defect Reporting System: Procedures should be established to formally and concisely report defects and gather the data required to compute the critical metrics. The bulk of this work should be the responsibility of individual project teams. A quality assurance function or other group should be responsible for consolidating and reporting information across projects.
Once the above areas are in place, defect information should be collected and analyzed. This information should be used to guide the ongoing process improvement activities.
IBM (see [CHIL92]) has done some very interesting work on a concept called Orthogonal Defect Classification (ODC), which can guide process improvement activities. ODC attempts to exploit the relationship between the cause of the defect and the type of defect. Defects are categorized into classes that are related to the part of the process that needs attention. Defect types are mapped into the process to provide the relation between the defect and the process. The figure below illustrates this association.
The assumption is that the distribution of defect types provides insight into status of a product. If for example, there are many defects associated with edit/validation rules found during system testing, then the distribution of defects would indicate that unit testing was inadequate and that the product was rushed prematurely into system test.
Defect Type and Process Associations
Defect Type
|
Process Association
|
Function
|
Requirements
|
Interface
|
Design
|
Edit/Validation Checking
|
Program Specification/Coding/Unit Test
|
Algorithm
|
Design/Program Specification
|
IBM is also researching the concept of a defect trigger. A defect trigger is a condition that allows a defect to be discovered. Unlike defect types, defect triggers can potentially be identified early in the life cycle of a defect. Like the defect type attribute, the defect trigger distribution can provide insight into the development process. It can also, however, provide insight into the verification process. Ideally, for example, the distribution of defects found in production should be similar to the distribution found during system testing. Significant differences in the two distributions would indicate weaknesses in the system testing.
|