Defect Management is a Powerful Ally in the Fight to Produce Quality Software
A bug is typically considered to be the result of a coding error where a defect is used as a broad term covering any deviation from the system’s business requirements. These terms tend to be used interchangeably and both indicate faults that need to be fixed. For the purpose of this discussion both terms will refer to incorrect operation (by whatever measure) of software system code.
Conventional wisdom holds that quality’s principle commission is to discover defects and report them to development for correction. This involves:
- dissecting the system specifications to determine use/test cases,
- executing tests/recording results and reporting those results to development and
- product management.
Notice that each of these functions involves communication with product management and/or development. Quality’s real function is to keep product status information in constant motion between management for decision making and development for corrective action.
This makes the processes and tools used for managing defects critically important to the quality process. They identify not only code faults but also development and test processes that need improvement.
Nuts and Bolts of Defect Management
Software development has a life cycle and so do defects. One of the givens of quality cost is that the sooner defects are identified and fixed, the lower the total cost of quality. Since defects can be introduced into any system process (requirements specification, technical document, use case, test case, code, etc.) they need to be identified and corrected as early in that process chain as possible.
Defect data management is a reductive process that needs as much input as possible. It works best when everyone reports discovered defects into the system. Software engineers, tech support personnel, product management, quality test techs and, yes, even customers all have vital defect input that needs to be recorded.
Software Defect Life Cycle
Once a defect is entered into the management system, it has one of five defect states:
- The defect has just been entered and is considered to be Open
- Defect has been fixed and that fix verified causing the defect report to be considered Closed
- A defect report can be Rejected because it is deemed invalid or the defect is considered a designed feature of the system
- Defect reports are Deferred if the defect won’t be fixed as part of the current release
- A defect may be Not Reproducible if neither development nor quality can observe its symptoms after the initial report
Defect Data Cleaning
As with any data collection process, defect reporting gathers up erroneous entries. Cleaning the defect list involves:
- Removing false positives caused by irregularities in the test environment, test code or test data or a test process errors
- Removing duplicate entries when one defect leads to results which look like multiple unrelated issues (and reports) to testers
- Making sure that scrubbing the list doesn’t discourage diligent defect reporting by all concerned
Note that, when using Test Driven Development (TDD), automated tests are built into the product design itself. The specified product development phase is not considered over until all the automated tests are executed, but this will involve numerous false defects caused by applying the embedded test scripts before the code is fully ready for them.
Who is Responsible for Defect Management?
Historically, defect management is owned by the QA group. However, a team which is cross-functional (a strident Agile aspect), comprising members from product development, product management, project management, etc. is accountable for managing defects reported by the team. Each functional team in a SCRUM must have a designated member to pursue defect recording and data management.
Software Defect Assessment
As defects are reported through the defect management system, the product management must consider the costs, risks and benefits associated with each report to decide whether the subject defect should be fixed or deferred. Defects must be assessed for:
- Severity (critical, high, medium, low) of impact to system operation and user perception of the finished product
- Priority for correction measured against other development work and resources required to fix the defect
- Process Implications for the development group to prevent future recurrences of similar defects
- Detailed Steps along with screenshots with which development can reproduce the defects
The primary push of Agile development is a high rate (velocity) of system version releases. To maintain that rate, teams may decide not to track detected defects for some part of or even the whole of the SDLC. While this is commonly done for efficiency and process overhead reduction, it drastically limits the available insights into software development capabilities and testing processes. It also impairs early process fault detection due to the lack of trustworthy data.
Many standards are available to help design the data collection process for defect reporting. Investigate IEEE 829, IEEE 1044 and ISO 25000 for examples of tried and proven defect management and classification systems.
The Upside of Software Defects
More than simply monitoring test progress, information captured in the defect management system supports development process and test procedure enhancement. Well recorded and managed defect data has a huge value for describing the state of a project (bugs discovered vs. fixed) and the state of the development process itself (recurrent bug causes).
Two important metrics that can be derived from a defect management system are:
- Rejection Ratio – the measure of false positives in the test results, literally how many bugs are actually not defects at all pointing to test process improvements
- Escape Rate – the count of bugs per version that are discovered in product releases (usually by customers) which is the final measurement of test efficacy
Well managed defect information holds the keys to both product and process improvement. It shows the way to make the code come out right the first time and improve customer/user satisfaction with the finished result.
Contact Us
Author: Gary James
