Implementation to Test

Implementation artifacts - primarily executable code - are the primary channel of information from development to test execution.

Implementation status is another important part of the flow of information. For example, in a large project some parts of the implementation will be ready before others. If the implementation strategy provides a complete testable interface with incomplete logic behind the interface then the test efforts can be directed toward the most advanced parts of the implementation. The test results can to some extent be predicted.

Backlog

Some of the artifacts in each class will represent approved, agreed-upon, active, or completed tasks.

Others will represent proposed tasks, work approved but not scheduled, etc. These artifacts comprise the backlog. A backlog is a natural and healthy part of the development process as long as it is managed correctly. One of the themes of this discussion will be to indicate when and how to address items in the backlog.

Here are some of the categories of the backlog.

Requirements that are published but not represented in specifications and/or test plans are in the design backlog.

Specifications that are not implemented are in the engineering backlog.

Implementation artifacts that have not been tested are in the test backlog.

Test plans and automation that have not been executed are also in the test backlog.

Test results that have not been reviewed are in the results backlog.

Customer reports that have not been triaged are in the support backlog.

Customer reports that have been triaged as defects but have not been fixed are in the engineering backlog.

Defects that have been fixed but where the fixes have not been delivered to customers are in the release backlog.

These are the most important components of the backlog.

Requirements to Test Results

Test results must correspond to requirements in terms of scope. That is, there must be a correspondence between features as described in requirements artifacts and successful test results. This mapping is driven by the coverage of features by test plans.

When required features are ranked by priority, test results can be used to evaluate release readiness. A release cannot be approved until all required, essential, or "release defining" features have been tested and these tests have passed.

Overview

This document will define the scope of our discussion of SDLC.

We begin with the definition of the six primary classes of project artifacts.

1. Requirements
2. Specifications
3. Implementation
4. Test Plans and Automation
5. Test Results
6. Customer Reports

Requirements are documents that describe the initial or general goals of the software to be produced. These are written in a natural language (English in this case) by product management or a business process analyst. Requirements are typically based on intangibles like imagination and insight and form the primary reference documents for a project. These documents are used in turn by the engineering and documentation functions as the basis for more artifacts.

Specifications are engineering documents that describe architecture, design and development plans. These are written in a natural language by engineering management. They are based on requirements documents but include much more detail.

Implementation is produced by the engineering team from specifications and is typically in a language that can be interpreted and executed by a computer. Implementation artifacts are the tangible assets produced by the development process. Some of these assets are transferred from the development group to the customers in the form of a product or system.

Test Plans are produced by test engineering and serve as a framework for evaluation of the implementation. Test plans are based on both requirements and specifications, but not on the implementation.

Test Results are produced by test execution. Results are often in both natural language and machine-readable form.

Customer Reports are natural language documents created ad hoc as a result of the use of the implementation.

Backlog is orthogonal to the other six. It is a measure of the portion of the available artifacts (of any type) that has not yet been used to generate work or other results.

Many other useful terms will be defined in the discussion to follow. The six artifact classes stand out because they are well-understood and can be clearly defined. Most project documents belong in one of these classes.

Each of these artifact classes has a relationship to every other artifact class. Every one of these relationships will be explored at some point in the documents to follow.