UNIT TESTING, BLACK-BOX TESTING AND WHITE-BOX TESTING

Read Time:9 Minute, 56 Second

UNIT TESTING

Unit testing is undertaken after a module has been coded and reviewed.

This activity is typically undertaken by the coder of the module himself in the coding phase.

Before carrying out unit testing, the unit test cases have to be designed and the test environment for the unit under test has to be developed.

Driver and stub modules

In order to test a single module, we need a complete environment to provide all relevant code that is necessary for execution of the module.

That is, besides the module under test, the following are needed to test the module:

The procedures belonging to other modules that the module under test calls.
Non-local data structures that the module accesses.
A procedure to call the functions of the module under test with appropriate parameters.

Modules required to provide the necessary environment (which either call or are called by the module under test) are usually not available until they too have been unit tested.

In this context, stubs and drivers are designed to provide the complete environment for a module so that testing can be carried out.

Stub:

The role of stub and driver modules is pictorially shown in Figure 10.3.

A stub procedure is a dummy procedure that has the same I/O parameters as the function called by the unit under test but has a highly simplified behaviour.

For example, a stub procedure may produce the expected behaviour using a simple table look up mechanism.

Figure 10.3: Unit testing with the help of driver and stub modules.

Driver:

A driver module should contain the non-local data structures accessed by the module under test.

Additionally, it should also have the code to call the different functions of the unit under test with appropriate parameter values for testing.

BLACK-BOX TESTING

In black-box testing, test cases are designed from an examination of the input/output values only and no knowledge of design or code is required.

The following are the two main approaches available to design black box test cases:

Equivalence class partitioning
Boundary value analysis

Equivalence Class Partitioning

In the equivalence class partitioning approach, the domain of input values to the program under test is partitioned into a set of equivalence classes.

The partitioning is done such that for every input data belonging to the same equivalence class, the program behaves similarly.

The main idea behind defining equivalence classes of input data is that testing the code with any one value belonging to an equivalence class is as good as testing the code with any other value belonging to the same equivalence class.

Equivalence classes for a unit under test can be designed by examining the input data and output data.

The following are two general guidelines for designing the equivalence classes:

1. If the input data values to a system can be specified by a range of values, then one valid and two invalid equivalence classes need to be defined.

For example, if the equivalence class is the set of integers in the range 1 to 10 (i.e., [1,10]), then the invalid equivalence classes are [−∞,0], [11,+∞].

2. If the input data assumes values from a set of discrete members of some domain, then one equivalence class for the valid input values and another equivalence class for the invalid input values should be defined.

For example, if the valid equivalence classes are {A,B,C}, then the invalid equivalence class is U-{A,B,C}, where U is the universe of possible input values.

Boundary Value Analysis

A type of programming error that is frequently committed by programmers is missing out on the special consideration that should be given to the values at the boundaries of different equivalence classes of inputs.

The reason behind programmers committing such errors might purely be due to psychological factors.

Programmers often fail to properly address the special processing required by the input values that lie at the boundary of the different equivalence classes.

For example, programmers may improperly use < instead of <=, or conversely <= for <, etc.

Boundary value analysis-based test suite design involves designing test cases using the values at the boundaries of different equivalence classes.

To design boundary value test cases, it is required to examine the equivalence classes to check if any of the equivalence classes contains a range of values.

For those equivalence classes that are not a range of values (i.e., consist of a discrete collection of values) no boundary value test cases can be defined.

For an equivalence class that is a range of values, the boundary values need to be included in the test suite.

For example, if an equivalence class contains the integers in the range 1 to 10, then the boundary value test suite is {0,1,10,11}.

WHITE-BOX TESTING

White-box testing is an important type of unit testing.

A large number of white-box testing strategies exist.

Each testing strategy essentially designs test cases based on analysis of some aspect of source code and is based on some heuristic.

Basic Concepts

A white-box testing strategy can either be coverage-based or faultbased.

Fault-based testing

A fault-based testing strategy targets to detect certain types of faults.

These faults that a test strategy focuses on constitutes the fault model of the strategy.

An example of a fault-based strategy is mutation testing.

Coverage-based testing

A coverage-based testing strategy attempts to execute (or cover) certain elements of a program.

Popular examples of coverage-based testing strategies are statement coverage, branch coverage, multiple condition coverage, and path coverage-based testing.

Testing criterion for coverage-based testing

A coverage-based testing strategy typically targets to execute (i.e., cover) certain program elements for discovering failures.

The set of specific program elements that a testing strategy targets to execute is called the testing criterion of the strategy.

For example, if a testing strategy requires all the statements of a program to be executed at least once, then we say that the testing criterion of the strategy is statement coverage.

We say that a test suite is adequate with respect to a criterion, if it covers all elements of the domain defined by that criterion.

Stronger versus Weaker testing

A white-box testing strategy is said to be stronger than another strategy, if the stronger testing strategy covers all program elements covered by the weaker testing strategy, and the stronger strategy additionally covers at least one program element that is not covered by the weaker strategy.

When none of two testing strategies fully covers the program elements exercised by the other, then the two are called complementary testing strategies.

Statement Coverage

The statement coverage strategy aims to design test cases so as to execute every statement in a program at least once.

The principal idea governing the statement coverage strategy is that unless a statement is executed, there is no way to determine whether an error exists in that statement.

It is obvious that without executing a statement, it is difficult to determine whether it causes a failure due to illegal memory access, wrong result computation due to improper arithmetic operation, etc.

It can however be pointed out that a weakness of the statement- coverage strategy is that executing a statement once and observing that it behaves properly for one input value is no guarantee that it will behave correctly for all input values.

Nevertheless, statement coverage is a very intuitive and appealing testing technique.

Design statement coverage-based test suite for the following Euclid’s GCD computation program:

int computeGCD(x,y)

int x,y;

{

while (x != y)

{

if (x>y) then 3 x=x-y;

else y=y-x;

}

return x;

}

Answer: To design the test cases for the statement coverage, the conditional expression of the while statement needs to be made true and the conditional expression of the if statement needs to be made both true and false.

By choosing the test set {(x = 3, y = 3), (x = 4, y = 3), (x = 3, y = 4)}, all statements of the program would be executed at least once.

Branch Coverage

A test suite satisfies branch coverage, if it makes each branch condition in the program to assume true and false values in turn.

In other words, for branch coverage each branch in the CFG representation of the program must be taken at least once, when the test suite is executed.

Branch testing is also known as edge testing, since in this testing scheme, each edge of a program’s control flow graph is traversed at least once.

It is easy to show that branch coverage-based testing is a stronger testing than statement coverage-based testing.

We can prove this by showing that branch coverage ensures statement coverage, but not vice versa.

Multiple Condition Coverage

In the multiple condition (MC) coverage-based testing, test cases are designed to make each component of a composite conditional expression to assume both true and false values.

For example, consider the composite conditional expression ((c1 .and.c2 ).or.c3).

A test suite would achieve MC coverage, if all the component conditions c1, c2 and c3 are each made to assume both true and false values.

Branch testing can be considered to be a simplistic condition testing strategy where only the compound conditions appearing in the different branch statements are made to assume the true and false values. It is easy to prove that condition testing is a stronger testing strategy than branch testing.

For a composite conditional expression of n components, 2n test cases are required for multiple condition coverage.

Thus, for multiple condition coverage, the number of test cases increases exponentially with the number of component conditions.

Therefore, multiple condition coverage-based testing technique is practical only if n (the number of conditions) is small.

Path Coverage

A test suite achieves path coverage if it exeutes each linearly independent paths ( o r basis paths ) at least once.

A linearly independent path can be defined in terms of the control flow graph (CFG) of a program.

Therefore, to understand path coverage-based testing strategy, we need to first understand how the CFG of a program can be drawn.

Control flow graph (CFG)

A control flow graph describes how the control flows through the program. We can define a control flow graph as the following:

A control flow graph describes the sequence in which the different instructions of a program get executed.

In order to draw the control flow graph of a program, we need to first number all the statements of a program.

The different numbered statements serve as nodes of the control flow graph (see Figure 10.5).

There exists an edge from one node to another, if the execution of the statement representing the first node can result in the transfer of control to the other node.

A CFG is a directed graph consisting of a set of nodes and edges (N, E), such that each node n N corresponds to a unique program statement and an edge exists between two nodes if control can transfer from one node to the other.

We can easily draw the CFG for any program, if we know how to represent the sequence, selection, and iteration types of statements in the CFG.

After all, every program is constructed by using these three types of constructs only.