SOFTWARE TESTING
Introduction
Software testing is one of the quality assurance activities, which aims at identifying and eliminating errors in a computer program. Software errors can cause software to fail, therefore the need for software testing cannot be over-emphasised. Between 30% and 40% of software project efforts can be devoted to testing. It begins after the codes have been generated, before delivery to customers. It involves executing the software with the aim of uncovering and correcting errors. Software testing requires designing test cases that have likelihood of uncovering and correctng errors. A test case is an execution path of a program during testing. Software testing techniques help in designing the test cases. Software testing techniques provide systematic guidance that execute the internal logic of the software together with the input and output domains of the program. Software testing can be performed by the software engineers or testing specialists. Software testing techniques are of two types, which are
- White box test case design: This tests the internal program logic
- Black box test case design: This tests the input and output software requirement.
- Fundamentals of Software Testing
One of the software myths says that you cannot catch bug in a software developed by a good programmer. The myth says that the reasons why errors are found in a program is because programmers are not good in what they are doing. Therefore, according to the myth, whenever testing uncovers errors, programmers should be guilty because they are not good programmers. The myth concludes by saying that testing and test case design is an admission of failure, which instills guilt because programmers are not good in what they are doing. However, the reality about this myth is in the fundamental objectives of software testing.
- Fundamental Objectives of Software Testing
The following serves as the fundamental objectives of software testing:
- Software testing is the process of executing a program with the aim of finding an error.
- A good test case design is one that has a high probability of finding an undiscovered error.
- A successful test is one that uncovers an undiscovered error.
Another myth about software testing, which is a long held view is that a successful test is one that does not uncover any error. Therefore, the above objectives about software testing is opposed to this long held view and myth about software testing. The objective in test case design is to design test that systematically uncovers different classes of errors with minimum amount of time and efforts.
Testing, therefore makes software appear to be working according to specification. It makes the behavioural and performance requirement of software appear to have been met.
- Principles of Software Testing
The following principles guides software testing:
- All tests are traced to customer requirements
- Tests are planned, long before testing begins.
- Pareto principles applies to software testing. This principle states that 80% of bugs are found in 20% of the program. This means that majority of the bugs in software are located in small part of the program. These small part of the program where the bugs are located must be found.
- Testing begins small and progresses big.
- Testing should be conducted by an indepemdent party.
- Software Testability
The term, testability means different things to defferent people. To some, it can mean how adequate a set of tests cover the product. To others, it can mean how easily a product can be checked and repaired in the field.
However, software testability does not mean any of the above. Software testability means how easily software can be tested. It means how easily the person charged to test software can design effective test cases, i.e. one with the highest probability of uncovering errors. A testable software will possess the following characteristics:
- Operability
The better it works, the more efficient it can be tested.
- Observability
What you see is what you test
- Controllability
The better we can control the software, the more the testing can be automated and optimized.
- Decomposability
Quick isolation of problems leads to smarter retesting.
- Simplicity
- Stability
- Understandability
- Test Case Design
There are two approaches to designing the test case of software:
- Black box testing
This approach is used to ensure that input is properly captured and output is correctly produced.
- White box testing
This approach is also called glass box testing. It uses the control structure of the program to design the test case. One of the techniques for white box testing is called Basis Path Testing.
- Basis Path Testing and Flow Graph Notation
| A |
This is a technique for white box testing, which uses logical complexity measure of the program as a guide for designing the basis set of execution paths. In order to obtain the logical complexity measure of the program, a notation called flow graph notation or program graph is used. Flow graph notation is a notation for representing the logical control flow of a program. We begin by considering the flow graphs of some of the simple control structures with simple logical expressions, as stated below:
F
| B |
while (A) T
{
| out |
B
}
| A |
do
| out |
| B |
{ T
A
} while (B) F
| A |
if (A) T
{ F
| B |
| C |
B
}
else
{
| out |
C
}
These simple control structures can be combined to form nested control structure, with simple logical expressions as illustrated below:
while(A)
{
B
if (C)
{
D;
E
}
else
{
if (F)
{
G
}
else
{
H
}
}
}
The following guideline can be used in developing the flow graph of such nested construct:
- In the flow graph, the node of the graph is represented as a circle.
- A sequence of a statement and a logical control structure will be combined to form a node.
- A sequence of two or more statements will be combined to form a node.
- A statement that stands alone will be represented as a node.
- A logical expression of a control structure will be represented as a node.
- The sequential flow of the program will be represented as edges/link of the flow graph.
- Each edge or link must terminate in a node. The node may represent a statement or logical expression in the program construct.
- Edges that originate from a node that evaluate logical expression can be labeled as T for TRUE and F for FALSE.
Using the above guideline, we can develop the flow graph of the above nested program construct as follows:
| A |
| BC |
T
F T
| F |
| DE |
F T
| H |
| G |
| out |
| HG |
| HGDE |
Figure 4.4.4 Flowgraph of the nested construct
However, some program constructs can contain one control structure with complex logical expression, as illustrated in the following examples
Examples: The flow graph of the following program constructs follow:
Some logical expressions that consists of two or more logical operations can be represented in a flow graph as follows:
| b |
| C |
| a |
if (a OR b)
{
C F T
}
else
{ T
D
| D |
| out |
} F
Figure 4.4.5
Consider another example as shown below:
If (a AND b)
| a |
{
| b |
| C |
C
} T F
else
{
D
| out |
| D |
} T F
Figure 4.4.6
More examples of complex logical expression and its flow graph.
if ((a OR b) AND c)
| a |
{
D F T
| c |
| b |
} T
else
{ F F T
| D |
| E |
E
}
| out |
Figure 4.4.7
Consider another example:
| a |
If ((a AND b) OR c)
{
| D |
| b |
| E |
| c |
| out |
D F T
} F
else
{ F T T
E
}
Figure 4.4.8
- Cyclomatic Complexity
We have said in the last section that the basis path testing technique uses logical complexity measure of the program as a guide for designing the basis set of execution paths (test case). Cyclomatic complexity is a software metric/measure that provides quantitative measure of the logical complexity of a program. Therefore, it can be used as a logical complexity measure of a program, as a guide in defining the basis set of execution path.The cyclomatic complexity determines the number of independent paths in the basis set of a program. An independent path is a path through the program that introduces at least one new statement or condition. An independent path, when defined in terms of flow graph must introduce at least an edge that it has not transversed before the path was created. In figure 4.4.4, the following are the independent paths:
Path 1: A -> Out
Path 2: A -> BC -> F -> H -> HG -> HGDE -> A -> Out
Path 3: A -> BC -> F -> G -> HG -> HGDE -> A -> Out
Path 4: A -> BC -> DE -> HGDE -> A -> Out
Paths 1, 2, 3 and 4 are the basis set for the flow graph in figure 4.4.4. The independent paths are not unique. The computation of the cyclomatic complexity helps to determine how many independent paths are available in a flow graph. It is computed using any of these formulae:
CC = E – N + 2 where E is the number of edges in the flow graph, and N is the number of nodes in the flow graph.
CC = P + 1, where P is the number of predicate nodes. Predicate nodes are nodes that evaluate logical expression.
4.6 Graph Matrix
This is a matrix representation of the flow graph. It is a square matrix whose dimension is the number of nodes of the flow graph. The edges of the flow graph is represented in the graph matrix by marking an X in the appropriate cell of the matrix. The graph matrix provides another method for computing the cyclomatic complexity of a program. The following example illustrates further.
Consider the flow graph in figure 4.4.8, the flow graph can be represented as graph matrix, as shown below:
| A | b | c | D | E | Out | |
| a | X | X | ||||
| b | X | X | ||||
| c | X | X | ||||
| D | X | |||||
| E | X | |||||
| Out |
The above graph matrix can be used to compute the cyclomatic complexity of the program construct as follows:
- In each row of the graph matrix, marked with at least one X, count the number of cells, marked X, and subtract 1 from the number counted.
- Add all the results you obtained from the above calculation, for all the rows.
- Add 1 to the result.
The final result will be the cyclomatic complexity, which will be the same as the other methods that we have outlined earlier. When we follow the above procedure, we shall obtain the following:
| a | B | c | D | E | Out | ||
| A | X | X | 2-1 = 1 | ||||
| B | X | X | 2-1 = 1 | ||||
| C | X | X | 2-1 = 1 | ||||
| D | X | 1-1 =0 | |||||
| E | X | 1-1 =0 | |||||
| Out |
Therefore, the cyclomatic complexity is 3+1 = 4
4.7 More Examples
Consider the following program construct
while ((a OR b) and (c AND d)
{
E
}
We are required to draw the program flow graph, and use the program flow graph to draw the graph matrix. Finally, use the graph matrix to determine the cyclomatic complexity of the program construct.
| a |
| c |
| d |
| E |
| b |
| out |
Figure 4.7.1
| a | B | c | d | E | out | ||
| a | X | X | 2-1=1 | ||||
| b | X | X | 2-1=1 | ||||
| c | X | X | 2-1=1 | ||||
| d | X | X | 2-1=1 | ||||
| E | X | 1-1=0 | |||||
| Out | |||||||
| 4+1=5 |
Exercise
Draw the graph matrix for the flow graphs in figures 4.4.4, 4.4.5, 4.4.6, 4.4.7 and use the graph matrix to determine the cyclomatic complexity of each of the program construct that each flow graph represents.
References
Software Engineering: APractitioner Approach by Roger S. Pressman