The 3 categories are :
1) Specification based or Black box testing.
2) Structured-based or White-Box Techniques.
3) Experienced Based Techniques.
As per the request of my blog readers I would like to post the "Structured-based or White-Box Techniques" first then later on continue with other design techniques.
TEST DESIGN TECHNIQUES for Structured-based (or) White-Box Techniques are:
-> Statement Testing Coverage
-> Decision Testing Coverage
{Statement Coverage & Decision Coverage : These 2 topics are covered in ISEB foundation Syllabus in 4th chapter "TEST DESIGN TECHNIQUES".}
Structured-based or White-Box Techniques :
White Box Testing :
-> Testing based on Knowledge of internal structure and logic.
-> Logic errors and incorrect assumptions are inversely proportional to a path’s execution probability.
-> We often believe that a path is not likely to be executed, but reality is often counter intuitive.
-> Measure Coverage.
Structured-based or White-Box Techniques is based on an identified structure of the software or system, as seen in the following examples:
Component level: The structure is that of the code itdelf, ie., statements, decisions or branches.
Integration level : The structure may be a call three (a diagram in which modules call other modules).
System level : the structure may be a menu structure, bussiness process or webpage structure.
Structure-based or white-box testing can be applied at different levels of testing. Here we will be focussing on white-box testing at the code level, but it can be applied wherever we want to test the structure of something - for example ensuring that all modules in a particular system have been executed.
** Further we will discuss about two code-related structural techniques for code coverage, based on statement and dicision, are discussed.
** For decision testing, a control flow diagram may be used to visualize the alternatives for each decision.
As said earlier, I focus main on code-related structural techniques. These techniques identify paths through the code tha need to be excercised in oredr to acheive the required level of code average.
These are methods that can be deployed that can make the identification of white-box test cases easier - one method is control-flow graphing, control flow graphing uses nodes, edges and regions.. I will show them in detail with examples here..
Now comming to the actual topic :
TEST DESIGN TECHNIQUES for Structured-based (or) White-Box Techniques are:
-> Statement Testing Coverage
-> Decision Testing Coverage
1. Statement testing & Coverage :
A Statement is:
>> 'An entity in a programming language, which is typically the smallest indivisible unit of execution' (ISTQB Def).
A Statement coverage is:
>> 'The percentage of executable statements that has been exercised by a test suite' (ISTQB Def)
Statement coverage:
-> Does not ensure coverage of all functionality
->
The objective if the statement testing is to show that the executable statements within a program have been executed at least once. An executable statement can be described as a line of program sourse code that will carry out some type of action. For example:
If all statements in a program have been executed by a set of tests then 100% statement coverage has been acheived. However, if only half of the statement have been executed by a set of tests then 50% statement coverage has been acheived.
The aim is to acheive the maximum amount of statement coverage with the mimimum number of test cases.
>>Statement testing test cases to execute specific statements, normally to increase statement coverage.
>>100% statement coverage for a component is acheived by executing all of the execuatbel statements in that component.
If we require to carry out statemtnt testing, the amount of statement coverage required for the component should be stated in the test coverage requirements in the test plan.We should aim to acheive atleast the minimim coverage requirements with our test cases. If 100% statement coverage is not required, then we need to determine which ares of the component are more important to test by this method.
>>Consider the following lines of code:
>> 1 test would be required to execute all three executable statements.
If our component consists of three lines of code we will execute all with one test case, thus acheiving 100% statement coverage. There is only one way we can execute the code - starting at line number 1 and finishing at line number 3.
>>statement testing is more complicated when there is logic in the code
>>For example..
>>Here there is one executable statement i.e., "Display error message"
>> hence 1 test is required to execute all executable statements.
Program code becomes tough when logic is introduced. It is likely what a component will have to carry out different actions depending upon circumstances at the time of execution. In the code examp,e shown, the component will do different things depending on whether the age input is less than 17 or if it is 17 and above. With the statement testing we have to determine the routes through the code we need to take in order to execute the statements and the input required to get us there!
In this example, the statement will be executed if the age is less than 17, so we would create a test case accordingly.
>> For more complex logic we could use control flow graphing
>> Control flow graphs consists of nodes, edges and regions
Control flow graphs describes the logic structure if the software programs - it is a method by which flows through the program logic are charted, usign the code itseld rather than the program specification. Each flow graph nodes and egdes The nodes represent computational statements or expressions, and the edges represent transfer of control between the nodes. Together the nodes and edges encompass an area known as a region.
In the diagram, the structure represents an 'If Then Else Endif' costurct. NOdes are shown for the 'If' and the 'Endif'. Edges are shown for the 'Then' ( the true path) and the 'Else ( the false path). The region is the area enclosed by the nodes and the edges.
>>All programs consists of these basic structures..
This is hetzel notation that only shows logic flow.
There are 4 basic structures that are used withn control-flow graphong.
The 'DoWhile' structure will execute a section of code whilst a feild or indicator is set to a certain value. For example,
The 'Do until' structure will execute a section of code until a field or indicator is set to a certain value. Foe example,
The evaluation of the condition occurs after the code is executed.
The 'Go To' structure will divert the program execution to the program section in question. For example
>> SO the logic flow code could now be shown as follows:
If we applied control-flow graphing to our sample code, then 'if Then Else' structure is applicable.
However, while it shows us the structure of the code, it doesn`t show us where the executabel statements are, and so it doesn`t help us at the moment with determining the tests we required for statement coverage.
>> we can introduce extra nodes to indicate where the executable statements are
>> And we can see the path we need to travel to execute the statemen in the code.
What we can do is introduce extra nodes to indicate where the statements occur in the program code.
NOw in our example we can see that we need to answer 'yes' to the question being posed to traverse the code and execute the statement on line 2.
>> Now consider this code and control flow graph:
>> We will need 2 tests to acheive 100% statement coverage.
Program logic can be a lot more complicated than the examples I have given so far!
In the source code shown here, We have executable statements associated with each outcome of the question being asked. We have to dosplay an error message if the age is less than 17( answering 'yes' to the question), and we have display 'costomer OK' if we answer 'No'.
We can only traverse the code only once with a given test; therefore we require two tests to acheive 100% statement coverage.
>> And this example...
>> We will need 3 tests to acheive 100% statement coverage.
NOw it get even more complecated!
In this example, we have a supplementary question, or what is know as a 'nested if'. If we answer 'yes' to 'If fuel tank empty?' we then have a further question asked, and each outcome of this question has an associated statement.
Therefore we will need two tests that answer 'yes' to 'if fuel tank empty'
* Fuel tank empty AND petrol engine ( to execute line 3)
* Fuel tanl empty AND NOT petrol engine( to execute line 5)
one further test will be required where we anser 'no' to 'if fuel tank empty' to enable us to execute the statement at line 8.
>>And this will be the last example for statement coverage.. we will then go for decision coverage.
>> We will need 2 tests to acheive 100% statement coverage.
In this example,,, we ahve two saperate questions that are being asked.
The tests have shown are
* A coffee drinker who wants cream
* A non coffee drinker who doesn`t want cream
Our 2 tests acheive 100% statement coverage, but equally we could have had 2 tests with:
* A coffee drinker who doesn`t want cream
* A no-coffee drinker who wants cream
If we were being asked to acheive 100% statement coverage, and if all statements were of equal importance, it would n`t matter which set if tests we chooose.
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Checking your calculation values:
Minimum tests required to acheive 100%
Decision coverage >= Statement coverage
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Decision testing & Coverage
A Decision is :
>> ' A program point at which the control flow has two or more alternative routes. A node with two or more links to saperate branches.'(ISTQB Def)
A Decision Coverageis :
>> ' The percentage if the decision outcomes that have been exercised by a test suite. 100% decision coverage implies both 100% branch coverage and 100% statement coverage.' (ISTQB Def)
Decision Coverage :
>>
The objective of decision coverage testing is to show all the decisions within a component have been executed at least once.
A decision can be described as a line of source code that asks a question.
For example:
If all decisions within a component have exercised by a given set of tests then 100% decision coverage has been achieved. However if only half of the decisions have been taken with a given set of tests then you have only achieved 50% decision coverage.
Again, as with statement testing, the aim is to achieve the maximum amount of coverage with the minimum number of tests.
>> Decision testing derives test cases to execute specific decision outcomes, normally to increase decision coverage.
>> Decision testing is a form of control flow testing as it generates a specific flow of control through the decision points.
If we are required to carry out decision testing, the amount of decision coverage required for a component should be stated in the test requirements in the test plan.
We should aim to achieve atleast the minimum coverage requirements with our test cases. If 100% decision coverage is not required, then we need to determine which areas of the component are more important to test by this method.
>> Decision coverage is stronger than statement coverage.
>> 100% decision coverage for a component is achieved by exercising all decision outcomes int he component.
>> 100% decision coverage guarantees 100% statement coverage, but not vice versa.
Decision testing can be considered as the next logical progression from statement testing in that we are not much concerned with testing every statement but the true and false outcomes from every decision.
As we saw in our earlier examples of the statement testing, not every decision outcome has a statement( or statements) to execute.
If we achieve 100% decision coverage, we would have executed every outcome of every decision, regardless of whether there were associated statements or not..
>>Lets take earlier example we had for statement testing:
>>This would require 2 tests to achieve 100% decision coverage, but only 1 test to achieve 100% statement coverage.
In this example there is one decision, and therefore 2 outcomes.
To achieve 100% decision coverage we could have two tests:
* Age less than 17(answer 'yes')
* Age equal to or greater than 17 (answer 'no')
This is a greater number of tests than would be required for statement testing as statements are only associated with one decision outcome(line 2).
>> Again, consider this earlier example :
>> we will need 2 tests to achieve 100% decision coverage & also 2 tests to achieve 100% statement coverage.
This example would still result in two tests, as there is one decision therefore 2 outcomes to tests.
However, we would need two tests to achieve 100% statement coverage, as there are statements with each outcome of the decision.
So,in this instance, statement and decision testing would give us the same number of tests. NOte that if 100% coverage is required, statement testing can give us the same number of tests as decision testing, BUT NEVER MORE!
>>Lets look at some more examples now..
>> We will need 3 tests to achieve 100% decision coverage, but only 1 test to achieve 100% statement coverage.
Here we have an example of a supplimentary question, or a 'nested if'.
We have 2 decisions, so you may think that 4 tests may be required to achieve 100% decision coverage( two for each decision).
This is NOT the case! We can achieve 100% decision coverage with three tests - we need to exercise the 'Yes' outcome from the first decision ( line 1) twice, in order to subsequently exercise the 'Yes' and then the 'No' outcome from the supplementary question(line 2).
We need a further third test to ensure we exercise the 'No' outcome of the first decision( line 1 ).
There is only one decision outcome that has an associated statement - this means that 100% statement coverage can be achieved with one test.
>> As more statements are added, the tests for decision coverage are the same:
>> 3 tests to achieve 100% decision coverage, and 2 tests to achieve 100% statement coverage.
We have now introduced a statement that is associated with 'No' outcome of the decision on line 2.
This change affects the number of tests required to achieve 100% statement coverage, but does NOT alter the number of tests required to achieved 100% decision coverage - it is still three!
>And again an example..
>> 3 tests to achieved both decision and statement coverage.
Finally, we have statements associated with each outcome of each decision - the number of tests to achieve 100% statement coverage and 100% decision coverage are now the same.
>> And Last Example..
>> We will need 2 tests to achieve 100% decision coverage and 100% statement coverage.
We looked at this example of the "if Then Else' structure when considering statement testing.
As the decisions are separate questions we only need two tests to achieve 100% decision coverage( the same as the number required for statement coverage).
You may have thought that four tests were required - exercising the four different routes through the code, but remember, with decision testing our concern is to exercise each outcome of each decision atleast once - as long as we have answered 'Yes' and 'No' to each decision we have satisfied the requirements of the techinique.
The tests we have illustrated would need the following input conditions:
* Coffee drinker wanting cream.
* Non Coffee drinker not wanting cream ( but milk).
Equally, we could have chosen the following input conditions:
* Coffee drinker not wanting cream( but milk).
* Non coffee drinker wanting cream.
>> Then What about loops?
>> If we choose an initial value of p=4, we only need 1 test to achieve 100% statement and 100% decision coverage.
The control-flow graphs we showed earlier depicted a 'Do While' construct.
To reiterate, thw 'Do While' structure will execute a section of code whist a field or indicator is set to a certain value. For example,
The evaluation of the condition occurs before the code is executed.
Unlike the 'If Then Else', we can loop around the 'Do While' structure, which means that we exercise different routes through the code with one test.
As in the above diagram, if we set 'p' with an initial value '4', the first time through the code will :
* Go from line 1 to line 2
* Answer 'Yes' to the 'If' on line 2 ( if p<5)
* Execute the statement in line 3 (p=p*2, so p now equals 8)
* Go from line 3, through line 4 to line 5
* Execute the statement on line 5 ( which adds 1 to 'p', making it`s value '9')
* Execute the statement on line 6, which takes it back up to line 1.
Again we execute the code, with the value of 'P' now '9'
* GO from line1 to line2
* Answer 'NO' to the 'if' on line 2 (If p>5)
* Go from line 4 to line 5
* Execute the statement on line 5( which adds 1 to 'p', making it`s value '10')
* Execute the statement on line 6, which takes it back up to line 1.
Once more we execute the code
* Line 1 - 'P' is not less than '10' ( it is equal to 10), therefore, we exit this structure.
1 test - it achieves 100% statement coverage and 100% decision coverage.
>> And it`s same for 'Do until' structure
>> IF we choose an initial value of A =15, we only need 1 test to achieve 100% Decision coverage and 100% statement coverage.
The control flow structures we showed earlier also depicted a 'Do Until' structure.
To reiterate, the 'Do Until' structure will execute a section of code until a field or indicator is set to a certain value. For example,
The evaluation of the condition occurs after the code is executed.
Unlike the 'If Then Else', we can loop around the 'Do Until' structure, which means that we exercise different routes through the code with one test.
In the example above, If we set 'A' with an initial value of '15', the first time through the code will:
* Go from line 1 to line 2
* Answer 'Yes' to the 'If' on line 2 (If A<20)
* Execute the statement on line 3 ( A=A*2,which makes A=30)
* GO from line 3, through the line 4 to line 5
* Execute the statement on line 5( which adds 1 to 'A', making its value '31'.
* Execute the statement on line 6, Which takes back to line 1
Again we execute the code, with the value of 'A' now '31'
* Go from line 1 to line 2
* Answer 'No' to the 'If' on line 2 (If A < 20)
* Go from line 2, through line 4 to line 5
* Execute the statement on line 5( which adds 1 to 'A', making it`s value '32')
* Execute the statement on line 6, which exits the structure('A' is greater than 31)
1 test - it achieves 100% statement coverage and 100% decision coverage.
END...
No comments:
Post a Comment