Differences Between Testing Strategies

Question:
What are the differences between sanity testing, smoke testing,
regression testing, and monkey testing?

Posted by prasad

Answer:
First, quite often the terms are not used correctly and you should learn
the meaning of the terms where you work and try to help adjust them to
industry norms. However, these terms may be used differently in your company since there are no formal standards for them, and most people wouldn't bother to learn the actual definitions if they existed.

Smoke tests get their name from the electronics industry. The circuits are
laid out on a bread board and power is applied. If anything starts smoking,
there is a problem. In the software industry, smoke testing is a shallow and
wide approach to the application. You test all areas of the application
without getting too deep. This is also known as a Build Verification test or
BVT.

In comparison, sanity testing is usually narrow and deep. That is they look
at only a few areas but all aspects of that part of the application. A smoke
test is scripted--either using a written set of tests or an automated
test--whereas a sanity test is usually unscripted.

A monkey test is also unscripted, but this sort of test is like a room full
of monkeys with a typewriter (or computer) placed in front of each of them.
The theory is that, given enough time, you could get the works of
Shakespeare (or some other document) out of them. This is based on the idea that random activity can create order or cover all options. Jamie Andrews
has been doing some work on an organised version of this at the University
of Western Ontario, London, Ontario, Canada. This method is called
coverage-checked random unit testing (CRUT). You can read about it at
http://www.csd.uwo.ca/faculty/andrews/papers/index.html.

Gorilla testing however has nothing to do with this. It is an intense round
of testing--quite often redirecting all available resources to the activity.
The idea here is to test as much of the application in as short a period of
time as possible.

Just to clarify here: a Sanity test is not the same as a Smoke Test or a
Build Verification test. The former is to determine a small section of the
application is still working after a minor change (which is not a good
policy, btw--you should do a regression test instead) and a Smoke or Build
Verification Test (BVT) is designed to touch every part of the application
in a cursory way. It's is shallow and wide. A Sanity test tends to be
unscripted, but you could (or should!) use a sub-set of your existing test
cases to verify a small part of your application. This is not quite a
regression test, where all areas of the application are tested using a
subset of the test cases.

A regression test is a more complete Smoke Test or BVT. A sanity test is a narrow regression test that focuses on a one or a few areas of functionality.

Posted by Walter Görlitz