Software Testing Basics
Note: Different testing strategies are described here. In practice, a detail test plan is helpful that may be formed with a subset of these strategies [and delivered to the testers] according to the need of the customers, and the company policies.
Five Fold Testing System
Any testing can be described in five dimensions:
1. Testers: Who does the testing?
2. Coverage: What needs to be tested? function, domain, extreme value?
3. Potential Problems: Why you are testing?
4. Activities: How you test?
5. Evaluation: How to measure the success of testing?
Some Forms of Testing
1. Function testing: Test all functions
2. Extreme Value Testing: Test for errors when extreme values of the variables are used
3. beta testing: make other people test your product
All testing involve five dimensions. When we tell to do function testing, we can list: who will do the test (programmers, testers, users of the product, internal employees), what needs to be tested like functions, what kind of problems they need to address such as pass by value or pass by reference, test method, evaluate your test
People Based Techniques
1. User Testing: Tests by the real users of the product
2. Alpha testing: By Test team, firendly insiders
3. Beta testing: Not insiders, not clients, but may be part of the target market, volunteers
Design Beta tests: appraise the design
marketing beta tests: to justify will customers buy the product?
compatibility beta tests: test hw/sw platform compatibility
4. bug bashes: in house testing, just before release one full day/half day testing
5. subject matter testing: give an expert on the subject to use and test the product
6. Paired testing: two person together do the tests and shares ideas
7. eat your own dogfood: use the software in-house for your own purpose and check reliability
Coverage based testing
1. Function testing: Test each function
White box function testing: testing in respect of code
Black box function testing: test functions in respect of features and commands
2. Function integration testing: test several functions together
3. Menu tour: check all menu items, all available choices
4. Domain testing: Test variable domains and their effect on functions. Find the possible values that a variable can take, classify the value domain, take representative values from each domain, test the variables and related functions
5. Equivalence class analysis: find equivalent domains and tests for one of them
6.Boundary test: Check for smallest and largest values for a variable
7. best representative testing
8. input field test catalogs or matrices
9. Map and test all the ways to edit a field
10. Logic testing: use logics like if then else. cause effect graphing is used for logic based testing
11. state based testing:
12. Path testing: All possible paths to come/go to a state
13. Statement and branch coverage: execute all statements and all branching
14. Configuration coverage: test how many types of printers your software support
15. Specification based testing: Test all claims made in the manual/specification
16. Requirements-based testing: test if all requirements are met
17. combination testing: testing two or more variables in combination with each other
Problems based techniques
1. Input constraints Tests: Check what kinds of input the program can handle. Also check input error protection
2. Output constraints: The inputs are legal but check if they lead to corresponding out value
3. computation constraints : the program may fail while calculating a value
4. Storage constraints: out of memory tests, too big output file to process
5. timimg: race condition, ordering of events
Activity based Testing
How you test?
1. Regression Testing: Test the same issue again.
a. Bug fix regression : bug is fixed but do the test again to prove the bug is not fixed
b. old bugs regression: new bug fix has unfixed old bugs. so test again
c. side effect regression/stability regression : the bug fix has created new bugs/side effects
2. Scripted Testing: execute step by step methods written by senior tester
3. smoke testing : kind of side effect regression testing. but test to prove that no side effect has come. show everything is working. assume may be some simple stuffs have caused the problems not the bug fix. if you can not prove then the new bug fix has created new errors
4. exploratory testing: keep the testers known about the project, product, market, risks. So new tests will be stronger than earlier ones as experience is more
5. guerilla testing: a fast and extensive exploratory test done by senior tester. test whole day a particular part extensively and decide if that area needs much/more testing or can be ignored
6. scenario testing:
a. test what the customer will really do
b. complex feature testing
c. easy to decide success/failure
tests that come from use cases.
7. Installation testing: install in various platform, in different methods, check the files, check if the program works?, check uninstallation
8. Load testing: Test the system at high enough load
9. long sequence testing: run the program for days, weeks. problems like wild pointers, memory leaks, stack overflows, and bad interaction among multiple features are caught by this duration testing
10. Performance testing: test the speed/efficiency of the software. A significant change in speed from time to time is a bad indication and the software need to be optimized
Evaluation based testing
Methods to determine if a test has passed or failed
1. Self verifying data:
2. compare with saved results: if results vary over the week/day, something is wrong
3. compare with specification: mismatch with the specification, failure
a. with history: software behavior consistent with past behavior?
b. with company image: matches with the image that company focuses on?
c. with claims: does what it is supposed to do
d. user's expectations: meets users expectations
e. within product: consistent with other functions
f. purpose: consistent with it's purpose?