Risk-based testing

Risk-based testing (RBT) is a type of software testing that functions as an organizational principle used to prioritize the tests of features and functions in software, based on the risk of failure, the function of their importance and likelihood or impact of failure.^[1][2][3][4] In theory, there are an infinite number of possible tests. Risk-based testing uses risk (re-)assessments to steer all phases of the test process, i.e., test planning, test design, test implementation, test execution and test evaluation.^[5] This includes for instance, ranking of tests, and subtests, for functionality; test techniques such as boundary-value analysis, all-pairs testing and state transition tables aim to find the areas most likely to be defective.

Assessing risks

Comparing the changes between two releases or versions is key in order to assess risk. Evaluating critical business modules is a first step in prioritizing tests, but it does not include the notion of evolutionary risk. This is then expanded using two methods: change-based testing and regression testing.

• Change-based testing allows test teams to assess changes made in a release and then prioritize tests towards modified modules.
• Regression testing ensures that a change, such as a bug fix, did not introduce new faults into the software under test. One of the main reasons for regression testing is to determine whether a change in one part of the software has any effect on other parts of the software.

These two methods permit test teams to prioritize tests based on risk, change, and criticality of business modules. Certain technologies can make this kind of test strategy very easy to set up and to maintain with software changes.

Types of Risks

Risk can be identified as the probability that an undetected software bug may have a negative impact on the user of a system.^[6]

The methods assess risks along a variety of dimensions:

Business or Operational

• High use of a subsystem, function or feature
• Criticality of a subsystem, function or feature, including the cost of failure

Technical

• Geographic distribution of development team
• Complexity of a subsystem or function

External

• Sponsor or executive preference
• Regulatory requirements

E-business failure-mode related

• Static content defects
• Web page integration defects
• Functional behavior-related failure
• Service (Availability and Performance) related failure
• Usability and Accessibility-related failure
• Security vulnerability
• Large scale integration failure

^[7]

References

1. Gerrard, Paul; Thompson, Neil (2002). Risk Based E-Business Testing. Artech House Publishers. ISBN 1-58053-314-0.
2. Bach, J. The Challenge of Good Enough Software (1995)
3. Bach, J. and Kaner, C. Exploratory and Risk Based Testing (2004)
4. Mika Lehto (October 25, 2011). "The concept of risk-based testing and its advantages and disadvantages". Ictstandard.org. Retrieved 2012-03-01.
5. Michael Felderer, Ina Schieferdecker: A taxonomy of risk-based testing. STTT 16(5): 559-568 (2014)
6. Stephane Besson (2012-01-03). "Article info : A Strategy for Risk-Based Testing". Software Quality Engineering IT. Stickyminds.com. Retrieved 2012-03-01.
7. Gerrard, Paul and Thompson, Neil Risk-Based Testing E-Business (2002)