Risk-Based Testing: How to Decide What Not to Test
The Quick Answer
Risk-based testing is a prioritisation method. It ranks what to test by how much damage a failure would cause and how likely that failure is, then spends testing effort in proportion to that ranking. Its real value is not in deciding what to test - most teams can list that easily - but in deciding, defensibly, what to leave alone.
The core idea: Risk equals impact multiplied by likelihood. A feature where failure would be catastrophic and the code is new and complex gets deep testing. A feature where failure is cosmetic and the code has not changed in two years gets a smoke check, or nothing.
Why it matters: Every team has less time than it would like. Without an explicit prioritisation method, that shortfall gets resolved implicitly - by testing what is easy, what is familiar, or what was tested last time - and nobody can explain to a stakeholder why a given area was skipped.
This article covers how to score risk, how to build and use a risk matrix, how to translate scores into concrete test depth, how to decide what to drop, and the failure modes that make risk-based testing an exercise in paperwork instead of a decision tool. For the wider planning context this sits inside, see our guide to QA testing strategy; for how to measure what you ended up covering, see test coverage.
What Risk-Based Testing Is - and Is Not
Risk-based testing is a way of allocating a fixed testing budget. It does not tell you how to test something, what techniques to use, or how to measure coverage. It tells you where the effort goes and, critically, where it does not.
The Assumption It Rejects
Uniform testing - treating every feature as equally deserving of attention - feels fair and is nearly always wrong. Features do not carry equal consequences. A failure in payment processing and a failure in a settings page label are not comparable events, and spreading effort evenly across both means the payment path is under-tested so that the label can be over-tested.
What It Is Not
Risk-based testing is not an excuse to skip testing and call it a strategy. The difference between prioritisation and negligence is documentation: a risk-based decision is written down, has a rationale, has been reviewed by people who understand the business consequences, and can be revisited. An undocumented decision to skip an area is just a gap.
It is also not a one-time exercise. Risk changes when code changes, when usage changes, and when the business changes. An assessment written at the start of a project and never revisited describes a product that no longer exists.
Scoring Risk: Impact and Likelihood
Risk has two independent dimensions, and conflating them is the most common mistake teams make.
Impact: How Bad Is the Failure
Impact is about consequences, not probability. Ask what actually happens if this area fails in production. Useful dimensions to weigh:
- Financial - lost revenue, incorrect charges, refunds, penalties
- Legal and regulatory - breach of a compliance requirement, reportable incident, audit finding
- Safety - physical harm, where the product can cause it
- Data - loss, corruption, or unauthorised exposure of customer data
- Reputation - visible public failure, loss of customer trust
- Blast radius - how many users are affected, and whether they can work around it
- Recoverability - how quickly the failure can be detected and reversed
Recoverability is the dimension teams most often forget, and it moves scores substantially. A failure that is instantly visible and fixable with a config change is materially less severe than a silent failure that corrupts data for three weeks before anyone notices.
Likelihood: How Probable Is the Failure
Likelihood is a judgement about the code and the process, not about the business. Reliable signals:
- Change recency and volume - code modified this sprint is far more likely to break than code untouched for a year
- Complexity - branching logic, calculations, state machines, and concurrency all raise probability
- Defect history - areas that have produced bugs before produce them again; this is the single best predictor available
- Number of integrations - each external dependency is a failure mode
- Team familiarity - code written by someone who has left, or in an unfamiliar framework, carries more risk
- Existing test depth - a well-covered area has a lower chance of an undetected defect reaching production
The most useful input: Defect history. If your bug tracker can tell you which modules have produced the most defects over the last year, you have an empirical likelihood estimate rather than a guess - and it usually contradicts the team's intuition in at least one place.
Building the Risk Matrix
Score each area on both dimensions, multiply, and rank. A three-point scale is enough for most teams; five-point scales invite false precision and long arguments about whether something is a three or a four.
| Impact ↓ / Likelihood → | Low (1) | Medium (2) | High (3) |
|---|---|---|---|
| Critical (3) | 3 - Thorough | 6 - Thorough | 9 - Exhaustive |
| Moderate (2) | 2 - Basic | 4 - Standard | 6 - Thorough |
| Low (1) | 1 - None this cycle | 2 - Basic | 3 - Basic |
Two observations about how this matrix behaves in practice. High impact with low likelihood still deserves real testing - the payment flow that has not changed in months is still the payment flow, and low probability of failure does not make a catastrophic outcome acceptable. Conversely, low impact with high likelihood is where teams over-invest: an area that breaks constantly attracts attention because it is noisy, not because the failures matter.
Choosing the Unit of Assessment
Score at a level that maps to something you can actually assign testing effort to. Whole products are too coarse to be actionable; individual test cases are too fine to be maintainable. Features, user journeys, or services usually work, at a granularity of roughly fifteen to forty items - small enough to review in one session, large enough to differentiate.
Turning Scores Into Test Depth
A ranked list is not yet a plan. The step that makes risk-based testing operational is defining, in advance, what each band of the ranking actually receives. Start from the score, then apply the two impact rules that follow the table, which is where a score of 3 gets settled.
| Risk Score | Test Depth | What That Means Concretely |
|---|---|---|
| 9 (highest) | Exhaustive | Full functional pass, boundary and negative cases, exploratory session, automated regression, verified on every release |
| 6 | Thorough | Full functional pass on the happy path plus key negative cases, automated regression, exploratory time if available |
| 3-4 | Standard | Happy path plus the most likely failure cases, included in the automated regression suite |
| 2 | Basic | Happy path only, covered by smoke or a single automated check |
| 1 (lowest) | None this cycle | Explicitly not tested; documented with a rationale and reviewed next cycle |
The product alone is not quite enough, because the same score can arrive from very different places. A score of 3 can mean critical impact that rarely fails, or trivial impact that fails constantly, and those two do not deserve the same treatment. Two rules on top of the score settle it, and they are the reason the matrix cells above do not read straight off the multiplication:
- Impact floor. Anything scored critical on impact never drops below Thorough, however unlikely the failure. A catastrophic outcome does not become acceptable because it is rare.
- Impact ceiling. Anything scored low on impact never rises above Basic, however often it breaks. Noisy is not the same as important.
Agreeing the bands and these two rules before scoring matters more than it sounds. If you score first and then decide what each band gets, the definitions bend to fit the conclusion people already wanted. Fix the policy, then apply it.
Once the bands are set, the mapping becomes mechanical: each area's score, adjusted by the floor and the ceiling, selects a depth, and the depth selects which test cases run. Holding those suites as reusable selections - a critical-path set, a standard regression set, a smoke set - is what makes this repeatable per release rather than a fresh negotiation each time. Assembling a run from the right subset is straightforward when suites are organised by risk band in test run building rather than reassembled by hand.
Deciding What Not to Test
This is the part teams skip, and it is the part that produces the value. Prioritisation that only adds effort at the top without removing effort at the bottom is not prioritisation - it is a wish list.
Good Candidates to Drop
- Unchanged, low-impact areas with a clean defect history. The strongest case for skipping: nothing changed, failure would be minor, and it has never broken.
- Features with near-zero usage. If analytics show a path is used by a handful of users a month, the blast radius is genuinely small.
- Cosmetic and non-blocking behaviour. Layout imperfections that do not prevent task completion.
- Duplicate verification. Behaviour already covered adequately at a lower level does not need re-verification at a higher one purely for reassurance.
- Deprecated functionality. Areas scheduled for removal rarely warrant the same investment as the code replacing them.
What to Never Drop
Non-negotiables: Authentication and authorisation, payment and billing, anything with a regulatory requirement attached, data deletion and export paths, and whatever your product's single most-used journey is. These stay in scope regardless of how stable they appear, because the impact side of the equation does not go down when the likelihood does.
Record the Decision
Every deliberate omission needs a line stating what was not tested, why, who agreed, and when it will be reconsidered. That record is what converts a gap into a decision. It also protects the team: when a defect eventually appears in a skipped area, the conversation is about whether the risk assessment was right rather than about who forgot to test it.
Who Decides, and How to Run the Session
Risk assessment done by QA alone produces good likelihood estimates and unreliable impact estimates, because testers do not always know what a failure costs the business. Impact needs product and business input.
A workable session takes about ninety minutes with four roles present: a product owner who can speak to business consequence, a developer or architect who knows where the code is fragile, a QA lead who holds the defect history and the current coverage picture, and - where relevant - someone from support or operations, who typically knows which areas generate real customer pain.
Run it in a fixed order. List the areas first, with no scores attached. Score impact next, before likelihood, so technical fragility does not contaminate the business judgement. Score likelihood after that, using defect history rather than opinion where the data exists. Then rank, apply the depth bands, and finally review the bottom of the list explicitly - reading out what will not be tested and confirming that the people who would be accountable for a failure are comfortable with it. That last step is the one that gets dropped, and it is the one that makes the assessment real.
Where Risk-Based Testing Goes Wrong
- It becomes a document instead of a decision. A scored spreadsheet that nobody consults when planning the next release has cost effort and changed nothing.
- Nothing is ever dropped. If every area ends up in the top two bands, the assessment has not prioritised anything.
- Impact and likelihood get merged. Scoring a single "riskiness" number loses the distinction that makes the method work, and it systematically under-weights rare catastrophic failures.
- It is never revisited. Risk moves with every change. An assessment older than a quarter is describing a different product.
- Scores are set by whoever argues hardest. Without evidence - defect history, usage data, incident records - scoring becomes a negotiation, and the loudest stakeholder's area always ranks highest.
- The scale is too granular. Ten-point scales generate long debates about one-point differences that do not change the resulting test depth.
Keeping the Assessment Alive
The maintenance cost of risk-based testing is what determines whether it survives past the first quarter. Keep it low deliberately.
Re-score on a trigger rather than a calendar where possible: a new integration, a significant refactor, an incident in production, a regulatory change, or a shift in what the product is used for. Add a light quarterly review for drift that no single trigger catches.
Feed real outcomes back into the scores. When a defect escapes to production, the area it came from had its likelihood under-estimated - adjust it. When an area scored high produces nothing over several cycles despite active development, consider whether the estimate was inflated. Over a few cycles this turns a set of opinions into a calibrated model, and calibration is what makes the bottom of the list trustworthy enough to actually skip. Defect data by area, pulled from your issue tracker integration and reporting, is what makes that feedback loop possible without manual bookkeeping.
Conclusion
Risk-based testing answers one question: given that you cannot test everything, where does the effort go? It answers it by scoring impact and likelihood separately, multiplying them, and mapping the resulting ranking onto pre-agreed levels of test depth.
The discipline that makes it work is at the bottom of the list, not the top. Any team can identify its critical paths. What separates a real prioritisation from a relabelled wish list is the willingness to name the areas that will not be tested this cycle, document why, get the people who own the consequences to agree, and revisit the decision when something changes.
Keeping that structure visible is largely a tooling question. QA Sphere lets you organise test cases by risk band, assemble a test run from the bands a release warrants, and see in reporting what was actually verified against what was planned. See pricing or book a demo.
Written by
QA Sphere TeamThe QA Sphere team shares insights on software testing, quality assurance best practices, and test management strategies drawn from years of industry experience.
