Testing methods
Smoke testing, regression testing and usability testing answer different questions
UxerProof Editorial · · 3 min read
“The tests passed” is incomplete information. A quick check that a page loads, a comparison against previous behavior and a study of someone's ability to complete a task can all produce passing results. They do not support the same release claims.
The useful question is what each test was designed to establish. Start there, then choose a combination that fits the risk of the change.
Name the question before the method
| Method | Main question | Example |
|---|---|---|
| Smoke testing | Is the deployed system available enough for further checking? | The public page and sign-in screen load. |
| Regression testing | Does previously expected behavior still hold after a change? | A valid draft still saves and can be reopened. |
| Usability testing | Can the intended participant complete the task, and where do difficulties appear? | A new user tries to find and edit a saved draft. |
These are working distinctions, not rigid tool categories. A team can automate parts of a smoke suite and parts of a regression suite. A usability investigation may combine observation, task outcomes and follow-up questions.
Apply all three to one change
Imagine an illustrative release that reorganizes a reporting menu. A smoke check confirms the application opens. Regression checks confirm saved reports still exist and export correctly. A usability task investigates whether someone can locate the report they need using the new navigation.
Passing the first check does not settle the next two. The application can be available while the menu hides an important action. Conversely, a participant can find the correct action while the save operation fails.
Keep the results separately named. This helps a reviewer see whether a concern is availability, functional behavior, navigation or a combination.
Keep automated and human evidence distinct
A synthetic participant can attempt a bounded browser task and produce observations about its path. Those observations can expose reproducible obstacles and generate research questions. They should not be presented as interviews with customers or evidence of how a population feels.
For questions about people's understanding, recruit appropriate participants and investigate their experience. GOV.UK describes moderated usability testing as watching participants attempt tasks, with opportunities to explore what they are doing and thinking. GOV.UK usability-testing guidance.
Report coverage rather than a universal pass
A release note might say: public-route smoke checks passed; draft-saving regression checks passed; the new navigation was assessed with one specified task; administrator workflows remain unassessed.
That wording is less dramatic than “fully tested,” but it gives the next person something actionable. They can decide whether the uncovered area matters before release and commission the appropriate check.
Choose the smallest collection of tests that answers the release questions. Expand it where risk or evidence gaps justify the effort, not because a larger test count looks more convincing.
Next step: Read AI usability testing: evidence and limits before interpreting synthetic journey results.
Put the evidence structure to work
Inspect UxerProof's fictional sample report, or use the free readiness check to prepare your next journey. External Circuit execution is not yet activated; see current launch status for availability.