What is a Regression Test?
Every Rainforest test is itself a regression test whether it's one or twenty steps long. But for the purposes of this article, a regression test is a longer and more complex test that is tailored to particular and specific action-path that an end-user could take while interacting with a website or application. Ideally, a suite of 15 to 20 regression tests should provide a sufficient substantial level of coverage.
The Regression Test's Defining Characteristics:
- Granular - At the step level, each step of a regression test is meant to describe a single action or small group of actions that a potential user could take during their interaction with the website or feature.
- Detailed - As regression tests are more complex than smoke tests, writing in an appropriate amount of detail into the steps can improve a tester's overall sense of how the test is supposed to function.
- Specific - A regression test is meant to provide coverage over a specific action-path. It has a defined starting point and guides a test toward definitive end-goal the tester is meant to reach.
When should I be running my regression tests?
Your regression tests should be run frequently throughout your developments and before any major releases. These tests are meant to provide coverage for any part of your app that can experience issues.
Why are regression tests important?
The goal of a regression test is that over time, it will become a reliable indicator of stability and consistency. The consistent passage of a regression test is meant to indicate an action-path is functioning as designed.
On the flip side, like with smoke tests, the failure of a regression test should sound an alarm that something is preventing a user from walking down an action-path as expected. Failures could indicate a legitimate bug with the feature, but could also indicate that the test, was unclear about what they were actually supposed to do in their interaction. Either way, regression tests become a valuable pipeline of feedback that can be distilled into future improvements.