What is ‘Consistency’?
Maintaining consistency when writing a step or test is important for your results. In a broad sense, consistency is recognized in an outcome that can be reliably and repeatedly reproduced. At Rainforest we believe that consistency can be understood at the level of the individual step and at the level of the test.
- Consistency within a step
Test steps should be written such that completing that action necessarily allows the tester to affirmatively ‘pass’ the step. For this to be the case, a step’s question must be consistent with the outcome the step instructions were intended to reproduce.
For example, asking a tester whether they successfully logged into a dashboard is consistent with instructions telling a tester to enter provided set of credentials in designated fields and clicking the ‘LOG IN’ button below the fields.
When steps are written with consistency, it not only becomes easy for testers to understand the tasks laid out for them, but it also allows the exact actions the testers performed in the step to be pinpointed and compared those actions that were specified in the test instructions.
- Consistency within a test
In the same way that individual steps should have consistency worked into them, tests should also. An individual regression test should be written whereby going through the workflow covered by the regression should produce an expected outcome every time the test is run. Each step, in turn, must be consistent with the preceding and proceeding step to produce the expected outcome.
Expanding on the example of the login flow outlined above, imagine that the outcome of this particular regression was to see if a profile picture could be uploaded successfully. To achieve the desired outcome of the test, the following steps should include instructions that direct the tester to:
- navigate to the appropriate location in the user portal
- download and save an image file
- upload the downloaded image file as the profile picture.
When a test written with consistency, testers will be able to successfully progress from one step to another (unless they encounter a bug!). In returned it is assured that the passage of this test means that the workflow specified can be completed as intended.
Ways to maintain consistency
- Maintaining a consistent use of terms:
Our testers are trained to follow a step as it was authored. To ensure that they do this consistently, use the same term to describe an element throughout the step and, if the same element appears multiple times, throughout the test. For example, if a test is meant to cover a workflow that begins with a sign-in process, testers shouldn’t be instructed to “log-in”.
- Have an end-goal in mind when writing a test:
As it was mentioned above, consistency is all about being able to reproduce an expected result reliably time and time again. To get to this level of confidence in a test more quickly, tests should cover one workflow and one workflow only. It’s fine when a tester must accomplish several subgoals in order to complete this end-goal, but do not try to have one test assess multiple workflows.
- Limit Ambiguity and Vagueness
When ambiguity and vagueness are limited within a test, number of instances where a tester could interpret instructions incorrectly, and thereby minimize their need to interpret or decode your instructions is minimized. Instructions that clearly direct a tester to perform a specific workflow is more likely to allow for consistent and reliable results.