The challenge for this week is dealing with wait statements and pauses in specifications.
My testers keep adding pauses into Given-When-Then. They claim it’s necessary because of how the tests work. I don’t like it because the numbers look random and do not reflect our business rules. Also, sometimes pauses aren’t long enough. The tests sometimes fail although there are no bugs in the system version we’re testing, because the pauses aren’t long enough. How do we avoid that?
This problem is symptomatic of working with an asynchronous process, often an external system or an executable outside of your immediate control. For example, think of an e-commerce platform that passes orders through a risk evaluation module before confirming. The risk check might take a few seconds, and the user may not be able to pay until the order is confirmed. In a visual user interface, this waiting period is often shown with a spinner or a progress bar. A bad, but common way of capturing this in Given-When-Then is to add a pause between actions:
Given a user checks out with a shopping cart worth $200 When the user submits an order And the system waits 3 seconds And the user authorises the payment Then ...
Given a user registers successfully When the account page reloads And the user waits 2 seconds Then the account page displays "Account approved"
In both these situations the testing process requires a bit of time between an action and observing its result. But the actual period will vary from test to test, even between two executions of the same test. It will depend on network latency, CPU load and the amount of data required to process. That is why it’s not possible to correctly select the period duration upfront. Choosing a short pause causes tests to occasionally fail because the asynchronous process sometimes does not complete when the test framework moves on to the next step, so people have to waste time chasing ghost problems. Setting a pause that’s long delays feedback and slows down testing excessively.
How would you rephrase and restructure this instead? Post your suggestions using the link below. If you’d like to send a longer comment, perhaps write a blog post and then just send us the URL (you can do that using the same form, link below).
Read-on for a selection of good ideas we’ve received and our suggestions on how to handle scenarios such as this one.
PS: … and don’t forget to share the challenge with your friends and team members by clicking on one of the social icons below 👇