How to deal with pauses and timeouts? #GivenWhenThenWithStyle

The challenge for this week is dealing with wait statements and pauses in specifications.

My testers keep adding pauses into Given-When-Then. They claim it’s necessary because of how the tests work. I don’t like it because the numbers look random and do not reflect our business rules. Also, sometimes pauses aren’t long enough. The tests sometimes fail although there are no bugs in the system version we’re testing, because the pauses aren’t long enough. How do we avoid that?

This problem is symptomatic of working with an asynchronous process, often an external system or an executable outside of your immediate control. For example, think of an e-commerce platform that passes orders through a risk evaluation module before confirming. The risk check might take a few seconds, and the user may not be able to pay until the order is confirmed. In a visual user interface, this waiting period is often shown with a spinner or a progress bar. A bad, but common way of capturing this in Given-When-Then is to add a pause between actions:

Given a user checks out with a shopping cart worth $200
When the user submits an order 
And the system waits 3 seconds
And the user authorises the payment
Then ...

In some cases, it’s not the system under test requiring pauses, but the way a test is automated. A common example is executing tests through a web browser automation tool such as Selenium or Puppeteer. The framework can load a web page using a browser, but that page might need to fetch external javascript files or additional content to fully initialise the user interface. Triggering an action immediately after a web page loads might cause the test to fail, because the related elements are not yet available. Adding pauses to Given-When-Then scenarios is also a common work-around for these kinds of issues:

Given a user registers successfully
When the account page reloads
And the user waits 2 seconds
Then the account page displays "Account approved"

In both these situations the testing process requires a bit of time between an action and observing its result. But the actual period will vary from test to test, even between two executions of the same test. It will depend on network latency, CPU load and the amount of data required to process. That is why it’s not possible to correctly select the period duration upfront. Choosing a short pause causes tests to occasionally fail because the asynchronous process sometimes does not complete when the test framework moves on to the next step, so people have to waste time chasing ghost problems. Setting a pause that’s long delays feedback and slows down testing excessively.

How would you rephrase and restructure this instead? Post your suggestions using the link below. If you’d like to send a longer comment, perhaps write a blog post and then just send us the URL (you can do that using the same form, link below).

Stay up to date with all the tips and tricks and follow SpecFlow on Twitter or LinkedIn.

PS: … and don’t forget to share the challenge with your friends and team members by clicking on one of the social icons below 👇