Specification by Example works well for deterministic processes, where you can know an expected result upfront. The next challenge deals with a situation when there is no clear expected result – generating random numbers.
We have a chat-bot that needs to simulate human typing. Instead of sending the entire message at once, we want it to insert short pauses between each letter, say between 0.2 and 0.5 seconds. And these pauses should vary randomly, so the human on the other end would not suspect it’s chatting with a robot. How can we describe this with Gherkin?
Option 1: Specify overall process properties
One option would be to directly translate business expectations into a single statement, and hide the details of the test in the automation layer.
Given the bot receives a message "Hi there" When the bot replies with "Hello" Then the bot should insert random pauses from 0.2 to 0.5 seconds between each letter
Option 2: specify randomness in individual actions explicitly
Alternatively, we could expose more information in the scenario, and leave individual actions to prove for the automation layer.
Given the bot receives a message "Hi there" When the bot replies with "Hello" Then the pause between typing H and e should be random between 0.2 and 0.5 seconds And the pause between typing e and l should be random between 0.2 and 0.5 seconds And the pause between typing l and l should be random between 0.2 and 0.5 seconds And the pause between typing l and o should be random between 0.2 and 0.5 seconds
Option 3: Test properties of a larger set
Although there’s no clear deterministic value for each individual action, non-deterministic processes may have some deterministic properties when applied to a large group. We could try specifying that instead of individual instances:
Given the bot receives 10000 messages When the bot replies to all messages Then the minimum delay between typing should be greater than 0.2 seconds And the maximum delay between typing should be less than 0.5 seconds And there should be at least 1000 unique delay values
Option 4: Fake the randomness for automation
Alternatively, we could make the process deterministic by providing a “fake” random number generator for testing purposes. In production, the system would use an actual random number generator. This would not prove end-to-end integration, but it would prove that we’re transforming the random number output (usually between 0 and 1) to the interval required by the scenario (0.2s to 0.5s).
Given the random number generator produces the following sequence: 0.9, 0.4, 0.1, 0, 0.6 When the bot replies with "Hello" Then the folling pauses should be be inserted after: | H | 0.47s | | e | 0.32s | | l | 0.23s | | l | 0.20s | | o | 0.38s |
Participate in the challenge
Choose one of the options you like the best, or propose your own solution – there are plenty of other ways to achieve the same thing. If you have an alternative proposal, perhaps publish a quick blog post and send us the link.
To participate, cast your vote using the link below before 25th January. We’ll publish the results on the 26th.
PS: … and don’t forget to share the challenge with your friends and team members by clicking on one of the social icons below 👇