Handle Date Contraints in your Scenarios

Last time, you voted for different ways of describing a feature that depends on a period of time. This article shows the results of the survey, an analysis of the votes, and some general ideas on how to solve similar problems.

And the winner is…

The most popular choice was implying the current date, and just referencing “more than a month ago” in the scenario. Following that, with the equal number of votes, are two options. One is using a special placeholder (TODAY) in the scenario and then referencing “TODAY – 1 month”. The second is explicitly setting the system timestamp before important steps.

The key reasons for using implied dates were that the scenarios were more customisable for business users, and that they are closer to human-readable language.

The key reasons for explicitly setting timestamps were being able to explore and document edge cases around leap years and seconds.

There were two suggestions in the “Other” category. The first was to tweak the criteria as “Given a transaction is authorised on TODAY – 1 month OR BEFORE”, making it clearer that we don’t want to match on the exact millisecond and ignore everything that happened before. Another suggestion in the “other” group involved the use of pre-processors to set explicit dates in the scenario and hide them from the business users, potentially applying another preprocessor formula to calculate the relative period start.

I kind of totally disagree with the community votes on this, read on to learn why. Before jumping to my conclusions, I’d also encourage you to read Seb Rose’s take on the same problem, in which he very thoughtfully explains an alternative position. Please note that the original version of this article contained a few typos in the list of examples, and Seb nicely spotted that. Some of the changes he proposed in his response are now integrated into the updated version that you are reading now.

For another outside perspective, I also recommend reading Dan North’s response, titled ‘The mystery of the missing date’. Dan identifies three different scenarios to analyse separately, and ends up with fewer examples than in my scenario outline below.

Acceptance criteria vs Examples

Many people like specifying examples by repeating words or phrases that business representatives used when describing a feature. You can see that in the votes for this challenge, but also in the votes for many previous challenges. The acceptance criteria for the whole feature in this challenge was to “cancel an authorisation if it has not been charged for a month”, so it sounds logical to use phrases such as Given a transaction was authorised more than a month ago in the examples. Using the business terminology in scenarios is great. It’s helping to form the ubiquitous language for the domain. But keeping ambiguous phrases just because they seem close to acceptance criteria is not necessarily a good idea.

I’m not too strict about the terminology around “acceptance criteria”, since none of the tools I use have that specific entity. Some companies need to formalise a process, and for them, figuring out how “acceptance criteria” relates to Given-When-Then is quite a challenge. “Acceptance criteria” tend to be short sentences that capture the needs or the expectations of business representatives, such as summaries of business rules. They usually need to be concise so they are easy to grasp, but they may lack details – especially regarding edge cases. Examples need to be precise and remove ambiguity. If a single sentence can perform both roles, then absolutely use it. But if there’s a conflict between the roles, it’s best to use different types of information.

With the warning that this may be overly generic, a good starting point is to think about acceptance criteria as bullet points in a work task description. During a collaborative analysis session, we can figure out what those rules mean exactly in different situations, and capture that with examples. Acceptance criteria are the input into the BDD conversation, examples are the output.

There are several hierarchical levels in Given-When-Then formats. Features contain scenarios, scenarios contain examples. Several popular tools also support the Rule keyword that can sit between the features and scenarios. This lets us capture sentences that business representatives want to see in many ways, and augment them with details on a lower level. Don’t get stuck on an ambiguous phrase just because it sounds like a business requirement.

Avoid mathematical formulas in examples

One typical cause of problems are mathematical formulas. Calculations (“today – 1 month”) and relations (“a month ago”) are great to summarise a business rule and to start a discussion, but they are usually bad for examples. That’s because they can be misleadingly simple, and hide important questions.

With relative periods, such as moving a month ahead or before, calculations sometimes get tricky. What date exactly is “TODAY – 1 month” on 30th March? How about on 31 March? Did you answer the same for both questions? If so, should you be concerned that a “TODAY – 1 month” on 10AM on 30th March is after the same formula for 9AM on 31st March?

Relations like “a month ago” might look obvious, but they are far from it. With daylight saving time corrections, time might seem as if it’s flowing in reverse unless we are careful. For example, clocks in Europe turned one hour back at 1AM on October 25th 2020. Try calculating the time for “a month ago” at (the first instance of) 00:59 on October 25th 2020, and for a minute later. Does the result make sense?

How accurate does it need to be?

A nice starting point to resolve such issues is to think about how accurate the result needs to be. With relative periods, different scenarios might require different levels of accuracy. Here are three examples:

• mobile phone billing: a second
• security token expiration: a minute
• tax reports: a business day
• old transaction archival: a week

For the sake of the discussion, let’s say that releasing authorised funds that were not charged for a month should be accurate to a business day. Releasing funds one hour earlier or later isn’t really going to be noticed by anyone. This at least lets us ignore all the problems with daylight saving time shifts and leap hours or seconds. We still need to look at tricky dates.

For example, what is one month after 31st March? (April doesn’t have 31 days). If it’s not possible to calculate it just by adding a calendar month, is it better to make the period shorter or longer? For releasing money reserved on client accounts, it might be better to release it early. In some other situation, especially if clients are paying daily fees, it might be much more profitable to extend “one month later” than to shorten it. These are important business model questions that need to be answered before delivery, and hiding them behind a simplistic mathematical formula is dangerous.

Choosing the right format

Although it got the least amount of votes, the “hard-coded dates” option looks best to me regarding the level of information. It allows us to explore and demonstrate all the problematic edge cases with period calculations, and implies the relative precision.

Hiding the calculation behind a formula, especially if the test automation layer also needs to apply the formula in some way, means that we could potentially have two different versions of a calculation in the test framework and in the system under test. This could lead to silly discrepancies. Or even worse, it could lead to hidden bugs, since both systems might have a mistaken implementation.

A single scenario, though, is not enough. I would love to see this expanded as a scenario outline, with the key edge cases clearly documented. Something similar to the scenario below:

Scenario Outline: cancelling uncharged authorisations

   To avoid locking up client funds indefinitely, we need to cancel an authorisation 
   if it has not been charged for a month. The accuracy level of 1 day is enough. When
   calculating the period, it's better to make it shorter than a calendar month if 
   the matching date does not exist.

   Given a transaction is authorised on 
   And the transaction was not charged
   When the cancellation job runs for 
   Then the authorisation should be 

   Examples: matching date exists

    If the next calendar day exists the following month, cancel 
    on that date.

   | auth date         |  processing date | status     |
   | 8 October 2020    |  8 November 2020 | authorised |
   | 8 October 2020    |  9 November 2020 | cancelled  |

   Examples: matching date does not exist

    If the next calendar day does not exist the following month 
    (usually because it's shorter), cancel on last day of month 

   | auth date         |  processing date  | status     |
   | 29 March 2020     |  29 April 2020    | authorised |
   | 29 March 2020     |  30 April 2020    | cancelled  |
   | 30 March 2020     |  29 April 2020    | authorised |
   | 30 March 2020     |  30 April 2020    | cancelled  |
   | 31 March 2020     |  29 April 2020    | authorised |
   | 31 March 2020     |  30 April 2020    | cancelled  |

   Examples: February (non leap year)

    February is a tricky case always because it is much
    shorter than the previous month, so we need to 
    check a wider range of end-of-month dates.

   | auth date         |  processing date  | status     |
   | 26 January 2019   |  26 February 2019 | authorised |
   | 26 January 2019   |  27 February 2019 | cancelled  |
   | 27 January 2019   |  27 February 2019 | authorised |
   | 27 January 2019   |  28 February 2019 | cancelled  |
   | 28 January 2019   |  27 February 2019 | authorised |
   | 28 January 2019   |  28 February 2019 | cancelled  |
   | 29 January 2019   |  27 February 2019 | authorised |
   | 29 January 2019   |  28 February 2019 | cancelled  |
   | 30 January 2019   |  27 February 2019 | authorised |
   | 30 January 2019   |  28 February 2019 | cancelled  |

   Examples: February (leap year)

    Leap years are often an edge case we want to test for
    specifically, because February is one day longer.
    2020 was a leap year.

   | auth date         |  processing date  | status     |
   | 27 January 2020   |  27 February 2020 | authorised |
   | 27 January 2020   |  28 February 2020 | cancelled  |
   | 28 January 2020   |  28 February 2020 | authorised |
   | 28 January 2020   |  29 February 2020 | cancelled  |
   | 29 January 2020   |  28 February 2020 | authorised |
   | 29 January 2020   |  29 February 2020 | cancelled  |
   | 30 January 2020   |  28 February 2020 | authorised |
   | 30 January 2020   |  29 February 2020 | cancelled  |

The key problem magic values (such as TODAY) try to solve is reliable test automation, especially when some piece of code depends on the system clock. However, the difficulty of capturing such examples should perhaps point to a modelling problem. If some piece of code (such as the transaction cancellation) depends on a period of time, it’s better to model that into the domain by providing the “processing date”, and avoid using the system clock directly in the model.