Fit-for-purpose Gherkin

A common mistake I see is the tendency to write Gherkin specification as if they were a series of instructions describing the algorithm for how the test is executed: “enter X” or “open page Y”. This is understandable; after all, it’s how you might write a standard test script. It’s very easy to continue using the same approach with Gherkin.

But the power of Gherkin comes from its abstraction – the ability to formulate steps in plain English (or any of the other supported languages), rather than as an algorithm. Instead of describing how something is done, Gherkin should describe what is being done without worrying about the implementation details.

This is a powerful tool, separating the algorithmic steps (bindings) from the actual description of the test intended to be read by humans. The primary goal of Gherkin is to formulate your scenarios so that they are quick to read, clearly formulated and understandable to all stakeholders – including non-technical team members.

OK, but what does that mean in real practical terms? Let’s take a look at a very basic example where we have a web application with various user roles, including administrators. Any test interfacing with the UI that involves the administrator dashboard will require the following initial steps in order to execute the test:

  1. Open the login page by navigating to the appropriate URL.
  2. Enter the user name.
  3. Enter the password.
  4. Click on the Log In button
  5. Select Administrator Dashboard from the menu.

These steps are important– you will definitely need to log in to the system somehow and navigate to the appropriate area to perform to execute the test. But let’s take a step back from the nitty gritty details, and ask ourselves what our precondition is: we need to be logged in as administrator, and we need to be viewing the administrator dashboard.

So, as tempting as it is to translate each of the steps listed above into individual steps in Gherkin, it is much more advisable to write the following:

Given I am logged in with administrator privileges

And I am viewing the administrator dashboard

These statement clearly capture the current state of the system required in order to perform the test. It goes without saying that being logged in as administrator involves logging in, but how this is done is not where the focus of your discussion should be. These steps are implicit implementation details that add nothing to the discussion about what can be done on the administrator dashboard itself. Furthermore, you may interface directly with your API and not even necessarily perform these steps using the UI.

The only place you need to worry about the implementation itself – how you log in to the system – is in the binding code for the Given step. There is no reason to explicitly state these steps in Gherkin; in fact, doing so runs the risk of making the feature file bloated, and thus harder to understand and navigate. As a side effect, you will also end up with a lot of unnecessary bindings for the unnecessary steps that could easily be grouped together.

Now if you have been critically evaluating the previous Gherkin step, you might even argue that “Given I am viewing the administrator dashboard” already implies you are logged in as administrator. And to a certain extent I would agree with you. In this case, my preferred approach would be to move the first step (Given I am logged in with administrator privileges) to the scenario background – assuming the feature file I am working on covers a range of administrator functions, all of which require the user to be logged in as an admin. Using background statements is another great way to reduce the clutter in your Gherkin files.

Another thing to remember is that there are always implicit steps; the question is where to draw the boundary. It is probably unnecessary to specify “Given I am logged in” as the prerequisite for every test, just like it is unnecessary to add the step “Given I have started the application”. Of course, you will want to specify the login state for certain, specific tests: testing the login process presumably requires the user to not already be logged in, so this is a step that should be specified. Similarly, access to the administrator dashboard is limited to a certain user type – being logged in as an administrator is therefore an important criteria. In this case, it also includes the information that other user types cannot perform these actions.

So let’s come back to the actual implementation of the steps. We have reduced a number of explicit steps to a single Gherkin statement, but we still need to execute all of those steps. That’s easily handled by writing code for each of the necessary steps that is called from the binding:

public void GivenIAmLoggedInWithAdminPrivileges()
{
    NavigateTo(URL);
    EnterUserName(“Admin”);
    EnterPassword(“P@ssw0rd”);
    ClickLogInButton(); 
}

As mentioned above, this might be abstracted to simply interface with your API, rather than actually performing these steps via the UI. Of course, just like you can reduce multiple explicit steps to a single Gherkin step, you can also group different sub-functions within a parent function, which I would probably do here:

public void GivenIAmLoggedInWithAdminPrivileges()
{
     Login(“Admin”,“P@ssw0rd”);
}

So I have one single function to handle the login process, which can obviously be reused elsewhere. Furthermore, any changes to the login process only require a single change to the test automation code and do not require any changes to the Gherkin file.

This makes it easier to maintain your tests and reuse steps that represent logical units, rather than having multiple tests share a large number of mini-steps that are brittle. Changes to the login process or your API do not require the test to be rewritten; they only affect the bindings of the tests. Furthermore, the step Given I am logged in as with administrator privileges can be used by any test involving an administrator account in a single binding method.

Of course, these concepts are not new. You will at some point have taken a function that has grown over time, and split it up into smaller sub-functions. You should adopt a similar approach to Gherkin: it may help to try and think more in terms of general functions, rather than individual statements. This may require a paradigm shift on your part: instead of being as explicit as possible in your feature files, reduce the steps to the essentials without obscuring any key information. The less bloated and more concise your tests/specifications are, the better suited they are to encouraging meaningful discussion between stakeholders, and the more understandable they are to non-technical team members. At the end of the day, that is probably the biggest benefit to this approach – it enables everyone to understand what is (supposed to be) going on and why.

As with any code (yes, feature files are code files too), a certain amount of refactoring and maintenance will always be required as your project grows. However, ensuring that your Gherkin files are fit-for-purpose from the start will mean that this refactoring will be a lot easier and less frequent. It will also have far less of an impact on your binding code if you reduce the frequency with which you need to update your Gherkin steps.

Sending Execution Reports via Email

We’ve had a number of users ask how to send execution reports via email after the test run has been executed. SpecFlow+ Runner 1.6 (a pre-release version is available on NuGet) introduces the option to give reports static names, i.e. without a time stamp, meaning sending the report as an attachment is now relatively easy to automate.

To help you out, we’ve outlined the necessary steps in the documentation here. The example uses mailsend, but you can obviously use other command line tools (or use telnet if you are really hardcore).

The same principles of course apply to any file you might want to send, and sending files via email is obviously not restricted to environments using SpecFlow+ Runner.