Turn your SpecFlow Examples into Automated Acceptance Tests
For me, one of the most important characteristics of a good test automation tool is that it does one thing very well. SpecFlow is no exception to this: it is very good at transforming human-readable specifications in the Gherkin Given/When/Then format into executable code, often also referred to as executable specifications.
What SpecFlow does not do, however, is providing you with tools, libraries or methods to actually interact with an application under test, once you’ve decided to turn your executable specifications into actual automated acceptance tests.
Therefore, to create automated acceptance tests using SpecFlow, you’ll have to use other tools together with SpecFlow. In this article, we’ll take a look at some of the tools that are frequently combined with SpecFlow to create different kinds of acceptance tests.
Before we dive into these tools, here’s a diagram that shows how they are connected to SpecFlow in a typical acceptance test solution:
Unit testing frameworks
The role of unit testing frameworks in a SpecFlow-based acceptance test solution is twofold:
- It offers the ability to create assertions in your tests. These assertions compare the actual outcome of an action in your test with an expected outcome expressed in code (also known as a codified oracle). These actions can range from individual method calls, to calls to a REST API or database, to actions performed on a graphical user interface.
- Unless you’re using the SpecFlow+ Runner, you’ll need a test runner that can take care of discovering and running your scenarios. SpecFlow is compatible with the most popular C# unit testing frameworks, such as NUnit, MSTest and xUnit.NET.
API testing libraries
If your scenarios involve interacting with web APIs, you’ll need to add a library that can do that for you. If you want full control, you can use the System.Net.Http library that comes with .NET. Alternatively, you can use third-party libraries. These typically provide an abstraction of System.Net.Http, removing boilerplate code and making it easier to read and write. Two popular, open source web API libraries for C# are RestSharp and Flurl.
User interface libraries
For your user interface-driven automation needs, by far the most popular libraries to combine with SpecFlow are:
- Selenium WebDriver for desktop browser automation, and
- Appium for native, hybrid and mobile web app automation
Both these libraries are open source and are supported by a large community, meaning there is almost always someone who has run into the problem you’re facing before and from whom you can learn how to solve it. Most of the acceptance testing solutions including SpecFlow that I’ve seen and worked with also had Selenium under the hood to drive at least part of the SpecFlow scenarios.
Database testing library
For storing data in and retrieving it from a database in your SpecFlow scenarios, you’ll need a dedicated library, or database engine, as they are typically referred to in the .NET space. Which one exactly you need depends on the type of database engine you’re using. Most popular database platforms, such as SQL Server, Oracle, MySQL, PostgreSQL and SQLite can be accessed from C# code, so it’s very likely you’ll be able to add database-based steps to your SpecFlow scenarios, too.
Assertion libraries
While SpecFlow does a great job of creating executable specifications that are readable for all stakeholders, the code that is executed when these specifications are verified is just as important. A proven way of improving the readability of your test code is by using dedicated assertion libraries, such as FluentAssertions or Shouldly. These libraries are very useful in making the most important part of your test code (the assertions) easy to read and understand, even for those that do not write C# code for a living.
Others
The types of libraries mentioned so far in this article are the ones that are most commonly used together with SpecFlow. However, that does not mean you’re options are limited to those. As long as your library is a C# library, too, or it offers a command line interface or other type of API that you can invoke from your C# codebase, you can combine it with SpecFlow. Executing a performance test to verify performance behaviour expressed in your scenarios? You could run Gatling or JMeter scripts using their command line interfaces and parse the results as part of your SpecFlow scenarios. Want to do a visual comparison? Applitools has powerful features and can be used to augment existing tests. The possibilities are nearly endless!
A final bit of advice
As you have seen in this article, you can use SpecFlow for a wide variety of test types and combine it with many different tools. I’d recommend you, though, to keep any implementation details about the behaviour of your application under test, and by association, about the tool you’re using to verify the behaviour of your application, out of your scenarios. Or, put differently, only describe ‘what’ your application does, not ‘how’ that behaviour is implemented. For example, instead of using steps in your scenarios like
'When Joe clicks on the login button'or
'Then the API returns an HTTP 200'keep your scenarios high-level, for example:
Given Susan is an existing customer
And she has an active bank account with a balance of 1000 dollars
When she withdraws 250 dollars
Then the withdrawal can be performed
And the remaining balance is 750 dollarsSpecifying your scenarios like this both ensures that they are crisp, clear, and readable for all those involved, and it makes it much easier for those responsible for automating the scenarios to pick the right tool for the job – even if you search for a Klimaanlage in Graz or a Sofa Konfigurator. The scenario above could be executed through the user interface (Selenium or Appium), using a web API (RestSharp or Flurl) or even directly on a database. And when the existing implementation needs to change (for example because at some point in time an API becomes available where previously everything needed to be done through the user interface), you don’t need to update the scenarios at all.
