Discover BDD with Gaspar Nagy and Seb Rose

Gaspar Nagy – who has been involved with SpecFlow since its inception – and Seb Rose – a core member of the open source Cucumber project – have teamed up to publish Discovery – Explore behaviour using examples. This is the first book in a series on BDD. Gaspar had the following to say about the motivation behind writing the book, and the type of content you can expect.

 

When it comes to BDD, there are generally 3 types of people:

  1. Those who do not know what BDD stands for at all (how sad!)
  2. Those who know BDD, and claim to implement it with their team, but where it turns out that their focus is only on the tools
  3. Those for whom BDD is a holistic approach

The initial idea was for a SpecFlow book that would essentially be a port of the “Cucumber for Java” book: a collection of technical tips etc. In other words, yet another technical book. While working on the core, it quickly became apparent that a SpecFlow book would never be complete without mentioning the collaborative aspects of BDD. So we ended up diving deeper into the BDD realm. At some point we made a decision: let this book be about BDD, instead of SpecFlow. In fact, let there be a BDD book series!

Why? Because in our opinion the BDD process can be divided into 3 stages:

  1. Discovery: explore and discover details through examples
  2. Formulation: convert examples to scenarios
  3. Automation: write test automation code

My painful experience is that the earlier a team adopts BDD, the better results they get. So why not give teams a helping hand (or even a compass) right at the start, in the discovery phase? We decided that this would be the essence of the first book in the series.

It took several months of intensive work with Seb to complete the 5 chapters of Discovery. At about 100 pages, it is relatively short, and said to be an easy read, so you could read it on a plane. The book is written for everyone: developers and testers, as well as product owners and test managers. Discovery – Explore behaviour using examples is also tool-agnostic: the emphasis is not on the technical process but on the collaborative aspect. The book covers the following:

Chapter 1 provides an introduction to the nature of BDD, where the important elements are explained.

Chapter 2 is an important chapter about structured conversation, providing hints on how testers and business stakeholders can be better involved in meetings, and how a shared understanding of the requirements can be achieved. A number of techniques, in particular Example Mapping (introduced by Matt Wynne), are described here through imaginative use cases.

Chapter 3 is about examples, and how there are more than just Given/When/Then test cases.

Chapter 4 covers tips on how a BDD approach can fit your project, regardless of whether you use Scrum, Kanban or any other framework.

Chapter 5 is all about how to sell BDD to business stakeholders, and make them interested in BDD. It includes tips on how to demonstrate that BDD is more than a mere technique applicable only to the tasks of developers and testers. Illustrative examples ensure that business stakeholders can relate to the advantages and share a common understanding as to why BDD is good for the entire team, saving time and effort on bug hunting. In order to reap the biggest benefits of BDD, the business side should be involved in the entire process from the start to finish.

Get your copy and let the discovery begin! You can find it on Amazon/Leanpub through http://bddbooks.com!

 

SpecFlow and PackageReference

By default, the packages used in SpecFlow projects are stored in packages.config. However the new Visual Studio projects instead use the PackageReference element to reference packages. When using the PackageReference element, generator plugins will not be found, and you will see errors similar to the following:

Unable to find plugin in the plugin search path: SpecRun. Please check http://go.specflow.org/doc-plugins for details.
#error TechTalk.SpecFlow.Generator

This is due to SpecFlow not searching for the package in the correct location. To ensure that the plugin is found, you need to specify the path explicitly in your app.config file.

To do so, enter the path to the appropriate path in the plugins element of your app.config file in this case, e.g.:

<plugins>
      <add name="SpecRun" path="%USERPROFILE%\.nuget\packages\specrun.specflow.2-3-0\1.7.0\lib\net45" />
</plugins>

IMPORTANT! You will need to manually update this path when upgrading to a newer version of the package, as the path contains the version number (e.g. “1.7.0” in the above example).

You can find a sample project using the new project format here. The relevant line in the app.config file is here.

Note: This requires SpecFlow 2.3.2 of higher. This step should not be necessary once SpecFlow 3 is released.

.NET Core Support for SpecFlow

We have been working on adding .NET Core support to SpecFlow. You can follow the current status here. If you are interested in using SpecFlow together with .NET Core, we would appreciate your input, feedback and even code contributions.

Our current estimates mean that we hope to be able to release a version of SpecFlow with support for .NET Core by Q3 of this year. The more testers and contributors we have, the quicker the process should be.

If you are interested in helping out, please visit the .NET Core Support sub-project on GitHub. Tasks tagged with “request-for-comment” are task where we are especially keen on getting feedback from the community. We would of course appreciate relevant feedback on the other tasks as well.

 

If you would like to contribute, please contact @SabotageAndi on GitHub first to prevent multiple contributors working on the same task. Not all issues on GitHub are worked out in full detail. In particular, issues with the “Request-for-comment” label or “[RFC]” in the title are not finalised. These tasks still require more discussion and evaluation. If you are able to contribute to the discussion, please do!

 

SpecFlow+ Runner and Visual Studio 2017 15.6

Visual Studio 2017 15.5 Preview 2 introduced a new real time test discovery feature. This feature is live with 15.6.

When using VS 2017 15.6 with SpecFlow+ Runner, the Test Explorer will not display any tests in VS unless you enable Additionally discover tests from built assemblies after builds under Tools | Options | Test:

We are looking into how we can support real time test discovery with SpecFlow+ Runner. Until then, please make sure you enable this option in VS if you are using SpecFlow+ Runner with Visual Studio 2017 15.6.

NuGet Packages – Reserved ID and Naming Conventions

Microsoft is introducing package identity verification for packages on nuget.org. This will allow developers to reserve particular ID prefixes used to identify. This in turn should help users identify which packages have been submitted by the owner of the ID prefix.

We have submitted a request to reserve the “SpecFlow” NuGet package prefix, which is used to identify official SpecFlow and SpecFlow+ packages. This will mean that new packages with the SpecFlow prefix can only be submitted by TechTalk, and will indicate that these packages are official.

We have also requested the SpecFlow.Contrib prefix be made publicly accessible for developers who want to release their own packages for SpecFlow. If you want to submit your own package for SpecFlow whose name begins with “SpecFlow”, you can use this prefix. This will indicate to users that the package is intended for use with SpecFlow, but is a third-party contribution.

These changes will not affect existing packages using the SpecFlow prefix that have already been submitted to nuget.org. If you are the owner of such a package, you should be able to update the package as usual. You may however want to change the name of your package to reflect the new convention.

 

In summary, here are the prefixes we have requested:

  • SpecFlow.*
  • SpecRun.*

 

You can find out more about the idea behind package IDs on the nuget.org blog.

Issue with SpecFlow+ Runner and Latest Visual Studio 2017 Version

Update: SpecFlow+ Runner 1.6.2 includes a fix for this issue. You now only need to enter a GUID if you are using the SpecFlow+ server (for adaptive test scheduling), as the GUID is used as the unique key for the test results.

 

We have discovered an issue that affects SpecFlow+ Runner with Visual Studio 2017. When adding the NuGet package to your project, a default profile is added to your project (default.srprofile). During the installation process, the projectName and projectId should be entered automatically in this profile.

When installing the packages with the latest version of Visual Studio 2017, the projectName and projectId are no longer filled in automatically. This results in your tests not being discovered, meaning they cannot be executed.

As a workaround, you can enter the projectName and projectId in your profile yourself. The projectName should be the name of the project you added the NuGet package to; the projectId should be the GUID of the project (this is the <ProjectGuid> in your .csproj, although in theory you can enter any valid GUID here).

An issue has already been opened, and you can track the status here.

Conflict with SpecFlow+ Runner and NUnit 3.7.1

Note: This issue has been fixed with NUnit 3.8.

NUnit 3.7.1 introduced changes that break NUnit tests executed with SpecFlow+ Runner when using NUnit’s Assert function. This results in an error similar to the following:

Object reference not set to an instance of an object.System.NullReferenceException: Object reference not set to an instance of an object. at NUnit.Framework.Assert.That[TActual](TActual actual, IResolveConstraint expression, String message, Object[] args)

This is a known issue. Until we are able to fix this, please stick to NUnit 3.6 if you are using SpecFlow+ Runner to execute your NUnit tests.

New SpecFlow+ Renewal Policy

A number of customers have contacted us in the past asking for option to renew their support period for SpecFlow+ so they can upgrade to a newer version. Until now, the only official option in this case was to purchase a new license at full cost.

This seems unfair on those users who want to keep up-to-date with the latest release and extend their support period each year. We have therefore decided to offer 1-year extensions to the SpecFlow+ support period for 50% of the cost of a new license. Extensions will be applied from the last day of the previous support period. For example, if your support period ends on 1 September 2017, the support period will be extended to 1 September 2018 (irrespective of the date on which you renew).

We are also giving all previous SpecFlow+ customers the option of renewing their SpecFlow+ licenses at the same price point, even if the upgrade period has already ended. In this case, the 1-year extension will begin from the date the renewal is purchased (instead of from the end of your last support period). This offer is only valid until the end of August 2017. If you are an existing SpecFlow+ customer and want to renew your license and support period, please visit this page and provide us with as much information on your licenses as possible. We will verify that you are a past customer and send you a payment link. If you have any questions, you can also contact us directly at support [at] specflow.org.

Podcast with SpecFlow creator Gaspar Nagy

SpecFlow creator Gaspar Nagy recently joined Aslak Hellesoy, Matt Wynne and Arti Mathanda for March’s Cucumber Podcast.

In the podcast, Gaspar talks about the origins of SpecFlow and how working on SpecFlow changed his life. He also sheds some light on the future of SpecFlow and his own plans, including teasing us with the prospect of his own book on the subject. As well as talking about SpecFlow and Cucumber, the panel also discuss some general topics, such as the role of QA in modern development teams. And last but not least, you will also learn how to pronounce Gaspar’s name like a true Hungarian.

New SpecFlow+ Sample Projects Available

We’ve added a number of new samples to the SpecFlow+ sample repository on GitHub:

  • WindowsAppDriver: Demonstrates using the WindowsAppDriver with SpecFlow+ and the Windows calculator.
  • CustomDeploymentSteps: Sample project based on the previous WindowsAppDriver project that demonstrates custom deployment steps.
  • ExcelExample: A relatively simple calculator project using SpecFlow+ Excel. Different operations (add/subtract/divide/multiply) are specified using a either a standard feature file, an Excel file, or a combination of a standard feature file and example data in an Excel file.
  • FilterExample: A sample project demonstrating how to use filters to only run a subset of tests.
  • MultipleReports: A sample project showing how to configure SpecFlow+ Runner to output multiple reports using custom templates.
  • TestThreadIsolation: A sample project showing the effects of different test thread isolation configurations for multithreaded testing