Xray Support for SpecFlow in JIRA

We are pleased to announce that Xray, a test management solution for JIRA, now supports SpecFlow+ Runner. Many software development teams work with JIRA, but integrating SpecFlow with JIRA can be challenging. Xray helps out by allowing you to manage tests in JIRA, generate the corresponding feature files, and import your test results so they are visible in JIRA itself.

When using Xray, you can choose to edit and manage your scenarios in Jira, or continue working with feature files within your source repository. Both these methods are described here.

If you choose to manage features in JIRA, you can generate the necessary .feature files using the Export to Cucumber option in the user interface:

 

If you choose to manage your features files as part of your repository, there are a few steps you should take, including tagging tests with an ID to ensure they are always mapped to the same test in Xray. These steps are covered towards the end of the page.

In order to generate test results that can be viewed in JIRA, you will need to output your test results as a JSON report using SpecFlow+ Runnner. A suitable template is included with SpecFlow+ Runner (located under Templates in your project’s SpecRun.Runner packages folder). This report can then be imported into JIRA to view the tests’ status there.

If you are interested in finding out more about Xray, there is a short video overview available here. You can find more detailed information, including pricing, on the Atlassian marketplace. There are also regular webinars that walk you through the Xray’s features.

End of Visual Studio 2013 Support Imminent

We will remove support for Visual Studio 2013 with the upcoming release of SpecFlow 3 (due to be released soon). Downloads for the Visual Studio extension account for only a fraction of 1% of all downloads (a ratio of roughly 400:1), so we want to invest our time and resources elsewhere. For this reason, we will no longer support Visual Studio 2013 with SpecFlow 3, meaning there will be no more updates of the VS 2013 integration.

If you are still using Visual Studio 2013 and will want to upgrade to SpecFlow 3, we suggest that you start mapping out your upgrade path so you are prepared for the new release in time.

SpecFlow+ Runner 1.8 Released

SpecFlow+ Runner 1.8 has been released and is available for download from NuGet. The main changes in this version are:

  • Support for SpecFlow 2.4
  • Change to the behaviour of the test adapter for compatibility with Visual Studio 15.8

This version also includes a number of smaller bug fixes.

SpecFlow 2.4 Released

SpecFlow 2.4 has been released and is now available for download from NuGet. This is likely to be the last release of SpecFlow before we release SpecFlow 3.

SpecFlow 2.4 includes a number of new features:

  • Added ability to convert type to same type or one of the base types #1110
  • Added support for customization of dependency injection at feature level via a runtime plugin event to raise feature dependencies #1141
  • Allow marking steps as Obsolete and have a configurable behavior for it #1140
  • IGherkinParser and IGherkinParserFactory interfaces added to support Gherkin Parser pluggability #1143
  • Assist: remove accents from column headers and property names when comparing objects to a table #1096
  • Added NUnit current TestContext scenario container registration. See #936
  • Array & List support for strings and enums when instantiating class from Specflow table #1018
  • MSBuild: Experimental support for deleting code-behind files on clean and rebuild when MSBuild #1167, #1208
  • MSBuild: Experimental support for Net.SDK project system, only for targeting desktop framework, .net core is supported and won’t work #1223
  • Changed Parameter handling of specflow.exe #1112

There are also a number of bug fixes. The full list of changes can be found here.

 

Don’t forget to regenerate your features and restart Visual Studio after upgrading!

 

Discover BDD with Gaspar Nagy and Seb Rose

Gaspar Nagy – who has been involved with SpecFlow since its inception – and Seb Rose – a core member of the open source Cucumber project – have teamed up to publish Discovery – Explore behaviour using examples. This is the first book in a series on BDD. Gaspar had the following to say about the motivation behind writing the book, and the type of content you can expect.

 

When it comes to BDD, there are generally 3 types of people:

  1. Those who do not know what BDD stands for at all (how sad!)
  2. Those who know BDD, and claim to implement it with their team, but where it turns out that their focus is only on the tools
  3. Those for whom BDD is a holistic approach

The initial idea was for a SpecFlow book that would essentially be a port of the “Cucumber for Java” book: a collection of technical tips etc. In other words, yet another technical book. While working on the core, it quickly became apparent that a SpecFlow book would never be complete without mentioning the collaborative aspects of BDD. So we ended up diving deeper into the BDD realm. At some point we made a decision: let this book be about BDD, instead of SpecFlow. In fact, let there be a BDD book series!

Why? Because in our opinion the BDD process can be divided into 3 stages:

  1. Discovery: explore and discover details through examples
  2. Formulation: convert examples to scenarios
  3. Automation: write test automation code

My painful experience is that the earlier a team adopts BDD, the better results they get. So why not give teams a helping hand (or even a compass) right at the start, in the discovery phase? We decided that this would be the essence of the first book in the series.

It took several months of intensive work with Seb to complete the 5 chapters of Discovery. At about 100 pages, it is relatively short, and said to be an easy read, so you could read it on a plane. The book is written for everyone: developers and testers, as well as product owners and test managers. Discovery – Explore behaviour using examples is also tool-agnostic: the emphasis is not on the technical process but on the collaborative aspect. The book covers the following:

Chapter 1 provides an introduction to the nature of BDD, where the important elements are explained.

Chapter 2 is an important chapter about structured conversation, providing hints on how testers and business stakeholders can be better involved in meetings, and how a shared understanding of the requirements can be achieved. A number of techniques, in particular Example Mapping (introduced by Matt Wynne), are described here through imaginative use cases.

Chapter 3 is about examples, and how there are more than just Given/When/Then test cases.

Chapter 4 covers tips on how a BDD approach can fit your project, regardless of whether you use Scrum, Kanban or any other framework.

Chapter 5 is all about how to sell BDD to business stakeholders, and make them interested in BDD. It includes tips on how to demonstrate that BDD is more than a mere technique applicable only to the tasks of developers and testers. Illustrative examples ensure that business stakeholders can relate to the advantages and share a common understanding as to why BDD is good for the entire team, saving time and effort on bug hunting. In order to reap the biggest benefits of BDD, the business side should be involved in the entire process from the start to finish.

Get your copy and let the discovery begin! You can find it on Amazon/Leanpub through http://bddbooks.com!

 

SpecFlow and PackageReference

By default, the packages used in SpecFlow projects are stored in packages.config. However the new Visual Studio projects instead use the PackageReference element to reference packages. When using the PackageReference element, generator plugins will not be found, and you will see errors similar to the following:

Unable to find plugin in the plugin search path: SpecRun. Please check http://go.specflow.org/doc-plugins for details.
#error TechTalk.SpecFlow.Generator

This is due to SpecFlow not searching for the package in the correct location. To ensure that the plugin is found, you need to specify the path explicitly in your app.config file.

To do so, enter the path to the appropriate path in the plugins element of your app.config file in this case, e.g.:

<plugins>
      <add name="SpecRun" path="%USERPROFILE%\.nuget\packages\specrun.specflow.2-3-0\1.7.0\lib\net45" />
</plugins>

IMPORTANT! You will need to manually update this path when upgrading to a newer version of the package, as the path contains the version number (e.g. “1.7.0” in the above example).

You can find a sample project using the new project format here. The relevant line in the app.config file is here.

Note: This requires SpecFlow 2.3.2 of higher. This step should not be necessary once SpecFlow 3 is released.

.NET Core Support for SpecFlow

We have been working on adding .NET Core support to SpecFlow. You can follow the current status here. If you are interested in using SpecFlow together with .NET Core, we would appreciate your input, feedback and even code contributions.

Our current estimates mean that we hope to be able to release a version of SpecFlow with support for .NET Core by Q3 of this year. The more testers and contributors we have, the quicker the process should be.

If you are interested in helping out, please visit the .NET Core Support sub-project on GitHub. Tasks tagged with “request-for-comment” are task where we are especially keen on getting feedback from the community. We would of course appreciate relevant feedback on the other tasks as well.

 

If you would like to contribute, please contact @SabotageAndi on GitHub first to prevent multiple contributors working on the same task. Not all issues on GitHub are worked out in full detail. In particular, issues with the “Request-for-comment” label or “[RFC]” in the title are not finalised. These tasks still require more discussion and evaluation. If you are able to contribute to the discussion, please do!

 

SpecFlow+ Runner and Visual Studio 2017 15.6

Visual Studio 2017 15.5 Preview 2 introduced a new real time test discovery feature. This feature is live with 15.6.

When using VS 2017 15.6 with SpecFlow+ Runner, the Test Explorer will not display any tests in VS unless you enable Additionally discover tests from built assemblies after builds under Tools | Options | Test:

We are looking into how we can support real time test discovery with SpecFlow+ Runner. Until then, please make sure you enable this option in VS if you are using SpecFlow+ Runner with Visual Studio 2017 15.6.

NuGet Packages – Reserved ID and Naming Conventions

Microsoft is introducing package identity verification for packages on nuget.org. This will allow developers to reserve particular ID prefixes used to identify. This in turn should help users identify which packages have been submitted by the owner of the ID prefix.

We have submitted a request to reserve the “SpecFlow” NuGet package prefix, which is used to identify official SpecFlow and SpecFlow+ packages. This will mean that new packages with the SpecFlow prefix can only be submitted by TechTalk, and will indicate that these packages are official.

We have also requested the SpecFlow.Contrib prefix be made publicly accessible for developers who want to release their own packages for SpecFlow. If you want to submit your own package for SpecFlow whose name begins with “SpecFlow”, you can use this prefix. This will indicate to users that the package is intended for use with SpecFlow, but is a third-party contribution.

These changes will not affect existing packages using the SpecFlow prefix that have already been submitted to nuget.org. If you are the owner of such a package, you should be able to update the package as usual. You may however want to change the name of your package to reflect the new convention.

 

In summary, here are the prefixes we have requested:

  • SpecFlow.*
  • SpecRun.*

 

You can find out more about the idea behind package IDs on the nuget.org blog.

Issue with SpecFlow+ Runner and Latest Visual Studio 2017 Version

Update: SpecFlow+ Runner 1.6.2 includes a fix for this issue. You now only need to enter a GUID if you are using the SpecFlow+ server (for adaptive test scheduling), as the GUID is used as the unique key for the test results.

 

We have discovered an issue that affects SpecFlow+ Runner with Visual Studio 2017. When adding the NuGet package to your project, a default profile is added to your project (default.srprofile). During the installation process, the projectName and projectId should be entered automatically in this profile.

When installing the packages with the latest version of Visual Studio 2017, the projectName and projectId are no longer filled in automatically. This results in your tests not being discovered, meaning they cannot be executed.

As a workaround, you can enter the projectName and projectId in your profile yourself. The projectName should be the name of the project you added the NuGet package to; the projectId should be the GUID of the project (this is the <ProjectGuid> in your .csproj, although in theory you can enter any valid GUID here).

An issue has already been opened, and you can track the status here.