What's new?

Releases of SpecFlow

SpecFlow Suite 3.9

2021-06-21
3.9
Affected Products: SpecFlow, SpecFlow+ LivingDoc, SpecFlow+ Runner

We are excited to announce that SpecFlow 3.9 is finally publicly released. In this release we fixed several bugs and tried to solve a long running issue with SpecFlow and Mac OS.

A big Thank You to all our amazing contributors!

As always this release would not be possible without you.

Here is a list of all the contributors to SpecFlow since the previous release:

Changelog:

SpecFlow

Features

  • Introduce NonParallelizableDecorator to opt out of feature level parallelization using tags #2409

Changes

  • Update to Gherkin 19.0.3
  • Replace Utf8Json with SpecFlow.Internal.Json
  • Remove ParallelizeDecorator introduced in #746. Fixes #2442

Bugfixes

  • Fix Type or namespace [TestNamespace]_XunitAssemblyFixture could not be found #1825
  • Fix Default value deserialization from specflow.json #2239 #2439

SpecFlow+ Runner

Features

  • Support for SpecFlow 3.9

SpecFlow+ LivingDoc

Bugfixes

  • Fix work item linking bug in the BuildTask, related to Github#2333

SpecFlow Suite 3.8

2021-05-21
3.8
Affected Products: SpecFlow, SpecFlow.ExternalData, SpecFlow+ LivingDoc, SpecFlow+ Runner

We are excited to announce that SpecFlow 3.8 is finally publicly released. The biggest new feature we added in this version is a dedicated API for additional output (text and files) for your scenarios.

Yes, you could this before by using the APIs of various test runners, but you had to have prior knowledge on how to use them. Also, having an abstraction layer makes it possible for us to hook into it and display further valuable information. The first thing we did with this new extension point is to bring the output to SpecFlow+ LivingDoc.

Test Output in SpecFlow+ LivingDoc

You can learn more about the Output API and its usage in our documentation. To use it, you simply update the SpecFlow NuGet packages to the latest version, and you are ready to go.

We developed this because our community upvoted this feature on our feature request list. The feature request became very popular; therefore, we decided to adjust our backlog priorities to implement this swiftly and bring more value to our users.

A big Thank You to all our amazing contributors!

As always this release would not be possible without you.

Here is a list of all the contributors to SpecFlow since the previous release:

Changelog:

SpecFlow

Features

  • Route output API messages to tool output
  • Extend ISpecFlowOutputHelper with attachment handling
  • Add events (IExecutionEvent) to track what happens and when

Bugfixes

  • Change CultureInfoScope to readonly struct
  • Improve performance / Reduce allocations of RuntimePluginLocator & LocationMerger
  • Fix spelling of IRuntimePluginTestExecutionLifecycleEventEmitter.RaiseExecutionLifecycleEvent
  • Improve performance by lazy initializing certain expensive members
  • Fixed bug in StepTextAnalyzer with empty string parameters
  • Update to NUnit 3.13.1 as minimum NUnit version
  • Step argument transformations should not be executed if a previous step had an error https://github.com/SpecFlowOSS/SpecFlow/issues/2382
  • Update “dotnet new” template in sync with the Visual Studio template
  • Fix NullReferenceException during assembly loading in PluginAssemblyResolver #2344 #2403
  • Update MsTest.TestFramework to 2.1.2 as minimum MsTest version

SpecFlow+ Runner

Features

  • Support for SpecFlow 3.8
  • Output API support

Fixes

  • Fix: execution on Octopus agent
  • Fix: circular reference error in JSON serialization
  • Fix: building problems in case of OS-specific TFMs

SpecFlow+ LivingDoc

Features

  • Support for SpecFlow 3.8
  • Output API support
  • Display Background steps at collapsed at every Scenario

Fixes

  • Fix problem with .NET Core 2.1
  • Handle Spaces in feature files and folders better

SpecFlow.ExternalData

Features

  • Support for SpecFlow 3.8

SpecFlow Suite 3.7

2021-02-17
3.7
Affected Products: SpecFlow, SpecFlow.ExternalData, SpecFlow+ LivingDoc, SpecFlow+ Runner

What is new?

We were finally able to fix a major bug that happens during the parallel execution of scenarios and was a problem for a lot of users. Parallel execution on feature-level (one scenario of multiple features is running in parallel) should work now without problems. If you have still one, please create an issue on GitHub.

A big Thank You to all our amazing contributors!

As always this release would not be possible without you.

Here is a list of all the contributors to SpecFlow since the previous release:

Changelog:

SpecFlow

Features:

  • support translations for true value

Fixes:

Changes:

  • Ignore tag handling: generate test framework specific ignore attributes

SpecFlow+ Runner

Features:

  • Support for SpecFlow 3.7

Fixes:

SpecFlow+ LivingDoc

Features:

  • Support for SpecFlow 3.7

Fixes:

  • Fixed an issue with xUnit that the TestExecution.json was generated in the wrong folder

SpecFlow.ExternalData

Features:

  • Support for SpecFlow 3.7

SpecFlow Suite 3.6

2021-01-26
3.6
Affected Products: SpecFlow, SpecFlow+ LivingDoc, SpecFlow+ Runner

What is new?

We started the new year with an open-source iteration so we can more closely work with our community on SpecFlow. This release includes a number of features and bug fixes for SpecFlow and SpecFlow+ product range.

A big Thank You to all our amazing contributors!

As always this release would not be possible without you.

Here is a list of all the contributors to SpecFlow since the previous release:

Changelog:

SpecFlow

Features:

  • support translations for true value

Fixes:

Changes:

  • Ignore tag handling: generate test framework specific ignore attributes

SpecFlow+ Runner

Features:

  • Support for SpecFlow 3.6

SpecFlow+ LivingDoc

Features:

  • Support for SpecFlow 3.6

SpecFlow & SpecFlow+Runner 3.5

2020-11-16
3.5
Affected Products: SpecFlow, SpecFlow+ Runner

What is new?

This release of SpecFlow and SpecFlow+ Runner combines a few small fixes. The biggest change is the support for .NET 5. More details about it can be read here. For a complete list have a look at the detailed changelog below.

Thanks to all our amazing contributors!

This release would not be possible without you.

These are contributors to SpecFlow since the last 3.4.31 release

Changelog:

SpecFlow

Features:

  • Support for .NET 5
  • Add support for Castle Windsor dependency injection via plugin

Fixed:

  • System.* packages are no longer referenced in .csproj files. Instead Framework Reference is used. #1962
  • Result collection has not been started issues #2008 #2159 #1956
  • Generated code causes CS1591 doc warning #2189

SpecFlow+ Runner

Features:

  • Support for SpecFlow 3.5
  • Support for .NET 5

Bugfixes:

Fix:

  • Result collection has not been started issues #2008 #2159 #1956

SpecFlow & SpecFlow+Runner 3.4

2020-08-19
3.4
Affected Products: SpecFlow, SpecFlow+ Runner

What is new?

This release of SpecFlow and SpecFlow+ Runner combines a few small fixes. The biggest change is the support for our new upcoming product SpecFlow+ LivingDoc Generator. For a complete list have a look at the detailed changelog below.

Thanks to all our amazing contributors!

This release would not be possible without you.

These are contributors to SpecFlow since the last 3.3.74 release

If you didn’t already, you can get some stickers from us here (When we are back in the office, because there are all our stickers stored).

Changelog:

SpecFlow

API Changes

  • IGherkinParser.Parse method signature change – external usages must be updated

Changes

  • Allow incremental builds to work correctly for projects with MSTest/NUnit/xUnit integration
  • Extend FeatureInfo with FolderPath information
  • RuntimePluginTestExecutionLifecycleEvents – allow runtime plugins to subscribe to test execution lifecycle events (similarly to the hooks in binding code)
  • Remove TestRunEnd extension point (obsolete with RuntimePluginTestExecutionLifecycleEvents)

SpecFlow+ Runner

Features

  • Support for SpecFlow 3.4

Bugfixes

  • Fix issue with test execution after publishing the test project

SpecFlow & SpecFlow+Runner 3.3

2020-06-24
3.3
Affected Products: SpecFlow, SpecFlow+ Runner

Yes, you are reading correctly. We are jumping directly from SpecFlow 3.1 to SpecFlow 3.3. This is to make it easier to understand, which version of SpecFlow works with which version of SpecFlow+ Runner. More details can be found here.

What is new?

This release of SpecFlow and SpecFlow+ Runner combines a lot of small and medium fixes, features, and performance increases. For a complete list have a look at the detailed changelog at the end.

There were two bigger changes, that made it necessary to increase the version number to 3.3.

Support for .NET Core SDK 3.1.200 and later

Because of some behavior changes in the .NET Core SDK 3.1.200 and later, we had to made adjustments to our Assembly loading logic, that SpecFlow is working with it again.

Supported .NET Versions

Starting with this version, we are supporting .NET Framework 4.6.1 and later and are following the Microsoft .NET Core Support policy. You can read more about this in this blog post.

This means, we support:

  • >= .NET Framework 4.6.1
  • .NET Core 2.1
  • .NET Core 3.1

Thanks to all our amazing contributors!

This release would not be possible without you.

These are contributors to SpecFlow since the last 3.1 release

If you didn’t already, you can get some stickers from us here (When we are back in the office, because there are all our stickers stored).

Changelog:

SpecFlow

Features

API Changes

  • Now allows override of method categories in NUnit to allow for plugin development on this function
  • (breaking) tableRow.GetEnum(..) has now a class contraint on T
  • (breaking) tableRow.GetTheEnumValue(..) has now a class contraint on T
  • added tableRow.GetDiscreteEnum
  • TechTalk.SpecFlow.Assist.ValueRetrievers.* public classes have undergone some refactoring. IValueRetriever interface remains unchanged.

Changes

  • Performance and memory improvements for table.CreateSet and table.CreateInstance
  • Improved performance of ValueRetrievers
  • Removed NullableValueRetriever as nullability is now handled by the ValueRetriever directly.
  • Changed all specflow.org hyperlinks to https (docs, generated code, packages)
  • Changed minimal .NET Framework version from .NET 4.5 to .NET 4.6.1
  • Changed minimal .NET Framework version of the Generator plugins and the SpecFlow.Tools.MSBuild.Generation package from .NET 4.7.1 to .NET 4.6.1
  • Downgrade to be able to use xUnit 2.4.0 (because VS Templates are using this version)

Fixes

  • Empty value for nullable enum should not throw an exception
  • RegEx performance fix in TEHelpers (Table column name validation)
  • Adjust assembly loading of Generator plugins to changed behavior in .NET Core SDK 3.1.200 and later – https://github.com/SpecFlowOSS/SpecFlow/issues/1912
  • add auto-generated comment for StyleCop to ignore the Assemblyhook files

Fixes for Development

  • Changed unit tests to ignore custom user overrides to data formatting in en-US locale
  • Some errors could randomly happen when running tests in parallel

SpecFlow+ Runner

Features

  • Support for SpecFlow 3.3

Bugfixes

SpecFlow & SpecFlow+Runner 3.1

2019-11-14
3.1
Affected Products: SpecFlow, SpecFlow+ Runner

We are happy to announce the release of SpecFlow and SpecFlow+Runner 3.1. This update fixes a lot of smaller issues found by our community members over the last months, but we have also added some bigger features to this release.

What’s new?

1. .NET Core 3.0 Support

With the addition of nullable reference types in C# 8, we needed to make some changes to our binding execution code. These changes have now been implemented, and all our tests for .NET Core 3.0 are successful.

2. Cucumber Messages

The integration of Cucumber messages is a joint effort with the Cucumber community to create a unified protocol for exchanging data between different components of SpecFlow and Cucumber. There will be no longer be a need to implement separate HTML report generators for each Cucumber implementation and SpecFlow. Cucumber messages allow you to use the same HTML report generators for SpecFlow and Cucumber.

SpecFlow 3.1 contains the first set of Cucumber Messages: TestRunStarted, TestCaseStarted, TestCaseFinished and TestRunFinished. We plan on implementing additional messages that allow you to generate different output formats/reports in future releases.

Cucumber messages are currently disabled by default. We plan to enable them by default with SpecFlow 3.2. You can find the current documentation here.

Changes Related to Cucumber Messages

For end users, the introduction of these messages does not bring any changes from previous versions, as we have not changed SpecFlow’s overall behavior.

However, we changed the internal workings of how scenarios are ignored. In the past, we generated the appropriate ignore attribute for the test runner in the code-behind file. But as we now also need to send a cucumber message for skipped scenarios, we no longer generate the ignore attribute and skip the scenario at runtime instead.

3. Docker Support in SpecFlow+Runner

One thing we learned is that just because SpecFlow is compiled and runs with .NET Core, that does not mean it will run on Windows, Linux and Mac OS. Not all aspects of.NET Core are applicable to all 3 operating systems. SpecFlow+Runner 3.1 now uses APIs that also work on Linux and MacOS. All SpecFlow+Runner features should also now work in Docker. If not, please open an issue.

If you want to generate SpecFlow+ Runner reports on non-Windows operating systems, you need to install mono. Details can be found here.

A sample SpecFlow+ Runner project demonstrating support for Docker support can be found here.

4. Gherkin 6 “Rule” Keyword

SpecFlow 3.1 supports the new Gherkin 6 Rule keyword.

However, the Visual Studio Extension does not yet support this keyword. Using it will stop the syntax highlighting from working.

5. Project Template

To make it easier to create new SpecFlow projects, we have created a project template that you can access with dotnet new. You can find more information on installing the template and using it to create new projects here.

Thanks to all our amazing contributors!

Thanks to the following users for their contributions to SpecFlow 3.1:

Changelog:

SpecFlow

Features

  • Support for .NET Core 3.0
  • Cucumber messages:
    • First set of cucumber messages are implemented:
      • TestRunStarted
      • TestCaseStarted
      • TestCaseFinished
      • TestRunFinished
    • Documentation: https://specflow.org/documentation/Cucumber-Messages/
    • Cucumber messages can be enabled via specflow.json and app.config
    • File sinks for Cucumber messages can be configured
  • Support for Gherkin v6 “Rule” keyword
  • Support for discovering step definitions from attributes that are derived from Given, When, Then or StepDefinition attributes (related to Issue 1745)

Fixes

  • AfterTestRun hook can take longer than 100ms on .NET Core (Issue 1348)
  • Adjust parameter names generation in scenario outlines (Issue 1694)
  • specflow.json can be used in SpecFlow+Runner with Process test thread isolation (Issue 1761)
  • specflow.json is copied to output directory automatically (Issue 315)
  • Step definition method with “object” parameter type might not match

SpecFlow+Runner

Features

  • Support for .NET Core 3.0
  • Support for SpecFlow 3.1

Fixes

  • Test execution in Docker works
  • Improved performance when executing projects with a lot of scenarios (> 10,000)

Changes

  • Changed the communication between main process and child processes from named pipes to TCP/IP

SpecFlow 3.0

2019-03-28
3.0
Affected Products: SpecFlow, SpecFlow+ Runner

Documentation on how to migrate can be found at https://docs.specflow.org/projects/specflow/en/latest/Guides/UpgradeSpecFlow2To3.html

A BIG THANK YOU TO ALL CONTRIBUTORS!

SpecFlow 3 would not have been possible without your contributions. 

alexvv
atanian123
atmcarmo
banovotz
Blackbaud-ChrisKessel
ChernyavskiyAlex
DanielTheCoder
david1995
Dietatko
epresi
gasparnagy
icnocop
inkinjo
kevinkuszyk
midgleyc
mmajcica
nbalakin
nesli92
nvborisenko
Regenhardt
SabotageAndi
sGeeK44
SirBoxBoxington
slig3
szaliszali
thomasouvre
Tragedian

Changelog:

SpecFlow

New features

  • Support for .NET Core!
  • Separate addition of default and non-default value comparers. ComparisonException indicates which value comparer was used for each difference
  • Uri and Null value retrievers added
  • BoolValueRetriver can now work with 1 and 0
  • Array & List support for any available item value retriever when instantiating class from Specflow table
  • Syntax and structural changes in value retriever/comparer registration
  • Check for non-default constructors using case-insensitive comparison
  • Add utility to get enum field value from TableRow.
  • Gherkin language data is used instead of an obsolete copy
  • Loader exceptions are appended to the exception message
  • Add CreateInstance support to a TableRow

Fixes

  • Visual Studio change detection integration for Net.SDK style projects via SpecFlow.Tools.MSBuild.Generation

Changes

  • Command line options now need to be prefixed with -- (double dash) rather than /

Breaking changes

  • Registration of value retrievers and comparers has been changed from RegisterValueComparer() to ValueComparers.Register()
  • Calling CreateInstance on an object with a constructor with default parameters populates the object using the default parameters instead of null values
  • Visual Studio change detection integration for Net.SDK style projects via SpecFlow.Tools.MSBuild.Generation

Known issue

  • AfterTestRun code fails to complete under certain circumstances, see this GitHub issue for details and possible workarounds.
  • Step bindings from external assembly are not found: set Copy to output directory for specflow.json.

Removed

  • The functions provided by specflow.exe (generate all, reports) have been removed completely.

SpecFlow+ Runner

New features

  • Support for .NET Core 2.0, 2.1, 2.2
  • Added DeleteFolder as a new deployment transformation step

Fixes

  • Reports: Display the correct success rate for retried scenarios
  • Profile files are no longer duplicated in the solution explorer
  • Display failing scenarios with at least one pending or successful retry as randomly failed

Changes

  • Command line options now need to be prefixed with -- (double dash) rather than /

Breaking changes

  • Report generation has been moved to a separate process
  • The base class for reports has been changed from TechTalk.SpecRun.Framework.Reporting.CustomTemplateBase to SpecFlow.Plus.Runner.Reporting.CustomTemplateBase
  • If you have customized the report templates, you will need to edit the first line of your templates accordingly.

Removed

  • Support for combined test executors (e.g. SpecRun+NUnit and SpecRun+MSTest) has been removed as a result of the new plugin infrastructure

Behavioral changes:

  • If CLR 2.0 is specified as the runtime, SpecFlow+ Runner uses the CLR 4.0 executor.

SpecFlow 2.4

2018-08-20
2.4
Affected Products: SpecFlow

SpecFlow 2.4 has been released and is now available for download from NuGet. This is likely to be the last release of SpecFlow before we release SpecFlow 3.

Changelog:

  • Added ability to convert type to same type or one of the base types #1110
  • Added support for customization of dependency injection at feature level via a runtime plugin event to raise feature dependencies #1141
  • Allow marking steps as Obsolete and have a configurable behavior for it #1140
  • IGherkinParser and IGherkinParserFactory interfaces added to support Gherkin Parser pluggability #1143
  • Assist: remove accents from column headers and property names when comparing objects to a table #1096
  • Added NUnit current TestContext scenario container registration. See #936
  • Array & List support for strings and enums when instantiating class from Specflow table #1018
  • MSBuild: Experimental support for deleting code-behind files on clean and rebuild when MSBuild #1167, #1208
  • MSBuild: Experimental support for Net.SDK project system, only for targeting desktop framework, .net core is supported and won’t work #1223
  • Changed Parameter handling of specflow.exe #1112

There are also a number of bug fixes. The full list of changes can be found here.

Don’t forget to regenerate your features and restart Visual Studio after upgrading!

SpecFlow+ Runner 1.8

2018-08-20
1.8
Affected Products: SpecFlow+ Runner

Changelog:

  • Support for SpecFlow 2.4
  • Change to the behavior of the test adapter for compatibility with Visual Studio 15.8

SpecFlow 2.3

2018-02-19
2.3
Affected Products: SpecFlow

SpecFlow 2.3 has been released with the following changes:

Changelog:

New features

Fixes

API Changes

Miscellaneous

  • A new NuGet package, “SpecFlow.Tools.MsBuild.Generation”, is available to easily enable the code-behind generation when building your project. The documentation will be updated soon.

SpecFlow+ Runner 1.7

2018-02-19
1.7
Affected Products: SpecFlow+ Runner

Changelog:

New Features

Fixes

  • runtests.cmd: uses msbuild.exe from path and not from %windir%\Microsoft.NET\Framework\v4.0.30319
  • Log entries are no longer written to the report file if you define the name of the report in your .runsettings file
  • VisualStudio.srProfile is used if it exists and the test run is started from within VisualStudio

SpecFlow+ Runner 1.6.3

2017-10-03
1.6.3
Affected Products: SpecFlow+ Runner

SpecFlow+ Runner 1.6.3 has been released and is available to download from nuget.org.

Changelog:

  • CommunicationObjectFaultedException thrown when using process separation in conjunction with DateTime
  • CommunicationObjectFaultedException with tests that take a long time to complete. This was caused by a time limit of 10 minutes for responses; the time limit has been extended to 24 hours.

SpecFlow+ Runner 1.6.2

2017-09-26
1.6.2
Affected Products: SpecFlow+ Runner

SpecFlow+ Runner 1.6.2 is now available to download from NuGet.org. This release addresses two issues in SpecFlow+ Runner

Changelog:

  • Workaround for issue 935, where the project name and projectId (GUID) are not automatically entered in your profile when installing the package. Note: You will still need to manually enter a GUID in your profile if you are using SpecFlow+ Server for adaptive testing. The projectId should be the GUID of the project (this is the in your .csproj, although in theory you can enter any valid GUID here).
  • Thread.Current.Name again returns “Test Thread #” when executing tests in multiple threads

SpecFlow+ Runner 1.6.1

2017-09-15
1.6.1
Affected Products: SpecFlow+ Runner

SpecFlow+ Runner 1.6.1 is now available from NuGet. This minor release and adds support for the “Run Functional Tests” task in TFS/VSTS and fixes the following issues.

Changelog:

  • Project paths containing a hash character (‘#’) no longer cause a file not found exception
  • Tests are no longer duplicated in the Test Explorer in VS