Case study

TOMRA Food and SpecFlow: Using BDD in IoT Edge Technologies

Introducing TOMRA Food

TOMRA Food’s industry-leading sorting machines utilize a broad range of sensor technologies coupled with real-time image processing engines over embedded and PC architectures.

The Machine Intelligence team, a new team within TOMRA Food, is leveraging new and existing data sources to improve our food sorting products in sorting performance, reliability, predictive maintenance, anomaly detection, and improved data/analytics feedback.

100,000 units installed
Over 80 markets worldwide
9,9 billion NOK in 2020
4300 employees globally
Listed in OSE:TOM

Behavior Driven Development (BDD) journey in Tomra Food

The machine development team in Dublin, Ireland were the first in TOMRA Food to embrace the Behavior Driven Development process. The team became familiar with the Gherkin language and deliberate discovery practice during the three amigos meetings. After some time, the team felt more comfortable specifying the machine software in terms of its behavior. Requirements described by scenarios allow for less ambiguous deliverables. We experienced that the process improves in structure, and the definition of done is more complete. Automating the scenarios enables a faster feedback loop and guards against regressions. As a result, the end-to-end and the integration test coverage have increased substantially.


We experienced that the process improves in structure, and the definition of done is more complete. Automating the scenarios enables a faster feedback loop and guards against regressions. As a result, the end-to-end and the integration test coverage have increased substantially.

Victor Sypniewski-Tom, Software QA Engineer at TOMRA Food

The scenarios written by the team to describe machine behavior are now used as training material, which in itself is a testament to the success of the BDD adoption and work done by the team. Newcomers, as part of the onboarding process, are provided with the feature files. They follow the scenarios at the machine. As a result, it now only takes a couple of days for a person to feel comfortable operating the sorting machine.

Outcome:

Less Ambiguous Deliverables
Structure Improvement
Faster Feedback
Increased Sustainability

High-level overview

This project makes use of TOMRA Food sorters, IoT Edge, cloud blob storage, Azure DevOps, and of course, the SpecFlow with SpecFlow+ Runner. The application under test (AUT) is the Connector. The central point of the solution, the IoT Edge runtime, is responsible for managing the connector module and routing its messages.

On the cloud side, three components are of interest when the automated tests run. The container registry holds the docker containers with the version of the AUT. IoT Hub through which the deployment of the correct version of the AUT and routing of the messages is achieved. Blob storage holds the messages from the machine.

Aim of AUT:
To deliver data from the machine to the cloud.

Objective for Automated Tests:
to verify that all the data is delivered in the correct format, and that logic applied by the IoT Edge connector module is in accordance with the specification.

Implementation

At the time of an automated build, the AUT pipeline triggers the testing pipeline using Azure Pipeline resources functionality. In this case, the AUT is the resource for the testing pipeline. This allows passing the version of the Connector to the testing pipeline.

The testing pipeline runs on the Ubuntu VM runner at which the desired version of the IoT Edge runtime is installed. SpecFlow is in charge of the events to follow.

3 Tasks of SpecFlow

1

Deploy the connector version

2

Run the scenarios

3

Report test results

1. Deploy the connector version to be tested

Before any tests run, SpecFlow is using a class and SpecFlow hooks to prepare the environment. [BeforeTestRun] hook updates the IoT Edge device deployment manifest in the IoT Edge Device twin in the cloud. In this step, SpecFlow instructs the environment to use a version of the connector module built in the AUT build that triggered the tests. It is implemented using .NET Azure IoT client API. This in turn, causes the IoT Edge runtime to download the AUT container version to the CI runner. SpecFlow then checks if the IoT Edge runtime is reporting the desired version of the Connector. This check is performed against the edge connector module reported properties in the cloud. When the correct version of AUT is confirmed, the test run can start.

2. Run the scenarios

SpecFlow runs several scenarios that use server implementation to which the AUT connects. Then the data from the scenario examples are used to form a message payload. When the stream connection is established, the message is sent to the connector client.

Upon receiving the message, Connector performs the data manipulation and timing logic and sends the message to the IoT Hub. Eventually, the message is present in the blob storage.

Following the steps in the scenario and using .NET Azure blob storage API, the SpecFlow extracts the message from the blob storage. The blob storage message is compared with the message sent by the Connector. We use Fluent Assertions for this purpose.

3. Report test-runs

The test report can be viewed directly in Azure DevOps after the tests are run. In case of failed tests, three attachments are produced: HTML report, log files, and the report in .trx format. Azure DevOps provides integration to view the results directly in the pipeline. This includes viewing the logs. Additionally, HTML report and logs can be downloaded.

Test Reports can be viewed as:

HTML
LOG
TRX

We believe that the success of the software project hugely depends on well-defined requirements. We also believe that the quality of the software product is one of its features. Behavior Driven Development is a tool that allows us to communicate the requirements better. SpecFlow makes it possible to check our code against its requirements specification consistently.

Victor Sypniewski-Tom, Software QA Engineer at TOMRA Food

Ready to get started? It's free!

Use SpecFlow

About TOMRA Food

TOMRA Food designs and manufactures sensor-based sorting machines and integrated post-harvest solutions for the food industry. We innovate the world’s most advanced analytical technology and apply it to grading, sorting, and peeling.

Over 12,800 units are installed at food growers, packers, and processors worldwide for confectionery, fruit, dried fruit, grains and seeds, potato products, proteins, nuts, and vegetables.

The company’s mission is to enable its customers to improve returns, gain operational efficiencies, and ensure a safe food supply via innovative, useable technologies. To achieve this, TOMRA Food operates centers of excellence, regional offices, and manufacturing locations within the United States, Europe, South America, Asia, Africa and Australasia.

TOMRA Food is a member of the TOMRA Group that was founded on an innovation in 1972 that began with the design, manufacturing and sale of reverse vending machines (RVMs) for automated collection of used beverage containers. Today TOMRA provides technology-led solutions that enable the circular economy with advanced collection and sorting systems that optimize resource recovery and minimize waste in the food, recycling and, mining industries and is committed to building a more sustainable future.

TOMRA has ~100,000 installations in over 80 markets worldwide and had total revenues of ~9.9 billion NOK in 2020. The Group employs ~4,300 globally and is publicly listed on the Oslo Stock Exchange (OSE: TOM).