TDD, BDD, ATDD, Acceptance Tests, Unit Tests – What’s the Relationship

A recent blog by Gojko Adzic asked for “How to choose between BDD and unit tests?”.  A recent blog of mine talked about the testing matrix, which also deals with the relationship between the processes and the tests.

An equivalent question has been asked many times during the workshops I give. The answer is that it’s like comparing apples and oranges. There is an overlap between what behavior the test is testing, the style of the test, the creators of the test, and the time of creation. 

External and Internal Behavior

There are basically two types of behavior that one tests – the external behavior that is visible to a user and the internal behavior of the implementation. 

External tests should be implementation independent. It shouldn’t matter whether the system is written in Java, JavaScript or Ruby. 

Internal tests are implementation dependent. The components that are created to pass external tests (methods, classes, microservices) have tests that specify their behavior. These are obviously implementation dependent.

Systems have varying numbers of external and internal tests. For an example, the tests for transferring a file from one computer to another might have tests revolving around the following areas:

External tests

Check that a file shows up on another computer with the same contents and appropriate associated data, such as date modified

Internal tests

Is each packet received correctly, e.g. are the checksums correct?  

Are the headers correct on each packet?

The implementation could use UDP, TCP, or some other protocol to transfer the file. The external test is independent of the implementation. There might be some other external requirement that suggests one implementation over another, but the external test above would still be the same.

As another example, the external tests may be very detailed.  For the tax computation on an order, the tests might look like:   

External Tests

Order from a particular jurisdiction has the correct tax applied

Tax for $10.00 order from Durham, NC is  $.75

Tax for $10.00 order from Chapel Hill, NC is $.70

In this case, the tax appears only as part of an order, but the computation itself is specified by an external user. So the test for the variations in taxes should be visible to that user. That makes it an external test. The responsibility for passing the test may be that of a component or a single method.  

Style of the test

A test can be written in Gherkin, an XUnit framework (JUnit, NUnit, etc.), a tabular format (e.g. FIT/FitNesse), or some other form.   

From my workshops, the customers prefer reading tests in Gherkin or FIT over XUnit tests.   (I don’t show other forms in the workshops).  So it’s recommended that external behaviors/tests be written in that form.  

For internal behavior, the tests are typically written in an XUnit framework. However, they could be expressed in Gherkin. This separates out the desired internal behavior from the implementation of that  behavior. In more complex situations, that can help in maintenance. The test represented by the Gherkin does not change even if the implementation is completely refactored. For example, there might be a internal test such as:

Scenario:   Packet has the correct header 
Given packet containing: 
|Data |
|abcdef| 
When the header is computed 
Then it should contain:
| Data Count | Checksum |
| 6                   | 2345           | 

Creators of the test and the time of creation.

The triad (Customer, Developer, Tester) collaborate on defining external behavior. That external behavior, as typically defined with scenarios, represents the tests of the implementation. They could create the tests after the implementation, but that would be wasteful. Instead, they employ Behavior Driven Development / Acceptance Test-Driven Development to create the scenarios/tests prior to implementation. The acceptance tests represent the external functional requirements for the system. BDD and ATDD are just two slight variations of the same process. BDD focuses on the scenarios as the behavior and ATDD focuses on the scenarios as the acceptance tests for that behavior. Specification by Example is another variation of this process.   

The developer, developer pair, or developer ensemble create the internal behavior tests. They are often referred to as unit tests. However, they should be testing units of behavior, not units of code (e.g. methods). The developers could do this after implementation, but that is typically wasteful. Instead, they employ Test Driven Development to create the tests just prior to implementation.  

An Answer to the Question

Should there be both external and internal tests that check the same behavior? There will always be internal tests that check a portion of an external behavior. Those tests could be derived from the external tests. So there might be a “Once and Only Once” issue. But the tests are used for two different purposes – testing external behavior and testing the components that help create that external behavior.  On the other hand, should there be both a Gherkin style and a XUnit style test against the same externally visible calculations?   If the team is collaborating, then the Gherkin style would be created first and thus there would be no need for an XUnit style test.