Approval Testing is well known as a good approach for getting Legacy Code under control. Did you know it can also be used in new development as part of a BDD process?
4 in a row
I expect you’ve played this game “4 in a row”? The board is a vertical grid and players take it in turns to drop coloured tokens in the top. The first player to get a row of 4 tokens in their colour wins. A colleague of mine, Olof Bjarnason, has been having fun implementing this game with Python and trying out an unfamiliar design approach. I worked with him on it and you can find the code on GitHub.
The picture shows a game that is in progress. When you click the mouse, the coloured token drops to the lowest available space in that column. As you can see, red has missed at least one opportunity to win already, and yellow is poised to take victory this move.
Think for a moment. How would you test that? And how would you develop this game using Behaviour Driven Development? Let’s work through this example and talk about how to do it with BDD and an Approval testing approach.
Discovery – examples workshop
Early in the development of this game you’d perhaps hold a workshop and invite developers, testers and business stakeholders. What kind of examples would you come up with? I’m imagining a whiteboard with a number of sketches looking a little like this:
The 4-in-a-row game naturally lends itself to visual tests. You want to check what’s on the board. This example sketch includes both the board position, the expected outcome ‘blue wins’ and a scenario name: ‘backslash’.
Formulation – designing a Printer
A developer and a tester generally work together to turn examples into executable tests. The tester knows what’s important to check for and the developer knows what’s possible to automate. They get together to formulate the sketches from the discovery workshop into something that is both descriptive, readable and automatable.
The scenario sketch above shows the position on the board, and to formulate that as a test you need to decide how to check that. One approach would be to compare the board shown on the screen pixel-by-pixel against a previously approved screenshot. Olof and I chose not to do it quite like that though. We developed a ‘Printer’ that can display the game state in plain text. ASCII art, if you like.
This is a snippet of the approved text for one of the test cases:
I think you should be able to read this and understand what this test checks for. The output includes a list of all the player moves that lead up to the final state. Yellow wins the game with a 4-in-a-row diagonal ‘backslash’.
One reason to use a textual description like this instead of an image for comparison, is that plain text is easy to version control and update. You can use a straightforward diff-merge tool to examine test failures. More importantly however, is the control it gives us over the level of detail the test checks for.
Later on in the approved text for this scenario there is a ‘simulation log’ that prints various other details. This is a short section of it:
The point of this section is to ensure relevant details of the game are working properly as each player places their pieces.
In the “Formulation” part of BDD the tester and the developer discuss what they want to check in the test. They can decide together what details of the position to log. What colours and mouse clicks and sounds are important to include in the checked output. This discussion begins before the program code is written, but the final version of the test is not finalized until the ‘automation’ part is done, and they can ‘approve’ the result.
Automation – implementing test cases and Printers
The developer begins the automation work with a good idea about what the formulated printed output should look like. They need to write some test code to simulate clicks and print the agreed details of the game state. I haven’t included that code here – it’s like the step definitions you have in a SpecFlow or Cucumber test. Ideally you shouldn’t need to read that code to understand what the test checks for. If you’re interested, the test code is in this file on GitHub.
In BDD the focus is on enabling better conversations with stakeholders. That is still true when you use an Approval Testing approach. You want to create executable specifications that are readable to all parties. In this example we showed the language you use can be a kind of textual printout or log showing the important actions and outputs of the system under test.
This approach is not exclusively for games and toys like four-in-a-row. I’ve used Approval testing in a BDD process on various business software applications too. The sweet spot seems to me to be on applications where the domain is visual. It matters what it looks like. Start with sketches in the example workshop and then design tests that check a relevant level of visual detail. Automate by building a printer and using Approvals to check the printout.
Approval testing tools are available for many programming languages. This four-in-a-row example is fun to play with and suits this technique. Why don’t you have a go at it coding it yourself?
Join Emily this November in her upcoming remote course Agile Development techniques: Approval Testing, to learn more about this topic.
About Emily Bache
Emily Bache is a Technical Agile Coach and long-term proponent of agile development practices like Test-Driven Development. She is a published author with Pluralsight and wrote „The Coding Dojo Handbook“ for which Emily is currently writing a follow up book called „Technical Agile Coaching“.