Acceptance Criteria as Requirements and Tests

Requirements can be very problematic. Either they are too detailed, and cost a fortune to create and maintain (if they are maintained at all), or they are too vague, which causes confusion, both for developers and testers. One of the major problems I have faced is that requirement changes are not communicated properly, and the requirements artifacts are not updated continuously during the project life cycle.

As a tester I have many times faced the situation where I test something, find that it is not according to the requirements document, create a bug report, and then get it rejected because the feature was working according to (non-documented updated) requirements.

I think that one of the reasons this happens is because the developer that updated the design/implementation does not see any value in also updating the requirements document. He/she already knows what changes have been done, and taking the time to write this down so that a tester can receive this information is perceived to add less value than it costs. Which in many cases can be correct from the developer’s perspective.

The reason for this is that the requirements artifact only adds information value to the tester, once the change has been implemented, and thus should not be owned by the developer (unless the developer is also the tester). An artifact, which is owned by someone who does not see any value in it for himself/herself, will not be maintained properly.

So a few months ago I wrote a presentation about that you could use exploratory test missions as requirements [1]. A Test Mission is a test artifact that adds value to the tester, and will be maintained by the tester. Product owners and other stakeholders more easily understand it compared to just having test cases as requirements. Having detailed test cases as requirements solves the problem of making sure that the requirements artifact is valued by someone (the tester), but test cases can be very difficult to understand for stakeholders. I thought test missions as requirements was quite a good idea, but I didn’t explore it further.

Fast forward to last week. I was having a conversation with a fellow tester and he explained to me how they were working with tests. He showed me that they were using acceptance criteria [2] as both requirements and tests. This was very similar to my idea, but much more compelling and easier understood by stakeholders than my test missions.

So the basic idea is that the tester (or person who performs the tests) takes part in design meetings and creates acceptance criteria during the design phase, which are reviewed and acknowledged by designers and developers. These acceptance criteria are written in a way so that they can be easily understood by all stakeholders, but are also written so that they can be used directly as input to exploratory testing [3].

This means that the acceptance criteria are both requirements and tests, and they are owned by the person/team who owns the acceptance testing. This team is responsible for participating in the design meetings, writing down the acceptance criteria, get the acceptance criteria accepted by the stakeholders, make sure that they are continuously updated, and secure that these acceptance criteria are met before the feature/functionality is considered completed.

The key obviously is to write the acceptance criteria in a way so that they are usable as both requirements and tests. They should be easy enough for stakeholders to read and understand, but detailed enough so that they can be used as input to exploratory testing.

This means that the acceptance criteria need to cover all relevant/prioritized parts of the test space – a term which I covered in a previous article [4].

Apart from features and functions you will also need to create acceptance criteria for system capabilities, which may be outside of the scope of specific feature teams. For large systems there may also be other types of acceptance criteria for system behavior that is also out of scope for feature teams.

This is by no means an easy task, but I think that if it is done correctly there are large potential gains to be made. I will try to use this method in the future and evaluate how well it works.

/Johan Hoberg

References

[1] Test Missions as Requirements

http://www.slideshare.net/JohanHoberg/test-missions-as-requirements

[2] Agile Acceptance Criteria

http://blogs.versionone.com/agile_management/2013/11/14/agile-acceptance-criteria/

[3] Exploratory Testing

http://www.satisfice.com/articles/what_is_et.shtml

[4] Game Testing: Exploring the Test Space

http://www.gamasutra.com/blogs/JohanHoberg/20140821/223852/Game_Testing_Exploring_the_Test_Space.php