How do you decide which tests to automate?

An end-to-end test pack needs to run quickly so that it does not slow developers, and at the same time provide useful feedback to the developers.  This makes deciding which categories of tests to include in the test pack challenging.

A ten-minute video from Russ Ackoff has helped me better understand my decisions on one category of tests to automate and helps me explain my choices better.

Ackoff says that “the essential or defining properties of any system are properties of the whole which none of its parts have” [1]. He illustrates his point by referring to a car: “The essential characteristic of an automobile is that it can carry you from one place to another. No part of an automobile can do that. The wheel can’t, the motor can’t”[1]. He also uses human beings to illustrate his point: “You have certain characteristics the most important of which is life. None of your parts live. You have life. You can write, your hand can’t write….You can see, your eye can’t see.”.[1]

The application I test is a system like the systems Ackoff described. The application has “essential or defining properties” that none of its parts have. The application that I test shares information. One part of the application, such as the UI, can not share information. The application as a whole has the property of sharing information, not its parts. 

A category of tests that should have end-to-end tests are tests for the “essential or defining properties” of the application. This is a useful analysis because if I define the application’s “essential or defining properties ” there will only be a limited number of tests and they will be for the key parts of the application. It is also a useful analysis because I can share this analysis and work with my team to define the “essential or defining properties” of the application.

References:[1]  Beyond Quality Improvement Dr Russ Ackoff – a 10-minute video (1994) This presentation is from an event hosted to capture the Learning and Legacy of Dr. W. Edwards Deming by Clare Crawford-Mason and Lloyd Dobyns, the journalists who made “If Japan can..Why can’t we?”.