Design a site like this with
Get started

A chart for measuring the quality of “spinning plates”

Quality metrics can be like measuring the wobble on spinning plates. 

The engineering teams are merging to the main branch, code is being deployed and you need metrics to show if there are issues with this process that require your attention. The teams working are rather like spinning plates. The teams are working and management needs to know if something is going wrong that they need to address. You can see that the plates are spinning and wobbling. You need to know if the plate is wobbling too much and will fall. If it is going to fall then you need to intervene.

Walter Shewhart created control charts to solve this problem at Bell Telephones. He saw variations in how the telephone production line worked and wanted a method to show when he needed to intervene. He plotted the metric he wanted to measure on the chart and then used the standard deviation to calculate the Upper Control Limit and Lower Control Limits. If the curve went above the upper limit or below the lower limit he needed to intervene. If the chart remained between the upper and lower limits he did not need to intervene.

Control charts are now used in many industries and can help us in software engineering.

John Willis recently wrote in his blog about how he used control charts. He used them to track his teams’ deployments to help him understand whether the team was improving.

I had a friend who was testing the work of two teams. One team was releasing new features and moved from making monthly releases to continuous delivery. The other team made monthly releases and needed developers to work extra hours to fix bugs when a release was made. In order to find out the difference between the two teams my friend decided to use control charts to analyse the work the two teams were doing. He created control charts that showed the number of unit tests added by each team each week. He found that the team practising continuous deployment created a fairly consistent number of unit tests each week and the curve on its chart was between the upper and lower limits. The chart for the team making monthly releases was quite different. There were weeks when the team making monthly releases added unit tests, there were also weeks when the team deleted unit tests and there were weeks the number of unit tests did not change. The curve on their chart often went below the lower limit. The control charts showed that a difference between the two teams was that the team practising continuous delivery were writing unit tests every week and the team making monthly releases frequently were not writing unit tests. The charts helped my friend to understand what was happening; the team that was making monthly releases were not writing unit tests. The understanding gained from the control charts enabled my friend to start a conversation with management about the team making monthly releases. 

Control charts are a useful tool that can help us understand quality. If the ‘wobble’ on a spinning plate is within the lower and upper limits you do not intervene. You only need to intervene if the ‘wobble’ goes outside the lower and upper limits. 

Thank you to Krizia Cureg and Nick Smith for the conversation that led to this blog post.

Further reading:






Published by Mike Harris

Mike has been working in testing for 20 years and is currently the lone tester for Geckoboard. He has been a Test Lead and has also worked as a part of waterfall, lean and agile teams. He is also Programme Secretary of BCS SIGiST. Mike has a B.Sc.(HONS) from Middlesex University and is an Associate of the University of Hertfordshire. He has set up and led a Testing Community of Practice and been part of a successful agile transition. He is Co-Programme Chair of the British Computer Society’s Specialist Interest Group in Software Testing. He also contributed to the e-book Testing Stories and has had articles published by the Ministry of Testing. Follow Mike on Twitter: @TestAndAnalysis

Join the Conversation


Leave a comment

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s

%d bloggers like this: