TestAndAnalysis

Three short stories about the Definition of Done

What does it mean when a card is put into the Done column? Done can mean that the coding is completed, it can also include that the code has been reviewed, that it includes unit tests and many other things. Teams sometimes create a “Definition of Done” to state exactly what “done” means and this can be a powerful way of improving quality.

A team I worked with that created a “Definition of Done” and did so to create a consensus across the engineering teams about what tasks need to be completed on a card before it could be put into the Done column. Team members put forward suggestions on changing the Definition when they felt it was appropriate, and what was included in the Definition was reviewed. This Definition of Done was part of a successful transition to improve quality and release features more quickly.

I heard about a company where the Definition of Done was negotiated as if it was a legal contract between management and the development teams. Once written the Definition was very difficult to update. The Definition was supposed to help improve quality. The status of the Definition as a type of contract showed the lack of trust that existed. The quality of the team’s software did not improve.

I also heard about a Team Lead whose team created a Definition of Done at a retro type meeting with all members of the team contributing to the discussion on what should be included. The discussion led to an in depth discussion of what needed to be completed before a card could be put into the Done column. This team worked together to deliver good quality new features quickly.

A Definition of Done can help teams deliver good quality software quickly. These stories show that using a Definition of Done can help a team improve quality if the Definition is agreed in a collaborative way and there is trust.

Further reading: Defining What Done Means

How can you work with your team to identify risks?

Identifying risks to their work helps development teams. Recently I spoke to a team who had talked through the ways in which their work could fail, and built on this to identify risks to their work.

The team adapted Failure Modes Effects Analysis (FMEA) to suit their needs. FMEA is a technique that is used in manufacturing to identify the modes of failure for a piece of work and calculate a number that represents the risk of each failure mode. A team using FMEA will, for each failure mode, assign a value for each of the following :

Severity (how severe is the failure)
Occurrence ( how likely the failure is to occur)
Detection (how likely it is that the failure will be detected)

And from these values calculate a value to represent the risk for each failure mode. This enables the team to identify the failure mode with the highest risk.

The team I heard about used a simplified version of FMEA. They found that identifying failure modes for their work would be useful, and that discussing severity, occurrence and detection for each failure mode helped them identify the risks to their project. They created a simplified spreadsheet, based on the one in the article below, with columns for: failure type, potential causes, potential impact and current process controls. However, assigning values and calculating risk for each failure mode felt to be more than was required for their project.

The team identified failure modes, entered them in the failure type column, and then completed the columns for each of the failure modes. The team were able to ensure that there were process controls in place for each failure mode, and identified an additional control to put in place. The team plans to revisit the process if they identify more failure modes for their work.

This simplified FMEA process gave the team confidence as it enabled the team to ensure that they had methods of reducing the risk of each failure mode of their work. The confidence the team gained enabled the team to move more quickly.

Further reading: https://www.isixsigma.com/tools-templates/fmea/fmea-quick-guide/

Quality is Free

Quality is Free is the name of a book by Philip B. Crosby in which he argues that quality is free and lays out a Zero Defects program to improve quality. Crosby’s ideas, along with Deming, Juran, and others, influenced the Quality Movement.

Crosby says that there is a cost of quality. The cost of quality comprises fourteen ingredients such as rework, warranty, engineering changes, and purchase order changes, and that these costs more than cover the costs of improving quality. He says that improving quality will reduce costs and so will increase profits without increasing sales.

The staff are not the cause of poor quality. Crosby argues that the staff are like a mirror and reflect the attitude of management towards quality.

Zero Defects is the name Crosby gives to his program to improve quality. Zero Defects is about preventing defects rather than fixing them and has the aim of getting work right the first time it is done. The promotional activities of the program such as giving every member of staff a badge saying “I’m for quality” have not aged well, but the Zero Defects program shows how a company can improve quality by removing the causes of defects. The program also includes teams setting their own goals for improving quality and that teams should work together on error cause removal.

A phrase used by Crosby in the book is “Quality is free, but requires effort”. He says this because, in the book, he shows that improving quality will cut costs and that effort is needed to improve quality.

Further reading:
A theory of management for improvement of quality vs a quality improvement plan, which helps us more?

How do you decide when to stop testing?

There is always more testing that can be done on a feature but there are reducing returns on testing a feature over time. At some point you need to stop testing, and the question as to when to stop is sometimes a difficult call. Also, sometimes there is more than one feature to be tested. Choosing when to stop testing the first feature and move to testing the second feature can be a difficult choice.

If I test a feature for a while I find that after a while I am finding less issues, and after a while longer I stop finding issues. At the point when you stop finding issues it makes sense to stop testing. Sometimes you don’t have the time to keep testing until you stop finding more issues and it makes sense to stop testing when you are finding only minor issues. In both situations I find that I find more issues at the beginning of testing than at the end and it is the slowing rate at which I find issues that prompts me to consider stopping testing.

That I find more issues, and more serious issues, early in testing than I do later is a pattern I have identified. The pattern is not unlike the Pareto Principle. Pareto was a nineteenth century Italian mathematician who observed that 80% and Italy’s land was owned by 20% of the population and found that this ratio applied in many other situations too. When I am testing it feels that 20% of the testing effort finds 80% of the issues.

One of the key principles of Joseph Juran, the American evangelist for quality, was to use the Pareto Principle. This meant identifying “the vital few and the trivial many” in order to focus on the key areas to improve quality. The Pareto Principle is the pattern that I observed in my testing where I found more issues earlier in my testing and less issues later in my testing. The “vital few” tests were those that I made earlier in my testing and the “trivial many” were the tests that I ran later and found less issues.

We need to know when to stop testing a feature and the Pareto Principle can help to decide when to stop. I stop testing when my testing ceases to find issues and I know that my testing has moved from the “vital few” tests to the “trivial many” tests.

Further Reading: https://www.juran.com/blog/a-guide-to-the-pareto-principle-80-20-rule-pareto-analysis/

The Quality Trilogy helps discussions with your team

We want to create quality features, products and services. Using a framework to think through how to achieve this is helpful as the framework ensures that vital points are covered. Testers can provide value by understanding these frameworks and using them to shape their own thinking in order to be able to prompt and/or lead discussions in their team that help to improve overall quality.

Joseph Juran was an American who helped Japan’s economy recover after World War 2 and later worked for companies such as Apple. He created The Quality Trilogy which is a framework that can be used for discussions with your team. The trilogy consists of:

Quality planning

This part of the trilogy can be described as designing the feature including the team understanding the customers needs. Quality planning helps me test because I want to understand the customer and for this reason I find it useful to have a discussion about what job the customer will be “hiring” the feature to do. It also helps to have a discussion at this early stage about what support for accessibility is appropriate for the new project.

Quality control

Tracking metrics and periodic checks are an important part of quality control because they tell us whether or not the feature we released is “working”. If there are no periodic checks in production that check that features are “working” then quality in production can fall without the company knowing. If quality in production falls this will affect customers who may leave or complain.

It is useful to discuss with the team how the new feature will be tested, including how we know the feature is working once it is deployed. Testing once the feature is deployed to production I think of this as “shift right” testing. As testers we need, not only, to “shift left” to identify issues before they become bugs and we also need to “shift right” to ensure that we know the feature continues to work after it is deployed. We need to ensure that the released product “is working” in production and continues to work in production. “Shift right” testing helps reduce the number of times a customer will surprise us by reporting bugs, reduces the time that bugs are live in production, enables us to plan fixes for bugs we find and will control quality by ensuring that the released feature “works”. If the discussion on testing does not happen then a new feature can be deployed to production that either does not work in production or starts to fail soon after it is deployed to production.

Quality improvement

Identifying areas where processes can be improved to improve quality is an important part of a team’s work. It is constructive to have a conversation about how the team could be improving quality, and it is also useful to have a conversation about what we can do to improve the quality of the new feature.

It is useful to have discussions within teams about quality and Juran’s Quality Trilogy gives a management framework in which we can have these discussions.

Thank you Jason Liggi for reviewing this blog entry

Further reading: Juran’s Quality Handbook

Testing Stories: Test Lead in an agile transformation

I am looking forward to speaking about my story in the ebook Testing Stories at the BCS SIGiST webinar on the 12th October because I want to show how Test Lead’s have an important role to play in making agile transformations successful. I will be speaking with three of the other testers who contributed stories to the ebook. We will all be speaking about the stories we contributed to the ebook. I am going to speak about the lessons learned from the transformation such as the importance of building relationships and that there is much to learn from change. During the transformation I found that the role of the tester can become more interesting and can contribute more value in an agile environment. Before the transformation the testers were in a separate test team which tested functionality to see if it “passed”. After the transformation the testers are part of a development team and work with their team to build quality into products. The testers role in an agile team is varied and I found that the role can be structured to enable the tester to gain and maintain a range of skills.

The premise of the ebook, Testing Stories, is that every tester has a story. Over 30 testers from around the world have each contributed a story to the book. All the stories are about testing and the subjects of the stories vary greatly from the story behind “Tester of the Day” to “My Journey Testing Internet networking Devices”. I am very pleased that my story “Test Lead in an agile transformation” was included in the book. The team that created the book was led by Melissa Fisher.

The book is available from lean publishing https://leanpub.com/testing_stories/ and the team decided that all proceeds of the book go to the mental health charity Open Sourcing Mental Illness (OSMI). The book has so far raised over $1000 for the charity.

Please buy the book,and please join me and the other authors, Andrea Jensen, Louise Woodhams and John McGee to hear our stories at the webinar. Book your place here: https://www.bcs.org/events/2021/october/webinar-testing-stories/

Added to the blog after the webinar:
You can now watch a video of the webinar: Webinar: Testing stories

Team Activities to Better Understand Your Customers and Team

We all need to understand our customers and the teams we are members of. Customers need to be understood as they keep the business alive. The team(s) we are members of need to be understood so that we can work with them to improve quality.

The best tool that I have found to understand customers is the Theory of Jobs to be Done. This is an innovation framework that asks what job did someone hire that product or service for? In “Competing against luck”[1] job is defined as “The progress that a person is trying to make in a particular circumstance”. Looking at the progress that a person is trying to make and the circumstances can help you understand your customers. It can help you and your team think of all the jobs that are done with the product or service that you are building. Jobs to be Done can be contrasted with personas. A persona describes the person, but does not give you the circumstance. I find the Theory of Jobs to be Done a useful way to focus on customers. and gave a presentation at SkillsMatter to share this.

In some ways teams are rather like families and I have recently found that the ideas of Victoria Satir are often referred to in books and articles I read. Katrina Clockie[2] and Lena Wiberg[3] both refer to the Satir Interaction Model. This model describes the process that happens internally when we communicate. Looking at the model can help us understand how our team communicates. Satir says that there are four steps:

Intake
Meaning
Significance
Response

Gerry Weinberg has pointed out that the first three steps all occur internally. It is useful to consider this in terms of remote communication. If we are with someone when we communicate with them we will see non visual signs of how they are reacting when the first three steps occur, but if we are remote we will only see their response. This is particularly important to consider when we are communicating bad news such as “I think I have just found a bug!” We won’t see the rest of the team’s non verbal communication while they are going through the first three steps, we will only see their response. It could be useful to discuss this model with your team to improve team communication and understanding.

I hope that these two ideas for activities are useful in helping you understand your customers and your team.

References:

[1] Competing Against Luck: The Story Of Innovation And Customer Choice by Clayton M. Christensen, Taddy Hall, Karen Dillon and David S. Duncan

[2] http://katrinatester.blogspot.com/2014/10/satir-interaction-model.html[3] Would Heu-Risk it? by Lena Wiberg

Introducing me

Hi, I have been working as a testing professional for twenty years and have been a member of a test team, a Test Lead and a lone tester. I have started this blog to share what I have learned and am learning about testing and quality.

I have automated tests with Playwright and carried out exploratory testing. I have also set up and led a Testing Community of Practice and been part of a successful agile transition.

I am Vice-Chair and Programme Secretary of the British Computer Society’s Specialist Interest Group in Software Testing. In this capacity, I help to organise software testing webinars and conferences.

I am one of the co-authors of the e-book “How Can I Test This?“. I also contributed to the e-book Testing Stories and have had articles published by the Ministry of Testing, LambdaTest, DZone and The QA Lead.

I have a B.Sc.(HONS) from Middlesex University and am an Associate of the University of Hertfordshire. I am also a Chartered Fellow of the British Computer Society.

Here is my story on (Extra)ordinary Tech Stories: Mike Harris: from sales to Solo Tester

Follow me:

On Bluesky: @testandanalysis.bsky.social
on Mastodon: @TestAndAnalysis@mastodonapp.uk

on LinkedIn: https://www.linkedin.com/in/mike-harris-citp-fbcs/