Improve quality by learning from your process data – a review of “Twenty Things You Need to Know” by Donald J. Wheeler

Image of cover of Twenty things You Need To Know

Donald Wheeler has over forty years of experience in the study, practice and collaboration in data analysis. In “Twenty Things You Need to Know”, he shares this experience in the form of questions and answers.

What are two mistakes in data analysis? Mistake one: “Interpreting the routine variation of noise as if it amounted to a signal of change in the underlying process, thereby sounding a false alarm” [1]. Mistake two: Thinking that a signal of change in the underlying process is merely the noise of routine variation and thereby missing a signal”[1].

“You need a technique that will:

  • Allow you to learn from your process data
  • Warn you when your process changes
  • Help you to identify the causes of those changes; and
  • Enable you to operate your process predictably and on target”[2]

Wheeler explains: think of a measurable product characteristic; list all the possible cause-and-effect relationships that have an impact upon the measured value of your characteristic; arrange these causes in descending order of impact.[p10] Shewhart called dominant uncontrolled causes Assignable Causes, and Deming called lesser uncontrolled causes Common Causes.[3]

Assignable Causes will be the source of most of the unexplained variation in the product, such as excess production costs and low quality.[4]

The Upper and Lower Natural Process Limits of the characteristic can be calculated from the measured characteristic’s average. “Whenever a point goes outside the Natural Process Limits, it identifies an…Assignable Cause of variation”[5]. Identifying these points on a Process Behaviour Chart provides a methodology for when to look for Assignable Causes. [6]

Process Behaviour chart of UK inflation

The above Process Behaviour Chart shows UK inflation as the UK came out of COVID as time series data, with an average running through it, an Upper Natural Process Limit above it and a Lower Natural Process Limit below it. Inflation goes above the Upper Natural Process Limit, and so has an Assignable Cause.

In Twenty Things You Need to Know, we learn about why process behaviour charts are created in the way they are, and that Wheeler tested process behaviour charts on 1143 probability models [7]. 

A friend of mine used the techniques in the book to understand the difference in the quality of output of two development teams by analysing the number of unit tests each team created each week. One team developed software with only a few bugs and created a similar number of unit tests each week. Its datapoints stayed within the Natural Process Limits. The second team’s software often contained bugs, and they created different numbers of unit tests each week. Their data points frequently went outside the Natural Process Limits. The process behaviour charts showed that the second team was not always creating unit tests as it developed code, which was an Assignable Cause of the bugs in their code. 

Twenty Things You Need to Know is a companion book to Understanding Variation The Key to Managing Chaos. 

I read this book with the Profound Book Club and would like to thank them for the insights I gained while reading it.

You can also use process behaviour charts to understand quality data by analysing your process data. Reading Understanding Variation The Key to Managing Chaos, and its companion book Twenty Things You Need to Know will help you do so. 

References

[1] Twenty Things You Need to Know by Donald J. Wheeler (2009, p2)

[2] Twenty Things You Need to Know by Donald J. Wheeler (2009, p17)

[3] Twenty Things You Need to Know by Donald J. Wheeler (2009, p14)

[4] Twenty Things You Need to Know by Donald J. Wheeler (2009, p15)

[5] Twenty Things You Need to Know by Donald J. Wheeler (2009, p20)

[6] Twenty Things You Need to Know by Donald J. Wheeler (2009, p21)

[7] Twenty Things You Need to Know by Donald J. Wheeler (2009, p36)

Additional Resources:

Donald J. Wheeler’s Wikipedia page

Articles by Donald J. Wheeler SPC Press

Donald J. Wheeler  QualityDigest

Control Charts Resources, including a spreadsheet template, from the American Association for Quality (process behaviour charts are also known as control charts)

A History of the Chart for Individual Values The ultimate in homogeneous subgroups by Donald J. Wheeler

DemingNEXT –  eLearning from the Deming Institute with two weeks free

Help for you on your systems thinking journey – A review of “Thinking in Systems: A Primer” by Donella H. Meadows

“Thinking in Systems: Primer” is a great resource that can help you whether you are beginning, or have already started your systems thinking journey. 

Meadows helps us understand what a system is. “A system is an interconnected set of elements that is coherently organised in a way that achieves something”[1]. “Once we see the relationship between structure and behaviour, we can begin to understand how systems work”[2]

She also helps us understand the elements of a system. “A stock is the foundation of any system. Stocks are the elements of the system that you can see, feel, count or measure at any given time” [p17]. “Stocks change over time through the actions of a flow[3]. “A feedback loop is formed when changes in a stock affect the flows in and out of that same stock”[4]. A balancing feedback loop stabilises the stock level [5]. A reinforcing feedback loop can be a vicious or virtuous circle because it amplifies and reinforces[6].

The book takes the reader on a visit to the “Systems Zoo”, which contains many examples of systems and draws lessons from them. Meadows also shows us where we can leverage a system.

Donella Meadows writes that we should “stay humble – stay a learner”[7]. That you should “expose your mental models to the light of day”[8], make them as rigorous as possible, test them against supported evidence and scuttle them if they are no longer supported. 

There are useful appendices to the book that include a glossary and a summary of systems principles.

“The systems thinking lens allows us to reclaim our intuition about systems and

  • Hone our abilities to understand parts
  • See interconnections
  • Ask “what if” questions about future behaviours and
  • Be creative and courageous about system design” [9]

To help my learning, I used the book to create systems diagrams of the systems for managing bugs that I am aware of. 

The first diagram shows a system in which managers encourage the reporting of bugs because they are dissatisfied with the quality of the product, and at other times, they discourage the reporting of bugs by stating that no bugs will be fixed this quarter, so that work is focused on a new feature. The changing perceived value of submitting a bug report creates a reinforcing loop that amplifies changes in the numbers of reported bugs. 

This system diagram enables me to see the faults in this system, understand the Stock of Bugs, to see its interconnection with the perceived value of submitting a bug report, to ask “what if” there was stable feedback about bugs to the development team, and to think about how to design a system where the development team did not have to manage the Stock of Bugs.

The second diagram shows a company which asked its developers to fix ten bugs each iteration, which is shown as a balancing loop that stablises the Stock of Ten Bugs. 

This system diagram enables me to understand a small Stock of Bugs as an input into development team planning, to see its connection with software failures, to ask “what if” we reduced the number of software failures, and to think about what the Stock of Ten Bugs represents. 

I read Thinking in Systems with the Profound Book Club, and would like to thank the members of the club for the useful discussions that we had.

Thinking in Systems is, as its title says, “a primer”; it is a great resource that will help you on your systems thinking journey. 

References

[1] Thinking in Systems: A Primer by Donella Meadows (2008, p. 11)

[2] Thinking in Systems: A Primer by Donella Meadows (2008, p. 1)

[3] Thinking in Systems: A Primer by Donella Meadows (2008, p. 18)

[4] Thinking in Systems: A Primer by Donella Meadows (2008, p. 25)

[5] Thinking in Systems: A Primer by Donella Meadows (2008, p. 27)

[6] Thinking in Systems: A Primer by Donella Meadows (2008, p. 31)

[7] Thinking in Systems: A Primer by Donella Meadows (2008, p. 180)

[8] Thinking in Systems: A Primer by Donella Meadows (2008, p. 172)

[9] Thinking in Systems: A Primer by Donella Meadows (2008, p. 6)

Additional Links

Quality comes first – A review of “Deming’s Road to Continual Improvement” by William W. Scherkenbach

William Sherkenbach was Corporate Director of Total Quality Planning and Statistical Methods at Ford Motors and Group Director Process Improvement at General Motors. He wrote that “both of these great companies are better because their journey included Dr. W. Edwards Deming”[1].  In this book, Sherkenbach explains “how to operationalize the Deming philosophy in business, government and academia”.[2]

Ford’s first guiding principle is that “Quality comes first”[3]. When writing the Mission, Values and Guiding Principles of Ford, Sherkenbach wrote that “we used terminology already used by Ford people: Quality is Job 1”[4]. 

Everything we “do can be described in terms of a process”[5], and there are opportunities to improve processes in all aspects of business. Sherkenbach coined the term Voice of the Process, which “is the result the process gives you” [6].

“Dr. Deming tells us that the customer is the most important part of the production line”[7]. “The Voice of the Customer communicates .. the wants and needs of your customers”[7]. “You must have a process to operationally define the Voice of the Customer” [8]. 

Often, the Voice of the Customer and the Voice of the Process do not match. This is an opportunity for improvement that Sherkenbach calls “the Gap”[9].

Integral to the Method of Continual Improvement is the Plan-Do-Study-Act (PDSA) Cycle[10].

“The quality profession itself is in need of major improvement. Everyone must be educated… to use the PDSA improvement cycle.”[11]. Sherkenbach operationally defines the PDSA Cycle as having eight steps:

“Plan

  1. Identify the opportunity for improvement
  2. Document the present process
  3. Create a vision of the improved process
  4. Define the scope of the improved process

Do

  1. Pilot the proposed changes on a small scale, with customers over time

Study

  1. Observe what you have learned about the improvement of the process

Act

  1. Operationalize the new mix of resources
  2. Repeat the Steps(Cycle) on the next opportunity”  [12].

I read this book with the Profound Book Club. I would like to thank the members of the book club for their discussions, Rob Park for organising the book club, and Cathy McGrath and Bill Sherkenbach for joining our discussions. Bill helped me better understand a mistake I made when introducing CI to a development team, and how to better deal with a similar situation in the future.

We can learn a great deal from William Sherkenbach about how to improve. I recommend reading this exceptionally well-written book and exploring the links below.

References

[1] Deming’s Road to Continual Improvement by William W. Sherkenbach (1991, p xii)

[2] Deming’s Road to Continual Improvement by William W. Sherkenbach (1991, p xi)

[3] Deming’s Road to Continual Improvement by William W. Sherkenbach (1991, p 131)

[4] Deming’s Road to Continual Improvement by William W. Sherkenbach (1991, p 132)

[5] Deming’s Road to Continual Improvement by William W. Sherkenbach (1991, p 5)

[6] Deming’s Road to Continual Improvement by William W. Sherkenbach (1991, p 14)

[7] Deming’s Road to Continual Improvement by William W. Sherkenbach (1991, p 12)

[8] Deming’s Road to Continual Improvement by William W. Sherkenbach (1991, p 157)

[9] Deming’s Road to Continual Improvement by William W. Sherkenbach (1991, p 164)

[10] Deming’s Road to Continual Improvement by William W. Sherkenbach (1991, p 162)

[11] Deming’s Road to Continual Improvement by William W. Sherkenbach (1991, p 159)

[12] Deming’s Road to Continual Improvement by William W. Sherkenbach (1991, p 63)

Additional Links

Collaborative customer-focused systems thinking – a review of Sys-Tao: Western Logic – Eastern Flow by Bob Browne

In this book, Bob Browne recounts how he drew on ideas from both the East and the West, including the philosophies of W. Edwards Deming and Eli Goldratt, to make the Great Plains Coca-Cola Bottling Company a success.

In 1980, Bob Browne, with the help of a few others, acquired the Great Plains Coca-Cola Bottling Company in a leveraged buyout. 

Initially, he was getting “good results” by doing the things he had been trained to do. It seemed that was all that mattered. It, however led to a crisis and there was unrest in the company. The “way out of the crisis represents a transformation in .. leadership philosophies”[1].

He used a synthesis of two world views to lead the company; he calls this sys-tao. Sys represents the traditional Western ways of thinking, and tao is symbolic of methods rooted in Eastern traditions. “The works of W. Edwards Deming, the American who taught the Japanese about quality, heavily influenced Sys-Tao.”[2]

Browne saw that “the left side of our brain is more narrowly focused on what we know is important”[3] and that “the right side of our brain is more broadly vigilant, and is open to relationships”[3]. 

Deming organised his thinking into four broad categories, one of which was systems thinking[3].   Systems thinking can be “a mapping exercise to better visualise big-picture ideas”[4]. This approach can be left-brained, mechanistic thinking.  

“Living systems are open, self-organising and nested”[4]. “Lifelike organisations seem to emerge from the bottom up”[4], but “top-down leadership philosophies…are not at all lifelike”[4]. “Humans do not naturally behave in predetermined and mechanistic ways” [4]. “Human organisations must be far more interdependent” [4]. “Interdependent relationships require a more collaborative approach to systems thinking”[4]. Systems thinking takes us to where “our vision becomes shared and it emerges into something bigger than ourselves.”[4]

The biggest single lesson Browne learned, he described as “Collaborative Customer-Focused Systems Thinking” [5].

When the business was sold in 2011, Browne and the other investors had made an average rate of return of 13% for each of the 32 years that they owned the company.

I would like to thank the members of the Profound Book Club for their informative discussions while we were reading this book, and Rob Park for organising the book club.

Sys-tao is the story of a business learning and through that succeeding. There is so much we can learn from this story. There are links below that will enable you to explore sys-tao further.

References

[1] Sys-Tao: Western Logic – Eastern Flow by Bob Browne (2015, Chapter 2)

[2] Sys-Tao: Western Logic – Eastern Flow by Bob Browne (2015, Preface)

[3] Sys-Tao: Western Logic – Eastern Flow by Bob Browne (2015, Chapter 3)

[4] Sys-Tao: Western Logic – Eastern Flow by Bob Browne (2015, Chapter 21)

[5] Sys-Tao: Western Logic – Eastern Flow by Bob Browne (2015, Some Concluding Thoughts)

Resources:

Sys-tao links 

Systems thinking and living systems 

Bob BrowneSys-Tao: Western Logic – Eastern Flow by Bob Browne. Get the book!

We can learn so much from how AI got here – A review of Rebels of Reason by John Willis with Derek Lewis

Much like the aliens in the sci-fi movie Arrival, AI has now arrived. Rebels of Reason tells us how AI arrived, and in doing so, provides us with a deeper understanding of AI. “We learn how foundational ideas have been refined over time, informing the rapid technological advances of our era”[1].

AI has been developed through a long history of technological innovation by people who questioned the frameworks they inherited. They were “rebels of reason”. Charles Babbage and Ada Lovelace built on the work of earlier rebels, such as Blaise Pascal and George Boole. Their work enabled later ‘rebels of reason’ such as the Polish cryptologists who broke the Enigma codes, Alan Turing, Claude Shannon, Fei-Fei Li, the creators of ChatGPT and many others. 

“We have to stop using ourselves as the benchmark for thinking and realise we have created an alien intelligence”[1]. AI is a new form of intelligence. Instead of being frustrated when an AI hallucinates, we need to learn the language of AIs.  

Testing professionals need to develop an understanding of AI that will help us to test applications that use AI and utilise tools that incorporate AI. Reading Rebels of Reason enabled me to appreciate how useful it is to have an understanding of the history of AI, ”because it reveals the evolutionary path from early rule-based systems to the advanced models (often beyond human comprehension) we see today.”[1] I have learned about past successes and setbacks, which have helped me gain a better understanding of the rapid changes that we are living through.

Rebels of Reason was published yesterday. I read pre-publication versions of Rebels of Reason with John Willis as a member of the Profound Book Club. I would like to thank John Willis and the members of the Profound Book Club, from whom I learned a great deal while discussing the book.

Testing professionals will gain from reading Rebels of Reason. AI is here now. We all need to develop a deeper understanding of what AI is. Reading this book provided me with insights into how AI evolved, and through this, I gained a more profound understanding of what AI is.

References

[1] Rebels of Reason by John Willis with Derek Lewis (2025, Epilogue)

Further reading

Profound – Having intellectual depth and insight – John Willis’ blog

Buy the book:

Rebels of Reason: The Long Road from Aristotle to ChatGPT and AI’s Heroes Who Kept the Faith by John Willis with Derek Lewis

Is the job market for testers over a century out of date?

Testers are building their careers in a job market that many experience as difficult. Our ancestors from the early years of the twentieth century would have recognised today’s job market. 

One of my relatives has researched the family tree. He found that when he researched individuals, they had different jobs each time they appeared in documentation. They changed jobs often, so, like many jobs today,  their jobs were probably insecure. 

However, there was a constant at the beginning of the twentieth century, just as there is today. My ancestors were always working with horses. They would, for example, be ostlers, grooms, jockeys, farriers or managing teams of horses. Today, there is also a constant: our LinkedIn profiles list many different job titles that relate to testing. Our skill is with testing; my ancestors’ skill was with horses.

The world has changed in the last hundred years, but our early twentieth-century ancestors would recognise today’s job market for testers, in which people have a core skill, change jobs regularly, and many jobs are insecure.

Today’s job market has so much in common with the job market of over one hundred years ago, which is sad. It makes me wonder if there is a better way.

Toyota’s model offers a successful alternative. Toyota rose from the ruins of post World War Two Japan to be the largest car manufacturer in the world. It recruits people at 18 or 22 years and has a 30-year plan for them[1]. Toyota’s model shows that there is a better way, and that way is successful.

My ancestors from over one hundred years ago would recognise today’s job market for testers. We can learn from Toyota and, in doing so, improve work.

References

[1] Lean Has Failed (or Has It?) with James Womack – Katie Anderson’s podcast (19 February 2025)

Further reading about Toyota:

A review of reviewing “Rebels of Reason – The Long Road from Aristotle to ChatGPT & AI’s Heroes Who Kept the Faith” by John Willis with Derek Lewis

As a member of the Profound Book Club, I feel privileged to be reviewing a pre-publication “galley copy” of John Willis’s latest book, Rebels of Reason. John Willis is one of the founders of the DevOps movement and co-author of The DevOps Handbook and Beyond the Phoenix Project.

Rebels of Reason “invites us to rethink our narrative of innovation” and John has published a sample of the Prolog to the book: First Look at Rebels of Reason. The book is full of great stories related to the history of AI.

I have learned so much while reviewing the book. I have learned from reading Rebels of Reason, from John Willis who has shared insights from his research for the book and career, from other members of the Profound Book Club, and from the discussions that we have had. 

I can’t comment on the contents of the book because it is pre-publication, except to say that it is well worth reading and that I am enjoying it!

We all need to know more about AI, and I recommend reading Rebels of Reason when it is published. To get on a waiting list for the book, sign up for John Willis’ DearCIO Newsletter. 

Thank you, John Willis, for another informative and useful book.

Learning from winners

When we are testing it helps to be aware of the perspectives of people who work in other fields within IT. I attended an audience with the 2024 British Computer Society Lovelace and Society Medal winners. It was great to hear talks from the medal winners and to network at the event. Listening to the medal winners’ presentations about their work enabled me to learn from their perspectives.

Professor Tom Crick MBE hosted the event and facilitated interesting conversations with the medal winners.

2024 BCS Society Medal Winners

  • Karl Flinders – Journalist, Computer Weekly

He spoke about his work as a journalist uncovering the Post Office Scandal, and said that people who work in IT should speak out when they know something is wrong.

  • Anne Marie Imafidon MBE – Founder and CEO, Stemettes

She spoke about the need for diversity and how Stemettes works to inspire girls, young women and non-binary people to work in STEM. She talked about what it was like to be the only woman and/or black person in the room. She also said that including the arts in STEM can be useful, making it STEAM. I now want to read her book She’s In CTRL.

2024 BCS Lovelace Medal Winners

  • Professor Aggelos Kiayias

He spoke about his work on Blockchain including a protocol that reduces its energy use.

  • Professor Philippa Gardner

She spoke about verification. I spoke to her after the presentation when she introduced me to Symbolic Execution, a form of testing,  and Hoare Logic, which can be used for validating software.

  • Dr Sue Sentance

She developed the PRIMM approach to teaching programming and spoke about how it has been introduced to schools around the world. She said it was best to be able to read code before learning to write it.

After the presentations, I networked with lots of great people. This was a great event because it celebrated the medal winners’ success and gave insights into different disciplines within IT. I learned a lot, including new ways to test software and the need for diversity in IT, and I met some great people.

Further reading:

The BCS Lovelace Medal

The BCS Society Medal

Dear Software Testing

This weekend, I saw a play that showed a manager driving out fear to lead his team to success. The play was ‘Dear England’ and dramatised Gareth Southgate’s term as manager of the England men’s soccer team.

The play shows Southgate using psychology. He hired the psychologist Dr Pippa Grange to be Head of People and Team Development. She was clear that she would not be doing performance psychology. Southgate wanted her to help the whole team. Dr Pippa Grange is shown working to remove fear from the team. 

Southgate also saw England’s soccer teams as a system. In the play, the England men’s and women’s teams work together as ‘One England’, and Southgate tells the players that ‘One England’ also includes the catering staff.

In the play, Southgate identifies taking penalties as a key weakness for the England team. Southgate, the team coaches and Dr Pippa Grange developed a theory to improve England’s ability to take penalties. Knowledge is built on theory[1]. The theory of how to take penalties was tested in the 2018 World Cup when England beat Colombia on penalties. The team learned that their theory of how to take penalties was a successful way to take penalties.

Southgate is a leader, he is a “coach and counsel, not a judge”[2].

In ‘Dear England’, we see psychology, systems thinking and the theory of knowledge transform the England men’s soccer team. We can use psychology, systems thinking and the theory of knowledge to transform our teams. This play is great management training.

Psychology, systems thinking and the theory of knowledge are three parts of Deming’s System of Profound Knowledge, which Deming used to rebuild the Japanese economy and help Ford become the most profitable car manufacturer in America.

Dear Software Testing, go to see ‘Dear England’ because it shows the use of techniques that will help you become a leader in test and quality(and the play will also make you laugh, smile and feel good!)

References

[1] The New Economics by W. Edwards Deming (1994, p102)

[2] The New Economics by W. Edwards Deming (1994, p126)

Further Reading

Exploring Systems Thinking

Systems thinking is an important part of a tester’s skill set. I really enjoyed discussing systems thinking on the Ministry of Testing’s Testing Planet with Simon Tomes, Sarah Byng and Rachel Kibler. It was great to learn from everyone on the broadcast. Thank you, Ministry of Testing, for inviting me to talk on Testing Planet about what I have learned on my systems thinking journey.

Please click on this link to listen to the broadcast: Exploring Systems Thinking – The Testing Planet News – Episode 09

Thank you software testing.

I have been a software testing professional since 1998 and am now changing my employment type on LinkedIn to “Mostly retired”. I saw that Dot Graham had set her employment type to “Mostly retired”, and it feels appropriate for me, too. I have retired, mostly. 

Software testing has been my second career. It has been a good career. I have met great people, learned a great deal and had fun. I would like to thank so many people:

  • Middlesex University for enabling me to make a career change and work in IT. 
  • Kalido, who gave me a start and gave me responsibility. 
  • ActiveStandards where I learned so much, we had success and so much fun. Thank you, Simon Lande and the management team.
  • CrownPeak, who made me the Global Test Lead.
  • Geckoboard for improving my test automation skills. Thank you to Paul Joyce for the way you led the company through COVID-19.
  • Melissa Fisher, for encouraging me to write a blog and for project managing the eBook ‘Testing Stories’.
  • Nicola Lindgren, for the idea for the eBook “How Can I Test This?” and for making it happen.
  • BCS SIGiST, with whom I enjoy organising events and conferences to share knowledge of testing. 
  • The Profound Book Club, from whom I have learned a great deal. Thank you, John Willis, for starting and Rob Park for organising the Profound Book Club.
  • The Testing Community, who have given me support, ran conferences at which I have spoken, listened and learned and from whom I have learned so much. Thank you to the Ministry of Testing, SkillsMatters, Indie Testing Community, Tester Hangout, BrowserStack, LambdaTest, Unicom, Geekle, QA Tech Talks, QA Evolution and others.
  • The British Computer Society for its work “making IT good for society”.
  • Everyone who has shared my testing and quality blog. 
  • Anyone I have forgotten to name, sorry and thank you.

I am still Vice-Chair and Programme Secretary of the BCS Specialist Interest Group in Software Testing and am helping to organise the 2025 SIGiST Conference. I hope to see you at the conference!

We need cooperation between testers and developers

The first web server at CERN

Sometimes there can be competition between testers and developers. Cooperation between testers and developers is better for the company and the customer. 

“A system must have an aim”[1]. A company is a system and so has an aim.  Testers and developers are components of the company’s system and so share the company’s aim. “The obligations of a component are to contribute its best to the system”[2].

“Competition leads to loss”[3]. If testers and developers are competitive towards each other, they will damage the system they are part of. Examples of damaging competition are: testers offending developers by celebrating each bug they find and developers damaging their relationship with testers by talking to testers as if testers have no technical skills.

“Every example of cooperation is one of benefits and gains to them that cooperate”[2]. There are many great examples of cooperation:

  • In an orchestra “the conductor, as manager, begets cooperation between the players, as a system”[4]. 
  • CERN (European Council for Nuclear Research) is an example of international cooperation. It has made many positive breakthroughs, including the first web server (shown above), which was “invented to allow an ever-increasing number of scientists to share information”[5]. Sharing information is a form of cooperation.

Testers and developers can cooperate in many ways including defining work, testing early in the development process, pair programming to develop automated tests, sharing knowledge of testing techniques, learning from failure, and many other ways. 

”What we need is cooperation”[3]. The company and the customer benefit when testers and developers cooperate because cooperation makes it easier to create quality software. 

Further Reading

References

[1] The New Economics by W. Edwards Deming (1994, p50)

[2] The New Economics by W. Edwards Deming (1994, p97)

[3] The New Economics by W. Edwards Deming (1994, p88)

[4] The New Economics by W. Edwards Deming (1994, p96)

[5] Advancing the frontiers of technology CERN

There are advantages to using the API to tear down automated end-to-end tests

When I automate a test I create a clean starting position for each test so that the automated tests can run concurrently. “The establishment of a known-good starting position for the test before it is run, and its re-establishment at its conclusion, is vital to avoid cross-test dependencies”[1]. 

Re-establishing the start position of the test is often referred to as “tear down”, and can be done by interacting with the UI using a testing tool or framework. Playwright has AfterAll() and AfterEach() that can be used for tear down. 

If you want to reduce the time it takes for tests to run, you can use a Lean metaphor to approach the problem of how to save time on the tests. The metaphor describes a process as a river that you want to run smoothly and looks for the largest rock that slows the river. Teardown occurs on every test so teardown can be the large rock slowing the tests.

When using an automated end-to-end testing framework, it is tempting to interact with the UI to tear down at the end of a test. However, clicking through the UI to tear down can be brittle and take a long time.

Tear down does not need to run via the UI because it is not part of the end-to-end test. To reduce the time it takes for tests to run, you can use the API of the application being tested instead of interacting with the UI to tear down.  The API will run quicker than interacting with the UI, so it will save time on every test.

Using the API of the application under test for tear down has additional the advantage that it will be less brittle because it is unaffected by changes to the UI. 

Tearing down the tests via the API also enables the test automation engineer to learn part of the API of the application they are testing.

There are several advantages to using the API of the application under test to tear down, it is quicker, less brittle and the engineer learns how to use the API.

References:

[1] Continuous Delivery by Jez Humble and David Farley (2011, p337)

Testers should not be defensive – A review of “Teaching Smart People How to Learn” by Chris Argyris

Testers often make suggestions about preventing bugs and sometimes find that these suggestions are rejected. Defensive reasoning can be a cause of their suggestions being rejected. Chris Argyris explains how defensive reasoning can prevent a company from learning and then explains how a company can overcome its defensiveness. I discovered Chris Argyris’s work through the Profound Book Club

Suggestions made by testers about preventing bugs are about learning because the tester has had to learn to make the suggestion. Also, both the team and the tester can learn by trying the suggestion.

Business success depends on learning[1]. Learning is not about motivation. Effective learning is about how you think[2]. Defensive reasoning brings learning to a halt [3].

 People act differently from how they espouse. Everyone develops a ‘theory of action’ a set of rules that they use to design and implement their behaviour[4]. However, most people’s behaviour, their theory-in-use, differs from their theory of action. There is a universal human tendency to behave according to four basic values:

  • To remain in unilateral control
  • To maximise ‘winning’ and minimise ‘losing’
  • To suppress negative feelings
  • To be as rational as possible

The values avoid embarrassment or threat, feeling vulnerable or incompetent, and lead to defensive behaviour and short circuits learning[5].

Sometimes, the rejection a tester experiences when their suggestion is rejected feels aggressive, but it is defensive because the rejection comes from the four points above.  

People can be taught to recognise their reasoning when designing and implementing actions[5]. Once a company has embarked on this type of learning they will discover the kind of reasoning that overcomes defensiveness. Organisations should learn to reason productively, and when they do they will discover the kind of reasoning necessary to reduce and overcome organisational defensiveness[6]. The first step is for managers to examine their theory-in-use critically. When managers do this they are learning how to learn[7]. One way this can be done is by connecting reasoning to a business problem, for example with a case study.[8] 

Reading this concise book is helpful for testing professionals. If testers can see the reasons for the defensive behaviour blocking their suggestions they can avoid becoming defensive and work on helping the company to reason productively. 

References:

[1] Teaching Smart People How to Learn by Chris Argyris. (2008, p1 )

[2] Teaching Smart People How to Learn by Chris Argyris. (2008, p5 )

[3] Teaching Smart People How to Learn by Chris Argyris. (2008, p21 )

[4] Teaching Smart People How to Learn by Chris Argyris. (2008, p23 )

[5] Teaching Smart People How to Learn by Chris Argyris. (2008, p25 )

[6] Teaching Smart People How to Learn by Chris Argyris. (2008, p44 )

[7] Teaching Smart People How to Learn by Chris Argyris. (2008, p64 )

[8] Teaching Smart People How to Learn by Chris Argyris. (2008, p47 )

Teaching Smart People How to Learn is available as a book and as an article. I have used page references from the book:

 Should we add new features or improve the usability and discoverability of existing features?

Some years ago, there was a disagreement within the company where I was working. The Test Manager and Customer Support Manager wanted to focus on learning about customers’ problems and solving them by improving the discoverability and usability of features. In contrast, the Product Owner wanted to engage customers by creating new features. 

I was recently introduced to a Thinking Process that would have helped us resolve the disagreement by reading Deming and Goldratt by Dr. Domenico Lepore and Oded Cohen with the Profound Book Club. I created these diagrams to help me understand the process described in the book. 

The Thinking Process tools visualise the cause-effect relationships and are part of Goldratt’s Theory of Constraints.

The first step is to describe the disagreement in a Core Problem Cloud. The view of the Product Owner is that to have A we must have B and, to have B we must have D. From the point of view of the Test Manager and Customer Support Manager to have A we must have C. The example Core Problem Cloud below describes the disagreement:

“The Core Problem Cloud is sufficient, in many cases, to get consensus on the problem”[1] If this is not the case you can construct a full Current Reality Tree.

Below is a Current Reality Tree that describes the disagreement. From the view of the Test Manager and Customer Support Manager D contains the Undesirable Effects and D1 represents the corresponding desirable effect.  From the view of the Product Owner D1 contains the Undesirable Effects and D represents the corresponding desirable effect. A is the common goal.

A Future Reality Tree can be created to design and control the implementation of a solution to the disagreement. The starting point for the creation of the Future Reality Tree is the Core Problem Cloud[2]. D* leads to the achievement of both B and C. The Future Reality Tree resolves the disagreement by learning about customers.

I learned a great deal from creating the diagrams. I went through quite a few iterations of each diagram. Whenever I could understand the problem better I updated one of the diagrams. Visualising the issues helped me understand them. I also felt that if I had worked collaboratively on the diagrams I would have gained a better understanding of the disagreement and learned how to resolve it.

If you have a disagreement in your team creating these diagrams will help you better understand the system and learn how to resolve the problem. 

I hope this high-level overview of Goldratt’s Thinking Processes encourages you to explore further.

I’d also like to thank Dr. Domenico Lepore for joining a discussion at the Profound Book Club and Rob Park for organising the book club.

References:

[1] Deming and Goldratt by Dr. Domenico Lepore and Oded Cohen (1999, p123)

[2] Deming and Goldratt by Dr. Domenico Lepore and Oded Cohen (1999, p134)

Additional Resources:

Podcast:

Profound – Dr Deming – S2 E8 – Domenico Lepore – Deming and Goldratt

Article

What Are the Thinking Processes?

Design a site like this with WordPress.com
Get started