Help to move from test execution to decision-making – A Review of Leadership is Language by L. David Marquet

As testers, we know the way we communicate matters. We can learn from how David Marquet used language to improve his submarine’s performance.

Marquet was a submarine captain in the US Navy, and now advises businesses. When he was put in charge of the nuclear-powered submarine USS Santa Fe, it was the “laughing stock of the fleet”[1]. Marquet had never captained a nuclear-powered submarine. He changed his method of leadership. He stopped giving orders; instead, he “provided intent”[2].  The USS Santa Fe rose “from worst to first”[4] in the fleet.

“Changing the way we communicated changed the culture”[4]. His team replaced the language of 

  • compliance with language of intent
  • “Prove and perform” with “improve and learn”
  • invulnerability and certainty with vulnerability and curiosity.

There are two different kinds of work: decision making(thinking) and execution(doing). “They require distinctly different mental processes and two different kinds of language.”[5] Marquet calls decision making “bluework” and calls execution “redwork”. You can find yourself stuck doing redwork without reflection. You can also find yourself stuck in bluework in perpetual reflection. Finding the right balance between “redwork” and “bluework” requires deliberately shifting from one to another.[5]

“Bluework and redwork exist in every organisation today, but red workers and blue workers do not have to”. This divide comes from an artificial construct invented during the industrial revolution. We have cultural labels that show whether you are a leader or a follower. [6]. I once had an allocated car parking space when others did not. 

The old style comes “from an older paradigm of leadership – the Industrial Revolution” [7], “exemplified by Frederick Winslow Taylor’s” philosophy [8]. After World War Two, “instigated by statistician W. Edwards Deming, leaders started to move from simply observing workers to asking them what they thought”[9].

Marquet found that the Oakland Raiders NFL team had a playbook that contained “plays” and he uses “plays” to describe how to use language. 

One “play” is to control the clock; this is a way to move from redwork to bluework. An example of a way to do this is if the leader asks, “How ready are we to shift to production? There is not a binary yes or no answer. This lets the team signal discomfort and opens the door to controlling the clock by pausing. A pause controls the clock.  It also helps if the team gives the pause a name; for example, the pause can be called “time out”.[10]. A team can also preplan the next pause. 

Another “play” is collaboration. There is an example in the book of you being the lone voice in a difficult situation. In the example, the person makes an observation and tries to avoid judgement. This is collaborative and invites the team to pause to consider their observation. [11]

There are seven sins in questioning; for example, asking “Why would you want to do that?” can make people defensive because you suggest “that” is a bad idea. It is better to ask “Tell me more about that”, “What is behind your decision? ” or ” How do you see the issue?” [12]

Testers can find themselves stuck doing only redwork, but can make a contribution to bluework. When you move from being a tester to being a test lead or a test manager, you move from mostly “redwork” to “bluework”. If a tester moves into quality engineering, they will be doing bluework. The techniques in this book helps testing professionals to make those changes.

I read this book with The Profound Book Club. I would like to thank members of the club for great discussions and Rob Park for organising the club.

Additional Resources

DavidMarquet.com

Leadership on a Submarine – a ten-minute video

References

[1] Leadership Is Language: The Hidden Power of What You Say — and What You Don’t by L. David Marquet (2020, p. 3)

[2] Leadership Is Language: The Hidden Power of What You Say — and What You Don’t by L. David Marquet (2020, p. 4)

[3] Leadership Is Language: The Hidden Power of What You Say — and What You Don’t by L. David Marquet (2020, p. 5)

[4] Leadership Is Language: The Hidden Power of What You Say — and What You Don’t by L. David Marquet (2020, p. 6)

[5] Leadership Is Language: The Hidden Power of What You Say — and What You Don’t by L. David Marquet (2020, p. 43)

[6] Leadership Is Language: The Hidden Power of What You Say — and What You Don’t by L. David Marquet (2020, p. 49)

[7] Leadership Is Language: The Hidden Power of What You Say — and What You Don’t by L. David Marquet (2020, p. 10)

[8] Leadership Is Language: The Hidden Power of What You Say — and What You Don’t by L. David Marquet (2020, p. 47)

[9] Leadership Is Language: The Hidden Power of What You Say — and What You Don’t by L. David Marquet (2020, p. 51)

[10] Leadership Is Language: The Hidden Power of What You Say — and What You Don’t by L. David Marquet (2020, p. 91)

[11] Leadership Is Language: The Hidden Power of What You Say — and What You Don’t by L. David Marquet (2020, p. 260)

[12] Leadership Is Language: The Hidden Power of What You Say — and What You Don’t by L. David Marquet (2020, p. 120)

Creative teamwork for quality – A Review of “Sidewinder – Creative Missile Development at China Lake” by Ron Westrum

The development of the Sidewinder missile is a successful example of how to run a creative research and development team. Like the author of this book, Ron Westrum, I don’t care about guided missiles, and, like him, I found this story irresistible[1].  It is intriguing that the methods used to develop the Sidewinder missile echo aspects of W. Edwards Deming’s philosophy. 

Bill McLean began working on missile development at China Lake during World War II and later became its director. Sidewinder started as an informal “skunk works” project[2] when China Lake was a testing station[3]. It was a weapon no one had asked for [4].  For a while, they could not get funding for missile development[5], so they obtained funding for “an intelligent fuse”.  The Sidewinder entered production in 1955[6] and remains in service today. Without the Sidewinder missile, more American pilots would have been lost[7].

The methods used by the team that developed the Sidewinder missile provide powerful examples of how to create a high-quality product.

Deming was not involved in missile development at China Lake, but many of the team’s ways of working there were in line with Deming’s philosophy. 

Led by Bill McLean, the team at China Lake relentlessly pursued the mission of developing a simple, low-cost guided missile.  This is consistent with the first of Deming’s 14 Points for Management: to have constancy of purpose toward improvement of product or service.

Both McLean and Deming worked to create psychological safety. McLean’s philosophy included “provide an environment where unfettered inquiry can take place”[8]. Deming’s philosophy includes driving out fear. 

At China Lake, they believed that a designer “must have more direct contact with this consumer than can ever be provided by written specifications[17]. One of China Lake’s most valuable characteristics was visiting the customer to ensure that weaponeers and users understood each other[9]. This is consistent with Deming, who wrote that “the consumer is the most important part of the production line”[10]. He also wrote that ”good quality and the right uniformity have no meaning except with reference to the consumer’s demands.”[11]

The China Lake team worked to get rapid responses from testing, such as by developing a way to quickly make sense of telemetry data[12]. Also, to get quick feedback, McLean asked machinists to create components from sketches instead of from full documentation[13]. Rapid responses from testing supports learning, as in Deming’s plan-do-study-act cycle, because teams can study the results of their actions soon after implementing them. “The PDSA cycle is meant to be cycled through quickly”[14]

One of Deming’s Points for Management is that you should break down barriers. China Lake broke down barriers for many women because it “employed about half the female technical professionals employed by the United States government”[15]. Barriers between teams were also broken down to enable cross-fertilisation between teams[16].

The China Lake team produced a quality product. Deming’s philosophy focused on quality. The success of Sidewinder illustrates the value of Deming’s philosophy because the methods used by the team at China Lake were in line with his principles.  The success of these methods provides useful examples for testers and quality engineers.  

I read this book with the Profound Book Club. I would like to thank the members of the book club for our discussions, and Rob Park for organising the book club. I would also like to thank Ron Westrum for joining a meeting of our book club and for leading an enlightening discussion.

Additional Resources:

References

[1] Sidewinder Creative Missile Development at China Lake by Ron Westrum (1999, pxi)

[2] Sidewinder Creative Missile Development at China Lake by Ron Westrum (1999, p12)

[3] Sidewinder Creative Missile Development at China Lake by Ron Westrum (1999, p18)

[4] Sidewinder Creative Missile Development at China Lake by Ron Westrum (1999, p1)

[5] Sidewinder Creative Missile Development at China Lake by Ron Westrum (1999, p42)

[6] Sidewinder Creative Missile Development at China Lake by Ron Westrum (1999, p123)

[7] Sidewinder Creative Missile Development at China Lake by Ron Westrum (1999, p268)

[8] Sidewinder Creative Missile Development at China Lake by Ron Westrum (1999, p162)

[9] Sidewinder Creative Missile Development at China Lake by Ron Westrum (1999, p129)

[10] Out of the Crisis by W. Edwards Deming (1986, p5)

[11]  Statistical Techniques and International Trade. Journal of Marketing, 17(4), 428-433. Sage Publications, Inc. , by W. Edwards Deming,  (1953, p428). 

[12] Sidewinder Creative Missile Development at China Lake by Ron Westrum (1999, p111)

[13] Sidewinder Creative Missile Development at China Lake by Ron Westrum (1999, p97)

[14] Quick Tips for Using the PDSA Improvement Cycle by John Hunter (2021)

[15] Sidewinder Creative Missile Development at China Lake by Ron Westrum (1999, p236)

[16] Sidewinder Creative Missile Development at China Lake by Ron Westrum (1999, p94)

[17] Sidewinder Creative Missile Development at China Lake by Ron Westrum (1999, p91)

How to get the most from the constraint on your (testing) process – A review of The Phoenix Project by Gene Kim, Kevin Behr, and George Spafford 

This book is a novel about the transformation of a company from one that experiences pain when it releases software into one that releases and maintains software easily. We can learn much from the company’s journey.

The Phoenix Project is a project that is vital to the company and which experiences many pain points. The project was two years behind schedule. When the project went live, it was the worst nightmare. The Phoenix Project has a “local hero”, Brent, who resolves issues for everyone but also “dictates the output of the whole system”[1]. Erik, a board candidate, coaches Bill, the VIP of IT Operations: ” We call this the constraint – or bottleneck”.[1]. “Until you create a trusted system to manage the flow of work to the constraint, the constraint is constantly wasted”[1]. Erik advises that Goldratt describes five focusing steps, and Bill uses them to get the most from Brent:

  1. Identify the constraint, which they have identified as Brent and that it is not Brent’s fault that he is the constraint.
  2. Exploit the constraint. Make sure that the constraint does not waste any time. Bill reduced reliance on Brent for unplanned work and works on improving methods to exploit Brent for other types of work
  3. Subordinate the constraint by making all decisions subordinate to the above two points. The team starts a monitoring project that helps them do this.[2]
  4. Elevate the constraint. Do whatever is required to get more cycles from Brent. The company creates a new project that does this. [2]
  5. Do not allow inertia to cause a system constraint. The company transforms to create a dynamic culture that does not allow inertia.[3]. “Dev and Ops working together, along with QA and the business, are a super tribe that can achieve amazing things” [4]

The constraint in the novel is on the whole development process. There could, however, be a constraint on the testing process, and Goldratt’s techniques can help you get the most from the constraint.

This novel is a gateway to learning because it shows how the company also benefits from using the four types of work and the three ways, and many other techniques. These techniques are described in a practical way in the novel.

The characters are not perfect; they make mistakes. This makes the novel more compelling.

This novel is a homage to The Goal by Eli Goldratt and gives a practical explanation of Goldratt’s Theory of Constraints. The goal of the characters in this novel is to save the jobs of the people who work for the company.

In many ways, this novel has become more relevant with AI. This is due to the need to understand the increased flow of work.

The Phoenix Project is easy to relate to because it captures the pain so many of us experience as testers, managers, and quality engineers. This makes it a great book to read with a group. I read it with the Profound Book Club, and we related to so many parts of the story. I’d like to thank Rob Park for organising the book club.

I learned so much from reading this book. In the novel’s Afterword, Gene Kim paraphrases Dr Deming: ”Learning is not compulsory…neither is survival”. I would encourage you to read the book, because you will learn techniques that will help you and your team.

References

[1] The Phoenix Project  by Gene Kim, Kevin Behr, and George Spafford (2013, chapter 15)

[2] The Phoenix Project  by Gene Kim, Kevin Behr, and George Spafford (2013, chapter 20)

[3] The Phoenix Project  by Gene Kim, Kevin Behr, and George Spafford (2013, chapter 2)

[4] The Phoenix Project  by Gene Kim, Kevin Behr, and George Spafford (2013, chapter 30)

Additional Resources

Books

The Phoenix Project  by Gene Kim, Kevin Behr, and George Spafford 

The Phoenix Project Graphic Novel:

Beyond the Phoenix Project by Gene Kim and John Willis 

The Unicorn Project by Gene Kim

Articles about The Phoenix Project


Cheat Sheet – Characters and quotes from The Phoenix Project
The Phoenix Project: 10 Years of Transformation, an article by Gene Kim
The Journey Ends Where It Began: Why The Phoenix Project Matters More in the Age of AI, an article by Leah Brown

The book “The Goal”

The Goal by Eli Goldratt
Beyond The Goal by Eli Goldratt
The Goal Summary & Book Review from The Theory of Constraints Institute

AI, Education and Online Safety – An audience with BCS Lovelace and Society Medal winners

It was great to be part of the audience with the winners of the 2025 BCS Lovelace and Society Medals in March 2026. The Lovelace Medal celebrates people from academia, industry or education who have had a major, notable impact in their field. The BCS Society Medal celebrates people who make extraordinary use of IT and computing for the benefit of society. Both medals are awarded by the British Computer Society.

The Winners of the Lovelace Medal

Fireside chat wtih Mirella Lapata and Michael Kolling

Winners of the BCS Lovelace Medal were Professor Mirella Lapata, University of Edinburgh and  Professor Michael Kölling, King’s College London. They both had a “fireside chat” with Emma Logan, Deputy President of BCS. 

Professor Lapata won the medal for her pioneering research in natural language processing and AI. She spoke about the use of vectors in AI and how AI uses plans.

Professor Kölling received his medal for his transformative impact on programming education worldwide. He spoke about the need to create joy in programming for school pupils and the need for everyone to have a good understanding of computer science.

Additional Resources:

Mirella Lapata will teach machines to reason
Mirella Lapata: The Turing Lectures: What is generative AI?

Interview with Professor Michael Kölling
inspirational people in CS- Interview with Prof. Michael Kölling

Winner of the BCS Society Medal

Fireside chat with Beeban Kidron

The winner of the BCS Society Medal was Baroness Beeban Kidron OBE, crossbench peer and founder of the 5Rights Foundation. She received the award for her leadership and advocacy in online safety, particularly her work to protect children and young people in the digital world. In her “fireside chat” with Emma Logan, she spoke about Age Appropriate Design, the Convention on the Rights of the Child and her role in influencing legislation.

Additional Resources:
5Rights Foundation
Age appropriate design: a code of practice for online services
The United Nations Convention on the rights of the child and what it means for online services
Research & Advocacy

 It is really useful to talk to people in IT who are outside of your specialist field. All of the speakers spoke about the importance of collaboration. This was a really interesting event organised by the British Computer Society. I learned from the speakers and from conversations with members of the audience. 

Additional Resources:

The BCS Lovelace Medal 
The BCS Society Medal
British Computer Society

Put quality at the centre of what you do – A Review of ‘Out of the Crisis’ by W. Edwards Deming

Photo of the cover of Out of the Crisis

I bought this book when I was a Test Lead at a startup that made a successful exit. We began working iteratively and discussed how to use the Deming Cycle. It was a useful book for me as a Test Lead because its philosophy helped us put quality at the centre of what we did.

“Poor quality begets poor quality and lowers quality all along the line”[1]. “An unhappy customer tells his friends” [1].

Deming shows that improving quality creates a chain reaction because if you improve quality, costs decrease because there are fewer bugs to fix, and productivity improves[2].

Gadgets and automation are not the answer[3]. Also, best efforts without the guidance of principles can do a lot of damage[4]. Deming defined fourteen points for management that contain the principles required[5]. The first point is to have constancy of purpose toward the improvement of product and service. Constancy of purpose enables everyone to work together to achieve the defined purpose. Another of the 14 points is that management should drive out fear; this will create psychological safety. Management should also institute leadership so that managers are “a coach and a counsel, not a judge”[6].

“Start as soon as possible to construct with deliberate speed an organisation to guide continual improvement of quality”[7]. The Shewhart Cycle (also known as the Plan-Do-Study-Act cycle or Deming Cycle) “will be helpful as a procedure to follow for improvement”[7].

“The reason to study the results of change is to try to learn how to improve tomorrow’s product”[7]. I found that the cycle helped us understand iterations, improve testing, and work together.

Misunderstandings between departments often have their roots in failure to state specifications in meaningful terms[8]. An Operational Definition will “put communicable meaning into a concept”[9]. An example of an operational definition is: define a specific test, define criteria to judge the test, and then decide if the criteria were met. We used this structure via user stories. A friend of mine is now using this structure to write code with Claude by defining a specification, acceptance criteria (including unit and integration tests), and then running the tests in CI.

The philosophy contained in the book contains so much that it can be useful to dip into “Out of the Crisis” to help understand an issue.

It was great to re-read this book with the Profound Book Club. I would like to thank the club for the insightful discussions.

I first started to read “Out of the Crisis” when I was a Test Lead at a start-up that went on to make a successful exit. I found the book very helpful and hope that you do too.

References

[1] ‘Out of the Crisis’ by W. Edwards Deming (1986, p12)

[2] ‘Out of the Crisis’ by W. Edwards Deming (1986, p3)

[3] ‘Out of the Crisis’ by W. Edwards Deming (1986, p13)

[4] ‘Out of the Crisis’ by W. Edwards Deming (1986, p19)

[5] ‘Out of the Crisis’ by W. Edwards Deming (1986, p23)

[6] The New Economics for Industry, Government, Education by W. Edwards Deming (1994, p126)

[7] ‘Out of the Crisis’ by W. Edwards Deming (1986, p88)

[8] ‘Out of the Crisis’ by W. Edwards Deming (1986, p277)

[9] ‘Out of the Crisis’ by W. Edwards Deming (1986, p276)

Additional Resources:

Your Quality Engineering will be richer if you know how Quality Engineering originated

Crocuses

It is useful to know that Quality Engineering originated in Japan in the 1950’s because it gives you long-term engineering memory. If a company has a long-term memory of previous engineers’ work, it can learn from it and build on it. This will enrich your Quality Engineering.

Taguchi wrote that Quality Engineering originated with his responses to product failures in 1950’s Japan[1]. This happened at a time when Japan was rebuilding its industry by improving quality. Knowing how Quality Engineering originated is an example of long-term engineering memory. 

An example of using long-term engineering memory is to use what Taguchi wrote: Quality Engineering is “an instrument to solve problems of .. product design, process design, process control or product control” [2].  We can learn from this and build on Taguchi’s practices by shifting our work to improve quality to the ‘left’ to include product design, process design, process control and product control.

Another example is to use that Taguchi also wrote “Quality is the loss a product causes to society after being shipped, other than any losses caused by its intrinsic functions” The meaning of loss is in two categories:

  1. Loss caused by variability of function.
  2. Loss caused by harmful side effects. [3]

An example of loss due to variability of function, could be an intermittent error.  An example of a harmful side effect could be a product outage.  When teams plan work or evaluate their product, they can build on Taguchi’s ideas to identify these two types of loss and work to minimise them.

When Taguchi originated Quality Engineering, Japan was rebuilding its economy through a strategy of improving quality. Japan understood that “as you produce quality, productivity automatically goes up and costs automatically go down”. [4] Another way to benefit from long-term engineering memory is to learn from the work of those, such as Walter Shewhart and W. Edwards Deming, who influenced Japanese industry at the time Taguchi was working. An example would be for you and your team to work using a plan-do-study-act cycle to improve learning and quality.

There are resources and communities based on the work of previous generations of engineers, such as Taguchi.  You can learn from these communities. I have included links below to resources you may find useful.

Companies with a long-term engineering memory do not need to recreate the methods that have helped others. They have richer Quality Engineering because they learn from and build on the methods and philosophy of others.

References:

[1] Taguchi’s Quality Engineering Handbook by Genichi Taguchi, Subir Chowdhury and Yuin Wu (2005, p 128)

[2] Taguchi’s Quality Engineering Handbook by Genichi Taguchi, Subir Chowdhury and Yuin Wu (2005, p 23)

[3] Introduction to Quality Engineering by Genichi Taguchi (1986, p 2)

[4] Quality or Else: The Revolution in World Business by Lloyd Dobyns and Clare Crawford-Mason (1991,p 2)

Additional Resources

Genichi Taguchi biography from the American Society for Quality

Genichi Taguchi Automotive Hall of Fame

Genichi Taguchi Six Sigma Study Guide

Genichi Taguchi Quality Gurus

American Supplier Institute 

Books:

Taguchi’s Quality Engineering Handbook by Genichi Taguchi, Subir Chowdhury and Yuin Wu (2005)

Introduction to Quality Engineering by Genichi Taguchi (1986)

Blogs:

Philippa Jennings

Curious Cat Management Improvement Blog by John Hunter

Profound – Having intellectual depth and insight by John Willis

4 Legs Software by Rob Park

Software Kaizen by Adam Hawkins

The Digestible Deming by Christopher R. Chapman

Collections of online resources:

William Scherkenbach files

Lonnie’s Technical Library Lonnie Wilson

Courses:

DemingNEXT – eLearning from the Deming Institute that has a two-week free trial

Learning resources from the Sergent Group

Katie Anderson’s Japan Leadership Experience

Podcasts:

Chain of Learning Podcast with Katie Anderson

Software Delivery in Small Batches

Profound Podcast by John Willis

In Their Own Words from The Deming Institute

The origins of Quality Engineering lie in the prevention of product failures in Japan.

Cover of Taguchi's Quality Engineering Handbook

The origins of Quality Engineering are attributed to the work of Genichi Taguchi, stemming from his response to product failures he experienced while working for Nippon Telegraph and Telephone Public Corporation (NTT)[1]. This happened at a time when Japan was rebuilding its economy by improving quality. 

In 1950, Taguchi began working at NTT’s Telephone Laboratory[2]. Demand for phones began to skyrocket due to the Korean War and the growth of the Japanese economy. However, like much of war-damaged post World War Two Japan, “the telephone system was not working”. [3]. Taguchi wrote that “once exchanges broke down or cables malfunctioned, we had to squander a tremendous amount of labour force and expense. When telephone machines went out of order, we had to repair them free of charge or replace them with new ones” [4]. NTT designed the telephone equipment so “there were no other people to make complaints against”. ”The Telephone Laboratory had to consider how to “design a product .., so that it does not cause failures and troubles”. A critical issue was “how to prevent such failures and problems or what types of test should be implemented”. Taguchi wrote: “The design of experiments that I instructed was a method of conducting a test and making an improvement ….This can be regarded as the origin of Quality Engineering”[4].

“Quality Engineering defines the following: “Quality of a product is a loss given to a society after it has shipped”[1], which means that ”quality is defined as the loss a product causes to society after it is shipped”[5]. Taguchi was improving quality by reducing the loss to society caused by product failures.

A key factor in the development of Quality Engineering is that it originated in post-World War II Japan.  This was a period in which Japan used the improvement of the quality of products to rebuild its economy. The Japanese Union of Scientists and Engineers (JUSE) wanted to adopt Walter Shewhart’s Quality Control, which included Statistical Process Control (SPC) and Plan-Do-Study-Act (PDSA) cycle,  [6] and, in 1950, invited W. Edwards Deming to address JUSE. Deming gave a highly influential summer lecture series, which included Shewhart’s SPC and PDSA cycle. [7]. Deming told his audiences that by following his philosophy, “they would be an economic force again in five years”[8]. However, “he was wrong: they did it in four”[8].

Taguchi wrote that Japan “practised what Deming preached”. “Based on his teaching, Japan had developed unique total quality management systems, together with various practical statistical tools, including Taguchi methods”. 

Japanese quality then influenced the USA and the rest of the world. The NBC documentary “If Japan Can, Why Can’t We?” introduced Deming to the public as the “father of quality in Japan”[9]. Following this documentary, Deming and Japanese quality were in great demand; for example, Ford hired Deming, and Taguchi gave lectures at Ford. Ford also made it mandatory to practice SPC and Taguchi’s Design of Experiments [9].

Quality Engineering originated in Japan with Taguchi’s response to product failures at a time when Japan was rebuilding its economy by improving the quality of its products and services.

Additional Resources

References

[1] Taguchi’s Quality Engineering Handbook by Genichi Taguchi, Subir Chowdhury and Yuin Wu (2005, p127)

[2] Taguchi’s Quality Engineering Handbook by Genichi Taguchi, Subir Chowdhury and Yuin Wu (2005, p126)

[3] Deming’s Journey to Profound Knowledge by John “Botchagalupe” Willis with Derek Lewis (2023, p94)

[4] Taguchi’s Quality Engineering Handbook by Genichi Taguchi, Subir Chowdhury and Yuin Wu (2005, p128)

[5] Introduction to Quality Engineering by Genichi Taguchi (1986, p11)

[6] Deming’s Journey to Profound Knowledge by John “Botchagalupe” Willis with Derek Lewis (2023, p94)

[7] Deming’s Journey to Profound Knowledge by John “Botchagalupe” Willis with Derek Lewis (2023, p96)

[8] Deming’s Journey to Profound Knowledge by John “Botchagalupe” Willis with Derek Lewis (2023, p97)

[9] Taguchi’s Quality Engineering Handbook by Genichi Taguchi, Subir Chowdhury and Yuin Wu (2005, p157)

Improve quality by learning from your process data – a review of “Twenty Things You Need to Know” by Donald J. Wheeler

Image of cover of Twenty things You Need To Know

Donald Wheeler has over forty years of experience in the study, practice and collaboration in data analysis. In “Twenty Things You Need to Know”, he shares this experience in the form of questions and answers.

What are two mistakes in data analysis? Mistake one: “Interpreting the routine variation of noise as if it amounted to a signal of change in the underlying process, thereby sounding a false alarm” [1]. Mistake two: Thinking that a signal of change in the underlying process is merely the noise of routine variation and thereby missing a signal”[1].

“You need a technique that will:

  • Allow you to learn from your process data
  • Warn you when your process changes
  • Help you to identify the causes of those changes; and
  • Enable you to operate your process predictably and on target”[2]

Wheeler explains: think of a measurable product characteristic; list all the possible cause-and-effect relationships that have an impact upon the measured value of your characteristic; arrange these causes in descending order of impact.[p10] Shewhart called dominant uncontrolled causes Assignable Causes, and Deming called lesser uncontrolled causes Common Causes.[3]

Assignable Causes will be the source of most of the unexplained variation in the product, such as excess production costs and low quality.[4]

The Upper and Lower Natural Process Limits of the characteristic can be calculated from the measured characteristic’s average. “Whenever a point goes outside the Natural Process Limits, it identifies an…Assignable Cause of variation”[5]. Identifying these points on a Process Behaviour Chart provides a methodology for when to look for Assignable Causes. [6]

Process Behaviour chart of UK inflation

The above Process Behaviour Chart shows UK inflation as the UK came out of COVID as time series data, with an average running through it, an Upper Natural Process Limit above it and a Lower Natural Process Limit below it. Inflation goes above the Upper Natural Process Limit, and so has an Assignable Cause.

In Twenty Things You Need to Know, we learn about why process behaviour charts are created in the way they are, and that Wheeler tested process behaviour charts on 1143 probability models [7]. 

A friend of mine used the techniques in the book to understand the difference in the quality of output of two development teams by analysing the number of unit tests each team created each week. One team developed software with only a few bugs and created a similar number of unit tests each week. Its datapoints stayed within the Natural Process Limits. The second team’s software often contained bugs, and they created different numbers of unit tests each week. Their data points frequently went outside the Natural Process Limits. The process behaviour charts showed that the second team was not always creating unit tests as it developed code, which was an Assignable Cause of the bugs in their code. 

Twenty Things You Need to Know is a companion book to Understanding Variation The Key to Managing Chaos. 

I read this book with the Profound Book Club and would like to thank them for the insights I gained while reading it.

You can also use process behaviour charts to understand quality data by analysing your process data. Reading Understanding Variation The Key to Managing Chaos, and its companion book Twenty Things You Need to Know will help you do so. 

References

[1] Twenty Things You Need to Know by Donald J. Wheeler (2009, p2)

[2] Twenty Things You Need to Know by Donald J. Wheeler (2009, p17)

[3] Twenty Things You Need to Know by Donald J. Wheeler (2009, p14)

[4] Twenty Things You Need to Know by Donald J. Wheeler (2009, p15)

[5] Twenty Things You Need to Know by Donald J. Wheeler (2009, p20)

[6] Twenty Things You Need to Know by Donald J. Wheeler (2009, p21)

[7] Twenty Things You Need to Know by Donald J. Wheeler (2009, p36)

Additional Resources:

Donald J. Wheeler’s Wikipedia page

Articles by Donald J. Wheeler SPC Press

Donald J. Wheeler  QualityDigest

Control Charts Resources, including a spreadsheet template, from the American Association for Quality (process behaviour charts are also known as control charts)

A History of the Chart for Individual Values The ultimate in homogeneous subgroups by Donald J. Wheeler

DemingNEXT –  eLearning from the Deming Institute with two weeks free

Help for you on your systems thinking journey – A review of “Thinking in Systems: A Primer” by Donella H. Meadows

“Thinking in Systems: Primer” is a great resource that can help you whether you are beginning, or have already started your systems thinking journey. 

Meadows helps us understand what a system is. “A system is an interconnected set of elements that is coherently organised in a way that achieves something”[1]. “Once we see the relationship between structure and behaviour, we can begin to understand how systems work”[2]

She also helps us understand the elements of a system. “A stock is the foundation of any system. Stocks are the elements of the system that you can see, feel, count or measure at any given time” [p17]. “Stocks change over time through the actions of a flow[3]. “A feedback loop is formed when changes in a stock affect the flows in and out of that same stock”[4]. A balancing feedback loop stabilises the stock level [5]. A reinforcing feedback loop can be a vicious or virtuous circle because it amplifies and reinforces[6].

The book takes the reader on a visit to the “Systems Zoo”, which contains many examples of systems and draws lessons from them. Meadows also shows us where we can leverage a system.

Donella Meadows writes that we should “stay humble – stay a learner”[7]. That you should “expose your mental models to the light of day”[8], make them as rigorous as possible, test them against supported evidence and scuttle them if they are no longer supported. 

There are useful appendices to the book that include a glossary and a summary of systems principles.

“The systems thinking lens allows us to reclaim our intuition about systems and

  • Hone our abilities to understand parts
  • See interconnections
  • Ask “what if” questions about future behaviours and
  • Be creative and courageous about system design” [9]

To help my learning, I used the book to create systems diagrams of the systems for managing bugs that I am aware of. 

The first diagram shows a system in which managers encourage the reporting of bugs because they are dissatisfied with the quality of the product, and at other times, they discourage the reporting of bugs by stating that no bugs will be fixed this quarter, so that work is focused on a new feature. The changing perceived value of submitting a bug report creates a reinforcing loop that amplifies changes in the numbers of reported bugs. 

This system diagram enables me to see the faults in this system, understand the Stock of Bugs, to see its interconnection with the perceived value of submitting a bug report, to ask “what if” there was stable feedback about bugs to the development team, and to think about how to design a system where the development team did not have to manage the Stock of Bugs.

The second diagram shows a company which asked its developers to fix ten bugs each iteration, which is shown as a balancing loop that stablises the Stock of Ten Bugs. 

This system diagram enables me to understand a small Stock of Bugs as an input into development team planning, to see its connection with software failures, to ask “what if” we reduced the number of software failures, and to think about what the Stock of Ten Bugs represents. 

I read Thinking in Systems with the Profound Book Club, and would like to thank the members of the club for the useful discussions that we had.

Thinking in Systems is, as its title says, “a primer”; it is a great resource that will help you on your systems thinking journey. 

References

[1] Thinking in Systems: A Primer by Donella Meadows (2008, p. 11)

[2] Thinking in Systems: A Primer by Donella Meadows (2008, p. 1)

[3] Thinking in Systems: A Primer by Donella Meadows (2008, p. 18)

[4] Thinking in Systems: A Primer by Donella Meadows (2008, p. 25)

[5] Thinking in Systems: A Primer by Donella Meadows (2008, p. 27)

[6] Thinking in Systems: A Primer by Donella Meadows (2008, p. 31)

[7] Thinking in Systems: A Primer by Donella Meadows (2008, p. 180)

[8] Thinking in Systems: A Primer by Donella Meadows (2008, p. 172)

[9] Thinking in Systems: A Primer by Donella Meadows (2008, p. 6)

Additional Links

Quality comes first – A review of “Deming’s Road to Continual Improvement” by William W. Scherkenbach

William Sherkenbach was Corporate Director of Total Quality Planning and Statistical Methods at Ford Motors and Group Director Process Improvement at General Motors. He wrote that “both of these great companies are better because their journey included Dr. W. Edwards Deming”[1].  In this book, Sherkenbach explains “how to operationalize the Deming philosophy in business, government and academia”.[2]

Ford’s first guiding principle is that “Quality comes first”[3]. When writing the Mission, Values and Guiding Principles of Ford, Sherkenbach wrote that “we used terminology already used by Ford people: Quality is Job 1”[4]. 

Everything we “do can be described in terms of a process”[5], and there are opportunities to improve processes in all aspects of business. Sherkenbach coined the term Voice of the Process, which “is the result the process gives you” [6].

“Dr. Deming tells us that the customer is the most important part of the production line”[7]. “The Voice of the Customer communicates .. the wants and needs of your customers”[7]. “You must have a process to operationally define the Voice of the Customer” [8]. 

Often, the Voice of the Customer and the Voice of the Process do not match. This is an opportunity for improvement that Sherkenbach calls “the Gap”[9].

Integral to the Method of Continual Improvement is the Plan-Do-Study-Act (PDSA) Cycle[10].

“The quality profession itself is in need of major improvement. Everyone must be educated… to use the PDSA improvement cycle.”[11]. Sherkenbach operationally defines the PDSA Cycle as having eight steps:

“Plan

  1. Identify the opportunity for improvement
  2. Document the present process
  3. Create a vision of the improved process
  4. Define the scope of the improved process

Do

  1. Pilot the proposed changes on a small scale, with customers over time

Study

  1. Observe what you have learned about the improvement of the process

Act

  1. Operationalize the new mix of resources
  2. Repeat the Steps(Cycle) on the next opportunity”  [12].

I read this book with the Profound Book Club. I would like to thank the members of the book club for their discussions, Rob Park for organising the book club, and Cathy McGrath and Bill Sherkenbach for joining our discussions. Bill helped me better understand a mistake I made when introducing CI to a development team, and how to better deal with a similar situation in the future.

We can learn a great deal from William Sherkenbach about how to improve. I recommend reading this exceptionally well-written book and exploring the links below.

References

[1] Deming’s Road to Continual Improvement by William W. Sherkenbach (1991, p xii)

[2] Deming’s Road to Continual Improvement by William W. Sherkenbach (1991, p xi)

[3] Deming’s Road to Continual Improvement by William W. Sherkenbach (1991, p 131)

[4] Deming’s Road to Continual Improvement by William W. Sherkenbach (1991, p 132)

[5] Deming’s Road to Continual Improvement by William W. Sherkenbach (1991, p 5)

[6] Deming’s Road to Continual Improvement by William W. Sherkenbach (1991, p 14)

[7] Deming’s Road to Continual Improvement by William W. Sherkenbach (1991, p 12)

[8] Deming’s Road to Continual Improvement by William W. Sherkenbach (1991, p 157)

[9] Deming’s Road to Continual Improvement by William W. Sherkenbach (1991, p 164)

[10] Deming’s Road to Continual Improvement by William W. Sherkenbach (1991, p 162)

[11] Deming’s Road to Continual Improvement by William W. Sherkenbach (1991, p 159)

[12] Deming’s Road to Continual Improvement by William W. Sherkenbach (1991, p 63)

Additional Links

Collaborative customer-focused systems thinking – a review of Sys-Tao: Western Logic – Eastern Flow by Bob Browne

In this book, Bob Browne recounts how he drew on ideas from both the East and the West, including the philosophies of W. Edwards Deming and Eli Goldratt, to make the Great Plains Coca-Cola Bottling Company a success.

In 1980, Bob Browne, with the help of a few others, acquired the Great Plains Coca-Cola Bottling Company in a leveraged buyout. 

Initially, he was getting “good results” by doing the things he had been trained to do. It seemed that was all that mattered. It, however led to a crisis and there was unrest in the company. The “way out of the crisis represents a transformation in .. leadership philosophies”[1].

He used a synthesis of two world views to lead the company; he calls this sys-tao. Sys represents the traditional Western ways of thinking, and tao is symbolic of methods rooted in Eastern traditions. “The works of W. Edwards Deming, the American who taught the Japanese about quality, heavily influenced Sys-Tao.”[2]

Browne saw that “the left side of our brain is more narrowly focused on what we know is important”[3] and that “the right side of our brain is more broadly vigilant, and is open to relationships”[3]. 

Deming organised his thinking into four broad categories, one of which was systems thinking[3].   Systems thinking can be “a mapping exercise to better visualise big-picture ideas”[4]. This approach can be left-brained, mechanistic thinking.  

“Living systems are open, self-organising and nested”[4]. “Lifelike organisations seem to emerge from the bottom up”[4], but “top-down leadership philosophies…are not at all lifelike”[4]. “Humans do not naturally behave in predetermined and mechanistic ways” [4]. “Human organisations must be far more interdependent” [4]. “Interdependent relationships require a more collaborative approach to systems thinking”[4]. Systems thinking takes us to where “our vision becomes shared and it emerges into something bigger than ourselves.”[4]

The biggest single lesson Browne learned, he described as “Collaborative Customer-Focused Systems Thinking” [5].

When the business was sold in 2011, Browne and the other investors had made an average rate of return of 13% for each of the 32 years that they owned the company.

I would like to thank the members of the Profound Book Club for their informative discussions while we were reading this book, and Rob Park for organising the book club.

Sys-tao is the story of a business learning and through that succeeding. There is so much we can learn from this story. There are links below that will enable you to explore sys-tao further.

References

[1] Sys-Tao: Western Logic – Eastern Flow by Bob Browne (2015, Chapter 2)

[2] Sys-Tao: Western Logic – Eastern Flow by Bob Browne (2015, Preface)

[3] Sys-Tao: Western Logic – Eastern Flow by Bob Browne (2015, Chapter 3)

[4] Sys-Tao: Western Logic – Eastern Flow by Bob Browne (2015, Chapter 21)

[5] Sys-Tao: Western Logic – Eastern Flow by Bob Browne (2015, Some Concluding Thoughts)

Resources:

Sys-tao links 

Systems thinking and living systems 

Bob BrowneSys-Tao: Western Logic – Eastern Flow by Bob Browne. Get the book!

We can learn so much from how AI got here – A review of Rebels of Reason by John Willis with Derek Lewis

Much like the aliens in the sci-fi movie Arrival, AI has now arrived. Rebels of Reason tells us how AI arrived, and in doing so, provides us with a deeper understanding of AI. “We learn how foundational ideas have been refined over time, informing the rapid technological advances of our era”[1].

AI has been developed through a long history of technological innovation by people who questioned the frameworks they inherited. They were “rebels of reason”. Charles Babbage and Ada Lovelace built on the work of earlier rebels, such as Blaise Pascal and George Boole. Their work enabled later ‘rebels of reason’ such as the Polish cryptologists who broke the Enigma codes, Alan Turing, Claude Shannon, Fei-Fei Li, the creators of ChatGPT and many others. 

“We have to stop using ourselves as the benchmark for thinking and realise we have created an alien intelligence”[1]. AI is a new form of intelligence. Instead of being frustrated when an AI hallucinates, we need to learn the language of AIs.  

Testing professionals need to develop an understanding of AI that will help us to test applications that use AI and utilise tools that incorporate AI. Reading Rebels of Reason enabled me to appreciate how useful it is to have an understanding of the history of AI, ”because it reveals the evolutionary path from early rule-based systems to the advanced models (often beyond human comprehension) we see today.”[1] I have learned about past successes and setbacks, which have helped me gain a better understanding of the rapid changes that we are living through.

Rebels of Reason was published yesterday. I read pre-publication versions of Rebels of Reason with John Willis as a member of the Profound Book Club. I would like to thank John Willis and the members of the Profound Book Club, from whom I learned a great deal while discussing the book.

Testing professionals will gain from reading Rebels of Reason. AI is here now. We all need to develop a deeper understanding of what AI is. Reading this book provided me with insights into how AI evolved, and through this, I gained a more profound understanding of what AI is.

References

[1] Rebels of Reason by John Willis with Derek Lewis (2025, Epilogue)

Further reading

Profound – Having intellectual depth and insight – John Willis’ blog

Buy the book:

Rebels of Reason: The Long Road from Aristotle to ChatGPT and AI’s Heroes Who Kept the Faith by John Willis with Derek Lewis

Is the job market for testers over a century out of date?

Testers are building their careers in a job market that many experience as difficult. Our ancestors from the early years of the twentieth century would have recognised today’s job market. 

One of my relatives has researched the family tree. He found that when he researched individuals, they had different jobs each time they appeared in documentation. They changed jobs often, so, like many jobs today,  their jobs were probably insecure. 

However, there was a constant at the beginning of the twentieth century, just as there is today. My ancestors were always working with horses. They would, for example, be ostlers, grooms, jockeys, farriers or managing teams of horses. Today, there is also a constant: our LinkedIn profiles list many different job titles that relate to testing. Our skill is with testing; my ancestors’ skill was with horses.

The world has changed in the last hundred years, but our early twentieth-century ancestors would recognise today’s job market for testers, in which people have a core skill, change jobs regularly, and many jobs are insecure.

Today’s job market has so much in common with the job market of over one hundred years ago, which is sad. It makes me wonder if there is a better way.

Toyota’s model offers a successful alternative. Toyota rose from the ruins of post World War Two Japan to be the largest car manufacturer in the world. It recruits people at 18 or 22 years and has a 30-year plan for them[1]. Toyota’s model shows that there is a better way, and that way is successful.

My ancestors from over one hundred years ago would recognise today’s job market for testers. We can learn from Toyota and, in doing so, improve work.

References

[1] Lean Has Failed (or Has It?) with James Womack – Katie Anderson’s podcast (19 February 2025)

Further reading about Toyota:

A review of reviewing “Rebels of Reason – The Long Road from Aristotle to ChatGPT & AI’s Heroes Who Kept the Faith” by John Willis with Derek Lewis

As a member of the Profound Book Club, I feel privileged to be reviewing a pre-publication “galley copy” of John Willis’s latest book, Rebels of Reason. John Willis is one of the founders of the DevOps movement and co-author of The DevOps Handbook and Beyond the Phoenix Project.

Rebels of Reason “invites us to rethink our narrative of innovation” and John has published a sample of the Prolog to the book: First Look at Rebels of Reason. The book is full of great stories related to the history of AI.

I have learned so much while reviewing the book. I have learned from reading Rebels of Reason, from John Willis who has shared insights from his research for the book and career, from other members of the Profound Book Club, and from the discussions that we have had. 

I can’t comment on the contents of the book because it is pre-publication, except to say that it is well worth reading and that I am enjoying it!

We all need to know more about AI, and I recommend reading Rebels of Reason when it is published. To get on a waiting list for the book, sign up for John Willis’ DearCIO Newsletter. 

Thank you, John Willis, for another informative and useful book.

Learning from winners

When we are testing it helps to be aware of the perspectives of people who work in other fields within IT. I attended an audience with the 2024 British Computer Society Lovelace and Society Medal winners. It was great to hear talks from the medal winners and to network at the event. Listening to the medal winners’ presentations about their work enabled me to learn from their perspectives.

Professor Tom Crick MBE hosted the event and facilitated interesting conversations with the medal winners.

2024 BCS Society Medal Winners

  • Karl Flinders – Journalist, Computer Weekly

He spoke about his work as a journalist uncovering the Post Office Scandal, and said that people who work in IT should speak out when they know something is wrong.

  • Anne Marie Imafidon MBE – Founder and CEO, Stemettes

She spoke about the need for diversity and how Stemettes works to inspire girls, young women and non-binary people to work in STEM. She talked about what it was like to be the only woman and/or black person in the room. She also said that including the arts in STEM can be useful, making it STEAM. I now want to read her book She’s In CTRL.

2024 BCS Lovelace Medal Winners

  • Professor Aggelos Kiayias

He spoke about his work on Blockchain including a protocol that reduces its energy use.

  • Professor Philippa Gardner

She spoke about verification. I spoke to her after the presentation when she introduced me to Symbolic Execution, a form of testing,  and Hoare Logic, which can be used for validating software.

  • Dr Sue Sentance

She developed the PRIMM approach to teaching programming and spoke about how it has been introduced to schools around the world. She said it was best to be able to read code before learning to write it.

After the presentations, I networked with lots of great people. This was a great event because it celebrated the medal winners’ success and gave insights into different disciplines within IT. I learned a lot, including new ways to test software and the need for diversity in IT, and I met some great people.

Further reading:

The BCS Lovelace Medal

The BCS Society Medal

Design a site like this with WordPress.com
Get started