First Thoughts on a System of Profound Testing

W. Edwards Demings’s System of Profound Knowledge (SoPK) is a valuable framework for leadership. SoPK provides a view, which Deming called a lens, “by which to understand the organisations we work in”[1]. The SoPK is “a framework for applying best efforts to the right tasks”[2].

There are four parts to the SoPK:

  • Appreciation of a system
    • The use of systems thinking to better understand the organisation. 
  • Knowledge about variation
    • There is always variation, but what is variation telling us about the process and the people who work in it?[3].
  • Theory of Knowledge
    • Knowledge is built on theory. Theory predicts the future and is based on observations of the past. [4] If a theory’s prediction does not occur, then the theory can be revised and in doing so knowledge is gained. 
  • Psychology
    • We need to understand people and the interactions between people.

The theory that is developed using SoPK can be used in a plan-do-study-act cycle in order to increase learning.

The four parts of the SoPK can not be separated, for example, “knowledge of psychology is incomplete without a knowledge of variation” [5], a reason for that would be that “fear invites wrong figures”[6].

The SoPK can also act as “a framework for applying best efforts to the right tasks”[2] for testing. If we look at the four parts of the SoPK we can apply them to testing:

  • Appreciation of a system
    • Systems thinking provides insights that can help us test. Systems thinking can be used to understand the organisation we work for and its customers. It can, for example, be used to understand how the organisation we work for and its customers are one system. 
  • Knowledge about variation
    • The variation in the test results is something that we need to fully comprehend. In the results of a performance test, we see variations that we need to understand. Metrics on quality will also show variation which needs to be understood.
  • Theory of Knowledge
    • We can use the SoPK to develop a theory, such as a testing charter,  on how to test the application or feature we are testing. We will learn from using the theory. It may be that the theory gives us what we want to achieve, or it may be that it does not. If it does not give us what we want we can learn by revising the theory. We can use the theory in a plan-do-study-act cycle to help us learn. 
  • Psychology
    • Psychology can help us understand interactions with our team and interactions with our customers. These are both useful. Testers sometimes have to give bad news, such as when they find a bug. Understanding the psychology of the team helps the tester deliver the message successfully. Psychology can also help us understand what the customer wants and needs from the product we are testing, and this can help us test.

Using the SoPK has a number of advantages for testers.

The use of the SoPK for testing will give testers a deeper point from which to start their analysis, which will enable testers to better understand their team and customers, and so be better testers.

Using the SoPK will give testers a lens, an external view, of their testing.

Deming wrote that if you use the SoPK you will be “a good listener”.[7]

Another advantage for testers using the SoPK is that it is a framework for leaders and by using it testers are preparing themselves for leadership. 

When the SoPK is used as a framework for testing we could call it the System of Profound Testing.

Please let me know your thoughts about the System of Profound Testing.

Thank you to John Willis for inspiration.

References

[1] The New Economics by W. Edwards Deming (1994, p92)

[2] Four Days With Dr Deming by William J. Latzo and David M Saunders (2002, p34)

[3] The New Economics by W. Edwards Deming (1994, p98)

[4] The New Economics by W. Edwards Deming (1994, p102)

[5] Four Days With Dr Deming by William J. Latzo and David M Saunders (2002, p34)

[6] The New Economics by W. Edwards Deming (1994, p94)

[7] The New Economics by W. Edwards Deming (1994, p93)

Other Learning Resources:

Deming’s System of Profound Knowledge by John Hunter

OpenAI contains tools that can help us with test automation

I often write javascript to manipulate the browser either as bookmarklets or as part of automated tests. OpenAI contains two AI tools that can provide solutions if I get stuck with a programming problem.

OpenAI is a research organization focused on developing artificial general intelligence through natural language processing, robotics, and machine learning. Through OpenAI you can access some useful AI tools, such as ChatGPT and the JavaScript Codex Sandbox.


If I need help with a programming problem I can ask ChatGPT for a solution. ChatGPT is a conversational AI developed by Microsoft that you can chat with. You can use ChatGPT via OpenAI. I have found that I can ask ChatGPT questions that help me test.  A simple example of this would be if I asked ChatGPT to write javascript to find two elements on the page that have the class widget. It returns this:

 This is a useful suggestion as to how to solve the simple example programming problem. 


The JavaScript Codex Sandbox is a web-based sandbox environment for developers to test and share code snippets that also can be can be accessed via OpenAI. It is developed by Cloudy Logic, Inc. In the sandbox, you can ask the codex to generate javascript. I asked it to find two elements on the page that have the class widget and it returned:

That the two OpenAI tools returned different solutions to my question about programming is useful as it gives me more than one way to solve the problem. I can decide which solution to use or adapt.

I would recommend exploring OpenAI to see what else you can find.

Useful links:

Create an OpenAI account: https://platform.openai.com/signup

OpenAI overview: https://platform.openai.com/overview

ChatGPT: https://platform.openai.com/playground

Codex Javascript Sandbox: https://platform.openai.com/codex-javascript-sandbox

Note: Unfortunately, OpenAI now carries this message: Codex Sandbox is no longer supported. On March 23rd 2023 we discontinued support for Codex.

A review of “Total Quality Control – the Japanese Way” by Kaoru Ishikawa

Kaoru Ishikawa was a significant figure in the development of quality in Japan. In his book Total Quality Control – The Japanese Way he describes many of the ways that Japanese businesses achieve quality. His book contains many points that are useful to testing professionals. 

He wrote that the very essence of Total Quality Control is quality assurance. It must be built into each design and process[1]. He saw quality assurance as a process that has Quality Control wrapped around it. He defines quality control as: ”to practice quality control is to develop, design, produce and service a quality product which is most economical, most useful and always satisfactory to the customer”[2]. Wrapped around Quality Control is Total Quality Control which he describes as “a thought revolution in management” [3], and it begins and ends with education about quality.

Central to these processes is the Deming Cycle. Ishikawa called the Deming Cycle the Control Circle as it controls all quality control work. He renamed the parts of the Deming cycle from plan-do-study-act to plan-do-check-act. He also divided the Plan part of the cycle into determining goals and targets and determining methods of reaching the goals and divided the Do part of the cycle into engaging in education and training and implementing work. [4] See the above diagram.

Ishikawa describes the roles that Deming and Juran took in helping Japan’s economy recover after World War Two. Deming taught the use of statistics and the Deming Cycle. Juran taught about management’s role in quality. Ishikawa described Deming as “a good friend of Japan who knows Japan”.[5]

The phrase “the next process is your customer” was invented by Ishikawa. He first used the phrase when he was working for a company where, instead of working together, the departments within the company viewed each other as enemies.[6]

The cause-effect diagrams that are called Ishikawa diagrams were invented by Kaoru Ishikawa and were named by Joseph Juran.[7]

Ishikawa believed that people are good and that believing this helps achieve quality. This belief underpins the view that education is important to help people be successful in their work.[8]

He argues that we need “cone-shaped engineers” not “well-shaped engineers”. This is very similar to the argument that we need T-shaped people, and I wonder if this idea is where the idea of the need for T-shaped people comes from. [9]

We must work towards the ideal state where there are no faults in the product and inspection is not required. [10]

A Quality Circle is a small group that performs quality control activities voluntarily.[11] Ishikawa gives advice on how to organise quality circles. He says the quality circles are based on voluntarism, self-development and mutual development. The advice he gives regarding quality circles would apply equally to communities of practice.

Ishikawa believed that the discussions and mutual development that came from Japanese quality control conferences were key factors in advancing quality control in Japan [12].

There are many lessons that testing professionals can learn from the book and I would recommend reading it.

References

[1] Total Quality Control – The Japanese Way by Kaoru Ishikawa (1985, p73)

[2] Total Quality Control – The Japanese Way by Kaoru Ishikawa (1985, p44)

[3] Total Quality Control – The Japanese Way by Kaoru Ishikawa (1985, p1)

[4] Total Quality Control – The Japanese Way by Kaoru Ishikawa (1985, p59)

[5] Total Quality Control – The Japanese Way by Kaoru Ishikawa (1985, p17)

[6] Total Quality Control – The Japanese Way by Kaoru Ishikawa (1985, p107)

[7] Total Quality Control – The Japanese Way by Kaoru Ishikawa (1985, p64)

[8] Total Quality Control – The Japanese Way by Kaoru Ishikawa (1985, p66)

[9] Total Quality Control – The Japanese Way by Kaoru Ishikawa (1985, p101)

[10] Total Quality Control – The Japanese Way by Kaoru Ishikawa (1985, p167)

[11] Total Quality Control – The Japanese Way by Kaoru Ishikawa (1985, p139)

[12] Total Quality Control – The Japanese Way by Kaoru Ishikawa (1985, p4)

Use a cause-and-effect diagram to achieve consensus when defining quality

When I gave the talks on “What is quality” I found it was not possible to provide a definition of quality on which everyone would agree. I recently read a book by Kaoru Ishikawa which included him describing quality using a cause-and-effect diagram[1].  It occurred to me that using a cause-and-effect diagram to describe something on which people have different opinions could be a way of building consensus. The effect the diagram was describing was quality and the causes the diagram showed were the causes of quality. This type of diagram is known as an Ishikawa or fishbone diagram. An illustrative example cause-and-effect diagram is at the top of this post. 

It can be difficult to achieve a consensus can when defining quality. It can be that some team members have definitions of quality that they believe are true. Differences of opinion over definitions of quality can make discussion difficult and this may make it hard to gain agreement on a form of words to define quality.

If a cause and effect diagram is used to define quality then you can use the diagram to create consensus on what quality is. It could be that even if people disagree on the words that they use to describe quality when they add items to a diagram there is agreement on the items they add. A facilitator can then use this as a way to create a consensus. 

The first step in creating a cause-and-effect diagram is to define the effect for which causes are to be identified. In this case, this is “Quality”. The name “Quality”  should be placed in the box on the diagram. Each of the causes of quality can then be placed in a box off the spine of the diagram. Subsidiary causes can be added as branches of the causes that have already been defined.

If people break down the ways to describe quality to specific items such as the use of continuous integration, they can add the items to the diagram as causes of quality. The creation of the diagram could start by adding those causes of quality on which everyone agrees. It could be that the group want to add code reviews as a cause of quality and this can be added to the diagram. Once items that everyone agrees on have been added the group can discuss adding other causes on which they do not agree. This process narrows down the areas of disagreement.

Defining quality is important because if you can not define quality then how can you achieve quality? Using a cause-and-effect diagram to define quality could be a way to create a consensus on an important subject. 

References

[1] What is Total Quality Control The Japanese Way by Kaoru Ishikawa (1985, p47)

Profound Testing with John Willis

I really enjoyed talking with John Willis on his Profound podcast. We talked about testing and the role Deming’s philosophy plays in helping me to test. I learned from the conversation. Thank you, John Willis.

Please click on this link to listen to the  podcast: https://www.buzzsprout.com/1758599/12422563

The Theory of Knowledge helps us learn from our work

To illustrate how the use of theory leads to learning W. Edwards Deming told the story of Chanticleer the cockerel[1]. Chanticleer crowed every morning, and after he crowed he saw that the sun rose. This led him to develop the theory that the sun rose because he crowed, however, one day he forgot to crow and the sun rose anyway. This forced him to revise his theory that the sun rose because he crowed. His knowledge increased because he revised his theory. If he had not had a theory to revise his knowledge would not have increased.

A theory is a prediction of the future based on our experience of the past. 

Dr Deming wrote that “rational knowledge requires theory and builds knowledge through systematic revision and extension of theory based on completion of prediction with observation”[1]

Testers and Test Leads/Managers can use theory to learn. 

As a tester, I may predict that if I test a piece of functionality with a certain test technique I will have a good chance to find a bug. After I have tested that functionality I may have to revise my theory, and because I had a theory to revise my knowledge will increase.

Much of management is prediction. As a test lead/manager I may have a theory that if I make a change to the way testers work it will help them test. When I implement the change I will find out whether the change had the desired effect, and may have to revise my theory. If I have a theory I will increase my knowledge because I will learn from the results of implementing my theory.

Theory is something that we can all use and learn from. An example of this would be that I recently decided to use Boundary Value Analysis to test some functionality. This caused me to reflect on the technique I used, and I now understand better how to apply that technique. If I had not had a theory about how to use that technique I would not have improved my knowledge of it by revising my theory. 

This post describes the Theory of Knowledge, which is one of the four parts of Dr Deming’s System of Profound Knowledge.

More Information about the System of Profound Knowledge:

References:

[1] The New Economics by W. Edwards Deming (1994, p102)

Don’t forget to test the steps in the Help documentation!

I test the steps in the Help documentation for the functionality that I am testing. It is easy to forget to test the examples in Help. Testing the steps used in Help can be useful in more than one way. 

It is important that the steps described in Help ‘work’ because they are what the user will refer to if they need to check how the functionality works.

I have tested using the Help pages everywhere that I have worked as a tester and many times they have helped me to find issues. 

You can find a fault in the software. When I have been using the Help pages to do testing I have sometimes found that the steps in Help do not work. This can be because of a bug in the software, in which case you can raise a bug card. Reviewing the automated tests to find the gap that allowed this bug will also be useful because you can then fill that gap in the automated tests. 

You can find a fault in the Help documentation. The steps in Help may not work because the steps described in Help are incorrect in some way, maybe due to the functionality being changed since the Help documentation was written. If the steps in Help are incorrect then it is useful to talk with the Help pages writer and show them the issue. Wherever I have worked I have found that the writers are pleased to know how Help pages can be improved and correct the steps in the Help pages. The Help pages are not only used by customers, they are often also used by Customer Success. This means that by finding issues with Help, you are not only helping customers, you are also helping your colleagues provide a better service. If you test the steps in Help also means that you provide the link in the process which keeps Help up to date.

If I am doing regression testing the cases in the Help pages form an important test case. If I am testing new functionality the Help pages may not have been written, but if they have been written the cases in them need to be tested because customers will follow the steps in them.

Whether you find a bug to be fixed or find an issue with the steps in the Help pages testing the steps in the Help pages is another way that your testing adds value. Don’t forget to test the steps in the Help documentation!

Using front-end tests for Test-Driven Development

From time to time, development teams need to replace back-end architecture and when doing so they need to ensure that the application’s front-end functionality remains the same. These projects can vary in size, from short projects to long-running pieces of work. This process contains risk as the backend unit tests passing does not guarantee that the front-end functionality the customer experiences still works. Creating failing automated front-end tests that initially fail, as in test-driven development, can help reduce this risk.

Test-driven development (TDD) is a development practice that has been called “test-first development”[1] because the tests are written before the code is written. The tests written in TDD are usually unit tests. These tests will fail until code is written that makes them pass. Kent Beck describes TDD as “an analysis technique, a design technique, really a technique for structuring all the activities of development”.[2]

If the backend of an application is changing the development team will want automated tests to ensure that the front-end functionality still ‘works’. Most teams have front-end tests that are automated via the UI and run on Continuous Integration (CI). The front-end tests are usually written using tools such as Cypress, Playwright, Ghost Inspector or Selenium that run tests against the UI. These tests can be used to help backend developers to test front-end functionality. It would be normal for the backend architecture replacement to be done behind a feature flag. If this is the case, initially, when the feature flag is applied to a branch all or some of the existing front-end tests will fail. The back-end development team then has failing front-end tests that they need to get to pass and keep passing.

As well as using the existing front-end tests to support back-end development, it can also be helpful to create new tests that test areas of functionality affected by the back-end changes. These new tests can be created against the main branch. The new front-end tests will probably fail when they are first run against a branch that uses the feature flag for the new back-end development. The development team will need to get these new failing front-end tests to pass on the branch with the feature flag and then keep them passing. 

It is useful if the back-end developers can run the tests themselves. This can be done in a number of ways such as a command line or by using GitHub actions.

The development team may fear that their backend changes will break some front-end functionality, TDD can help because it “is a way of managing fear during programming”[3]. Using front-end tests to support back-end development by creating failing tests that the team then need to get to pass and then keep passing can be described as test-driven development. It can be described as TDD because the front-end tests are written before the code and code needs to be written so that the tests pass.

References:

[1] Test-Driven Development: By Example by Kent Beck (2003, p203)

[2] Test-Driven Development: By Example by Kent Beck (2003, p204)

[3] Test-Driven Development: By Example by Kent Beck (2003, pxi)

A thought regarding boundary value analysis

“The Art of Software Testing” by Glenford J. Myers is a classic book about software testing and I often use it as a reference. In the book, Glenford J. Myers wrote that “test cases that explore boundary conditions have a higher payoff than cases that do not”[1]. Boundary Value Analysis is a widely used testing technique that is used to explore boundary conditions.  

Myers gives a detailed overview of how to conduct boundary value analysis. A very high-level synopsis of Myers’ description of the testing technique could be described in the following way: 

  • A “well selected”[2] test case meets two criteria: it reduces by one the number of test cases to be developed and it covers a large set of test cases. These two criteria can be met by using equivalence partitioning. 
  • He describes how to conduct equivalence partitioning. He says that you can identify equivalence classes by taking a condition, often a phrase in the specification, and partitioning it into two or more classes. If a test case in an equivalence class finds an error then testing other members of the equivalence class should find the same error.
  • Boundary value analysis is preceded by equivalence partitioning and tests the boundaries of equivalence classes. Myers points out that the test case design should consider the output of the test as well as the inputs to the tests.
  • Myers gives an example of test cases developed using boundary value analysis for an input condition specifying a range of values. He wrote that you should create test cases for the ends of the range and for test cases for invalid inputs beyond the end of the range. 

Myers also says that “it is difficult to present a cookbook for boundary value analysis”, that boundary value analysis “requires a degree of creativity” [1] and that boundary value analysis “is more a state of mind than anything else”[3].

A friend of mine recently had to test a UI with two numeric input fields and started by testing the boundary values of the two fields. He quickly found a bug. My friend said that he had used boundary value analysis to find the bug but he had not created equivalence classes. My friend just started testing the boundary values. 

Myers says boundary value analysis “is more a state of mind than anything else” so, using Myers’ comment, could my friend say that he had used boundary value analysis to find the bug? No, I don’t think he could describe my friend’s testing as “boundary value analysis” because he had not created equivalence classes before testing boundary values.

References:

[1] The Art of Software Testing by Glenford J. Myers (1979, p. 50)

[2] The Art of Software Testing by Glenford J. Myers (1979, p. 45)

[3] The Art of Software Testing by Glenford J. Myers (1979, p. 51)

Definitions of Done, Team Agreements, Ways of Working and Checklists have so much in common

Many teams, create a definition of done to clarify what putting a card in the Done column on a scrum or kanban board means. A good number of teams also have team agreements or ‘ways of working’ which define ways in which the teams work. Also, some teams have checklists of items that need to be completed when an action is carried out. These artefacts; a definition of done, team agreements, a ‘ways of working’ document and checklists, are in some ways different but they also have a lot in common:

  • They are all shared within the team.
  • They all facilitate teamwork.
  • They are all ways of sharing knowledge.
  • They are all ways to help the team by, where appropriate, defining what the team regards as the best way of doing something 
  • They all require being reviewed so that the team knows whether they are still relevant.
  • They all are a way of helping the team improve because when the best way of doing a practice has been defined it can be reviewed and improved. 
  • They should all be updated from time to time because if they are not updated they will cease to describe how the team is working.
  • If a definition of done, team agreement, ‘ways of working’ document or checklist have not been updated in a long time you can be sure it is not used and no longer reflects current practice.
  • They can all be shared with other teams to facilitate learning and collaboration between teams.

I am sure that there are other similarities too and would love to hear from you if you can add some more. 

The points show that definitions of done, team agreements, ‘ways of working’ documents and checklists help teams in two ways:

  1. They all help to aid teamwork and collaboration.
  2. They all can be used to help a team improve its work by defining the best way of doing something and this definition can then be reviewed.

It is never too late to reassess how you define quality

We all need to be able to reevaluate issues and concepts. We have also all heard it said that adapting to change is harder for older people. Dr Joseph Juran is one of the significant figures in quality and he changed how he defined quality when he was over 95 years old. 

In the third edition of his Quality Control Handbook, which was published in 1979, he defined quality as “fitness for use”. The fifth edition of the Handbook was published in 1999 and still used  “fitness for use” as its definition of quality. In the sixth edition of his Quality Handbook, which was published in 2010, he changed his definition of quality to “fitness for purpose”. 

Juran was born in 1904 and died in 2008. He was 95 years old when the fifth edition was published and it must have been after the publication of the fifth edition that he reasoned that “quality means fitness for use” was no longer applicable because “more and more service industries use the methods of managing for quality, the prior definition is not applicable enough” [1].

The new definition of quality is in the section called “How to think about quality” in the sixth edition of Juran’s Quality Handbook. This section is worth reading for its insights on understanding quality. The sixth edition of his Handbook was published in 2010 two years after his death at the age of 103, so Juran’s contributions to the Handbook were written when he was aged between 95 and 103. 

Being mentally flexible is something to value. That Juran could reassess and redefine a core concept when he was over 95 years old should make us think that we all should be able to reassess and redefine important issues and concepts such as quality and also that age is not a barrier to mental flexibility.

References

[1] Juran’s Quality Handbook Sixth Edition

Resources about Dr Joseph Juran:

Web pages:

YouTube

A review of Dr Joseph Juran’s autobiography: “Architect of Quality”

“Architect of Quality”[1] is the autobiography of Dr Joseph Juran. Juran’s autobiography is the moving story of a Romanian immigrant to the USA who rose from poverty to being a world leader in quality and receiving honours from the Emperor of Japan and the President of the USA. Those of us who work in quality have much to learn from him.

Juran worked at Bell Telephones Hawthorne works at the same time as Walter Shewhart was working there and while W. Edwards Deming was an intern there. That three of the giants of quality were working at the same plant at the same time has always seemed remarkable to me. I learned from “Architect of Quality” that this occurred when Bell Telephones were building the first national telephone system for the United States of America. Bell Telephones Hawthorne plant would, at that time, have been the electronics hub for the USA. It would seem that these three giants all came to work at the plant due to the scale of the engineering task that Bell Telephones were undertaking. 

Bell Telephones used statistics to develop their systems. Juran describes how Walter Shewhart invented Control Charts at Bell Telephones as a way to improve quality and how the use of statistics at Bell Telephones led to the creation of Statistical Quality Control. 

Juran found his knowledge of mathematics extremely valuable in several different ways. There is an entertaining section in the book where he describes how he used his knowledge of mathematics to win on roulette tables that were run by Al Capone. 

Writing was important to Juran and he describes how he came to start to write editions of Juran’s Quality Control Handbook. The Handbook is a reference book for all who are involved in quality, and each edition of the Handbook has sold tens of thousands of copies. I have three editions of the Handbook and find them to be useful books to consult when I want to consider an issue more deeply. 

He, along with W. Edwards Deming, helped to rebuild the Japanese economy after World War Two and in his autobiography he describes the lectures and courses he gave in Japan. He also writes about Deming’s work in Japan. Lloyd Dobyns and Clare Crawford-Mason’s overview of Juran and Deming’s work is more valuable because it is written from the viewpoint of a third party who is trying to understand the work of the two men[2].

Universal methods were something that interested Juran. This interest came from his knowledge of mathematics. He describes how this led to him creating universal methods in management planning and control. 

If you work in “quality” then I would recommend this book to you and would also recommend exploring Juran’s work.

References:

[1] Architect of Quality : The Autobiography of Dr. Joseph M. Juran by Dr Joseph Juran

[2] Quality or Else by Lloyd Dobyns and Clare Crawford-Mason

Resources about Dr Joseph Juran:

Web pages:

YouTube

Books

A chart for measuring the quality of “spinning plates”

Quality metrics can be like measuring the wobble on spinning plates. 

The engineering teams are merging to the main branch, code is being deployed and you need metrics to show if there are issues with this process that require your attention. The teams working are rather like spinning plates. The teams are working and management needs to know if something is going wrong that they need to address. You can see that the plates are spinning and wobbling. You need to know if the plate is wobbling too much and will fall. If it is going to fall then you need to intervene.

Walter Shewhart created control charts to solve this problem at Bell Telephones. He saw variations in how the telephone production line worked and wanted a method to show when he needed to intervene. He plotted the metric he wanted to measure on the chart and then used the standard deviation to calculate the Upper Control Limit and Lower Control Limits. If the curve went above the upper limit or below the lower limit he needed to intervene. If the chart remained between the upper and lower limits he did not need to intervene.

Control charts are now used in many industries and can help us in software engineering.

John Willis recently wrote in his blog about how he used control charts. He used them to track his teams’ deployments to help him understand whether the team was improving.

I had a friend who was testing the work of two teams. One team was releasing new features and moved from making monthly releases to continuous delivery. The other team made monthly releases and needed developers to work extra hours to fix bugs when a release was made. In order to find out the difference between the two teams my friend decided to use control charts to analyse the work the two teams were doing. He created control charts that showed the number of unit tests added by each team each week. He found that the team practising continuous deployment created a fairly consistent number of unit tests each week and the curve on its chart was between the upper and lower limits. The chart for the team making monthly releases was quite different. There were weeks when the team making monthly releases added unit tests, there were also weeks when the team deleted unit tests and there were weeks the number of unit tests did not change. The curve on their chart often went below the lower limit. The control charts showed that a difference between the two teams was that the team practising continuous delivery were writing unit tests every week and the team making monthly releases frequently were not writing unit tests. The charts helped my friend to understand what was happening; the team that was making monthly releases were not writing unit tests. The understanding gained from the control charts enabled my friend to start a conversation with management about the team making monthly releases. 

Control charts are a useful tool that can help us understand quality. If the ‘wobble’ on a spinning plate is within the lower and upper limits you do not intervene. You only need to intervene if the ‘wobble’ goes outside the lower and upper limits. 

Thank you to Krizia Cureg and Nick Smith for the conversation that led to this blog post.

Further reading:

Books:

Blog:

Websites:

eLearning:

Articles published on other websites

  1. LambaTest
  2. British Computer Society It Now Magazine
  3. British Computer Society It Now Newsletter
  4. Ministry of Testing
  5. The CTO Club
  6. DZone

The Five S’s create a structure for test automation

I use the Five S’s to create a disciplined structure that helps me create and maintain automated tests. Mary and Tom Poppendiek recommend using the Five S’s to create the discipline necessary to develop quality software. They wrote that “the five S’s are a classic lean tool to organise a workspace” [1]. 

Mary and Tom Poppendiek describe the Five S’s by translating the original Japanese words into English and give examples of using each of the S’s in the kitchen, in the workplace and for a java development project. I have taken their idea and created a table showing the translation of the Japanese words and examples of using the Five S’s in the kitchen, workplace and test automation.

JapaneseEnglishKitchenWorkspaceTest Automation
SeiriSortSort through kitchen tools and throw away any unused tools.Sort through the servers, and find old files that will never be used. Delete or back them up.Remove dead code, unused imports, and unused methods.
SeitonSystemiseFind a place for everything and make it easy to find.Craft desktop layouts and file structures so that things are easy to findOrganise projects and packages so that everything is easy to find.
SeisoShineClean the kitchen.Clean whiteboards and desktops.Resolve TODOs and improve performance.
SeiketsuStandardiseFill and run the dishwasher every nightPut automation and standards in place.Reduce complexity and improve ease of maintenance.
ShitsukeSustainKeep up the disciplineKeep up the discipline.Use and follow standard procedures.

I am sure that you can think of additional examples of how to use each of the S’s to aid your test automation. 

Test Automation is software development and benefits from the disciplines that are used in software development. The Five S’s can help improve test automation that uses test frameworks such as Selenium and Playwright. They also can help automation projects that use low code tools such as Ghost Inspector.

I have found that sustain is the most important of the Five S’s because if you do not sustain the standards and practices that you have created with the other four S’s you can not improve. Sustaining standards and practices is a practice where value is gained.

My thinking about how to automate tests and maintain the tests has been shaped by the Five S’s. This is because the Five S’s enable me to view the issues I encounter, such as how to maintain a library of functions, through a framework that helps me improve the tests. 

The Five S’s help me keep my automated tests in a state where they are easy to maintain, have no flaky tests and are easy to read. I hope that the Five S’s can help you do the same.

References:

[1] Implementing Lean Software Development By Mary and Tom Poppendiek

Design a site like this with WordPress.com
Get started