Don’t ask “Why did the tester miss that bug?”

“Why did the tester miss that bug?” is a question I have heard in many places but it is the wrong question. W. Edwards Deming’s philosophy shows that if something has gone wrong we need to look at the process and not blame the individual. A tester can be working in a system that the tester can not change and that creates software that contains bugs. If there are bugs we need to look at the system that produced them.

The “Red Beds Experiment” was regularly conducted by Deming. He first ran the “Red Beads Experiment” in Japan in 1950. The book ”Four Days with Dr Deming” gives an example of how the Yawata Steel Company benefited from changing its process as a result of seeing the “Red Beads Experiment”. In the 1980s and early 1990s, Deming ran the ”Red Beads Experiment” in the USA as part of his seminars for executives.

In the “Red Beads Experiment” Deming asked members of the audience at his seminar to act as Willing Workers, a Secretary, and Inspectors in a factory that makes white beads. Unfortunately, there are red beads, as well as white beads, in the raw material that is delivered to the factory. The Willing Workers are to remove the white beads with a paddle and are given detailed instructions on how to do so. The Inspectors count how many red beads have been removed on the paddle and the Secretary records the results. Each scoop of the paddle by a worker represents a day’s work. Unfortunately, because the raw material contains both red and white beads each scoop of the paddle also contains red and white beads. On day four management is disappointed at the number of red beads being collected in the paddles and develops a plan. The plan recommends keeping only the best workers, so the workers with the worst results that day are laid off. The remaining staff are warned that unless results improve the factory will be closed. On day five the results have not improved as red beds are still being scooped up with white beads, so the factory is closed.

In “The New Economics” Deming lists fourteen lessons from the experiment. One of the lessons is that the Willing Workers were victims of the process as they could not, under the manager’s rules, improve their performance.

Software quality and testers can be victims of the process, like the Willing Workers.  We should not ask “Why did the tester miss that bug?”, we should instead ask how to improve the process.

More Information:

  • eLearning from the Deming Institute: DemingNext contains a course on the Red Beads Experiment

State transition diagrams help to define tests

State Transition Diagram of sports matches

Identifying what states exist in functionality and how changes in state take place enables testers to identify tests that should be executed. State transition diagrams provide simple notation for summarising states and how changes in states take place. I recently used a state transition diagram on a project and it helped me identify the tests to execute.

To create a state transition diagram you need to first identify objects that may be in one of several states depending on what events have occurred. An event causes an action which may change the state of the object. 

The above state transition diagram shows the states of sports matches and the transitions between them. The notation used in the diagram above is:

  • Circles represent to the states that the system may be in
  • Labelled arrowed lines from states show events that cause actions
  • Text on a line represents actions that are taken when events occur
  • Arrowed lines from actions show a change to another state

The state transition diagram identifies for us the states of matches and the changes that cause matches to change state. We can then  test all of the states and test all the actions that cause the matches to change state. This gives us a level of test coverage and so gives us confidence in our testing.

Many charting tools that support the creation of state transition diagrams have free plans.I used Lucidchart for the diagram in this post. 

I find that creating a state transition diagram is an iterative process from which I learn about the functionality I am to test. Whenever I come back to the diagram I review how the diagram represents the functionality and this helps me get a better understanding of the functionality that is to be tested. As I get a deeper understanding of the functionality I update the diagram and in this way creating a state transition diagram helps me gain a fuller appreciation of the functionality my team is developing. 

The diagram can also be shared and reviewed. Sharing the diagram helps collaboration within your team. Something I learned from using a state transition diagram on a recent project was that the diagram is easier for the rest of the team to use if, instead of having one crowded diagram, you break the functionality down into several clear diagrams.

Further reading: Computer Science 4th Edition by C. S. French

Three Things That Have Helped Me in My Testing Career

The Ministry of Testing Bloggers Club has asked bloggers to write a post that describes t three things that have helped them in their testing career. Many things have helped me in my testing career, and here are three of them:

  1. The Testing Community

The Testing Community has helped me throughout my testing career. I started working as a tester over twenty years ago. Back then I remember there being two testing newsletters and BCS SIGiST conferences in Grosvenor Square. The conferences and newsletters introduced me to other testers and to resources that helped me develop my testing skills.One way I have found conferences useful is that I go to the conference with a problem I was trying to solve and ask people if they have the same problem. I have often found that other testers have a similar problem, and that a discussion with them helped to find a solution. 

Today there are so many more organisations and forums for testers, such as the Ministry of Testing. I am now working as a lone tester and being able to share ideas with other testers is a great help and support for me. The online discussions, the newsletters and and events all help me learn new ideas and validate those that I have developed.

During lockdown the on-line testing community was a great help by providing ideas, discussion and support. 

So, I would like to say a big thank you to all the testers who organise events, speak at events, take part in online discussions, write blogs, write courses, write articles, review articles, write newsletters, contribute to forums, make podcasts, make videos, mentor people wanting to become testers and generally support the testing community.

  1. Continual Learning

Testers need to be hungry to learn new things, and I enjoy learning new skills. There are fundamental principles in testing such as “building quality in”, rather than trying to test quality in. Reading books by W. Edwards Deming has helped me get a better understanding of these principles. How these principles can be applied is something which I continue to explore and learn about.

We also need to keep learning new ways of working and new techniques. I am continuing to learn how to use new tools, expand our technical knowledge, and develop our soft skills .

  1. Great Colleagues

Great colleagues mean that you can get so many positive things from work. I have learned so much from people I work with and continue to do so. I have learned new testing skills, new programming skills, learned about agile and lean, learned about CI/CD and learned new soft skills. 

Which programming languages should a tester learn?

It is helpful for a tester to be able to program as it enables a tester to get a better understanding of the application they are testing. Being able to code also helps create test data, automate tests and build a better relationship with developers.

But which programming language(s) should a tester learn? Over the last year or so I have learned some javascript. This has helped me work with front end developers, automate tests  and create test data. To help backend developers I used Node.js to automate tests for the API, but was this the best thing to do?

Javascript is used by frontend developers. If I want to support developers I should use the programming language that they use and not use Node.js. Our backend developers use golang and for this reason I have started to learn golang. I am using the go track on exorcism.org to learn golang. Exercism is free, and I like its style of learning. As an exercism.org student you write solutions to exercises, and the exercise passes if the tests for the exercise passes. It is sort of TDD style learning! You can also ask exercism.org for a mentor.

I have also created PR’s for my solutions to the exercises and got feedback from backend developers on the golang code that I have written. This has been really useful as it shifts my perspective from just getting the solution to the exercise to pass the tests to a perspective of trying to write good code.

Other free tools that are helping me to learn golang are:

Golang is quite different to javascript as it is a compiled language that is statically typed, and I am learning how these differences affect developing code. Learning golang is expanding my technical knowledge and is beginning to help me to work with backend developers.

Learning golang will also enable me to read the code that I am testing and pinpoint where in the code a bug is occurring.

It is great that where I work gives me time and support to learn new skills.

Being able to code in javascript has helped me as a tester, being able to code in golang will help me too when I work with backend developers to test API’s and create test data.

Testing messaging with a decision table

Decision Table

Recently I was testing the messages sent from an app to the user when I noticed that the messages were dependent on the interaction of three conditions. I used a decision table to analyse the messaging because decision tables show how many tests are required to test something that is affected by a group of conditions . 

In an earlier post I gave the steps for creating a decision table.

I had thought that decision tables would be useful for testing API endpoints. I have learned that testing messaging would be another use for decision tables. In the scenario I was testing there were three conditions. I found that all the three conditions could be true or false and that the message contained varied depending on whether each of the three conditions were true or false. Analysing this problem was most easily done with a decision table as the actions defined by the table would be the tests for the messages the app creates. 

Decision tables have one row for each condition. The number of rules in a decision table is 2 to the power of the number of conditions. There were three conditions so I would need eight columns for the rules. The actions in the table would cover all the test cases. What I needed to do was ensure that I had listed the conditions correctly as the decision table would then show me how many actions were required to test the messages. See the image above for an example of the decision table.

When I had created a decision table and I could execute the tests in the table. The decision table would ensure that I defined tests for each of the possible variations of the three conditions.

I shared my decision table with my team to help them in their work as I don’t want to find bugs, I want to help prevent them. 

Using a decision table had a number of advantages. It ensured that the tests covered all the functionality as I knew that I had covered all the conditions. It also gave me something I could share with the developers and help them in their developer testing which would mean that there are less bugs for me to find.

Further Reading: Computer Science 4th Edition by C. S. French

A review of “Artificial Intelligence and Software Testing”

“Artificial intelligence and Software Testing” is edited by Adam Leon Smith and has recently been published by the British Computer Society. The book covers both using Artificial Intelligence (AI) to test software and testing AI systems. The book will help testers who want to learn about AI and testing. 

Rex Black writes in the introduction about how AI is a wave of change and that we need to test AI in order to learn if we can trust AI. Testing AI provides new challenges as AI changes its behaviour in response to testing, and we need to rethink many aspects of testing.

In the first chapter Adam Leon Smith writes about AI trustworthiness and quality. He writes about the importance of being able to trust AI, about quality problems such as understanding how an AI system optimises itself, and also gives useful examples.

Quality and bias are the subjects of the second chapter by James Davenport. The chapter is an exceptionally good explanation of the issues relating to bias in AI. The chapter covers the difference  between bias and fairness and issues with bias in AI.The examples given in the chapter about bias in AI and in everyday life give a deeper picture of the issues relating to bias. 

The fourth chapter is about testing machine learning (ML) systems. In this chapter Adam Leon Smith writes about the role of the tester, the nature of machine learning systems and what testing metrics and testing techniques are appropriate for ML systems. The role of a tester in ML is described in a way that a tester in a team that does not use AI can relate to. The examples of testing metrics and techniques are different to those in non AI projects and so are really interesting.

Jeremias Rößler writes about AI based test automation in the next chapter. He writes about the problems in using AI for test automation and also covers AI being used to generate unit tests and UI tests. 

Ontologies for software testing are described by Joanna Isabelle Olszewska in the sixth chapter. She explains what ontologies are, how they relate to AI and gives examples of several ontologies for software testing.

In the seventh chapter Jonathan Wright describes a shift-right approach to testing called “digital twin testing” which he used when collaborating with MIT. This approach was used when Jonathan led the QA and testing efforts for the COVID Pathcheck Foundation.


Many testers are now thinking about how to use AI to assist testing and how to test and this book will help them. Reading this book is a great way to become better informed about AI and software testing. You can get your copy from: https://shop.bcs.org/store/221

I have changed my mind about…Test Plans and Test Documentation

When I started as a tester I learned that test documentation, such as test plans, needed to gain approval from the Director and could be regarded as a project deliverable. I have learned that this is not helpful and that test documentation should support testing and the development.

I remember that, before testing started, we wrote detailed test plans for each project. This documentation included test plans that conformed to IEE standards and all the documentation was written in Word docs. These plans needed the approval of the director but I am not sure that anyone ever read them apart from the testers. We used the plans to test the new project but getting the directors approval for the documentation was really very important. The testers discussed whether test documentation was a project deliverable as these documents had to be produced for every project and needed the approval of the director. I found that as a project progressed I learned more about the project and that this learning was not included in the documentation as the documentation had already been written. As I explored the functionality I frequently found bugs from tests I ran that were not included in the tests I had written in the test plans. 

Now I only create documentation if it helps testing and development. I have learned to have test documentation that supports testing as this way I can better support the development of the feature I am testing. I create test documentation such as mind maps, spreadsheets, decision tables, flow charts and state transition diagrams that help me analyse the features being developed. This type of documentation helps me test and can help the developers, designers and product owners too. I also create test strategy documents for a number of reasons which include that writing the strategy enables me to clarify my thoughts and enables me to share ideas. The test plans I create now are to help me test and to help me collaborate with the other members of my team. Documentation is no longer thought of as a project deliverable. The deliverable is a high quality product and the test documentation helps the team to achieve this.

I have changed my mind about documentation.It should not be about gaining the directors approval or being a project deliverable. Documentation should be about supporting testing and development because that way it can help improve quality.

An easy way to start testing your API

If you have not tested an API then starting to test an API can feel quite daunting as you may feel that you need strong technical skills for this type of testing. You can, however, start to test your API easily and then learn technical skills to test the API.

An API is an application programming interface and is used to interact with an application. It is an interface to the application that we are testing just as the user interface (UI) is an interface is. 

An easy  way to start to test an API is to use the Interceptor on Postman. Postman is an API testing tool that has a free plan. The Interceptor captures API requests in Postman where you can explore them and test them. Once you have captured the requests in Postman you can learn skills to test them.

To use the Postman Interceptor you need to install the Postman desktop app and then follow the instructions to install the Interceptor: https://learning.postman.com/docs/sending-requests/capturing-request-data/interceptor/

The Interceptor can help you start to test your API by capturing the requests made by the API calls that you want to test. First you need to capture the API calls in Postman:

  1.  In chrome navigate to the page that makes calls to the API that you want to test. In Postman click on capture requests->via interceptor. Choose a collection to save the API requests to, then click start capture. 
  2. On your web page make the user actions that invoke the API calls that you want to test
  3. In Postman click on Stop. 

You should now see a list of API requests in Postman. There will be more requests than the ones that you are interested in.

Postman will have intercepted the authorisation that the request requires so each request should run if you click on Send. I would work through the requests to see what each one does and how it works. If your API has documentation it will help you understand what each request does and how to use it. When you are doing this you are starting to test your API.

It will probably be best to start your exploration of the API with understanding authorisation as the authorisation on the request you intercepted will expire after a period of time. The API’s documentation will explain how authorisation works for the API.

Once you have intercepted requests in Postman you can then explore the API by doing things like seeing what happens when you change a parameter. When you change the values for an API parameter it is, in many ways,  like changing the values you enter in a text box. The testing techniques that you know from testing via the UI can be used for testing an API. You will build on your knowledge as you test the API.

By using Postman Interceptor you can start to test your API quickly and without deep technical knowledge. To develop your API testing skills further you could take courses from the API path at the Test Automation University.

How Can I Test This? an app with microservices

“How Can I Test This?” is a book that contains examples of how to test different sites and apps along with test heuristics applied, the reasoning as well as ideas for which tools will help testing. The book aims to be useful both to new testers and to more experienced testers, and has just been published on Lean Publishing.

The book already has great testimonials from Federico Toledo and Julia Pottinger:

Each chapter gives an example of how to test a different application or site and covers the features to be tested, heuristics to apply, any assumptions made, how to limit scope and what tools to use. 

I am really pleased to have contributed a chapter to the book. I wrote the chapter by drawing on my experience and from my reading of one of Gerry Weinberg’s books. 

I have been testing an app with a microservices architecture and I have used my chapter in the book to share what I have learned about testing an app that has a microservices architecture. Testing an application that has a microservices architecture can’t only focus on the UI. Microservices are increasingly used and they provide new opportunities and challenges for testing. I hope that testers find the ideas in my chapter for testing microservices useful.

Writing the chapter has enabled me to clarify my thoughts in a way that I have found useful and I hope others do too. Reviews of the chapter were so helpful and I have learned a lot from the process of writing the chapter. I would like to thank the reviewers: Lisi Hocke and Phil Wong.

I would also like to thank Nicola Lindgren for leading the process of producing the book.This has involved her in a lot of work in many different roles, including being the project manager and promoting the book, all of which she has done brilliantly.

I hope that you read the book “How Can I Test This?” and that the examples in the book are useful to you in your role as a testing professional. 

The book is available here: https://leanpub.com/howcanitestthis

Coaching tools for testers

Sometimes testers need to “coach”, when I do so I draw on my knowledge as a junior cricket coach and use the PoP framework. My cricket coaching qualification also included using the coaching tools that best suit the player, outcome and practice.

The tools that we can use to coach are:

  • Instruct
  • Demonstrate
  • Observe
  • Analyse
  • Intervene
  • Praise
  • Use silence
  • Question
  • Feedback

Having a list of tools to use for coaching is useful as it helps the coach consider which tools to use when leading a practice. The list of tools also helps the coach to review the practice after the practice has been completed as the coach can consider if they used the appropriate tools for the practice. It also helps you talk to other coaches about coaching techniques as you can discuss which tools are being used.

I will first illustrate how I use the tools when coaching cricket and then show how they can be used when “coaching” as a tester. 

If the practice is for something new to the players, for example if they are young or new to the game, then the tools I would tend to use are demonstrate and instruct. I would demonstrate the new skill and instruct the players how to execute the skill. I would also analyse their use of the new skill and give praise for their performance.

When I am working with more experienced players I often use silence, in order to let them solve problems for themselves. While I am using silence I would also analyse how they are doing and give feedback at the end of the practice.

Choosing which coaching tools are appropriate is something that we need to do as testers when we are coaching.

Facilitating a retrospective can involve using coaching skills. When I am facilitating a retrospective there are occasionally times when the facilitator intervening stifles discussion. When I am the facilitator I am not afraid of being silent as it is something I can use to let the team bring forward ideas. This means that I am less likely to intervene and the team can discuss the problem they are focussing on. Using silence is an important coaching skill. When using silence you need to be considering what are the players trying to do, whether if you say anything now will it help and what else can you be doing when you are silent such as observe or analyse.

I recently made some short videos with a colleague to explain Boundary Condition Analysis and Equivalence Partitioning to developers. In these videos I used different tools from when I facilitated retrospectives. I used demonstration as the coaching tool. I demonstrated the two testing techniques because they were new to the developers. I also analysed their responses to the first video before making the second video.

Being aware of the tools that you can use when “coaching” helps me make choices and review the choices that I have made.

A coaching framework for testers

Sometimes testers need to “coach” and when I do I use a coaching framework to help me. At the weekend I coach junior cricket and I use what I have learned from cricket to help me as a tester. The PoP framework we use in coaching cricket can be useful to us as testers.

 In cricket we practice player-centred coaching using PoP because we want to focus on the players’ needs. PoP helps us do that. Pop stands for Player, Outcomes, Practice.

I’ll first show how I use PoP when coaching cricket and then show how it can be used for “coaching” as a tester.

PoP starts with the needs of the Player. What are the needs of the player? When looking at the needs of the player we need to look at the whole person, including their stage of development as a player. It could be, for example,  that the player is relatively new to the game and that the players’ drive shot is great but that their forward defensive shot could be better.

We should then decide on the Outcome for the practice that meets the players needs. If you were working with the batter in the above example you could aim to have an improved forward defensive shot as the outcome of the practice.

We then design a Practice to help the player achieve that Outcome. 

After each practice it is important to reflect on the practice to see what you learned as a coach as there is always something to learn. Sometimes a practice does not go as well as you had hoped, this is something to reflect on and learn from.

This coaching framework followed by reflecting on the practice works as a plan-do-study-act cycle. PoP gives you a structure to plan your practice, you then do the practice and then reflect on the practice. Reflecting on the practice enables you to study and learn from the practice. What you learn from reflecting you can take into planning the next practice. 

The PoP framework can help us as testers when we need to “coach”.

If you want to coach or a group of people a testing technique you can use PoP. You should start by thinking about the needs of who you want to coach, then consider what outcome you want before designing a practice for the coaching. The framework means that you consider the needs of the person or people being coached before you consider the outcome that you want to achieve. 

This coaching framework can be useful for working with your team, for example when you are planning a retrospective. You can view the team as the Player and the retrospective as the Practice. If you want to plan a retrospective for your team you will want to consider the needs of the team. Maybe the team needs to get a better understanding of a particular issue at the retrospective, so the team getting a better understanding of that subject would be the Outcome you want from the retrospective. You would then design the retrospective to achieve that Outcome. After the retrospective you should reflect on what went well, what went badly and use the knowledge gained when planning the next retrospective.

If you have reports then you will want to help them develop themselves.When I’ve had reports I found this framework useful for planning 1:1’s and annual reviews.  In these situations the Player in PoP is your report and the 1:1’s or reviews are the Practice in PoP. The Outcome that you want from the practice will help you plan the review or 1:1. The framework encourages the manager to think about how to develop their staff  because it helps you think about the needs of the person you manage and how to help them grow.

When using this framework you will also need to think about what practice will work in the team’s cultural context. It is useful to listen to others and take advice to help ensure that what you plan works within your team’s culture.

Four insights from “a reference book for all who all who are involved with quality”

Joseph Juran wrote in the 5th Edition that Juran’s Quality Handbook ”is a reference book for all who are involved with quality of products, services, and processes”. I find it a useful book to refer to if I am thinking about a problem.

There are eight editions of the Juran’s Quality Control Handbook. They are large books and so are best viewed as reference books. The Handbooks have very useful subject indexes.

Dr. Joseph Juran, like W. Edwards Deming, worked for Walter Shewhart at Bell Telephone Labs. Later Juran taught quality control at New York University and he also helped to revive the Japanese economy after World War Two. Juran went on to work with a number of companies and also worked with Steve Jobs.

I have found many useful insights in the handbooks, for example: 

  1. The article by Frank Gryna in the 4th Edition on Quality Costs gives a background to the concept of quality costs that is missing in Crosby’s writing on quality costs. The article also groups costs in categories, which is helpful when creating a model of costs that can be shared in your company.
  1. The Juran Trilogy of Quality Planning, Quality Control and Quality Improvement can be used to create a useful way of discussing quality with your team and of designing quality in.
  1. Juran’s definition of quality also gives an insight into different perspectives of quality. He says that two of the meanings of quality are:

“1. Quality consists of those product features which meet the needs of customers and thereby provide product satisfaction.
2. Quality consist of freedom from deficiencies”

In the 5th Edition of the Handbook Juran writes how managers can disagree on whether improved quality costs more or less. This confusion is resolved by this definition of quality as items that improve quality in point 1 tend to increase costs, and items that improve quality in point 2 tend to decrease costs.

  1. Use of the Pareto Principle to identify the “vital few” to assure that resources are used where they do most good.

I have copies of the 4th, 5th  and 6th Editions and use them as a reference. I think that there are many more insights to be found while exploring the Handbooks and I look forward to discovering them.

Further reading:

Juran’s Quality Control Handbook 4th Edition
Juran’s Quality Handbook 5th Edition
Juran’s Quality Handbook, 6th Edition

Are metrics always helpful? an example of a success and a failure

Engineering teams often keep metrics on their work, but are these metrics always helpful?. 

Metrics could be something that the manager or director has asked for or metrics something that the team has created to help the team. I have found that creating metrics on quality is something testers often do. Creating useful metrics is not easy and creating metrics is an underrated skill that testers have.

If you work in a startup you will have more freedom as to what metrics to create and if you work in a larger company you will probably have less influence over what metrics are created.

Metrics that support the engineering team can help the team improve. If the metrics do not support the team, then the team can feel that it is being judged or measured and the team can become defensive.

I once worked in a team which had made big improvements in the quality of its work and I kept a chart showing each time we had to fix a bug in production. This was good news for the team as the improvements the team had made meant that we did not have any incidents in production and only had to fix a bug once every six weeks. This metric showed the team what it had achieved, helped the team feel good and encouraged further improvement.

A friend of mine once worked in a team that made monthly releases. They put a post it note on the wall for each bug put into production, and grouped the post it notes by release, soon the wall was too small for all the post it notes. My friend’s line manager liked it because it visualized the scale of the problem, but the management team saw it as a threat. The metrics created with the post its notes contributed to tension amongst the management team and so had no positive effect on quality.

Creating metrics that support the team can have a positive effect on the team and contribute to continuous improvement.

What job should you hire a tester to do?

Jobs Theory gives us a way of understanding clients by asking what job did they “hire” a product or service for. An example of Jobs Theory is given in “Competing Against Luck”: by Clayton M. Christensen, Taddy Hall, Karen Dillon and David S. Duncan. The example they give explains why people “hired” milk shakes. Research found that the jobs people “hired” milk shakes to do were to make their commute to work more interesting and to treat their children in the afternoon. With this insight the company that was selling milkshakes was able to work out who their competitors were, and understand that there was more than one motivation for people buying milk shakes. Jobs Theory can be applied to understanding many problems. If we were to apply Jobs Theory to the hiring of testers, what job should a tester be hired to do?

Job One: Testers do the testing for a development team

A tester’s job title suggests that they are hired to do the testing for the development team. A manager may feel that after the developers have written the code a tester should test it before the code is deployed to production. This is the traditional waterfall view of testing. This approach assumes that testing the code after it is written will improve the quality of the code. The tester does not need to be able to read or write code, or have an involvement in planning or in the process of writing the code. 

However, testing at the end of development is expensive, it is more efficient to build quality into the product . In “Out of the crisis” Deming described visiting a factory in which all the finished items were tested and those items that failed the test went to a “rework” production line to be fixed. The factory manager said that 21% of staff time went in fixing substandard items, and that another 21% of his staff’s time went in making the substandard items. Deming argued that it would be better to build quality into the factory’s products at each stage in production, this way products could be released without having to fix substandard items. 

Job Two: Testers help a development team to build quality into the product

If the tester tests at the end of the development process then when the tester makes their tests the bugs are already in the code, and each time a bug is found the code needs to go back to the developer to be fixed. Testing at the end of development is rather like the factory that Deming visited. So, should the job that a tester be hired to do be to help build quality into the product?

As testers we can help to build quality into the product by:

  • Contributing to planning by helping to define the requirements of the functionality
  • Automating tests
  • Working with programmers to manually test their code
  • Questioning
  • Trying to give the customers point of view
  • Facilitating collaboration on testing and quality with designers, engineers and Product Owners
  • Tracking metrics on quality
  • Designing tests
  • Sharing knowledge about testing
  • Being an advocate for accessibility

If the tester helps the developers throughout the development process they can help the developers define the requirements of the code at an early stage and avoid bugs reaching production or even being created. A tester in this second “job” will add more value to the team as they are helping the team to build quality in rather than trying to test quality in at the end of development.

These are the two jobs that I have seen testers hired to do. Should a tester’s job be something else? What do you think?

I often “see” functionality I am testing as a tree diagram

Creating a tree diagram can be a helpful way to visualise functionality that is accessed via hierarchical structures like menus and the tree diagrams can help to plan testing. In my mind’s eye I often see functionality as a tree diagram.

C.S French says that trees are “hierarchical data structures rather like the familiar family tree” and that “the elements of a tree are called nodes” [1].  The example tree diagram below shows the settings menus on a website. 

Tree diagrams are hierarchical and show the items that you are analysing as nodes in a hierarchy like the hierarchy they are part of. This means that in the example above you see the menu items in the tree diagram similar to how they appear in the menu hierarchy in the app you are testing. 

In the diagram above each node is a menu item, if your diagram was for config options each node would be a different config option. 

It could also be that there are a series of choices that a user can make that are not in a menu. The user starts by choosing a or b, but then for either a or b there can be more choices and then for those choices there may also be more choices that can be made. The structure of these choices is a hierarchy that can also be analysed in a tree diagram.

Visualising the choices in the functionality you are testing as nodes on a tree diagram can help you plan to get the test coverage you want to achieve.You can show in the diagram which nodes you are going to test, or you can use the diagram to ensure that you do some testing of all the nodes. It could be that for the changes that you are testing you need to test all menu options, or it could be that you only need to test all the menu options at a particular level in the menu. Your tree diagram can sometimes contain more nodes than you have time to test. If there are more nodes in the tree diagram than you are going to test then being able to visualise your choices can make choices as to what to test.

Diagrams are also useful to share. If you have created a tree diagram you can share it with your team members and ask them to review it. The team can then see what you are going to test, and you can explain your reasoning. This is really helpful as it can improve understanding of the functionality and team members can offer suggestions that improve the diagram. The diagram can also be useful to share with managers, particularly if you need to show that there is insufficient time to test all the items.

I find tree diagrams are a useful way to visualise functionality that is accessed via menus as a tree diagram. I hope that you find tree diagrams useful too.

References

[1] Computer Science 4th Edition by C.S French

Design a site like this with WordPress.com
Get started