Mike Harris' Testing and Quality Blog
An end-to-end test pack needs to run quickly so that it does not slow developers, and at the same time provide useful feedback to the developers. This makes deciding which categories of tests to include in the test pack challenging. A ten-minute video from Russ Ackoff has helped me better understand my decisions on one …
Continue reading “How do you decide which tests to automate?”
Sometimes testers are asked how long it will take to automate a batch of tests. Planning how long your test automation should be simple, however, the plan will have missed some requirements. There are three types of requirements[1]: “We will never understand all the requirements of a story ahead of time”[2], this applies to test …
I am using Playwright to automate end-to-end tests, and have started to complement my Playwright tests with Chromatic tests. The Playwright tests are really useful, but each test needs to run through several steps to create a scenario I want to test. Creating a Playwright test for each input variation for each part of the …
Continue reading “Do Chromatic tests complement Playwright tests?”
I am interested in how thinking slowly can help me test and I have been reading Gene Kim and Steven Spear’s new book Wiring the Winning Organization: Liberating Our Collective Greatness Through Slowification, Simplification, and Amplification with the Deming Profound Book Club. Spear and Kim acknowledge how Daniel Kahneman, who recently passed away, influenced them …
Continue reading “Think slowly when you are testing And Think slowly when you are automating”
When we are automating and maintaining tests we need good feedback on test runs. Playwright gives quick and useful feedback in its Trace Viewer which provides feedback on the run of tests. Trace Viewer’s feedback is one reason I like automating tests with Playwright. Through Trace Viewer you can: If a Playwright test fails, Trace …
Tests that are automated in Playwright run fast, which is great. However, sometimes they run so fast that assertions need to be used to stop the test being flaky. Playwright uses Auto-waiting to help it run tests fast. Auto-waiting means that Playwright performs several checks before making an action, for example, it checks that an …
Continue reading “Sometimes you need to assert that Playwright tests are not running too fast!”
Whichever technique is used to create tests they will contain assumptions about the nature of bugs. Each technique targets a different set of bugs. If development teams react to bug reports by fixing the bugs that have been reported and by finding and fixing similar bugs, then running the same tests is unlikely to find …
Continue reading “The Pesticide Paradox is a reason to do exploratory testing”
How long should an automated test pack take to run? Some people say that a certain number of minutes is too long, and others say that a different number of minutes is too long. How can we determine what is “too long”? And what can we do about it? What is an acceptable length of …
React is an open-source User Interface (UI) JavaScript library created by Facebook. I am doing some React tutorials to help me better understand the UI and what I am learning is helping me to automate tests. The React tutorials I am doing are here: https://react.dev/learn. The tutorials include exercises that can be done in a …
Continue reading “Gaining a knowledge of React is helping me automate tests”
I have started to automate tests in Playwright with TypeScript. I am now writing standards for the tests. I view writing the standards as a journey because I am adding to the standards as I learn more. I have started to create these standards because: I have found lots of useful advice in testing and …
Continue reading “Writing test automation standards is a journey, not a destination”
Recently it was pointed out to me that Playwright has a test generator and that I could use it to record tests. Many automated testing tools include a tool or extension that can be used to record tests. I don’t find them useful. My experience of using tools to record tests is that they record …
When I create or maintain an automated test I use a “learning loop” so that I learn from the test results. This enables me to create stable test packs that return accurate results. I automate tests with Playwright using TypeScript and with the low code tool Ghost Inspector. The “learning loop” is the same for …
We can make it easier to automate tests by giving feedback if incorrect values are passed as parameters. Developers can get feedback when they try to pass an invalid parameter if I use union types when using TypeScript. I have been using TypeScript to automate tests with Playwright. We often pass strings to functions as …
Continue reading “Make test automation easier with better feedback for parameters with TypeScript”
Automated tests can be run multiple times with different values by parameterising tests. Parameterising your tests can improve test coverage because a value in the test can be parameterised which means that the test can iterate through a number of values. This enables one automated test to cover a number of test cases. You could, …
Continue reading “Improve test coverage by parameterising tests with Playwright and other tools”
I often write javascript to manipulate the browser either as bookmarklets or as part of automated tests. OpenAI contains two AI tools that can provide solutions if I get stuck with a programming problem. OpenAI is a research organization focused on developing artificial general intelligence through natural language processing, robotics, and machine learning. Through OpenAI …
Continue reading “OpenAI contains tools that can help us with test automation”
TestAndAnalysis