Role
Testing and QA
What do you drop first when a release comes faster than the testing? Usually the regression - and that is usually where the bug sits that comes back as a user report.
Intro call
Four questions we would start with
What do you drop first when a release date is tight?
The answer describes where the quality debt in your process arises.
How do you know the area changed in this release has been tested?
If it is because the tester thinks so, the coverage is a conviction rather than a fact.
How many times does the same bug come back after a fix?
If it happens regularly, what is missing is a test that would stop it - not a developer's attention.
What do you not do, even though you know you should?
Usually: you do not test the edge cases and the rare scenarios. Always for lack of time, never for lack of awareness.
We take over the preparation and the repetition, you keep designing the tests
Preparing test data, running regression, reporting results and maintaining the test documentation can be taken off the desk. What is to be tested and why is designed by the tester.
- Regression does not compete with new tests. - If the repeatable scenarios run themselves, the tester's time goes on what nobody has checked before.
- Test data does not block the start. - Preparing a data set for a scenario stops being a half-day task before every round of testing.
- Coverage is a fact, not an impression. - It is visible which areas changed in a release have tests and which do not - and that is the basis for a release decision.
Scope
What can be taken off a tester
Nineteen example tasks. Each one has a human approval point. Below is a selection, and the scope we fit to your development process.
Designing tests
- A draft of test cases from a requirement (assessment: the tester)
- Pointing out edge cases to consider
- Linking a case to a requirement and an area of the system
- A signal when a requirement has no test assigned
Test data
- Generating data sets for a scenario
- Data covering different states and configurations
- Restoring the state of the environment before a round of testing
- Anonymising data taken from production
Execution and reporting
- Running the repeatable scenarios without the tester
- Recording the result with the steps to reproduce
- A draft bug report with the context (content: the tester)
- A summary of the results of a test round
Regression testing
- Pointing out the areas covered by a change in the release
- A regression set matched to the scope of the change (decision: the tester)
- A signal about an area changed and not covered by a test
- Comparing the result with the previous round
Quality after release
- Linking users' reports to areas of the system
- Pointing out bugs an existing test should have caught
- Bugs summarised by area and cause
A day's work
Today
- You prepare test data by hand before every round
- You run regression by clicking through the same scenarios
- Under a tight deadline you test what you manage to
- You do not know whether the area changed in the release has any tests
- You write a bug report from scratch, with the steps
- After the release you do not know which bug could have been caught earlier
With AI assistance
- A data set for a scenario is produced with no half-day preparation
- The repeatable scenarios run themselves
- The time goes on cases nobody has checked before
- An area changed and not covered by a test is pointed out
- A bug report has a draft with the steps to reproduce
- Bugs after release are linked to gaps in the tests
- Test coverage
it is visible what is checked and what only looks checked
the cases are linked to requirements and areas
- Regression time
the regression fits before the release rather than after it
the repeatable scenarios run themselves
- Bugs found after release
fewer matters reaching users
the tester's time goes on new cases
- Repeatable testing
the same test gives the same result whoever runs it
the data and the state of the environment can be reproduced
We start with test data, because it most often delays the start of testing.
- Step 1
Data and a reproducible environment
Data sets for the scenarios and a repeatable starting state.
- Step 2
Repeatable scenarios
Running regression without the tester on every pass.
- Step 3
Coverage against the changes
Pointing out the areas changed and not covered by a test.
- Step 4
Quality after release
Linking reports to areas and to gaps in the tests. Last, because it needs production data.
Common questions
Questions that come up at the first meeting
Will this not lower the tester's standing in the team?
Usually the opposite, because the tester stops being the person clicking through regression and becomes the person designing what is worth checking at all. That is the harder part of this work and the more visible one to the team.
Will testers still be needed?
Generated tests do not find what matters - and we say so openly a few paragraphs below. A tester is needed to design what is worth checking at all, and that is the harder part of this work than clicking through regression. When regression runs itself, there is time for it. Teams that go through this usually end up with stronger QA, not smaller.
Does this fall under the AI Act?
Testing software is not a high-risk use. The only point needing care is using production data as test data - the data protection rules apply there and anonymisation is part of the scope rather than an add-on.
Still have a question that is not here? Book a call
Next step
How much time before the last release went on preparing test data?
We will show you what testing looks like when regression does not compete with new cases and the coverage is visible before the release decision.