Test management: strategy, risks, and estimates 🧪
Test management: the (im)possible craft 🧑💻
Managing tests is not just a matter of processes and documents: it’s an art of balancing business needs, quality, timelines, and — let’s be honest — a good dose of diplomacy. The test manager is the silent director orchestrating teams, tools, environments, and stakeholders, often trying to square the circle between impossible requests and limited resources.
Beyond writing the test plan, the test manager must:
- Define the test policy (why do we test?) and communicate it so everyone understands it (even those who think tests are a waste of time)
- Write the test strategy (what and how we test), adapting it to the context and the project’s evolution
- Launch, monitor, and adjust the testing process, solving problems before they become emergencies
- Verify exit criteria and make sure no one “escapes” ahead of schedule
- Produce clear and useful reports, not just to fill slides but to guide decisions
- Mediate between developers, PMs, QA, and business, often translating different languages (and swinging moods)
- Promote a quality culture, even when the temptation is “well, it works on my machine”
The test policy is the big-picture vision, the strategy is the plan of attack. Without one, the other isn’t much use. And without a good test manager, often neither is of any use at all.
Test plan: if you don’t plan, you plan to fail 📝
A test plan isn’t just a document to file away at the bottom of a dusty folder. It’s the compass that keeps you from shipwrecking among bugs, deadlines, and last-minute requests. Here’s what it can’t miss:
- Scope: what do we test (and what not)?
- Approach: how do we intend to test? Agile, waterfall, freestyle?
- Objectives: what do we want to achieve (besides surviving)?
- Risks and constraints: budget, time, resources, testability…
- Who does what: roles and responsibilities (spoiler: not everything is the test manager’s fault)
- Scheduling: when we start and, above all, when we finish
- Metrics: how do we measure success (or disaster)?
- Budget and templates: because Excel never lies
Test planning is a continuous process: you start with a draft, gather feedback, and update it. And you repeat, until the product goes into production (or the test manager changes jobs). Reference standard? ISO/IEC/IEEE 29119-3 (for those who love acronyms).
Test strategies: choose your weapon ⚔️
There’s no universal strategy, but at least seven ways to complicate (or simplify) your life:
- Analytical: risk-based tests. More risk, more tests. Simple, right?
- Model-based: you start from a model (business, state, reliability) and design tests accordingly.
- Methodical: checklists, defect taxonomies, standards. For rule lovers.
- Process compliant: you follow an external standard (ISO, industry, company processes). Rigorous, but audit-proof.
- Direct: you listen to the experts (even those who don’t test). Consultative, but often enlightening.
- Regression averse: you bet everything on automation and test reuse. Perfect for those who hate surprises.
- Reactive: you test while developing, often in exploratory mode. For adrenaline lovers.
Entry & exit criteria: when to start and when you can head to the bar 🚦
Entry criteria: the Definition of Ready
If the prerequisites aren’t there, no testing (and no excuses). Entry criteria exist to avoid starting testing when essential pieces are missing: incomplete specs, an environment that isn’t ready, missing test data, unconfigured tools. A good entry criterion is like the bouncer at the door: if you don’t have everything, you’re not getting in.
Practical examples of entry criteria:
- All requirements have been approved and documented
- The test environment is operational and stable
- Test data is available and validated
- Technical dependencies (APIs, external services) are accessible
Exit criteria: the Definition of Done
When all tests pass, critical bugs are closed, and the PM is smiling, you can close it out (maybe). Exit criteria exist to avoid declaring “done” too early: they must be objective, measurable, and shared. If they aren’t clear, you risk staying in testing forever (or going to production with a few “surprises”).
Practical examples of exit criteria:
- All planned test cases have been executed
- Critical and major bugs have been fixed or accepted
- Test coverage has reached the agreed threshold (e.g. 80%)
- Test reports have been shared with stakeholders
Tip: define entry & exit criteria at the start of the project and update them as needs change. This way you avoid endless discussions and can really head to the bar guilt-free.
Risk analysis: the real extreme sport 🧨
If you thought bungee jumping was risky, try managing risk in a software project! Risk analysis is the heart of quality: it helps predict where (and how) the project might derail, and put countermeasures in place before it’s too late. It’s not just a formality: it’s what separates a team that “hopes everything goes well” from one that actually knows what it’s doing.
Product risk: when the software makes customers angry 😡
Product risks are the ones that directly impact the end user. If something goes wrong here, get ready for scorching reviews and endless support calls.
- The software doesn’t do what it promises (missing features, blocking bugs)
- It ignores users’ real needs (confusing UX, unintuitive flows)
- Inadequate architecture, wrong calculations, sloth-like performance, neglected security…
Practical examples:
- An e-commerce site that doesn’t correctly calculate shipping costs
- A banking app that shows incorrect balances
- A management system that freezes with too much data
Product risks are also called quality risks: if you don’t manage them, quality goes out the window.
Project risk: when the project derails 🚂
Project risks undermine timelines, costs, and overall quality. This is where the real game is played between a “successful project” and a “nightmare project”.
- Delays, evaporating budget, last-minute changes (the classic “scope creep”)
- Insufficient resources, wobbly priorities, nightmare communication
- Test environment not ready, vendors disappearing, tools that don’t work
Practical examples:
- The development team changes halfway through the project
- The customer asks for new features a week before release
- The test environment differs from production (and bugs only emerge after deployment)
Managing project risk also means knowing how to say “no” (or at least “not now”) when needed.
How do you fix it?
The test manager doesn’t have a magic wand, but can:
- Hire contractors to fill temporary gaps
- Work at night (but only if strictly necessary…)
- Improve estimates and communicate risks promptly
- Manage changes with clear, shared processes
- Promote retrospectives and discussion moments to learn from mistakes
Risk management is a continuous process: better to prevent than to cure (and better to cure than to ignore).
Risk-based testing: theory is nice, practice is better 🧮
Risk-based testing (RBT) is the strategy that puts risks at the center of test decisions. It’s not just about “doing more tests where there’s more risk”, but about choosing techniques, priorities, and depth based on what can really go wrong.
- Impact: how bad is it if something goes wrong? (1-5 scale, where 5 is “total disaster”)
- Probability: how likely is it to happen? (1-5 scale, where 5 is “happens every time it rains”)
- Risk level = impact x probability
Practical example:
| Function | Impact | Probability | Risk level (I x P) |
|---|---|---|---|
| Payment | 5 | 4 | 20 |
| Login | 4 | 3 | 12 |
| Email notifications | 2 | 4 | 8 |
| Reporting | 3 | 2 | 6 |
Testing is a form of risk mitigation: fewer surprises, fewer sleepless nights. And remember: theory is useful, but practice (and experience) make the difference.
Estimation techniques: how long does it really take? ⏳
Estimating the time (and cost) of testing is an art that improves with experience, but a few tricks of the trade can help avoid nasty surprises and impossible deadlines.
- Break down the activities: split the work into small, clear tasks (analysis, writing test cases, execution, automation, reporting…)
- Use historical data: if you’ve already done similar tests in the past, start from there. Historical metrics are more reliable than gut feelings.
- Estimate by analogy: compare the new project with others you’ve already tackled (“how long did it take last time we tested a login?”)
- Involve the team: ask whoever will actually run the tests how much time is really needed. Expert judgement is often more accurate than a thousand spreadsheets.
- Add a margin: unexpected issues and bugs are always around the corner. A 10-20% buffer can save your reputation (and your weekend).
- Update the estimates: as the project evolves, update the estimates. Better to correct course than to sail straight into the iceberg.
Practical estimation example:
| Activity | Estimate (days) |
|---|---|
| Requirements analysis | 2 |
| Writing test cases | 3 |
| Manual test execution | 4 |
| Test automation | 5 |
| Reporting | 1 |
| Total | 15 |
Remember: the perfect estimate doesn’t exist, but a reasoned (and updated) estimate is better than a promise made at random.
In summary
Test management isn’t just bureaucracy, but the art of preventing disasters, optimizing resources, and — why not — making everyone’s life a little simpler. And remember: those who don’t plan, plan to fail. 😉