Testing Methodology Selection Criteria
3.4 Testing methodology selection criteria¶
Not all digital solutions at DHI require the same testing approach. A small internal tool with limited users has different testing needs than a large, customer-facing MIKE application or a multi-tenant cloud platform. Similarly, a simple feature with stable requirements needs less intensive testing than a complex system with many integrations.
This section provides a framework for selecting appropriate testing methodologies based on six key factors that characterise your project or solution. By assessing these factors, teams can determine which types and levels of testing are most critical for their specific context, ensuring efficient use of resources while maintaining quality standards.
3.4.1 The Selection Criteria¶
| Factors | Unit Test | Integration Test | System Test | Regression Test | UAT | Beta Test | Non-functional Test | Smoke / Sanity Test |
|---|---|---|---|---|---|---|---|---|
| Project Size | ||||||||
| Small | Mandatory | Recommended | Recommended | Optional | Optional | N/A | Optional | Mandatory |
| Large | Mandatory | Mandatory | Mandatory | Mandatory | Mandatory | Mandatory | Mandatory | Mandatory |
| Exposure & Risk | ||||||||
| High | Mandatory | Mandatory | Mandatory | Mandatory | Mandatory | Mandatory | Mandatory | Mandatory |
| Low | Mandatory | Recommended | Recommended | Recommended | Optional | N/A | Recommended | Mandatory |
| Complexity | ||||||||
| High | Mandatory | Mandatory | Mandatory | Mandatory | Mandatory | Mandatory | Mandatory | Mandatory |
| Low | Mandatory | Recommended | Recommended | Recommended | Optional | N/A | Optional | Mandatory |
| Requirements Stability | ||||||||
| Stable | Mandatory | Mandatory | Mandatory | Mandatory | Recommended | Recommended | Recommended | Mandatory |
| Volatile | Mandatory | Recommended | Recommended | Optional | Optional | Optional | Optional | Mandatory |
| Available Resources | ||||||||
| Tight | Mandatory | Recommended | Recommended | Recommended | Recommended | Recommended | Recommended | Mandatory |
| Ample | Mandatory | Mandatory | Mandatory | Mandatory | Mandatory | Mandatory | Mandatory | Mandatory |
Note:
- Unit testing and Smoke/Sanity testing are baseline expectations for all deployable solutions at DHI (T&I).
- Non-functional testing (Performance, Security, reliability) is mandatory for exposed and critical systems.
- Beta testing applies only to customer-facing products and is not applicable to internal tools or bespoke customer projects.
3.4.1.1 Project/Solution Size¶
Size refers to the scale and scope of the solution being developed, typically measured by development effort, number of features, or codebase complexity.
-
Small projects - Focus on unit, smoke/sanity, and minimal system testing with automated unit tests
-
Large projects - Include unit, integration, system, regression, UAT, beta, and relevant non-functional tests (performance, security, reliability). Automation and a structured testing strategy are critical.
3.4.1.2 Exposure & Risk¶
Exposure measures the potential impact of bugs on users, business operations, and DHI's reputation.
- High exposure / critical systems (e.g., public-facing MIKE Cloud, client deliverables) - Require all testing levels, including security, performance, usability, and compatibility tests. Regression and beta testing are essential.
- Low exposure / internal tools - Can prioritise unit, integration, and system testing
3.4.1.3 Complexity¶
Complexity is determined by the number of modules, integrations, dependencies, and workflows within the solution.
- High complexity → Must include integration, system, regression, and non-functional testing to validate all interactions and dependencies.
- Low complexity → Focus on unit, system, and functional testing
3.4.1.4 Requirements Stability¶
Stability refers to how frequently requirements change during the development lifecycle.
- Stable requirements → Invest in automation for unit, regression, and integration tests
- Volatile requirements → Prioritize smoke/sanity and functional system testing; automate only stable parts.
3.4.1.5 Available Resources¶
Resources include time, budget, tools, and team capacity available for testing activities.
- Tight timelines → Prioritise smoke/sanity, critical functional tests, and targeted regression.
- Ample resources → Full spectrum of unit, integration, system, UAT, beta, and non-functional testing with comprehensive automation.