Skip to content

Quality Engineering Tools and Standards

4.3 Quality Engineering Tools and Standards

This section defines the core practices and tools that ensure consistent, high-quality software delivery at DHI. It covers standards for CI/CD, code quality, and test automation, supported by a recommended set of tools. Existing tools may remain if they are consistent within the product.

4.3.1 CI/CD Integration

  • DHI's preferred CI/CD provider is Azure DevOps/GitHub Actions.
  • Automated tests (unit, integration, regression, performance where applicable) must run in pipelines.
  • Static code analysis must also be integrated into pipelines.
  • Users are encouraged to review the CI/CD guideline for more detailed information about configuration, standards, and recommended practices.

4.3.1.1 Code Analysis

Code analysis is the process of reviewing and evaluating source code to understand its structure, behaviour, and potential issues. It helps developers improve code quality, find bugs, enforce coding standards, and ensure software reliability and maintainability.

There are two types of code analysis:

4.3.1.1.1 Static Code Analysis (Required)

Static code analysis reviews the source code without executing it. It helps detect issues early in the development process, before testing or deployment.

Purpose:

  • Enforce coding standards and best practices.
  • Identify potential bugs and security issues.
  • Improve maintainability and reduce technical debt.

When to apply:

  • Every commit or pull request should trigger static code analysis in the CI/CD pipeline.
  • Results must be reviewed and fixed before merging into main.

Examples of what it detects:

  • Unused variables, null reference risks.
  • Violations of naming or formatting conventions.
  • Potential security vulnerabilities (e.g., unsafe input handling).
  • Complexity or dependency issues in the code.

Outcome: A static code analysis report should be visible in the build pipeline or dashboard, highlighting issues by severity. Critical and Major issues must be fixed before merge. Minor issues can be tracked and resolved later.

Implementing static code analysis depends on the language being used.

Language Recommended Tool Other Valid Tools
C#/.NET Roslyn Analyzers ReSharper, SonarQube
Python Ruff Pylint, SonarQube
Java JFrog SonarQube
JavaScript/TypeScript ESLint TSC, OxLint
C/C++ Clang-Tidy SonarQube

Dynamic analysis reviews code during execution to detect issues that only appear at runtime.

Purpose:

  • Identify memory leaks, performance bottlenecks, and runtime exceptions.
  • Validate system behaviour under real operating conditions.
  • Complement test automation and monitoring.

When to apply:

  • During integration, system, or performance testing.
  • Especially valuable for long-running or compute-intensive components (like simulation engines or cloud microservices).

Examples of what it detects:

  • Memory leaks or unclosed file handles.
  • Performance degradation under load.
  • Threading and concurrency issues.
  • Runtime exceptions not caught by unit tests.

Outcome: Dynamic code analysis helps teams improve reliability and efficiency. Results should feed into post-build reports or monitoring dashboards (see Monitoring and Observability Guidelines for further details).

Language Recommended Tool Other Valid Tools
C#/.NET Visual Studio Performance Profiler PerfView
Python Py-spy tracemalloc
Java Java Flight Recorder (JFR) + Java Mission Control (JMC) Async Profiler
JavaScript/TypeScript Chrome DevTools (incl. Node Inspector) Node
C/C++ Clang/GCC Sanitizers (ASan, TSan, UBSan) Valgrind

4.3.1.2 Unit Testing – Tools and Standardisation

Unit testing is the foundation of our quality approach. All new backend components must have unit tests, and we should plan how to introduce unit tests for existing components that currently do not have coverage.

Frameworks

The unit testing framework depends on the programming language. Below are the recommended frameworks at DHI, along with valid alternatives when needed:

Language Recommended Framework Frameworks valid for existing unit tests and products/components
C#/.NET xUnit MSTest, NUnit
Python unittest (built-in) pytest, nose
Java JUnit TestNG
JavaScript/TypeScript Vitest Jest, Mocha
C/C++ GTest Catch2
Guidelines for Using Frameworks
  • New components and applications should use the recommended framework for their language.
  • Existing products do not need to switch to the recommended framework if they already have unit tests running.
  • A single product should not mix different unit test frameworks — consistency makes maintenance easier.
How We Standardise Unit Testing at DHI
  • Unit tests must be part of the CI pipeline and run automatically on each commit.
  • They should test one function, class, or module at a time and avoid external dependencies such as databases, APIs, or file shares.
  • Mocking or stubbing should be used for all external interactions.
  • Projects should follow a simple folder structure such as: src/ for code and tests/ for unit tests.
  • Test names should be clear and descriptive so anyone can understand the intention of the test.
Unit Test Coverage

Unit test coverage helps us understand how much of the code is actually being tested. Coverage should always be measured automatically as part of the build pipeline when the unit tests run.

Recommended Coverage Tools

Different languages use different coverage tools. Below are the recommended options at DHI and valid alternatives:

Language Recommended Tool Other Valid Tools
C#/.NET Coverlet dotCover (JetBrains)
Python Coverage.py —
Java JaCoCo —
JavaScript/TypeScript Istanbul / nyc —
C/C++ Gcov —

Coverage Expectations

The coverage level depends on the type and importance of the component. The goal is not to chase 100%, but to ensure meaningful and reliable tests where it matters most.

Component Usage Required Coverage
Standard Backend Components >70%
Critical Backend Components >80%
Less critical, limited-use components >50%

These targets help us maintain a consistent testing standard across DHI while giving teams flexibility based on the complexity and importance of their components.

4.3.1.3 Integration Testing – Tools and Standardisation

Integration testing helps us verify that different parts of a system work together as expected. At DHI, this applies to both desktop products (like MIKE and FEFLOW) and cloud applications. Because our product landscape is mixed, the integration testing approach differs slightly depending on the technology.

Integration testing focuses on:

  • How modules communicate with each other.
  • How data flows between components.
  • How services, APIs, or desktop modules behave when combined.
  • Whether the system still behaves correctly when real dependencies are involved.

This is not UI testing and not full end-to-end testing — it sits in the middle.

For systems that use APIs, microservices, or databases, we follow a more traditional integration testing approach:

Area Recommended Tools Other Valid Tools
API Integration TestContainers, Postman + Newman CLI Pytest, MSTest, Swagger/NSwag clients
Service-level Integration TestContainers Docker-based custom harnesses
Database Integration TestContainers, EF Core testing tools, pytest + sqlite Local test DB setups

Desktop applications don't use REST APIs. Instead, integration testing focuses on internal modules, native APIs, shared libraries, and computational engines.

Approaches:

  • Integration via desktop API layers (e.g., scriptable APIs, engine interfaces).
  • Testing interactions between computation engine, UI logic, and file formats.
  • Verifying modules such as: pre-processing ↔ computation engine, engine ↔ post-processing, and file I/O ↔ model execution.
Area Recommended Tools Other Valid Tools
Desktop API Integration Python API harnesses (MIKE/FEFLOW APIs), MSTest or pytest-based wrappers Custom command-line harnesses
Engine/Component Testing Batch runners + assertions, Python automation —
File-Based Integration

These tests are often run through scripting interfaces or command-line tooling, rather than UI automation.

How We Standardise Integration Testing at DHI
  • Include integration tests in CI. Tests should run automatically for main branches or merge requests.
  • Integration tests must use controlled test environments and must not use production data or services.
  • Test data must be version-controlled, stable, and reusable. This avoids flaky tests and ensures results are predictable.
  • Integration tests should be narrow and only test module interactions.
  • Automate setup and teardown. Use Docker/TestContainers for cloud systems. Use scripted harnesses for desktop engines. Avoid manual steps or local machine dependencies.

4.3.1.4 UI Testing – Tools and Standardisation

UI testing focuses on verifying that users can interact with the system as expected. At DHI, UI testing applies to web applications, desktop software (MIKE, FEFLOW), and hybrid platforms. Because our landscape is diverse, our focus is on using the right tools for the right platform without overcomplicating the workflow.

UI testing complements unit and integration tests. It should focus only on what truly needs UI validation.

When doing automated testing, the tool to use depends on the platform. New components and assets should use the recommended framework as they support running in pipelines. Existing automated UI tests are allowed to use other valid tools.

Platform/Framework Recommended Tool Other Valid Tools
Web applications Playwright Selenium, Cypress
WinForms, WPF WinAppDriver Ranorex
Qt Squish Ranorex
Avalonia Avalonia.ReactiveUI with xUnit Selenium-based setups
How We Standardise UI Testing at DHI
  • UI automated tests are optional but recommended for critical user workflows.
  • Keep the number of UI tests small, as UI automation is fragile.
  • All UI tests must run automatically in CI pipelines.
  • Desktop UI tests must avoid relying on local machines; use virtualised or containerised agents where possible.

4.3.1.5 Performance Testing

Performance testing ensures our software behaves reliably under real-world load and stress. This applies to web applications, cloud systems, APIs, and simulation engines. It helps us validate response time, throughput, scalability, stability during long runs, and behaviour under heavy load.

Platform/Framework Recommended Tool Other Valid Tools
Web, Cloud APIs JMeter, K6 Locust, Gatling
Simulation engines (to be confirmed)
How We Standardise Performance Testing at DHI
  • Performance testing is recommended for critical systems.
  • Tests should run in stable, dedicated environments that mimic production.
  • Avoid running performance tests on shared or development environments.

4.3.1.6 Regression Testing – Tools and Standardisation

Regression testing ensures new changes do not break existing functionality. At DHI, regression testing applies to every product type — cloud systems, desktop applications, APIs, and simulation engines.

Regression testing helps confirm:

  • Previously working features still behave correctly.
  • Bug fixes do not reappear.
  • Updates in one module don't break others.
  • Releases remain stable across versions.
How Regression Testing Should Be Done at DHI

Regression testing can be carried out through a mix of automated and manual tests.

Automated Tests:

  • Unit tests.
  • Integration tests.
  • UI tests (selected critical workflows).

Most regression coverage should come from unit and integration tests, not UI tests.

Manual Tests:

  • When automation is not feasible.
  • When workflows are too visual or too complex.
  • For validating complex scientific modelling scenarios (MIKE/FEFLOW).

Regression testing does not require a single tool. It uses existing automation layers (unit tests, integration tests, UI tests). The tools are mentioned in the respective sections above.

How We Standardise Regression Testing at DHI
  • Regression tests must be part of the CI pipeline.
  • All critical user flows should have some form of regression coverage.
  • Desktop applications should maintain reference datasets for validation.
  • Cloud services should maintain a minimal regression suite for every API version.
  • Release cycles must include regression execution and reporting.