Skip to content

Metrics, Reporting, and Continuous Improvement

4.6 Metrics, Reporting, and Continuous Improvement

4.6.1 Metrics

Every project should collect a minimal but meaningful set of metrics. The focus is not on tracking everything, but on getting the right insights to make decisions.

  • Test coverage
  • How much of the system is tested?
  • Can be measured as % of requirements covered by tests or % of code covered by automated tests.
  • Helps ensure we are not leaving critical areas untested.
  • Bug trends
  • Track the number of bugs found, fixed, and outstanding.
  • Must include severity levels (Critical, Major, Minor) to show the impact.
  • Also look at bug leakage (issues found after release that escaped testing).
  • This helps understand both product quality and testing effectiveness.
  • Automation rate
  • Percentage of test cases automated vs. manual.
  • Helps measure progress toward more efficient regression testing and faster releases.
  • Performance benchmarks
  • Response time under normal load.
  • System behaviour under peak load.
  • Stability and resource usage (memory, CPU, etc.).
  • Ensures products meet customer expectations in real usage.
  • Security checks
  • Number of vulnerabilities found and resolved.
  • Coverage of security tests (static scans, penetration tests, etc.).
  • Ensures alignment with ISO 27001 and other security requirements.

4.6.2 Reporting

  • Test results and quality metrics must be shared regularly with stakeholders (Product Owners, Technical Leads, QA, and Architecture Board).
  • Reports should be simple and visual where possible (dashboards, charts).
  • Key checkpoints:
  • At the end of each sprint or iteration.
  • At major/minor release milestones.
  • After Beta testing or user acceptance testing

4.6.3 Continuous Improvement

  • Every project must conduct post-release reviews to identify lessons learned.
  • Root causes of critical bugs must be analysed and addressed.
  • Metrics and feedback must be used to improve test strategies, automation, and processes.
  • Standards and guidelines will be updated regularly by QA and the Architecture Board, based on project feedback and industry practices.