Test Execution Time

Applicability

Definition (Hover Text)

Source Tools

Jira, Azure Boards, Rally

Graph type

Line Chart

Filters

<None>

Hover Format on KPI

Sprint Name: <<Value>>

Fields on Explore

1. Sprint Name

2. Defect ID

3. Defect Description

4. Linked Story ID

5. Story Description

6. Root Cause

7. Defect severity

8. Defect priority

9. Defect status

10. Time spent

11. and more…

Business Logic

Calculation Formula

It's calculated by assigning weights to different severity levels (e.g., critical, high, medium, low) and then aggregating these weighted values to provide an overall index score.

Defect Severity Index (DSI) = Sum of (Defect * Severity Level) / Total number of defects

Example:

If a project has 3 critical defects (severity 5), 6 high defects (severity 4), 10 medium defects (severity 3), and 11 low defects (severity 2), the DSI would be calculated as: ((35) + (64) + (103) + (112)) / (3+6+10+11) = 3.03. 

KPI Settings

1. Custom field (requires processor run)

   1. Defect Severity

2. Issue types mapping
   
   

   Open

   

3. Defects field mappings

   Open

   

    

Trend

A higher DSI indicates a greater severity of defects, potentially requiring more urgent attention. 

Maturity Levels

None

Target KPI Value

Target KPI Value denotes the bare minimum a project should maintain for a KPI.

Global Configurations (Field Mapping)

Processor Fields

Whenever we update the defect mapping and issue type mapping, whether we add or remove any issue type, we must run the processor. This is necessary to show the changes in the KPI.

Defect Mapping :

Open

Mandatory fields

Project Settings

1. Navigate to Jira/ Azure Board Settings: Go to Settings → Projects → Edit Config → Mappings

2. Mandatory Field: In the Mapping section, you’ll find the Mandatory Field. This is where you’ll configure the necessary global mapping fields.

3. Configure the Fields:

Defect Mapping :

• Issue Type to identify Defect - all the issue type that signify defect

Open

How to Validate KPI

Suggested ways of working

1. Configure a standardized set of issue types and workflow statuses in the field mappings for your project

2. Avoid frequent changes to the settings as that may affect the trend

3. Verify the sprint snapshots of closed sprints

Sample JQLs

project = "Your Project Name" AND issuetype = Bug AND Sprint in openSprints() ORDER BY priority DESC

Best Practices

Shift-Left Testing

Involve QA early in the development cycle — review user stories, write test cases during grooming, and automate early validations through unit and API tests.

Strong Code Reviews

Enforce peer reviews focused on edge cases, exception handling, and non-functional requirements (performance, security, etc.).

Automated Regression Suites

Maintain robust automation across critical flows to catch regressions early before high-severity defects hit production.

Clear Acceptance Criteria & Definition of Done

Ensure all stories have well-defined acceptance criteria and that DoD includes test coverage, security checks, and performance validation.

Root Cause Analysis (RCA) & Defect Triage

Perform RCA for high-severity defects to identify systemic causes — whether it's lack of test coverage, unclear requirements, or technical debt.

Benefits of KPI

Prioritize Quality Focus

Highlights the impact of critical defects over just volume, helping teams focus on severity, not just count.

Risk Visibility

Surfaces high-risk areas in the product that may affect customer experience or stability.

Drives Preventive Action

Encourages root cause analysis for severe defects, promoting long-term quality improvements.

Release Readiness Decisions

Assists in go/no-go calls by indicating whether unresolved issues are minor or business-critical.