WoW - APM Tools

Overview

PSknowHOW support collecting data from Jira & Azure Boards. The basic of SCRUM or KANBAN need to be followed for KnowHOW to be calculate all KPI at project level and aggregate at above levels.
Following are the highlights of expected ways of working…

Any project can be configured with either Jira OR Azure.Based on the default setting of PSKnowHOW, all of the required fields will be populated. If a user wants to change the default setting, below are some key points to be considered.
SCRUM Dashboard Filters operates on the basis of Sprint and on KPIs, be default last 5 sprints data are shown, which can be configured to show upto 15 sprint.

Iteration Board KPI’s

Iteration board is configured to seek data from different Scrum projects. Based on selected projects and sprint, Iteration KPI’s shall populate with corresponding data points.

^{key Points -}
- Only Scrum projects are supported.
- The selected project needs to have at least one sprint available for the KPI’s to show up.
- Any user other than Superadmin shall have access to a Scrum project for data to be displayed on the Iteration board.
- Any update on the Iteration board shall be followed by refreshing the data from Itertaion dashboard for a quick data update on Iteration board.

KPI’s Based on Defects -

Defects should be linked to Stories. defects that are not linked with stories will not be considered for these KPI data calculations.
For DSR, the defect should have an identifier discovered on production.
All defects should be linked with a story, even if the story is part of the previous Sprint which has been closed.
Defect status should be reflected via the appropriate stage in the workflow, rather than using comments or using another status.
- Resolved defects should be reflected via their status as done/closed.
- Rejected defects should be reflected via their status as canceled or resolution as configured in PSKnowHOW.
  Similarly, it should be followed for other stages in the defect life cycle like Open, In Progress, In Development, Ready for QA, etc.
RCA drop-down field should be added in the Defect issue type. RCA values drive the info for RCA KPI.
Any Defect/Story being considered to work upon in a particular sprint shall be tagged to same.

KPI’s Based on Test Cases -

All Test cases should be linked with a Story (to get In Sprint Automation Percentage value).
if custom fields are used to identify regression and in-sprint test cases, check the custom fields related to it on the setting screen of PSKnowHOW.
Regression KPI can either be set up by Label or Customfield and is not dependent on Sprint.
Only those Test Cases which are to be automated will be considered for calculation and check the corresponding setting.
Test Execution Percentage and Capacity to be updated manually from the settings > upload.

Azure Boards Limitations

Area	Behavior	Guideline

Area	Behavior	Guideline
Sprint Status	Last available sprint of the project will show as CURRENT as long as subsequent sprints are not defined, even when the sprint’s end date is out of sync with the calendar date	Projects using Azure Boards must create future sprints in advance to avoid data discrepancies KnowHOW will generate sprint snapshots based on the Sprint Status only, i.e., a sprint will be considered as Active as long as the status is CURRENT in Azure Board, irrespective of the end date of the sprint
Area Path	If workitems are tagged across different area paths and the area paths are not included in the WIQL, KnowHOW will not fetch those as they don’t satisfy the criteria	At KnowHOW, project boards should be configured in such a way that it requires minimal updates
Scope Changes	Daily snapshots are formed at KnowHOW during an active sprint. The final snapshot is frozen on closure of the sprint.	KnowHOW compares the latest snapshot with the previous day’s snapshot to determine the following scope changes Workitems added Workitems removed Snapshot captured on Day 1 of the sprint is considered the Initial Scope of the sprint The sprint snapshots will not get update based on changes that happen after the sprint is completed
Scope Completion		KnowHOW compares the latest snapshot with the previous day’s snapshot to determine the following completed workitems not completed workitems
Spilled issues		Not supported

KPI Level Practise Guide -

Segment	KnowHOW KPI	Best practices for KnowHOW integration	Minimal required Jira configuration

Segment	KnowHOW KPI	Best practices for KnowHOW integration	Minimal required Jira configuration
Speed	Sprint Velocity	Estimation completed on story points before start of iteration Story point baselining should be practiced (i.e. start with base story with story point 1, and compare rest other stories with base story to estimate)	Estimation field in Jira story should be only “Story Point”. Any other estimation field on standard issuetype (i.e. Story etc) should be avoided. Sprint should be added in standard issuetype (i.e. Story)
	Commitment Reliability	Avoid scope change Each planned story must have Sprint value set before start of sprint.	Nothing additional
	Sprint Predictability	Avoid scope change Each planned story must have Sprint value set before start of sprint.	Nothing additional
	Issue Count	None	Nothing additional
	Sprint Capacity Utilization	Time tracking (Original estimate, Remaining Estimate, Time Spent) field should be captured on sub-task level Original estimate should filled in each sub-task before start of sprint Time spent should captured before each end of the day during sprint execution	Time tracking (Original estimate, Remaining Estimate, Time Spent) field should be added only on sub-task level
	Code Build Time	Integration with CI/CD, Repo tools	Nothing additional
	Scope Churn	Estimation planning on story points Sprint tagging to the issues before sprint start	Story point should be added for estimation in standard issuetype (i.e. Story) Sprint should be added in standard issuetype (i.e. Story)
	Build Frequency	Integration with CI/CD tools	Nothing additional Jira integration with one of the CI/CD tool (Jenkins, Bamboo, Github Action, Azure Pipeline, Teamcity, Argo CD)
Quality	Defect Injection Rate	All logged defects must get linked with stories planned in sprint.	Provision to mandate that, if defect created then must get linked with story
	First Time Pass Rate	Run time status update All logged defects must get linked with stories planned in sprint.	Provision to mandate that, if defect created then must get linked with story
	Defect Density	All logged defects must get linked with stories planned in sprint. Root cause field is updated	Provision to mandate that, if defect created then must get linked with story Root cause field is configured in defects
	Defect Seepage Rate	Run time status update All logged defects must get linked with stories planned in sprint. Escaped defect field is configured	Provision to mandate that, if defect created then must get linked with story Escaped defect field is configured in defects
	Defect Removal Efficiency	Defects are planned in sprint	Nothing additional
	Defect Rejection Rate	Defects are planned in sprint	Nothing additional
	Defect Count By Priority	Link all defects with the stories in the sprint by priority	Nothing additional
	Defect Count By RCA	Link all defects with the stories in the sprint by root cause	Root cause field is configured in defects
	Created vs Resolved Defects	Defects are planned in sprint	Nothing additional
	Regression Automation Coverage	In test management tool, select regression, automation options and complete traceability by mapping with story	Nothing Additional
	In-Sprint Automation Coverage	In test management tool, select regression, automation options and complete traceability by mapping with story	Nothing Additional
	Unit Test Coverage	Integration with SonarQube
	Sonar Violations	Integration with SonarQube
	Tech Debt - Sonar Maintainability	Integration with SonarQube
	Sonar Code Quality	Integration with SonarQube
	Test Execution and pass percentage	Integration with test management tool	Nothing additional
Value	Release Frequency	Link Jira issues tagged with the fix version The release should have a start and end date	Nothing additional Jira integration with one of the CI/CD tool (Jenkins, Bamboo, Github Action, Azure Pipeline, Teamcity, Argo CD) Check the provision to mandate release start and end date when added fix version.
	Value Delivery (Cost of Delay)	Epic (considered as feature in JIRA) should have the fields User-Business Value, Time Criticality, Risk Reduction and Opportunity Enablement	Epic should have User-Business Value, Time Criticality, Risk Reduction and Opportunity Enablement fields configured
	PI Predictability	Planned and Actual value should be added in the Epic (considered as feature in JIRA)	Epic should have Planned Value and Actual Value fields configured
Iteration	Iteration Commitment	Estimation planning on story points Before the start of the sprint, all items should have an estimate and added to the sprint	Story point should be added for estimation in standard issuetype (i.e. Story)
	Planned Work Status	The due date must be added to the issues	Duedate should be configured in JIRA
	Quality Status	All logged defects must get linked with stories planned in sprint.	Nothing additional
	Dev Completion Status	Dev due date must be added to the issues	Dev duedate should be configured in JIRA
	Work Remaining	Time tracking (Original estimate, Remaining Estimate, Time Spent) field should be captured on sub-task level Original estimate should filled in each sub-task before start of sprint Time spent should captured before each end of the day during sprint execution	Time tracking (Original estimate, Remaining Estimate, Time Spent) field should be added only on sub-task level
	Defect Count By Status	All logged defects must get linked with stories planned in sprint.	Nothing additional
	Defect Count By RCA	Link all defects with the stories in the sprint by root cause	Root cause field is configured in defects
	Unplanned Work Status	The due date must be added to the issues	Duedate should be configured in JIRA
	Defect Count By Priority	Link all defects with the stories in the sprint by priority	Nothing additional
	Closure Possible Today	The due date must be added to the issues Run time status update Time spent should captured before each end of the day during sprint execution	Duedate should be configured in JIRA Time tracking (Original estimate, Remaining Estimate, Time Spent) field should be added only on sub-task level
	Issues Likely to Spill	The due date must be added to the issues	Duedate should be configured in JIRA
	Wastage	Runtime status update	Nothing additional
	First-Time Pass Rate	Run time status update All logged defects must get linked with stories planned in sprint.	Provision to mandate that, if defect created then must get linked with story
	Risk and Dependencies	Link all risk and dependencies in a sprint	Nothing additional
	Estimation Hygiene	Time tracking (Original estimate, Remaining Estimate, Time Spent) field should be captured on sub-task level Original estimate should filled in each sub-task before start of sprint Time spent should captured before each end of the day during sprint execution	Time tracking (Original estimate, Remaining Estimate, Time Spent) field should be added only on sub-task level
	Estimate vs Actual	Time tracking (Original estimate, Remaining Estimate, Time Spent) field should be captured on sub-task level Original estimate should filled in each sub-task before start of sprint Time spent should captured before each end of the day during sprint execution	Time tracking (Original estimate, Remaining Estimate, Time Spent) field should be added only on sub-task level
	Iteration Burnup	Due date in all issues in a sprint	Nothing additional
	Daily Standup	None	Nothing Additional
Developer	Mean Time to Merge	Integration with the Repo tool	Nothing additional
	PIckup Time	Integration with the Repo tool	Nothing additional
	PR Size	Integration with the Repo tool	Nothing additional
	Rework Rate	Integration with the Repo tool	Nothing additional
	Check-Ins & Merge Requests	Integration with the Repo tool	Nothing additional
Release	Release Burnup	Link all issues with the release The release should have a start and end date Dev duedate & Due date must be added to issues Estimation planning on story points	Dev duedate and Due date should be configured in JIRA
	Release Plan	Link all issues with the release The release should have a start and end date Dev duedate & Due date must be added to issues Estimation planning on story points	Dev duedate and Due date should be configured in JIRA
	Release Progress	Link all issues with the release The release should have a start and end date	Nothing Additional
	Defect Count By Status	Link all defects with the release	Nothing Additional
	Defect Count By Assignee	Link all defects with the release Defects should be assigned	Nothing Additional
	Defect Count By RCA	Link all defects with the release Root cause field is updated	Root case field should configured in JIRA
	Defect Count By Priority	Link all defects with the release Add priority	Nothing Additional
	Defect By Testing Phase	Link all defects with the release Add Testing Phase	Configure testing phase in Defects
	Epic Progress	Runtime status update of epic in JIRA	Nothing Additional
DORA	Change Failure Rate	Integration with the SCM tools	Nothing Additional Jira integration with one of the CI/CD tool (Jenkins, Bamboo, Github Action, Azure Pipeline, Teamcity, Argo CD)
	Deployment Frequency	Integration with the SCM tools	Nothing Additional
	Lead Time for Change	Integration with the SCM tools	Nothing Additional Jira integration with one of the CI/CD tool (Jenkins, Bamboo, Github Action, Azure Pipeline, Teamcity, Argo CD) Check the provision to mandate release start and end date when added fix version.
	Mean Time To Recover	Create and close Incident on run time in JIRA	Configure Incident issuetype in JIRA
Backlog	Backlog Readiness	Estimation completed on story points before start of iteration Story point baselining should be practiced (i.e. start with base story with story point 1, and compare rest other stories with base story to estimate)	Estimation field in Jira story should be only “Story Point”. Any other estimation field on standard issuetype (i.e. Story etc) should be avoided. Sprint should be added in standard issuetype (i.e. Story)
	Iteration Readiness	None	Nothing Additional
	Production Defects Ageing	Link all defects with the release Add Testing Phase	Configure testing phase in Defects
	Defect Reopen Rate	None	Reopen transition and status should configured in defects from “In Testing”, “Ready for testing”, In UAT & Closed