Implementing a successful metrics system in an IT company Muntean Ramona Measurements & Best Practices @ISDC
IT’S IMPORTANT TO YOU IT’S REDUCING UNCERTAINTY TELLS YOU MORE THAN YOU KNEW BEFORE
Better informed decisions Tracking the progress of the objectives Managing the achievements of goals
WE CAN MANAGE WHAT WE CAN EASILY MEASURE
GOALS Business growth rate Profit margin Bids success rate Personnel turnover
SUCCES FACTORS Productivity Predictability Quality Competitiveness Client Satisfaction Employee Satisfaction
Goal
Success Factors Key Performance Indicators Metrics / Data
Projects, Processes, People
TOP DOWN APPROACH – COMPANY GOALS
QUALITY Cost of Quality (CoQ) Cost of Appraisal (CoA)
Reviews (1st time) Code Requirements Design Test plans, test scripts
Testing (1st time) Audits
Cost of Prevention (CoP)
Cost of Non-Conformance (CNC)
Training Procedures, methods Tools, systems Planning Quality improvement Measurements
Fixing defects Re-work code review Re-work doc review Re-reviews Re-testing
QUALITY TRENDS
Improve
KPI’s Productivity Predictability Quality Client Satisfaction Team Satisfaction
Metrics, Trends
Key Performance Indicators
METRICS ANALIZED Velocity: Story Points/sprint Estimation deviation %Re-work, DDR, CoQ trends Demo / Release success score
Tooling, Data
Projects, Processes, People
BOTTOM UP APPROACH – PROJECT’S KPI
TRENDS Story Points done / sprint 60 55 50 45 40 35 30 25 20 15 10
55 43
41
43 32 29 23
22
S4
S5
S6
S8
S9
S10
22
S11
S12
S13
TRENDS Defect Detection Rate (DDR) 90.00%
We can learn how review is influencing the product quality in correlation with defects found by testers.
80.00% 70.00% 60.00% 50.00% 40.00% 30.00% 20.00% 10.00% 0.00%
S4
S5
S6
DDR_Review
S7
S8 DDR_Testing
S9
S10
S11
DDR_Customer
S12
S13
DEVELOPING MEASUREMENT PROCESS DO
PLAN Define procedures:
Collect metrics:
- Metrics to be collected - Collection practices & tooling - Analysis & reporting - Feedback integration
- Tailor procedures (if the case) - Record and store data - Use proper tools & templates - Check data accuracy
Identify scope: - Goals, objectives - “Problems� that need answer - Measurements needs
Evolve process: - Evaluate progress - Evaluate process - Recommend improvements
ACT
Analyze metrics: - Aggregate, correlate data - Prepare reports - Discuss trends, identify thresholds - Revise procedures
CHECK
MEASUREMENT PARADOX? I want a car that uses as little petrol as possible. I have chosen 2 cars and will do 3 test drives with both of them. Test drive 1
Test drive 2
Test drive 3
Average
Car A
1 km/l
1 km/l
4 km/l
2 km/l
Car B
1.5 km/l
1.5 km/l
1.5 km/l
1.5 km/l
Test drive 1
Test drive 2
Test drive 3
Average
Car A
1 l/km
1 l/km
0.25 l/km
0.75 l/km
Car B
0.667 l/km
0.667 l/km
0.667 l/km
0.667 l/km
The car that uses less petrol is car A which does more kilometres per litre. But, if I had measured litres per kilometre, car B uses less litres per kilometre and is the better car.