TEST METRICS
Agenda ●
Why Test Software
●
Why Do We Need Metrics
●
Types of Metrics
●
OS/ Browser compatibility Matrix
●
DRE(Defect Resolution Effectiveness)
●
Test Execution Burndown Chart
●
Test Case Defect Density/ Test Case Efficiency or Test Efficiency
●
Test Effectiveness
●
TER Copyright © by QA InfoTech. All rights reserved.
Why Test Software? Any software product, no matter how accomplished by today’s testing technologies, has bugs. Software testing is the process of detecting and removing the bugs from the software product. The goal of software testing is to ensure software product quality according to the customer requirements. Copyright Š by QA InfoTech. All rights reserved.
Some typical software testing questions ● ● ● ● ● ●
How long will it take to test the software? How much will it cost to test? What kind of bugs were found? How many of the bugs that were found were fixed? Will the product be ready on time? How much is the risk of release? --Metrics are ways of answering these questions. --There are numerous metrics for software testing.
Copyright © by QA InfoTech. All rights reserved.
Why Do We Need Metrics? TEST METRICS HELPS US: ● Take decisions for the next phase of activities. ● As an evidence on the basis of which we can predict. ● Understand the type of improvement required. ● Take decisions on the change in process or technology.
Copyright © by QA InfoTech. All rights reserved.
Type of metrics Base Metrics (Direct Measure): Base metrics constitute the raw data gathered by a Test Analyst. These metrics are used to provide project status reports to the Test Lead and Project Manager, they also act as an input to the formulas used to derive Calculated Metrics. Eg: # of Test Cases, # of Test Cases Executed
Calculated Metrics (Indirect Measure): Calculated Metrics convert the Base Metrics data into more useful information. These types of metrics are generally the responsibility of the Test Lead and can be tracked at many different levels (by module, tester, or project). Eg: % Complete, % Test Coverage Copyright Š by QA InfoTech. All rights reserved.
Test Metrics Test metrics are classified into two categories: ● Product Metrics: Metrics that are related to test results or quality of software under test
● Process Metrics: Metrics that used for evaluation of testing process effectiveness
Copyright © by QA InfoTech. All rights reserved.
OS/Browser Compatibility Matrix:
Copyright © by QA InfoTech. All rights reserved.
DRE(Defect Resolution Effectiveness) Formula to Calculate DRE :
Total Valid Defects Resolved -------------------------------------- *100 Total Valid Defects Logged
Copyright Š by QA InfoTech. All rights reserved.
OVERALL DRE FOR THE PRODUCT: (CALCULATED BASED ON PRIORITY):
Copyright © by QA InfoTech. All rights reserved.
Test Execution Burndown Chart
Copyright © by QA InfoTech. All rights reserved.
Test Case Defect Density/Test Case Efficiency or Test Efficiency Total number of errors found in test scripts v/s developed and executed. â—? (Defective Test Scripts/Total Test Scripts) * 100
Example: Total test scripts developed are 1360, total test scripts executed are 1280, total test scripts passed are 1065, total test scripts failed are 215 So, test case defect density is
215 ------------------------- X 100 = 16.8% 1280
This 16.8% value can also be called as test case efficiency %, which depends on the total number of test cases which uncovered the defects Copyright Š by QA InfoTech. All rights reserved.
Test Effectiveness “Test” Effectiveness ● Number of defects found during “testing” activities in any phase of the SDLC ● Total number of defects found including the ones in production (used a “warranty period”) Test effectiveness is calculated as: # of defects found during testing phase ----------------------------------------------------total # of defects found (Testing phase + Post release or customer reported defects)
Copyright © by QA InfoTech. All rights reserved.
Test Execution Report (TER) ● ● ●
Managerial Summary about the health of the product at the end of each testing cycle Helps in determining Release Readiness Prepared at the end of each testing cycle (Iteration, Sprint)
TER Contents: ● ● ●
●
Scope of testing Environment details Coverage details ● Features ● Test Cases ● Pass Rate ● Results List of open defects Copyright © by QA InfoTech. All rights reserved.
●
Defect Resolution Effectiveness
●
QA is ➔ Ready to release ➔ Ready to release with recommendations ➔ Not ready to release Copyright © by QA InfoTech. All rights reserved.
Thank You
info@qainfotech.com www.qainfotech.com