Reducing False Positives in Automated Testing
FALSE POSITIVES ARE “FAILED” TESTS THAT SHOULD HAVE PASSED, aka FALSE ALARMS.
WHY DO THEY OCCUR?
WHY DO THEY OCCUR? • AUTOMATION APPROACH
WHY DO THEY OCCUR? • AUTOMATION APPROACH • IMPLEMENTED FRAMEWORK
WHY DO THEY OCCUR? • AUTOMATION APPROACH • IMPLEMENTED FRAMEWORK • THE WRITTEN TEST ITSELF
THERE ARE SEVERAL POTENTIAL CAUSES FOR FALSE POSITIVES IN AUTOMATED TESTING:
Cause #1:
RELYING ON UI
Small changes in UI can cause test breakage
Cause #2:
CHANGE IN UI ELEMENT PROPERTIES
Any change to:
ID
Any change to:
ID CSS
Any change to:
ID CSS CLASS NAME
Can result in test breakage
Cause #3:
SHARED ENVIRONMENTS
Functional & automated testing require separate environments
Cause #4:
REFERENCE DATA CHANGE
Scripts may change configuration data required by a test
Cause #5:
SLOW PERFORMANCE
Slow performance = Slow test execution & test failure
Cause #6:
MANUAL INTERVENTION
Some data must be cleared manually before/after test execution
Cause #7:
RELYING ON SEQUENCE OF EXECUTION
Any failure in test sequence impacts the entire test suite
KEYS TO REDUCING FALSE POSITIVES
USE OPTIMAL CONFIGURATIONS
USE OPTIMAL CONFIGURATIONS CONTROLLED AUTOMATION ENVIRONMENT
USE OPTIMAL CONFIGURATIONS CONTROLLED AUTOMATION ENVIRONMENT
KEEP TESTS SHORT
USE OPTIMAL CONFIGURATIONS CONTROLLED AUTOMATION ENVIRONMENT
KEEP TESTS SHORT KEEP TESTS INDEPENDENT
USE OPTIMAL CONFIGURATIONS CONTROLLED AUTOMATION ENVIRONMENT
KEEP TESTS SHORT KEEP TESTS INDEPENDENT USE RIGHT LOCATORS FOR OBJECT IDENTIFICATION
VIEW THE ENTIRE
QASOURCE WEBINAR AND READ THE COMPLETE Q & A: Pre and Post Webinar Questions Answered: Reducing False Positives in Automated Testing QUESTIONS ABOUT FINANCIAL TESTING? VISIT US AT QASOURCE.COM