Limited Business Circulation
Testing Essentials Unit 4 QSIT- Quality Solutions for Information Technology Pvt. Ltd., A 706 & 707, Mittal Tower, BANGALORE- 560 001,INDIA URL: www.qsitglobal.com PH: +91 80 41134334 Fax: +91 80 25597445
Software Testing Essentials
1
www.qsitglobal.com
2
www.qsitglobal.com
Testing Life Cycle Overview
•
Test Strategy
•
Test Planning
•
Test Case Design
•
Test Execution
•
Testing Types
•
Testing Tools Overview
Software Testing Essentials
QSIT Copyright
Page:1 Version 1.1/Testing Essentials/001
Limited Business Circulation
Testing Life Cycle Overview Normally, testing of any Large Systems will be in TWO
parts.
The functional verification and validation towards the Requirement Spec and Performance evaluation against the indicated requirements
Testing activity is involved right from the beginning of the project V&V Process model to successfully deliver a project on time This model is followed for a complete project life cycle model
Software Testing Essentials
3
www.qsitglobal.com
SDLC Vs STLC URS
User Acceptance Testing
UAT Planning
System test planning
SRS
Verification
System Testing
Validation
Integration test planning
HLD
LLD
Unit test planning
Integratio n testing
Unit
Delivery production deployment
testing Maintenance and
Coding Software Testing Essentials
QSIT Copyright
4
enhancement www.qsitglobal.com
Page:2 Version 1.1/Testing Essentials/001
Limited Business Circulation
Software Testing Life Cycle
Software Testing Essentials
5
www.qsitglobal.com
STLC Phases
Requirement analysis
Test Strategy
Prepare Test Plan
Test Case Designing
Test Case Execution
Bug Reporting, Analysis and Regression Testing
Inspection and Release
Client Acceptance and Support During acceptance
Test Summary Analysis
Software Testing Essentials
QSIT Copyright
6
www.qsitglobal.com
Page:3 Version 1.1/Testing Essentials/001
Limited Business Circulation
Test Strategy - Objective All the business and test objectives are identified and test strategy shall be defined in order to meet those objectives. As far as possible all the test objectives to be met are defined in quantitative terms in the test strategy. Examples of test objectives can be like 90 % of code coverage, finding at least a minimum of 100 defects etc.
Software Testing Essentials
7
www.qsitglobal.com
Test Strategy A test strategy will typically cover the following aspect
Definition of test objective Strategy to meet the specified objective Overall testing approach Test Environment Test Automation requirements Metric Plan Risk Identification, Mitigation and Contingency plan Details of Tools usage Specific Document templates used in testing
Software Testing Essentials
QSIT Copyright
8
www.qsitglobal.com
Page:4 Version 1.1/Testing Essentials/001
Limited Business Circulation
Test Strategy
Test strategy is statement of overall approach of testing to meet the business and test objectives. It is a plan level document and has to be prepared in the requirement stage of the project. It identifies the methods, techniques and tools to be used for testing . It can be a project or an organization specific. Developing a test strategy which effectively meets the needs of the organization/project is critical to the success of the software development An effective strategy has to meet the project and business objectives Defining the strategy upfront before the actual testing helps in planning the test activities
Software Testing Essentials
9
www.qsitglobal.com
Test Strategy - Approach Test approach will be based on the objectives set for testing Test approach will detail the way the testing to be carried out Types of testing to be done viz Unit, Integration and system testing The method of testing viz Black–box, White-box etc., Details of any automated testing to be done
Software Testing Essentials
QSIT Copyright
10
www.qsitglobal.com
Page:5 Version 1.1/Testing Essentials/001
Limited Business Circulation
Test Strategy – Test Environment
All the Hardware and Software requirements for carrying out testing shall be identified in detail. Any specific tools required for testing will also be identified If the testing is going to be done remotely, then it has to be considered during estimation
Software Testing Essentials
11
www.qsitglobal.com
Test Strategy – Metric Plan Metric plan will contain the measures that will indicate the effectiveness of testing . All the metrics to be tracked for testing like defect density ,residual defect density ,code coverage etc are identified in test strategy . The objectives set for testing are also measured in the metric plan.
Software Testing Essentials
QSIT Copyright
12
www.qsitglobal.com
Page:6 Version 1.1/Testing Essentials/001
Limited Business Circulation
Test Strategy – Risks Risk analysis should carried out for testing phase The risk identification will be accomplished by identifying causesand-effects or effects-and-causes The identified Risks are classified into to Internal and External Risks. The internal risks are things that the testing team can control or influence. The external risks are things beyond the control or influence of the testing team
Software Testing Essentials
13
www.qsitglobal.com
Test Strategy – Risks
Once Risks are identified and classified, the following activities will be carried out – – – –
Identify the probability of occurrence Identify the impact areas – if the risk were to occur Risk mitigation plan – how avoid this risk? Risk contingency plan – if the risk were to occur what do we do?
Software Testing Essentials
QSIT Copyright
14
www.qsitglobal.com
Page:7 Version 1.1/Testing Essentials/001
Limited Business Circulation
Test Strategy – Tools Usage Tools required in the project and rationale to use that particular tool(s) should be documented. Automation requirements should be identified and analyzed Which phases need to be automated How much of testing needs to be automated Based on the automation requirements, tools strategy should be prepared.
Software Testing Essentials
15
www.qsitglobal.com
Prepare Test Plan This involves following activities:
Objective
Scope
Environment
Schedule/ Mile Stone
Entry/Exit Criteria
Resources & Responsibility
Software Testing Essentials
QSIT Copyright
testplan template
IEEE testplan template
16
www.qsitglobal.com
Page:8 Version 1.1/Testing Essentials/001
Limited Business Circulation
Test Case Test Case Writing: A specific set of steps and data along with expected results for a particular test objective. A test case should only test one limited subset of a feature or functionality.
Software Testing Essentials
17
www.qsitglobal.com
Test Case Design A typical test case sheet contains these fields‌
Test Case ID: It is unique number given to test case in order to be identified.
Test description: The description if test case you are going to test.
Revision history: Each test case has to have its revision history in order to know when and by whom it is created or modified.
Function to be tested: The name of function to be tested.
Priority/Build/time: The priority of the Test Case, Build version of the test case and the Minimum time-to-execute the test case is also mentioned.
Software Testing Essentials
QSIT Copyright
18
www.qsitglobal.com
Page:9 Version 1.1/Testing Essentials/001
Limited Business Circulation
Test Case Design (Contd…)
Environment: It tells in which environment you are testing.
Test Setup: Anything you need to set up outside of your application for example printers, network etc.,
Test Execution: It is detailed description of every step of execution.
Expected Results: The description of what you expect the function to do.
Actual Results: Pass /Failed. If pass – What actually happens when you run the test If failed - Description of what you've observed.
Software Testing Essentials
19
www.qsitglobal.com
Test Case Design (Contd…) Characteristics of a Good Test:
Tests are likely to catch bugs No redundancy Not too simple or too complex.
TIP: Test case is going to be complex if you have more than one expected results.
Software Testing Essentials
QSIT Copyright
20
www.qsitglobal.com
Page:10 Version 1.1/Testing Essentials/001
Limited Business Circulation
Test Data Inputs Test Data Inputs:
Before testing, the tester should plan what kind of data he is giving for test.
Give data inputs as functional, boundary, stress, performance, usability values etc.
Software Testing Essentials
21
www.qsitglobal.com
Test Case Design (Contd‌) Test Scripts:
A test script is the executable form of a test.
It defines the set of actions to carry out in order to conduct a test and it defines the expected outcomes and results that are used to identify any deviance in the actual behavior of the program from the logical behavior in the script (errors during the course of that test).
In essence it is a program written for a human computer (tester) to execute.
TestCase Template
Software Testing Essentials
QSIT Copyright
22
TestCase Status Template
www.qsitglobal.com
Page:11 Version 1.1/Testing Essentials/001
Limited Business Circulation
Test Case Template
Software Testing Essentials
23
www.qsitglobal.com
Test Execution • Test execution is culmination of testing activities which involves executing the planned test cases and conducting of the tests. • Test execution phase broadly involves execution and reporting.
Software Testing Essentials
QSIT Copyright
24
www.qsitglobal.com
Page:12 Version 1.1/Testing Essentials/001
Limited Business Circulation
Test Execution - Activities Test execution consists of following activities to be performed Creation of test setup or Test bed Execution of test cases on the setup Test Methodology used Collection of Metrics Defect Tracking and Reporting Regression Testing
Software Testing Essentials
25
www.qsitglobal.com
Test Execution - Cycle
Test
Data: Inputs which are to exercise the system under test.
Software Testing Essentials
QSIT Copyright
26
www.qsitglobal.com
Page:13 Version 1.1/Testing Essentials/001
Limited Business Circulation
Test Execution – Defect Tracking & Reporting All the defects/bugs found during testing is logged and they are classified based on the severity and priority. They are assigned to developers for fixing. Each defect found is logged and tracked to closure. Test summary report shall be prepared containing the details like numbers of test cases passed/failed ,number of cycles of testing. Generally test defect logging and tracking tools are used for this purpose.
Test report template Software Testing Essentials
27
www.qsitglobal.com
Test Execution – Defect Tracking & Reporting Bug A fault in a program which causes the program to perform in an unintended or unanticipated manner or deviation from the requirement specification or the design specification is referred as a bug.
Software Testing Essentials
QSIT Copyright
28
www.qsitglobal.com
Page:14 Version 1.1/Testing Essentials/001
Limited Business Circulation
Bug Life Cycle?
Defer
Submit Bug
Update
Review, verify and qualify
More Info
Assign
Duplicate/ Invalid
In-work
Fix/Change Cancel Validate
Close
Software Testing Essentials
29
www.qsitglobal.com
Defer Bug Deferred is where in the developer says that he cannot fix that bug
Reasons for that would be: It may change entire contents Technology Problem Cost of fixing Or Whenever we accept some changes in the modules Whenever there is less time available they may say that they will fix in future release
Software Testing Essentials
QSIT Copyright
30
www.qsitglobal.com
Page:15 Version 1.1/Testing Essentials/001
Limited Business Circulation
Reporting/Logging a Bug/Defect A Bug/Defect is reported with the following details
Defect ID : Bug ID _1546
Build ID : B04
Module Name : Sent Items
Status : Open /Assigned/Fixed/Closed/Reopen
Severity : Blocker/Critical/Major/Minor
Priority : High/Medium/Low
Test Case Name: ABC/Yahoo_sent_items
Test Environment : WinXP, Netscape Navigator
Found By : Name of the tester
Software Testing Essentials
31
www.qsitglobal.com
Reporting/Logging a Bug/Defect Brief Description : Sent mails are not listed in sent items page Description : Following are the steps to Reproduce the defects
Open browser and enter “Url�
Login with a valid user name and password
Click on compose link
Enter valid data in To, CC, BCC and in body of the mail and click on send button
Click on sent items
Expected results : sent mail should be listed in sent items page
Actual Result : Sent items is not displayed
this difference shows you the defect Software Testing Essentials
QSIT Copyright
32
www.qsitglobal.com
Page:16 Version 1.1/Testing Essentials/001
Limited Business Circulation
Reporting/Logging a Bug/Defect Process to be followed while you track a defect manually
Find a bug Check in the repository to make sure its not duplicate Prepare a defect report Send a copy to dev lead or development team distribution list and mark a cc to testing lead Enter the details in defect tracking sheet (optional)
Software Testing Essentials
33
www.qsitglobal.com
Classification of Bugs Two attributes are used whenever a Bug/Defect is detected Severity – Impact of the bug on the application Blocker: This bug prevents testers from testing the software. Critical : The software crashes, hangs, or causes you to lose data. Major : A major feature is broken. Trivial : A cosmetic problem, such as misspelled or misaligned text. Enhancement: Request for new feature or enhancement.
Priority – Impact on the developer/management High Medium Low
Software Testing Essentials
QSIT Copyright
34
www.qsitglobal.com
Page:17 Version 1.1/Testing Essentials/001
Limited Business Circulation
Examples on Severity and Priority Example on Low Severity and Priority Bug Which Does not Effect the Performance of the Application Spelling Mistakes Grammatical Mistakes Example 1: Enter Pwsd Severity : Minor Priority : Low
Example on low severity and a high priority Bug Example 2: Well to Yahoo Welcome page of Yahoo shows Well to Yahoo – Severity : Low Priority : High Software Testing Essentials
35
www.qsitglobal.com
Examples on Severity and Priority Example on Low Severity and High Priority Bug If you are going to test Dominos Pizza web site, in the home page the door delivery phone number is displaying wrong one in this situation severity wise it won’t affect any thing but it will affect the stake holder business so its “Low severity & High Priority”
Example on High Severity and High Priority Bug
If you take the example of Citi bank web site think the login page is not working then its “High severity & High Priority”
Software Testing Essentials
QSIT Copyright
36
www.qsitglobal.com
Page:18 Version 1.1/Testing Essentials/001
Limited Business Circulation
Examples on Severity and Priority In the big bazaar application the items that is to be brought has to be listed and has to be categorized according to category, it should not take under food items cloths and under cloths it should not take food Major Severity , Low Priority Logo Has Changed Critical Severity, Low Priority minimizes when the application is invoked, solution is: the application should be Maximized Minor Severity, Medium Priority Accepts blank in the Password, Accepts Critical Severity, Medium Priority You have entered Wrong password/Wrong Username Minor Severity, High Priority
Software Testing Essentials
37
www.qsitglobal.com
Test Execution – Start & Stop Criteria Testing is done in rounds. Software passes though several rounds before it gets released. Testing is stopped when there are many major bugs and critical system errors, when further testing is not feasible Testing is stopped when the system becomes un stable during test. When those critical errors are fixed and system is stable, the next round of testing is started . The test cases are executed all over again in each round.
Software Testing Essentials
QSIT Copyright
38
www.qsitglobal.com
Page:19 Version 1.1/Testing Essentials/001
Limited Business Circulation
Test Execution – Metrics Metrics are important to collect for any project as it helps in monitoring the health of project /product /application under test. Metrics are collected during testing depending on the objective of testing. There are different kinds of metrics which can be used in testing. For example LOC (Lines of Code) is a metric for Unit testing. i.e., Code related metrics comes in the category of Unit testing. Test case related metrics can be used for all kinds of testing types, since for every type of test, we will write test cases. Normally source of errors/defects is requirements, design or Implementation. Testing can not be the source of errors/defects.
Software Testing Essentials
39
www.qsitglobal.com
Test Execution – Metrics • Typical testing metrics – – – – – – – – – – – –
No of LOC tested per day No of test cases executed per day Number of test cases failed Number of bugs reported Number of rejects of bugs reported Number of persons working on the test plan during the month Total Effort spent on test execution in person hours Number of simulations made during the month Number of out of norms per month Average time taken to find a problem categorized by priority Defect density Review efficiency
Software Testing Essentials
QSIT Copyright
40
www.qsitglobal.com
Page:20 Version 1.1/Testing Essentials/001
Limited Business Circulation
Test Execution – Metrics • Having a complete Metrics data helps in building the “Trustworthy, Capable, Reliable and Predictable” Product or Application. – Productivity Increase – Field error rate Decrease – Customer Overhead reduces – Improvement release on release – On time delivery – Risks in Program identified and proactively intimated – Decrease in Cost of Quality – Decrease in Re-work – Helps in identifying the area which needs focus in the project/organization for improvement – Helps in arriving at better estimates for the project
Software Testing Essentials
41
www.qsitglobal.com
Test Execution – Testing Rules ? • Software testing 10 rules?
– Test early and test often. – Integrate the application development and testing life cycles. – Formalize a testing methodology; you'll test everything the same way and you'll get uniform results. – Develop a comprehensive test plan; it forms the basis for the testing methodology. – Use both static and dynamic testing.
Software Testing Essentials
QSIT Copyright
42
www.qsitglobal.com
Page:21 Version 1.1/Testing Essentials/001
Limited Business Circulation
Test Execution – Testing Rules ? –
Define your expected results.
–
Understand the business reason behind the application. You'll write a better application and better testing scripts.
–
Use multiple levels and types of testing (regression, systems, integration, stress and load).
–
Review and inspect the work, it will lower costs.
–
Don't let your programmers check their own work; they'll miss their own errors.
Software Testing Essentials
43
www.qsitglobal.com
Testing • What makes a good test engineer? – A good test engineer has a 'test to break' attitude. – An ability to take the point of view of the customer – A strong desire for quality, and an attention to detail. – Tact and diplomacy for maintaining a cooperative relationship with developers. – An ability to communicate with both technical (developers) and non-technical (customers, management) people.
Software Testing Essentials
QSIT Copyright
44
www.qsitglobal.com
Page:22 Version 1.1/Testing Essentials/001
Limited Business Circulation
Testing –
Previous software development experience can be helpful, gives the tester an appreciation from the developers' point of view, and reduce the learning curve in automated test tool programming.
–
Judgment skills are needed to assess high-risk areas of an application on which to focus testing efforts when time is limited.
–
Notice little things that others miss/ignore (See symptom not bug)
–
Good testers use files, Databases and all the other accoutrements of an organized mind
–
Testers are fundamentally honest and incorruptible Last but not the least is “Patience”
–
Software Testing Essentials
45
www.qsitglobal.com
Testing Team Structure An effective testing team includes a mixture of members who has – Testing expertise – Tools expertise – Database expertise – Domain/Technology expertise
Software Testing Essentials
QSIT Copyright
46
www.qsitglobal.com
Page:23 Version 1.1/Testing Essentials/001
Limited Business Circulation
Testing Team – Role and Responsibilities The testing team must be properly structured, with defined roles and responsibilities that allow the testers to perform their functions with minimal overlap. There should not be any uncertainty regarding which team member should perform which duties. The test manager will be facilitating any resources required for the testing team.
Clear Communication protocol should be defined with in the testing team to ensure proper understanding of roles and responsibilities. The roles chart should contain both on-site and off-shore team members.
Software Testing Essentials
47
www.qsitglobal.com
Testing Team – Role and Responsibilities • Test Manager – Single point contact between onsite and offshore team – Prepare the project plan – Test Management – Test Planning – Interact with onsite lead Client QA manager – Team management – Work allocation to the team – Test coverage analysis
Software Testing Essentials
QSIT Copyright
48
www.qsitglobal.com
Page:24 Version 1.1/Testing Essentials/001
Limited Business Circulation
Testing Team – Role and Responsibilities • Test Lead – Resolves technical issues for the product group – Provides direction to the team members – Performs activities for the respective product group – Review and Approve of Test Plan / Test cases – Review Test Script / Code – Approve completion of Integration testing – Conduct System / Regression tests – Ensure tests are conducted as per plan – Reports status to the Offshore Test Manager
Software Testing Essentials
49
www.qsitglobal.com
Testing Team – Role and Responsibilities • Test Engineer – – – –
Development of Test cases and Scripts Test Execution Result capturing and analysing Defect Reporting and Status reporting
Software Testing Essentials
QSIT Copyright
50
www.qsitglobal.com
Page:25 Version 1.1/Testing Essentials/001
Limited Business Circulation
Collaborative Approach - Roles of Developers & Testers Knowledge transfer from Development Team
How the Product/Module Works?
Intentions of each feature
Areas or features which are more risk/error prone
the schedule for delivering the remaining feature sets
Concerns about hidden/abstract activities
Knowledge transfer from Testing Team
Test coverage including types & levels of testing
The feature as understood by the testers
Required information so as to make feature more granular
Testing phases and schedule
What testers expect from the development team Software Testing Essentials
QSIT Copyright
51
www.qsitglobal.com
Page:26 Version 1.1/Testing Essentials/001