Test Automation

Page 1

Testing induction

Testing Essentials Unit 12 Introduction to the Automation QSIT- Quality Solutions for Information Technology Pvt. Ltd., A 806 & 807, Mittal Tower, BANGALORE- 560 001,INDIA URL: www.qsitglobal.com PH: +91 80 41134334 Fax: +91 80 25597445

Software Testing Essentials

1

www.qsitglobal.com

Basic Automation Glossary

Test Case – Definition – Set of Predefined inputs to Produce a definitive output, organized into a sequential execution

Types of Test case – Manual – Automation

Software Testing Essentials

QSIT Copyright

2

www.qsitglobal.com

Page:1


Testing induction Manual Testing

Draw backs: – Hard to repeat – Not always reliable – Costly – Time consuming – Certain scenarios not practically possible

Software Testing Essentials

3

www.qsitglobal.com

Test Automation

Need for Automation: –

– –

Develop additional test cases to achieve better coverage Time available for Ad-hoc Testing Process improvement in Manual testing

Software Testing Essentials

QSIT Copyright

4

www.qsitglobal.com

Page:2


Testing induction Automation Testing

Benefits of Automation: – Reliable – Good return on investment – Run attended anytime – Able to simulate scenarios

Software Testing Essentials

5

www.qsitglobal.com

Test Automation

Software test automation refers to –

Activities and efforts that intend to automate engineering tasks and operations in a software test process.

Software test automation uses –

Well-defined strategies and systematic solutions.

Software Testing Essentials

QSIT Copyright

6

www.qsitglobal.com

Page:3


Testing induction Objectives of Test Automation

To free engineers from tedious and redundant manual testing operations

To speed up a software testing process, and to reduce software testing cost and time during a software life cycle

To increase the quality and effectiveness of a software test process by achieving pre-defined adequate test criteria in a limited schedule

To reduce manual testing activities and redundant test operations using a systematic solution to achieve a better testing coverage.

Software Testing Essentials

7

www.qsitglobal.com

Test Automation Process

Plan Software Test Automation Design Test Automation Strategies & Solutions

Select and Evaluate Available Software Testing Tools

Develop & Implement Test Automation Solutions Introduce and Deploy Test Automation Solutions Review and Evaluate Software Test Automation

Software Testing Essentials

QSIT Copyright

8

www.qsitglobal.com

Page:4


Testing induction Test Automation Process

Step #1: Test automation planning – This is the initial step in software test automation. The major task here is to come out a plan that specifies the identified test automation focuses, objectives, strategies, requirements, schedule and budget.

Step #2: Test automation Design – The primary objective of this step is to draw out the detailed test automation solutions to achieve the major objectives and meet the given requirements in a test automation plan.

Software Testing Essentials

9

www.qsitglobal.com

Test Automation Process

Step #3: Test tool development – At this step, the designed test automation solutions are developed and tested as quality tools and facilities. The key in this step is to make sure that the developed tools are reliable and reusable with good documentation.

Step #4: Test tool deployment – Similar to commercial tools, the developed test tools and facilities must be introduced and deployed into a project or onto a product line. At this step, basic user training is essential, and proper user support is necessary.

Software Testing Essentials

QSIT Copyright

10

www.qsitglobal.com

Page:5


Testing induction Test Automation Process Step #5: Review and evaluation – Whenever a new tool is deployed, a review should be conducted to identify its issues and limitations, and evaluate its provided features. The review results will provide valuable feedback to the test automation group for further improvements and enhancements.

Software Testing Essentials

11

www.qsitglobal.com

Test Process Tools Costs Automation involved in Automation

Test team shall research to ascertain and help shape management expectation. – 1.Improved requirements definition. – 2.Improved performance testing. – 3.Improved load/stress testing. – 4. Quality measurements and test optimization. – 5. Improved partnership with development team. – 6.Improved system development life cycle.

Outline phased implementation – Effort distributed – Cost Distributed

Get the large purchase discount.

Identify project where the tool can be used.

Number of licenses required concurrently.

Software Testing Essentials

QSIT Copyright

12

www.qsitglobal.com

Page:6


Testing induction Conclusion First , the test team could select a less expensive tool that supports test requirements adequately for the near term Second, it could outline the cost savings or performance enhancing benefits in a way that convinces management that the tool is worth the upfront investment Third, it could scale down the implementation of the test tool and plan for additional implementation during the next budget period.

Software Testing Essentials

13

www.qsitglobal.com

Classification of Software Test Tools

White Box Testing Tools Functionality Testing Tools Test Management Tools Test-Case Generation Tools Performance Testing Tools

Software Testing Essentials

QSIT Copyright

14

www.qsitglobal.com

Page:7


Testing induction Some test case for different types of testing

Software Testing Essentials

Functionality Reliability

Load / stress testing

Performance

Internationalization

Check whether log works Repeat log in operation in loop for 48 hrs Perform log in from 10000 clients Measure time taken for log in operations in different conditions Run log in operation from a machine running japanese language

15

www.qsitglobal.com

Skills required for automation Automation-First generation

Automation-second generation

Automation-third generation

Skills for test case

Skills for test case automation

Skills for test case automation

Skills for Framework

Scripting languages

Scripting languages

Scripting languages

Programming languages

Record-playback tools usage

Programming languages

Programming languages

Designing and architecture skills for framework creation

Knowledge of data generation techniques

Designing and architecture of the product under test

Generic test requirement for multiple products

Usage of the product under test

Usage of the framework

automation

Software Testing Essentials

QSIT Copyright

16

www.qsitglobal.com

Page:8


Testing induction Basic Principles of the test Automation tools

What is test Script? What are the verification Point? What are synchronization points?

Software Testing Essentials

17

www.qsitglobal.com

Verification points

Verification points verify that a certain action has taken place, or verify the state of an object

When you create a verification point‌ – You are capturing information about an object in the application to establish this as baseline information for comparison during playback

Software Testing Essentials

QSIT Copyright

18

www.qsitglobal.com

Page:9


Testing induction Test Script

A test script is a short program written in a programming language used to test part of the functionality of a software system

A written set of steps that should be performed automatically can also be called a test script, however this is more correctly called a test case

Software Testing Essentials

19

www.qsitglobal.com

Synchronization points

synchronization means the coordination of simultaneous threads or processes to complete a task in order to get correct runtime order and avoid unexpected race conditions

Software Testing Essentials

QSIT Copyright

20

www.qsitglobal.com

Page:10


Testing induction Tool classification

Object Based automation tools Bitmap Based automation tools

Software Testing Essentials

21

www.qsitglobal.com

Approaches for test automation

Record and playback Scripting Framework

Software Testing Essentials

QSIT Copyright

22

www.qsitglobal.com

Page:11


Testing induction Integration of the functional testing tools with test management tools

Best practices – – – – – – – – – – –

Document the process Manage expectations Use a pilot project Verify test tool compatibility Verify test tool upgrades Baseline the system setup Configuration before installing a new tool Avoid incompatible test tools Avoid unnecessary software installation in an already-base lined Environment Testing environment Understand the overall test program objectives

Software Testing Essentials

23

www.qsitglobal.com

Best practices cont…..

Remember that not everything should be automated Keep automation simple Follow test procedure design and development standards Conduct automated versus manual test analysis Conduct reuse analysis Encourage test team partnership with all teams involved in the system development life cycle Keep the communication flowing Ensure schedule compatibility Involve the customer from the beginning of the system development life cycle

Software Testing Essentials

QSIT Copyright

24

www.qsitglobal.com

Page:12


Testing induction Best practices cont‌.

Document and report defects Use an automated test expert or tool advocate to sell the benefits of the automated tool within the project team Clarify test team assignments Participate in user group meetings and testing discussions on the Web Suggest test tool improvements to the vendor Become a beta tester Take advantage of the knowledge of the specialty topic experts (tool experts)

Software Testing Essentials

25

www.qsitglobal.com

Myths of Automation‌

Automated testing has the advantage over manual testing in that it is easily repeatable This is flavored when doing regression testing This however is not always the case as automated tests may be poorly written and can break during playback Since most systems are designed with human interaction in mind, it is good practice that a human tests the system at some point

Software Testing Essentials

QSIT Copyright

26

www.qsitglobal.com

Page:13


Testing induction White Box Testing Tools

Tests – Coding Standards – Memory Leaks – Code Complexity – Code Coverage

Tools available – Rational Purifier – C++ Test from Parasoft – Rational Quantifier – Jtest from Parasoft – Rational Purecoverage

Software Testing Essentials

27

www.qsitglobal.com

Functionality Testing Tools

Features: – Record and Playback capability – Object Recognition & Object Repository – Synchronization – Parameterization – Checkpoints or Verification points (Standard, Bitmap, Database etc) – Batch Testing – Error Handling – Configurable Options – Initialization Scripts – Support Client server and Web Based Applications

Software Testing Essentials

QSIT Copyright

28

www.qsitglobal.com

Page:14


Testing induction Functionality Testing Tools (Contd…) Functionality Testing Tools contd.

Features for web based applications – Functions to know when the page has finished loading. – Making the tool to wait until an image appears on the page – Can I extract data from the web page itself? E.g. the title? A hidden form element? – Testing Links – State of the web based objects (enabled, disabled, contain data etc) – Are there facilities that will allow me to programmatically look for objects of a certain type on a web page or locate a specific object?

Software Testing Essentials

29

www.qsitglobal.com

Functionality Testing Tools available today…

Rational Robot from IBM Win-Runner, Quick Test Professional from HP Silk-test from Borland Test-Partner from Compuware Etc…

Software Testing Essentials

QSIT Copyright

30

www.qsitglobal.com

Page:15


Testing induction Functionality Testing Tools available today… Test Management Tools

Test Management Tools address – Requirements – Managing manual test suites, scripts – Managing Test Automation Scripts – Test Execution Activities – Generation of various reports on execution – Defect Tracking Tools available… – Rational Test-Manager from Rational – Test Director from Mercury etc

Software Testing Essentials

31

www.qsitglobal.com

Test Case Generation Tool

Generate Test Cases based on inputs passed on to the tool.

Tools available… – Caliber RBT

Software Testing Essentials

QSIT Copyright

32

www.qsitglobal.com

Page:16


Testing induction Performance Testing Tools

Capacity – Capacity means whether the web application is able to handle large number of concurrent users. Throughput – Throughput means whether the number of clients requests processes within a certain time. Response Time – Response time is the time from when the browser has finished sending a request to when it starts receiving the response from the server. – Response time and capacity both are inversely proportional to each other.

Software Testing Essentials

33

www.qsitglobal.com

Performance Testing Tools contd.

Load-Runner from HP Rational Performance from IBM Silk-Performer from Borland Etc..

Software Testing Essentials

QSIT Copyright

34

www.qsitglobal.com

Page:17


Testing induction Essential needs of Test Automation •

A dedicated work force for test automation

The commitment from senior managers and engineers

The dedicated budget and project schedule

A well-defined plan and strategy

Talent engineers and cost-effective testing tools

Maintenance of automated software tests and tools

Software Testing Essentials

35

www.qsitglobal.com

Issues in Software Test Automation ♦ ♦

Poor manually performed software test process Late engagement of software test automation in a software product life cycle

Unrealistic goals and unreasonable expectations

Organization issues

Lack of good understanding and experience of software test automation

Software Testing Essentials

QSIT Copyright

36

www.qsitglobal.com

Page:18


Testing induction Benefits of Test Automation

Speed and Accuracy – It can be as much as 50 times faster, depending upon the speed of the driver machine and the speed of the application to process information

Accessibility – Automation tools allow access to objects, data, communication protocols, and operating systems that manual testers cannot access

Accumulation – Once tests are developed, long-term benefits are derived through reuse

Software Testing Essentials

37

www.qsitglobal.com

Benefits of Test Automation contd.

Manageability – Ability to manage artifacts through automation tools

Discovery of issues – Automated testing assists with the discovery of issues early in the development process, reducing costs

Repeatability – An automation suite provides a repeatable process for verifying functionality on the functional side and scalability on the performance side

Software Testing Essentials

QSIT Copyright

38

www.qsitglobal.com

Page:19


Testing induction Intangible Benefits of Test Automation (contd..)

Formal process – Automation forces a more formal process on test teams, due to the nature of the explicitness of the artifacts and the flow of information that is needed

Retention of customers – Performing correct and systematic automated testing helps assure a quality experience for the customer – both internal and external

Software Testing Essentials

39

www.qsitglobal.com

Intangible Benefits of Test Automation (contd..)

Greater job satisfaction for Testers – The Test Engineers no longer manually execute the same test cases over and over – They would utilize a programming-like IDE and language that is more challenging, rewarding and portable to other positions

Software Testing Essentials

QSIT Copyright

40

www.qsitglobal.com

Page:20


Testing induction Causes for failure in Test Automation Lack of structured automation methodology Test automation is not treated as a project with proper project planning (i.e. scope, resources, time-to-market) Testing is performed at the end of the development cycle (the waterfall method) No modularization (use of functions) in automation scripts Test engineers are untrained in tool interface and programming techniques After initially creating automation suite, customer does not maintain the suite for future builds

Software Testing Essentials

41

www.qsitglobal.com

Solutions to the reasons for failure

Implement a pragmatic approach to testing that is – Manageable, such that the project can be decomposed into modular, defined tasks with assigned resources and timelines –

Repeatable, such that others can easily carry forward the process that has been defined

Measurable, such that the effort is quantifiable how many defects found in each stage

what is trend of different severities of defects

how close is the testing cycle to completion

how long does a transaction take to complete

Software Testing Essentials

QSIT Copyright

42

www.qsitglobal.com

Page:21


Testing induction Solutions to the reasons for failure

Implement a pragmatic approach to testing that is – Improvable, such that each build becomes more efficient in producing defects – The goal of this measurable and improvable process is to produce more defects in the testing life cycle so that less are found in production –

Automated, to build a data-driven regression and scalability/performance suite that takes advantage of the best-of-breed testing software

Software Testing Essentials

43

www.qsitglobal.com

Myths and realities of automation

An automated test tool cannot be expected to support 100% of the test requirements of any given test effort Automated testing my increase depth and breadth of test coverage, yet there is still not enough time or recourses to perform a 100% exhaustive test

Software Testing Essentials

QSIT Copyright

44

www.qsitglobal.com

Page:22


Testing induction Different types of tools : Cross-Platform tools Eggplant : Automated user-interface testing of a device running on any platform that uses the Virtual Network Computing (VNC) protocol Squish : Automated GUI testing tool on Windows, Linux/Unix, Mac OS X, embedded Linux for Qt, Web/HTML, Java/SWT, Java/Swing, Mac OS X, Tk, 4JS, XView, ... Silk Test: Is a leading functional GUI testing tool for Windows, and several flavors of UNIX

Software Testing Essentials

45

www.qsitglobal.com

Testing Tools Windows test tools IBM Rational Functional Tester (RFT) is a leading Functional GUI testing tool, supports testing Java, .NET, Web Apps(IE, Firefox, Mozilla, Netscape), Siebel and SAP GUI environments. Badboy is an inexpensive automated testing tool targeted specifically at testing of web applications. KeepLiquid TestAutomation is a keyword and data driven GUI test automation tool. NUnitForms is an open source extension for unit testing. QuickTest Professional (QTP) is a leading functional GUI testing tool. Ranorex is a free GUI test and automation Library for C++, Python and for all .NET languages. SilkTest is a leading functional GUI testing tool. TestDrive-Gold, code-free testing of GUI and browser applications. TestPartner an automated functional and regression testing tool from Compuware Corporation. WinRunner is a leading functional GUI testing tool. Wintask is an automation and regression test tool for GUI. TestComplete is functional GUI testing tool. Visual_Test from Microsoft / Rational was the first automation testing tool. Visual Studio Team Test is the successor to Visual_Test from Microsoft.

Software Testing Essentials

QSIT Copyright

46

www.qsitglobal.com

Page:23


Testing induction Testing Tools Web only test tools Selenium - Selenium tests run directly in a browser. Runs in IE, Mozilla and Firefox on Windows, Linux, and Macintosh. Watir - An opensource webtesting framework written in Ruby. Works only under Windows/IE6. WET Web Tester - An opensource webtesting framework inspired by Watir.Includes a script generation utility Sahi - Open source, java based testing tool for automated testing of web/html/ajax applications Mercury Interactive Tools - Win Runner, QTP Compuware Tools - QA Tester Empirix Tools - eTester [edit] Related tools AutoHotkey is a free GUI automation/scripting tool for windows applications, source available under the GPL. CodePro AnalytiX - Automated Java testing and sofware quality tools for Eclipse. Jameleon is an automated testing framework that uses plug-ins to test database, desktop, web, and mainframe software applications. Used mainly for integration, regression, functional, and acceptance testing at the code/object level.

Software Testing Essentials

QSIT Copyright

47

www.qsitglobal.com

Page:24


Turn static files into dynamic content formats.

Create a flipbook
Issuu converts static files into: digital portfolios, online yearbooks, online catalogs, digital photo albums and more. Sign up and create your flipbook.