Wp qualisystems testautomation services nexiilabs aug17 copy(1)

All-in-One Test Automation Platform for Accelerated Testing Contents 1

Abstract ............................................................................................................................................................2

2

What is QualiSystems TestShell? ......................................................................................................................2

3

Why QualiSystems? ..........................................................................................................................................2

4

Test Shell Components .....................................................................................................................................3

5

6

4.1

Quali Server ..............................................................................................................................................3

4.2

Studio ........................................................................................................................................................3

4.3

Resource Manager ................................................................................................................................3

4.4

Portal........................................................................................................................................................3

Test Architecture ..............................................................................................................................................4 5.1

Test Management ...................................................................................................................................4

5.2

Resource Manager....................................................................................................................................4

5.3

Studio ........................................................................................................................................................4

5.4

Test Reports ..............................................................................................................................................5

Automation Framework ...................................................................................................................................6 6.1

7

8

9

Rapid Automation.....................................................................................................................................6

Create a test block in Studio .............................................................................................................................7 7.1

Canvas.......................................................................................................................................................7

7.2

Interface ...................................................................................................................................................8

7.3

Execution ..................................................................................................................................................8

Benefits of QualiSystems over Scripting ...........................................................................................................9 8.1

All-in-one platform ...................................................................................................................................9

8.2

In-built libraries ........................................................................................................................................9

8.3

Ease of automation...................................................................................................................................9

8.4

Ease of debugging.....................................................................................................................................9

8.5

Central test data and reports ...................................................................................................................9

Conclusion ........................................................................................................................................................9

Test Automation using QualiSystems

Nexiilabs

Confidential

1 Abstract Test Automation framework development needs investment and skilled professionals. It takes considerable amount of time and budget to design, develop and maintain an automation framework.

Added to this, it needs extra time to build execution engine and test reporting tools to alert the QA team through mails. There is a lot of coding involved in developing test blocks for the framework and utility tools. Manual testing team also need to develop utility scripts. QualiSystems identified these pain areas in test automation and came up with revolutionary platform called Test Shell. This white paper discusses the application of Test Shell platform in test automation.

2 What is QualiSystems TestShell?

TestShell is an integrated test environment which will meet the need of both automation and manual teams. Automation team can develop automated test blocks, execute and report test results. Manual testing team can build utility blocks quickly and execute them. It has a canvas on which test steps are inserted in the form of a flow chart blocks. All the paths in test block are visible and easy to develop and review the test.

3 Why QualiSystems?

For software development, there are RAD (Rapid Application Development) tools like Microsoft Visual Studio which is an IDE (Integrated Development Environment). Similarly for automated test modules development, QualiSystems is a RTD (Rapid test development) tool with ITE (Integrated Test Environment). It will enable the QA team to create tests, run and debug the test blocks pretty fast. 

Unlike traditional script based testing, QualiSystems provides a large repository of reusable libraries and built-in functions. These are dragged and dropped to create automated tests.



QualiSystems help in creating tests in a flash. Automated or manual testing is 20X faster using QualiSystems ecosystem compared to scripting approach.



It has a suite of tools which will enable the QA team to collaboratively test a product and report results with complete visibility at all phases.

A sample test looks like in below diagram:

Test Automation using QualiSystems

Nexiilabs

Confidential

4 Test Shell Components Below diagram shows the Test Shell server and its main components. It has several other components which are not shown here.

4.1 Quail Server It is the central server which will serve the requests from the clients like Studio, Resource Management and others. Test data and test results are stored at this central server which are accessible from any client station.

4.2 Studio QualiSystems TestShell Studio is similar to Microsoft Visual Studio which has all the features to create a test block, run the test and debug the test block. A test is created as a flow chart with START and END blocks in Studio. Test steps can be well organized in the form of blocks in Studio which helps in understanding and analyzing the test block quickly.

4.3 Resource Manager Lab resources like Server, switch or a device are all maintained in a central repository called Resource Manager. Hardware resources which are utilized to execute a suite are reserved for a time period and locked for a team. This feature will avoid any conflicts which arises when the same lab resource is being used by different teams at the same time. Test bed details like server IP address, username and password are fetched from resource manager into the test blocks. So, resource manager eliminates hard-coding of any lab details in the test blocks.

4.4 Portal If a QA manager wants to view the progress of test execution of the teams, it is possible to login to the Portal and view the work done by the team. There is no need to install any tool or software. Test results are visible to everyone and saved on this website for debugging any failed test cases. All the Test Automation using QualiSystems

Nexiilabs

Confidential

test runs against the System under Test (SUT) are visible in the test execution history.

5 Test Architecture Test Architecture with QualiSystems TestShell looks like in the diagram below. QA team picks up the test cases from the test management tool and automates them using TestShell Studio. Test bed details are fed into Resource Manager from which test modules pulls the required hardware details.

Automated tests are run against the SUT and the test reports are generated which can be viewed in Studio or portal.

5.1 Test Management Test cases are picked from a test management tool like Test Link and test blocks are developed as per the steps in the test cases. It is possible to integrate the test management tool with Studio so that the test blocks and reports are mapped to test case IDs.

5.2 Resource Manager Lab hardware resources required for execution of tests are entered in Resource Manager with a unique name for each resource. Test Shell has provided API to pull the resource details into test blocks. A set of resources can be reserved so that other team does not use them till the reservation ends.

5.3 Studio Studio is the main component in which test blocks are developed and executed against the SUT. Tests appear in the form of flowchart blocks. Re-usable functions can be developed which can be dragged and dropped to create new tests quickly.

Test Automation using QualiSystems

Nexiilabs

Confidential

5.4 Test Reports Report generation can be set using logging profile with different levels. Based on the logging level set, detailed reports are generated in any format like PDF, Word, Excel or text. Initially, logging level can be set to detailed level and once the tests are stable, the logging level has to be reduced which will speed up the test execution.

Test Automation using QualiSystems

Nexiilabs

Confidential

6 Automation Framework Automation framework is developed in layered approach by Nexiilabs team using re-usable function tests for several customers. A subset of functionality which is repeated in many tests is developed as a function block. This block can be reused by dragging and dropping into the new test blocks.

6.1 Rapid Automation Using in-built terminal tools and recording of executed commands, Studio automatically creates automated tests for SSH, Telnet or PowerShell commands as shown in the diagram below. It shows the commands to connect to an FTP server, list the files available on the server and download a file to local machine.

Test Automation using QualiSystems

Nexiilabs

Confidential

7 Create a test block in Studio 7.1 Canvas A test block is created in Studio Canvas with test steps as shown below. Test starts with START block and ends with END block. Test steps are coded in between these blocks. Each step function has defined inputs and outputs which are used for validation.

Each of the blocks can be run individually and the output can be verified. When f_step1 is working fine, then the f_step2 is run and the output is verified. In this way test is created as well as verified in parallel. Using traditional blocks, sub-functions have to be verified using dummy input separately and then needs to be integrated into main test. Canvas stores the previous execution results and helps in running the succeeding tests with earlier outputs. Test blocks are connected to each other by passing the outputs of previous test to succeeding test as shown below.

Test Automation using QualiSystems

Nexiilabs

Confidential

7.2

Interface

Inputs and outputs for each step function are defined in interface page as shown below. It is possible to publish the variables as IN and OUT.

7.3

Execution

While test is executed against the SUT, each step result is shown as PASS or FAIL as shown in the diagram below. A step which has passed is shown with green Tick mark and which has failed is shown in red cross mark. Failed step is investigated to identify the reason and report the bug to development team.

Test Automation using QualiSystems

Nexiilabs

Confidential

8 Benefits of QualiSystems over Scripting 8.1

All-in-one platform

Using traditional scripting approach, one has to build execution engine and utility tools to build complete test automation framework from scratch. QualiSystems provides a platform with all the basic features of testing so that QA team can focus only on creating tests for their SUT or DUT. 8.2

In-built libraries

There is a repository of libraries for File I/O, Restful API, VMWare, Database etc. QA team can drag these libraries and utilize the functions provided.

8.3

Ease of automation

Terminal tools have recording feature which will help in executing the commands manually and generate automated test blocks.

8.4

Ease of debugging

Several log files have to be analyzed to debug any issue in case of scripting approach. TestShell has full features Canvas which will allow the automation engineer to watch all the variables, code, execution results at one place.

8.5 Central test data and reports A test may be executed from any of the clients but the reports are stored on the central server. It will prevent any loss of previous logs and make them available for development team or managers to view the results and reports of all executions done by QA team.

9 Conclusion The all-in-one QualiSystems TestShell or cloushell is an integrated testing environment that effectively overcome the challenges in traditional testing. Automation team can develop automated test scripts, execute and report test results. Manual testing team can build utility scripts quickly and execute them. Unlike traditional script based testing, QualiSystems provides a large repository of re-usable libraries and built-in functions. It is a rapid test development platform which will help in creating tests in a flash. Automated or manual testing is 20X faster using QualiSystems ecosystem compared to scripting approach. Its suite of tools enables the QA team to collaboratively test a product and report results with complete visibility through all phases of the testing process.

Test Automation using QualiSystems

Nexiilabs

Confidential

About Nexiilabs Nexiilabs team is a pioneer in developing test automation frameworks in Storage, Telecom, data-com, control systems and embedded systems domains. It has a pool of test architects who are specialists in test automation using traditional scripting as well as QualiSystems. Nexiilabs architects have observed several gains and are able to deliver automated tests in much less time and with improved coverage using QualiSystems. Nexiilabs team has delivered test automation frameworks to several customers and helped them to run the tests on QualiSystems.

Test Automation using QualiSystems

Nexiilabs

Confidential