TEST Magazine - September-October 2010

IN TOUCH WITH TECHNOLOGY

T H E E U RO P E A N S O F T WA R E T E S T E R Volume 2: Issue 3: September 2010

INDIA: TESTING POWERHOUSE Yogesh Singh and Ruchika Malhotra on optimising web performance testing

Inside: Offshore testing | Penetration testing | Business alignment Visit T.E.S.T online at www.testmagazineonline.com

www.testmagazineonline.com

Leader | 1

The ‘mind change’ threat T H E E U RO P E A N S O F T WA R E T E S T E R

IN TOUCH WITH TECHNOLOGY

T H E E U RO P E A N S O F T WA R E T E S T E R Volume 2: Issue 3: September 2010

INDIA: TESTING POWERHOUSE Yogesh Singh and Ruchika Malhotra on optimising web performance testing

Inside: Offshore testing | Penetration testing | Business alignment Visit T.E.S.T online at www.testmagazineonline.com

Editor Matthew Bailey matthew.bailey@31media.co.uk Tel: +44 (0)203 056 4599 To advertise contact: Grant Farrell grant.farrell@31media.co.uk Tel: +44(0)203 056 4598 Production & Design Dean Cook dean.cook@31media.co.uk Toni Barrington toni.barrington@31media.co.uk Editorial & Advertising Enquiries 31 Media Limited, Media House, 16 Rippolson Road, London, SE18 1NS Tel: +44 (0) 870 863 6930 Fax: +44 (0) 870 085 8837 Email: info@31media.co.uk Web: www.testmagazineonline.com Printed by Pensord, Tram Road, Pontllanfraith, Blackwood. NP12 2YA © 2010 31 Media Limited.All rights reserved.

T.E.S.T Magazine is edited, designed, and published by 31 Media Limited. No part of T.E.S.T Magazine may be reproduced, transmitted, stored electronically, distributed, or copied, in whole or part without the prior written consent of the publisher. A reprint service is available.

A

n Oxford University brain researcher whose speciality is the physiology of the brain, Baroness Susan Greenfield CBE, has sparked a furious debate over the effects modern information technology has on the brain and its development.

In a speech at the British Science Festival in Birmingham she told delegates that the issue could pose the greatest threat to humanity after climate change! She has called on the Government and the private sector to join forces to examine the effects that computer games, the Internet and social networking sites like Twitter, Facebook and MySpace are exerting on the human brain.

She has coined the term ‘mind change’ to describe the effects that exposure to long periods working on computers can have. All potentially very worrying, until you take in the opinions of the wider scientific community where the consensus seems to be that it is way too early to come to any conclusion, either good or bad about the effects of prolonged exposure to these technologies.

Indeed, UK science journalist and self-confessed ‘nerd champion’ Ben Goldacre has criticised Greenfield for making the claims without having undertaken any research or, he says, properly evaluating the available

evidence. Goldacre called on her to “formally write up her concerns about computers damaging childrens’ brains” for formal peer review. To which she replied that he is “like the people who denied that smoking caused cancer”.

To be fair to Baroness Greenfield she stressed the possible benefits of modern technology may include higher IQ and faster processing of information, while, on the other hand, she also suggested that using internet search engines to find facts may affect people's ability to learn and that computer games in which characters get multiple lives might even foster recklessness, I wouldn’t claim any deep scientific knowledge, truth be told I ran off to be a journalist before I finished my biology degree, but I can’t see how social networking specifically could be any worse than any other sort of screen-based activity, including watching TV for long periods of time.

And it would perhaps be interesting to go back to the ’60s and ’70s to make a direct comparison with the era when TV was becoming ubiquitous (in the UK).

I’m old enough to remember being told that watching too much TV would be bad for my eyesight, although to be fair I don’t remember my parents suggesting the physiology of my brain was likely to be affected. Surely the thing to do is ask some of the real pioneers of social networking – and indeed personal computing – say, Bill Gates and Steves Jobs and Wozniak – to undergo a few tests. Thinking about it, in many ways, if these guys are the result of prolonged exposure to computers, is that necessarily a bad thing? Don’t answer that!

Opinions expressed in this journal do not necessarily reflect those of the editor or T.E.S.T Magazine or its publisher, 31 Media Limited.

ISSN 2040-0160

Published by:

Matt Bailey, Editor

In a speech at the British Science Festival in Birmingham she told delegates that the issue could pose the greatest threat to humanity after climate change! She has called on the Government and the private sector to join forces to examine the effects that computer games, the Internet and social networking sites like Twitter, Facebook and MySpace are exerting on the human brain. Matt Bailey, Editor www.testmagazineonline.com

September 2010 | T.E.S.T

A Global Virtual Learning Environment for the Software Testing industry Positioned to guide you to success in testing education and certification ■ ■

■

■

■

Options to fulfil your requirements – from start to finish Access to online and classroom courses, virtual classrooms, ebooks and a range of other testing-related content Sample exam questions, papers, revision sessions and live exam service Scalable to support individuals to the largest corporations Supported 24/7 with a global network of accredited training providers

www.learntesting.com For details of your local Learntesting provider, visit www.learntesting.com and begin your journey to success with Learntesting

Contents | 3

CONTENTS SEPT 2010

1

Leader column

4

Cover story – Testing web performance

The threat of ‘mind change’ and the brain-addling nature of social meadia.

Testing the performance of web pages and, increasing, web-based applications is a real

challenge because conventional testing techniques may not be applicable. Yogesh Singh and Ruchika Malhotra offer advice on optimising web performance testing

8

The testing landscape of India Vipul Kochur, chairman of the Indian Testing Board takes an overview of the Indian software testing industry, where it stands today, how has it evolved and where is it headed.

12

Offshore testing with a global reach

Matt Bailey speaks to Dr Sashi Reddi the founder and executive chairman of AppLabs a

real power house in the Indian offshore testing business.

16

What does the future hold for software testing?

Can we learn from software testing failures in today’s world to improve it in the future?

Geoff Thomson, chairman of the UK Testing Board looks into his crystal ball.

20

Now the offshore ‘Honeymoon’ is over, where next?

4

What are the challenges currently facing the offshore testing industry as the global financial situation puts the focus firmly back on cost? Adam Ripley reports.

24

Targeting new technology

Although it has stiff competition from the giants of the software industry, Facilita has

carved out a strong niche supplying a complete load testing solution to its customers.

T.E.S.T speaks to the company’s Gordon McKeown.

28

Evolve or die!

Industry analysts say that as much as 75 percent of all attacks are now targeted at the

12

application layer. David Harper looks at the changing face of penetration testing and tackles its fundamental flaws.

30

Is the testing world still flat?

34

Could better testing have saved the Deepwater Horizon?

s an advocate of testing for its business benefits, Susan Chadwick argues for the A need to align testing with business objectives, ensuring that companies are not just testing for the sake of it.

rian Luff casts a critical eye over the lack of independent software testing on oil B rig systems and how this needs to change to prevent disasters like the recent BP oil spill in the Gulf of Mexico from happening again.

36

Testers – equal amongst equals?

41

T.E.S.T Directory

48

Last Word – Dave Whalen

30

I n the rapidly developing world of software testing Angelina Samaroo says the fight for recognition goes on, but there is light at the end of the tunnel.

his issue Dave lets us in on his personal list of most annoying web design T techniques and tricks. Warning – may contain traces of sarcasm.

www.testmagazineonline.com

36 September 2010 | T.E.S.T

4 | T.E.S.T cover story

Testing web performance Testing the performance of web pages and, increasingly, web-based applications is a real challenge because conventional testing techniques may not be directly applicable and if your site isn’t up to the job, the visitor will quickly move to a competitor.

Yogesh Singh and Ruchika Malhotra* of the University School of Information Technology in Delhi, India offer advice on optimising web performance testing.

T.E.S.T | September 2010

www.testmagazineonline.com

T.E.S.T cover story | 5

W

eb pages and web sites have become integral part of modern civilisation.

Every day a new website for some specific applications is hosted and joins the bandwagon of the Internet.

We may visit a website and find a good number of web pages designed for specific applications. The quality of a web application must be assured in terms of response time, ease of use, number of users, ability to handle spikes in traffic and provide accurate information.

Compromise in any of the above parameters may compel the customers to move on to a competitor’s site.

Testing performance of these web pages is a real challenge because conventional testing techniques may not be directly applicable.

How long does an application take to respond to a user click? We must ensure that the user does not wait for a service for long, otherwise the potential user will move on to the competitor’s site. To ensure that the web application can bear the load during the peak hours along with serving the user in a timely and reliable manner, performance tests including load and stress tests need to be conducted.

One of the key advantages of web applications is that numerous users can have access to the them simultaneously. Hence the performance of the application during the peak periods must be tested and monitored carefully.

Several factors that may influence performance include: • Response time; • Memory available; • Network bandwidth; • Number of users; • User type; • Time to download; • Varied client machine configurations.

The goal of performance testing is to evaluate the application’s performance with respect to real world scenarios. The following issues must be addressed during performance testing: • Performance of system during peak hours (response time, reliability, availability);

www.testmagazineonline.com

• P oints at which the system performance degrades or system fails; • Impact of the degraded performance on the customer loyalty, sales and profits.

The above issues require metrics to be applied to the system under test in order to measure the system behaviour with respect to various performance parameters.

Load testing Load testing involves testing the web application under real world scenarios by simulating numerous users accessing the web application simultaneously. It tests the web application by providing a maximum load. The development of plans for load testing should begin as early as possible during the software life cycle.

Early testing will help in detection of problems prior to deployment of the web application. Load testing may follow the steps below in order to ensure reasonable performance during peak hours: 1. Defining environment for a load test; 2. Defining testing strategy and determining number of users; 3. Identifying potential metrics; 4. Choosing the right tool and executing the load test; 5. Interpreting the results.

1. Defining environment for a load test: This step involves the setting up of separate testing environments that simulate the production environment.

If the attributes of the machine such as speed, memory, and configuration are not the same, then it may be very difficult to extrapolate the performance during production. The guideline is to identify the necessary resources in order to establish the lab for conducting load tests. In addition to identification of resources the load testing team members must be determined to solve specific problems in the following areas: • Team leader: takes charge of the load test activity; • Database: identifies and removes database problems. Administers database activities;

To ensure that the web application can bear the load during the peak hours along with serving the user in a timely and reliable manner, performance tests including load and stress tests need to be conducted.

• N etwork: setups lab, identifies and removes network bottlenecks; • Business experts: defines quality levels of the application; • Performance analyst: analyses the available measures; • Quality management: ensures quality testing.

The configuration of environment for load tests include network configuration, setting up server, and database set up etc.

2. Defining testing strategy and determining number of users: This step includes writing of test cases focusing on the number of concurrent user requests, synchronisation of user requests, system bottlenecks and stability issues. The number of users may be determined by approximating the percentage of users hitting particular areas in the web site. The load test team may consult the requirement specification document to collect the specific targets of user hitting the web site.

September 2010 | T.E.S.T

6 | T.E.S.T cover story

3. Identifying potential metrics: Selection of appropriate metrics enables the practitioner to determine the system performance with respect to the established quality benchmarks.

Metrics allow the tester to quantify the results of the load tests. There are various tools available which allow to measure the time of running a test case. The metrics presented in table 1 may be used to monitor the system’s performance during load testing.

Working with the load testing team earlier in the software life cycle is important, as it will help in identifying metrics and ensure performance.

Execution of a load test requires time, effort, and commitment in order to perform tests and simulate actual test environment.

The load tests may be executed multiple times (repeatedly) and the measures may be recorded for future evaluations. The metrics corresponding to each change and retest must be recorded.

T.E.S.T | September 2010

4. Choosing the right tool and executing the load test: To implement the specified testing strategy and perform the established measurements, the right load testing must be chosen. The selection of tool depends on various factors such as nature and complexity of the web application, technology used in developing the web application and available resources (time, cost, skilled professionals) for testing the web application. For example, Microsoft’s Application Center Test (ACT) or Empirix’s e-Test tools may be used for testing complex web application and ACT may be used for testing application developed in .NET. ACT requires skilled professional whereas e-Test requires payment of license fees. Some of the load test tools for web applications include Microsoft’s ACT, Neolys’s NeoLoad, Radview’s WebLoad, Red Gate’s ANTS profiler, Yahoo’s Yslow, Webperformer’s Web Performance Load Tester, Empirix’s e-Test.

Execution of a load test requires time, effort, and commitment in order to perform tests and simulate actual test environment. The load tests may be executed multiple times (repeatedly) and the measures may be recorded for future evaluations. The metrics corresponding to each change and retest must be recorded.

Table 1: Load Testing Metrics

5. Interpreting the Results: The metrics collected by monitoring tool or manually must be analysed to check whether the system’s performance meets the user expectation. If the system’s performance does not meet the specified targets, then corrective action should be taken.

Stress testing Stress testing involves execution of a web application with more than maximum and varying loads for long periods. Unlike performance and load testing, stress testing evaluates the response of the system when system is given load beyond its specified limits.

It is also used to monitor and check the www.testmagazineonline.com

T.E.S.T cover story | 7

reliability of a web application when available resources are on beyond maximum usage. The behaviour of the system is monitored to determine when system under stress test fails and how does it recovers from the failure. Stress tests may test the web application for the following: • CPU and memory usage; • Response time; • Backend database; • Different types of users; • Concurrent users.

The system performance is expected to degrade when large number of users hit the site simultaneously.

After the completion of stress tests, the testing team must analyse the noted system’s performance degradation points and compare them with the acceptable performance of the system. Risk analysis may be used to make decisions of the acceptable level of performance degradation of the system.

About the authors

Yogesh Singh

Yogesh Singh is a professor with the University School of Information Technology, Guru Gobind Singh Indraprastha University, Delhi, India. He was founder Head (1999-2001) and Dean (2001-2006) of the University School of Information Technology. He received his master’s degree and doctorate from the National Institute of Technology, Kurukshetra, India. His research interests include software engineering focusing on planning, testing, metrics, and neural networks. He is co-author of a book on software engineering, and is a Fellow of IETE and member of IEEE. He has more than 200 publications in international and national journals and conferences.

Performance testing checklist Performance testing checklist may provide the opportunities for testers to check the issues related to performance requirements. A checklist may also discipline and organize performance testing activities in a web application. Table 2 shows a list of questions that may be checked while conducting performance tests.

However organizations and website owners may modify the checklist according to needs and requirements of their web application.

Table 2 presents a generic list of performance testing questions that are common to most of the web applications. The performance testing checklist may help to uncover the major performance related problems early in software development process.

Conclusion Performance testing includes load and stress tests issues that prevail during peak hours and under extreme conditions. Such conditions include exhaustion of memory, network www.testmagazineonline.com

Ruchika Malhotra

Table 2: Performance Testing Checklist

bandwidth, disk space, CPU cycles etc, due to simultaneous access of web applications. The effectiveness of performance tests is based on application of efficient methodology and correct analysis of results.

Performance has become a very critical issue for the popularity of a web application. It has become more prevalent due to very heavy competition amongst the similar type of websites. A search engine provides many options to a user and sustainability may be maintained if performance issues are handled effectively. Therefore, success of a website is dependent on the effective handling of performance related issues.

Ruchika Malhotra is assistant professor with the University School of Information Technology.

Prior to joining the school she worked as full time research scholar and received a doctoral research fellowship and her master’s and doctorate degree in software engineering from University School of Information Technology. Her research interests are in software testing, improving software quality, statistical and adaptive prediction models, software metrics, neural nets modelling, and the definition and validation of software metrics.

She has published more for than 35 research papers in international journals and conferences.

September 2010 | T.E.S.T

8 | Indian Testing

The testing landscape of India Vipul Kochur, chairman of the Indian Testing Board and co-president of Pure Testing takes an overview of the Indian software testing industry, where it stands today, how has it evolved and where is it headed.

I

deas are like fashion, they keep getting recycled. Some of the ideas in the testing world that keep getting recycled are - centralised testing team versus testing teams as part of development teams, focus on developer testing versus testing by independent testers, manual testing versus automated testing, early test design versus late test design and the debate of outsourcing versus in-sourcing versus in-house testing.

Indian software and testing industry has seen and participates in some of these debates. The concepts keep getting recycled in tune with what happens around the world as

T.E.S.T | September 2010

information is available at Internet speed and debates probably even faster. One important point to keep in mind is that while debates are a dime a dozen, skills are relatively rare and take time to acquire.

When I started my career as a developer, I was trained by my company in the various black arts of software development including good design and coding practices, debugging practices and surprisingly, software testing. Of course, testing was more geared towards unit and integration testing with focus on creation of automated tests and creation of tools for checking memory leaks, memory

corruption and performance. Those were good old days of C programming and developers were supposed to test their code and then integrate it and test it before releasing the product.

System testing was unknown to most organisations.

Since then testing has come a long way. System testing has become a norm, large and small software houses take pride in their testing centres of excellence (TCoE). Every aspect of software testing - people, processes or tools - has seen a sea change.

Testing in India Some of the key timeframes and www.testmagazineonline.com

Indian Testing | 9

thought processes prevalent at that time that shaped the Indic testing landscape are: • Early 90s – What is testing?

• Late 90s-2001 – I know what testing is but who wants to go there?

• Slowdown in 2001-2002 – Testing is an interesting career choice (I don’t have any other job) • 2003-2005 – I am proud to be a software tester • 2006 – Present – Testing Centres of Excellence (TCoE), era of pure-play software testing organisations Software test outsourcing has gradually increased and today all ISVs offer independent software testing as part of their outsourcing portfolio.

In addition, there are a lot of ‘captive’ centres which are in the IT divisions of various organisations such as banks, insurance and healthcare companies.

There is hardly any global organisation that does not have its offices in India. An unofficial account says there are more than 300,000 software testers in India.

The growth did not happen all of a sudden. When organisations started looking at outsourcing as a strategic move to tap skills and cost savings, maintenance appeared to be a logical first step to assess capabilities of the outsourcing organisation. A large part of any maintenance is the regression testing exercise. With organisations gaining expertise in testing, pure-play testing outsourcing has also become an industry in its own right.

With time organisations and individuals gained expertise in test design, automation, performance testing, test management, tool development and other areas of software testing. Today there are many players in this niche of outsourced testing where the organisations offer functional, www.testmagazineonline.com

non-functional and unit and API testing related services. Some niche organisations also offer tool development services in addition to automation.

Participation Interestingly, and probably logically, the growing testing expertise has resulted in increasing participation of Indian testers in the world forums.

People like Pradeep Soundararajan, Rahul Verma, Parimala Shankaraiya, Ajay Balamurugadas and many others are names that are recognised by many testers the world-over. Rahul Verma’s articles on Python meets Pareto are unique and so is his upcoming work on Test Automation Architecture.

Santosh Tuppad is another example of a person who has won a number of testing challenges and bug battles. A lot of testing innovation has happened in India, some of it in public domain and most of it as part of intellectual property of outsourcer organisations. Sahi is an example of a powerful web-based application testing automation tool coming out of India, created by Narayan Raman.

TestersDesk.com, created by Ashwin Pallarpathi is another example which provides a number of online test design tools for free. New test design techniques such as extension-tonoun-and-verb technique and new frameworks such as Q-Patterns, Hypothesis-Based Testing (HBT) from T Ashok have also come out of India.

Many Indians participate routinely in testing conferences around the world as speakers, key-note speakers, tutorial presenters and participants.

The number of testing conferences in India has gone up from one in 1999 to four or more large conferences and numerous small conferences in various cities.

Today there are many players in this niche of outsourced testing where the organisations offer functional, non-functional and unit and API testing related services. Some niche organisations also offer tool development services in addition to automation.

September 2010 | T.E.S.T

10 | Indian Testing

There is a hunger to acquire more knowledge, push the boundaries, do innovative work, deliver value to the customers and stakeholders and participate in the world movement towards better quality software.

These resources are ready to be tapped.

The coming few years promise to be interesting years for software industry in general and software testing industry specifically.

T.E.S.T | September 2010

Vipul Kocher Chairman Indian Testing Board www.istqb.in

Testing processes

Diversity

The testing world has also gained in the area of testing processes.

Executing testing projects spread across multiple countries and time zones with developers and testers separated by time and space has resulted in the development of processes capable of handling such situations. Interestingly, agile projects are also being successfully executed even when there is no co-location of developers and testers. New methods have been developed to cater to the needs of agility without sacrificing the cost or extended hours of test execution.

Increased demand for testing has resulted in increased demand for certification as well. Testing certification is often used by testing organisations to gain skills, showcase their expertise and motivate their testers. The last five years have seen a surge in demand for certification.

Indian Testing Board is an ISTQB approved national board for India which has been at the forefront of certifications. India today accounts for approximately 22 percent of worldwide ISTQB certifications and has more than 30,000 ISTQB certified testers.

In addition to people certifications, TMMi is gaining a lot of traction in Indian market. TMMi is being viewed by many organisations as a potent test process improvement method and framework, more than just a marketing tool. Currently there are many organisations undergoing formal and informal TMMi assessment and 2011-12 should see a large number of TMMi certifications for Indian companies.

Diversity of ideas is a hallmark of progress. In addition to people who promote certification there is also a an anti-certification group who do not see certification as necessary or some even find it useless. There are frequent debates among various groups related to various current topics in testing - exploratory and scripted testing, schools of testing, automation versus manual testing, unit testing versus independent testing led quality consciousness, usefulness of certifications etc being some of them.

There are Seven main IT hubs in India: Bangalore; NOIDA-Delhi-Gurgaon (Collectively called NCR or National Capital Region); Hyderabad; Chennai; Pune; Mumbai and Kolkata. Each city has its own niche such as general outsourcing, banking, engineeringservices, embedded systems and many other streams.

Today testing forms an important part of the software industry in India.

There are plenty of organisations with very talented, computer science engineers and domain experts, software testers. They are well versed with testing techniques, tools, processes, approaches and terminology. There is a hunger to acquire more knowledge, push the boundaries, do innovative work, deliver value to the customers and stakeholders and participate in the world movement towards better quality software. These resources are ready to be tapped. The coming few years promise to be interesting years for software industry in general and software testing industry specifically. http://india.istqb.org www.testmagazineonline.com

TM

Powerful multi-protocol testing software

Can you predict the future?

Forecast tests the performance, reliability and scalability of IT systems. Combine with Facilita’s outstanding professional services and expert support and the future is no longer guesswork. visit Facilita at:

WINTER 2010

4th October

7th December

Guoman Tower Hotel. London

Plaisterers Hall, London

Facilita Software Development Limited. Tel: +44 (0)1260 298 109 | email: enquiries@facilita.com | www.facilita.com

12 | Offshore testing

Offshore testing with a global reach T.E.S.T editor Matt Bailey speaks to Dr Sashi Reddi the founder and executive chairman of AppLabs a real power house in the Indian offshore testing business.

I

ndia is rapidly attaining global superpower status when it comes to software testing. As we have seen elsewhere in this issue, the country’s relatively high standards of education and its growing number of skilled professionals willing to work at lower wage levels than their colleagues in the UK, mainland Europe and the USA make it an ideal offshore software development and testing location.

And while the skilled and upwardlymobile testers may soon start to demand higher wages and better conditions, India remains a low cost-base location for offshore testing compared to its first-world competitors.

T.E.S.T | September 2010

A testing giant Dr Sashi Reddi is the founder and executive chairman of AppLabs, one of the world's largest software testing and quality management companies.

Since its inception in April 2001, Reddi has grown AppLabs from a three person outfit in a shared office space in Philadelphia, USA, to its current size with 2,000 employees in the US, UK, and India.

A serial entrepreneur, Reddi started two other companies prior to AppLabs. His first was EZPower Systems, a developer of products for building and maintaining largescale web applications. That business was acquired by DocuCorp and then eventually by Oracle, its technology

continues to be a part of Oracle’s document management offerings to this day.

Following EZPower, Dr Reddi founded iCoop, a group-purchasing dotcom that went bust in the dotcom bubble.

He received his BTech degree in Computer Science from the Indian Institute of Technology in Delhi and went on to gain an MS in Computer Science from New York University and later a PhD from The Wharton School, University of Pennsylvania.

“I founded AppLabs in April 2001,” remembers Reddi, “soon after the dotcom bubble when my company iCoop went bust. The crash came and funding became an issue. It was a time of cost cutting and as all my www.testmagazineonline.com

Offshore testing | 13

development work was being done in India I decided to package that service for the US market. The type of small technology companies we were working for were firmly rooted in development, so they were reluctant to outsource that function, but they didn’t mind outsourcing the testing side as this posed no threat to their intellectual property.”

While a typical customer didn’t mind spending money on development, they didn’t like spending on testing.

“I decided to go out and try to build something for the testing sector,” says Reddi. “But people didn’t like spending money on testing, they weren’t bothered about spending on development, but testing was a different matter. By 2006 it was becoming clear that there was only so much test spend smaller companies could afford; the real money was in enterprise space. We bought a company in the UK doing testing in the enterprise sector. Since we shifted our focus four years ago this sector has grown to account for 70 percent of our revenue.”

Key verticals Since then AppLabs has had success testing custom applications in key verticals for big players in big markets.

It’s most successful market remains financial services (stock exchanges, insurance, banking), followed by technology (large and small software vendors) and then the airlines where AppLabs has served five of the world’s top ten airlines. “We were initially testing the customisation of big software products for these enterprise customers,” says Reddi. “Now, more often, we test custom-built applications for them.”

Two thirds of AppLabs’ revenue comes from the USA, with the remaining third predominantly accounted for by the UK. “Outside these two key markets we don’t see much business currently,” says Reddi, “but we are expanding into continental Europe and doing work for a few Indian customers including the Bombay Stock Exchange, but as you would expect, margins are lower in the subcontinent.

“Our biggest competition is from the big software players who want to offer

the whole package. But we say testing is better done by independent third party players. We compete with the big vendors for all our major contracts and when we win the work we have to work with them – or often three or more vendors at a time - to complete the project. In contracts like these, we are the glue that provides a quality bridge across these streams of work.”

Testing in the UK It is key for AppLabs to have a strong operation in the UK. Sashi Reddi admires the level of attainment in Britain. “The skill sets in testing in the UK are very mature,” he enthuses.

“There is great project management and the testing community is well connected to each other. There’s a strong community with lots of sharing of information. Practitioners are in testing because they want to be. They are professional testers who look at testing as a career.”

While there may be a storm coming in the UK as the public sector is hit by cuts, Reddi remains optimistic. “We won’t be hit by the freeze in public

Over half Of the UK’s tOp 50 e-COMMerCe sites* test with Site ConfidenCe. shOUldn’t yOU? *source: comscore Media Metrix (dec 09)

For web site monitoring & load testing: Call 08445 380 127 Email info@siteconfidence.com Web www.siteconfidence.com

siteconfidence an ncc group company

14 | Offshore testing

sector spending because we have never picked up too much business in that area,” he says. “There has been a sea change in our customers. Six months ago things looked bleak. But now the level of uptake has increased. We are finding more business on the corporate side.”

another 1,000 people in the next 12 months. The software development business is massive and approximately 25 to 30 percent of that business is testing. I see no reason to get into other business areas, we need to maintain our focus on quality,” says Reddi. “And we will continue to grow in this space – adding Part of the process. value for the customer; bringing to “We are doing a lot of Agile work bear the tools and methodologies and some of these practises by their that we have developed with our nature have to be integrated with major clients. Our recent acquisition the customer’s work. We have not of ValueMinds, an award-winning found it difficult to be integrated – the developer of automated testing tools, offshore testing can be a part of the further endorses our commitment and development process,” says Reddi. focus. With this acquisition, AppLabs “Now more than ever there is the continues to add to its toolkit of technology available to make testers proprietary testing tools to improve part of the ongoing process even the productivity of its workforce and when they are offshore. Our teams are help customers accelerate ROI on now mixed. We have people working Quality Assurance. at customer sites and in testing labs “We are committed to growing offshore; it’s a global team.” through the right mix of organic And increasingly the value added by and inorganic means. We are the offshore testers is the important actively seeking ERP testing companies part, rather than the money saved in the $40-50MM range to help by using an offshore resource. “The us address the large enterprise way we sell today is less about the market. Many of our customers are money to be saved, but about how demanding highly specialized ERP much money poor quality will cost you testing skills and we would be open without adequate testing – we promise to an acquisition in that space,” savings,” stresses Reddi. “But it’s also adds Reddi. about improving quality across an “We still have a high volume of organisation and how we can translate work in the UK and USA and we have that into a dollar value, rather than a long way to go and a lot of work just better software. When defects to do in our key verticals in these get into production the costs can be markets. There are opportunities to massive. This is what we will commit grow in back-end operations and this to reducing.” may be an area where we could build The cheaper workforce factor a test lab as part of our global test is also becoming less of an issue. infrastructure.”

“These days when you supply testing As in all other areas of the IT services, it’s no longer about the cost industry, cloud computing is having of the individuals,” states Reddi, “that an effect on AppLabs. “Cloud worked five years ago, but now the big computing has a big impact for us,” customers have big teams around the says Reddi. “When looking at the worlds. Now it is about the processes test infrastructure, the cloud is an and tools we can bring to bear. For obvious area for expansion. example, the skills we learned working But if a client already has the with the five major airlines we can test infrastructure we will make bring to bear on problems at smaller a private cloud for them. Many companies.” applications are being cloud enabled and we are testing many applications that are being converted for the cloud.

Brilliant future And we are sourcing testers through “We’re about 2,000 people strong the cloud too.” currently and I can picture adding T.E.S.T | September 2010

Sashi Reddi Executive chairman AppLabs www.applabs.com

“These days when you supply testing services, it’s no longer about the cost of the individuals,” states Reddi, “that worked five years ago, but now the big customers have big teams around the worlds. Now it is about the processes and tools we can bring to bear.

For example, the skills we learned working with the five major airlines we can bring to bear on problems at smaller companies.” www.testmagazineonline.com

DELIVERING THE BEST TESTING TALENT We believe that the right person can transform a business and the right job can change a life.

That’s why we make it our mission to find you the best talent that the market place has to offer.

We know that guaranteed assurance of delivery and peace of mind is critical when up against tight deadlines and go live dates. Our Testing & Quality Assurance practice recruits professional contract and permanent testing specialists who can ensure that your projects are delivered on time and within budget.

If you need a well informed opinion, advice on a specific area of recruitment or wider staffing project, please contact Sarah Martin at testing@hays.com or call +44 (0)1273 739272.

hays.co.uk/it

With offices nationwide, our testing practice has expert consultants who will work with you to scope and design an effective bespoke solution to cater for your project requirements. By focusing purely on the testing market, our practice occupies a unique place in the market, offering a highly specialised personal approach along with the resources, scalability and infrastructure to deliver across multiple projects or larger organisations.

16 | Future testing

What does the future hold for software testing?

Can we learn from software testing failures in today’s world to improve the future of software testing? Geoff Thomson, chairman of the UK Testing Board looks into his crystal ball.

T.E.S.T | September 2010

www.testmagazineonline.com

Future testing | 17

H

eathrow’s T5 software testing issues highlighted the importance of software testing, I hope that lessons have been learnt and that CIOs across the world will take note in the future.

I believe that in the future will we see changes in six key areas of software testing:

1. Accurate information provision (in the language of the recipient) Software testing’s role is the provision of data to enable go live decisions to be made. If we get this information wrong, wrong decision will be made. If testing can’t provide the right information it actually adds little value.

Key to this is the understanding of what data is required. So many times I see that test managers have just produced what they think is right without asking, or even worse don’t understand the goals of the project so don’t know what data should be collected to indicate whether those goals have been achieved.

Test management tools can help, but they are only as good as the way they are implemented. In a lot of cases this is straight out of the box, with no configuration or understanding of what is to be reported, with a view that the standard reports will be good enough! Test cases are then loaded and then it’s found that the right data is not available.

I believe the way forward for information provision is actually to learn something from the Agile world and start to work/talk to each other. Everyone’s requirement for information is different and so the closer projects work together and people mix the more we can understand and the closer we will be.

I think a software delivery project is much like an orchestra, if each part works together in harmony it can be beautiful, but if only one element works on its own the result will be disastrous. We need to work together so that the software delivery orchestra works, and delivers beautiful results.

2. Building quality in, rather than testing it out, prevention rather than detection Traditionally software testing’s objective is seen as finding defects, www.testmagazineonline.com

whereas in reality it is a great way of preventing defects.

It continually amazes me how many clients I see who leave testing until the last possible minute. When will companies understand that the earlier a test team is involved, and of course the better trained they are, the more they will recognise problems early in the lifecycle enabling them to be resolved before they become execution defects?

What do I mean? Well, if you knew your back wall was going to fall down, would you wait until it did or do something to stop it falling?

The same can be said for a software tester. If early enough in the lifecycle it is identified that elements of the software may fail would it not be better to stop it failing by some corrective action, rather than to leave it to fail and then checking it has?

What I am referring to here is a risk-based approach to testing, understanding what risks the product has at the earliest possible point and putting in place mitigating actions to lessen or completely remove the risk of failure

3. Ensuring that quality is considered and not seen as a burden, alongside time and cost Many have said that you can’t balance the three sides of the eternal project triangle: time, cost and quality. The view seems to be that you can’t have all three, you have to pick two.

In my experience, by adopting a collaborative approach and being able to get involved at the earliest point in a project, it is possible to achieve all three.

While working for a large insurance company, I liaised very closely with the development manager lending him my testing staff so that he could be sure that when his developers had finished developing, the code would work.

They asked for a week’s extra time, which I agreed to. When the code was delivered to test it just worked and we therefore reduced the lifecycle timescales by six weeks and saved over £400k. The developers were happy, the testers were happy and most of all the business got their software early.

The testing cost roughly the same as the back-ended budget, but was spread out from a far earlier point, all

I believe the way forward for information provision is actually to learn something from the Agile world and start to work/talk to each other.

Everyone’s requirement for information is different and so the closer projects work together and people mix the more we can understand and the closer we will be.

I think a software delivery project is much like an orchestra, if each part works together in harmony it can be beautiful, but if only one element works on its own the result will be disastrous. We need to work together so that the software delivery orchestra works, and delivers beautiful results.

September 2010 | T.E.S.T

18 | Future testing

of the savings were in development not having to support an extended testing window.

The key to this success was the joint ownership of the quality between all parts of the project team. So by focusing on quality early in the lifecycle you can meet the joint objectives of cost, time and quality.

4. Software testing qualifications The reality is that a good software tester or test manager needs a mixture of skills and knowledge to make them successful. In my view the qualifications account for about 10 to 15 percent of the knowledge required for the practical application in the workplace.

I believe software testing qualifications have a very important part to play in the future education and verification of software testers and test managers. The reality is that the qualification together with real practical experience is what makes a good tester. After all would you allow a surgeon who had just passed their exams but had no practical experience to operate on you? I don’t think so.

Employers need to consider what structures and support need to be in place subsequent to a qualification course to ensure what has been learnt is put into practice in the best possible way.

5. Standards The few standards currently out there are developed by a wide working international group (can be as many as 1,000 contributors) who build and review the content. However, by necessity they are generic and some concessions have to be made to gain global agreement.

It continually amazes me that even knowing that the standards exist, T.E.S.T | September 2010

testers prefer to start from a blank sheet of paper when constructing their processes and templates. They may not be perfect but to start from a point above zero has to be better than no help at all?

I believe we need to embrace the standards more and take from them the information that is right for the project.

6. Test process measurement models There are many test process measurement models available for testers to review what they do against. For some very strange reason, a lot of test consultancies will build their own version, mainly for the creation of a sales opportunity and not to help the customer. As a result, the models are woefully short of the real detail required to help improve processes.

There are many standard models out there, but in my opinion the most effective model available today is the new TMMi Model. It is a model that has been developed out of research (rather than commercial interest) and therefore provides a very independent and structured view. It works along the same lines as CMMi, providing five levels of maturity to measure against and supporting key process areas.

Again as in the fifth point above, it may not be perfect but like standards it is there and has many hours of experience vested in it, so why do people still start from a blank sheet of paper when looking at their process, when a short cut is available?

Details of this model, which has been developed by the TMMi Foundation, an international organisation whose aim is to establish a non commercial test process assessment model for the industry, can be found at www.tmmifoundation.org

Geoff Thomson Chairman UKTB www.uktb.org.uk

It continually amazes me that even knowing that the standards exist, testers prefer to start from a blank sheet of paper when constructing their processes and templates.

They may not be perfect but to start from a point above zero has to be better than no help at all? www.testmagazineonline.com

In Touch With Technology

SUBSCRIBE TO T.E.S.T. by Supported

er 20 09

T E S T E R T H E E U RO P E A N S O F T WA R E T E S T E R N S O F T WA R E T H E SETUERRO P E A Volume 2: Issue 1: March 2010 THE EUROPEAN SO A R E T Ee 1: Issue 4: December 2009 F T WA R E T E S T E R Volum Volum e 2: Issue 2:

t es

pte mb Iss ue 3: Se Vo lum e 1:

IN TOUCH WITH T ECHNOLOGY

: ig ide .T D Ins T.E.S

SOFTW ROPEAN

T.E.S.T THE EU ROPEAN SOFTW ARE TESTER

ESTER ARE T OFTW EAN S UROP THE E

ARE TESTER EAN SOFTW THE EUROP

T T.E.S.

THE EU

T.E.S.T THE EUROPEAN SOFTWARE TESTER

ECH WITH T T.E.S.T

CH IN TOU

Supported by

IN TOUCH WITH TECHNOLOGY

e ag -p 16

ECHNOLOGY YCH WITH T TOU N IONL O G

June 2010

VOLUME 2: ISS U E 2 : J U LY 2 0 1 0

VOLUME 2: ISSUE 1: MARCH 2010

009 ECEMBER 2 ISSUE 4: D VOLUME 1:

9 R 200 TEMBE 3: SEP ISSUE ME 1: VOLU

HIGH HEELSDEALING G N ! H TECW ILEHIG BRIDGI IPHATE AGIN HITH DEBT A G THE of Agile Devon Smith with a woman’s view of a hi-tec Philindustry Kirkham tackles technica n takes on the cult Dave Whale l debt approach ile ag e e epted testing ic th s User acc serv virtual worlds |Inside: ke a ta as tie g Risk-based is in testing | Checking automation |Insid st urity | Testing Usere:testing | aTesec g Sma in ll-sca st Dat le James Chr testi de: te ng d | Reporting | owInsi

Enhanced application testi ng nline.com agazineonline.com magazineo online at www.testm Visit T.E.S.T online at www.testmagazineonline.com www.test Visit T.E.S.T online at Visit w T.E.S. no T onlin in e at www.testmagazineon T.E.S.T line.com

sting | Cr -based te Inside: Risk

Simply visit www.testmagazine.co.uk/subscribe Or email subscriptions@testmagazine.co.uk *Please

note that subscription rates vary depending on geographical location

Published by 31 Media Ltd Telephone: +44 (0) 870 863 6930 Facsimile: +44 (0) 870 085 8837

www.31media.co.uk

Email: info@31media.co.uk Website: www.31media.co.uk

The European Software Tester

20 | Offshore testing

Now the offshore ‘Honeymoon’ is over, where next?

What are the challenges currently facing the offshore testing industry as the global financial situation puts the focus firmly back on cost? Adam Ripley reports.

T.E.S.T | September 2010

I

f there were any doubts about the positive future ahead for the offshore IT services market the period of recession certainly put paid to them. The cost focus has reasserted itself on the IT services landscape and it is certainly number one on the list of reasons to take projects offshore.

However, the past two years have seen a gradual shift towards companies taking a more careful approach to using offshore testing services.

Surveys by the leading analysts have highlighted the drawbacks of companies simply adopting offshore testing services without considering some of the potential issues: • Lack of end-to-end coordination of the test process including a flow from Business Analysis; • Complex stakeholder and expectation management; • Lack of domain knowledge initially; • ‘Command and Control’ culture suppresses issue identification in offshore centre; • Inadequate requirements gathering; • Data security concerns; • Missing feedback and innovation loop.

In particular loss of ‘innovation’ and lack of understanding of the customers business, were cited as the top issues in Forrester’s survey of 300 top European companies that use offshore IT services.

Of course none of the above are show stoppers and many of them already arise in a project that is completely onshore. However, unlike a project that is completely onshore, the ability for a business to work around issues, by quickly assigning a business analyst to the team or focusing the team on requirements gathering, is not readily available when it is offshore.

In truth each of these issues is exacerbated by the distance between www.testmagazineonline.com

Offshore testing | 21

the two entities involved. When you ask a friend to buy you a sandwich at the shop you’d better be clear about your requirements or you will get something you didn’t want because you won’t be there to influence things when the purchase is made!

Structure & process maturity So if these issues drive the shift towards an environment where structure and process maturity is a must to exploit offshore capability then it is a very positive one. It means they are going into a deal with their eyes wide open, understanding the pitfalls, and considering the preparation they will need to make to ensure the project is a success.

Potentially this is making life easier for offshore providers, but it also means that new customers are seeking to establish how the offshore services company will address the potential issues when they select their provider.

So if the users are getting more sophisticated, then how should the offshore companies respond? To simply qualify out of opportunities where potential customers do not have adequate processes is unlikely to work. On the other hand to win the deal they cannot just respond by simply offering more onshore staff.

The costing model is probably the most critical aspect of the proposal.

Let’s consider the costing model for a moment. In a competitive bid the offshore staff day rates are pretty well comparable between the various suppliers. Therefore the most critical factor becomes the split of offshore/ onshore staff and the level of innovation and efficiency that can be promised. New customers are looking to maximise the use of the lower cost onshore staff against more expensive onshore staff. www.testmagazineonline.com

In fact a few years ago winning a deal was a matter of proposing to do more of the work offshore, thereby reducing the total project costs. But, savvy customer are now realising that simply taking the proposed approach by the potential supplier will land them with many, if not all, of the issues identified above.

The enlightened offshore generation So in this new age of enlightened offshore purchasing the offshore testing service providers are responding and differentiating their service by more than just cost.

They are proactively responding to the issues in a way that gives customers confidence. They are building onshore capability to consult and help existing and future customers address the following critical questions: Is it the right project? It will be an indication of the quality of the supplier on how much work they do to ensure the project fits well with their capabilities and the attributes are well matched to it being taken offshore.

For example, is it big enough? It needs to be approximately 10 people or more to offer the savings to match the effort involved and really anything below 20 may mean it should meet other criteria such as duration and quality of documentation. A second consideration is whether the project is clearly defined and autonomous from other onshore activities, migrations and conversions are suitable because they can be tested separately and the users don’t need to be involved until acceptance testing. Software packages and products are also a good match.

Other considerations will be matters such as whether they have the correct skills match and how easily they can take on the skills needed.

An offshore test team can only test the system well if they have the information necessary. A good supplier will develop and communicate a detailed plan. Onshore consultants will be integral to executing and maintaining the plan, in many cases taking charge of the project close to the client’s business team.

September 2010 | T.E.S.T

22 | Offshore testing

What is the best engagement model? Considering factors such as knowledge acquisition, formal training, automation, mobilisation, transfer of skills and offshore/onshore split.

Leading offshore testing providers are addressing these in a less prescriptive way and marketing their flexibility.

This is a direct response to the needs of potential customers. For example some companies we encountered over the past two years have spent time carefully selecting the onsite team with the supplier, recognising that the calibre of people initially onshore will be critical to the project’s success.

Despite the increased costs, they are also accepting proposals to begin with a larger onsite team before scaling down as maturity develops.

How do we address stakeholder management? The success of the project will always depend on how well the project is managed from the customer’s side. As a user you know your requirements better than your offshore test centre. An offshore test team can only test the system well if they have the information necessary.

A good supplier will develop and communicate a detailed plan.

Onshore consultants will be integral to executing and maintaining the plan, in many cases taking charge of the project close to the client’s business team.

The test project will be broken down in such a way that offshore teams have the various elements required (eg, software, hardware, data) in a timely manner and can execute simultaneous streams of test. A ‘common time window’ is agreed where the onshore and offshore teams can openly interact. Conference calls will be regular, dynamic information provided prior to the call for review and actions promptly distributed by email.

How do we ensure that we have a mature process in place? Not only T.E.S.T | September 2010

that the process is mature, also that the levels of maturity are matched between offshore and onshore.

Testing is inherently difficult to measure; when the test team is offshore the problem is amplified. So having a clearly defined process with detailed scripts and a repository to keep the test cases and results will be important.

What about test data privacy and IP protection? Data sent offshore faces the equivalent risks to data kept in-house. Yet the potential backlash from a leakage will be much more damaging if it happens offshore. Also of concern would be the different legal coverage that companies are covered by. For this reason you will expect a potential partner to cover this area in depth. ISO27001 is important certification in this area, but this is only part of the picture, you will need to ensure your supplier really understands the data and the software they will be receiving and can explain their process for ensuring it is managed and made secure.

Integrated global sourcing As offshore testing services companies continue to address the above issues their services have, as a result, become more proactive and sophisticated, in fact the level of capability they require in the onshore location rivals the more traditional onshore based testing services companies. So where next for offshore testing services companies?

Gartner predicts a continued growth in the market for global sourcing, in its definition of onsite, offsite, nearshore and offshore are integrated and the supplier controls the movement.

It estimates the market for global sourcing will rise substantially. Where it was nine percent of the total worldwide IT services market it in 2009 it will grow to between 13 and 16.5 percent by 2013.

In a Global Sourcing model the boundaries to where the sourcing takes place are blurred and a culture of communication optimises the location of execution based on time and cost. Of course there is a natural affinity for testing to fit into this model, in particular flexibility on time of execution is a critical factor.

Imagine a world where test execution could take place immediately the code drop was ready, rather than being dependent on availability of environments and people.

What this will mean of course is that the concept of an ‘offshore project’ will no longer exist. The sourcing of test services will be provided by global suppliers who manage the delivery through their global capability.

Of course this will only apply to the areas of testing that are suitable for being executed in a location away from the business teams.

Functional testing, some nonfunctional testing and acceptance testing are often seen as working better when executed closer to the business implementation, particularly if the integrator is supplying development and testing services.

It is therefore likely that these areas are going to continue to be addressed by specialist suppliers where the customer likes to keep some form of independent verification of the delivery.

Finally, on the topic of innovation; it is currently seen as an issue for companies where their development and/or testing is being undertaken offshore. They miss the strategic input and currently look to more specialist suppliers for that input.

For the smaller pure play offshore companies the development of more global capability, with increased onshore firepower, will hopefully address this gap.

Adam Ripley Managing director Certeco www.certeco.co.uk

www.testmagazineonline.com

WHAT HAPPENS WHEN YOUR USERS HIT ‘ENTER’?

Test your application performance in live network conditions • • • •

How many users can your application support?

What is the impact of users on your servers and network?

How does your application perform over different access technologies?

Is your cloud application hosting secure?

For your free test methodology prepared by a Spirent expert, call Daryl Cornelius on +44 (0)1293 767970. Or email EMEA-leads@Spirent.com The world’s leading communications companies use Spirent’s solutions to evaluate performance of the latest applications, technologies and networks.

www.spirent.com

24 | T.E.S.T Profile

Targeting new technology Although it has stiff competition from the giants of the software industry, Facilita has carved out a strong niche supplying a complete load testing solution to its customers. T.E.S.T speaks to the company’s managing director Gordon McKeown.

F

acilita claims to be among the few companies supplying the testing industry that is able to offer a complete load testing solution. It is the creator of Forecast , which the company describes as an innovative and cost-effective performance testing tool. Facilita’s clients include many of the UK’s largest organisations, such as government departments, major financial institutions, systems integrators, household names in e-commerce, specialist testing consultancies and software product vendors.

Gordon McKeown is an IT veteran, experienced software designer and joint founder of Facilita. He is deeply T.E.S.T | September 2010

engaged with the technical challenges of developing load testing solutions as part of his role as Facilita’s managing director.

Work and Pensions, commissioned Facilita to create a load testing tool for the comparative benchmarking of servers from suppliers such as IBM. We developed and supported the software T.E.S.T: What are the origins of the and sold it commercially in a limited company; how did it start and develop; way (mainly to the DSS suppliers). The how has it grown and how is it software was then developed further structured? (becoming Forecast V2) and was sold Gordon McKeown: Facilita is privately to several blue chip organisations. owned with headquarters near The decision was made around Congleton in Cheshire. We started as the year 2000 to specialise as a a Software design and development load testing vendor and to create a service company and the founders, commercial load testing tool that built who are still at the core of company, on the experience we had gained. With are seasoned software developers. The Forecast Version 3 (V3) we started mid 1990s saw our first significant to make significant inroads into the involvement in test tool creation.

UK test tools market. Our product The DSS, now the UK Department of development continues to be very www.testmagazineonline.com

T.E.S.T Profile | 25

Gordon McKeown, managing director, Facilita

intensive. When we created Forecast V4 we carried out a major re-design, incorporating customer feedback, and created a flexible platform for adding functionality, targeting new technology and enabling the testing of difficult user applications.

The Forecast toolset is central to the company. Our ambition is to create the most effective and sophisticated load testing tool and to deliver outstanding support and highly focused services. We are not trying to be a generalist testing services company. Rather, we offer specialist services that support and complement our software and which exploit the wealth of experience that we have in the highly technical discipline of load and performance testing. We also work with our delivery and technical partners to deliver comprehensive testing and QA solutions.

potential customers who are swayed by large vendors pushing ‘suites’ of testing, planning and monitoring software. Why settle for an inferior or less cost-effective load testing product because you have chosen a particular test planning tool or monitoring software?

A distinguishing feature of Facilita is our high standard of support. Load testing is technically demanding and customers really appreciate our backup. Apart from support and formal training we offer mentoring where an experienced Facilita staff member works within a customer team either to ‘jump start’ a project or to cultivate advanced testing techniques. Forecast simplifies test script creation using intelligent generation from application traces. However, even with Forecast’s outstanding script automation features, scripting remains a challenge for some complex applications, we, therefore offer a direct scripting service to our clients. We advise on all aspects of performance testing and carry out testing either by providing expert consultants or fully managed testing services.

We partner with specially selected technology companies that provide excellent complimentary products to broaden our offering. For example Itrinegy supply network emulation tools, Original Software functional testing tools and DynaTrace deep server diagnostics and monitoring.

T.E.S.T: Does the company have any specialisations within the software testing industry?

GM: We are focused on load and performance testing believing that by specialising we can provide much better products and services. However, T.E.S.T: What range of products and we do have a wider perspective. services does the company offer?

GM: Forecast is our core product. It is a The distinction between load testing modular load and performance testing and related areas such as functional testing and performance engineering tool that we sell along with related is not clear cut, for instance our services. Forecast has advanced product is ideal for multi-user features and targets a wide range of functional testing. By exploiting the technologies. It is open in design and works well with other tools. It has the flexible nature of our technology we are expanding tool functionality features needed for Agile testing as well as ‘traditional’ testing approaches. into several new areas in response to market and technology change.

The ease of integration reassures www.testmagazineonline.com

There is a growing acceptance of the need to invest in testing, including load testing. The picture is very mixed in terms of the sophistication and effectiveness of testing in practice. We still get calls along the lines of “Please test our new web site, it is going live at the end of next week.”

On the other hand some of our customers have implemented fantastic testing practices and test infrastructures around our products.

September 2010 | T.E.S.T

26 | T.E.S.T Profile

Technical challenges will multiply rather than diminish. Systems are being designed as ever more complex mix-ins with multiple end-user interfaces (desktop, mobile, rich web) so there is a terrific challenge to tool developers. Flexible and open design will be paramount for commercial testing tools.

There are grounds for optimism with regards to testing in general and load testing in particular.

One engine for positive change is the Agile testing movement which has an influence even on organisations that are not fully committed to this approach.

T.E.S.T | September 2010

T.E.S.T: Who are the company’s main customers today and in the future?

GM: Our customers span a variety of sectors, for instance: e-commerce & consumer websites: Shop Direct, ASOS, Unilever; Financial and Banking: CMC Markets, ING Bank, Copperdime; Government: Department of Work and Pensions, Insolvency Service; Defence and Engineering: EADS, Intergraph Command & Control Systems; Application Vendors: Bond Adapt Recruitment Software, NCode Operational Monitoring.

As a tool vendor we directly compete with the market leader HP and the LoadRunner tool. Some of our customers use both tools. Where we win it is for a variety of reasons including value, support, license flexibility and technical capability. The corporate ‘LoadRunner space’ is an important target market.

Another target is organisations that have tightly integrated load testing with development, possibly using Agile methods. Here we are mainly competing against open source tools where we have to demonstrate the added value of our functionality and support. A third market sector is software vendors. Because we offer flexible licensing and the ability to customise the tool to target complex applications, a number of software vendors have adopted Forecast rather than create a bespoke test harness.

Finally, I should mention testing service companies and Systems Integrators who like our licensing because it allows them to bundle tool and services in an economic package for their clients.

Forecast’s extensibility allows the targeting of vertical markets characterised by technology, for instance trading systems. This sort of development does not have to be carried out by Facilita. We are planning initiatives to create a market in addins from independent vendors. A recent example of successful vertical targeting is mobile technology.

T.E.S.T: What is the company’s commitment to corporate social responsibility, ie, ‘green’ issues?

GM: We have resisted the temptation to ‘green wash’ our marketing but would make the simple but often overlooked point that effective load testing provides the empirical data needed to avoid over-specifying hardware for production. This not only saves energy and other resources it also saves money. Forecast is also very efficient in its use of hardware.

T.E.S.T: What is your view of the current state of the testing industry and how will the recent global economic turbulence affect it.

What are the challenges and the opportunities?

GM: The demand for testing is linked to IT investment which in turn depends on general economic activity and business confidence. Despite moves towards integrating testing with development the demand for testing resources still appears to be biased towards the end of the project cycle. Therefore suppliers of testing services and tools seem to experience a time lag in terms of moving in and out of recession. We have grown significantly despite the recent problems with the economy so feel well positioned for the recovery. We seem to be witnessing an upswing in demand for testing services and tools so, let’s hope we are all spared the dreaded ‘double dip’!

A positive outcome of the economic situation is that organisations are searching for better value options. So, rather than automatically choosing the market leader for new tool purchases or renewing expensive maintenance contracts, they are now talking to us.

There is a growing acceptance of the need to invest in testing, including load testing. The picture is very mixed in terms of the sophistication and effectiveness of testing in practice. We still get calls along the lines of “Please test our new web site, it is going live at the end of next week.” On the other hand some of our customers have www.testmagazineonline.com

T.E.S.T Profile | 27

implemented fantastic testing practices and test infrastructures around our products.

A persistent problem is the underestimation of the effort and technical expertise required to carry out load testing properly. As tool vendors we are committed to reducing this effort and increasing tester productivity. But let’s be realistic. Load testing is a technically demanding discipline and there is a genuine problem assembling teams with the right mix of skills. Bluntly, there is a shortage of good quality practitioners with in-depth technical experience, which provides both challenges and opportunities. Understanding this, our approach is to combine innovative product development with highly skilled support and services.

T.E.S.T: What are the future plans for the business?

GM: Our focus will remain on product development and broadening Forecast’s scope by adding new protocols and features such as system monitoring using artificial transactions.

Targeting new technologies and client feedback are the main drivers of our development roadmap.

We are developing new ways of delivering test services, for example Cloud based testing. We are investing in our customer support services to maintain and improve our high levels of customer satisfaction while we continue to expand in the UK and internationally. We intend to seek additional strategic technology, delivery and re-sale partners.

T.E.S.T: How do you see the future of the software testing sector developing in the medium and longer terms?

GM: Technical challenges will multiply rather than diminish. Systems are being designed as ever more complex mix-ins with multiple end-user interfaces (desktop, mobile, rich web) so there is a terrific challenge to tool developers. Flexible and open design will be paramount for commercial testing tools. www.testmagazineonline.com

There are grounds for optimism with regards to testing in general and load testing in particular. One engine for positive change is the Agile testing movement which has an influence even on organisations that are not fully committed to this approach. We expect to see greater integration of automated multi-user functional testing and load testing within the development process.

Though this will not necessarily mean an end to the ‘classic’ independent test team that load tests either regular or infrequent versions of software prior to release. Many organisations will combine both approaches and there are organisations that source their systems externally that will require episodic QA exercises including load testing as part of acceptance and commissioning.

Will the trend continue of commercial test tool providers being absorbed by enormous companies then losing their creativity and providing less responsive support? As an independent vendor we think this is likely and will therefore give us a competitive advantage.

What of open source tools?

Undoubtedly they will play a role but we don’t think they will displace good, well supported commercial tools as they lack functionality and tend to require advanced programming skills for creating anything more than simple tests.

There is a tension between the more Agile approach to testing and outsourcing. It will be interesting to see how that is resolved. The emerging economic powers are bound to impact both as suppliers and consumers of tools and services. Organisations will be more discerning and will mix and match outsourcing, in house development, Agile and traditional methods. That should increase opportunities for specialist firms with a strong track record as will the increasing desire to maximise value for money.

T.E.S.T: Gordon McKeown, thank you very much www.facilita.com.uk

There is a tension between the more Agile approach to testing and outsourcing.

It will be interesting to see how that is resolved. The emerging economic powers are bound to impact both as suppliers and consumers of tools and services.

Organisations will be more discerning and will mix and match outsourcing, in house development, Agile and traditional methods. That should increase opportunities for specialist firms with a strong track record as will the increasing desire to maximise value for money.

September 2010 | T.E.S.T

28 | Penetration testing

Evolve or die!

Industry analysts say that as much as 75 percent of all attacks are now targeted at the application layer. David Harper looks at the changing face of penetration testing and tackles its fundamental flaws.

F

or a long-time we have relied on penetration testing to address the threat of attacks aimed at the application layer.

There are several ways to conduct penetration testing: black box testing assumes no prior knowledge of the system being tested and is often conducted as an outside hacker, white box provides the tester with complete knowledge of the infrastructure and therefore considers the internal threat or someone with inside knowledge.

Grey box testing is variations between the two. While the relative merits of these approaches are debated, there are a number of reasons why penetration testing, as it currently stands, is fundamentally flawed: T.E.S.T DIGEST | September 2010

It isn’t deterministic: Despite the increasing sophistication of the tools available, penetration testing will still come down to two key factors: the skill of the tester, and the time available. If you want to test this theory then the next time you commission a penetration test give the tester more time and he will find more issues! Alternatively, get two different testers to perform a penetration test on the same application and you will find that you get a different list of issues back.

It provides the wrong information: Penetration test reports are despised by the development organisation., Chiefly they report vulnerabilities based on the URL without any real www.testmagazineonline.com

Penetration testing | 29

advice on the underlying cause so it is then left for the developers to ponder the problem, pontificate the possibilities and often through a process of elimination discover how this relates to the code that they have developed. This, combined with the lack of security knowledge within the development organisation, make vulnerabilities difficult to fix.

It occurs at the wrong time: The nature of penetration testing means that it can only occur at the end of the development life-cycle. The problem is that this is really the worst possible time to fix an issue. As an order of magnitude, it is cheaper and quicker to fix an issue if it is discovered during development.

A better way Organisations are starting to allocate larger budgets to get code right in the first place and to embed security activities through the software development life-cycle. As the code is developed it needs to be checked for common coding errors that lead to attacks like SQL Injection and Crosssite Scripting attacks. During testing the security controls need to be fully tested and, yes, you still need to perform penetration testing but now its role is a final QA check not as the primary means of defence.

These security activities can’t be left to an individual project team to define. We suggest establishing a software security assurance (SSA) programme that is responsible for ensuring all software is developed to an appropriate security standard and also provides resources to assist the development teams to meet this challenge. A given is that the www.testmagazineonline.com

organisation needs to create a holistic programme that fits its requirements as a generic approach is not likely to succeed – this is one area where one size most definitely does not fit all.

Then there are the people within the organisation - when securing the applications an organisation uses is a key strategic priority, with buy-in from senior management, staff understand that this is not just a passing fad but something that is truly a major directive for the organisation that will have tangible business benefits.

While tools and technology play a critical role in the success of an SSA programme, they are by no means the only cog in this wheel - software security practitioners have a variety of tools available, ranging from static and dynamic analysis tools to binary analysis and fuzzing.

The Software Assurance Maturity Model (SAMM) is an open framework to help organisations formulate and implement a strategy for software security assurance that is tailored to the specific risks facing the organisation. It was defined with flexibility in mind so that it can be utilised by small, medium, and large organizations using any style of development. As an open project, SAMM content will always remain vendor-neutral and freely available for all to use. Visit www.opensamm.org for more information.

Penetration testers are not suddenly going to disappear off the face of the earth. Instead, we will see the practice undergo a transformation and be reborn as part of a tightly integrated approach to security. Penetration testing as a stand-alone solution is dead, long live penetration testing.

David Harper EMEA service director Fortify Software www.fortify.com

Penetration testers are not suddenly going to disappear off the face of the earth. Instead, we will see the practice undergo a transformation and be reborn as part of a tightly integrated approach to security. Penetration testing as a stand-alone solution is dead, long live penetration testing.

September 2010 | T.E.S.T

30 | Business objectives

Is the testing world still flat?

I As an advocate of testing for its business benefits, Susan Chadwick argues for the need to align testing with business objectives, ensuring that companies are not just testing for the sake of it.

T.E.S.T | September 2010

have a real passion for testing, and the contribution and real business benefit it can add to business change. I believe that testing is fundamentally a risk management process and the overall test approach for any organisation, change programme or individual IT project must be aligned to meeting business objectives and ensuring the business benefits are achieved.

In commercial environments, testing carried out purely for its own sake is often ineffective and costly from a business perspective.

This leads to testing being viewed as an overhead rather than contributing business benefit. Some indicators that testing is not anchored to business objectives nor being actively managed to meet them might include: • A test approach that aims to cover everything to the same level of detail and coverage; • A test approach that is not reviewed even though the project profile, timescales and risk profile have changed; • A lack of user acceptance criteria and no clear definition of when to stop testing; • Subjective and not objective progress and management reporting; • Never ending testing cycles; • Late engagement of testing in the project lifecycle;

• A large test strategy document which no-one has referred to since it was signed off; • Test process improvement that is not applied in the context of the businesses objectives.

Business objectives Business objectives should be used to ensure that any testing strategy is defined in context and that progress is measured against those objectives. In the same way that you would expect the approach to be different for an internal, non-critical business system to that of a safety critical system, you would also expect the strategy to be driven by business priorities in relation to elements such as cost benefits realisation, time to market or achieving competitive edge.

There are some key deliverables and activities for which testing can and should be aligned to business objectives and these include: • Corporate test policy and strategy; • User acceptance criteria; • Ensuring early testing engagement; • Risk-based delivery approaches; • Context-based test process improvement; • Engaging commitment through full communication of objectives and decisions.

Corporate test policy and strategy: Ensuring there is a corporate test policy and test strategy in place www.testmagazineonline.com

31 | Business objectives

Have you noticed how the super hero turns up just in time to snatch you from the jaws of disaster? It has never happened to me personally, but that is how the story goes. Now, have you ever wondered if our super hero’s time could be better spent preventing those disasters from occurring in the first place?

T.E.S.T DIGEST | December | September 2009 2010

establishes a clear framework for the testing requirements within an organisation. This ensures alignment to corporate objectives, maintains consistency internally, gives direction to third parties and avoids ‘re-invention of the wheel’ every time a new programme or project is initiated.

User acceptance criteria: User acceptance is clearly a key element of business engagement and there is a strong link here between the definition of user acceptance criteria and ensuring that the ultimate deliverables are fit for purpose.

That purpose includes, for instance, providing a solution which will deliver the defined business benefits. This is often referred to as User Acceptance Testing, but I prefer to shorten it to User Acceptance.

As a key part of the development lifecycle its purpose is to build confidence for the business that the solution, when implemented, will deliver the defined business benefits, meet the stated business requirements and is fit for business purpose. That does not have to be achieved purely by waiting until the end of the project and carrying out explicit user acceptance testing. Building business confidence, and hence carrying out User Acceptance, should be part of the end to end project lifecycle and can include static reviews, business process reviews, walkthroughs, witnessing tests, reviewing the output of previous testing and engaging business users in earlier testing phases.

Early testing engagement: Early engagement of the person who will be managing the testing is key.

This includes having the right test manager – emphasising the ‘managing’ and clearly differentiating it from ‘co-ordinating'. The test manager should establish a test approach that is optimal given the business objectives and constraints and manage testing to deliver to quality, cost and time.

That often means the project specific approach will be dynamic over the lifetime of the project.

The test manager must also have a focus on business objectives and

deliver real business benefit through a project specific approach which is effective and efficient. If a particular delivery is made to testing late, the right test manager will recognise the risks, review the approach and propose solutions or options. They must also have a good understanding of testing principles and practices and be able to apply that knowledge to establish the most beneficial approach for the project and the business.

Without early engagement there is a danger that the focus falls purely on finding faults (late in the process) and not enough on prevention. Have you noticed how the super hero turns up just in time to snatch you from the jaws of disaster? It has never happened to me personally, but that is how the story goes. Now, have you ever wondered if our super hero’s time could be better spent preventing those disasters from occurring in the first place? It is widely accepted that cost to fix increases exponentially the later in the development lifecycle an error is found. Hence relatively small investments in static testing techniques such as Requirements Testing can have a huge, beneficial impact on overall project delivery costs.

Risk-based delivery approaches: Risk-based testing approaches have a significant role to play in the current environment of pressures on time to market and cost while ensuring quality delivery. The complexity of today’s systems means that it is not practical to even attempt to test all possible combinations of variables and scenarios and so coverage is rationalised based on assessing the likelihood/probability of there being an issue in a particular area and the impact were an issue to occur.

The Boston Matrix approach is a firm favourite of mine as it supports a robust and simple approach which can be presented to management in their terms and reviewed objectively.

The impact element can be leveraged to drive and protect certain business objectives: high impact elements may include, for instance: www.testmagazineonline.com

Business objectives | 32

• Major revenue loss; • No workaround; • Unable to achieve mandatory business benefit; • Failure results in penalties; • Severe reputational impact; • Regulatory or legislative obligation not met; • Customer down time; • Security breach; • No recovery option.

Establishing these categories for an organisation or project creates a number of benefits: • It is in business language and can be signed off up front by business stakeholders and sponsors; • It provides objectivity for prioritisation, test coverage and focus of testing effort; • It supports pro-active test management.

Testers should also be equipped with a number of test design techniques which enable them to optimise test coverage whilst managing the actual numbers of tests required, again effectively applying a risk-based approach. These include, for instance classification trees, decision tables and my all time favourite – principally because of the beauty of the mathematics and the logic! – pairwise testing.

Context-based test process improvement: Similarly you can ‘improve’ your test process as much as you like but unless this is driven by clear business objectives you run the risk of not delivering real business benefit. Improving processes, including testing, can deliver benefits in terms of quality, time and cost. However, before embarking on such an exercise it is critical that it is put in the context of your particular business objectives.

This enables you to focus process improvement on the areas that will make a real difference. You may have test processes in some areas which are basic but still meet your business objectives. Changing those processes merely for the sake of increasing your level of testing maturity should therefore be questioned.

In my experience there are some key initiatives which consistently deliver www.testmagazineonline.com

process improvement and clear time, cost and quality savings. These can be aligned to critical areas reviewed as part of a short, sharp test process improvement assessment, providing recommendations for change aligned to pre-defined business objectives.

Engaging commitment through full communication of objectives and decisions: Testing teams are often not aware of the business objectives and constraints for a particular project and yet I have found that fuller communication of the real cost of change and the business drivers can have a transforming effect on the testing contribution and approach.

As an example, I was recently involved in a programme where the business users were taking part in the User Acceptance but had no concept of the actual costs involved.

Making them aware of the cost levels and therefore the actual cost to the business, for instance, of introducing changes late in the process made them much more aware of the impact of their activity and the context in which the project board and sponsor might reject rather than take on a change request.

Focus on the business objectives Testing is at a level of sophistication where more can be expected of its practitioners across a number of areas.

There is a toolkit of methods and techniques which should be used to better effect. By all means implement good testing practices, but also focus on the business objectives.

The only way we can move testing away from being seen purely as an overhead or a necessary evil is by tying it to achieving business objectives and aligning it to managing risk and delivering business benefit. For many organisations there is a view that testing costs too much and takes too long and frankly, they are often right.

As an industry, it is time to move our focus from finding errors in software to making sure they do not get there in the first place and recognising testing as a holistic process embedded in core business strategy.

You can ‘improve’ your test process as much as you like but unless this is driven by clear business objectives you run the risk of not delivering real business benefit. Improving processes, including testing, can deliver benefits in terms of quality, time and cost. However, before embarking on such an exercise it is critical that it is put in the context of your particular business objectives.

Susan Chadwick Co-founder EdgeTesting www.edgetesting.co.uk

September 2010 | T.E.S.T DIGEST

34 | Critical testing

Could testing have saved the Deepwater Horizon?

Brian Luff casts a critical eye over the lack of independent software testing on oil rig systems and how this needs to change to prevent disasters like the recent BP oil spill in the Gulf of Mexico from happening again.

T.E.S.T | September 2010

www.testmagazineonline.com

Critical testing | 35

A

s always happens, the news media has now lost interest in a story with which they were totally obsessed only a few weeks ago. The catastrophic effects of the BP oil spill in the Gulf of Mexico will be felt for a long time, perhaps for more than a decade, but the media have moved on to the next headline. Maybe the long term effects aren’t as obvious and dramatic as a flock of oil-sodden sea birds struggling pathetically to survive in their ruined habitat. They are felt by the proprietors of, and workers in, a devastated tourist industry. They are felt by pensioners whose investments are shrunk by the need to divert billions from what would have been profits into reparations and damages. They are felt by all of us for whom prices will go up as a result of a diminished appetite for deepwater drilling.

Although the media may have moved on, more responsible interested parties will be spending a long time and a lot of effort trying to figure out what caused the Deepwater Horizon explosion in April 2010, an explosion lest we forget that not only caused an environmental disaster but also claimed the lives of eleven people.

Perhaps, despite their best efforts, investigators will never be able to tell us what happened in which case we’ll simply have to be satisfied with speculation, or educated guesswork.

Such speculation has started already which has certainly struck a chord with us. How many people know what’s involved in drilling the sea bed for oil? Far from being a simply mechanical process, it actually depends on a lot of softwareintensive control systems. It’s not widely appreciated, but most of the sophisticated technology that shapes all our lives depends on a lot of software. Sometimes, software failures are an inconvenience. So you had to restart your PC? Big deal. How about if the pilot’s ‘glass cockpit’ packs up in the middle of your holiday flight. That gives a whole new meaning to the ‘blue screen of death’!

A software issue?

In the case of Deepwater Horizon, it’s clear from the Transocean interim report to the Waxman committee that control system software is falling under suspicion1. Reports have already surfaced in the Houston Chronicle2 that “display screens at the primary workstation used to operate drill www.testmagazineonline.com

controls on the Deepwater Horizon, called the A-chair, had locked up more than once before the deadly accident.”

Given the amount of embedded software in oil-rig systems, or the dozens of operations that are carried out under software control, it’s no wonder that software is getting the third degree.

Software is relatively easy to write.

Reliable, safety-critical software can be complex and challenging: however in truth often it isn’t much harder to write than the software that’s powering the browser that you use.

To really get close to perfection, it requires independent testing so that the developers’ assumptions, and even egos, are not allowed to stand in the way of the quest for those last few elusive bugs.

Independent testing of something that’s already been tested in the normal way, by its developer, is undeniably an extra expense. It’s not a prohibitive expense though – just the one that’s most likely to be cut when money’s tight and financial control is wielded by those that don’t really understand the true value of what they’re cutting. Our experience is that cost pressures are all too often allowed to bear on the safety-critical part of the software development process. Do we skip physical safety checks on trains and boats and ’planes? Not likely! So how is it OK to let finance directors and others of their ilk cause the cutting of corners when it comes to the more abstract and less tangible factors in the safety equation?

Until independent testing, by truly qualified testers, is recognised as sacrosanct within safety-critical developments then we’ll continue to have aircraft falling out of the sky, runaway cars, space-launch disasters and yes, oil rig disasters. At the dawn of a new era of nuclear power generation, it’s time to start changing attitudes now.

Footnotes: 1. "Deepwater Horizon Incident—Internal Investigation," draft report, Transocean, 8 June 2010, p. 15; http://energycommerce. house.gov/documents/20100614/ Transocean.DWH.Internal.Investigation.

Update.Interim. Report.June.8.2010.pdf.

2. B. Clanton, "Drilling Rig Had Equipment Issues, Witnesses Say— Irregular Procedures also Noted at Hearing," Houston Chronicle, 19 July 2010; www.chron.com/disp/story.mpl/ business/7115524.html.

Brian Luff Chairman Critical Software www.criticalsoftware.com

How many people know what’s involved in drilling the sea bed for oil? Far from being a simply mechanical process, it actually depends on a lot of software-intensive control systems. It’s not widely appreciated, but most of the sophisticated technology that shapes all our lives depends on a lot of software.

September 2010 | T.E.S.T DIGEST

36 | Testing training

Testers – equal amongst equals?

In the rapidly developing world of software testing Angelina Samaroo says the fight for recognition goes on, but there is light at the end of the tunnel.

T.E.S.T | September 2010

www.testmagazineonline.com

Testing training | 37

A

s a profession, we have had to fight for recognition as equals amongst equals.

When I started my career as a test engineer, I was told there was a possibility that I may be a bright girl, and so I should aspire to be a programmer. Because how much skill can reasonably be attributed to a person who just checks someone else’s work?

As you know, there is a big difference between possibility and probability.

I learnt how to write programs, and even wrote a few deep and meaningful ones. But I never aspired to becoming a programmer. In all probability then, I was never going to be equal. But, still, my Mum said she’d love me regardless, so how bad could it really get?

The good old days So I figured, with nothing to lose now, already a ‘failure’, and only just turned 21, I may as well just enjoy myself, at work and at play. So I carried on testing, regardless. And what fun it was, before and after hours. We did the job – by day, we tested; we wrote about it; we fought for fix. The software quality increased. There were fewer defects in production. There seemed to be something in this testing lark. By night – we tested, who could eat the hottest nuclear hot pizza, who could drink the most foul of concoctions, by the pint.

Then people began to leave the company – for the city, big money, bigger nights! Our company was quick on the uptake, they saw the danger, and dealt with it – comprehensively.

Our wages went up, and up, until we had pulled away from our peers– yes, there was a good side to Y2K. www.testmagazineonline.com

Fast-forward to today.

Now I’m all grown up. No more those heady nights, I’ve really got to set an example to my niece and nephew.

They’re about to head off to university – I need them to work hard so they can pay my pension. As an adult I’ve managed to ruin the planet and the economy for them - not by myself of course, but I’m not sure that’s an excuse.

As professionals though, they will face similar choices to mine.

Which job, which country? When I graduated, the advice was to choose the company with the best training programme on offer, not the one with the highest salary. Train first, gain later. Nowadays, like many things, companies are outsourcing their training programmes. In-house trainers still rattle around, but the hot courses seem to be the ones with the external certificates attached. These have their cachet, but they don’t come cheap.

Also, the recession is ignorant of its victims. It will hit anyone, including training and exam providers. They too will dive for cover. Training providers will try to fill up their classes by offering price reductions, exam providers will attempt to cut costs.

This may be a reason for the increasing number of multiple-choice-based exams on offer. These certainly have their value, but not at all levels, in all topics.

Testing has its prescriptive elements – that all testing should follow a process from planning through to closure; that a risk-based approach to testing requires identification, analysis and mitigation of risks; that boundary value analysis requires equivalent partitions to be identified first, amongst many others. The rule-book

When I graduated, the advice was to choose the company with the best training programme on offer, not the one with the highest salary. Train first, gain later. Nowadays, like many things, companies are outsourcing their training programmes. Inhouse trainers still rattle around, but the hot courses seem to be the ones with the external certificates attached.

September 2010 | T.E.S.T DIGEST

38 | Testing training

however, does not and cannot claim to inform on what the decision should be when things do not go to plan.

Effective decision making requires full understanding of all of the facts. It also requires that know-how – of the people, of the politics, of the budget, of past experiences. The rule book might say that if at first you don’t succeed, try try again. The rule book does not know that each exam costs time and money.

Buyer beware I know that the external certificates have a certain ‘sugar hit’ – a few days in the classroom, adrenalin pumping exam, certificate through the door.

However, we now need to consider – those bucks, that bang – will they match?

I am getting feedback that is not always the case – the hype did not turn into reality. The market likes papers – they make it easier to filter prospective new recruits. Although I do like the option of just throwing away half the CVs – you don’t want unlucky people working for you! (I took this from The Office – a comedy programme in the UK and US). And even if you are lucky, remember that at first review time, it will be you that will be assessed, not your CV. Those papers – where will they be?

So, if you decide that you must have a piece of paper, download the syllabuses and read them. Don’t rely just on the course outlines created by the training providers – they have a purpose – sales. You have a responsibility – budget. Know what you’re buying.

The syllabuses are freely available, read them! And yes, I know you’ll end up reading well over 100 pages for some – if you can’t stay the distance to read them all then put away that purchase order until you have. Buy the sample exam paper and check if the test is applicable to your needs. If T.E.S.T DIGEST | September 2010

there are no full sample papers, then keep that PO in its box. Remember the fundamental rule of trade – caveat emptor – let the buyer beware. Trade is trade, even if it is qualifications being traded. You are a professional – trust that you know what you want, check that you’re likely to get it, and evaluate from end to end. Everyone in the chain is accountable, from delegate to course provider to exam provider. This is the ‘Age of Austerity’ – definitely not the age of blind trust.

Back to the real world.

In 2010, our main worries are: Micro – what price our daily bread?

Macro – what price our daily wheat?

Intergalactic – wonder if we can get wheat to grow on Mars? OK, a bit off the wall, but you’ve got to admit, our own planet just isn’t playing ball these days, it just keeps on spouting stuff at us – clouds, oil, water, disease, fat cat bonuses.

At work, in order for us testers to earn our crust and pass those reviews, our testing skills must be complemented by other skills. As a test analyst, we would typically need to analyse specifications; liaise with business analysts; write test scripts; work with developers to get defects fixed; and write test reports.

Usually we would be part of a team, so we would need to be able to interact with people effectively. As a test manager we would typically be able to participate in risk workshops; write test strategies and plans; assess extent of testing carried out; write reports and present findings. As consultants we would be expected to be all things to all men, and an expert from day one, so pretty fast talking is essential. We also of course have to manage the bods back at base – really fast talking needed here!

To be effective in any of these roles we need to know not just which boxes to tick, we sometimes need

To be effective in any of these roles we need to know not just which boxes to tick, we sometimes need to create the boxes first. We need to gather requirements; write reports; deal with people; manage conflict; give presentations; manage budgets; amongst many others. www.testmagazineonline.com

Subscribe FREE to the most VitAL source of information e ag -p 16

st

/ June 2010 Volume 3 : Issue 5 : May

vital vital : ige ide L D Ins VitA

mod ern busin ess Inspi ratio n for the

VitAL : Inspiration for the modern business

for the modern business VitAL : Inspiration

vital

Inspiration for the modern business Volume 3 : Issue 6 : July / August 2010

Inspira tion for the moder n busine ss Volume 4 : Issue 1 : September / October 2010

IT delivers for Britain

standard IT Broadcast adc aster’s IT strategy The world’s biggest bro

How to do more with less in the public sector

5 : May/June 2010 Volume 3 : Issue

Volume 3 : Issue 6 : July/August 2010

A perfect storm

Rediscovering the value of IT

Dave Ramsden says it’s time to add more value

Reaching cloud nine

Securing the IT estate

Selecting the right cloud partner

Doing more with less

Greening the enterprise

climate IT in a tough economic

Punching above your weight

Using the cloud for rapid growth

The need for speed

Strategies for going green

Does faster broadband mean faster applications?

THIS ISSUE: CLOUD COMPUTING SPECIAL VISIT VitAL ONLINE AT:

WWW.VITAL-MAG.NET

VISIT VITAL ONLINE AT: WWW.V ITAL-MAG.NET

News, Views, Strategy, Management, Case Studies and Opinion Pieces

vital Inspiration for the modern business

www.vital-mag.net/subscribe 31 Media will keep you up to date with our own products and offers including VitAL Magazine. If you do not wish to receive this information please write to the Circulation Manager at the address given.

Please tick here ■ if you do not wish to receive relevant business information from other carefully selected companies.

40 | Testing training

Albert Einstein said “Try not to become a man of success but rather a man of value.” He said this rather a long time ago, and of course by man he clearly meant tester – he was way ahead of his day after all.

T.E.S.T DIGEST | September 2010

to create the boxes first. We need to gather requirements; write reports; deal with people; manage conflict; give presentations; manage budgets; amongst many others.

In addition, we need to keep up with new technologies in order to stay ahead of the game. The lingo of the last decade won’t in itself set you apart from the rest. Knowing what a mash-up is and what it has to do with you just might. Knowing what other healthcare providers around the world are doing, would be invaluable to those in healthcare. Knowing how other countries are dealing with the supply of IT in the public sector might prove useful.

So perhaps we should seek out recognition of not just our testing skills, but our soft skills, our db skills, our IT knowledge. Recognition that our vocabulary also includes words like Agile; SOA; SCRUM; computing in the cloud; QC; accessibility; facebook;

iphone; no one’s listening; the developers are over there and I’m over here; the requirements keep changing; timescales are way too short; the customer sets my test window; none of those answers match my thinking.

Big society thinking In other words, to be a tester in this decade and beyond requires that ‘Big Society’ thinking. But how new is this Big Society idea that we should be aware of the world around us and not hide in our boxes? Albert Einstein said “Try not to become a man of success but rather a man of value.” He said this rather a long time ago, and of course by man he clearly meant tester – he was way ahead of his day after all.

To attempt an answer to this, in good old-fashioned speak, tune in next time, where we will revisit those good old days, to see what they (and Einstein) can do to make us equals amongst equals.

Angelina Samaroo Managing director Pinta Education www.pintaed.com

www.testmagazineonline.com

T.E.S.T company profile | 41

Parasoft Improving productivity by delivering quality as a continuous process For over 20 years Parasoft has been studying how to efficiently create quality computer code. Our solutions leverage this research to deliver automated quality assurance as a continuous process throughout the SDLC. This promotes strong code foundations, solid functional components, and robust business processes. Whether you are delivering Service-Orientated Architectures (SOA), evolving legacy systems, or improving quality processes – draw on our expertise and award winning products to increase productivity and the quality of your business applications.

Parasoft's full-lifecycle quality platform ensures secure, reliable, compliant business processes. It was built from the ground up to prevent errors involving the integrated components – as well as reduce the complexity of testing in today's distributed, heterogeneous environments.

Load/performance testing: Verify application performance and functionality under heavy load. Existing end-to-end functional tests are leveraged for load testing, removing the barrier to comprehensive and continuous performance monitoring.

Specialised platform support: Access and execute tests against a variety of platforms (AmberPoint, HP, IBM, Microsoft, Oracle/BEA, Progress Sonic, Software AG/webMethods, TIBCO).

Security testing: Prevent security vulnerabilities through penetration testing and execution of complex authentication, encryption, and access control test scenarios.

What we do

Trace code execution:

Parasoft's SOA solution allows you to discover and augment expectations around design/development policy and test case creation. These defined policies are automatically enforced, allowing your development team to prevent errors instead of finding and fixing them later in the cycle. This significantly increases team productivity and consistency.

Provide seamless integration between SOA layers by identifying, isolating, and replaying actions in a multi-layered system.

End-to-end testing: Continuously validate all critical aspects of complex transactions which may extend through web interfaces, backend services, ESBs, databases, and everything in between.

Advanced web app testing: Guide the team in developing robust, noiseless regression tests for rich and highly-dynamic browserbased applications.

Application behavior virtualisation: Automatically emulate the behavior of services, then deploys them across multiple environments – streamlining collaborative development and testing activities. Services can be emulated from functional tests or actual runtime environment data.

Continuous regression testing: Validate that business processes continuously meet expectations across multiple layers of heterogeneous systems. This reduces the risk of change and enables rapid and agile responses to business demands.

Multi-layer verification: Ensure that all aspects of the application meet uniform expectations around security, reliability, performance, and maintainability.

Policy enforcement: Provide governance and policy-validation for composite applications in BPM, SOA, and cloud environments to ensure interoperability and consistency across all SOA layers.

Please contact us to arrange either a one to one briefing session or a free evaluation.

Web: www.parasoft.com Email: sales@parasoft-uk.com Tel: +44 (0) 208 263 6005

www.testmagazineonline.com

September 2010 | T.E.S.T

42 | T.E.S.T company profile

Learntesting There are many reasons for individuals and businesses to increase knowledge, skills and gain industry-recognised certification. However, with growing financial and time constraints, more flexible solutions are needed to realise the benefits.

Certification

Learntesting delivers a flexible, innovative online learning service designed to put you in control of all your software testing learning and certification needs. It delivers this unique service via a global network of expert training providers, a range of high-quality testing courses and content, powered by learning technology used by some of the largest businesses in the world.

- Advanced Test Manager

• ISTQB Certified Tester Foundation Level (CTFL); • ISTQB Certified Tester Advanced Level (CTAL); - Advanced Test Analyst - Advanced Technical Test Analyst • ISEB Intermediate Certificate in Software Testing.

General Testing • Agile Testing

The Learntesting ‘Virtual Learning Environment (VLE)’ guides and supports your testing education on an ongoing basis with:

• Test Techniques

• High Quality online learning with a variety of content to suit different learning styles and budgets, including fully accredited ISTQB and ISEB courses;

As an independent recognition of achievement, after only six months of operation Learntesting was selected as a finalist for the prestigious ‘Learning Technologies Solution of the Year’ award from the Institute of IT Training in February 2010.

• ‘Live Virtual Classrooms’ led by experienced tutors for exercise revision sessions and exam preparation; • A private library of 50 testing and testing related ebooks 24x7; • Global support with 24x7 access to accredited tutors worldwide;

• A Private Library of Testing Books, Templates & Information

Visit www.learntesting.com for your nearest Learntesting provider and self-register for free. For more information, please contact info@learntesting.com

• Online exam style question papers and answers with detailed explanations; • Self-registration system with access to a range of free valuable content in the Learntesting ‘Testers Treasure Chests’; • Exams available globally.

At Learntesting, we recognize that everyone is different and so provide something to suit everyone according to: • Personal goals; • Existing knowledge and experience; • Budget; • Study availability; • Preferred Learning Styles.

We can provide this because we have invested in an ‘industrial strength’ solution – scalable from the individual to the largest corporations in the world.

Learntesting provides support for the complementary ISTQB and ISEB software testing certification schemes and also other aspects of software testing:

www.learntesting.com info@learntesting.com Learntesting Ltd, 5th Floor 117-119 , Houndsditch, London EC3A 7BT.

T.E.S.T | September 2010

www.testmagazineonline.com

T.E.S.T company profile | 43

iTrinegy Network emulation & application testing tools iTrinegy is Europe’s leading producer of network emulator technology which enables testers and QA specialists to conduct realistic pre-deployment testing in order to confirm that an application is going to behave satisfactorily when placed in the final production network.

Delivering more realistic testing As more and more applications are being delivered over wide area networks (WANs), wireless LANs (WLAN), GPRS, 3G, satellite networks etc, there is a growing need to test application performance in these environments. iTrinegy Network Emulators enable you to faithfully recreate these types of network environments for testing applications, including VoIP, in the test lab or even at your desktop.

Ease of use Our network emulators do not require you to be a network expert in order to use them. They let you quickly and easily create and control network characteristics such as bandwidth, latency, jitter and packet error or loss in order to test software in the conditions that will be encountered in a live production environment or in a proposed new network environment.

Email: info@itrinegy.com Tel: +44 (0)1799 543 345

Any test scenarios you create can be saved and quickly retrieved for subsequent reuse. We also provide you with an extensive range of predefined network scenarios to get you started.

Our products also work seamlessly with load generation and performance tools to further enhance software testing.

A comprehensive range to suit your needs iTrinegy’s comprehensive range of network emulators is designed to suit your needs and budget. It includes: • Software for installation on your own desktop or laptop (trial copies available); • Small, portable inline emulators that seat silently on the desktop and can be shared amongst the test team; • Larger portable units capable of easily recreating complex multi-path, multi-site, multi-user networks for full enterprise testing; • High performance rack-mount units designed to be installed in dedicated test labs; • Very high performance units capable of replicating highspeed, high-volume networks making them ideal for testing applications in converged environments If you would like more information on how our technology can help you ensure the software you are testing is “WAN-ready” and going to work in the field, please: Visit us at Test Expo Winter – 7th December 2010 to discuss our testing partner programme.

Web: www.itrinegy.com

T.E.S.T Focus Groups The T.E.S.T Focus Groups is a complimentary event specially designed and targeted at senior software testers, testing managers, QA & project managers, who wish to discuss and debate some of their most pressing challenges in a well thought out yet informal setting.

T.E.S.T Magazine, the T.E.S.T Focus Groups sister product, spends a lot of time spends a lot of time speaking and listening to its customers and then seeking out innovative ways to meet their needs. It has become apparent that senior decision makers wish to discuss their current challenges in a meaningful and structured manner with a view to finding pragmatic and workable solutions to what are invariably complex issues. Suppliers, who are naturally keen to meet these professionals want to gain a clearer understanding of these challenges and identify how, through meaningful dialogue, they can assist.

This logic coupled with T.E.S.T Magazine’s consistent desire to drive the market forward lead us to launch the T.E.S.T Focus Groups for 2011!

Due to the demands put on modern managers and the subsequent limited opportunities available to join together and voice opinions – the challenges consistently faced by today’s army of testers and testing management tend not to get resolved as quickly as enterprise would like. As a market-leading publisher and events business the organiser understands there should be a format that empowers meaningful debates to assist managers & directors overcome their issues. The T.E.S.T Focus Groups therefore provides ten specially designed syndicate rooms, each containing a specialist subject for delegates to discuss and debate the matter in hand with a view to finding pragmatic and workable solutions.

With some of the industry’s leading minds on hand to help facilitate and steer each session the T.E.S.T Focus Groups will quickly become a ‘must-attend’ event for anyone serious about software testing & QA. Add to this there are plenty of networking opportunities available in addition to a small exhibition, and each

F O C U S

G R O U P S

delegate is provided a fabulous opportunity to interact with their peers, source the latest products and services, and develop meaningful relationships in an informal yet professional setting.

Subjects to be debated are: People or Technology – Who Gets the Cash?

The Value of Testing Requirements Does The User Matter?

Agile Testing Crowd Testing Outsourcing Qualifications, Accreditation, & Exams Event Sponsors Subject Identifying Tester Related Risks Tester Training If you are interested in being a delegate at the T.E.S.T Focus Groups please visit: www.testfocusgroups.com/delegates.html or to register visit: www.testfocusgroups.com/register.html If you are interested in sponsoring this event and hosting a session please visit: www.testfocusgroups.com/sponsor.html Or to discuss any aspect of the event please contact Grant Farrell on +44 (0) 203 056 4598 or email: grant.farrell@31media.co.uk

Web: www.parasoft.com Email: sales@parasoft-uk.com Tel: +44 (0) 208 263 6005 www.testfocusgroups.com +44 (0) 870 863 6930 grant.farrell@31media.co.uk

www.testmagazineonline.com

September 2010 | T.E.S.T

44 | T.E.S.T company profile

TechExcel TechExcel is the leader in unified Application Lifecycle Management as well as Support and Service solutions that bridge the divide between product development and service/support.

This unification enables enterprises to focus on the strategic goals of product design, project planning, development and testing, while enabling transparent visibility with all customer-facing initiatives.

TechExcel has over 1,500 customers in 45 countries and maintains offices in UK, US, China and Japan.

Application Lifecycle Management DevSuite is built around the best-practices insight that knowledge is central to any product development initiative. By eliminating the silos of knowledge that exist between different teams and in different locales, DevSuite helps enterprises transform their development processes, increasing efficiency and overall quality.

DevSpec DevSpec is an integrated requirements management solution that is specifically designed to provide visibility, traceability and validation of your product or project requirements. DevSpec provides a framework to create new requirements, specifications and features that can be linked to development and testing implementation projects.

DevPlan DevPlan is a project, resource, and task management tool. It allows users to plan high level areas of work, assign team members to work in these areas, and then track the tasks needed to complete the activities.

DevTrack DevTrack is the leading project issue and defect tracking tool that is used by development teams of all sizes around the globe. Its configurable workflows allow DevTrack to meet the needs of any organisation's development processes.

DevTest From test case creation, planning and execution through defect submission and resolution, DevTest tracks and manages the complete quality lifecycle.

DevTest combines the test management features of DevTest, DevTrack and TestLink for test automation into one integrated solution.

KnowledgeWise KnowledgeWise is the knowledge management solution at the core of the entire suite.

It is the centralised knowledge base for all company documents including: contracts, processes, planning information and other important records as well as customer-facing articles, FAQs, technical manuals and installation guides.

More information at: www.techexcel.com/products/devsuite.

Service and Support Management Service and Support Management solutions provide enterprises with total visibility and actionable intelligence for all service desk, asset management and CRM business processes.

ServiceWise ServiceWise is a customisable and comprehensive internal Helpdesk, ITSM- and ITILcompliant solution. Automate and streamline services and helpdesk activities with configurable workflows, process management, email notifications and a searchable knowledge base. The self-service portal includes online incident submission, status updates, online conversations and a knowledgebase. ServiceWise includes modules such as incident management, problem escalation and analysis, change management and asset management.

CustomerWise CustomerWise is an integrated CRM solution focused on customer service throughout the entire customer lifecycle. CustomerWise allows you to refine sales, customer service and support processes to increase cross-team communication and efficiency while reducing your overall costs. Combine sophisticated process automation, knowledgebase management, workflow, and customer self-service to improve business processes that translate into better customer relationships.

AssetWise AssetWise aids the process of monitoring, controlling and accounting for assets throughout their lifecycle. A single and centralised location enables businesses to monitor all assets including company IT assets, managing asset inventories, and tracking customerowned assets.

FormWise FormWise is a web-based form management solution for ServiceWise and CustomerWise.

Create fully customised online forms and integrate them directly with your workflow processes. Forms can even be routed automatically to the appropriate individuals for completion, approval, and processing, improving your team's efficiency. Web-based forms may be integrated into existing websites to improve customer interactions including customer profiling, surveys, product registration, feedback, and more.

DownloadPlus DownloadPlus is an easy-to-use website management application for monitoring file downloads and analysing website download activities.

DownloadPlus does not require any programming or HTML. DownloadPlus provides controlled download management for all downloadable files, from software products and documentation, to marketing materials and multimedia files.

More information at: www.techexcel.com/products/itsm/

Training Further your investment with TechExcel, effective training is essential to getting the most from an organisation's investment in products and people. We deliver professional instructor-led training courses on every aspect of implementation and use of all TechExcel’s software solutions as well as both service management and industry training. We are also a Service Desk institute accredited training partner and deliver their certification courses.

More information at: www.techexcel.com/support/ techexceluniversity/servicetraining.html

For more information, visit www.techexcel.com or call 0207 470 5650.

T.E.S.T | September 2010

www.testmagazineonline.com

T.E.S.T company profile | 45

Hays Experts in the delivery of testing resource Setting the UK standard in testing recruitment We believe that our clients should deal with industry experts when engaging with a supplier. Our testing practice provides a direct route straight to the heart of the testing community. By engaging with our specialists, clients gain instant access to a network of testing professionals who rely on us to keep them informed of the best and most exciting new roles as they come available.

Our testing expertise

Tailored technical solutions

We provide testing experts across the following disciplines:

Automated Software Testing: including Test Tool selection, evaluation & implementation, creation of automated test frameworks;

With over 5,000 contractors on assignment and thousands of candidates placed into very specialised permanent roles every year, we have fast become the pre-eminent technology expert. Our track record extends to all areas of IT and technical recruitment, from small-scale contingency through to large-scale campaign and recruitment management solutions.

Performance Testing: including Stress Testing, Load Testing, Soak Testing and Scalability Testing;

Unique database of high calibre jobseekers

Functional Testing: including System Testing, Integration Testing, Regression Testing and User Acceptance Testing;

Operational Acceptance Testing: including disaster recovery and failover; Web Testing: including cross browser compatibility and usability; Migration Testing: including data conversion and application migration; Agile Testing; Test Environments Management.

The testing talent we provide • Test analysts; • Test leads; • Test programme managers; • Automated test specialists; • Test environment managers; • Heads of testing; • Performance testers; • Operational acceptance testers.

Our expert knowledge of the testing market means you recruit the best possible professionals for your business.

When a more flexible approach is required, we have developed a range of creative fixed price solutions that will ensure you receive a testing service tailored to your individual requirements.

As we believe our clients should deal with true industry experts, we also deliver recruitment and workforce related solutions through the following niche practices: • Digital; • Defence; • Development; • ERP; • Finance Technology; • Infrastructure; • Leadership; • Public, voluntary and not-for-profit; • Projects, change and interim management; • Security; • Technology Sales; • Telecoms.

We build networks and maintain relationships with candidates across these areas, giving our clients access to high calibre jobseekers with specific skills sets.

To speak to a specialist testing consultant, please contact: Sarah Martin, senior consultant, Tel: +44 (0)1273 739272 Email: testing@hays.com Web: hays. co.uk/it

Our specialist network Working across a nationwide network of offices, we offer employers and jobseekers a highly specialised recruitment service. Whether you are looking for a permanent or contract position across a diverse range of skill sets, business sectors and levels of seniority, we can help you.

Web: hays.co.uk/it Email: testing@hays.com Tel: +44 (0)1273 739272

www.testmagazineonline.com

September 2010 | T.E.S.T

46 | T.E.S.T company profile

Spirent Communications Spirent Communications plc is a global leader in test and measurement products and services. The world’s leading organisations rely on Spirent’s solutions and expertise to evaluate the performance of their applications, products and network services – because no other company can offer stringent application testing in a realistic network context, embracing any combination of Local Area, Wide Area, fixed line and wireless access scenarios.

application needs to serve any combination of multiple users across a range of access technologies – including broadband business connection, home ADSL, GSM, WiFi – each with their own protocols, speeds and typical user behaviour profiles.

From wireline to wireless to satellite, Spirent offers a complete portfolio of solutions to enhance user and customer quality of experience (QoE). They enable today’s communication ecosystem as well as tomorrow’s emerging enterprises to deploy lifeenriching communication networks, devices, services and applications.

With Spirent these ‘real life’ test scenarios are not only possible, but made easy by a simple graphical user interface, enabling rapid creation of complex and realistic everyday usage models as well as extreme crisis test conditions – without any specialist scripting skills.

Sectors that rely on Spirent testing include: financial services, cloud service providers, major equipment vendors, operators, government organisations and the military.

Broad industry expertise Spirent is recognised worldwide for its expertise in next-generation communication networks, devices and applications, and its engineers are chosen to advise many of the leading communication standards organisations. It provides the equipment, training and test methodology to help many of the leading technology labs and forums evaluate the performance of emerging technologies.

Spirent pioneered the testing of Ethernet networks, IP Telephony and VoIP, VPNs, TriplePlay, 2G and 3G wireless, and Location Based Services. Today, it is helping to test the first deployments of high-speed Ethernet, next-generation Internet , 4G wireless and IPv6 networks in Asia, Europe and North America.

Areas of expertise include: • Enterprise Networks: load testing, system performance, network security; • Convergence: VoIP, IP VPNs, IPTV; • Broadband Networking: DSL, Gigabit Ethernet, and IP; • Next-Generation Internet: IPv6; • Wireless: CDMA, UMTS, Location Based Services and LTE; • Satellite Navigation: GPS, GLONASS, Galileo.

The leader in web application testing Spirent’s expertise is utterly critical to the performance and QoE offered by cloud-based applications. Other solutions exist, offering sophisticated application testing, but only Spirent solutions take into account the complexities of the web environment and its potential impact on the service.

Unless your test scenario can emulate such a complex mix of protocols and user behaviours, you cannot be certain how the application will perform across the web.

Spirent solutions for web application testing Two factors are key to realistic web application testing – user emulation and network empairment emulation.

Spirent Network Impairment Emulators bring the realism of live network conditions into the lab. They enable testers to accurately create the whole range of actual and possible delays and impairments that occur over live networks.

Spirent Ethernet Network Emulators are essential for validating and evaluating new products and technologies.

Examples include: testing multimedia products and services; storage networks; circuit emulation services; Timing over Packet; disaster recovery; data centre moves; mobile/wireless infrastructure; WAN and TCP acceleration; and multi-play applications.

Spirent Avalanche appliances generate the most realistic possible emulation of large user populations for application, web services and content delivery testing.

Any combination of users, access technologies, and applications can be easily programmed via a point and click graphical interface – specifying such variables as connections, transactions and sessions per second – then multiplied to emulate massive traffic conditions. Real time performance statistics are presented and summarised into customised test reports.

Spirent Avalanche Virtual is the Industry’s only ‘all-in-one’ cloud testing solution designed to test and measure the performance, availability, security and scalability (PASS) of virtualised network infrastructures and of applications deployed in the cloud. A software solution that resides in virtualised data centre environments, Spirent Avalanche Virtual lets enterprises and cloud service providers measure the impact of large-scale cloud-based application deployments against their expected quality of experience.

Realistic testing based on flexible impairment and usage programming is the only way to ensure a fully developed solution and an increase in quality of experience for the user.

Whether it is a pizza restaurant remote ordering system, an Inland Revenue tax return, or the BBC’s iPlayer, the

Spirent Communications plc Tel: +44(0)7834752083 Email: Daryl.Cornelius@spirent.com Web: www.spirent.com

T.E.S.T | September 2010

www.testmagazineonline.com

Can you predic T.E.S.T company profile | 47

Facilita Load testing solutions that deliver results Facilita has created the Forecast™ product suite which is used across multiple business sectors to performance test applications, websites and IT infrastructures of all sizes and complexity. With this class-leading testing software and unbeatable support and services Facilita will help you ensure that your IT systems are reliable, scalable and tuned for optimal performance.

Forecast, the thinking tester's power tool A sound investment: A good load testing tool is one of the most important IT investments that an organisation can make. The risks and costs associated with inadequate testing are enormous. Load testing is challenging and without good tools and support will consume expensive resources and waste a great deal of effort.

Forecast has been created to meet the challenges of load testing, now and in the future. The core of the product is tried and trusted and incorporates more than a decade of experience but is designed to evolve in step with advancing technology.

Realistic load testing: Forecast tests the reliability, performance and scalability of IT systems by realistically simulating from one to many thousands of users executing a mix of business processes using individually configurable data.

Comprehensive technology support: Forecast provides one of the widest ranges of protocol support of any load testing tool.

1. Forecast Web thoroughly tests web-based applications and web services, identifies system bottlenecks, improves application quality and optimises network and server infrastructures. Forecast Web supports a comprehensive and growing list of protocols, standards and data formats including HTTP/HTTPS, SOAP, XML, JSON and Ajax.

2. Forecast Java is a powerful and technically advanced solution for load testing Java applications. It targets any non-GUI client-side Java API with support for all Java remoting technologies including RMI, IIOP, CORBA and Web Services.

3. Forecast Citrix simulates multiple Citrix clients and validates the Citrix environment for scalability and reliability in addition to the performance of the hosted applications. This non-intrusive approach provides very accurate client performance measurements unlike server based solutions.

4. Forecast .NET simulates multiple concurrent users of applications with client-side .NET technology.

5. Forecast WinDriver is a unique solution for performance testing Windows applications that are impossible or uneconomic to test using other methods or where user experience timings are required. WinDriver automates the client user interface and can control from one to many hundreds of concurrent client instances or desktops.

Facilita Tel: +44 (0) 1260 298109

www.testmagazineonline.com

6. Forecast can also target less mainstream technology such as proprietary messaging protocols and systems using the OSI protocol stack.

Powerful yet easy to use: Skilled testers love using Forecast because of the power and flexibility that it provides. Creating working tests is made easy with Forecast's script recording and generation features and the ability to compose complex test scenarios rapidly with a few mouse clicks. The powerful functionality of Forecast ensures that even the most challenging applications can be full tested.

Facilita Software Development Limited. Tel: +44 (0)12

Supports Waterfall and Agile (and everything in between): Forecast has the features demanded by QA teams like automatic test script creation, test data management, real-time monitoring and comprehensive charting and reporting.

Forecast is successfully deployed in Agile ‘Test Driven Development’ (TDD) environments and integrates with automated test (continuous build) infrastructures. The functionality of Forecast is fully programmable and test scripts are written in standard languages (Java, C#, C++ etc). Forecast provides the flexibility of open source alternatives along with comprehensive technical support and the features of a high-end enterprise commercial tool.

Flexible licensing: Geographical freedom allows licenses to be moved within an organisation without additional costs. Temporary high concurrency licenses for ‘spike’ testing are available with a sensible pricing model. Licenses can be rented for short term projects with a ‘stop the clock’ agreement or purchased for perpetual use.

Our philosophy is to provide value and to avoid hidden costs. For example, server monitoring and the analysis of server metrics are not separately chargeable items and a license for Web testing includes all supported Web protocols.

Services In addition to comprehensive support and training, Facilita offers mentoring where an experienced Facilita consultant will work closely with the test team either to ‘jump start’ a project or to cultivate advanced testing techniques. Even with Forecast’s outstanding script automation features, scripting is challenging for some applications. Facilita offers a direct scripting service to help clients overcome this problem.

We can advise on all aspects of performance testing and carry out testing either by providing expert consultants or fully managed testing services.

Email: enquiries@facilita.co.uk Web: www.facilita.com

September 2010 | T.E.S.T

48 | The Last Word

the last word...

Stupid web tricks Dave Whalen is hunting out bad practice on the web.

Beware, may contain sarcasm.

S

o you just finished taking an HTML course at your local college and you’ve hung out your banner announcing your newly acquired skills as a Web Master.

As a token of good will, you volunteer your services to your local club or organisation. Not so fast!

After almost 20 years in web design and development – including teaching it for four years – and web testing, I thought I had seen it all. But I’m still amazed at some of the things I see, from both novice Web site builders and professionals. Here’s a tongue-incheek list of the some of the things that users despise and many web designers and developers love to do, and they are all things that I have seen...

Multiple scroll bars: Vertical scrolling is typically unavoidable and most users will tolerate it. But horizontal scrolling should be avoided. You know what’s really cool? Combine the two so a user has to scroll both ways or, even better, build your site using frames and make the user scroll in every frame.

Multiple scroll bars rock! Here’s a special tip – require scrolling, but turn off the scroll bars!

Bizarre colour combinations: Make the background lime green and use hot pink text. Even better, use a black background with navy blue text, who needs to read it anyway? Why limit yourself to only four or five colours, most monitors support over 16 million. Try to use as many as you can.

If you don’t burn my retinas, you're not really trying.

Sound: I’m sitting at my computer, with my new Jimmy Buffett CD in the player and I go to your site and I’m immediately hit with the electronic version of YMCA you have playing on your web site... And don’t give me a way to turn it off.

Animated GIFS: When I taught web development, I used to tell my students that I never gave Fs - if they T.E.S.T | September 2010

wanted them, they would have to earn them! One sure way to earn one was to put anything on the page that moved. Animated GIFs and marquees were always sure bets. Even better – use lots of them!

Large graphic files: I have nothing better to do than wait for that eleventy-jillion-pixel picture of your cat to load. Use lots of them; then I’ll have time to go get another Landshark Lager... from Las Vegas.

Text, text, and more text: There is nothing I like better than an entire screen of nothing but text. The web is just an electronic book. Just to make sure it all fits on a single screen, use a really small font.

Changing layouts: Make sure every page has a different layout! Keep me on my toes and keep moving stuff around, like buttons and links so I really have to look for them. No two pages should ever look alike.

Background images: If the text on your site isn’t hard enough to read, you can always display it on top of a picture.

Fade-ins/outs: Who would want pages to be displayed immediately when you can slowly fade them in.

And you should chain them together so not only do I have to wait for the next page to be displayed, I also have to wait for the current page to disappear.

HTML leaks: Everyone loves broken HTML. No page is complete unless "font>" is displayed somewhere.

Never view or test your pages!

Last updated: Always include a ‘last updated’ notice on your site, but never change it. Who wants current information?

Counters: Everyone cares how many visitors come to your site, so be sure to waste plenty of space telling them.

Flash: Not only will it require you to buy expensive software to create it, it also requires the user to download a plug-in to see it.

After almost 20 years in web design and development – including teaching it for four years – and web testing, I thought I had seen it all.

Awards: Most don’t mean anything, and nobody really cares. If you really want one, I can design one for you!

Empty drop-down boxes/selection lists: Options, who needs ‘em? Also, be sure to include some type of instructions as one of the options.

Something like: ‘Please select an item‘ is good. Don't forget to allow me to actually select it. Make the boxes change width with each selection and blow out the entire page. Or make the width so small I can't read the options.

Ads: Everyone loves ads and pop-ups.

I always wondered what those popup blockers did. The more browser windows I can have open the better!

Cursor tails: Sometimes I can’t find my cursor on the page. Please help me by attaching some nonsense text or a graphic to my cursor that will follow it all over the screen. Or change the cursor to something else so it’s really hard to lose!

Broken links/graphics: That red X icon is much more appealing than any picture you could possibly have – and it downloads so much faster. Give me a valid link and I may leave your site and that would be bad! Oh, and of course, never, ever use the ALT attribute. Keep me guessing.

Dave Whalen President and senior software entomologist Whalen Technologies http://softwareentomologist.wordpress.com

www.testmagazineonline.com

F O C U S

G R O U P S

Helping you overcome obstacles

21st June 2011 ●

One Day Event ● 80 Decision Makers ● 10 Thought Leading Debate Sessions Peer-to-Peer Networking

●

Exhibition

●

Cutting Edge Content

For more information Contact Grant Farrell on +44 (0) 203 056 4598 Email: registration@testfocusgroups.com Website: www.testfocusgroups.com

Held at: Park Inn Hotel, Heathrow