TEST Magazine - December 2011-January 2012

g s in er ad vid Le ro 20 ing P st Te

iNNoVATioN FoR SoFTWARE QuALiTy Volume 3: issue 6: December 2011

Testing centres of excellence Focussing on collaboration

inside: Static analysis | Data obfuscation | Testing tools Visit TEST online at www.testmagazine.co.uk

BORLAND SOLUTIONS FROM MICRO FOCUS DESIGNED TO DELIVER BETTER SOFTWARE, FASTER

Borland Solutions are designed to:

•

Align development to business needs

•

Strengthen development processes

•

Ensure testing occurs throughout the lifecycle

•

Deliver higher quality software, faster

Borland Solutions from Micro Focus make up a comprehensive quality toolset for embedding quality throughout the software development lifecycle. Software that delivers precisely what is needed, when it is needed, is crucial for business success. Borland Solutions embed quality into software delivery from the very beginning of the development lifecycle whichever methodology you use – traditional, Agile or a mixture – from requirements, through regression, functional, performance and load testing.

The result: you meet both business requirements and quality expectations with better quality software, delivered faster.

Visit Micro Focus at EuroSTAR 2011, stand 45 & 46, and check out what the Borland Solutions have to offer you. Register now at http://www.eurostarconferences.com/ Micro Focus Ltd. Email: microfocus.communications@microfocus.com © 2011 Micro Focus IP Development Limited. All rights and marks acknowledged.

ing ers ad id Le rov 20 ng P sti Te

TEST : INNOV ATION FOR SO FTWARE QUAL ITY

Leader | 1 Feature

INNOVATION FO R SOFTWARE QU ALITY

Volume 3: Issue 6: December 2011

Testing centres of excellence

Focussing on collabora tion

VOLUME 3: ISS UE 6: DECEMB ER 2011

iNNoVATioN FoR SoFTWARE QuALiTy

Inside: Static analysis

| Data obfuscation |

Testing tools

Visit TEST online at www.testmag azine.co.uk

TEST’s top twenty

F

irstly i would like to introduce you to our special new section – which we hope to repeat on an annual basis – 20 Leading Testing Providers. in this special section we have gathered together twenty of the top providers of products, tools and services to the testing industry in a handy alphabetical reference section.

We hope you find it useful.

Since late September when the initial details were publicised, another OPERA experiment was run, further data were processed and the same results were observed, seemingly confirming the initial flabbergasting results that the neutrinos were indeed travelling faster than light - and thus undermining the whole idea of cause and effect by theoretically allowing scientists to send data back in time. The second set of results were publicised just as I was working on the feature.

Matt Bailey, Editor

© 2011 31 Media Limited.

All rights reserved.

TEST Magazine is edited, designed, and published by 31 Media Limited. No part of TEST Magazine may be reproduced, transmitted, stored electronically, distributed, or copied, in whole or part without the prior written consent of the publisher. A reprint service is available.

Opinions expressed in this journal do not necessarily reflect those of the editor or TEST Magazine or its publisher, 31 Media Limited.

ISSN 2040-0160

www.testmagazine.co.uk

Apart from our testing top 20, another thing you can depend upon is change. When I first interviewed Axel Naumann for the CERN feature on page 16 of this issue, details had just been released, literally in the same week, about how the astonishing results of the OPERA experiment carried out by CERN in which 15,000 beams of neutrinos were fired from Geneva to the Gran Sasso Laboratory in Italy seemed to show that the neutrinos had travelled faster than the speed of light.

CERN researchers advocated prudence in the face of the early results, although at the time they believed their observations – in which the neutrinos made the 730 km journey 60 nanoseconds, ie, faster than light – were accurate.

Since late September when the initial details were publicised, another OPERA experiment was run, further data were processed and the same results were observed, seemingly confirming the initial flabbergasting results that the neutrinos were indeed travelling faster than light – and thus undermining the whole idea of cause and effect by theoretically allowing scientists to send data back in time. The second set of results were publicised just as I was working on the feature.

Editor Matthew bailey matthew.bailey@31media.co.uk Tel: +44 (0)203 056 4599 To advertise contact: Caspar Ingham Caspar.ingham@31media.co.uk Tel: +44(0)203 056 4386 Production & Design Toni barrington toni.barrington@31media.co.uk Dean Cook dean.cook@31media.co.uk

In the latest response to the research however, published just yesterday as I write this, a team of scientists led by the Gran Sasso National Laboratory claims to have refuted the OPERA faster-than-light neutrino result.

The latest experiment used much shorter pulses to try and remove a potential systematic error from the set-up. To make sure that they could prove the time correlation between the received neutrinos and the originating event, the latest test used pulses three nanoseconds long, with more than 500 nanoseconds between pulses.

This experiment claims that the neutrinos arrived 57.8 ns (±7.8ns) too early, seemingly corroborating the original OPERA data.

The Gran Sasso group however says that in attaining faster than light velocities the neutrinos should lose energy by producing photons and electron/anti-electron pairs.

This, they assert, has not been observed; their spectrum seems to demonstrate that they didn’t lose that energy, therefore they can’t be passing the speed of light...

If this is the case, then perhaps the guys at CERN need to run the data processing software through a bit more static analysis.

Whatever the facts, it’s clear that life on the cutting edge of theoretical physics is never simple – and by the time you read this, there will probably be a whole new set of results to confound us.

Until next time...

Matt bailey, Editor

Editorial & Advertising Enquiries 31 Media Ltd, Three Tuns House 109 borough High Street London SE1 1NL Tel: +44 (0) 870 863 6930 Fax: +44 (0) 870 085 8837 Email: info@31media.co.uk Web: www.testmagazine.co.uk Printed by Pensord, Tram Road, Pontllanfraith, blackwood. NP12 2YA

December 2011 | TEST

TM

Powerful multi-protocol testing software

Can you predict the future?

Don’t leave anything to chance.

Forecast tests the performance, reliability and scalability of your business critical IT systems backed up by Facilita’s specialist professional services, training and expert support.

Facilita Software Development Limited. Tel: +44 (0)1260 298 109 | email: enquiries@facilita.com | www.facilita.com

Contents | 3

Contents... december 2011 1 Leader column

Changing faster than the speed of light.

4 News 6 Cover story – Testing centres of excellence: Focussing on collaboration “More than machinery, we need humanity.” What does this quote have to do with testing and well performing testing centres of excellence? According to Archie Roboostoff, it forms the core of some of the best TCE implementations around the world.

12 Super-testers?

People with Aspergers Syndrome and autism often have a number of attributes that can make them highly suited to software testing tasks. Peter Macdonald gets some help from his colleagues to outline what skills an ‘aspie’ can bring to bear on software testing.

16 Testing at the boundaries of science

6

With CERN once again in the news and the possibility of particles travelling faster than light making the scientific community reassess its very foundations, Matt Bailey speaks to Axel Naumann, part of the team of physicists that developed the crucial numbercrunching software at the heart of the organisation, about how static analysis is improving its software quality.

20 Twelve questions to prevent a testing data breach

Howard Wiener says that answering the 12 questions he poses in this feature can help your organisation examine the challenges it faces when handling sensitive data during

16

software testing.

22 Christmas wishes It’s Christmas time... And top of Angelina Samaroo’s wish list? That Christmas comes early for every project!

24 Design for TEST – Living labs

This issue Mike Holcombe is in the living lab – a place where users can become designers.

22

26 Test case paths: Happy, sad and bad Senior testing consultant Nishant Verma pits the QTP testing tool against his own Agile tool, Twist and finds that QTP could do better.

29 TEST’s 20 leading testing product and service providers 41 TEST Directory 48 Last Word – Dave Whalen

It’s Christmas and Dave Whalen is at the buffet...

www.testmagazine.co.uk

26 December 2011 | TEST

Testers misunderstood by management

A

survey targeting engaged UK test and QA professionals reveals strong belief in increasing popularity of automated testing and emerging technologies, despite 69 percent claiming their management won’t fully understand or support software testing initiatives A survey asking 78 London-based influential software testers and QA professionals for their views on the key drivers and sources of innovation and skills that will influence their industry over the next three years reveals that 83 percent back the continued popularity of automated testing over manual testing. However, respondents were considerably less optimistic about relationships between management and IT. When asked what won’t change over the next three years, more than two-thirds of testing professionals (69 percent) believed that software testing will still not be fully

understood or supported by managers in that time-frame. In addition, 45 percent were resigned to the fact budgets for software testing and QA would not increase over the same period.

When asked what would define testing innovation over the next three years, almost two thirds of survey respondents (64 percent) pointed at demand for mobile applications.

The second most popular response to the question was increased adoption of Agile testing procedures and principles (39 percent), while testing in virtualised environments was also identified as a significant driver of innovation for more than a third of participants (35 percent).

Despite 40% of respondents agreeing that testing in the Cloud to reduce costs and improve agility would become ubiquitous, it appears that few organisations are actually

ready to fully commit to the technology. Just 17 percent of test professionals said that they expected more than half of applications in their business to be migrated to the cloud within three years, while 47 percent were adamant that this would not happen.

Testers were also asked what technical skill sets they believed would be in greatest demand for software testers and QA professionals over the next three years.

In accordance with other responses, automation was voted the most likely to be in demand, followed by Agile, mobile and then security testing. When asked what non-technical skill sets might be in greatest demand, communications skills were regarded as the most sought-after, followed by business experience and collaborative working, with austerity project planning thought of as a less important quality.

CERN collaboration highlights London Testlab launched the importance of testing

A

W

hen the media announced back in 2008 that the centre for nuclear research CERN was conducting a series of ground-breaking experiments involving particle accelerator technology, the public accordingly went into a paranoid frenzy. While all doomsday theories have since been discredited by scientists worldwide, it recently emerged that CERN has in fact employed the services of Coverity, a software company that tests source codes for potentially critical software flaws, on the Large Hadron Collider project. The full story and an interview with Axel Naumann, chief physicist on CERN’s ROOT Development Team appears on page 16 of this issue.

So far Coverity's Static Analysis testing tool has allegedly found over 40,000 defects in CERN’s software, which could have severely compromised the project had they not been detected. This case aptly

TEST | December 2011

showcases the importance of rigorous testing in a scientific experiment of the magnitude of the LHC project.

Director of software testing company BugFinders, Martin Mudge commented on the safety precautions involved in software testing: “Software testing is vital when the project involves safety critical systems. Sure, it’s bad when you come across a glitch in one of your iPhone apps, but if a computerised plane system gives the wrong information, it could end with serious loss of life. I wouldn’t dream what the consequences would be if a critical bug went unnoticed on a project involving nuclear energy.”

The fervent media controversy surrounding the Large Hadron Collider may have been overhyped, but there have been however past cases where a lack of testing has proven to be costly, sometimes even fatal.

leading European provider of carrierneutral colocation data centre services, has announced the deployment of its second Cloud Testlab, a turnkey cloud hosting test environment enabling service providers and system integrators to test and develop cloud services at high speed and with best-in-class performance guarantees.

The Interxion Testlab deployment is a collaboration between Interxion, Flexiant, Boston Limited and GTT and acts as an online proof-of-concept (PoC) environment for cloud service providers. It can be used for testing scenarios before deploying a cloud infrastructure with the proof-of-concept being run on a reference architecture. This Testlab offers a framework that supports multiple hypervisors and allows service providers to build both private and public clouds.

The new test environment is located in Interxion’s City of London data centre and will deploy Flexiant’s Extility platform solution, have network connectivity via GTT and be housed on Supermicro hardware provisioned

by Boston Limited.

“The rapid adoption of cloud computing has seen cloud-based services become a central part of business strategy,” comments Greg McCulloch, UK managing director, Interxion. “The launch of this UK Cloud Testlab deployment further reflects our commitment to the cloud and offers both customers and prospects the opportunity to test and develop cloud services at high speed and with best-inclass performance guarantees.”

www.testmagazine.co.uk

g

orilla Logic, has released FoneMonkey for Android, its latest open source automated testing tool for mobile applications.

The new Android testing capabilities complement the popular FoneMonkey for iOS (iPhone and iPad) automated testing tool and extends Gorilla Logic’s comprehensive open source automated testing solution set by bringing Android developers and QA testers the same recording/playback and test script generation already enjoyed by iOS developers. It benefits developers by providing rapid creation of functional testing scripts; allows testers without in-depth Android Software Developer Kit (SDK) knowledge to easily and quickly create testing scripts; and, offers increased application quality and faster time-to-market.

“FoneMonkey for Android addresses a major gap in the Android development

tool chain,” said Stu Stern, president and CEO of Gorilla Logic and co-creator of FoneMonkey. “The FoneMonkey family of tools is the only record/playback functional testing tool for iOS and Android. With the addition of FoneMonkey for Android, Gorilla Logic has established its position as the pre-eminent provider of next

Seven software companies form a new supplier to the mid–market

S

even UK business software companies have unified under a new senior management team and a single brand to form a supplier that they says is dedicated to meeting the requirements of mid-sized organisations.

Calyx Software, Gyrosoft, Trinity Computer Services, and divisions of MentecPlus, Touchstone Group, Sys-Care, and Avant-Garde have been integrated through acquisition into a single entity called m-hance. The announcement follows a series of recent acquisitions that has seen m-hance grow to 150 staff serving almost 1,000 direct customers across the UK and Ireland and a further 1000 through channel partners. The company operates from offices in Manchester, London, Loughborough, Dublin and Glasgow.

Mark Thompson, CEO of m-hance, comments, “The creation of m-hance allows us to go to our customers with a single, clear identity and message. The company benefits from the stability, long trading history and deep client relationships of each of the component entities. We’ve combined the strengths of seven respected specialists into one financially strong leader with an experienced management team backed-up by a solid balance sheet containing no external debt”.

The new structure allows the company to further develop its product set along vertical market specific lines. For example, in the distribution and manufacturing sectors it had been providing a best of breed industry solution including finance, CRM and other business critical functionality to more than 750 organisations for over 25 years and can now expand this model further into not-forprofit, education, financial services and hospitality and leisure, where it has a strong customer footprint.

www.testmagazine.co.uk

generation functional testing tools for the mobile market.”

According to Gorilla, FoneMonkey is the only test tool for iOS and now Android that records all actions with the iPhone, iPad and Android phone while in use and plays them back as a test script at any time. It enables the interactive creation, editing and playback of automation scripts that exercise an application's user interface. Using FoneMonkey, developers and quality assurance team members can create suites of tests that automate and perform user operation sequences, and then verify results. FoneMonkey supports development as well as QA testing, and tests can easily be incorporated into continuous integration environments.

FoneMonkey for Android is available now. because it is an open source tool, it is free to download at www.gorillalogic.com/fonemonkey.

NEWS

FREE AuToMATED TESTiNg TooL FoR ANDRoiD NoW AVAiLAbLE

Working together for safer tomorrow

F

uzzing and security testing are to become part of education of iT professionals in germany according to Finnish security testing specialist Codenomicon. The company and brandenburg university of Applied Sciences have announced a cooperation agreement to help educate german iT professionals. Codenomicon's Defensics software will be available for students in Security management programme in FH brandenburg.

“Information technology is becoming more and more sophisticated, complicated and as a result, more vulnerable,” says Anton von Troyer, vice president DACH from Codenomicon Deutchland. “We want to help training the future information security professionals to meet the challenge they are up against.”

Codenomicon's Defensics test tool relies on fuzz testing, a technique where invalid or unexpected input is fed to the system under test to reveal potentially exploitable vulnerabilities. The technique is particularly effective in finding previously unknown, so called zero-day vulnerabilities. Unknown vulnerabilities are a challenge to product security, since there are no patches available, no way to protect the system against attacks targeted to unknown vulnerabilities.

“It is a fertile cooperation for both sides", says Prof. Sachar Paulus from FH brandenburg. “beside the use of the software in teaching, possible innovations in the context of product use are examined together.”

TestResponsibly.com launched

A

new website focuses on demonstrating real-world embedded software testing activities has been launched

by Vector Software. The company, a provider of dynamic software test tools for embedded systems, says the new website aids development organisations in evaluating test tools for embedded software applications by demonstrating how to test complex code samples. www.prweb.com//releases/2011/10/prweb8888216.htm

December 2011 | TEST

6 | Cover story

TEST | December 2011

www.testmagazine.co.uk

Cover story | 7

Testing centres of excellence – focussing on collaboration What does a quote from a 1940s movie starring Charlie Chaplin have to do with testing and well performing testing centres of excellence (TCE)? More than one would think according to Archie Roboostoff. In fact, some would say it forms the core of some of the best TCE implementations around the world.

“More than machinery, we need humanity.”

Charlie Chaplin

A

testing centre of excellence (TCE) provides a framework and methodology for bringing specialists and toolsets together that better leverage knowledge, technology and resources. A basic TCE structure brings together test managers, architects, automation engineers, subject matter experts, quality assurance and testers. While TCEs vary across verticals and companies, the basic premise is the same, create an organisation that delivers the highest quality software in the most efficient manner.

Given that mainstream, commercial test automation tools have been around since the late 1980s, organisations have had plenty of time to fine tune and adjust a wide variety of TCEs. From outsourced offshore to localised TCEs, there have been many derivatives that have both failed and succeeded. Every TCE methodology puts a strong emphasis on testing (ie, Feature-Driven Development, Test-

www.testmagazine.co.uk

Driven Development, Behaviour Driven Development, Insert you methodology here-Driven Development, etc) yet, most development organisations still cannot seem to get it right.

Product deliveries across the globe are still delayed and those that aren’t sometimes have quality issues. When a quality issue is discovered after a product has gone to market, the cost to recover is exponential. Having to pull development resources off future products to fix existing quality issues can severely impact not only the ongoing cost of the development team but will adversely affect the revenue generating capabilities of that organisation.

Now, back to that Charlie Chaplin quote and what it means to productive TCEs. The bottom line, it’s more important to focus on collaboration than it is to focus on the technology within a TCE. Collaboration leads to two benefits within a TCE, predictability and productivity. The impetus of TCEs was to foster collaboration yet as market and product cycles shrink, silos within TCEs are starting to emerge. It

is human nature after all, that when under stress, most people shut down to try to recover.

In the past, this lack of collaboration remained hidden given the long delivery cycles of most software applications. Those TCEs that do not collaborate well will not have the luxury of hidden recovery given that the voracious technical appetite for social media driven applications, mobile devices, on demand content, wireless services and connected appliances is rapidly shrinking product lifecycles.

Now, more than ever, TCEs are being pushed to the limits to get products out to market that work, perform, and function.

Building a collaborative TCE step 1: Requirements Evolving a TCE into a collaborative ‘well-oiled machine’ that brings instant productivity to the people within the TCE and constant predictability to the management layers above the TCE is actually easier to accomplish than most organisations make it out to be. The tip of the spear, so to

December 2011 | TEST

8 | Cover story

By bringing the TCE into the loop during the requirements phase, and linking that information with testing assets, any change to the requirement and any affected policy changes are instantly communicated across the entire SDLC and TCE. This aspect of technology integration is a prime example of focusing on collaboration rather than pure technology that will ultimately drive efficient testing practices and an overall better product.

TEST | December 2011

speak, is getting everyone on the same page with requirements. It is notoriously difficult for business users to communicate needs to development teams. As a matter of fact, up to 70 percent of all production defects stem from this stage when requirements are poorly defined.

Requirements come from numerous stakeholders that may be globally distributed and in a variety of formats.

This is further complicated by the fact that users often don’t know what they want until they actually see it and the fact that the only constant with requirements is change. This leads to wasteful cycles where development teams try to implement what they understood and the TCE is trying to verify quality on misaligned goals.

This results in missing capabilities, unnecessary features, and – down the road – a lot of rework. And that rework quickly adds up, especially when issues are discovered late in the development process.

Visual business scenarios are an effective means of communicating business process flows and concepts but are usually only propagated to the various business and development stakeholders. by sharing process visualisations coupled with simulations with all functional groups within the TCE, this helps reduce ambiguity and presents an instantaneous way to recognise sequencing issues, broken decision points, missing process steps, and opportunities for innovation. The overall benefit to the TCE is that the TCE now has the ability to create test cases while products are still in the planning phase.

Effort put into defining the paths that users can take within an application is a wise investment because this information is used to create test cases for each path. With the ability to calculate the magnitude of requirements risk and complexity, QA staff can identify high risk areas in an application and in doing so help set testing priorities. This in turn ensures that effort is effectively spent and not wasted on over- or underengineered functionality. Generating tests cases directly from requirements

also means that QA staff becomes active participants in the development process from inception and not as an afterthought. by doing this, an environment where test planning and effort estimation can become much more realistic. And thus, they become more predictable and productive because they have been validated early in the process. by bringing the TCE into the loop during the requirements phase, and linking that information with testing assets, any change to the requirement and any affected policy changes are instantly communicated across the entire SDLC and TCE. This aspect of technology integration is a prime example of focusing on collaboration rather than pure technology that will ultimately drive efficient testing practices and an overall better product.

building a collaborative TCE step 2: Collaborative testing framework building a successful test automation practice within a TCE is a challenging task for many development organisations. Mature development organisations realise that simple UI test automation techniques like record/playback do not provide the collaboration required for the speed of today’s product delivery cycles.

There are many frameworks for test automation that have evolved over the years. Keyword-driven Testing (KDT) is a widely used test automation technique that many mature TCEs rely on to overcome the disadvantages of simple record/playback test automation. There are, however, some major disadvantages with managing the complexity inherent in KDT. Many applications require that thousands of automation keywords be developed to make use of KDT. Navigating and constructing test cases based on these keywords can be cumbersome and unpractical.

Acceptance Testing Frameworks (ATF) use a similar approach to structuring test cases (acceptance tests) using keywords that are implemented using coded fixtures.

www.testmagazine.co.uk

Cover story | 9

These frameworks do not support users in navigating all available actions (keywords). As with KDT frameworks, ATFs do not support collaboration since the structured fixtures (the implementation of the actions) cannot be easily reused and maintained.

Keeping in mind the end goal for a productive and predictable TCE organisation is to bring collaboration to the forefront; advanced testing frameworks should be considered.

Again, focusing on people vs. technology, frameworks should be considered that allows testers to create automated tests as early (and change as often) as possible.

The ideal state is to allow a test automation engineer to create test cases based on a product that still may be on the whiteboard.

State-driven Testing (SDT) is a framework that drives collaboration and addresses the maintenance and complexity issues of KDT by providing a UI-state transition model. by defining the given state at any moment of the user interface, the test automation engineer instantly knows what steps the application can take from there.

SDT reduces the test case possibilities from thousands of allowable actions down to a manageable list of between 10 and 20 allowable actions. SDT uses a domain specific language (DSL) to define the test automation framework which provides a highly maintainable and simple productive approach to structuring a test framework.

SDT is the next evolution of collaborative frameworks with an approach that provides the easeof-use for creating tests, enabling non-technical subject matter experts to write tests while eliminating the complexities and maintenance issues inherent in previous frameworks that created test silos.

Choosing a highly flexible framework that brings people together from the inception of an application all the way through to the testing endpoint overcomes many issues that are present in today’s frameworks.

There are numerous reasons why many TCEs still struggle with test automation projects:

www.testmagazine.co.uk

• Lack of required skills (test automation requires technical staff); • Maintenance effort (test automation is sensitive to application change and volatility); • Inconsistency of execution (test automation tools can deliver inconsistent results due to unresolved synchronisation points between test scripts and AUT); • Lack of relevance/test documentation (without associating tests with specific requirements and/or code and without clearly documented test cases, it is not clear what actually has been tested); • Lack of situational awareness (if a requirement changed, who informs the test automation engineer?).

While no framework is going to be the ‘holy grail’ of test automation, SDT does provide answers to many test automation challenges that remain within TCEs. SDT is an approach that suits organisations that want to deliver productivity through collaboration to their test automation practices without the pain and effort and limitations of relying on less capable commercial and open source tools.

building a collaborative TCE step 3: increase agility As product lifecycles continue to shrink, software development practices are evolving to keep pace. It has become increasingly important that TCEs test and modify software from the perspective of the group that will ultimately decide the fate of an organisation's business their customers. With the customer perspective in mind, Agile methods are becoming a more common development approach. Companies value the flexibility, transparency and speed of these Agile development procedures, which are getting adopted more and more even for large development and testing projects.

Agile development enables organisations to deliver products early while making any necessary changes for future roll-outs along the way.

Agile testing plays a critical role in this.

Collaborative testing at an early stage – and in parallel with always visible

SDT is the next evolution of collaborative frameworks with an approach that provides the easeof-use for creating tests, enabling nontechnical subject matter experts to write tests while eliminating the complexities and maintenance issues inherent in previous frameworks that created test silos.

December 2011 | TEST

10 | Testing the tester

The movement toward Agile adoption promotes environments where organisations can deliver and benefit from real-time visibility into testing activities and the quality status of a project. This is the driving reason to start fostering collaboration early so that old habits can be eliminated.

In order to deliver quality software, TCE using Agile testing methods collaborate more effectively with the business to improve project delivery outcomes.

TEST | December 2011

requirements – ensures that the quality of the software satisfies all necessary requirements regardless of change.

Most importantly, it enables the TCE to find potentially costly errors earlier in the development process, saving time, money and resources.

As stated before, quality assurance teams have typically lacked visibility into what is being tested, the actual test results, the business requirements driving testing and the extent to which they are reflected in the testing requirements. The movement toward Agile adoption promotes environments where organisations can deliver and benefit from real-time visibility into testing activities and the quality status of a project. This is the driving reason to start fostering collaboration early so that old habits can be eliminated. In order to deliver quality software, TCE using Agile testing methods collaborate more effectively with the business to improve project delivery outcomes.

As organisations continue to turn to Agile methods to speed up time-tomarket, reduce risk, and improve the overall quality of their software projects, we have pulled together some practical tips for TCE organisations on how they can optimise their software testing procedures: Test early and often: Not just some, but all relevant test procedures for a given project must be carried out comprehensively as early as possible in the lifecycle of a project. This means that unit tests, functional tests and load tests must be built into a single visible portal so that management, developers, and TCE members are on the same page.

Integrate unit testing: Unit testing is a critical step in the development process that ensures code being used will actually do what it is designed to do and will behave in the way it is intended.

Load test daily: As more software is designed for massively scalable deployments, load testing has emerged as an important step in the development process. The best product in the world will be useless if it crashes under the weight of too many users.

Know the start points: Test starting points must be clearly defined throughout the

testing process, so it is important to take starting points into consideration even in the development stage.

Be flexible: One of the benefits of Agile testing is the flexibility it delivers to the development teams. Software development projects are not static priorities, goals and requirements can be fluid and often change throughout the development lifecycle. Having an infrastructure goal that fosters collaboration can ensure that the team is always testing against the most up to date set of requirements and living up to the flexible environment that Agile development was designed to deliver.

The trend in technology is to bring people together. The first iPhone was laughed at by resident technocrats for not having as much technology as its competitors at the time. By focusing on how people work and bringing them together, Apple has continued to keep the iPhone the market leader in smartphones. Testing practices and TCEs have not followed suit. Rather than focus on how people work together, TCEs have compensated by throwing more technology and tools at the problem. By taking a step back and understanding that open and flexible communication through the tools being used will make sure everyone is on the same page. This collaboration delivers productivity through shortened development and testing cycles while at the same time bringing predictability into the environment.

The TCE will be able to see and track how subtle changes from the business that once created uncertainties, are now understood, corrected, and learned from.

Borland solutions are different than anything else in the market. Where competitors might focus on throwing complicated technology at a problem, Borland focuses on fostering collaboration between the people developing the technology. Borland’s first priority is providing customers with tools that turn whiteboard ideas into customer facing products faster than anyone in the industry. From cost savings to faster product delivery cycles, organisations using our tools will continue to build better software faster, regardless of framework, platform or system.

Archie Roboostoff Director of product management, Borland Solutions Micro Focus www.microfocus.com

www.testmagazine.co.uk

WHAT IS A GOOD FUZZING TOOL? Fuzz testing is the most efficient method for discovering both known and unknown vulnerabilities in software. It is based on sending anomalous (invalid or unexpected) data to the test target - the same method that is used by hackers and security researchers when they look for weaknesses to exploit. There are no false positives, if the anomalous data causes abnormal reaction such as a crash in the target software, then you have found a critical security flaw.

In this article, we will highlight the most important requirements in a fuzzing tool and also look at the most common mistakes people make with fuzzing.

PROPERTIES OF A GOOD FUZZING TOOL There are abundance of fuzzing tools available. How to distinguish a good fuzzer, what are the qualities that a fuzzing tool should have?

Model-based test suites: Random fuzzing will certainly give you some results, but to really target the areas that are most at risk, the test cases need to be based on actual protocol models. This results in huge improvement in test coverage and reduction in test execution time.

Easy to use: Most fuzzers are built for security experts, but in QA you cannot expect that all testers understand what buffer overflows are. Fuzzing tool must come with all the security knowhow built-in, so that testers only need the domain expertise from the target system to execute tests.

Automated: Creating fuzz test cases manually is a time-consuming and difficult task. A good fuzzer will create test cases automatically.

Automation is also critical when integrating fuzzing into regression testing and bug reporting frameworks.

Test coverage: Better test coverage means more discovered vulnerabilities. Fuzzer coverage must be measurable in two aspects: specification coverage and anomaly coverage.

Scalable: Time is almost always an issue when it comes to testing.

User must also have control on the fuzzing parameters such as test coverage. In QA you rarely have much time for testing, and therefore need to run tests fast. Sometimes you can use more time in testing, and can select other test completion criteria.

Documented test cases: When a bug is found, it needs to be documented for your internal developers or for vulnerability management towards third party developers. When there are billions of test cases, automated documentation is the only possible solution.

Remediation: All found issues must be reproduced in order to fix them. Network recording (PCAP) and automated reproduction packages help you in delivering the exact test setup to the developers so that they can start developing a fix to the found issues.

MOST COMMON MISTAKES IN FUZZING Not maintaining proprietary test scripts: Proprietary tests scripts are not rewritten even though the communication interfaces change or the fuzzing platform becomes outdated and unsupported.

Ticking off the fuzzing check-box: If the requirement for testers is to do fuzzing, they almost always choose the quick and dirty solution. This is almost always random fuzzing. Test requirements should focus on coverage metrics to ensure that testing aims to find most flaws in software.

Using hardware test beds: Appliance based fuzzing tools become outdated really fast, and the speed requirements for the hardware increases each year. Software-based fuzzers are scalable in performance, and can easily travel with you where testing is needed, and are not locked to a physical test lab.

Unprepared for cloud: A fixed location for fuzz-testing makes it hard for people to collaborate and scale the tests. Be prepared for virtual setups, where you can easily copy the setup to your colleagues, or upload it to cloud setups.

NEW PLATFORM RELEASE! NEW HIGHLIGHTS! • Improved user interface • Scalable test cases • Infinite text execution • Interoperability improvements • Enhanced reporting

Visit http://www.codenomicon.com/test mag to participate in the raffle!

12 | Special Testers

Super-testers?

People with Aspergers Syndrome and autism often have a number of attributes that can make them highly suited to software testing tasks.

Peter Macdonald gets some help from his colleagues to outline what skills an ‘aspie’ can bring to bear on software testing.

TEST | December 2011

www.testmagazine.co.uk

Special Testers | 13

People with Aspergers Syndrome often have a very good memory for facts and events.

They may be terrible with people's names but can often easily remember product keys, IP addresses and even entire routines. In testing, this can mean that they remember a lot of detail about their particular data set and it can result in both increased efficiency and early detection of problems.

W

hy is it often a good idea to employ people with Aspergers Syndrome as testers or developers? This was the question TEST editor Matt Bailey put to Garry Burge, an Asperger’s advocate and campaigner in Australia, who in turn sought the help of Gavin Bollard. The question ultimately found its way into my inbox in a rather grey Newcastle upon Tyne in the UK. If you type software testing autism into Google you’ll find many much more educated answers to the above question, but just to maintain your attention, Gavin has kindly summarised it for you here with a few of my own edit’s.

The trouble with any article like this is that we walk on dangerous territory, since we are purposefully labelling people, and that’s troubling. It’s important to remember that people are people, and people who are different as a result of a condition are still people. All people with Aspergers Syndrome are different and most have significant differences in their co-conditions. Not all of the information here will apply to all people with Asperger Syndrome.

I’m also sure that those of you already working in the testing community probably work with some people with Aspergers, knowingly or unknowingly and already think that the condition doesn’t matter as long as the job gets done.

People with Aspergers Syndrome are often particularly suitable for software development for a number of reasons:

www.testmagazine.co.uk

and events. They may be terrible with people's names but can often easily remember product keys, IP addresses and even entire routines. In testing, this can mean that they remember a lot of detail about their particular data set and it can result in both increased efficiency and early detection of problems.

In the coding world, this ‘gift’ often means that people with Aspergers Syndrome can develop without having to constantly refer to help and manuals.

API calls and their parameters are easily recalled and whole routines used in other projects can easily be remembered without having to open old projects. It makes for much speedier coding.

Hyperfocus When a person with Aspergers is really concentrating on something, particularly if it's closely aligned with their ‘special interest’, they can go into a state referred to as ‘hyperfocus’.

In this state, the person tends to shut off most input around them and concentrate solely on the task.

It's this single-minded drive which enables many people with Aspergers complete large amounts of work in a seemingly small amount of time. It also ensures great clarity of thought.

The main drawback with hyperfocus is simply that the person can be so absorbed in the subject that they will accidentally miss meeting appointments, lunch and even knocking off time. While this is usually a great feature for employers, care must be taken to ensure that long durations do not cause harm, such as keyboard overuse injuries.

Excellent rote memory

Attention to detail

People with Aspergers Syndrome often have a very good memory for facts

Software development and testing is one of those places where attention

December 2011 | TEST

14 | Special Testers

to detail is actually helpful if not essential. Coding relies upon very strict grammatical rules, and in all languages a clear syntax. It helps to have someone who is obsessive about the way that things are written, displayed and put together. Apply the same trait together with a curious mind to testing and fewer stones are left unturned.

Rules orientation People with Asperger’s Syndrome like routine and structure in their working environment and are often punctual in getting into work on time and finishing tasks/projects on time. Many also like doing tasks to a set of rules and are conscientious in their work as a result and if not then the rules need to be improved.

Unique point of view People with Aspergers often have quite a different frame of reference which leads to different points of view.

Often the new point of view is the result of looking at excessive details and the big picture at the same time – something that most people find difficult. When data and procedures are looked at in this manner, different patterns tend to emerge and these can result in radical approaches which cut development, testing or processing time significantly.

The different point of view is also quite useful when it comes to debugging.

People with Aspergers may try to view the code from the point of view of an object or variable. They can tell you what they can and can't see at the time, leading to easy tracing of undeclared objects and unpassed variables. It's quite a good debugging technique.

Peter Macdonald Managing Director Autism Works Ltd www.autismworks.co.uk

TEST | December 2011

Motivation The special interest is a major motivating force in the life of a

person with Aspergers Syndrome. An employee with a special interest in computers will enjoy computing to such an extent that they do large amounts of research and testing in their own time at home.

This results not only in much better education but also cost savings as the ‘paid for’ time that an employee spends on a problem is drastically reduced.

If all this makes a person with Aspergers sound like a superhero, then it's probably not telling the entire story. People with Aspergers Sydnrome can make very capable employees and in some fields they are actually much more efficient than neurotypicals. Unfortunately, these people have issues too, chiefly in their interactions with other people.

You may find that your ‘aspie workers’ have difficulty in social situations but if that's not a major job requirement, then they're probably the best people to have in testing and development.

And this is exactly why we created our company. The reality is that despite all of the above in the UK alone there are half a million people with a diagnosis of Aspergers and Autism, with only 15 percent in employment. All that Autism Works sets out to do is positively discriminate; by that I mean that we just make it easier for someone with Aspergers to work with us. To the best of my knowledge there are only a few companies doing something similar in the world, check out Specialisterne, Passwerk and Aspiritiech.

Hopefully by shining a light on what can be achieved more companies will adopt our policies and we will not only create better working environments but also be much more tolerant of people’s differences.

The different point of view is also quite useful when it comes to debugging. People with Aspergers may try to view the code from the point of view of an object or variable. They can tell you what they can and can't see at the time, leading to easy tracing of undeclared objects and unpassed variables. It's quite a good debugging technique.

www.testmagazine.co.uk

16 | Staic analysis

Testing at the boundaries of science With CERN once again in the news and the possibility of particles travelling faster than light making the scientific community reassess its very foundations, Matt bailey speaks to Axel Naumann, part of the team of physicists that developed the crucial number-crunching software at the heart of the organisation, about how static analysis is improving its software quality.

TEST | December 2011

www.testmagazine.co.uk

Staic analysis | 17

C

ERN is the European organisation for Nuclear Research.

Headquartered in geneva, Switzerland, the world’s leading laboratory for particle physics has rarely been out of the news in the last few months with stories about how the lab might have detected a neutrino travelling faster than the speed of light – a feat thought impossible a few months ago, but now very much under serious investigation.

CERN is one of the world’s largest and most respected research institutions; it has successfully deployed static analysis to improve the integrity of the source code found across a number of projects analysing data from its famous Large Hadron Collider (LHC), the world's largest and highest-energy particle accelerator which is addressing some of the most fundamental questions of physics and advancing our understanding of the deepest laws of nature.

The LHC lies in a tunnel 27km in circumference, 175m beneath the Franco-Swiss border near Geneva.

It was built to test various predictions about high-energy physics, including testing for the existence of the hypothesized Higgs boson and of the large family of new particles predicted by the supersymmetry theory.

During operation, the LHC produces vast quantities of data that have to be processed in order to detect the presence of these exotic particles. One of LHC’s core software ingredients, ROOT, is a program used by CERN’s physicists to store, analyse, and visualise petabytes of data about the experiments. The lab used static analysis from test automation specialist Coverity to help improve their software’s quality. “better quality software translates to better research results,” confirms Axel Naumann, a member of CERN’s ROOT Development Team. “Like CERN, Coverity finds the unknown; its development testing solution, Static Analysis, discovers the rare, unpredictable cases that can’t be recreated in a test environment.”

Testing at CERN The experiments conducted around the LHC generate approximately 15 petabytes data per year, equivalent to 15,000 standard disk drives. Given the size and scale of these experiments,

www.testmagazine.co.uk

CERN has implemented a number of processes to ensure data generated by the LHC experiments are accurate and as bug free as possible. “ROOT is used by all 10,000 physicists, so software integrity is a major issue,” says Naumann. “A bug in ROOT can have a significant negative impact on the results of the LHC experiments and physicists' data analyses.”

Within the first week of implementing static analysis, CERN’s ROOT Development team found thousands of possible software defects that could have impacted software integrity and research accuracy, including buffer overflows and memory leaks, with very few false positives. To improve the integrity of its source code, the ROOT team spent six weeks on resolving the errors and continues to use the solution in production daily to prevent further software defects from occurring.

“We used a massive amount of testing before,” say Naumann. “The software has to be working properly to ensure the results you are seeing are right. I would say that the amount of effort we put into testing the software is more than we put into its writing. For each stage in the experiment software we have independent tests like unit tests, memory checkers, and performance checkers. We test coverage, we enforce coding rules – and we were actually doing a pretty good job.”

CERN works in a different way to most software development environments.

“CERN is not a company, we don’t have teams which do things, we have physicists who write the code to get the jobs they need doing done and then they write the tests because they want to make sure their code actually works and that it continues to work even if other people change their code,” says Naumann. “The important thing is that we realise that we cannot operate without static analysis. We had a massive testing framework before and we thought that dynamic tests were sufficient, and they’re really not, especially when you want to cover the rare cases when dynamic tests don’t get you all the way. Now we have static analysis checking our code and that by itself help a lot. I believe that CERN is not the only one having fallen into the trap that if you just do enough run time testing you will be covered and that is really not the case. Whether you follow ISO or not doesn’t matter, you’ve got to do static code analysis.

Axel Naumann part of CERN’s ROOT development team

The most important change is that the developers feel watched, because they know if they upload sloppy code to the repository there will be somebody looking at it and slapping their hands if it’s not done properly. My physicists tell me that it is a dramatic change in the way they develop their algorithms and I think that is invaluable.

December 2011 | TEST

18 | Staic analysis

“Two out of the four experiments are now using this in production, so any code change will be checked by static analysis,” says Naumann. “It reduces the probability of new bugs coming into the software. The most important change is that the developers feel watched, because they know if they upload sloppy code to the repository there will be somebody looking at it and slapping their hands if it’s not done properly. My physicists tell me that it is a dramatic change in the way they develop their algorithms and I think that is invaluable. Even though there are procedures that call for testing, you should not assume that these will actually help. It helps controlling the issue, but actually getting things fixed still is based on motivation, you have got to motivate the developers to look at the report and deal with them properly.”

Static analysis tested 50 million lines of software code in the LHC software, helping CERN find and fix more than 40,000 defects.

The Root of the problem ROOT is a data processing software used for the Higgs boson experiment.

It runs statistical analysis on the huge amounts of data generated by the search for the particle. “It is one of the ingredients for the experiment,” explains Naumann. “Our physicists have their own code and do their own tests but they all use ROOT for the basics like statistical analysis of huge amounts of data, reading and writing huge amounts of data and utilisation of that data.

“The way we test is very mixed,” he adds, “because there is no overarching structure that people have to follow. In our ROOT development team we use something very close to an Agile approach. We also have continuous testing; we have code reviews and we have small steps that we test regularly - it’s not a formalised Agile approach, but it is very close.

For ROOT we have a strong lead developer who basically decides what goes in and what doesn’t featurewise. We have users around the world who contribute patches and new developments and it’s our job to make sure it all fits together.”

Matt bailey Editor 31 Media Limited www.31media.co.uk

TEST | December 2011

Testing at the boundaries of science Even though the realm of theoretical particle physics is as cutting edge as it gets in the scientific community, there

are still some known quantities and constants that make testing a lot easier.

“We know our test coverage and we know how our software behaves. When we release it, we know it is working correctly if it proves the same physics results as the previous version and if it doesn’t we sit down and find out where the difference comes from,” says Naumann.

“We can even measure the effects of software bugs in the physics results because when we have a bug it can change the measurement and we get a wider distribution of measurements.

We don’t just measure things once, we measure them billions of times and then we use statistical analysis which gives us a certain distribution of measurement values and if we have bugs this distribution becomes wider. To get the real value we need to measure more often – taking data longer, maybe two years instead of one year.

“We know and measure the effects of software bugs and we take them into account in our physics results. That said, we don’t want to have bugs that are sensitive to physics. At CERN we are looking for the Higgs boson, a particle we assume exists. If we were to have a bug which is triggered by the Higgs particle being found in the detector, that would be a disaster because it could prevent us from finding it!”

Even though, in principle, Naumann’s team have the bugs well under control, they have to make sure that they find as many as possible to improve the software’s efficiency and the quality of their results. “Testing is done by the people who know what the code is supposed to be doing,” says Naumann.

“This situation is good and bad at the same time. but it’s not like the developers are testing their own code all the time. This does happen, but there are also more complex stages where the experiment as a whole is testing the software as a whole to verify the results.”

Testing at the speed of light The remit of the static analysis solution has now also expanded to include the software underpinning specific LHC experiments, which equates to 50 million lines of code. Static analysis is now used by thousands of developers across CERN. Using Coverity's web interface, even non-professional developers are able to quickly understand and remediate errors in the software, as well as the reports produced from the analysis. Moreover,

Coverity has achieved viral adoption across the CERN developer community within a period of two months.

Given the recent news that an experiment had once again appeared to show neutrinos moving at speeds faster than light, the question of whether the ROOT software was used to number crunch the data had to be asked: “I couldn’t say for certain,” replies Naumann, “but I know of no experiment in our environment which doesn’t use ROOT, so I assume that they did.”

“We know and measure the effects of software bugs and we take them into account in our physics results.

That said, we don’t want to have bugs that are sensitive to physics. At CERN we are looking for the Higgs boson, a particle we assume exists. If we were to have a bug which is triggered by the Higgs particle being found in the detector, that would be a disaster because it could prevent us from finding it!”

www.testmagazine.co.uk

Break free!

– a unique cloud-based learning solution – simple monthly licensing ■

Save 90% on total cost of learning

■

Increase certification pass rates

■

On-demand, 24×7 access to global testing training content and resources

■

Value-based licensing – low cost of engagement and implementation

■

Replicate classroom benefits

■

Scalable, global, secure, private & proven

■

Blend Self Study with Virtual Classrooms and Real Classrooms

■

Virtualization with real tutors, real time and real benefits

Visit www.learntesting.com or email enquiry@learntesting.com to find out more about Freedom Learning

Please quote TESTMAG for an additional 10% discount off all Learntesting licenses for TEST Magazine readers

www.learntesting.com

20 | Testing data

12 Questions to prevent a testing data breach Howard Wiener, director of Professional Services at Direct Computer Resources says that answering the 12 questions he poses in this feature can help your organisation examine the challenges its faces when handling sensitive data during software testing

C

omedian Groucho Marx may be best recognised for saying that he would “refuse to join any club that would have (him) as a member.” When it comes to the Data Privacy Breach Club, we embrace his witticism readily. Companies that have experienced business ramifications of the unintended exposure of Personally Identifiable Information (PII) include those in credit card processing, clothing and home goods, marketing, and gaming entertainment systems.

At this point in software development history, everyone in the testing community should already understand that using production data for testing is not only wrong, but also violates longestablished rules and regulations from both governments and industry.

Still, the numbers are staggering. The San Diego, California-based Privacy Rights Clearinghouse reports that more

TEST | December 2011

than 542 billion records have been breached in the U.S. since 2005. And if you’re thinking that you just need a better firewall or intrusion detection system, it’s time to reconsider. Of those 542 billion breached records, greater than 32 million were a result of internal mismanagement, business operation failures or insider data theft.

What could be more sadly ironic than concerned citizens in Tampa, Florida who purchased home security systems only to discover later that their personal information was stolen and used to file fake tax returns in their names?

The money involved is also astounding. In March 2011, the Ponemon Institute’s annual study of the cost of a data breach put the price tag at $214 per compromised record. In another case, Ponemon estimated that the total exposure of an entertainment and gaming company resulting from a large data breach could total more than $2Billion.

Despite Americans’ trust in legislative healthcare-related security

requirements like HIPAA, another Ponemon report pegs the expense of data breaches in the healthcare system at $6 billion annually. In July, a California-based hospital system paid $865,000 to federal regulators to settle claims of unauthorised access to medical records. The complaints cover 2005 to 2009, a time during which hospital employees were repeatedly caught and fired for peeping at the medical records of dozens of celebrities.

These reported healthcare data breaches from 2005 through 2009 – a four-year time period – point the way towards other disturbing statistics. An April 2010 Accenture survey revealed that repeated security breaches are an ongoing challenge for many organisations. Fifty-eight percent of executives polled said they had lost sensitive personal information, and for nearly 60 percent of those who had a breach, it was not an isolated event during the 12-month period. In sum, companies compromised once are

www.testmagazine.co.uk

Testing data | 21

not bridging the security gap quickly enough to prevent additional incidents.

The cost of security Outright expenditures resulting from data breaches encompass numerous areas, including development and implementation of new internal polices, technical remediation of data systems, increased auditing services, public relations programmes, customer notifications, identity theft monitoring services and legal judgments.

Other costs, such as lost customers, are less tangible. Even companies that pride themselves on outstanding faceto-face customer service suddenly fall short when personal data is exposed.

Customers truly fear identity theft and they feel violated when they hear media reports or receive a data breach notification.

The 12 question challenge

Howard M. Wiener Director of Professional Services Direct Computer Resources www.datavantage.com

Answering the questions below can help your organisation examine the challenges you face when handling sensitive data during software testing: 1) How does information flow across your enterprise?

2) How much customer data do you actually need to collect and process?

3) How long do you hold on to PII? 4) Do you have internal security policies related to who has access to which data?

5) Have you trained company associates about these policies and do you monitor their adherence to them?

6) Are your data privacy policies supported by relevant technology to both prevent and monitor for data leaks?

7) Are your customers aware of your data privacy policies and practices?

8) Are you using production data for testing or other non-production purposes? If so, is it being masked or obfuscated first?

9) Do you know how many copies of production data exist internally and externally with subcontractors and vendors?

10) Do your subcontractors and other vendors understand and abide by your data privacy requirements?

11) Could you pass internal and external audits?

12) How many of your company’s divisions, departments and executives are involved in keeping sensitive data secure?

www.testmagazine.co.uk

If your organisation is conscientious about data privacy, the questions above will not have rocked your world.

If a few were difficult, it’s time to do more than create a committee to study the issue.

Technology considerations Data obfuscation is a set of processes by which production data is extracted and transformed so that it may be used for non-production purposes, such as systems development or testing, without compromising any personally-identifiable information it may contain. The trick to doing it well is to minimise the amount of work necessary, while producing a data set with the same logical relationships and characteristics as the original data. One complicating factor is that processes defined for a particular set of data structures will have to be revised if the structures are changed, which is a likely result of the systems development processes that are a primary driver of the need to obfuscate in the first place.

As you envision a new privacy initiative or work to enhance your current practices, consider this list of technology-related questions that relate to the data obfuscation process: • Is it easy to deploy and customise, requiring minimal custom programming?

• Is it automated and repeatable?

• Can it create realistic-looking data in sufficient quantities?

• Does it maintain logical relationships identical to the original source data?

• Will it work to select and to subset data from all the types of data stores you’re using, including relational, hierarchical or object databases or flat files?

• Will implementing it force you to restructure existing application data stores?

• Will it work with all releases of your operating platform(s)?

• From a cost effectiveness perspective, will it facilitate the security you’re seeking while minimising business disruptions?

After reviewing the questions pertaining to privacy breaches and data obfuscation above, we have one final piece of wisdom from Groucho Marx regarding the gravity of your prospective membership in the Data Privacy Breach Club. In his 1933 film Duck Soup, Marx opined, “I’ve got a good mind to join a club and beat you over the head with it.”

Data obfuscation is a set of processes by which production data is extracted and transformed so that it may be used for non-production purposes, such as systems development or testing, without compromising any personallyidentifiable information it may contain. The trick to doing it well is to minimise the amount of work necessary, while producing a data set with the same logical relationships and characteristics as the original data.

December 2011 | TEST

22 | Feature Training Corner

Christmas wishes It’s Christmas time... And top of Angelina Samaroo’s wish list?

That Christmas comes early for every project!

A

t the start of every project we would be invited to a walkthrough of the new project. At this meeting, the product specialist team will explain what the new product will be, why it is required, what the budget is, and what the drop-dead date for delivery is.

The latter being the real date required – no giving us an earlier end-date because a slip is inevitable. Starting off with false information is more likely to cause this than anything that professional software engineers might miss.

Angelina Samaroo Managing director Pinta Education www.pintaed.com

TEST | December 2011

As soon as requirements are documented, either as part of the business requirements, functional specification or user stories, we would be invited to review them. This may or may not be a formal affair, depending on the software development model being followed. Whether this is a fullblown statement of requirements or a set of user-stories, the perspective of the test community is essential at this early stage, if overall development costs are to be reduced. The tester perspective here will show up likely issues in functionality well before the system is implemented.

If functionality has been defined up-front, we would be given time to analyse them so that we can apply our tester mindset to what has been written. We would firstly look to apply the cause-effect technique to what has been written. Does every effect have an identified cause – in other words, when this functionality is executed, then this next functionality is activated, and so on. In other words, have we identified the whenthen? Also, we would place the new requirements in context with the current system functionality. We would ask the question – do they complement what we already have, or are there likely conflicts? Is the user experience likely

to be improved or reduced? Have we increased the number of interactions with the system to effect a transaction or reduced it? In other words, we look for usability issues up-front.

When requirements are not documented, we are not shouted down when we ask for some statement of the business need.

Developers, on releasing code into test, would provide us with a set of release notes. The notes would show the areas of functionality changed; the updated versions of software packages; the extent to which testing has been carried out at developer level; outstanding defects and a risk assessment statement. In this way, we are informed on likely risks to product and project as soon as development has been completed.

Thus our risk assessment exercise is timely and relevant.

When testing in earnest, we have agreed defect turnaround times. If a defect cannot be addressed by the agreed time, we are informed, rather than having to chase.

When the system goes live, we take time out to celebrate. It is important to take time out to enjoy life. This brings me to the all important worklife balance.

Work life balance A long while back, or so it now seems, I recall congratulating myself that I was born in today’s world. The world wars long behind us; oil prices meaning that you put another jumper on, not another war; the great outdoors as a child meaning that I could run around the neighbourhood with my friends all day, without a care in the world. At school, you were streamed. If you wanted to go to university, you were pushed, and encouraged. You were not daunted by fees. You just needed to make the grade, and your parents and the government took care of the rest.

However, it didn’t stop the ‘too-muchterm-left-at-the-end-of-allowance’

The perspective of the test community is essential at this early stage, if overall development costs are to be reduced. The tester perspective here will show up likely issues in functionality well before the system is implemented.

www.testmagazine.co.uk

Join the Revolution Don’t let your legacy application quality systems hamper your business agility At Original Software, we have listened to market frustrations and want you to share in our visionary approach for managing the quality of your applications. We understand that the need to respond faster to changing business requirements means you have to adapt the way you work when you’re delivering business-critical applications.

Our solution suite aids business agility and provides an integrated approach to solving your software delivery process and management challenges.

Find out why leading companies are switching to Original Software by visiting: www.origsoft.com/business_agility

24 |Design for TEST

dilemma! This wasn’t all bad though, it made sure you were first to apply for those summer jobs.

I remember vividly a discussion with my colleagues when I first started out as an engineer. It went something along the lines of – work five out of seven days every week (the term 24/7 was yet to be coined); have 22 days paid holiday (but dates to be approved by your boss); do this each and every year for the next 42 years; at age 64 work four days out of seven to prepare you for the apparent shock of being the boss of all 24 hours of your day); then be the boss of your whole day. At 22 you can imagine my bewilderment at this projection of my future.

Add to this the life bit, which is not all about fun, and we can no longer measure our free time in days but in hours. Sometimes, life can throw you a curve ball or two. Real life comes with real problems. You have to man-up and face them. The family springs into action and work provides a welcome break. The work-life balance now takes on new meaning. You need work to balance life. It provides shape and structure to your day. My wish is that life’s curve balls are few enough so that we can keep complaining about work!

This brings to mind a dinner party. My friend’s husband was loading up the dishwasher. The dishes were many as were his frowns. He had a very stern look on his face. I asked if he was ok.

Yes, he said, but if I look like I’m happy she’ll ask me to do it again!

Then try counting your ‘me’ time. This is in minutes in today’s world. In today’s world, the technology means that we no longer need to wait for things, we can demand instant responses. Salespeople can call right into our home, just as we’re having dinner with the family; or they can call on a Sunday, when we’ve finally managed to wind down and get into that lazy Sunday afternoon feeling. I can switch off the mobile, but not the landline – my wish is that this channel of communication is for our families, and is respected as such.

When we provide our details for a specific need, my wish is that data protection laws also apply to us.

Encouragingly, I see more and more families on their bikes around town. Our elongated autumn has been a wonderful gift to a southern softie; guess there’s a few of us around. The evening football field was buzzing last Friday night with the happy laughter of children and their parents. My wish is that we remember to switch off the technology and go out and play ball.

As we head into 2012, my personal New Year’s resolution is to resolve not to feel guilty about not making one, yet again.

Best wishes to one and all for a very Merry Christmas and a Happy New Year.

TEST | December 2011

Living labs This issue Mike Holcombe is in the living lab – a place where users can become designers.

T

he idea of a living lab has become fashionable in some quarters. It is a place where users or potential users can interact with software designers, ICT service providers and managers to explore new ideas and new solutions. It is often a place where public sector, private sector and the ‘people’ can meet.

It is not a user testing facility or a focus group but a significant development of those ideas in a move towards the empowerment of users in influencing the systems that affect them. These living labs are often focused on a particular town or city and many of the examples involve local authorities, university researchers and the private sector. They may have a specific focus, eg e-services for citizens, healthcare etc.

There are some in the UK – Coventry, Manchester, Newcastle, Sheffield etc. The European Network of Living Labs (www. openlivinglabs.eu) has more details.

Testing in the living lab What is the connection with testing? Clearly, having access to users of systems in realistic environments and with the appropriate facilities for observing their behaviour when using prototype software, is one benefit. But it is more fundamental than that. The real benefit is to allow users to contribute more the identifications of needs, the outline of potential solutions and the integration of new services into a coherent ICT environment.

So far so good but the concept of user-driven innovation is only just beginning to percolate through to the industry and to those who commission software. It raises a number of issues for developers and testers. The first, and one I have discussed before, is that testers should be involved at the beginning of all projects in order to help focus the project on outcomes and performance rather than on abstract design. Not all software developers and managers are comfortable with that. What the Living Labs does is to bring the users into the equation as a driver and innovator of new systems. The emphasis is again the introduction of a stakeholder into the start of the process.

It might be difficult to adjust to but this is surely a positive development.

What is the connection with testing? Clearly, having access to users of systems in realistic environments and with the appropriate facilities for observing their behaviour when using prototype software, is one benefit. But it is more fundamental than that. The real benefit is to allow users to contribute more the identifications of needs, the outline of potential solutions and the integration of new services into a coherent ICT environment.

Mike Holcombe Xxxxxx Xxxxxx www.xxxxxxxxxxx

www.testmagazine.co.uk

Industry-leading Cloud, CEP, SOA and BPM test automation Putting you and the testing team in control Since 1996, Green Hat has been helping organisations around the world test smarter. Our industry-leading solutions help you overcome the unique challenges of testing complex integrated systems such as multiple dependencies, the absence of a GUI, or systems unavailable for testing. Discover issues earlier, deploy in confidence quicker, turn recordings into regression tests in under five minutes and avoid relying on development teams coding to keep your testing cycles on track.

GH Tester ensures integrated systems go into production faster: • • •

Easy end-to-end continuous integration testing Single suite for functional and performance needs Development cycles and costs down by 50%

GH VIE (Virtual Integration Environment) delivers advanced virtualized applications without coding: • • •

Personal testing environments easily created Subset databases for quick testing and compliance Quickly and easily extensible for non-standard systems

See Green Hat’s capabilities for yourself at EuroSTAR in Manchester, November 21-24

Every testing team has its own unique challenges. Visit www.greenhat.com to find out how we can help you and arrange a demonstration tailored to your particular requirements. Discover why our customers say, “It feels like GH Tester was written specifically for us”.

Support for 70+ systems including: Web Services • TIBCO • webMethods • SAP • Oracle • IBM • EDI • HL7 • JMS • SOAP • SWIFT • FIX • XML

26 | Testing Featuretools

Test case paths: Happy, sad and bad Senior testing consultant Nishant Verma pits the QTP testing tool against his own Agile tool, Twist and finds that QTP could do better.

I

am a tester in the team that is developing a testing tool called Twist. In past I have worked on variety of tools including QTP, Loadrunner, SoapUI, etc. I remember using QTP 8.0 in past to automate one of the legacy application developed in .NET 1.1, this was some three to four years ago. I was reading some blogs where I was reminded of QTP and I decided to download the latest version of it and see what’s new with the tool and also to see how it’s keeping pace with the changing scene of the IT industry where Agile methodology is becoming more popular and replacing traditional Waterfall methodology.

So I downloaded QTP (Trial Version 11) from the website. I have a Dell Machine with Windows 7 installed on it. Browsers installed on my machine are: Firefox (Version 6.0.2); IE (Version 9.0.2); and Chrome (Version 13.0.782.220). As part of my regular testing activity I keep my browsers updated to see whether our tool (Twist) supports latest browsers. I installed QTP with one click, a very easy process. It ships with four default addins : Standard Windows Application; Web Object/Applications; ActiveX

TEST | December 2011

Controls; and Visual Basic Applications.

QTP allows me to setup the environment before recording the test. So I can navigate to Automation, Record and Run Settings. The window below (fig 1) shows up. I can set the application URL there and when I pull the browser drop down, it just shows up Microsoft Internet Explorer.

If I am testing a web application I would want to test it on various browsers available, or at least Firefox (Version 4, 5, 6), Chrome (Version 12, 13) and IE 8, 9 (on Windows 7).

Fig 2

Fig 3

Fig 1

Though I have Firefox and Chrome installed on my box, they do not shows up in the list. So when I opened the “HP QuickTest Professional Product Availability Matrix” pdf (present in installed location) I was shocked to see the matrix Below (fig 2 & 3). Are you serious?

I looked back at my tool which I am using (Twist) and I saw the list which appears below (fig 4). Read more here about the supported web browsers (www.thoughtworks-studios.com/ twist-agile-test-automation/2.3/help/ system_req.html)

Fig 4

When it comes to web application testing, QTP still lacks basic browser support. We have so many browsers around and Firefox is releasing new

www.testmagazine.co.uk

Testing tools | 27

Fig 5

www.testmagazine.co.uk

versions aggressively. Also Amazon has recently launched a new browser on their device ‘Fire’ called Amazon Silk and people are expecting the OS version of that browser soon so that they can check compatibility with their websites. So QTP is way behind other functional test automation tool by just supporting IE as the browser available for recording. This is precisely one reason why I would use Twist for web application testing and not QTP. Once I am past the record and run setting, I start using it to automate test against my sample application which is a project management tool ‘Mingle’.

I have used QTP before and I am aware of the basic things like test creation, actions, keyword-driven approach etc. I found the ‘Process Guidance’ really good for first time users. It guides the user as to ‘how to create and organize your test in a step by step manner’ (One thing which is not present in Twist and I would want to add). This is mostly because my first 30 minutes with the tool forms my basic opinion. So a tool should be easy to use with a rich user experience.

I started by creating a layout of the project by creating actions which are logical names for Test scenarios like ‘Login’, ‘Create Project’, ‘Create Card’ etc in QTP. The left side of the image below (fig 5) shows how the test looks in QTP and right side (fig 6) shows how it looks in the tool I am using (Twist).

So once the project layout was created in QTP I started recording the test.

Recording is very simple in QTP and remains one of the powerful features of the tool. Fig 7 shows how it looks in QTP. QTP shows me the recorded script and I can add comments to that script to say why it was recorded. However recorded script doesn’t help anyone apart from me to understand what this test does. So it remains very QA oriented. Action name helps me understand the intent of the test but doesn’t convey much about what my test is doing. It’s not test case based.

Fig 6

Fig 8

Fig 7

Let’s go to the tool I am developing and using: Twist. Once the project layout is created, I can write what my test will do, basically, what the test steps are. I can further break down my test scenario into different test steps and record each of them one by one. Also, all this is in plain English and any one can understand it. So it’s easy for me to write my acceptance test cases and then automate it.

When I record the test for each step, it generates an underlying Java method which looks as shown below. I have

December 2011 | TEST

28 | Testing tools

FIG 10

FIG 9

highlighted the method name.

I have my tests written in domainspecific language and it can be easily read by any one on the team irrespective of their knowledge about the code. Anyone can check out the code from the repository and read it for their own understanding.

This basically takes away the pain of interacting with two testing tools/ software: one to write my manual test cases and other one to automate.

This is another reason why I would use Twist for test automation and not QTP.

Analysing the tools Let’s analyse both the tools on the aspects of test framework and test automation maintenance. When you launch QTP, you will notice that the Process Guidance panel shows you a path to achieve what QTP has popularised in the testing community as ‘keyword-driven testing’.

“Keyword-driven testing is a technique that separates much of the programming work from the actual test steps so that the test steps can be developed earlier and can often be maintained with only minor updates, even when the application or testing needs to change significantly.”

Nishant Verma Senior consultant nishuverma@gmail.com www.nishantverma.com

QTP Help File on Keyword Driven Testing This technique relies on three pillars: • Test steps; • Keywords; • Object repository.

TEST Digest | December 2011

A tester generally writes test steps using the keywords, while a tester with technical knowledge creates the function library using built-in keywords or their own user-defined keywords and maintains the Object Repository.

So a slight change in application would require changes in the Object Repository and then the functions.

“The keyword-driven methodology is especially useful for organisations that have both technical and less technical users because it offers a clear division of automation tasks.

This enables a few experts to maintain the resource framework while less technical users design and maintain automated test steps. Additionally, after the basic infrastructure is in place, both types of users can often do their jobs simultaneously.”

QTP Help File on Keyword Driven Testing The QTP keyword-driven testing approach actually requires a significant manpower to create a test suite and maintain it. Also if you are following this test framework approach you need to put a significant effort into creating the infrastructure and then start writing the test cases.

So it takes up a considerable amount of time before you see the benefits of QTP. On the other hand if you are using Twist, it’s like you download it, install it and start automating your tests right from the day-one. You see an immediate benefit from the tool unlike QTP (Where you will have to wait for quite some time, maybe a month, two months or 15 days till the basic infrastructure is built). Twist does provide a user some guidelines to use

the tool but you don’t need such heavy infrastructure to write tests. All you need is the domain knowledge or the application under test.

“Writing your scenarios at the right level of abstraction is critical to building an effective test suite. Here are a few guidelines that can help you get there: • Tests should be independent of the user interface of the application; • Tests should strive to be at the highest level of abstraction possible, without sacrificing readability; • Don't repeat yourself: Repeating a series of steps at any level – at a scenario, or in the implementation can increase the maintenance overhead; • Use Extract Concept and Push to Implementation to achieve granularity that reflects your domain and also promotes reuse; • Your test should ideally be an illustration of the functionality as opposed to some UI mechanics.

For instance, if a test reads: Shopping cart – enter ‘5’ in units and ‘10.0’ in price and select ‘air shipping’ in shipping dropdown.

The description of the UI clutters the intent here. Instead the test would be better as: Shopping cart – create order for ‘5’ units at price ‘10.0’ for ‘air shipping’.

This helps to clarify the intent.”

Twist Help File on ‘guidelines for your Test Suite’ Following a page object pattern is a nice way to group all the actions that are possible on a given screen.

Twist provides you more benefits than QTP when it comes to test suite creation and maintenance.

www.testmagazine.co.uk

20 iNNoVATioN FoR SoFTWARE QuALiTy

LEADING TESTING PROVIDERS

20 Leading Testing Providers

APiCA

Do you watch your Web performance?

Apica gives you the insights you need.

Are you alerted if performance degrades?

use of our cloud-based test centers allows load testing with unmatched scalability.

Can you handle peak traffic?

Create and monitor real world user scenarios with Selenium. you can even re-use your existing scripts.

Number of sites: uK – 20 globally – 200 T: +44 (0)1628 440826 E: mike.howse@apicasystem.com W: www.apicasystem.com Contact: Mike Howse

AuToMATioN CoNSuLTANTS Automation Consultants (AC) is a leading independent consultancy offering a comprehensive range of cloud based testing products. We also offer full end-to-end testing services ranging from fully outsourced test management to the provision of both functional and non functional test resources.

Products AC markets a number of advanced testing tools, including TestWave, a test management tool hosted in the cloud, which allows users to set up test projects in minutes, dispense with servers, and pay by subscription.

As well as managing testing, TestWave incorporates an Automation Framework, which optimises a test team's automation efforts so as to maximise the business benefits of automation.

1) Test Management: AC provides a high quality, independent test management service. Customers value AC's ability to manage complex test programmes in a way that balances risk with the need to meet tight budgets and deadlines. Customers also appreciate AC's independence, which allows it to provide a true and unbiased view of the systems being tested.

2) Test Automation: Automation Consultants has in depth expertise in functional test automation, using industry standard tools such as HP UFT/QTP, Selenium and IbM Rational Functional Tester. The company delivers automation using a methodology designed to maximise return on investment and the maintainability of scripts.

Test Services

3) Performance Testing and optimisation: Ensuring application performance is increasingly important, especially with the shift of enterprise IT towards the cloud.

With over a decade of experience in performance testing and optimisation, Automation Consultants offers an unparallelled service in this area. Tests are performed with industry standard tools such as HP LoadRunner, and AC's custom tools and harnesses. Testing is followed up with detailed analysis of results to identify performance bottlenecks and potential solutions.

Automation Consultants offers a comprehensive software testing service spanning both functional and non-functional testing. The service is built on in-depth knowledge acquired from working with many blue-chip and government clients.

4) Training: Automation Consultants provides training in leading HP Application Lifecycle Management (ALM) products including Quality Center, Unified Functional Testing (QTP) and LoadRunner.

Another leading tool is the AC Latency Analyser. The Latency Analyser predicts the effects on performance of physically moving a Data Centre from one location to another. This can deliver major cost savings by anticipating performance problems before they occur.

Number of sites: uK – 2 globally – 4

MERLiN HouSE, bRuNEL RoAD, THEALE, READiNg, bERKSHiRE, uK. Rg7 4Ab T: +44 (0)1189 323001 F: +44 (0) 1189 323003 E: info@automation-consultants.com W: www. automation-consultants.com Contact: Jeff Cunliffe

TEST Digest | December 2011

www.testmagazine.co.uk

20 Leading Testing Providers

bugfinders ltd It’s time to call in a software tester, but do they have the skills and motivation to find your bugs?

BugFinders offers you the chance to employ a globe-spanning community of professional testers to do the job for you - and we only charge our clients per bug! This means you pay for results approved by you personally and are in complete control of all costs and testing services.

BugFinders delivers a complete coverage of platforms, browsers and testing types. We operate on a 24/7 basis, allowing us a fast turnaround - we can complete hundreds of hours work overnight! Our rapid response and scalable approach to business is what makes us stand out from the crowd, so you can get your software tested at the click of a button.

You can also visit our site and check out our free tools and online training courses.

All you have to do is register!

BugFinders’ unique business model is simply the best way to get your testing done in time, to budget and at high quality – we help you meet the challenges of today’s market.

Entry Level Price: £1,000

3 Rockfield Business Park, Old Station Drive, Leckhampton, Cheltenham, GL53 0AN, United Kingdom T: +44 (0) 844870 8710 E: enquiries@bugfinders.com W: www. bugfinders.com Contact: Martin Mudge

Codenomicon Codenomicon is a leading vendor of proactive security testing solutions and abuse situation awareness tools.

The company is known for its innovations in negative black-box testing.

Founded in 2001 as a spin-off of the widely acclaimed PROTOS test tool project of the Oulu University Secure Programming Group (OUSPG), the company continues to demonstrate a high degree of innovation by incorporating the latest research into its testing technique.

Codenomicon’s solutions are used by top governments and leading software companies, operators, service providers and manufacturers to secure critical networks and to provide robust and reliable products and services.

Codenomicon has over 200 customers including AlcatelLucent, AT&T, Cisco Systems, Nordea, Sony-Ericsson, Microsoft and Nokia Siemens Networks.

It’s what you don’t know that makes you vulnerable

Both in software security testing and network management, it is essential to know your products and systems to protect them. With Codenomicon’s proactive security testing software and situation awareness tools you can discover the problems at the earliest possible moment.

Proactive security testing

Codenomicon Defensics is a proactive testing solution for finding and mitigating both known and unknown vulnerabilities in software even before deployment, improving application and network security. In the core Number of sites: UK – 20 Globally – Over 200

of the solution is fuzz testing, a method where invalid input are fed to the system under test to expose vulnerabilities.

While traditional testing has been overwhelmed with the fast adoption rate of new technologies, Codenomicon’s testing tools have proven to be easily adaptable. Fuzzers do not focus on finding known vulnerabilities, but rather look for new instances of security problems that cannot be discovered with traditional security measures such as antivirus software or firewalls. codenomicon.com/defensics

Situation awareness

Managing critical infrastructure requires making informed decisions quickly. For that, it is essential to acquire relevant information, and understand the relationship between individual events. The problem is that there is too much information available, making it extremely challenging to distinguish significant data from irrelevant pieces of information. Codenomicon's situational awareness tools – Network Analyzer, AbuseHelper and VSRoom – help collecting, filtering and visualising the information in real time, allowing you to make the right decisions. codenomicon.com/clarified Codenomicon is headquartered in Oulu, Finland, with offices in Helsinki, Silicon Valley, Singapore, Hong Kong and Shanghai. The company sells its products directly and through international partners. Codenomicon is privately held with investments from Verdane Capital and Prime Technology Ventures.

Entry Level Price: Euro 10,000

Tutkijantie 4E, 90590 Oulu, Finland T: +358 424 7431 F: +358 8 340 141 E: sales@codenomicon.com W: www.codenomicon.com Contact: Ashley Benn

www.testmagazine.co.uk

December 2011 | TEST Digest

20 Leading Testing Providers

EDgEWoRDS Expert training from Edgewords Specialising in public and on-site training courses in all the HP automated testing tools; QuickTest Professional (QTP), Quality Center and LoadRunner, Edgewords is recognised throughout the software testing industry as one of the leading training providers in the UK. Established in 2003, we were one of the original Mercury Interactive training partners, and almost a decade on, we remain one of the very best.

World Class Training Courses Our training courses have all been carefully created by leading real-world automated testers, enabling us to share over a decade of valuable testing expertise and extensive industry knowledge. All Edgewords' trainers are professionally qualified and deliver the highest quality courses; providing you with the ideal combination of theory and practice.

Competitive Training Costs When you choose Edgewords, you can be confident that you will receive the very best training, and at a highly competitive price.

Locations throughout uK Our public courses are held at dedicated training facilities throughout the UK, in London, Cardiff and the Midlands. A full list of course dates for each location can we found on our website www.edgewords.co.uk .

on-site Training In addition to our public schedule, we can also deliver training on-site at your own premises. This can significantly reduce the training costs if you have four or more delegates. We provide everything required to deliver the courses.

Clients Edgewords has trained over a thousand testers from across all industries, from testing consultancies to global organisations, including bT, HbOS, E.ON, Egg, Llyods TSb. "Just wanted to drop you a line and say a big thank you for delivering an excellent QTP course. My colleague and I were talking afterwards and both agreed it was one of the most useful courses we had attended in a long time! The whole of the three days were a thoroughly enjoyable experience and we both learned a great deal."

"Just wanted to say how pleased I am with some of the things that I have been sent since I attended the Advanced QTP course a while back. The QTP and QC quizzes went down a treat with some of my colleagues recently, and this looks like it could certainly provide help in the future."

9 Wells Close, Bridgnorth, Shropshire, WV16 5JQ T: +44 (0)1746 766246 E: info@edgewords.co.uk W: www.edgewords.co.uk Contact: Tom Millichamp

ExPERiMENTuS LTD Experimentus is an IT solutions and services company, focussed on software quality management.

Working in partnership with our clients to understand their business and technical objectives, our consultants implement solutions designed to: • Accelerate time to market • Prevent software defects rather than detect them • Reduce the overall cost of the software development lifecycle • Meet stakeholder expectations Software quality management spans project management, requirements management, configuration management and testing, and must be closely monitored and managed throughout all phases of the software development lifecycle.

To help our clients we have developed four key solutions aimed at optimising their approach to software quality management.

benefit. Upon completion we can certify your organisation’s capability against one of these global accreditations.

• Trusted Advisory and Delivery Regarded as the thought leaders in the industry, we offer expertise, methodologies and apply best practice techniques with practical experience. Our clients benefit from a pragmatic Consulting service for when they actually need it.

• Process Improvement According to Rummler (1996) an organisation is only as good as its processes. At Experimentus we help our clients develop existing and implement new software quality management processes, which ultimately improve business and IT efficiency.

• Metrication Our metrication solution provides our clients with the expertise to implement robust repeatable qualitative and quantitative measurements that provide the right information to the right people.

• Assessment and Certification Using industry models (TMMi, CMMi and more), we formally assess your approach to quality; we understand the level of maturity and then recommend improvements for implementation, while taking careful consideration of their respective tangible and intangible

These address the ever increasing complex issues surrounding enterprise computing. The solutions provided

by Experimentus allow organisations to focus on their business objectives by ensuring that requirements are translated into first class deliverables. In addition to supporting some of the larger systems integrators, the client base of Experimentus predominantly represents corporations operating in the finance, media, telecommunications and technology sectors.

Number of sites: uK – 1 globally – 1

17a Dorset Square, Marylebone, London, NW1 6Qb, uK T: +44 207 871 2300 F: +44 207 723 4597 E: julian.clarke@experimentus.com W: www.experimentus.com Contact: Julian Clarke

TEST Digest | December 2011

www.testmagazine.co.uk

20 Leading Testing Providers

Experior Experior is a dedicated testing consultancy, continually delivering successful independent SAP testing services across all verticals. Experior is the only SAP Services Partner dedicated to testing.

By combining a risk-based testing approach with a unique delivery methodology, Experior can provide clear and measurable results to customers, including: • Accelerated test execution by up to 80% • Reduced testing investment by as much as 50% • Increased quality, predictable outcomes, service is assured & critical issues are removed • Lower costs, increased quality and reduced timelines allowing businesses to remain agile • Confidence – go-live earlier and with zero critical defects • Flexibility – delivering results across SAP and non-SAP landscapes

Performance Testing – Mitigate risk and avoid expensive downtime

Experior’s performance testing service combines a blend of tools, methods and skilled resource to enhance new or growing systems during and beyond go-live; ultimately striving to maximise the responsiveness and reliability of systems to all business and technology users.

Choosing the right Testing Tools

There are a number of toolsets available today to support your testing activities. Experior works with customers to recommend, implement Test automation reduces cost and improves the efficiency of your test activity and mobilise toolsets while transferring knowledge & best practise to internal teams. without compromising the quality and integrity of your solution. Experior has developed an execution based pricing model to enable customers to start Functional Testing – Deliver change into the business faster realising the benefits of automation – reducing test cycles by up to 87%, with A tailored solution, providing SAP functional test templates specific to your no upfront costs. business requirements, this service provides a robust and consistent approach to validating change and can involve SAP and non-SAP applications.

Upgrade Testing – Zero critical defects on go-live

Remove the cost barrier to Automation

Upgrades can be a highly stressful time for an organisation, Experior is repeatedly assisting customers with upgrades/projects that go live with zero critical defects and reducing SAP upgrade testing times by 50%.

Mobility Testing – Have confidence in your mobile solution

Enterprise mobility solutions will change the way organisations operate.

Exposure to a larger customer base, in a public forum, creates additional risks and reinforces the need for performance testing. Experior works closely with customers to develop strategies for testing the latest mobile technologies.

Number of sites: UK – 25+ client sites Globally – 10+ client sites

Test Strategy & Roadmaps – Strategic roadmap of manageable stages

Need a strategic long-term view of your testing but too tied up with numerous current projects? Experior delivers a consultancy led service advising on successful, enterprise wide test strategies.

Test Management – Deliver UAT faster

Experior’s Test Management approach has been specifically developed for functional testing projects and can reduce your UAT timeline by up to 60%. *All metrics based on previous Experior projects

Entry Level Price: Price on application – dependant on service/landscape/project

T: +44 (0)20 8439 9560 E: info@experiorgroup.com W: www.experiorgroup.com

Facilita Software Development Load testing solutions that deliver results Are your applications and server infrastructure optimised for optimal performance?

Load and performance testing using Facilita Forecast™ shows in advance how your applications, IT systems and websites will perform before they go live.

Reduce risk, optimise performance Comprehensive load testing minimises the risk of system failure, poor application performance and a damaging user experience. Facilita Forecast™ is a powerful load testing tool that genuinely meets the “real world” needs of testers, QA specialists and developers. Forecast™ is used by companies across many business sectors as well as Government and Public Sector organisations to ensure the performance and reliability of their business critical applications and systems.

Facilita Testing Services Load testing is challenging. Facilita provides first-class professional services from fully managed testing engagements through to product mentoring. Utilizing Cloud based servers, Facilita can provide a cost effective tailored performance testing infrastructure.

Number of sites: UK – Many hundreds of clients use Facilita Software and Services throughout the United Kingdom.

Globally – Facilita is increasing its global reach with many new international clients and partnerships.

Entry Level Price: Performance testing solutions to suit all budgets. Please contact enquiries@facilita.com.

Somerford Business Court, Holmes Chapel Road, Congleton. Cheshire. CW12 4SN T: 01252 405468 F: 01260 298335 E: enquiries@facilita.co.uk W: www.facilita.com Contact: Gareth Heaton

www.testmagazine.co.uk

December 2011 | TEST Digest

20 Leading Testing Providers

gREENHAT Experimentus is an IT solutions and services company, focussed on software quality management.

Working in partnership with our clients to understand their business and technical objectives, our consultants implement solutions designed to: • Accelerate time to market • Prevent software defects rather than detect them • Reduce the overall cost of the software development lifecycle • Meet stakeholder expectations Software quality management spans project management, requirements management, configuration management and testing, and must be closely monitored and managed throughout all phases of the software development lifecycle.

To help our clients we have developed four key solutions aimed at optimising their approach to software quality management.

benefit. Upon completion we can certify your organisation’s capability against one of these global accreditations.

• Trusted Advisory and Delivery Regarded as the thought leaders in the industry, we offer expertise, methodologies and apply best practice techniques with practical experience. Our clients benefit from a pragmatic Consulting service for when they actually need it.

• Process Improvement According to Rummler (1996) an organisation is only as good as its processes. At Experimentus we help our clients develop existing and implement new software quality management processes, which ultimately improve business and IT efficiency.

• Metrication Our metrication solution provides our clients with the expertise to implement robust repeatable qualitative and quantitative measurements that provide the right information to the right people.

• Assessment and Certification Using industry models (TMMi, CMMi and more), we formally assess your approach to quality; we understand the level of maturity and then recommend improvements for implementation, while taking careful consideration of their respective tangible and intangible

These address the ever increasing complex issues surrounding enterprise computing. The solutions provided

by Experimentus allow organisations to focus on their business objectives by ensuring that requirements are translated into first class deliverables. In addition to supporting some of the larger systems integrators, the client base of Experimentus predominantly represents corporations operating in the finance, media, telecommunications and technology sectors.

Number of sites: uK – 1 globally – 1

17a Dorset Square, Marylebone, London, NW1 6Qb, uK T: +44 207 871 2300 F: +44 207 723 4597 E: julian.clarke@experimentus.com W: www.experimentus.com Contact: Julian Clarke

LEARNTESTiNg Learntesting is a cloud-based, Virtualized Learning Environment for the Software Testing Industry, supported by a global alliance of testing organisations.

Learntesting provides software testing education solutions for certification and beyond – both theoretical and practical. Learntesting is an ISTQb and bCS Accredited Training Organisation and also provides e-learning courses leading to the IREb (Requirements Engineering) plus classroom courses leading to Certified Agile Tester (CAT) via partners.

With nearly 10 years of e-learning experience, Learntesting brings you the best blended learning solutions of self-study, virtual classrooms and real classrooms – with real tutors. Learntesting features strong support for its students with an integrated Helpdesk, a team of 10 accredited tutors for support, plus many resources including a library of 50 leading software testing and related e-books. Learntesting measured pass rates for testing certification over 10 years are higher than average pass rates for traditional classroom training.

Learntesting is a truly global business and has strategically placed resellers in most major regions in the world. This includes Accredited Training Organizations with their own branded Learntesting distribution portals, providing local support to local clients and part of a network that can support the largest global businesses.

Learntesting has been running classroom-based Testing Academies for 5 years, where ‘Putting the Theory into Action’ is delivered by a ‘hands-on’ approach to the complete testing life-cycle. From Q1 2012, Learntesting will be delivering its Practical Test Academy via self-study and virtual classrooms.

Our Virtualized Learning Environment is highly scalable and can be configured with private courses and multi-branded Portals. These

can be localized for the largest global organizations, including multi-language support.

Our new ‘Freedom Learning’ delivery model extends the flexibility even further: As much content as you want…

For however many people that need it…

For as long as you need it… one Simple Monthly License Fee. • Save 90% on total cost of learning • On-demand, 24×7 access to global testing training content and resources • Replicate classroom benefits • Blend Self Study with Virtual Classrooms and Real Classrooms • Virtualization with real tutors, real time and real benefits • Increase certification pass rates • Value-based licensing – low cost of engagement and implementation • Scalable, global, secure, private & proven Learntesting innovation was recognised by the Institute of IT Training when it was selected as a finalist for the 'Learning Technologies Solution of the year award' in 2010.

117-119 Houndsditch, London, EC3A 7bT E: enquiry@learntesting.com W: www.learntesting.com

TEST Digest | December 2011

www.testmagazine.co.uk

20 Leading Testing Providers

MiCRo FoCuS Borland Solutions from Micro Focus The borland software quality suite of products ensures that quality assurance is embedded throughout the entire development lifecycle from requirements definition to ‘go live’. borland Solutions put the focus on identifying and eliminating defects at the beginning of the process, rather than removing them at the end of development. It provides capabilities across three key areas: Requirements: Caliber®, the requirements definition and management tools, uniquely combine requirements definition, visualization, and management into a single ‘3-Dimensional’ solution. This gives managers, analysts and developers the right level of detail about how software should be engineered. by removing ambiguity in the requirements definition and management process, the direction of the development and QA teams is clear, dramatically reducing the risk of poor business outcomes.

- Streamlined requirements collaboration; - End to end traceability of requirements; - Fast and easy simulation to verify requirements; - Secure, centralized requirements repository.

Change: StarTeam® enables development teams to regain control in their constantly shifting world with a single ‘source of truth’ to prioritise and collaborate on defects, tasks, requirements, test plans, and other in-flux artefacts. Even when software is built by global teams with complex environments and methods, Micro Focus controls change and increases the quality of outputs.

- A single source of key information for distributed teams; - Streamlined collaboration through a unified view of code and change request; - Industry leading scalability combined with low total cost of ownership.

Quality: The Silk™ family automates the entire quality process from inception through to software delivery. Unlike solutions that emphasize ‘back end’ testing, Silk ensures that tests are planned early and synchronised with business goals, even as requirements and realities change.

- Ensures that developed applications are reliable and meet the needs of business users; - Automates the testing process, providing higher quality applications at a lower cost; - Prevents or discovers quality issues early in the development cycle, reducing rework and speeding delivery. bringing the business and end-users into the process early makes business requirements the priority from the outset as software under development and test is continually aligned with the needs of business users. borland solutions provide an open framework which integrates diverse toolsets, teams and environments, giving managers continuous control and visibility over the development process to ensure that quality output is delivered on time. by ensuring correct deliverables, automating test processes, and encouraging reuse and integration, enterprise critical software is continually and efficiently validated.

Number of sites: 18,000 customers and over two million licensed users, including 91 of the Fortune global 100 companies.

The Lawn, 22-30 old bath Road, Newbury, berkshire, Rg14 1QN, uK T: +44 (0)1635 565472 E: anna.wake@microfocus.com W: www.microfocus.com Contact Anna Wake

oDiN TECHNoLogy LTD Axe from Odin Technology is a test automation platform that greatly increases productivity and drastically reduces maintenance overheads for test automation efforts using leading ALM tools and open source tools, like Selenium. Aimed at non-technical testers primarily, Axe uses simple spreadsheets as a means to define and model test scenarios and code generation techniques to rapidly generate robust, self-documented automation code. Axe is a vital addition to any ALM tool line-up.

Number of sites: uK – 49 globally – 77

Entry Level Price: £3000 + Support and Maintenance

Atlantic House, imperial Way, Reading, berkshire, Rg2 0TD T: +44(0)118 903 6101 F: +44(0) 118 903 6100 E: info@odintech.com W: www.odintech.com Contact: David Tracey

TEST Digest |December 2011

www.testmagazine.co.uk

20 Leading Testing Providers

original software Original Software’s innovative approach to testing provides practical solutions for Application Quality Management (AQM) and Automated Software Quality (ASQ).

We help customers achieve real successes by offering an effective ‘application quality eco-system’ that delivers greater business agility, faster time to market, reduced risk, decreased costs and an early return on investment.

Our success has been built on a solution suite that provides a dynamic approach to quality management and automation, empowering all stakeholders in the quality process, as well as uniquely addressing all layers of the application stack.

The Original Software solution embraces the full spectrum of Application Quality Management across a wide range of applications and environments. We partner with customers and help make software quality a business imperative. Our solutions include a quality management platform, manual testing, test automation and test data management software, all delivered with the control of business risk, cost, time and resources in mind. Our test automation solution is particularly suited for testing in an agile environment.

Current users encompass a wide range of industries and include the likes of Coca-Cola, Unilever, Barclays Bank, Cargill, HSBC, FedEx, Pfizer, DHL and many others.

Number of sites: UK – 150 Globally – 300

Entry Level Price: £10,000

Grove House, Chineham Court, Basingstoke, Hampshire RG24 8AG T: +44 1256 338 666 E: press@origsoft.com W: www.origsoft.com Contact: Tulin Pledger

outsource UK ltd Outsource Testing is an operation catering exclusively to the Quality Assurance & Software Testing sector. As a market specialist we have an experienced, dedicated team of consultants singularly focused within the QA & Testing field.

We aim to add value through our consultants who are market specialists, encouraged to gain ISEB Certification, enabling us to better define your business requirements through depth of understanding.

Outsource Testing can assist your organization ease the pressure to search, screen and hire quality software testing resources for your IT projects. Software testing is a specialised area in the recruitment market and we aim to help you find the most qualified resources to fulfil quality assurance and software-testing needs.

Our purpose is to provide expertise in the recruitment of testing specialists.

By combining our experienced recruitment consultants with testing specialists at all stages throughout the placement process, we are able to put forward the most suitable candidates based on your specific requirements.

Specialising in this niche sector of software testing we have a dedicated database of test professionals of circa 21,000

individuals (as at November 2011) as well as a secondary network of testers that is second to none. This enables us to tap into your local geographical area to source the best local and qualified testers to fulfil your needs.

We can source junior testers (newly qualified ISEB/ISTQB) through scripters, test analysts, automated testers, test leaders through to test managers and test programme directors. Whatever the level and discipline within testing, we will be able to help.

However, we also acknowledge that our role is not solely to identify candidates who possess the right skills but also to find the right culture fit, which is crucial to today’s organisations. We believe in working closely with all our clients and really getting to know what makes their businesses tick. We have extremely low turnover (average tenure is currently 6 years) which means that long term relationships are not only possible, they are the norm.

Working with us as a specialist division within Outsource UK Ltd also has the additional benefit of giving you direct access to our colleagues who can support your requirements for non-testing IT professionals. We have been involved in the recruitment sector since 1993 and as such are well placed to help with any of your IT staffing requirements.

Number of sites: We work at over 150 UK based client sites, and have a growing exposure to European clients.

Churchward House, Fire Fly Avenue, Swindon, Wilts. SN2 2EY T: 01793 430021 F: 01793 432337 E: awilliams@outsource-uk.co.uk W: www.outsource-uk.co.uk Contact: Andrea Williams

www.testmagazine.co.uk

December 2011 | TEST Digest

20 Leading Testing Providers

Parasoft Parasoft SOAtest was designed from the ground up to provide testers with an easy way to test at the message layer. Tests are able to be immediately automated with no further work, and in general require NO scripting.

Imagine being able to put together an end to end test that uses a web page to create an account, take the account number and run it through a web service test to generate an order, check that the order exists within the database, and finally check that the order arrives on the warehouse web page for processing, all without scripting!

But a good test does not just rely on the end results. If you are bug hunting then you will want to examine the message queues along the way. SOAtest will allow you to do this, read the contents of JMS or MQ queues or sniff at the traffic in an ESB! Parasoft SOAtest is an integrated solution for: • End-to-end testing: To continuously validate all critical aspects of complex transactions, which may extend beyond the message layer through a web interface, ESBs, databases, and everything in between.

• Environment management: To reduce the complexity of testing in today's heterogeneous environments–with limited visibility/control of distributed components or vendor-specific technologies.

Number of sites: UK – 50+ Globally – 1,000+

•Q uality governance: To continuously measure how each service conforms to the often dynamic expectations defined by both your own organization and your partners.

•P rocess visibility and control: To establish a sustainable workflow that helps the entire team efficiently develop, share, and manage the evolution of quality assets throughout the lifecycle.

Parasoft Load Test allows you to load test your SOAtest tests to verify functionality and performance under load.

Support is also provided for load testing non-Parasoft components such as JUnits or lightweight socket-based components, and for detecting concurrency issues.

Parasoft's customers, including 58% of the Fortune 500, rely on SOAtest and Load Test for: •E nsuring the reliability, security, and compliance of SOA and web applications; •R educing the time and effort required to construct and maintain automated tests; •A utomatically and continuously validating complex business scenarios; • F acilitating testing in incomplete and/or evolving environments; alidating performance and functionality •V expectations under load.

Entry Level Price: £4,168

The Lansdowne Building, 2 Lansdowne Road, Croydon. CR9 2ER T: 0208 263 6005 E: andrew.thompson@parasoft.com W: www.parasoft.com Contact: Andrew Thompson

Pinta Education Limited Does your job include the following?

• Testing and challenging requirements • Designing tests • Setting up the test environment • Making use of automated tools • Raising and managing defects • Creating test strategies and plans • Setting organisational test policies • Deciding on test methodologies and techniques • Effective communications with others • Using IT safely and responsibly • Managing others doing the above.

Would you like a once-and-for-all-time recognition of your skills as a test professional? If yes, then seeking registration as a professional software engineer is for you. Testing as a profession has matured. We can now move on up to the designatory letters after our name. The certificates are useful, but once registered, we will not need to walk around with them – we will just sign our name.

Why? So that your current and future employers will know that you have been formally and rigorously assessed in your profession, by your peers.

What’s the benefit to you as a person? It is that quiet confidence in knowing that you have been tried and

tested, and you passed. If not first time, then the next – this will be your show, in your time and at your convenience.

So, when seeking your next pay rise or next promotion or next company, you will be in the driving seat.

Pinta Education Limited provides courses in software testing and support to those wishing to become registered with the Engineering Council as an ICTTech, IEng or CEng. We offer the BCS/ISEB/ISTQB Foundation and Intermediate courses to those just wanting the short-sharp certificate courses.

Beyond these levels we also offer practical courses designed to develop a well-rounded individual, with registration in mind. As well as testing skills, these include communication skills in testing and legal concepts when using IT, such as data protection and copyright.

To learn more about our courses or registration, please visit us at www.pintaed.com, or email angelina.samaroo@pintaed.com. Angelina is co-author of the ISEB Foundation and Intermediate books on software testing; a Chartered Engineer and a Fellow of the Institution of Engineering and Technology (IET). She can advise on which level of registration you should aim for and mentor you through the process, if required.

Pinta Education Limited looks forward to working with you to raise the recognition of testers worldwide to the highest levels possible.

Trafalgar House, Fullbridge, Maldon, Essex CM9 4LE T: 01344 779842 E: angelina.samaroo@pintaed.com W: www.pintaed.com Contact: Angelina Samaroo

TEST | December 2011

www.testmagazine.co.uk

20 Leading Testing Providers

Seapine Software, Inc. Seapine Software’s application lifecycle management solutions help QA, development, and project teams plan, execute, and manage the product development process to produce highquality solutions. Teams can manage the entire development lifecycle from requirements, through tests and issues, to source code and other digital assets. Integrated reporting and project management enable teams to manage projects of all sizes. QA teams can also automate and accelerate testing, while ensuring application quality, with Seapine’s automated functional and regression testing, and web load testing, solutions.

Seapine’s solutions are especially suitable for regulated, safety-critical, or mission-critical product development where traceability, auditability, and repeatability of processes, activities, and artifacts is critical. By linking requirements or user stories with downstream artifacts, such as test cases, defects, and source code, the solutions enable teams to manage regulatory compliance initiatives. Customizable workflows, with time-stamped approvals and electronic sign-off, provide full traceability of process and repeatability of tasks defined in the workflow.

Instead of defining and implementing a specific software methodology, Seapine lets teams decide the best approach for their project and skill sets, and easily customize the solution for that approach. Seapine offers an exceptional out-of-the-box user experience, combined with an unprecedented ability to configure the solutions to meet team needs. Virtually every

Number of sites: Approximately 8,500 customers globally

screen, term, and activity can be tailored to meet the unique needs of a team and project. Seapine’s solutions also integrate well with tools you may already be using, giving you a more comprehensive solution for managing product quality.

When you're managing a portfolio of projects, you need a reporting platform to provide insight into schedules and team performance across all projects. The Seapine ALM Reporting Platform (Seapine ALM RP), which warehouses data in a central database, gives you and your management team insight into every stage of the development lifecycle, enabling fact-based reviews of each project and team. With Seapine ALM RP, you can review the schedule status of active projects, investigate test plan performance across teams, or analyze bug counts across release cycles, to name a few possibilities.

Whether your team uses Waterfall, Agile, or a hybrid methodology, Seapine’s ALM solution provides the most seamless way to manage product lifecycle processes available today.

Your entire team works from the same information and can make the best decisions for the project as a whole. Contact Seapine to learn how your application quality will benefit from a robust and easy-to-use solution that fits into your project.

Entry Level Price: please contact for pricing information

5412 Courseview Drive, Suite 200, Mason Ohio 45040, USA T: +1 513 754-1655, +44 (0) 208 948 9460 (UK) F: +1 513 754-1660, +44 (0) 208 948 9462 (UK) E: agara@seapine.com W: www.seapine.com Contact: Ashley Agar

SmarteSoft Inc. Total Test Solutions Achieve test success with SmarteSoft's comprehensive automated testing solutions, customised according to your specific business needs.

Whether you follow agile, waterfall or a hybrid approach, SmarteSoft can help. Our industry-leading services and tools for functional and performance testing include: SmarteScript™ – Truly scriptless functional enterprise test automation Test automation is fast and easy to implement and maintain, with no scripting skills needed – even for complex functions. SmarteScript supports data driven testing and supports legacy, client-server, webbased and mobile software testing.

SmarteStudio™ – Flexible, extensible, open-architecture for enterprise software test automation and cross-browser testing SmarteStudio is both an end-user test tool and a test automation platform for GUI-driven testing with an open, extensible architecture.

Get the flexibility to automate where other tools fail. Modify and extend key functions by creating custom plug-in libraries with JavaScript.

SmarteStudio supports desktop, web-based, point-of-sale, legacy and mobile application test and a wide and growing range of environments, including Java, GWT, YUI, AJAX, Infragistics Windows Forms Controls, DevExpress, Adobe Flash/Flex/AIR, Microsoft .NET, Win32 and Windows Presentation Foundation with MAA. SmarteStudio supports eProgesa, HCLL and more.

SmarteStudio has the strongest support in the industry for cross-browser testing: create one test script and execute without modification across all major browsers including Microsoft Internet Explorer, Mozilla Firefox and Google Chrome.

SmarteLoad™ – Enterprise-class performance test Fast, easy to use, and efficient for data correlation; verify your system’s

ability to cope under any scenario. Simulate usage patterns and ramp load up, analysing bottlenecks and optimising performance.

SmarteLoad is highly user-extensible. Add custom protocols via open API structure. No need to buy additional licenses every time you add new protocol support to load test capabilities.

SmarteQM™ – Test lifecycle management: requirements through support Manage project requirements, releases, test cases, issues and tasks in one environment. Take control of project lifecycle and synchronise development and testing teams with flexible configuration options.

Reduce budget with this affordable, enterprise – class TLM tool.

The end result SmarteSoft’s tools provide improved application quality, reduced time to market, and flexibility and extensibility you don’t find in other tools.

Benefit from satisfied customers and improved customer retention, lower support and maintenance costs, improved brand/product reputation and significant competitive advantage.

SmarteSoft community We serve the Federal Government, energy, retail and Insurance industries with a special focus and dedicated solutions for specific healthcare environments. We are dedicated to providing superior customer service. Join the SmarteSoft community; we’ll help you succeed!

Entry Level Price: SmarteStudio $999 USD; SmarteQM $999 USD; SmarteScript $1,795 USD; SmarteX $899 USD;

SmarteTime $1,500 USD; SmarteLoad $499 USD.

4301 Westbank Drive, Building A, Suite 125A, Austin, Texas 78746 T: +1.512.782.9409 F: +1.512. 531-7493 E: gordon.macgregor@smartesoft.com W: www.smartesoft.com Contact: Gordon MacGregor

www.testmagazine.co.uk

December 2011 | TEST Digest

20 Leading Testing Providers

TECHExCEL TechExcel DevSuite provides an integrated platform for development and defect tracking, requirements management, project planning and quality management. With tools to effectively support both agile and traditional processes, DevSuite allows you to manage development your way. Achieve balanced and sustainable development with a solution that truly fits your needs.

Portfolio – Product Portfolio Management Product Portfolio Management tool provides a comprehensive and real-time view of all internal and external processes and resources to enhance the management practices for all of an organization’s project and program portfolios. Integrating PPM across all DevSuite modules bridges the gaps between development teams communicating project status, managers reporting progress, and decision-makers quantifying projects with the greatest business value.

DevSpec – Requirements Management DevSpec enables organization-wide collaboration for the management of requirements, specifications, product ideas or agile stories, with integration points that allow you to drive development and testing directly from completed requirements. Whether you wish to adopt a simple agile process for building a product backlog or need to implement a defined and regulated requirements process, you will quickly achieve full traceability throughout the project lifecycle.

DevPlan – Development Project Planning A better alternative to traditional project management tools, DevPlan provides robust development project planning and resource management. by dynamically linking your project timeline to implementation tracking in DevTrack, you achieve real-time visibility into the current project status, risk areas, burndowns and expected delivery dates.

DevTrack – implementation and Defect Tracking DevTrack comprehensively tracks and manages all aspects of a Number of sites: UK – 60 Globally – 1,500

development project, from feature stories and implementation tasks to product defects and change requests. Teams can effectively plan, organize and execute development work, and with tools to support both agile and traditional practices, DevTrack can easily be configured to fit your needs.

DevTest – QA Test Management From test case creation, planning and execution through defect submission and resolution, DevTest manages the complete quality lifecycle. Implement quality processes earlier in the development lifecycle to manage shorter deadlines, address complex contemporary testing challenges, and improve your deliverable software.

KnowledgeWise – Knowledge base DevSuite includes KnowledgeWise, a central knowledge repository for managing documents, images, Wiki articles, Wiki books, and other digital assets and attachments. KnowledgeWise provides a fully configurable user interface and definable workflow process for tracking knowledge creation, review, publishing, and approval processes, and items in KnowledgeWise can be accessed from all areas of DevSuite.

DevTime – Timesheet Management Adding the optional DevTime module to DevSuite creates a complete time sheet management system that is integrated with development time tracking in DevTrack.

Entry Level Price: £1,500

Crown House, 72 Hammersmith Road, London, W14 8TH T: +44 207 470 5650 F: +44 207 470 5651 E: emeainfo@techexcel.com W: www.techexcel.com

TESTPLANT TestPlant is the developer of the leading user interface test tool eggPlant. eggPlant is a powerful test tool that creates an abstraction of a GUI for any device type, enabling automation of screen-based testing through ‘search and compare’. With a patented innovative approach, eggPlant is non-invasive and a unique and invaluable feature for testers working in defence, security, media, IT, and entertainment. It is currently relied on by more than 500 businesses and organizations, including government agencies, household names, and leading brands.

Through a non-invasive abstraction of the visual screen to be tested, eggPlant’s image recognition algorithm intelligently searches and compares for variances including icons, buttons, message boxes, prompts, texts, background, tones of color, and inconsistencies across screen types, including PCs, touchscreens, touchpads, and mobile screens. eggPlant is technology agnostic, with full browser compatibility and the ability to run in dynamic environments including HTML5, Silverlight, Flex and Flash. It is highly scalable and can be set up across hardware and operating systems in test labs and global networks. eggPlant is user-friendly with an integrated command language, called SenseTalk, that is accessible to all user levels including developers and non-developers. Its detailed reporting captures inconsistencies between visual outputs and pinpoints failures with easy-reference screenshots.

The company has a global footprint with American and European offices in Washington D.C., boulder, CO, and London, with partners across the world. It is backed by Seraphim Capital.

Entry Level Price: £3,800

6 Snow Hill London EC1A 2Ay united Kingdom T: +44 20 7002 7888 F: +44 20 7002 7887 E: info@testplant.com W: www.testplant.com Contact: Richard Ward

TEST Digest | December 2011

www.testmagazine.co.uk

Can you predic TEST company profile | 41

Facilita Load testing solutions that deliver results Facilita has created the Forecast™ product suite which is used across multiple business sectors to performance test applications, websites and IT infrastructures of all sizes and complexity. With this class-leading testing software and unbeatable support and services Facilita will help you ensure that your IT systems are reliable, scalable and tuned for optimal performance.

Forecast, the thinking tester's power tool A sound investment: A good load testing tool is one of the most important IT investments that an organisation can make. The risks and costs associated with inadequate testing are enormous.

Load testing is challenging and without good tools and support will consume expensive resources and waste a great deal of effort.

Forecast has been created to meet the challenges of load testing, now and in the future. The core of the product is tried and trusted and incorporates more than a decade of experience but is designed to evolve in step with advancing technology.

Realistic load testing: Forecast tests the reliability, performance and scalability of IT systems by realistically simulating from one to many thousands of users executing a mix of business processes using individually configurable data.

Comprehensive technology support: Forecast provides one of the widest ranges of protocol support of any load testing tool.

1. Forecast Web thoroughly tests web-based applications and web services, identifies system bottlenecks, improves application quality and optimises network and server infrastructures. Forecast Web supports a comprehensive and growing list of protocols, standards and data formats including HTTP/HTTPS, SOAP, XML, JSON and Ajax.

2. Forecast Java is a powerful and technically advanced solution for load testing Java applications.

It targets any non-GUI client-side Java API with support for all Java remoting technologies including RMI, IIOP, CORBA and Web Services.

3. Forecast Citrix simulates multiple Citrix clients and validates the Citrix environment for scalability and reliability in addition to the performance of the hosted applications. This non-intrusive approach provides very accurate client performance measurements unlike server based solutions.

4. Forecast .NET simulates multiple concurrent users of applications with client-side .NET technology.

5. Forecast WinDriver is a unique solution for performance testing Windows applications that are impossible or uneconomic to test using other methods or where user experience timings are required. WinDriver automates the client user interface and can control from one to many hundreds of concurrent client instances or desktops.

6. Forecast can also target less mainstream technology such as proprietary messaging protocols and systems using the OSI protocol stack.

Powerful yet easy to use: Skilled testers love using Forecast because of the power and flexibility that it provides. Creating working tests is made easy with Forecast's script recording and generation features and the ability to compose complex test scenarios rapidly with a few mouse clicks. The powerful functionality of Forecast ensures that even the most challenging applications can be full tested.

4

G

Facilita Software Development Limited. Tel: +44 (0)12

Supports Waterfall and Agile (and everything in between): Forecast has the features demanded

by QA teams like automatic test script creation, test data management, real-time monitoring and comprehensive charting and reporting.

Forecast is successfully deployed in Agile ‘Test Driven Development’ (TDD) environments and integrates with automated test (continuous build) infrastructures. The functionality of Forecast is fully programmable and test scripts are written in standard languages (Java, C#, C++ etc). Forecast provides the flexibility of open source alternatives along with comprehensive technical support and the features of a high-end enterprise commercial tool.

Flexible licensing: Geographical freedom allows licenses to be moved within an organisation without additional costs. Temporary high concurrency licenses for ‘spike’ testing are available with a sensible pricing model. Licenses can be rented for short term projects with a ‘stop the clock’ agreement or purchased for perpetual use.

Our philosophy is to provide value and to avoid hidden costs. For example, server monitoring and the analysis of server metrics are not separately chargeable items and a license for Web testing includes all supported Web protocols.

Services In addition to comprehensive support and training, Facilita offers mentoring where an experienced Facilita consultant will work closely with the test team either to ‘jump start’ a project or to cultivate advanced testing techniques. Even with Forecast’s outstanding script automation features, scripting is challenging for some applications. Facilita offers a direct scripting service to help clients overcome this problem.

We can advise on all aspects of performance testing and carry out testing either by providing expert consultants or fully managed testing services.

Facilita Tel: +44 (0) 1260 298109 Email: enquiries@facilita.co.uk Web: www.facilita.com

www.testmagazine.co.uk

December 2011 | TEST

42 | TEST company profile

Parasoft Improving productivity by delivering quality as a continuous process For over 20 years Parasoft has been studying how to efficiently create quality computer code.

Our solutions leverage this research to deliver automated quality assurance as a continuous process throughout the SDLC. This promotes strong code foundations, solid functional components, and robust business processes. Whether you are delivering Service-Orientated Architectures (SOA), evolving legacy systems, or improving quality processes – draw on our expertise and award winning products to increase productivity and the quality of your business applications.

Specialised platform support:

Parasoft's full-lifecycle quality platform ensures secure, reliable, compliant business processes. It was built from the ground up to prevent errors involving the integrated components – as well as reduce the complexity of testing in today's distributed, heterogeneous environments.

Trace code execution:

What we do Parasoft's SOA solution allows you to discover and augment expectations around design/ development policy and test case creation. These defined policies are automatically enforced, allowing your development team to prevent errors instead of finding and fixing them later in the cycle.

This significantly increases team productivity and consistency.

End-to-end testing: Continuously validate all critical aspects of complex transactions which may extend through web interfaces, backend services, ESBs, databases, and everything in between.

Advanced web app testing: Guide the team in developing robust, noiseless regression tests for rich and highly-dynamic browserbased applications.

Access and execute tests against a variety of platforms (AmberPoint, HP, IBM, Microsoft, Oracle/ BEA, Progress Sonic, Software AG/webMethods, TIBCO).

Security testing: Prevent security vulnerabilities through penetration testing and execution of complex authentication, encryption, and access control test scenarios.

Provide seamless integration between SOA layers by identifying, isolating, and replaying actions in a multi-layered system.

Continuous regression testing: Validate that business processes continuously meet expectations across multiple layers of heterogeneous systems. This reduces the risk of change and enables rapid and agile responses to business demands.

Multi-layer verification: Ensure that all aspects of the application meet uniform expectations around security, reliability, performance, and maintainability.

Policy enforcement: Provide governance and policy-validation for composite applications in BPM, SOA, and cloud environments to ensure interoperability and consistency across all SOA layers.

Please contact us to arrange either a one to one briefing session or a free evaluation.

Application behavior virtualisation: Automatically emulate the behavior of services, then deploys them across multiple environments – streamlining collaborative development and testing activities. Services can be emulated from functional tests or actual runtime environment data.

Load/performance testing: Verify application performance and functionality under heavy load. Existing end-to-end functional tests are leveraged for load testing, removing the barrier to comprehensive and continuous performance monitoring.

Spirent CommunicationsEmail: plc Tel:sales@parasoft-uk.com +44(0)7834752083 Email: Web: www.spirent.com Web: www.parasoft.com Tel:Daryl.Cornelius@spirent.com +44 (0) 208 263 6005

TEST | December 2011

www.testmagazine.co.uk

TEST company profile | 43

Seapine Software TM

With over 8,500 customers worldwide, Seapine Software Inc is a recognised, award-winning, leading provider of quality-centric application lifecycle management (ALM) solutions. With headquarters in Cincinnati, Ohio and offices in London, Melbourne, and Munich, Seapine is uniquely positioned to directly provide sales, support, and services around the world.

Built on flexible architectures using open standards, Seapine Software’s cross-platform ALM tools support industry best practices, integrate into all popular development environments, and run on Microsoft Windows, Linux, Sun Solaris, and Apple Macintosh platforms.

Seapine Software's integrated software development and testing tools streamline your development and QA processes – improving quality, and saving you significant time and money.

TestTrack RM TestTrack RM centralises requirements management, enabling all stakeholders to stay informed of new requirements, participate in the review process, and understand the impact of changes on their deliverables. Easy to install, use, and maintain, TestTrack RM features comprehensive workflow and process automation, easy customisability, advanced filters and reports, and role-based security. Whether as a standalone tool or part of Seapine’s integrated ALM solution, TestTrack RM helps teams keep development projects on track by facilitating collaboration, automating traceability, and satisfying compliance needs.

TestTrack Pro TestTrack Pro is a powerful, configurable, and easy to use issue management solution that tracks and manages defects, feature requests, change requests, and other work items. Its timesaving communication and reporting features keep team members informed and on schedule. TestTrack Pro supports MS SQL Server, Oracle, and other ODBC databases, and its open interface is easy to integrate into your development and customer support processes.

TestTrack TCM TestTrack TCM, a highly scalable, cross-platform test case management solution, manages all areas of the software testing process including test case creation, scheduling, execution, measurement, and reporting. Easy to install, use, and maintain, TestTrack TCM features comprehensive workflow and process automation, easy customisability, advanced filters and reports, and role-based security. Reporting and graphing tools, along with user-definable data filters, allow you to easily measure the progress and quality of your testing effort.

QA Wizard Pro QA Wizard Pro completely automates the functional and regression testing of Web, Windows, and Java applications, helping quality assurance teams increase test coverage. Featuring a nextgeneration scripting language, QA Wizard Pro includes advanced object searching, smart matching a global application repository, datadriven testing support, validation checkpoints, and built-in debugging. QA Wizard Pro can be used to test popular languages and technologies like C#, VB.NET, C++, Win32, Qt, AJAX, ActiveX, JavaScript, HTML, Delphi, Java, and Infragistics Windows Forms controls.

Surround SCM Surround SCM, Seapine’s cross-platform software configuration management solution, controls access to source files and other development assets, and tracks changes over time. All data is stored in industry-standard relational database management systems for greater security, scalability, data management, and reporting.

Surround SCM’s change automation, caching proxy server, labels, and virtual branching tools streamline parallel development and provide complete control over the software change process.

www.seapine.com Phone:+44 (0) 208-899-6775 Email: salesuk@seapine.com United Kingdom, Ireland, and Benelux: Seapine Software Ltd. Building 3, Chiswick Park, 566 Chiswick High Road, Chiswick, London, W4 5YA UK Americas (Corporate Headquarters): Seapine Software, Inc. 5412 Courseview Drive, Suite 200, Mason, Ohio 45040 USA Phone: 513-754-1655

www.testmagazine.co.uk

December 2011 | TEST

44 | TEST company profile

Micro Focus Deliver better software, faster.

Software quality that matches requirements and testing to business needs.

Making sure that business software delivers precisely what is needed, when it is needed is central to business success. Getting it right first time hinges on properly defined and managed requirements, the right testing and managing change. Get these right and you can expect significant returns: Costs are reduced, productivity increases, time to market is greatly improved and customer satisfaction soars.

The Borland software quality solutions from Micro Focus help software development organizations develop and deliver better applications through closer alignment to business, improved quality and faster, stronger delivery processes – independent of language or platform.

Combining Requirements Definition and Management, Testing and Software Change Management tools, Micro Focus offers an integrated software quality approach that is positioned in the leadership quadrant of Gartner Inc’s Magic Quadrant.

The Borland Solutions from Micro Focus are both platform and language agnostic – so whatever your preferred development environment you can benefit from world class tools to define and manage requirements, test your applications early in the lifecycle, and manage software configuration and change.

Requirements Defining and managing requirements is the bedrock for application development and enhancement.

Micro Focus uniquely combines requirements definition, visualization, and management into a single '3-Dimensional' solution, giving managers, analysts and developers precise detail for engineering their software. By cutting ambiguity, the direction of development and QA teams is clear, strengthening business outcomes.

For one company this delivered an ROI of 6-8 months, 20% increase in project success rates, 30% increase in productivity and a 25% increase in asset re-use.

Using Micro Focus tools to define and manage requirements helps your teams: ollaborate, using pictures to build mindshare, drive •C a common vision and share responsibility with rolebased review and simulations.

•R educe waste by finding and removing errors earlier in the lifecycle, eliminating ambiguity and streamlining communication.

• Improve quality by taking the business need into account when defining the test plan.

Caliber ® is an enterprise software requirements

definition and management suite that facilitates collaboration, impact analysis and communication, enabling software teams to deliver key project milestones with greater speed and accuracy.

Software Change Management StarTeam® is a fully integrated, cost-effective software change and configuration management tool.

Designed for both centralized and geographically distributed software development environments, it delivers: • A single source of key information for distributed teams • Streamlined collaboration through a unified view of code and change requests • Industry leading scalability combined with low total cost of ownership

Testing Automating the entire quality process, from inception through to software delivery, ensures that tests are planned early and synchronize with business goals even as requirements and realities change. Leaving quality assurance to the end of the lifecycle is expensive and wastes improvement opportunities.

Micro Focus delivers a better approach: Highly automated quality tooling built around visual interfaces and reusability. Tests can be run frequently, earlier in the development lifecycle to catch and eliminate defects rapidly.

From functional testing to cloud-based performance testing, Micro Focus tools help you spot and correct defects rapidly across the application portfolio, even for Web 2.0 applications.

Micro Focus testing solutions help you: • Align testing with a clear, shared understanding of business goals focusing test resources where they deliver most value • Increase control through greater visibility over all quality activities • Improve productivity by catching and driving out defects faster Silk is a comprehensive automated software quality management solution suite which enables users to rapidly create test automation, ensuring continuous validation of quality throughout the development lifecycle. Users can move away from manual-testing dominated software lifecycles, to ones where automated tests continually test software for quality and improve time to market.

Take testing to the cloud Users can test and diagnose Internet-facing applications under immense global peak loads on the cloud without having to manage complex infrastructures.

Among other benefits, SilkPerformer ® CloudBurst gives development and quality teams: • Simulation of peak demand loads through onsite and cloud-based resources for scalable, powerful and cost effective peak load testing • Web 2.0 client emulation to test even today’s rich internet applications effectively Micro Focus, a member of the FTSE 250, provides innovative software that enables companies to dramatically improve the business value of their enterprise applications. Micro Focus Enterprise Application Modernization, Testing and Management software enables customers’ business applications to respond rapidly to market changes and embrace modern architectures with reduced cost and risk.

For more information, please visit www.microfocus.com/solutions/softwarequality

TEST | December 2011

www.testmagazine.co.uk

TEST company profile | 45

Original Software Delivering quality through innovation With a world class record of innovation, Original Software offers a solution focused completely on the goal of effective software quality management.

By embracing the full spectrum of Application Quality Management (AQM) across a wide range of applications and environments, we partner with customers and help make quality a business imperative. Our solutions include a quality management platform, manual testing, test automation and test data management software, all delivered with the control of business risk, cost, time and resources in mind. Our test automation solution is particularly suited for testing in an agile environment.

Setting new standards for application quality Managers responsible for quality must be able to implement processes and technology that will support their important business objectives in a pragmatic and achievable way, and without negatively impacting current projects.

These core needs are what inspired Original Software to innovate and provide practical solutions for Application Quality Management (AQM) and Automated Software Quality (ASQ). We have helped customers achieve real successes by implementing an effective ‘application quality eco-system’ that delivers greater business agility, faster time to market, reduced risk, decreased costs, increased productivity and an early return on investment.

Our success has been built on a solution suite that provides a dynamic approach to quality management and automation, empowering all stakeholders in the quality process, as well as uniquely addressing all layers of the application stack.

Automation has been achieved without creating a dependency on specialised skills and by minimising ongoing maintenance burdens.

An innovative approach Innovation is in the DNA at Original Software. Our intuitive solution suite directly tackles application quality issues and helps you achieve the ultimate goal of application excellence.

Empowering all stakeholders The design of the solution helps customers build an ‘application quality eco-system’ that extends beyond just the QA team, reaching all the relevant stakeholders within the business. Our technology enables everyone involved in the delivery of IT projects to participate in the quality process – from the business analyst to the business user and from the developer to the tester. Management executives are fully empowered by having instant visibility of projects underway.

Quality that is truly code-free We have observed the script maintenance and exclusivity problems caused by code-driven automation solutions and has built a solution suite that requires no programming skills. This empowers all users to define and execute their tests without the need to use any kind of code, freeing them from the automation specialist bottleneck. Not only is our technology easy to use, but quality processes are accelerated, allowing for faster delivery of businesscritical projects.

Top to bottom quality Quality needs to be addressed at all layers of the business application. We give you the ability to check every element of an application - from the visual layer, through to the underlying service processes and messages, as well as into the database.

Addressing test data issues Data drives the quality process and as such cannot be ignored. We enable the building and management of a compact test environment from production data quickly and in a data privacy compliant manner, avoiding legal and security risks. We can also manage the state of that data, so that it is synchronised with test scripts, enabling swift recovery and shortening test cycles.

A holistic approach to quality Our integrated solution suite is uniquely positioned to address all the quality needs of an application, regardless of the development methodology used.

Being methodology neutral, we can help in Agile, Waterfall or any other project type. We provide the ability to unite all aspects of the software quality lifecycle. Our solution helps manage the requirements, design, build, test planning and control, test execution, test environment and deployment of business applications from one central point that gives everyone involved a unified view of project status and avoids the release of an application that is not ready for use.

Helping businesses around the world Our innovative approach to solving real pain-points in the Application Quality Life Cycle has been recognised by leading multinational customers and industry analysts alike. In a 2011 report, Ovum stated: “While other companies have diversified, into other test types and sometimes outside testing completely, Original Software has stuck more firmly to a value proposition almost solely around unsolved challenges in functional test automation. It has filled out some yawning gaps and attempted to make test automation more accessible to non-technical testers.”

More than 400 organisations operating in over 30 countries use our solutions and we are proud of partnerships with the likes of Coca-Cola, Unilever, HSBC, Barclays Bank, FedEx, Pfizer, DHL, HMV and many others.

www.origsoft.com Email: solutions@origsoft.com Tel: +44 (0)1256 338 666 Fax: +44 (0)1256 338 678 Grove House, Chineham Court, Basingstoke, Hampshire, RG24 8AG

www.testmagazine.co.uk

December 2011 | TEST

46 | TEST company profile

Green Hat The Green Hat difference In one software suite, Green Hat automates the validation, visualisation and virtualisation of unit, functional, regression, system, simulation, performance and integration testing, as well as performance monitoring. Green Hat offers codefree and adaptable testing from the User Interface (UI) through to back-end services and databases.

Reducing testing time from weeks to minutes, Green Hat customers enjoy rapid payback on their investment.

Green Hat’s testing suite supports quality assurance across the whole lifecycle, and different development methodologies including Agile and test-driven approaches. Industry vertical solutions using protocols like SWIFT, FIX, IATA or HL7 are all simply handled. Unique pre-built quality policies enable governance, and the re-use of test assets promotes high efficiency. Customers experience value quickly through the high usability of Green Hat’s software.

Focusing on minimising manual and repetitive activities, Green Hat works with other application lifecycle management (ALM) technologies to provide customers with value-add solutions that slot into their Agile testing, continuous testing, upgrade assurance, governance and policy compliance.

Enterprises invested in HP and IBM Rational products can simply extend their test and change management processes to the complex test environments managed by Green Hat and get full integration.

Green Hat provides the broadest set of testing capabilities for enterprises with a strategic investment in legacy integration, SOA, BPM, cloud and other component-based environments, reducing the risk and cost associated with defects in processes and applications. The Green Hat difference includes: • Purpose built end-to-end integration testing of complex events, business processes and composite applications. Organisations benefit by having UI testing combined with SOA, BPM and cloud testing in one integrated suite.

• Unrivalled insight into the side-effect impacts of changes made to composite applications and processes, enabling a comprehensive approach to testing that eliminates defects early in the lifecycle.

• Virtualisation for missing or incomplete components to enable system testing at all stages of development. Organisations benefit through being unhindered by unavailable systems or costly access to third party systems, licences or hardware.

Green Hat pioneered ‘stubbing’, and organisations benefit by having virtualisation as an integrated function, rather than a separate product.

• ‘ Out-of the box’ support for over 70 technologies and platforms, as well as transport protocols for industry vertical solutions. Also provided is an application programming interface (API) for testing custom protocols, and integration with UDDI registries/repositories.

•H elping organisations at an early stage of project or integration deployment to build an appropriate testing methodology as part of a wider SOA project methodology.

Corporate overview Since 1996, Green Hat has constantly delivered innovation in test automation. With offices that span North America, Europe and Asia/Pacific, Green Hat’s mission is to simplify the complexity associated with testing, and make processes more efficient.

Green Hat delivers the market leading combined, integrated suite for automated, end-to-end testing of the legacy integration, Service Oriented Architecture (SOA), Business Process Management (BPM) and emerging cloud technologies that run Agile enterprises.

Green Hat partners with global technology companies including HP, IBM, Oracle, SAP, Software AG, and TIBCO to deliver unrivalled breadth and depth of platform support for highly integrated test automation. Green Hat also works closely with the horizontal and vertical practices of global system integrators including Accenture, Atos Origin, CapGemini, Cognizant, CSC, Fujitsu, Infosys, Logica, Sapient, Tata Consulting and Wipro, as well as a significant number of regional and country-specific specialists. Strong partner relationships help deliver on customer initiatives, including testing centres of excellence. Supporting the whole development lifecycle and enabling early and continuous testing, Green Hat’s unique test automation software increases organisational agility, improves process efficiency, assures quality, lowers costs and mitigates risk.

Helping enterprises globally Green Hat is proud to have hundreds of global enterprises as customers, and this number does not include the consulting organisations who are party to many of these installations with their own staff or outsourcing arrangements. Green Hat customers enjoy global support and cite outstanding responsiveness to their current and future requirements. Green Hat’s customers span industry sectors including financial services, telecommunications, retail, transportation, healthcare, government, and energy.

• Scaling out these environments, test automations and virtualisations into the cloud, with seamless integration between Green Hat’s products and leading cloud providers, freeing you from the constraints of real hardware without the administrative overhead.

• ‘Out-of-the-box’ deep integration with all major SOA, enterprise service bus (ESB) platforms, BPM runtime environments, governance products, and application lifecycle management (ALM) products.

sales@greenhat.com www.greenhat.com

TEST | December 2011

www.testmagazine.co.uk

TEST company profile | 47

T-Plan T-Plan since 1990 has supplied the best of breed solutions for testing. The T-Plan method and tools allowing both the business unit manager and the IT manager to: Manage Costs, Reduce Business Risk and Regulate the Process.

By providing order, structure and visibility throughout the development lifecycle from planning to execution, acceleration of the "time to market" for business solutions can be delivered.

The T-Plan Product Suite allows you to manage every aspect of the Testing Process, providing a consistent and structured approach to testing at the project and corporate level.

What we do Test Management: The T-Plan Professional product is modular in design, clearly dierentiating between the Analysis, Design, Management and Monitoring of the Test Assets.

• What coverage back to requirements has been achieved in our testing so far?

Test Automation: Cross-Platform Independence (Java) Test Automation is also integrated into the test suite package via T-Plan Robot, therefore creating a full testing solution.

T-Plan Robot Enterprise is the most flexible and universal black box test automation tool on the market. Providing a human-like approach to software testing of the user interface, and uniquely built on JAVA, Robot performs well in situations where other tools may fail.

• Platform independence (Java).

T-Plan Robot runs on, and automates all major systems, such as Windows, Mac, Linux, Unix, Solaris, and mobile platforms such as Android, iPhone, Windows Mobile, Windows CE, Symbian.

• Test almost ANY system.

As automation runs at the GUI level, via the use of VNC, the tool can automate any application.

E.g. Java, C++/C#, .NET, HTML (web/browser), mobile, command line interfaces; also, applications usually considered impossible to automate like Flash/Flex etc.

• What requirement successes have we achieved so far?

• Can I prove that the system is really tested?

• If we go live now, what are the associated Business Risks?

Incident Management: Errors or queries found during the Test Execution can also be logged and tracked throughout the Testing Lifecycle in the T-Plan Incident Manager.

“We wanted an integrated test management process; T-Plan was very exible and excellent value for money.”

Francesca Kay, Test Manager, Virgin Mobile

Web: hays.co.uk/it Email: testing@hays.com Tel: +44 (0)1273 739272

www.testmagazine.co.uk

December 2011 | TEST

48 | The last word...

the last word...

Step up to the buffet It’s Christmas time and Dave Whalen is at the buffet.

I

love buffets! i can pick what i want in the combinations i want. Sometimes, those combinations only make sense to me. Today’s plate will most likely be completely different than last week’s plate.

Next week’s will probably be different from this week’s. oh some of the same things may be there, the ‘staples’, but others will probably vary. Test tools are a lot like buffets.

I like to maintain a smorgasbord of test tools. I’m constantly adding new ones to the fare along with some favourites, which I tend to pick often.

They’re like the cheesecake of test tools. I have to have it. Everything else usually depends on the situation.

What sounds good today? Maybe I’ll try something new? Maybe I’ll try the apple pie this time? It’s good to have choices.

The latest addition to my test tool buffet is an open source Graphical User Interface (GUI) test tool. Of course, using open source tools has its challenges. It doesn’t do everything I want it to do, but it should allow me to modify it by writing the code needed to support my unique testing situation.

I’m not sure if I like it yet, but I’m keeping an open mind. I’m frustrated by some basic things that it won’t do.

Automatically launching a browser has become a bit of a challenge. Whether I end up selecting and using the tool on my current project or not has yet to be determined. Much will depend on my frustration level and I’m getting pretty frustrated. One of the major frustrations with using any open source tool is the lack of documentation.

In the case of this tool, a new version has been released but the updated user documentation is not yet available. There is also a very small user community. I can typically find a local consultant to help me out but not this time. My next stop – surely someone has written a book on the subject? Nope.

TEST | December 2011

The jury is still out on the tool. So far, I’m not real happy with it. The quest continues.

The majority of my current test efforts are still in the backend. There is no user interface. I don’t have much of a need for a GUI tool right now, but I will soon.

Our goal is 100 percent automation so I need to find something. Now as far as the backend testing goes, I’m pretty happy. Like the GUI tool, we had a variety of open source tools to choose from. This time we selected a tool that I knew rather than trying something new.

This one is much more popular so there is a large and vocal user community. There are even a couple of books available; but they’re terrible.

We have been able to review posts from other users for solutions to most of our problems. As we suspected, others have had similar issues to ones we have uncovered and we have been able to work around most of them thanks to their help. but we are constantly breaking new ground.

Thankfully, I have a really supportive development team at my disposal. I merely have to mention that I need a little help and they are willing to write any code I need. Sure, I could do it myself, but who has the time? There are some blocks of code that are there just for me – the customer will never know they are there. It’s a very team-oriented approach – teamwork at its best – I absolutely love it! back to the buffet to find a performance testing tool, this time a tool that I had never used but I had heard of, a popular tool with a wide user base. We tried it and liked it. It isn’t the easiest tool to use and the user documentation is somewhat sparse, but the user pool is deep. We have been able to solve most problems with a simple Internet search. Other problems were solved by putting our heads together and trying something new. Most of our solutions come from me asking “I wonder what would happen if…”

Even with known tools, we are using most of them in unique and different ways. We are always finding new and better ways to do things. Many of the approaches we take may be new ways to do things. We are more than happy to share what we are doing with the user community. but we all constantly complain about the lack of any real source of documentation.

I mentioned a few days ago that since there are no good books available, that perhaps we should write one. It was a joke but the team has taken it very seriously. They think we should really do it. It may not be a bad idea. but what to call it? I know, Smorgasbord – The Definitive Guide to the Test Tool buffet. Maybe not. I’ll get back to you.

I like to maintain a smorgasbord of test tools.

I’m constantly adding new ones to the fare along with some favourites, which I tend to pick often. They’re like the cheesecake of test tools. I have to have it. Everything else usually depends on the situation.

What sounds good today?

Maybe I’ll try something new? Maybe I’ll try the apple pie this time? It’s good to have choices.

Dave Whalen

President and senior software entomologist Whalen Technologies softwareentomologist.wordpress.com

www.testmagazine.co.uk

INNOVATION FOR SOFTWARE QUALITY

Subscribe to TEST free! e 201 1 Issu e 3: Jun Vol um e 3:

LIT RE QUA

Y

INNOV ATION INNOVATION FOR SOFTWARE QUALITY FOR SO FTWA Volume 3: Issue 4: 2011 3: August June 2011

Vol um e 3: Issu e 5: Oc tob er 201 1

: ide est Ins Dig ST TE

FTWA FOR SO

TEST : INNOV ATION FOR SO FTWAR E QUA LITY

ATION

TEST : INNOVATION FOR SOFTWARE QUALITY

LITY E QUA FTWAR FOR SO ATION INNOV TEST :

INNOV

RE QUA LIT

Y

Apps don’t stand alone Tact & Diploma cy g Banishin Banishing gs u b y it r u security bugs sec Professor Mike Holcombe on fitting good ideas into the world

Devyani Bo rade on developin g the right attitude

VOLUM E 3: IS SUE 5: OCTOB ER 201 1

V O LVUOML U E M3 E: I3S: S IUS ES U4 E: A3 U UN S TE 2 0 1 1 : GJ U

011 JUNE 2 SUE 3: E 3: IS VOLUM

n Ari Takane tisjärvi and g zzinMiia fu Miia Vuon of er Vuontisjärvi and Ari Takanen on the pow on the power of fuzzing

testing t | Safety velopmen Inside: Lo ad testing ication de pl Ap | | testing Test auto Inside: Performance testing | Gettingdevelopment real | The ethics of testing Inside: Risk-based testing | Application | Safety testing mation | k-based Requirem .uk .co Inside: Ris ine ents agaz Visit TES www.testm online at Visit TEST

Visit TEST online at www.testmagazine.co.uk

T online at

www.testm agazine.co .uk

For exclusive news, features, opinion, comment, directory, digital archive and much more visit

www.testmagazine.co.uk Published by 31 Media Ltd Telephone: +44 (0) 870 863 6930 Facsimile: +44 (0) 870 085 8837

www.31media.co.uk

Email: info@31media.co.uk Website: www.31media.co.uk

INNOVATION FOR SOFTWARE QUALITY