TEST Magazine - August-September 2009 by 31 Media

IN TOUCH WITH TECHNOLOGY

T H E E U RO P E A N S O F T WA R E T E S T E R Volume 1: Issue 3: September 2009

BRIDGING THE GAP

James Christie takes the agile approach

Inside: Risk-based testing | Crowd testing | Testing as a service T.E.S.T in now online at www.testmagazineonline.com

Supported by

Test Case Management

Satisfy your quality obsession.

TestTrack® TCM

www.seapine.com/testmag

[

Test Planning & Tracking

TestTrack® Studio

Surround SCM®

Configuration Management

Seapine CM®

Change Management

Automated Testing

QA Wizard® Pro

[

Full-Time Quality Assurance Manager—Immediate Opening

• Achieve complete traceability between test cases and defects with seamless TestTrack Pro integration.

• Ensure all steps are executed, and in the same order, for more consistent testing.

Test Case Management

• Streamline the QA > Fix > Re-test cycle by pushing test failures immediately into the defect management workflow.

• Manage suites of platform-specific compliance tests, functional tests, and performance tests in one central location. • Assign tests to your QA team, track results, and report on performance and workload.

Issue Management

• Know instantly which test cases have been executed, what your coverage is, and how much testing remains.

• Use test variants to target multiple platforms with the same test case for more efficient test case management.

TestTrack® Pro

TestTrack TCM puts you in control of test case planning and tracking, providing better visibility over the testing effort and giving you more time to manage your team. With TestTrack TCM your team can write and manage thousands of test cases, select sets of tests to run against builds, and process the pass/fail results using your development workflow.

TestTrack® TCM

Don’t work yourself to death. Use TestTrack® TCM to manage your testing effort.

www.testmagazineonline.com

Leader | 1

T H E E U RO P E A N S O F T WA R E T E S T E R

IN TOUCH WITH TECHNOLOGY

Supported by

T.E.S.T THE EUROPEAN SOFTWARE TESTER

T H E E U RO P E A N S O F T WA R E T E S T E R Volume 1: Issue 3: September 2009

VOLUME 1: ISSUE 2: JUNE 2009

BRIDGING THE GAP

James Christie takes the agile approach

Inside: Risk-based testing | Crowd testing | Testing as a service T.E.S.T in now online at www.testmagazineonline.com

Editor Matthew Bailey matthew.bailey@31media.co.uk Tel: +44 (0)1293 934464 To advertise contact: Grant Farrell grant.farrell@31media.co.uk Tel: +44(0)1293 934461 Production & Design Dean Cook dean.cook@31media.co.uk Toni Barrington toni.barrington@31media.co.uk Editorial & Advertising Enquiries 31 Media, Crawley Business Centre, Stephenson Way, Crawley, West Sussex, RH10 1TN Tel: +44 (0) 870 863 6930 Fax: +44 (0) 870 085 8837 Email: info@31media.co.uk Web: www.testmagazineonline.com

Introducing T.E.S.T Online

welve months ago we launched The European Software Tester (T.E.S.T) to an expectant industry. The magazine was born into what it’s fair to say were very ‘interesting’ times. The financial sector was still coming to terms with the credit crunch and the global economy was firmly in the grip of the worst recession since World War Two. It will come as no surprise then that we were a little anxious that our newest venture might founder and the magazine would go the way of so many other brave but ultimately doomed periodicals. However, after only a relatively short period of time it became clear that T.E.S.T was not only desired by a largely voiceless industry, but was in extremely high demand from all quarters. Subscriptions came in thick and fast from Europe of course, but also from the Americas, India, and Australasia. Editorial content was submitted at such a rate it made my head spin – clearly the industry needed an outlet and cometh the hour, cometh the magazine!

Printed by Pensord, Tram Road, Pontllanfraith, Blackwood. NP12 2YA © 2009 31 Media Limited.All rights reserved. T.E.S.T Magazine is edited, designed, and published by 31 Media Limited. No part of T.E.S.T Magazine may be reproduced, transmitted, stored electronically, distributed, or copied, in whole or part without the prior written consent of the publisher. A reprint service is available.

This is now the fourth issue of T.E.S.T and we have covered an awful lot of ground over the last year. From agile to outsourcing, from risk-based testing to the black art of masking, obfuscating and scrambling, throughout we have strived to maintain the business-focus that we believe sets T.E.S.T apart from other titles and services that are focussed on the technicalities of testing. While this is undoubtedly very important, it is our business-led approach that we believe the industry not only wants but requires at the present time. The next step is to take the project a stage further. Our aim has always been to publish T.E.S.T in as many formats as possible so with that in mind we are delighted to announce that as well as the quarterly printed magazines, T.E.S.T Online has now well and truly gone live. So for the latest news; in-depth thought leadership features; case studies; cutting edge opinion pieces; best practice and strategy articles; industry events news; vendor listings; digital archive; PDF versions of past issues; and a whole lot more, much of it exclusive to the website, you can now visit: www.testmagazineonline.com We hope you enjoy the site and very much look forward to any comments or feedback you may have. Happy surfing...

Opinions expressed in this journal do not necessarily reflect those of the editor or T.E.S.T Magazine or its publisher, 31 Media Limited. ISSN 2040-0160

Published by:

Matt Bailey, Editor

Our aim has always been to publish T.E.S.T in as many formats as possible so with that in mind we are delighted to announce that as well as the quarterly printed magazines, T.E.S.T Online has now well and truly gone live at: www.testmagazineonline.com Matt Bailey, Editor

www.testmagazineonline.com

September 09 | T.E.S.T

In Touch With Technology

SUBSCRIBE TO T.E.S.T. Sponsored by

HNOL WITH TEC IN TOUCH

PEA THE EURO

AR N SOFTW

OGY

E TESTER

risk Handling the g -based testin ht with Risk Getting it rig ile; The ering with ag Inside: Deliv

the QA g; Anarchy in tware testin future of sof

Simply visit www.testmagazine.co.uk/subscribe Or email subscriptions@testmagazine.co.uk *Please

note that subscription rates vary depending on geographical location

Published by 31 Media Ltd Telephone: +44 (0) 870 863 6930 Facsimile: +44 (0) 870 085 8837

www.31media.co.uk T.E.S.T | September 09

Email: info@31media.co.uk Website: www.31media.co.uk

The European Software Tester www.testmagazineonline.com

Contents | 3

CONTENTS SEP 09

Leader column

Cover story – Bridging the gap with agile

Editor Matt Bailey announces the arrival of the all new T.E.S.T Online website.

Independent consultant James Christie looks at the sometimes troubled relationship between usability designers and software engineers and sees agile as a possible bridge between the two.

Use the best to test

John Lunt discusses the importance of software testing, the possible pitfalls and challenges of outsourcing, and how to conduct it in-house.

A risky business

Why does testing SAP need business process expertise and why has risk-based process testing become a necessity? Matt Pickin explains.

Seven books that shaped my career

Test consultant Phil Kirkham champions the cause of literacy on a journey through the

seven books that have shaped his career.

The sound of the crowd

Testing bloggers Andréas Prins and Ewald Roodenrijs discus the emerging phenomenon of crowd testing and its implications for the testing industry.

When requirements go bad

In the first of a two-part series, Kurt Bittner shows how understanding the causes for requirements errors can help prevent them.

A name to bank on

Santander Consumer Bank AS in Norway is weathering the financial storm remarkably well. Its services live and die by the functionality of their IT. T.E.S.T editor Matt Bailey spoke to the bank’s Nordic region CTO Geir Berglind about its software testing activities.

STaaS in their eyes

Many companies will tackle today’s economic challenges by deploying new software, but how can they be sure it won’t let them down? Richard Terry explains how software testing as a service (STaaS) could help.

Welcome to the new BCS

David Clarke, chief executive of the BCS, explains the changes that have taken place in the organisation, the direction in which it is now heading and the challenges that lie ahead.

T.E.S.T Directory

48 The Last Word – Joel Montvelisky Joel Montvelisky searches his homeland of Costa Rica for some signs of a global testing community and decides we’re not quite there yet.

www.testmagazineonline.com

36 September 09 | T.E.S.T

4 | Test cover story

or most of its history software engineering has had great difficulty building applications that users found enjoyable. Far too many applications were frustrating and wasted users' time. That has slowly changed with the arrival of web developments, and I expect the spread of agile development to improve matters further. I'm going to explain why I think developers have had difficulty building usable applications and why user interaction designers have struggled to get their ideas across. For simplicity I'll just refer to these designers and psychologists as UX. There are a host of labels and acronyms I could have used, and it really wouldn't have helped in a short article.

Why software engineering didn't get UX

Bridging the gap with agile Independent software testing consultant James Christie looks at the sometimes troubled relationship between usability designers and software engineers and sees agile as a possible bridge between the two.

T.E.S.T | September 09

Software engineering's problems with usability go back to its roots, when geeks produced applications for fellow geeks. Gradually applications spread from the labs to commerce and government, but crucially users were wage slaves who had no say in whether they would use the application. If they hated it they could always find another job! Gradually applications spread into the general population until the arrival of the Internet meant that anyone might be a user. Now it really mattered if users hated your application. They would be gone for good, into the arms of the competition. Software engineering had great difficulty coming to terms with this. The methods it had traditionally used were poison to usability. The Waterfall lifecycle was particularly damaging. The Waterfall had two massive flaws. At its root was the implicit assumption www.testmagazineonline.com

Test cover story | 5

that you can tackle the requirements and design up front, before the build starts. This led to the second problem; iteration was effectively discouraged. Users cannot know what they want till they've seen what is possible and what can work. In particular, UX needs iteration to let analysts and users build on their understanding of what is required. The Waterfall meant that users could not see and feel what an application was like until acceptance testing at the end when it was too late to correct defects that could be dismissed as cosmetic. The Waterfall was a project management approach to development, a means of keeping control, not building good products. This made perfect sense to organisations who wanted tight contracts and low costs. The desire to keep control and make development a more predictable process explained the damaging attempt to turn software engineering into a profession akin to civil engineering. So developers were sentenced to 20 years hard labour with structured methodologies, painstakingly creating an avalanche of documentation; moving from the current physical system, through the current logical system to a future logical system and finally a future physical system. However, the guilty secret of software engineering was that translating requirements into a design wasn't just a difficult task that required a methodical approach; it's a fundamental problem for developers. It's not a problem specific to usability requirements, and it was never resolved in traditional techniques. The mass of documentation www.testmagazineonline.com

Why UX didn't get software development Not surprisingly, given the massive documentation overhead of traditional techniques, and developers' propensity to pragmatically tailor and trim formal methods, the full process was seldom followed. What actually happened was more informal and opaque to outsiders. September 09 | T.E.S.T

6 | Test cover story

The UX profession understandably had great difficulty working out what was happening. Sadly they didn't even realise that they didn't get it. They were hampered by their naivety, their misplaced sense of the importance of UX and their counter-productive instinct to retain their independence from developers. If developers had traditionally viewed functionality as a matter of what the organisation required, UX went to the other extreme and saw functionality as being about the individual user. Developers ignored the human aspect, but UX ignored commercial realities – always a fast track to irrelevance. UX took software engineering at face value, tailoring techniques to fit what they thought should be happening rather than the reality. They blithely accepted the damaging concept that usability was all about the interface; that the interface was separate from the application. This separability concept was flawed on three grounds. Conceptually it was wrong. It ignored the fact that the user experience depends on how the whole application works, not just the interface. Architecturally it was wrong. Detailed technical design decisions can have a huge impact on the users. Finally separability was wrong organisationally. It left the UX profession stranded on the margins, in a ghetto, available for consultation at the end of the project, and then ignored when their findings were inconvenient. An astonishing amount of research and effort went into justifying this fallacy, but the real reason UX bought the idea was that it seemed to liberate T.E.S.T | September 09

them from software engineering. Developers could work on the boring guts of the application while UX designed a nice interface that could be bolted on at the end, ready for testing. However, this illusory freedom actually meant isolation and impotence. The fallacy of separation encouraged reliance on usability testing at the end of the project, on summative testing to reveal defects that wouldn't be fixed, rather than formative testing to stop these defects being built in the first place. There's an argument that there's no such thing as effective usability testing. If it takes place at the end it's too late to be effective. If it's effective then it's done iteratively during design, and it's just good design rather than testing. So UX must be hooked into the development process. It must take place early enough to allow alternative designs to be evaluated. Users must therefore be involved early and often. Many people in UX accept this completely, though the separability fallacy is still alive and well. However, its days are surely numbered. I believe, and hope, that the agile movement will finally kill it.

Agile and UX – the perfect marriage? The mutual attraction of agile and UX isn't simply a case of ‘my enemy's enemy is my friend’. Certainly they do have a common enemy in the Waterfall, but each discipline really does need the other. Agile gives UX the chance to hook into development, at the points where it needs to be involved to be effective. Sure, with the Waterfall it is possible to smuggle effective UX

If developers had traditionally viewed functionality as a matter of what the organisation required, UX went to the other extreme and saw functionality as being about the individual user. Developers ignored the human aspect, but UX ignored commercial realities – always a fast track to irrelevance.

www.testmagazineonline.com

Test cover story | 7

There will still be tension between UX and software professionals who've been used to working remotely from the users. However, agile should mean that this is a creative tension, with each group supporting and restraining the others. This is a great opportunity. Testers will get the chance to help create great applications, rather than great documentation!

techniques into a project, but they go against the grain. It takes strong, clear-sighted project managers and test managers to make them work. The schedule and political pressures on these managers to stop wasting time iterating and to sign off each stage is huge. If UX needs agile, then equally agile needs UX if it is to deliver high quality applications. The opening principle of the Agile Manifesto states that â&#x20AC;&#x2DC;Our highest priority is to satisfy the customer through early and continuous delivery of valuable softwareâ&#x20AC;&#x2122;. There is nothing in agile that necessarily guarantees better usability, but if practitioners believe in that principle then they have to take UX seriously and use UX professionals to interpret users' real needs and desires. This is particularly important with web applications when developers don't have direct access to the end users. There was considerable mutual suspicion between the two communities when agile first appeared. Agile was wary of UX's detailed analyses of the users, and suspected that this was a Waterfall-style big, up-front requirements phase. UX meanwhile saw agile as a technical solution to a social, organisational problem and was rightly sceptical of claims that manual testing would be a thing of the past. Automated testing of the human interaction with the application is a contradiction. Both sides have taken note of these criticisms. Many in UX now see the value in speeding up the user analysis and focussing on the most important user groups, and agile has recognised www.testmagazineonline.com

the value of up-front analysis to help them understand the users. The agile community is also taking a more rounded view of testing, and how UX can fit into the development process. In particular check out Crispin and Gregory's four agile testing quadrants. UX straddles quadrants two and three. Q2 contains the up-front analysis to shape the product, and Q3 contains the evaluation of the results. Both require manual testing, and use such classic UX tools as personas (fictional representative users) and wireframes (sketches of webpages). Other people who're working on the boundary of agile and UX are Jeff Patton, Jared Spool and Anders Ramsay. They've come up with some great ideas, and it's well worth checking them out. Microsoft has also developed an interesting technique called the RITE method. This is an exciting field. Agile changes the role of testers and requires them to learn new skills. The integration of UX into agile will make testing even more varied and interesting. There will still be tension between UX and software professionals who've been used to working remotely from the users. However, agile should mean that this is a creative tension, with each group supporting and restraining the others. This is a great opportunity. Testers will get the chance to help create great applications, rather than great documentation! James Christie's comments about the relationship between developers and testers will appear at: www. testmagazineonline.com in October.

James Christie Independent software testing consultant www.clarotesting.com

September 09 | T.E.S.T

8 | Outsourcing testing

Use the best to test Managing director and co-founder of IT asset management specialist Certero, John Lunt, discusses the importance of software testing, the possible pitfalls and challenges of outsourcing, and how to conduct it in-house.

here is no point in companies investing large amounts of money in developing software when it can potentially cause costly problems in the future. Testing is a vital part of the software development lifecycle and ensures that once installed it is error-free. Over-looking this important stage from the outset can seriously affect sales, customer satisfaction, the brand name, and potentially increase the workload to fix unscheduled problems. Although testing does take time and effort, it

T.E.S.T | September 09

reduces any unnecessary risk and cost in the future. In fact, fixing a problem is often more expensive if left until the product has gone to market. Prevention is better that cure! Software Quality Assurance (SQA) is incredibly important to ensure products are designed and developed to a high quality that meets regulatory standards. A company that ignores this, runs the risk of non-tested software failing. As customers look at ways to modernise their back-office operations and IT infrastructure to support their growth, they look for www.testmagazineonline.com

Outsourcing testing | 9

deadlines are met and product releases take place on time. When deciding to outsource testing, businesses need to determine how project delays affect their bottom line profits compared to the costs involved.

The pitfalls

Although testing does take time and effort, it reduces any unnecessary risk and cost in the future. In fact, fixing a problem is often more expensive if left until the product has gone to market. Prevention is better that cure! software that is reliable and fully compliant without risks attached. Products below quality assurance levels can put customers off and force them to look elsewhere for a solution. It is easy for an organisation to gain a negative reputation for providing unreliable software products and the hazards of launching a product that is not fully prepared can be catastrophic and cause irreversible damage.

Outside help Companies increasingly hire third parties to carry out testing on their behalf, particularly in tough financial times. Outsourcing is an attractive option as it means companies do not have the burden or costs associated with employing an in-house testing team or investing in internal testing technology. Third parties offer access to a skilled team of specialist testers who have the ability and knowledge to use specially designed softwaretesting tools. Improved turnaround times and reduced software delays ensure www.testmagazineonline.com

Although an enticing option, outsourcing does have its problems. Outsourcing needs constant management, which can be extremely time consuming and, in general, is the most common reason why outsourced projects fail. If a company does decide to outsource testing, it needs to dedicate a manager to be responsible for the relationship. This manager needs to have the capacity to be in constant communication with the supplier in order to build a close working relationship and effectively manage the project. In some instances, outsourcing can compromise the quality of testing, where a company can be left feeling dissatisfied. To avoid this, and ensure quality testing is delivered and continuous product improvements are met, the manager needs to check that the outsourced company understands the testing requirements at every level. There are offshore software testing companies all around the world, including the UK, US and technologyrich countries such as Japan, India and Singapore. Geographical locations can create further frustrations when managing a relationship that spans from one country to another. This can increase workloads and can be more time consuming than working with someone in the same country. Time differences can put additional strain on the relationship and cause delays, especially when a problem needs resolving quickly. Constant communication can combat this and by arranging scheduled telephone meetings every day or week, issues can be identified and resolved in a timely manner. September 09 | T.E.S.T

10 | Outsourcing testing

In-house: the belt and braces approach Conducted comprehensively, in-house testing should ensure quality and standards are met and should provide value for money. Not having an outside relationship to manage can also lead to a happier workforce. If companies decide to carry out their software testing in-house, they need to embrace a ‘belt and braces’ approach to guarantee quality assurance. This involves a strict and uncompromising method - once a product, or an enhancement to an existing product, is made, then it needs to be tested by a separate testing function, not by the development function. After passing all set criteria, the product should be driven through an external beta testing programme where companies can use a selected number of existing customers to identify any performance issues or bugs prior to an official product release. Software should never be released without beta testing, as it is impossible for developers to test their own products in all environments or predict how a new product will react with particular software and hardware. Only when this process is complete, should the software be made available for general release. There is the perception that it is hard for companies to be critical of their own developments. However, T.E.S.T | September 09

having two separate teams in place – one to write the software and one to conduct testing – means that products are examined exhaustively. Testing technicians that employ scripts and plans to check every software function, guarantee systematic analysis and precise examination.

Some considerations Outsourcing: Do not outsource just because it is a financially attractive option. Think about what you want to get out of it before entering into a partnership with anyone.

Essential testing Software testing is an integral part of the development system and is not an afterthought. Whether a company decides to conduct software testing in-house or outsource to an external tester depends on its needs and resources. Software development companies should assess what they aim to get out of testing to help them evaluate the costs involved and help them determine what approach to take. Regardless of which route an organisation decides upon, software testing is essential. It enhances the quality of the product offering and ultimately, guarantees customer satisfaction. What is the point of developing something that is unreliable and may cause future headaches? Software testing reduces unnecessary risks for the long term and ensures that companies are offering solutions they can be proud of. Producing a product that fails and is unreliable not only leaves you red faced, but it can damage your brand immensely.

Sourcing oversees help: How would you manage a long distance relationship from one country to another? Do you have someone that can dedicate time to managing this relationship?

John Lunt Managing director Certero www.certero.co.uk

Keeping it in-house: Remember to use a method that ensures you are critical of your own development and find those all important beta testers!

www.testmagazineonline.com

The Whole Story Print Digital Online

For exclusive news, features, opinion, comment, directory, digital archive and much more visit

www.testmagazineonline.com www.31media.co.uk

www.testmagazineonline.com

The European Software Tester September 09 | T.E.S.T

12 | Risk-based testing

A risky business Matt Pickin principle consultant at Experior explains why testing SAP requires business process expertise and why risk-based process testing has become a necessity.

t’s becoming more and more apparent that IT and the business need to be better aligned if companies want to deliver successful business services and achieve competitive advantage. With the right approach and methodology, testing will help a business achieve its objectives - going live ‘on time’ and ‘to budget’. In addition, if aligned correctly with the business, testing should drive improvements in business activities by ensuring the system delivered by the IT department is truly supporting the heart of the business. Companies that T.E.S.T | September 09

adopt a risk-based process approach to their testing can see their total project timelines shorted by up to 25 percent leading to cost savings, internal resource liberation – driving both business agility and competitive advantage. With this being the case it’s surprising that so few companies have recognised the role testing plays in achieving these business outcomes and still focus efforts on testing down at the most granular of levels ie, unit testing and basic integration tests. The reality within an SAP landscape, as within the majority of www.testmagazineonline.com

Risk-based testing | 13

other application landscapes, is that if you are not testing your business processes then you are wasting time, effort and resource, while not properly protecting against failure and performance issues. In today’s economic climate, no organisation can afford to do this. Testing is still seen by most people as costly and time consuming. This is hardly surprising when you consider the number of companies that have delayed go-live dates on multiple occasions due to failures identified in testing. Others have had to increase the spend associated with testing by up to three times original estimates due to Gantt Chart driven test plans or suffer stabilisation periods as long as two years because of poor application quality. Testing is supposed to remove these challenges for organisations, not add to them. Companies frequently ask, “What should we test?” Surely, this basic question should be easily answered by every tester; you need to test what presents the business with the most risk, what truly matters to the business. Knowing how to determine this true understanding of business risk can only come from an understanding of the business processes. This will enable alignment of testing activities to critical business processes and drive efficiencies into the test cycle. This seems like common sense, but still you can witness approaches such as, 1) attempting to test everything 2) testing only what is allowed by project timelines, or 3) testing only the system, technology & code and not taking into account reality - the business processes.

Why testing has failed to meet the challenge of Business Process Testing Born as a discipline of IT, testing is a victim of its own heritage and, from the start became overly focused on tools. The roles and skills required by testers along with the importance of intelligent testing methodologies have been neglected. Experior has a testing methodology that combines business process and analysis knowledge, with SAP Systems knowledge and testing expertise – ensuring that testing not only aids the business in reaching its objectives, but also achieves ROI faster. www.testmagazineonline.com

In the past, businesses deploying ERP solutions demanded heavily bespoke systems leaving IT (and subsequently testing) focused on resolving technical integration issues. Removed from the business and skilled in test tool functionality, testers were compromised by their limited knowledge of processes, applications or the dependencies behind the data they were testing. The approach was inevitably orientated around technology. In testing SAP, business processes were over-looked in preference to individual transactions. The focus was largely by SAP module, with a limited appreciation of how each business process could span multiple modules or how processes had the potential to affect seemingly disparate ends of an organisation. Testers, focused on the merits of their test tools, lacked a knowledge or appreciation of business goals and could not deliver the predictable outcomes sought by the business. Today we have to recognise that the key to unlocking the value in testing is through early involvement, understanding business processes and the potential knock-on effects that exist within companies’ unique business frameworks. In addition to possessing business understanding and acumen, testing experts need to understand how SAP works; the dependencies behind the data, its enterprise-wide integration and the technical challenges presented by NetWeaver and SOA. Only then will an individual’s technical understanding of test tools be of maximum value to testing SAP and the business. This blend of skill is not common but the rewards are well worth the effort.

What is risk-based process testing? One of the common mistakes is to take a blanket approach and trying to test everything. However, this is never an option; to replicate every variation of every activity that a business conducts in a test phase would always be too costly. A riskbased approach is not just desirable or the latest big idea, but a necessity. When you can’t test everything deciding what to test is crucial. Once you have a thorough understanding of business and business process, only then can you

The reality within an SAP landscape, as within the majority of other application landscapes, is that if you are not testing your business processes then you are wasting time, effort and resource, while not properly protecting against failure and performance issues. In today’s economic climate, no organisation can afford to do this. September 09 | T.E.S.T

14 | Risk-based testing

Today we have to recognise that the key to unlocking the value in testing is through early involvement, understanding business processes and the potential knock-on effects that exist within companies’ unique business frameworks.

have a true understanding of risk. Risk comes in the form of both a business impact, the impact of the system no longer supporting a particular route through the business process and the technical risk of any part of the system no longer working as specified. Combining these two elements together gives you the true picture of what presents the most risk to your business. Risk-based process testing (RBPT) gives you the systems and processes to understand and manage your business risk combining the best of breed tools - IDS Scheer’s market leading ARIS Business Architect for SAP and ARIS Test Scenario Composer. Underpinned by, SAP Solution Manager and SAP Quality Center by HP with Experior’s SAP testing methodology.

How does risk-based process testing work? Business process is at the core of RBPT starting with a definition of your T.E.S.T | September 09

business activities in SAP Solution Manager. This information is passed through to ARIS Business Architect for SAP where your business processes are mapped and enriched. Your business processes are then passed back into Solution Manager where your system is built to support your business processes. This should be related to standard solution design and system development project activity. ARIS Test Scenario Composer can then be used to examine your business process maps and generate both your test requirements and your end-toend process tests to be loaded as the foundation stones for the risk-based test approach in SAP Quality Center by HP. Test collateral is then built in Quality Center and linked to the business requirements. In Quality Center you then complete your risk assessment - updating test requirements with both business impact and technical risks. When changes are made to your solution, for example by installing a

support or enhancement pack, SAP’s business process change analyser tool in Solution Manager will highlight the test requirements that are impacted. In Quality Center you then use the release functionality to group these impacted test requirements. The risk analysis in Quality Center is then used to determine the scope of your testing. The industry-leading test management functionality in Quality Center can not only be used to execute your testing but to raise and manage defects too. This information can then be passed through to Solution Manager to give your project, development and test Leads the same view of the test information, creating a single version of the truth.

Not just another IT initiative The requirement to test should no longer be seen as just another IT initiative, it should be seen for its true value in the context of delivering continual business process www.testmagazineonline.com

Risk-based testing | 15

improvement. Indeed, testing should be structured and viewed as accelerating project lifecycles and reducing costs – rather than as a beleaguered must-have that only adds to the expenditure and delay of projects. An effective testing approach validates the value of testing, aligning it with other corporate initiatives that continually enhance an organisation’s competitive advantage. Whether it is technically-driven tests to optimise performance, ‘keep the lights on’ release packs or qualitative-driven exercises such as UAT, all testing must be designed with specific reference to validating and improving business process and mitigating risk. Businesses should start using methods that allow business analysts to reuse and automate test assets expediting test cycles while maintaining high levels of accuracy and consistency. When you adopt best practice methods such as these in

automation you can free-up resources formerly distracted with the mundane requisites of testing, allowing dedicated attention to the less certain, more critical areas of change within the business. Risk based process testing solves common issues while mitigating risk, lowering costs, speeding up timelines, achieving faster ROI and reducing TCO.

Matt Pickin Principle Consultant Experior www.experiorgroup.com

EuroSTAR 2009 is Europe’s Largest Software Testing Conference & Expo! It is currently in it’s 17th year and includes an incredible line up of inspirational testing experts & the very latest insights in world-class testing. Visit the website for full programme details.

STOCKHOLM, 30 NOV- 03 DEC

The theme for EuroSTAR 2009 is...

‘Testing for Real, Testing for Now‘ Send us your suggestion for the EuroSTAR 2010 theme to be in with a chance to win one of 2 FREE Conference places to Stockholm later this year! Enter online before September 25th at...

EuroSTAR 2009 Are you ‘Testing for Real?’ Can you afford noT to be there?

www.eurostarconferences.com/competitions/enter-testmag.aspx Book before September 25th to receive a 10% discount on conference fees. Register your test team now to avail of great Group Discounts! EuroSTAR Online Community coming soon – Join the debate! Forum, Blog, Facebook, Archives, Resources and much more…

w w w. e u r o s t a r c o n f e r e n c e s . c o m www.testmagazineonline.com

September 09 | T.E.S.T

16 | Testing literature

Seven books that shaped my career “The man who doesn't read good books has no advantage over the man who can't read them.” Test consultant Phil Kirkham ponders the wisdom of Mark Twain and wishes he’d bought shares in Amazon as he lists the seven books that have shaped his career.

have always loved books, the 'Amazon recommends' feature always sucks me in and my wish list is now four pages long. Lee Copeland in his Nine Forgettings talk bemoans the fact that the majority of testers haven't read a book on testing and with the recent increase in blog and online communities (hello Software Testing Club members!) and the Kindle from Amazon then maybe books are dying. Or maybe not, there is something about the feel of a book in your hands and the look of them on a bookshelf that is still very appealling. A recent blog post by Michael Lopp (aka Rands in Repose, the author of Managing Humans), The Book Staker1, outlined his love of books and how he didn't really trust someone until he had seen their bookshelf. He also admitted to having seven precious books. As a great bibliophile, I loved reading it and it gave me the idea to list the seven books that have influenced me most.

the now 30-year old dog-eared copy and keep finding it cropping up when reading testing sites and books. In the book, Pirsig explores the meaning and concepts of ‘quality’ – impossible to measure but its presence or absence is what makes something good or bad. The quest for quality (hopefully not taken the Pirsig’s obsessive lengths) and how to measure it is an obvious one for all of us in this field - and so I think there should be the equivalent of Godwin’s Law – call it Pirsig’s Law – that states that a discussion about quality is over when someone drags out a quote from this book.

2. In Search of Excellence Lessons from America's BestRun Companies by Tom Peters and Robert Waterman

Full of stories about how the top companies became top and what they do to stay there, how their management works and how they treat their employees. Being reasonably good at maths I knew the possibility of a lottery win and 1. Zen and the Art of retirement to a desert island was very Motorcycle Maintenance low so that meant I would be working by Robert M Pirsig for most of my life so why not try to I first read it when I was 14, just get something from it. Plenty of tales getting interested in motorbikes in the book of wanting to produce an and found this copy on my brother’s excellent product, an appreciation for bookshelf. I was disappointed to find it wasn't really about motorcycles and the customer and attention to the small details – isn't that a job spec for I didn't really understand it but there was just something about it. I still have a tester? T.E.S.T | September 09

The quest for quality (hopefully not taken the Pirsig’s obsessive lengths) and how to measure it is an obvious one for all of us in this field - and so I think there should be the equivalent of Godwin’s Law – call it Pirsig’s Law – that states that a discussion about quality is over when someone drags out a quote from this book. www.testmagazineonline.com

Test literature | 17

The book can be an interesting read now to see if the companies mentioned in it are still there and still being excellent.

3. Testing Computer Software, 2nd Edition by Cem Kaner, Jack Falk, and Hung Q. Nguyen The first testing book I read when I was thinking of moving from writing bugs to finding them. It gave me a huge shock as I realised that what we had called testing was nothing more than keyboard bashing. It gave me an idea of the complexity and potential that the testing field offered and was the push that led to me changing from being a developer who wrote defects to a being a tester who found them. My copy is now looking a bit sorry for itself as this is the book I usually hand out to newbie testers.

4. Everyday Scripting with Ruby: for Teams, Testers, and You by Brian Marick I was investigating Watir to try out some automation testing and this led me to looking at Ruby and hence this book. It showed me how automation could be more than just automating the program under test and could be used for other mundane monotonous tasks. Ruby is a fun language to play around with and writing and running programs – even small quick ones – is a good reminder about what developers do and how easily mistakes can be introduced into a program.

5. Lean Software Development: An Agile Toolkit by Mary and Tom Poppendieck This got me thinking about agile and how wasteful some projects I had worked on had been. For example, two and a half years working with a team of top-notch developers to produce a program that sold less than 100 copies – which would be OK if each copy sold for a million pounds but this was supposed to be a shrink-wrapped mass production program. This led me into finding out more about agile and Scrum which have grown tremendously since I first read this book. After reading this book I went on to learn about the Toyota www.testmagazineonline.com

September 09 | T.E.S.T

18 | Testing literature

Production System, the Theory of Constraints and I am currently reading up on one of the latest buzzes, Kanban. The arguments still continue on mailing lists and blog posts about whether producing a computer program is like engineering or art, can we apply lessons from building bridges or learn from how drama companies produce a play?

6. The Secrets of Consulting by Jerry Weinberg No list of test/QA books could be complete without a Weinberg book. All seven of my selections here could have been his books and the man himself is a fine example – he is in his 70s and still writing and still enthusiastic and keen to share his knowledge and experience. I settled for this one as I was moving into the consulting field and so needed to know about the Law of Raspberry Jam, the Orange Juice Test, Prescott's Pickle Principle and The Jiggler Role. As you might be able to tell from that list, the book is not a checklist of actions to be done – it makes you think about yourself, interactions with people and the role of a consultant.

7. Bridging the Communication Gap by Gojko Azdic Skillsmatter offers free courses and I attended one about agile testing presented by Antony Marcano. There is usually a small audience at these talks so it’s easy to get to know other people and I found myself outside a pub chatting with Antony and a T.E.S.T | September 09

guy called Gojko Azdic. I'd heard of him through his work on Fitnesse, we exchanged email addresses and a while later I was one of several people helping review his latest work – Bridging the Communication Gap. An excellent read and I was flattered to find my name on the credit and a quote from me at the start. This has given me some inspiration for thinking about possibly writing a book myself – and this article could be the first small step. The book itself has been getting (quite rightly) excellent reviews and explains how specification by example and agile acceptance testing work and the problems they attempt to solve. As you can see from the photo, my bookcase has far more than seven books in but the books listed above are the ones that have really made me think, been an influence on me – and now potentially others as I lend out my copies (I have become known as 'The Librarian' at work). This article might now influence others. I'll have to keep an eye on the Amazon sales ranking to see if there is a spike of sales in any of the books listed! Phil Kirkham's comment, War! What is it good for? Absolutely nothin’! appears on the new T.E.S.T Online website at www.testmagazineonline.com. www.randsinrepose.com/archives/2009/08/08/ the_book_stalker.html 1

Phil Kirkham Test consultant www.acutest.co.uk

No list of test/QA books could be complete without a Weinberg book. All seven of my selections here could have been his books and the man himself is a fine example – he is in his 70s and still writing and still enthusiastic and keen to share his knowledge and experience. www.testmagazineonline.com

www.testmagazineonline.com

September 09 | T.E.S.T

20 | Crowd testing

The sound of the crowd Testing bloggers AndrĂŠas Prins and Ewald Roodenrijs discuss the new phenomenon of crowd testing and share their vision of the way traditional testing is changing and how testing companies will fit into this developing picture.

owadays we see huge changes in the world around us, borders are fading and the economy is working around the clock. Much of this has been driven by the Internet and its inherent advantages. The changes also influence our private lives â&#x20AC;&#x201C; just think about the online communities where you participate and the social networks and the blogs you use for personal broadcasting - and have an impact on how we organise our work and the work around us. T.E.S.T | September 09

And all this will change the way we test! Now we test based on risks, test coverage and/or budget, but if we deliver our testing activities to a crowd, they will test it for us and if they accept it and give their agreement the job is done! The blurring of boundaries in our private life will have an impact on the boundaries we nowadays have between companies. A lot of people already use search engines and blogs to gain information, but formally we still do our work at our companies. Have you ever thought about how you www.testmagazineonline.com

Crowd testing | 21

will work in the future? What will it be like? And what will you do when the boundaries finally disappear?

The crowd

An important part of crowd testing is the crowd itself and its makeup. Indeed the crowd is the success formula of crowd testing. So it is important to Crowd testing have the crowd you or the client wants What exactly is crowd testing and for testing the application. So who what does it do? Crowd testing is a is in the crowd? It consists of many form of crowd sourcing where you different people who want to test. use both known and unknown people These people can be testers (beginners, – the ‘crowd’ – to test software. In experts and specialists), end users, other words crowd testing is the use or just interested people or potential of a crowd of people and knowledge users. It consists of your audience and when testing software applications if necessary paid testers. or systems. An important difference When determining the use of the compared to traditional testing is that crowd it is important to know what the test is focused on accepting the software instead of finding defects and they can or cannot do, why they has a different reward compared to the want to test, what is their knowledge or background and origin. If this current models. information is needed it needs to be A crowd consists of people all over available. This can be done by adding the world from different background, cultures and companies. This potential this information, and more, to a ‘crowd atlas’ which details who your crowd is group of billions can test for you and your client! So everybody in the world and what it can do; so your crowd is a could potentially be part of the crowd. known quantity. How do you build up this crowd? So others will test the software we This is perhaps the most difficult task. testers normally test? What happens The test crowd can be built up using to my work? existing networks like communities If done properly, crowd testing around crowd sourcing, around specific shouldn’t be undertaken on software topics like OWASP or around networks that hasn’t already been tested. It like LinkedIn and can include all who should be ready for acceptance. The are interested in a particular kind of professional testers need to have software. These people can be friends, executed their system tests and can family, co-workers, competitors, or in fact go on to be participants in the anybody else you may know. And crowd if they wish. www.testmagazineonline.com

A crowd consists of people all over the world from different background, cultures and companies. This potential group of billions can test for you and your client! So everybody in the world could potentially be part of the crowd. September 09 | T.E.S.T

22 | Crowd testing

they – the crowd – are organised in a crowd atlas.

The reasons for crowd testing After answering the question ‘who is your crowd’, another question comes up: ‘Why do companies want to use the crowd for testing?’ The quick answer is to reduce costs. A crowd tester can be everybody and you don’t have to pay them as an expert (further down this article we’ll explain what the reward for the tester can be). When you use the crowd to test you don’t have to pay expensive experts or buy expensive tooling and you can leave your in-company end users to do the work they’re paid to. But most of the time reducing costs isn’t the most important argument; with crowd testing the focus is on accepting the application, so it allows you to interact with the market and your audience. The reason for a company to go for crowd testing is made up of three parts: company, people and product. And the interaction between these three parts is crucial. It is the interaction between company, people and product that makes crowd testing so desirable. Figure 3 shows the major reasons for crowd testing, but there are probably many more.

Activities for the crowd Now we know why we should use crowd testing, the next step is to find out what tests can be done. As previously stated, tests are not focused on the maximum coverage or finding defects but on accepting the software. From this perspective there are many tests that can be done. First, where do you position crowd testing in the V-model or other development lifecycle models? Crowd testing is focused on accepting the product and therefore it is easy to position it as an acceptance test. When executing system or development tests it is much more difficult to use a crowd. It is, however, possible to work with a crowd for development or system tests, but T.E.S.T | September 09

because of the accepting of the product you shouldn’t release a product that is full of bugs when crowd testing. When a product is still full of bugs, acceptation by the crowd will be less than anticipated. Of course there can still be defects, but the application does needs to work. When this isn’t the case, word can get round that the application is bad and full of bugs. That isn’t good marketing for the software and acceptance will be a long way off. Activities that can be executed by the crowd include: • Testing where a lot of feedback is required; • Testing with no sensitive data of functionality; • Testing multi-platform software; • Testing large end-to-end systems over multiple companies; • Testing games; • Testing beta software. Testing on multi-platforms is an activity that will become more and more important over the coming years. People are getting more flexible and have more software running. As a result there is a greater requirement for applications that can work on more than one platform, especially platforms like mobile phones. End-to-end testing over multiple companies is a relatively new idea. Testing across companies brings many challenges, including the use of end users. The crowd can be build around the existing end users and they can reduce the challenge to get everybody involved on time at the same moment. When you have a dedicated crowd - with the right reward - the challenges are reduced and everybody is working towards the same goal.

fig.3 Reasons for crowd testing

fig.4 multi-platform testing with mobile phones

A rewarding job There has to be a specific reward available for your dedicated testers out in the crowd, it is an important motivator to get them testing. As a company you have to answer the following question: Why will this crowd test for me? There are people who want to try out a new application or new game. www.testmagazineonline.com

Crowd testing | 23

In this case the games industry has an advantage over other software producers. People are more likely to try out games compared to administrative software etc. But with the right reward people can also be motivated to test business and admin applications. When people want, or need, to use specific software they will tend to help you with it. A good example is the beta testing of Windows or Google Apps. One of the motivations for people testing this software is that they want a good application in the end. Everybody ideally wants to work for money. But if you compare crowd testing with crowd sourcing for example you see that the rewards can be, but are not always, non-financial. Rewards include: • Being part of a social network. Because the (social) network needs to use an application, people in the network are pleased to test the software for the whole group. • Enjoyment of the tests. Testers don’t always do those tests that are fun; they do the tests that are needed. But it can be fun to execute tests that you think are most www.testmagazineonline.com

•

rewarding to you. So in the crowd you can do your own personal tests. Income per test. A tester in the crowd can be paid. They personally execute the tests and get money for it. Interaction with different cultures. Some people like to interact with different cultures. When the crowd expands globally, multiple cultures will be present. It can be rewarding to learn from them. Get a fee if a product is sold. Potential users test a product and when the product goes on the market they receive a fee for each product sold. Competition to be the best. When people want to be the best, working in a crowd is an excellent place to show their skills. With the friendly competition in the crowd it is possible to prove your talent. There is a star rating for the best testers in the competition and by executing more difficult tests their rating increases. Companies can ask the best testers with the highest rating to execute their tests for the best results.

Maybe the most valuable role for a test provider is to manage the crowd atlas which is linked to the rating system as described earlier. The test provider can use the atlas to see who has the capabilities and is available for a test. If you have a clear overview of the participants in the crowd then you have added value for the client. September 09 | T.E.S.T

24 | Crowd testing

fig.5 Rewards diagram

• S haring knowledge. People love to share their knowledge. They are glad to share what they know and receive knowledge back from others in return. • Charity. Charity is another strong motivation for people to test for free. This can be with charitable organisations, but also as a gesture to programmers. Think about open source applications or freeware. • Discount on software’s release. When a commercial product is finished, its testers receive a discount on that product. • Vouchers. Testing a product and receiving vouchers to buy other products cheaper or for free from the same company can be a reward. This can also be linked with receiving the product for free after a certain test effort. As a result it can be said that the reward for crowd testing can be a financial, but also non-financial, focused on being a member of a community or personal honours or developing own skills and doing something you enjoy. These rewards can be set in a diagram where the honour and financial rewards can be set.

Control of the test

fig.6 Control crowd testing

T.E.S.T | September 09

After all these things are taken into account, a new question comes up. For which sorts of software is crowd testing suitable? The answer is almost all, but there can be a problem with the sensitivity of the tested software. It can be difficult to convince a manager to let an unknown person test sensitive software. But this problem can be addressed by how you set out your crowd. As mentioned earlier, the crowd can consist of a range of different often unknown people, but also known people. For example the crowd can comprise only the end users of the program. But, clearly, for classified software crowd testing is not a sensible option. Classified software or content needs a high level of control, for example, it is not a good idea to use a crowd to test a national security system; this should be done with screened testers using traditional testing methods. But perhaps the most important part of crowd testing is the control of what is tested. With traditional testing

an analysis of what are the most important parts that need to be tested most thoroughly is done first. These parts are equally important in the crowd test so there should be a log of what is and isn’t tested available. There is clearly a need to control what is tested. Control is implemented by tracking the tests or the presence of individual testers in the software. By doing this a client and/or test manager can exert a high degree of control on the test. Where certain important parts of the software are receiving less attention and other parts which are less important are tested thoroughly there should be ways to divert the crowd to the important parts. The best way to do this is by removing the attractive, but unimportant part of the software altogether and attracting more attention to the important bits – there should be an option to ‘remove’ parts of the software without disrupting the application itself. To achieve all this there needs to be a way of tracking and analysing the behaviour of the crowd within the application.

The conditions for crowd testing Now you know who the crowd is; why a company may want to use a crowd; why the crowd wants to test; what to do to control the test; and how IT companies can use the crowd, the next step is to do it! To use crowd testing there are conditions that need to be met to get the full potential. First, the crowd needs to be established. If there is no crowd available this should be addressed first. When there is a crowd and crowd atlas available, it needs to be determined which testers should be contacted to help with the testing. This is when a specific specialty or group is needed; for example, for security testing the client needs the best security testers in the crowd. Of course, other testers can also participate. Contacting them will be done later, when the software is almost ready for testing. When the software is ready for testing a few prerequisites need to be met. One of these conditions has already been mentioned, the application needs to be divided into separate components which can work independently of one other. The reason for this is that parts of the software www.testmagazineonline.com

Crowd testing | 25

will need to be removed from and inserted into certain parts of the application in order to control what needs to be tested and how the flow of the test proceeds. An additional prerequisite is accessibility to the testing environment. Testers can come from around the world, working with every available system (if needed), so the testing environment should be accessible through the Internet which is the best option for all devices and applications to connect. The system under test should also work well enough as it stands so that a crowd test can proceed. Crowd testing is about accepting the application and when the system works with a lot of (technical) faults the base for acceptation will be low. The crowd will also be less motivated to test the application if it is full of bugs. Thus there should be a pre-test on the test environment to prove that the application will run well enough for acceptance.

Therefore a third party can arrange the contacts and divide the work in the network model. The testing provider The future for must know the crowd and the entities testing companies who participate in the crowd. The question at the end of this article Other profitable jobs for the test is: how can existing testing companies provider could be the management earn their money with crowd testing? It is obviously unacceptable for them to of the test infrastructure, offering post their personnel for free in a crowd tooling for the registration of the crowd, and defect management, all of just for honour or charity. Before we which can be managed from a central can answer the question we need to explain what makes testing companies position. The advantage of managing this all from a central position is that it special: allows you to make reports and status • They have their own test updates for the client. methodologies; Managing the star rating system • They know their clients and the – giving the best tester the highest requirements they have for testing; • Test experts know everything about rating – however, is not a task for the test provider. The rating needs to be the right test strategy and test done by the crowd itself or by the coverage; client. Ratings are awarded when the • Test communities have years of experience with defect management, client is satisfied with the work of the test crowd. Making this ratings system test infrastructure, end-to-end available to the client can eventually be testing, etc. In a nutshell, testing companies have a task for the test provider though. Maybe the most valuable role for a to provide a crowd testing service to test provider is to manage the crowd their clients when they ask to use a atlas which is linked to the rating crowd. With this role they can earn system as described earlier. The test their money! provider can use the atlas to see who The test provider can be a link has the capabilities and is available for between the current hierarchic organisations and the crowd. Currently, a test. If you have a clear overview of the participants in the crowd then you organisations aren’t aware of how the have added value for the client. network model of the crowd works Andréas and Ewald’s blog can be and don`t have the contacts from the found at: http://testingthefuture.net hierarchic model in the network. www.testmagazineonline.com

fig.7 Tasks for the test service provider

Ewald Roodenrijs Testing blogger http://testingthefuture.net

Andréas Prins Testing blogger http://testingthefuture.net

September 09 | T.E.S.T

26 | Testing requirements

When requirements go bad In the first of a two-part series, Kurt Bittner, chief technology officer at Ivar Jacobsen International shows how understanding the causes for requirements errors can help prevent them.

hat error in requirements is a contributing cause to the failure of many projects is nothing new. It is easy to attribute the problem to lack of clarity in the requirements - that if only the ambiguities could be eliminated all would be well. Ambiguous, unclear or incomplete requirements are serious problems, to be sure, but they are not the sole source of requirements errors. Requirements errors have many sources, however, and ambiguous or unclear requirements are only part of the problem. In order to reduce requirements errors, it's worth exploring a number of common types of errors to find their source and strategies for dealing with them. For the purposes of discussion, I find it useful to group requirements errors into three major categories:

T.E.S.T | September 09

- errors of conception - errors of specification - errors of implementation Each one of these types of errors has separate symptoms and causes, and each must be solved in a different way.

What is a requirement, really? Before talking about requirements errors, it's worth pausing for a moment to consider what requirements are. A requirement is a statement that describes a desirable, or sometimes a mandatory, aspect of a solution. The degree to which the requirement is mandatory is often the subject of lengthy scoping debates. The observation that a requirement describes a solution is important, but at first this statement probably seems innocuous and perhaps even, well, obvious. The importance of recognising a requirement as

In the extreme case they may actually be contradictory, much like the old logic teaser: The following statement is true. The previous statement is false. Taken individually, each of these statements is clear and unambiguous, but taken together they are nonsensical. It is easily possible for requirements to exhibit the same qualities. www.testmagazineonline.com

Testing requirements | 27

describing a solution comes from the possibility that the solution that the person writing the requirement has in mind may not actually be the only solution, let alone the best solution, to the problem at hand. If more than one person is writing requirements it is possible that each person actually has a slightly different mental picture of the solution, which means that even if individual requirements are clear and unambiguous, taken as a whole the requirements may not be consistent with one another. In the extreme case they may actually be contradictory, much like the old logic teaser: The following statement is true. The previous statement is false. Taken individually, each of these statements is clear and unambiguous, but taken together they are nonsensical. It is easily possible for requirements to exhibit the same qualities. Some teams try to look at requirements as a complete and unambiguous definition of the solution to be delivered. In today's world where a premium is typically placed on speedy delivery of a "good enough" solution, this perspective is often impractical. It is virtually impossible to specify every interesting aspect of a solution; just as it is impossible to completely and precisely describe even the simplest of chemical compounds, it is also impossible to completely and precisely describe every interesting aspect of something as complex as a software application. Even if it were possible to do, it would be prohibitively expensive to do so. This leads to something more pragmatic: the requirements of a solution are the means by which the stakeholders and the development team arrive at a shared understanding of what it is the project is trying to www.testmagazineonline.com

do. The requirements themselves can be useful in facilitating interesting discussions and documenting understanding, but the important thing is the communication that occurs, not the requirements themselves. The view that requirements are a complete and unambiguous definition of the solution derives from the use of requirements as a kind of contract. This may be appropriate in cases where an external firm is being hired to build the system, but there are typically contractual mechanisms for accomplishing this. Contracts in legal settings are most useful as establishing the terms for lawsuits. In the context of software development, if you have to resort to ‘contractual commitments’ the project has typically failed and the participants are simply trying to lay blame. It is better to work more proactively to achieve success, using the requirements as a communication vehicle rather than as a contractual document. In this respect not all requirements have to be stated in an absolutely context-free and unambiguous manner. As long as there is a consensus about what is needed, some requirements can be less formal and open to creative alternatives, whereas some other requirements must be absolute. As an example, the exact look and feel of the user interface often presents a great opportunity for creative innovation, but the financial calculations performed by a billing system are absolute and not open to creative interpretation. Treating all requirements with the same rigor is a waste of time and may squelch creativity. The key is to understand what must be precise and what can be less formal, and make

In the context of software development, if you have to resort to ‘contractual commitments’ the project has typically failed and the participants are simply trying to lay blame. It is better to work more proactively to achieve success, using the requirements as a communication vehicle rather than as a contractual document. September 09 | T.E.S.T

28 | Testing requirements

sure that everyone understands the decision criteria. Armed with this perspective, we can now turn our attention to different ways in which this communication can break down.

actively communicating the wrong information. The usual source of the problem is talking to the wrong people, not finding the right subject matter expert with the right perspective Errors of conception to define the right solution. A story Errors of Conception occur when the about a failed project illustrates the requirement is poorly conceived, when point: A customer service system it solves the wrong problem or is based was developed and deployed at on flawed assumptions about the great expense. The leader of the solution. These types of errors usually requirements effort was very keen to occur because the objectives of the use the latest windowing technologies, solution are poorly understood. When with lots of sophisticated capabilities these errors are present, the resulting to provide expert advice on how to requirements many be unambiguous handle different kinds of inquiries. The and clear, but nevertheless wrong. As system was released to great fanfare a result of these errors, even when and was, surprisingly, a complete the implementation is sound and of failure. high quality, the resulting solution is It turned out that the customer of little or no value. It is in cases such service representatives who used the as this where resorting to the defence system were accustomed to using an that “the requirements were satisfied” old CRT system that had keystrokeis of little consolation. Symptoms of accelerator commands for frequently errors of conceptions include: performed actions. In addition, the - lack of clarity in the solution goals or staff were extensively trained on the problem being solved; customer scenarios and had years of - unused or unneeded features; industry experience and did not need - unneeded complexity. or welcome the intrusive guidance There have been some interesting on how to do their jobs. As a result studies that suggest that this is of all the extra mouse-clicks and actually a fairly large problem. These distractions of the new system, call studies have shown that up to 50 handling throughput actually declined percent of consumer electronics and the old system was eventually product returns are due to complexity turned back on. - users could not figure out how to use What went wrong here? Basically, the products (but presumably wanted the ‘expert’ was solving problems that the functionality that the product did not need to be solved and had provided). The important thing about ignored the real needs of the intended this is that significant effort was users. Because the requirements expended to create the capabilities dictated the ‘wrong’ solution and were that people were unable to use, or not questioned, the development team perhaps did not even want. built the wrong solution. A couple of Other data from the Standish things could have been done to avoid Group's Chaos Report suggests that this problem: even for successful projects, up to 30 - First, find the right subject matter percent of the function they delivered expert. Since the development team were unneeded or unused. Clearly rarely has the necessary business someone is not clear about what is subject matter expertise, finding the really needed in these solutions. right person is essential. In a sense, you should do enough investigation of the business problem to have a sense Root causes and when purported experts really know avoidance strategies what they are talking about. It would be easy, but wrong, to write - Second, get more than one opinion. these errors off as simply due to poor There should be a consensus among communication. Poor communication the stakeholders about the nature is always present, but in most cases of the problem being solved, and as of errors of conception, someone is T.E.S.T | September 09

www.testmagazineonline.com

Testing requirements | 29

the definition of the solution evolves you should continuously validate the "concept" of the solution with the stakeholders. - Third, this continuous validation can't occur if you are heads-down writing requirements documents. You need to capture the ideas simply and succinctly and communicate them frequently. An old adage to remember is that users don't know what they want, but they know what they don't want when they see it. This means that you will probably have to develop a number of different prototypes to walk-through various aspects of the system in order to make sure that you are building the right solution. Other sources for errors of conception relate to poor understanding of the needs of the business (or sponsoring organisation if the solution is being developed for a not-for-profit organisation or government agency), which really means the needs of the stakeholders for the project. By this I mean going beyond what they say they need and uncovering the real needs that often lie hidden. An example from another domain illustrates the difference: A patient walks into a doctor's office saying that he needs to get a prescription for painkillers. When the doctor asks why, the patient complains of severe headaches, but insists that it's no big deal, he just needs to get something to dull the pain and he doesn't need more tests or treatments. Against these protestations the doctor insists on more tests and eventually finds that the patient has an operable but benign tumour. After the operation, the patient's headaches disappear completely and he makes a speedy recovery. In this example, giving the patient what he is asking for is exactly the wrong course of treatment. Similarly, if you give your users exactly what they ask for you may simply be treating the symptoms of a problem and fail to find a better solution to the real problem at hand. Users are not very good at coming up with solutions to their problems – that's where you can add real value – and letting them ‘dictate’ www.testmagazineonline.com

a solution by dictating requirements usually ends up with everyone less than satisfied. There are a variety of techniques to breaking through the initial requirements posed by the users, but they all centre on focusing on uncovering the needs of the stakeholders and not, at least initially, soliciting requirements from them. Eventually you will uncover requirements, and as you do you need to relate them directly back to needs; if the connection to needs is weak then eliminate the requirement - it's either not needed (or there is a need that you are not clear about). A final problem can occur that is a kind of ‘error of conception’: sometimes the problem to be solved or the needs are clear, but it's just not a problem that is worth solving. This situation occurs when the cost of developing a solution outweighs the benefits, or the solution is simply technically infeasible. It's best to figure this out early, and so focussing early on identifying needs and then exploring the cost and technical feasibility of solutions through one or more prototypes is the best way to figure this out quickly.

Studies have shown that up to 50 percent of consumer electronics product returns are due to complexity users could not figure out how to use the products (but presumably wanted the functionality that the product provided). The important thing about this is that significant effort was expended to create the capabilities that people were unable to use, or perhaps did not even want.

Disambiguation Errors of conception are the most significant kind of requirement errors, and they are the hardest to detect, but avoiding them has the greatest benefit. By preventing them you ensure that the solution will better meet the real needs of stakeholders; failing to prevent them, no amount of ‘disambiguation’ of requirements language will yield a good result. No amount of precision can help a requirement that specifies the wrong behaviour. In this first article in a two-part series, I have covered the ‘errors of conception’, which relate to requirements errors that arise from problems with the basic conception of the solution. In the next issue I will discuss ‘errors of specification’, which relate to errors in how the requirement is expressed, as well as ‘errors of implementation’, which arise from problems in how the requirements are implemented.

Kurt Bittner Chief Technology Officer Ivar Jacobsen International

www.ivarjacobson.com

September 09 | T.E.S.T

30 | Customer profile

A name to bank on Santander Consumer Bank AS in Norway is weathering the financial storm remarkably well. Like all financial institutions in the modern world, its services fundamentally live and die by the functionality of their IT. T.E.S.T editor Matt Bailey spoke to the bank’s Nordic region CTO Geir Berglind about its software testing activities.

ew would deny that the banks have been having a rocky time of it recently. There can’t be many of the larger global financial institutions that have been left untouched by the travails of the credit crunch and the fallout from the global economic downturn. And despite the proverbial ‘green shoots’ of recovery that may or may not be starting to sprout as I write, many in the industry are still wary of announcing a slowdown in these tough times, let alone any more permanent good news. One institution that has weathered the storm remarkably well however is Santander. Indeed, in 2008 when many a once rock-solid name in the banking world was going under, it became the third largest international bank in the world based on profits. With over 14,000 branches (more than any other international bank) in more than 40 countries and over 90 million

T.E.S.T | September 09

customers, Santander grew at an astonishing rate.

Banking on technology In a world where money is more often than not ‘virtual’, existing only as code inside the electronic brains of increasingly complex financial systems, it’s no exaggeration to say that the role of the software is critical. To emphasise just how critical, take a moment to recall the panic that surrounded the ‘millennium bug’ and its implications a decade ago. Now, factor in how far things have come since then and you will get an idea of the scale of the challenges faced by the testing professionals inside large financial organisations. Geir Berglind, CTO of Santander Consumer Bank AS (SCB), took time out of his busy globetrotting schedule to tell us a little about the company and its software testing activities. “We are a part of Grupo Santander

Take a moment to recall the panic that surrounded the ‘millennium bug’ and its implications a decade ago. Now, factor in how far things have come since then and you will get an idea of the scale of the challenges faced by the testing professionals inside large financial organisations. www.testmagazineonline.com

Customer profile | 31

which is one of the largest banks in the world,” he said. “SCB Nordic, which covers Norway, Sweden, Finland and Denmark, has the aim of becoming the leading Nordic car, leisure and consumer finance company. In fact, our goal is to be one of the leading consumer finance companies anywhere in the world. As a step towards achieving this strategy, we have recently concentrated on broadening our product and service range as well as expanding our operations in all of the Nordic countries. In 2007, we established companies in Finland and Denmark providing a full range of Santander Consumer Bank’s products and services.” Berglind’s role as CTO gives him responsibility for much of the group’s IT infrastructure. The role of Seapine Software’s testing solutions in the organisation has been crucial since before he joined the company. “I www.testmagazineonline.com

am the Nordic CTO and this gives me responsibility for architectural guidance, infrastructure, tools and so on,” he explained. “Actually Seapine products were already present when I started working in SCB two years ago. We are currently using TestTrack Pro, Surround SCM, TestTrack TCM, and QA Wizard Pro from Seapine.” As an application lifecycle management (ALM) solution provider, Seapine offers many tools to support the application development lifecycle. Specifically, it offers QA Wizard Pro for automated functional and regression testing, Surround SCM for software change management, TestTrack RM for requirements management, TestTrack Pro for issue and development process management, and TestTrack TCM for test case planning and management. It also offers multiple levels of training for all products, custom software development, data conversion, and process consulting services.

Over the past 14 years Seapine has grown from its initial two employees to over 100, and has added offices in London, Munich, and Melbourne. With all of this growth, the company says it has remained faithful to its internal culture and a philosophy of exceeding customer expectations. September 09 | T.E.S.T

32 | Customer profile

“Integration is a key feature of our products,” says Rick Riccetti. “TestTrack Pro and Surround SCM support linking issues and change requests with source code changes for improved traceability of software changes. QA Wizard Pro integrates with TestTrack Pro to automate pushing test failures into users’ defect tracking workflow. QA Wizard Pro also integrates with TestTrack TCM so users can link test scripts with TestTrack TCM tests cases and automate running tests. TestTrack Pro and TestTrack TCM are integrated in a single application so defects can be traced to the test that generated them. Defects can also be promoted to new test cases to help improve test coverage.” T.E.S.T | September 09

“TestTrack Pro has been used in SCB for quite a while,” comments Berglind. “It is currently used for bug tracking on production systems and in ongoing development projects. It is used as a tool both internally in the IT/Project departments as well as a tool for communicating with external developers. Most of our IT development is outsourced.” Seapine’s solutions are designed to help quality assurance organisations create, enforce, automate, track, and measure development and QA processes. Beginning with TestTrack TCM, they give companies a specialised tool which helps them manage the thousands of test cases needed to test today’s complex applications. And, as already stressed, applications in the financial sector can be massive and complex. With TestTrack Pro, companies like Santander can have a solution for managing issues found during software testing to ensure they are resolved or at least considered in future builds. QA Wizard Pro is a platform for automating functional and regression testing and Seapine brings expertise to companies to help them implement a formal automated testing programme. “Surround SCM is of course used for source control, both in our Java and .Net environments; it is also connected

to CruiseControl for daily builds,” confirms Berglind. “TestTrack TCM and QA Wizard Pro are new products for us, and we are currently exploring ways to use them in our organisation.”

A partner in quality After fourteen years in the business of supplying solutions and training to software testers, Cincinnati-based Seapine Software can rightfully claim to be one of the pioneers in the sector. The company specialises in helping customers like Santander Consumer Bank AS identify the problems associated with developing quality products and implementing costeffective solutions to those challenges. These solutions can come in many forms, such as consulting or tools, or a combination of both. Seapine is a privately-owned corporation headquartered in Cincinnati, Ohio in the USA. It was founded by husband and wife team Rick and Kelly Riccetti in 1995. The company’s founding mission was to reduce the complexity of software development tools, allowing developers and quality assurance professionals more time to focus on their tasks in hand—producing quality software. And that remains its mission to this day. Over the past 14 years Seapine has grown from its initial two employees www.testmagazineonline.com

Customer profile | 33

Santander Consumer Bank AS Norway history 1963 ELCON Leasing established as one of the first finance companies in Norway 1982 ELCON Leasing changes name to ELCON Finans through a merge with Inter Finans 2000 ELCON establishes business in Sweden 2001 ELCON establishes business in Denmark 2001 Bankia Bank established 2004 ELCON is sold to Santander Consumer Finance, a company within Grupo Santander, Spain. 2005 ELCON demerges through separation and sale of factoringand equipment leasing part of the business to Societe Generale, France 2005 Santander Consumer Finance acquires Bankia Bank which merges with ELCON 2005 Through this merger, the company changes name to Santander Consumer Bank 2006 Santander Consumer Bank establishes business in Finland 2007 Santander Consumer Bank establishes business in Denmark

to over 100, and has added offices in London, Munich, and Melbourne. With all of this growth, the company says it has remained faithful to its internal culture and a philosophy of exceeding customer expectations.

Integration “Integration is a key feature of our products,” says Rick Riccetti. “TestTrack Pro and Surround SCM support linking issues and change requests with source code changes for improved traceability of software changes. QA Wizard Pro integrates with TestTrack Pro to automate pushing test failures into users’ defect tracking workflow. QA Wizard Pro also integrates with TestTrack TCM so users can link test scripts with TestTrack TCM tests cases and automate running tests. TestTrack Pro and TestTrack TCM are integrated in a single application so defects can be traced to the test that generated them. Defects can also be promoted to new test cases to help improve test coverage.” TestTrack RM, Seapine’s new requirements management solution, rounds out their ALM offering by providing end-to-end traceability of requirements, issues, and tests. With TestTrack RM, customers can easily manage the complete requirement lifecycle, including planning, www.testmagazineonline.com

workflow, traceability, review, change management, and reporting. The company also offers the QualityReady Assessment (QRA). According to Seapine, this online assessment helps companies measure their product development capabilities in the four key areas of testing, change control, tracking, and automation. After completing the assessment, the survey results and recommendations for improving any noted deficiencies are emailed to the respondent within a few minutes. On the subject of the current economic uncertainty, Rick Riccetti is upbeat, seeing it as an ideal opportunity to automate. “Economically turbulent times are a wakeup call for companies to improve efficiencies across the organisation,” says Riccetti. “For software development companies and IT teams, test automation can have the biggest impact in helping them accomplish more with their QA resources. So there is a real opportunity to introduce automated testing to companies. However, these tools have a higher cost-per-seat than many other tools and cash flow is being watched closely. The challenge (and opportunity) is to demonstrate to companies how quickly an ROI can be realised on the automated testing investment.”

Geir Berglind Nordic region CTO Santander Consumer Bank AS Norway

September 09 | T.E.S.T

34 | Testing Feature as a service

STaaS in their eyes Many companies will tackle today’s economic challenges by deploying new software, but how can they be sure it won’t let them down? Richard Terry, Deputy CEO for Sogeti, explains how software testing as a service (STaaS) could help.

oftware can transform a business: it empowers customers to serve themselves, helps marketers to find the right prospects, and makes sure everybody gets paid at the end of the month. When it fails, though, it can transform the business in an altogether different way. Software applications are woven into the fabric of every business. If that fabric tears, the business is no longer covered for even its most basic activities. As companies face up to the challenging economy, many will look at how they can use software to cut their costs and improve their effectiveness. At the same time, they must minimise the risk of software failure.

The big picture Applications, be they custom or standard packages, cannot just be dropped into an organisation’s IT infrastructure. They need to be tested so that performance can be measured, security holes and bugs identified – in short, to make sure it’s T.E.S.T | September 09

A business that bypasses the testing stage is risking an underperforming application, full of glitches and security threats that can have a ripple effect on the whole IT infrastructure. And that can lead to something far more damaging – a loss in shareholder confidence. actually doing what it’s meant to as efficiently as possible. A business that bypasses the testing stage is risking an underperforming application, full of glitches and security threats that can have a ripple effect on the whole IT infrastructure. And that can lead to something far more damaging – a loss in shareholder confidence. A lot of organisations already use in-house teams to test their IT, but in a difficult economy this is maybe not an ideal way of doing things. In the majority of enterprises, the testing team will experience periods where

they have nothing to do. Sure, the people can be repurposed for other jobs, or they can just sit around playing Tetris. But hiring people that will not do what they’re there for, for much of the time, is not a good strategy when budgets are tight. Unless a lot of applications need testing consistently throughout the year, an organisation is unlikely to have enough work to keep an in-house testing team busy.

Testing as a service A new model has emerged for outsourcing software testing, called www.testmagazineonline.com

Testing as a service | 35

‘Software testing as a service’ (STaaS). It’s inspired by software as a service, a model where software is provided over the internet, requiring minimal infrastructure and enabling rapid startup and low maintenance operation. STaaS makes software testing similarly accessible, enabling projects to begin quickly, and making it viable for smaller software testing projects to be undertaken. By outsourcing testing, a business can minimise software risks and control costs, ensuring that it only spends money on application testing when it needs to. Studies have shown that outsourced software testing can cut the cost of testing by 25 percent or more. It’s not just for companies without an in-house team either. STaaS can help to smooth the cost of an in-house team by creating flexible capacity. In some larger enterprises, the in-house testing team could find itself in a position where it is unable to efficiently manage the volume of testing required at times. The business could hire more staff, www.testmagazineonline.com

but when the workload decreases to manageable levels again, it’s going to need to keep paying them. STaaS means this long-term investment is no longer necessary to address the short term bottleneck. An outsourced testing professional can work with the in-house testing team to help them clear the workload much faster.

Fast response

One of the reasons that traditional software testing is often expensive is that it can take a long time to get underway. Setting up can be a process that can sometimes take months. And when things do finally get underway, the results can be a long time coming. In many cases too long – businesses need to react swiftly to changes in an unpredictable market. They need to be able to counter competitor moves. They need to able to bring new products or services to market as soon as possible. In short, with the world as it is, businesses need results, and they need them yesterday. Outsourced testing specialists have dedicated, industry-approved testing environments, which are ready to go

Businesses need to react swiftly to changes in an unpredictable market. They need to be able to counter competitor moves. They need to able to bring new products or services to market as soon as possible. In short, with the world as it is, businesses need results, and they need them yesterday. September 09 | T.E.S.T

36 | Testing as a service

at any time. As a result, for the vast majority of applications, tests can be started in as little as two days. This, of course, means that the results can be delivered in a much shorter timeframe than with the traditional testing model. STaaS ensures that testing doesn’t cause a costly bottleneck during application deployment, which is one less headache for the organisation, and allows the business much greater operational flexibility. The specialised infrastructure, test environment, and methodology that testing outsourcers offer also mean that applications can be tested, even if the business has little or no documentation. In that case, the outsourcer could even help the organisation to start producing the critical documentation that will support future tests. STaaS will deliver cost and time savings to many businesses, at a time when they need them most. But it does have its limitations. If you’re looking to deploy an expansive enterprise-wide application, such as T.E.S.T | September 09

Siebel CRM or SAP ERP, STaaS would certainly not be the best way to go.

Losing control Some organisations might be uncomfortable with the idea of handing over control of their testing. What these companies may not realise is that, although the testing processes are handled by a third party, the relationship is still collaborative. Before starting tests, a reliable STaaS provider will visit the company and ensure it has a complete understanding of the organisation’s systems. The provider will make certain it knows what the customer wants and expects, so it can ensure this is what it delivers. Through collaboration, the testing project can benefit from a combination of testing experience and a deep understanding of the customer’s corporate culture and systems. The recession is a challenging time for many companies, but smart businesses can save themselves a lot time and money by letting someone else do the software testing for them.

Richard Terry Deputy CEO Sogeti www.uk.sogeti.com

STaaS will deliver cost and time savings to many businesses, at a time when they need them most. But it does have its limitations. If you’re looking to deploy an expansive enterprisewide application, such as Siebel CRM or SAP ERP, STaaS would certainly not be the best way to go. www.testmagazineonline.com

www.testmagazineonline.com

38 | Testing Feature qualifications

Feature sponsored by:

Welcome to the new BCS David Clarke, chief executive, explains the changes that have taken place in the BCS, the direction in which the organisation is now heading and the challenges that lie ahead.

CS was created in 1957 to bring together a number of different groups to try and deliver on the grand dream of making all aspects of computing into a compelling public benefit. We have made great strides since then, and BCS has evolved continually to meet the ever changing world we serve, never more so than in the last few years.

Transformation As we evolve, we see the world changing quicker than ever, and we know this pace of change will only increase. We must, therefore, respond and adapt quickly and more broadly to be relevant to the world we now live in so that we are ready for the future. We need not just to change BCS but to completely T.E.S.T | September 09

transform it, and this month we are launching ‘BCS, The Chartered Institute for IT’, with the aim not just of serving the ever changing world of the IT/computing professional in all its many guises, but more broadly ‘Enabling the Information Society’ as an international force. More than a million people work directly in information technology in the UK alone and computer science lies behind almost everything that happens in society today. To really support not only these knowledge workers but an increasingly IT-dependent society as a whole, our activities and capabilities have to become even more relevant to our members, the academic community, industry, government, volunteers and the wider public.

Today’s economy is built on knowledge and it’s on our ability to process, share and manage information that our society thrives or dies. How well the UK does these activities will in the end determine our financial standing. This information is the most vital commodity we possess. www.testmagazineonline.com

Testing qualifications | 39

Feature sponsored by:

IT could reasonably be called the first truly global profession, and the world needs an organisation that leads an IT profession fit to drive the information society, not just to follow it. BCS, The Chartered Institute for IT, is intent on becoming that organisation, and this re-launch is our next major step in a long-term plan to achieve this position.

Knowledge

and persistent gulf exists between the information haves and have-nots. Information vulnerability: From identity theft to benefit fraud, money laundering and deception â&#x20AC;&#x201C; information is still not secure. Lax information management: Data is still not highly valued by the public, by modern institutions or even by our profession. Project failure: Many public and private IT projects still fail to deliver: too often, and too often on a grand scale. Skills shortages: Nationally, and even globally, too few people are training in IT and too few are trained in the underlying fundamentals of computer science to solve our pressing problems. Lack of a clear career path: Finally, too few of those who do train actually make it up the chain of command into management whether inside or outside the IT function. We have developed a clear strategy based around five specific pillars, which you will see much of in the coming years, in order to address these challenges. Information society They will support: IT could reasonably be called the first 1. Bridging the gap between education truly global profession, and the world practice and research; needs an organisation that leads an IT profession fit to drive the information 2. Giving practitioners the professional development and career support society, not just to follow it. BCS, The they deserve; Chartered Institute for IT, is intent on 3. Informing public policy on how IT becoming that organisation, and this can contribute to society; re-launch is our next major step in a long-term plan to achieve this position. 4. Ensuring everyone benefits from IT; 5. Championing the global IT To put IT to work, we must actively profession. address a range of social integration This new BCS will take shape challenges: The digital divide: Contrary to popular between now and the end of belief, the digital divide has not closed. 2010, with a wide range of new qualifications, products and services We too easily mistake broadband all supported by a brand new website access for information literacy. A real Todayâ&#x20AC;&#x2122;s economy is built on knowledge and itâ&#x20AC;&#x2122;s on our ability to process, share and manage information that our society thrives or dies. How well the UK does these activities will in the end determine our financial standing. This information is the most vital commodity we possess. We increasingly rely upon it to protect us from crime and terror; diagnose and treat us when we are ill; monitor and mitigate our impact on a wounded planet; provide open access to goods and services to fulfil our human needs; help us to save and spend our money wisely and profitably; collect, allocate and redistribute our taxes; ensure efficient and transparent markets; and work more effectively and efficiently on a daily basis. Ultimately, we rely upon information-based technologies to support the most human need of all, to communicate: sharing our needs, desires and aspirations with others.

www.testmagazineonline.com

(check out www.bcs.org) offering simplified sign-up, subscription management, better industryacademia contact, state of the art knowledge sharing and career development tools. The challenge and the opportunity for BCS is clear. The UK and the world need an IT profession that understands its social impact and has a long-term strategy to address it.

Software testing Certification for software testers, which began with the launch of the ISEB Foundation Certificate in 1998, has grown steadily in importance in the UK and around the world ever since. Employers have embraced the qualifications as evidence of effort and attainment, through improved communications between software development and testing professionals, and in greater consistency of testing practice, and many have incorporated a level of certification into the criteria for software testing jobs. Training providers have enjoyed a steady stream of attendees for their accredited certificate courses. Individual testers have benefited from the enhanced status of software testing as a discipline and from the codification of bodies of knowledge in the form of syllabi and glossaries to support them.

ISEB Qualifications ISEB has a comprehensive portfolio of qualifications, of which software testing is one of the most popular in terms of certificates awarded. In recent times ISEB has recognized the value of treating its qualifications as a September 09 | T.E.S.T

40 | Testing qualifications

Feature sponsored by:

coherent portfolio with the potential for multiple routes that can both broaden and enrich qualifications; at the same time it has concentrated effort on the quality and consistency of its operations. The ISEB qualification program supports the Continuous Professional Development that is so important to all IT professionals. The software testing qualifications have been leaders in this process, with well documented and effectively managed examination processes that ensure complete consistency wherever the qualifications are offered in the world. Some ISEB Foundation and Intermediate exams are also available online via Prometric and Pearson VUE.

International The international IT certification market is very much influenced by the IT industry. Many certificates are offered by the well known IT vendors. Some domains are more influenced by vendor neutral certification, such as the ITIL certification which dominates the market of service management. Since the beginning of the T.E.S.T | September 09

certification of software testers by ISEB in 1998, the software testing market has developed into an international market. The International Software Testing Qualifications Board (ISTQB) was established as a result of this growth. ISTQB has adopted a strategy of encouraging as much international participation in its scheme as possible, each member country having its own national board. These national boards work together in a loosely federated structure. ISEB can be positioned as an international certification body, working closely together with national boards, especially the UK test board and the Dutch and Belgian test boards. The original foundation exam has grown into a global exam, translated into many languages and delivered through a variety of channels. ISEB is market leader in this area. ISEB examinations are set centrally and translated as necessary and appropriate for an international audience. The same processes and quality standards apply to all examinations. The ISEB model of quality has been steadily improved

over the years, and is now well documented and subject to regular audit, both internally and by external bodies. The quality and consistency of ISEB processes, and the clear benefits of acquiring ISEB certificates for both individuals and organisations, especially where organisations operate internationally and need consistency for all their areas of operation, has helped to maintain and enhanced the reputation of the software testing qualifications.

Software Testing Foundation The Foundation Certificate continues to be a popular qualification. It provides not only a broad-based introduction to the software testing specialism, but also a sound introduction to the terminology and basic approach of software development in its various forms. The average pass rate is 80 percent and most candidates pass the exam at the end of a three-day accredited course. For many the Foundation Certificate meets all their needs, but there is a significant number who wish or www.testmagazineonline.com

Testing qualifications | 41

Feature sponsored by:

need to progress to a more in-depth understanding of how testing works. The syllabus is maintained on a regular basis and the programme continues to attract many participants. The Foundation certificate is by far the most popular, also in comparison with some other software testing qualifications available in the market. ISEB has accredited over 30 training providers to offer the foundation level training for software testing.

questions at each level. ISEB has adopted an assessment strategy that begins with a relatively broad examination at the Foundation level, the examination, lasts one hour and contains 40 multiple choice questions with a defined mixture of questions at the lower (K1 – K3) levels and a balance of syllabus topics. The Intermediate level is a one hour examination with 25 multiple choice scenario-based questions. There is a defined structure of questions at the K2 – K4 levels and a balance of The higher levels of the syllabus topics. The scenarios provide software testing programme context for practical questions The training on the Foundation level requiring some analysis of the is very much an introduction. Those interested in more in-depth knowledge scenario to select correct answers. At the current highest level, the two and a practical perspective will find Practitioner qualifications are each the ISEB Intermediate Certificate a examined in a two hour written great next step. The Intermediate Certificate explains how the principles examination at the K2 – K6 levels. At the ISTQB Advanced level the of testing are actually applied in the required knowledge is in the K2 – K4 context of real software projects. band and is examined during a three The Intermediate Certificate prepares hour multiple choice examination with its holders with a sound practical 65 weighted questions, some of which knowledge of the testing discipline are scenario-based. from which it is possible to develop the skills and knowledge of a software testing professional. Unmatched experience The Intermediate level was launched ISEB’s long and unmatched experience in Autumn 2007 and since then across a spectrum of IT disciplines, 2,700 candidates have taken the coupled with its commitment to examination. software testing certification and a The Practitioner level currently strong desire to enhance and extend provides the knowledge and skills that its certification scheme in cooperation effectively define the more senior roles with the corporate market, the training in software testing. providers and organisations such as ISTQB has introduced the advanced ISTQB, make ISEB a powerful partner certificate for software testing which for any organisation. includes three specialist disciplines within software testing. The Advanced Leading the IT profession level covers all three of these These are exciting and important disciplines. times for us. As the natural leader In time, the various streams of of the IT profession, responsible for software testing certification for the awarding the much sought-after higher professional levels are likely Chartered IT Professional (CITP) to be harmonised so that valuable status, we must stand up and act, as experience can be harnessed for the a hugely respected and independent benefit of testers around the world. professional body, for the benefit of the entire industry. This now embraces all professionals The exams and practitioners from software In setting the objectives for its engineers to business analysts, and certificates, ISEB has made extensive the critical world of academia with use of Bloom’s taxonomy, a scale of its computer science foundations six levels of knowledge labelled K1 for all technology. Across the globe. to K6, which provides criteria to Join us in enabling the information characterise each level and offer society. guidance for examiners setting www.testmagazineonline.com

These are exciting and important times for us. As the natural leader of the IT profession, responsible for awarding the much sought-after Chartered IT Professional (CITP) status, we must stand up and act, as a hugely respected and independent professional body, for the benefit of the entire industry.

David Clarke Chief Executive BCS www.bcs.org

September 09 | T.E.S.T

42 | T.E.S.T company profile

ISEB The Information Systems Examinations Board (ISEB) is part of the BCS, the Charted Institute for IT, and is an international examination body created to raise the standard of competence and performance of people working in IT. We’re leading the way in qualifications for IT professionals – delivering more than 380,000 exams in over 200 countries. Our qualifications are internationally recognised and cover eight major subject areas in: Software Testing, ITIL /IT Service Management, IT Assets and Infrastructure, Systems Development, Business Analysis, Project Management, IT Governance, Information and Security and our new qualification Sustainable IT. These are available at Foundation, Practitioner and Higher Level to suit each individual candidate. ISEB Professional Level is also available. For more information visit www.iseb-exams.com. These qualifications are delivered via a network of high quality accredited training and examination providers. The breadth and depth of our portfolio is one of its key strengths as it encourages knowledge, understanding and application in specific business and IT areas. Candidates develop their competence, ability and aptitude – and therefore their professional potential – giving employers the edge they’re looking for

BCS BCS, The Chartered Institute for IT, promotes wider social and economic progress through the advancement of information technology science and practice. We bring together industry, academics, practitioners and government to share knowledge, promote new thinking, inform the design of new curricula, shape public policy and inform the public. As the professional membership and accreditation body for IT, we serve over 70,000 members including practitioners, academics and students, in the UK and internationally. A leading IT qualification body, we also offer a range of widely recognised professional and end-user qualifications.

BCS membership for software testers BCS membership gives you an important edge; it shows you are serious about your career in IT and are committed to your own professional development, confirming your status as an IT practitioner of the highest integrity. Our growing range of services and benefits are designed to be directly relevant at every stage of your career.

Industry recognition Post-nominals – AMBCS, MBCS, FBCS & CITP – are recognised worldwide, giving you industry status and setting you apart from your peers. BCS received its Royal Charter in 1984 and is currently the only awarding body for Chartered IT Professional (CITP) status, also offering a route to related Chartered registrations, CEng and CSci.

Membership grades Professional membership (MBCS) is our main professional entry grade and the route to Chartered(CITP) status. Professional membership is for competent IT practitioners who typically have five or more years of IT work experience. Relevant qualifications, eg a computingrelated degree, reduce this requirement to two or three years of experience. Associate membership (AMBCS) is available for those just beginning their career in IT, requiring just one year’s experience. Joining is straightforward – for more information visit: www.bcs.org/membership where you can apply online or download an application form.

Best practice By signing up to our Code of Conduct and Code of Good Practice, you declare your concern for public interest and your commitment to keeping pace with the increasing expectations and requirements of your profession.

Networking opportunities Our 44 branches, 16 international sections and over 40 specialist groups including Software Testing (SIGIST) and Methods & Tools, provide access to a wealth of experience and expertise. These unrivalled networking opportunities help you to keep abreast of current developments, discuss topical issues and make useful contacts.

Specialist Group in Software Testing (SIGIST) With over 2,500 members SIGIST is the largest specialist group in the BCS. Objectives of the group include promoting the importance of software testing, developing the awareness of the industry’s best practice and promoting and developing high standards and professionalism in software testing. For more information please visit: www.sigist.org.uk.

Information services The BCS online library is another invaluable resource for IT professionals, comprising over 200 e-books plus Forrester reports and EBSCO databases. BCS members also receive a 20 percent discount on all BCS book publications. This includes Software Testing, an ISEB Foundation and Intermediate. As well as explaining the basic steps of the testing process and how to perform effective tests, this book provides an overview of different techniques, both dynamic and static, and how to apply them.

Career development A host of career development tools are available through BCS including full access to SFIA (the Skills Framework for the Information Age) which details the necessary skills and training required to progress your career.

BCS, First Floor, Block D, North Star House, North Star Avenue, Swindon, SN2 1FA, United Kingdom Tel: +44 (0) 1793 417655 Fax: +44 (0) 1793 417559 Email: isebenq@hq.bcs.org.uk Web: www.iseb-exams.com

T.E.S.T | September 09

www.testmagazineonline.com

T.E.S.T company profile | 43

Seapine Software TM

With over 8,500 customers worldwide, Seapine Software Inc is a recognised, award-winning, leading provider of quality-centric application lifecycle management (ALM) solutions. With headquarters in Cincinnati, Ohio and offices in London, Melbourne, and Munich, Seapine is uniquely positioned to directly provide sales, support, and services around the world. Built on flexible architectures using open standards, Seapine Software’s cross-platform ALM tools support industry best practices, integrate into all popular development environments, and run on Microsoft Windows, Linux, Sun Solaris, and Apple Macintosh platforms. Seapine Software's integrated software development and testing tools streamline your development and QA processes—improving quality, and saving you significant time and money.

TestTrack RM TestTrack RM centralises requirements management, enabling all stakeholders to stay informed of new requirements, participate in the review process, and understand the impact of changes on their deliverables. Easy to install, use, and maintain, TestTrack RM features comprehensive workflow and process automation, easy customisability, advanced filters and reports, and rolebased security. Whether as a standalone tool or part of Seapine’s integrated ALM solution, TestTrack RM helps teams keep development projects on track by facilitating collaboration, automating traceability, and satisfying compliance needs.

TestTrack Pro TestTrack Pro is a powerful, configurable, and easy to use issue management solution that tracks and manages defects, feature requests, change requests, and other work items. Its timesaving communication and reporting features keep team members informed and on schedule. TestTrack Pro supports MS SQL Server, Oracle, and other ODBC databases, and its open interface is easy to integrate into your development and customer support processes.

TestTrack TCM TestTrack TCM, a highly scalable, cross-platform test case management solution, manages all areas of the software testing process including test case creation, scheduling, execution, measurement, and reporting. Easy to install, use, and maintain, TestTrack TCM features comprehensive workflow and process automation, easy customisability, advanced filters and reports, and role-based security. Reporting and graphing tools, along with user-definable data filters, allow you to easily measure the progress and quality of your testing effort.

QA Wizard Pro QA Wizard Pro completely automates the functional and regression testing of Web, Windows, and Java applications, helping quality assurance teams increase test coverage. Featuring a next-generation scripting language, QA Wizard Pro includes advanced object searching, smart matching a global application repository, data-driven testing support, validation checkpoints, and built-in debugging. QA Wizard Pro can be used to test popular languages and technologies like C#, VB.NET, C++, Win32, Qt, AJAX, ActiveX, JavaScript, HTML, Delphi, Java, and Infragistics Windows Forms controls.

Surround SCM Surround SCM, Seapine’s cross-platform software configuration management solution, controls access to source files and other development assets, and tracks changes over time. All data is stored in industry-standard relational database management systems for greater security, scalability, data management, and reporting. Surround SCM’s change automation, caching proxy server, labels, and virtual branching tools streamline parallel development and provide complete control over the software change process.

www.seapine.com United Kingdom, Ireland, and Benelux: Seapine Software Ltd. Building 3, Chiswick Park, 566 Chiswick High Road, Chiswick, London, W4 5YA UK Phone:+44 (0) 208-899-6775 Email: salesuk@seapine.com Americas (Corporate Headquarters): Seapine Software, Inc. 5412 Courseview Drive, Suite 200, Mason, Ohio 45040 USA Phone: 513-754-1655

www.testmagazineonline.com

September 09 | T.E.S.T

44 | T.E.S.T company profile

Parasoft Improving productivity by delivering quality as a continuous process. For over 20 years Parasoft has been studying how to efficiently create quality computer code. Our solutions leverage this research to deliver automated quality assurance as a continuous process throughout the SDLC. This promotes strong code foundations, solid functional components, and robust business processes. Whether you are delivering Service-Orientated Architectures (SOA), evolving legacy systems, or improving quality processes – draw on our expertise and award winning products to increase productivity and the quality of your business applications. Parasoft's full-lifecycle quality platform ensures secure, reliable, compliant business processes. It was built from the ground up to prevent errors involving the integrated components—as well as reduce the complexity of testing in today's distributed, heterogeneous environments.

What we do Parasoft's SOA solution allows you to discover and augment expectations around design/development policy and test case creation. These defined policies are automatically enforced, allowing your development team to prevent errors instead of finding and fixing them later in the cycle. This significantly increases team productivity and consistency.

End-to-end testing: Continuously validate all critical aspects of complex transactions which may extend through web interfaces, backend services, ESBs, databases, and everything in between.

Advanced web app testing: Guide the team in developing robust, noiseless regression tests for rich and highly-dynamic browser-based applications.

Application behavior virtualisation: Automatically emulate the behavior of services, then deploys them across multiple environments – streamlining collaborative development and testing activities. Services can be emulated from functional tests or actual runtime environment data.

Load/performance testing: Verify application performance and functionality under heavy load. Existing end-to-end functional tests are leveraged for load testing, removing the barrier to comprehensive and continuous performance monitoring.

Specialised platform support: Access and execute tests against a variety of platforms (AmberPoint, HP, IBM, Microsoft, Oracle/BEA, Progress Sonic, Software AG/webMethods, TIBCO).

Security testing: Prevent security vulnerabilities through penetration testing and execution of complex authentication, encryption, and access control test scenarios.

Trace code execution: Provide seamless integration between SOA layers by identifying, isolating, and replaying actions in a multi-layered system.

Continuous regression testing: Validate that business processes continuously meet expectations across multiple layers of heterogeneous systems. This reduces the risk of change and enables rapid and agile responses to business demands.

Multi-layer verification: Ensure that all aspects of the application meet uniform expectations around security, reliability, performance, and maintainability.

Policy enforcement: Provide governance and policy-validation for composite applications in BPM, SOA, and cloud environments to ensure interoperability and consistency across all SOA layers. Please contact us to arrange either a one to one briefing session or a free evaluation.

Web: www.parasoft.com Email: sales@parasoft-uk.com Tel: +44 (0) 208 263 6005

T.E.S.T | September 09

www.testmagazineonline.com

T.E.S.T company profile | 45

TechExcel TechExcel is the leader in unified Application Lifecycle Management as well as Support and Service solutions that bridge the divide between product development and service/support. This unification enables enterprises to focus on the strategic goals of product design, project planning, development and testing, while enabling transparent visibility with all customer-facing initiatives. TechExcel has over 1,500 customers in 45 countries and maintains offices in UK, US, China and Japan.

Application Lifecycle Management DevSuite is built around the best-practices insight that knowledge is central to any product development initiative. By eliminating the silos of knowledge that exist between different teams and in different locales, DevSuite helps enterprises transform their development processes, increasing efficiency and overall quality.

DevSpec DevSpec is an integrated requirements management solution that is specifically designed to provide visibility, traceability and validation of your product or project requirements. DevSpec provides a framework to create new requirements, specifications and features that can be linked to development and testing implementation projects.

DevPlan DevPlan is a project, resource, and task management tool. It allows users to plan high level areas of work, assign team members to work in these areas, and then track the tasks needed to complete the activities.

DevTrack DevTrack is the leading project issue and defect tracking tool that is used by development teams of all sizes around the globe. Its configurable workflows allow DevTrack to meet the needs of any organisation's development processes.

DevTest From test case creation, planning and execution through defect submission and resolution, DevTest tracks and manages the complete quality lifecycle. DevTest combines the test management features of DevTest, DevTrack and TestLink for test automation into one integrated solution.

KnowledgeWise KnowledgeWise is the knowledge management solution at the core of the entire suite. It is the centralised knowledge base for all company documents including: contracts, processes, planning information and other important records as well as customer-facing articles, FAQs, technical manuals and installation guides. More information at: www.techexcel.com/products/ devsuite.

Service and Support Management Service and Support Management solutions provide enterprises with total visibility and actionable intelligence for all service desk, asset management and CRM business processes.

ServiceWise ServiceWise is a customisable and comprehensive internal Helpdesk, ITSM- and ITILcompliant solution. Automate and streamline services and helpdesk activities with configurable workflows, process management, email notifications and a searchable knowledge base. The self-service portal includes online incident submission, status updates, online conversations and a knowledgebase. ServiceWise includes modules such as incident management, problem escalation and analysis, change management and asset management. CustomerWise CustomerWise is an integrated CRM solution focused on customer service throughout the entire customer lifecycle. CustomerWise allows you to refine sales, customer service and support processes to increase cross-team communication and efficiency while reducing your overall costs. Combine sophisticated process automation, knowledgebase management, workflow, and customer self-service to improve business processes that translate into better customer relationships. AssetWise AssetWise aids the process of monitoring, controlling and accounting for assets throughout their lifecycle. A single and centralised location enables businesses to monitor all assets including company IT assets, managing asset inventories, and tracking customerowned assets.

FormWise FormWise is a web-based form management solution for ServiceWise and CustomerWise. Create fully customised online forms and integrate them directly with your workflow processes. Forms can even be routed automatically to the appropriate individuals for completion, approval, and processing, improving your team's efficiency. Web-based forms may be integrated into existing websites to improve customer interactions including customer profiling, surveys, product registration, feedback, and more.

DownloadPlus DownloadPlus is an easy-to-use website management application for monitoring file downloads and analysing website download activities. DownloadPlus does not require any programming or HTML. DownloadPlus provides controlled download management for all downloadable files, from software products and documentation, to marketing materials and multimedia files. More information at: www.techexcel.com/products/itsm/

Training Further your investment with TechExcel, effective training is essential to getting the most from an organisation's investment in products and people. We deliver professional instructor-led training courses on every aspect of implementation and use of all TechExcelâ&#x20AC;&#x2122;s software solutions as well as both service management and industry training. We are also a Service Desk institute accredited training partner and deliver their certification courses. More information at: www.techexcel.com/support/ techexceluniversity/servicetraining.html

For more information, visit www.techexcel.com or call 0207 470 5650.

www.testmagazineonline.com

September 09 | T.E.S.T

46 | T.E.S.T company profile

31 Media 31 Media is a business to business media company that publishes high quality magazines and organises dynamic events across various market sectors. As a young, vibrant, and forward thinking company we are flexible, proactive, and responsive to our customers' needs.

www.31media.co.uk

T.E.S.T Online Since its launch in 2008 T.E.S.T has rapidly established itself as the leading European magazine in the software testing market. T.E.S.T is a publication that aims to give a true reflection of the issues affecting the software testing market. What this means is that the content is challenging but informative, pragmatic yet inspirational and includes, but is not limited to: In-depth thought leadership; Customer case studies; News; Cutting edge opinion pieces; Best practice and strategy articles The good news is that the T.E.S.T website, T.E.S.T Online has had a root and branch overhaul and now contains a complete archive of previous issues as well as exclusive web-only content and testing and IT news. At T.E.S.T our mission is to show the importance of software testing in modern business and capture the current state of the market for the reader.

www.testmagazine.co.uk

VitAL Magazine VitAL is a journal for directors and senior managers who are concerned about the business issues surrounding the implementation of IT and the impact it has on their customers. Today senior management are starting to realise that implementing IT effectively has a positive impact on the internal and external customer and it also influences profitability. VitAL magazine was launched to help ease the process.

www.vital-mag.net

Customer Magazine Customer Magazine was launched to address and assist with the various challenges senior professionals face when establishing a customer-centric business. Our editorial takes a pragmatic approach to what has become a series of complex issues and delivers dynamic, provocative, and insightful articles, case studies, opinion pieces, and news stories that not only challenge our readers but also bring clarity and vision to the many challenges they face.

www.customermagazine.co.uk

31 Media www.31media.co.uk info@31media.co.uk Crawley Business Centre, Stephenson Way, Crawley, West Sussex, RH10 1TN, United Kingdom Phone: +44 (0) 870 863 6930 Fax: +44 (0) 870 085 8837

T.E.S.T | September 09

www.testmagazineonline.com

T.E.S.T company profile | 47

iTrinegy Network emulation & application testing tools iTrinegy is Europe’s leading producer of network emulator technology which enables testers and QA specialists to conduct realistic pre-deployment testing in order to confirm that an application is going to behave satisfactorily when placed in the final production network.

Delivering more realistic testing Increasingly, applications are being delivered over wide area networks (WANs), wireless LANs (WLAN), GPRS, 3G, satellite networks etc, where network characteristics such as bandwidth, latency, jitter and packet error or loss can have a big impact on their performance. So, there is a growing need to test software in these environments. iTrinegy Network Emulators enable you to quickly and easily recreate a wide range of network environments for testing applications, including VoIP, in the test lab or even at your desktop.

Ease of use Our network emulators have been developed for ease of use: • No need to be a network expert in order to use them • Pre-supplied with an extensive range of predefined test network scenarios to get you started • Easy to create your own custom test scenarios • All test scenarios can be saved for subsequent reuse • Automated changes in network conditions can be applied to reflect the real world • Work seamlessly with load generation and performance tools to further enhance software testing.

A comprehensive range to suit your needs iTrinegy’s comprehensive range of network emulators is designed to suit your needs and budget. It includes: • Software for installation on your own desktop or laptop (trial copies available) • Small, portable inline emulators that sit silently on the desktop and can be shared amongst the test team • Larger portable units capable of easily recreating complex multi-path, multi-site, multi-user networks for full enterprise testing • High performance rack-mount units designed to be installed in dedicated test labs • Very high performance units capable of replicating high speed, high volume networks making them ideal for testing applications in converged environments. If you would like more information on how our technology can help you ensure the software you are testing is ‘WANready’ and going to work in the field, please contact iTrinegy using the details below:

Email: info@itrinegy.com Tel: +44 (0)1799 543 345 Web: www.itrinegy.com/testmagazine

The Software Testing Club The Software Testing Club is a relaxed yet professional place for software testers to hang out, find likeminded software testers and get involved in thoughtful and often fun conversations. Interesting things happen at The Software Testing Club. It started out as an experiment. Now two years on it has turned into vibrant online community of software testing professionals. You'll find members are dedicated to their profession and you can find them in deep conversation within the forums. However, it's more than just forums and your standard niche social network. As the club grows we are finding things happening. This includes things like a Job Board, a Mentoring Group, a collaborative Software Testing Book and a crowd sourced testing initiative called Flash Mob Testing. The Software Testing Club is a grassroots effort. It's for the members and grows according to what we believe they want. Come join and let us know what you think.

Rosie Sherry – Founder & Community Manager Email: rosie@softwaretestingclub.com Tel: +44 (0)7730952537 Web: www.softwaretestingclub.com

www.testmagazineonline.com

September 09 | T.E.S.T

48 | The Last Word

the last word... We’re on our way to a global testing community, but not there yet… Joel Montvelisky of Practitest searches his homeland of Costa Rica for some signs of a global testing community.

was back in my homeland of Costa Rica during August this year. Even though I visit there every couple of years this was the first time – after 15 years living in Israel and working in software testing and QA – that I decided to learn more about the status of testing in the local professional environment. I got in touch with the National Chamber of IT and Communications and organised a seminar on software testing and testing Intelligence. The seminar sold-out within a couple of days, so there clearly was an audience for the subject of testing, but what caught my attention was that over half this audience were not testers but project managers and business analysts.

A different evolutionary path During the seminar and in the meetings I had after the event I learned a couple of interesting things: 1. The reality for Costa Rica is very similar to that of the whole Central American region (and still relevant for a vast number of South American countries); 2. Many of the working assumptions I had taken for trivial during my work as a tester were less trivial in these parts of the World. I understood that because of a different evolutionary path and maybe even a cultural gap, the subject of software testing had developed less and in a different way than in most of the places I had visited and worked in so far. For example, in many local companies testing was something coordinated by the project managers that utilised end users or subject matter experts to validate that the software did what was expected from it. This is what I would call UATs or user acceptance tests in my process, but the rest of the testing tasks and operations appeared to be missing from the process. T.E.S.T | September 09

I don’t want to be misunderstood; these countries are not falling short on their technology or engineering skills. They have very competent developers working with the latest and greatest technologies creating good-looking software products. There are also some companies in these countries where they use a very robust testing process, but these are isolated firms that mostly belong to or are closely linked to international corporations that bring with them their end-to-end processes. As I already stated, I think this is the result of a separate evolution taken by the software market in these countries. Places where ten or 15 years ago they had little or no local software development and operations had created a local development culture from scratch in a very short time, managing to close the par in the technical levels but leaving the methodological aspects behind, and with it most of the areas related to the testing process. Also in these places there is a cultural aspect where the average end users appears to be less demanding and thus allow local firms to ‘get away’ without doing much testing, at least for now. The most interesting and frustrating part of this phenomenon is the fact that the people leading the development processes understand they are missing something, but they lack the tools and the know-how to find and fill the missing pieces.

Is there change on the horizon? I think that in the same way that nature flows to close existing gaps and reach a state of equilibrium so do geographical markets. This means that two things should start happening (if they are not already) in these places: 1. Software development companies that want to export their products to other markets will start running into

commercial barriers due their lack of testing processes; 2. Local buyers will stop being so ‘forgiving’, since they get access to international firms that provide the same products at competitive prices but with higher quality. The effects of both of these processes will be to generate the need for change within the local development markets and firms, triggering an improvement process that should push the quality of their products and processes up to international standards. This will be the time when we will start seeing a more active representation of these countries within the international testing community, when places like Central America or the Middle East start taking an active role in the development and enrichment of our testing methodologies and techniques. This change may take place within the next couple of months or years, depending on each separate country and market, but I have no doubt it will take place, closing the existing gaps and generating a truly global testing community.

The most interesting and frustrating part of this phenomenon is the fact that the people leading the development processes understand they are missing something, but they lack the tools and the know-how to find and fill the missing pieces.

Joel Montvelisky Xxxxxxxx Practitest www.practitest.com

www.testmagazineonline.com

Subscribe to the most VitAL source of information

News, Views, Strategy, Management, Case Studies and Opinion Pieces

vital Inspiration for the modern business

www.vital-mag.net If you have not already subscribed then visit www.vital-mag.net to download a subscription form which you can fax to +44 (0) 870 085 8837 or post to: 31 Crawley Business Centre, Stephenson Way, Crawley, Westplease Sussex, 1TN. 31 Media will keep you up to dateMedia, with our own products and offers including VitAL Magazine. If you do not wish to receive this information write to RH10 the Circulation Manager at the09 address given. www.testmagazineonline.com September | T.E.S.T Please tick here â&#x2013; if you do not wish to receive relevant business information from other carefully selected companies.

Promote yourself...Test yourself ISEB Software Testing certification • • • • •

International benchmark of skills and experience for Software Testers 80% of all Software Testing exams worldwide taken through ISEB Hierarchical structure providing a basis for skills and career progression Over 70,000 Software Testing exams delivered worldwide Supported by ISEB Foundation and Intermediate level publications www.bcs.org/books

ISEB is part of the BCS, The Chartered Institute for IT, and is an International examination body. We have delivered over 380,000 exams worldwide and our exams are available in over 200 countries. For more information visit www.iseb-exams.com or call 01793 417655

Book a training course at www.iseb-exams.com/st Book an exam at www.iseb-exams.com/computerbasedexams The British Computer Society (Registered charity no. 292786) MTG/AD/689/0909

T.E.S.T | September 09