TEST Magazine Launch - October 2008 by 31 Media

I n T o u c h W it h T e c h n o l o g y

Sponsored by

T h e E u r o p e a n S o f tw a r e T e st e r

Handling the risk Getting it right with Risk-based testing Inside: Delivering with agile; The future of software testing; Anarchy in the QA

Looking for a better way to manage test cases?

Learn how to write and manage test cases more effectively. Read our TestTrack TCM best practices white paper. Download it today at www.seapine.com/tstcmbp

Seapine

TestTrack TCM ®

Software for test case planning, execution, and tracking You can’t ship with confidence if you don’t have the tools in place to document, repeat, and quantify your testing effort. TestTrack TCM can help you thoroughly test your applications in less time. Seapine ALM Solutions: TestTrack Pro

Issue & Defect Management

TestTrack TCM

Test Case Planning & Tracking

Surround SCM

Configuration Management

QA Wizard Pro

!UTOMATED &UNCTIONAL 4ESTING

In TestTrack TCM you have the tool you need to write and manage thousands of test cases, select sets of tests to run against builds, and process the pass/fail results using your development workflow. With TestTrack TCM driving your QA process, you’ll know what has been tested, what hasn't, and how much effort remains to ship a quality product. Deliver with the confidence only achieved with a well-planned testing effort.

s %NSURE ALL STEPS ARE EXECUTED AND IN THE SAME order, for more consistent testing. s +NOW INSTANTLY WHICH TEST CASES HAVE BEEN EXECUTED WHAT YOUR COVERAGE IS AND HOW MUCH testing remains. s 4RACK TEST CASE EXECUTION TIMES TO CALCULATE HOW much time is required to test your applications. s 3TREAMLINE THE 1! &IX 2E TEST CYCLE BY pushing test failures immediately into the defect management workflow. s Achieve complete traceability between test cases and defects with seamless TestTrack Pro integration.

Leader | 1

T h e E u r o p e a n S o f tw a r e T e st e r

IN TOUCH WITH TECHNOLOGY

T H E E U RO P E A N S O F T WA R E T E S T E R

Welcome to

The European Software Tester

Handling the risk Getting it right with Risk-based testing Inside: Delivering with agile; The future of software testing; Anarchy in the QA

Editor Matthew Bailey matthew.bailey@31media.co.uk Tel: +44 (0)1293 934464 Advertising Sales Ian Trevett ian.trevett@31media.co.uk Tel: +44 (0)1293 934463 Production & Design Dean Cook dean.cook@31media.co.uk Toni Barrington toni.barrington@31media.co.uk Publisher Grant Farrell grant.farrell@31media.co.uk Tel: +44 (0)1293 934461 Editorial & Advertising Enquiries 31 Media, Crawley Business Centre, Stephenson Way, Crawley, West Sussex, RH10 1TN Tel: +44 (0) 870 863 6930 Fax: +44 (0) 870 085 8837 Email: info@31media.co.uk Web: www.testmagazine.co.uk Printed by Pensord, Tram Road, Pontllanfraith, Blackwood. NP12 2YA

aunching into stormy waters may seem a little rash, but as some of our columnists and contributors have demonstrated within, the world has never needed the disciplines of software testing more than it does today. And cometh the hour, cometh the magazine! As editor of this inaugural issue of T.E.S.T, I think a little bit of background is called for. I am an experienced business to business editor with over 13 years’ served on a range of highly technical industrial and business titles, from aerospace to packaging, logistics to engineering I have written about them all. I currently also edit T.E.S.T’s sister IT title VitAL for 31 Media. Perhaps more important than the basic editorial skills is the ability to get involved and become part of an industry. To this end I intend to get out and mix it up with as many companies and people as I can – it’s the only way you can really get under the skin of an industry in my experience. Before getting the ‘calling’ to be a business journalist I worked as a marketing communications consultant for a major telecoms company, so I do have a little experience at the ‘coal face’ of the IT sector. Indeed, the company in question

was at the time in the process of writing and (more importantly) testing the software for BT’s first generation of digital telephone exchanges, so I like to think I have some history with the industry. I only wish I paid more attention at the time; hindsight is a wonderful thing. This issue of T.E.S.T tackles the key issues and challenges facing software testing as it stands today. And I think we cover the ground from the in-depth techie, to the more general, holistic and strategic thought leadership features within these pages. Rather than skilled editing, however, this is a testament to the range of talented individuals we have contributing their opinions. Inside, experienced consultant Angelina Samaroo gives guidance on riding the rollercoaster that is the current commercial environment. It’s probably inevitable given the current seismic shifts in the business world that most of our features touch on this topic in one way or another – and hopefully offer valuable advice on coming out the other side in better shape!

Matt Bailey

© 2008 31 Media Limited.All rights reserved. T.E.S.T Magazine is edited, designed, and published by 31 Media Limited. No part of T.E.S.T Magazine may be reproduced, transmitted, stored electronically, distributed, or copied, in whole or part without the prior written consent of the publisher.A reprint service is available. Opinions expressed in this journal do not necessarily reflect those of the editor or T.E.S.T Magazine or its publisher, 31 Media Limited.

Published by:

I intend to get out and mix it up with as many companies and people as I can – it’s the only way you can really get under the skin of an industry in my experience. Matt Bailey, Editor October 08 | T.E.S.T

4VCTDSJCF UP UIF NPTU 7JU"- TPVSDF PG JOGPSNBUJPO

/FXT 7JFXT 4USBUFHZ .BOBHFNFOU $BTF TUVEJFT BOE 0QJOJPO 1JFDFT 9%3 ) AM IN THE 5+ AND ) WOULD LIKE TO RECEIVE A .0/5) 46#4$3*15*0/ SIX ISSUES TO 7JU"- -AGAZINE AT A COST OF a 9%3 ) AM /VERSEAS AND ) WOULD LIKE TO RECEIVE A .0/5) 46#4$3*15*0/ SIX ISSUES TO 7JU"- -AGAZINE AT A COST OF a ) ENCLOSE A CHEQUE FOR

a 5+

0LEASE INVOICE MY COMPANY FOR 0URCHASE /RDER .UMBER 4ITLE *OB 4ITLE

a /VERSEAS 0AYABLE TO -EDIA ,TD

a 5+

3IGNATURE

&ULL .AME #OMPANY

!DDRESS

0OST #ODE

4EL

&AX 1V[XQZI\QWV NWZ \PM UWLMZV J][QVM[[

%MAIL 3IGNATURE

$ATE

XXX WJUBM NBH OFU

*G ZPV IBWF OPU BMSFBEZ TVCTDSJCFE UIFO WJTJU XXX WJUBM NBH OFU UP EPXOMPBE B TVCTDSJQUJPO GPSN PS TJNQMZ DPNQMFUF UIF GPSN CFMPX BOE GBY UP PS QPTU UP .FEJB $SBXMFZ #VTJOFTT $FOUSF 4UFQIFOTPO 8BZ $SBXMFZ 8FTU 4VTTFY 3) 5/ *G ZPV BMSFBEZ TVCTDSJCF UIFO XIZ OPU -EDIA WILL KEEP YOU UP TO DATE WITH OUR OWN PRODUCTS AND OFFERS INCLUDING 6IT!, -AGAZINE )F YOU DO NOT WISH TO RECEIVE THIS INFORMATION PLEASE WRITE TO THE #IRCULATION -ANAGER AT THE ADDRESS GIVEN T.E.S.T |QBTT UIJT GPSN UP BOZ DPMMFBHVFT ZPV GFFM XPVME CFOFm October 08 U GSPN SFDFJWJOH UIFJS PXO DPQZ PG 7JU"- .BHB[JOF 0LEASE TICK HERE â&#x2013; IF YOU DO NOT WISH TO RECEIVE RELEVANT BUSINESS INFORMATION FROM OTHER CAREFULLY SELECTED COMPANIES

Sponsor's Statement | 3

Hold the front page! Here it is at last, what the industry’s been waiting for, welcome to the first edition of T.E.S.T magazine.

ith Software Testing increasing its status as a profession across the UK and Europe, welcome to the launch of the definitive software testing magazine. We are now entering the end of 2008 and what a crazy year it has been. On the plus side we have witnessed two very successful sporting events taking place: the Beijing Olympics and closer to home, Euro 2008. While on the down side the global economy has gone from bad to worse and is continuing to sink like the proverbial stone. And incidentally, for advice on surviving the credit crunch and information on how testers and being affected, you could do much worse than checking out Angelina Samaroo’s ISEBsponsored thought leadership feature on page ten. All is not lost though, and we at ISEB are proud to sponsor this new, innovative magazine. “Who are ISEB?” I hear you all ask, well let me give you a bit of background information on who we are and what we do... ISEB is an exam body that has been operating for over 40 years; we provided exams in software testing 15 years

ago and we were instrumental in the development of the original software testing syllabus that was instrumental in creating the ISTQB syllabus. We now offer over 50 products, with almost 350,000 exams being delivered worldwide (60,000 of these in software testing). In fact, our courses have been delivered in over 50 countries including, but not limited to: South Africa, Brazil, USA, India, Japan, Australia, and across Europe. We are therefore delighted to sponsor the inaugural T.E.S.T magazine and as the leading examination institute for software testers worldwide, we are pleased to be supporting testers throughout their careers. It is essential that software testers have a clear career path and are supported in an ongoing manner, so ISEB in conjunction with T.E.S.T magazine aims to create an essential source of information for testers, providing relevant and important news and features on a regular basis. Enjoy your read.

Pete Bayley Director, Qualification Products, BCS

It is essential that software testers have a clear career path and are supported in an ongoing manner Pete Bayley

October 08 | T.E.S.T

T.E.S.T | October 08

Contents | 5

CONTENTS oct 08

Leader column

Editor Matt Bailey welcomes you to this inaugural issue of The European Software Tester. 3

T.E.S.T Sponsor’s statement

Cover story

Goldilocks and the two bears. No fairytale endings for testing, which never gets to the third bowl of porridge - it’s never just right. But risk-based software testing offers a solution.

Thought leadership

While the planet and the economy are in meltdown together, Angelina Samaroo asks what this means for software testers?

Delivering with agility

Tackling the challenges of implementing agile processes in the software testing environment and successfully integrating agile with automation.

20 EuroSTAR 2008 - The future of software testing on show What’s in store at the EuroSTAR 2008 international conference that takes place November 10th -13th in The Hague, the Netherlands.

22 Predicting the future of software testing An increasing emphasis on the four Rs: repeatability, reliability, re-use and robustness, has opened a new path to testing, where it will need to change to accommodate trends and become a business-led activity

Can you afford not to train?

Don’t make the expensive mistake of freezing training requirements. The question is not whether a company can afford to train, but rather can you afford not to?

Anarchy in the QA

There is unrest brewing in IT QA departments all over the globe and a revolution that is gaining momentum, and is forcing people to re-think their attitude towards software test automation.

Teaching a new dog an old trick

By looking back at some old techniques and applying them to today’s systems we can shave weeks off the time taken to build an automated test system.

Keeping it real

Without a realistic environment, software testing is meaningless and with an ever-changing array of different environments software testers need to ensure that the appropriate context is included in the testing regime.

T.E.S.T Company Profiles

40 October 08 | T.E.S.T

6 | Risk-based testing

You know the story of Goldilocks and the Three Bears? Well, guess what, in testing we never get to the third bowl of porridge, it’s never just right. Like it or not, we are going to have to accept that we only have limited budget and limited time to test this application. Risk-based software testing offers a solution says Tony Simms, principle consultant at Roque Consulting.

Goldilocks and the two bears

et’s face the truth, we are testers, we test, that’s what we do. We love it. A good day for us is when everything development delivered to us is sent back with a host of Sev 1s. Oh yes, we pretend we are concerned about ensuring quality, protecting the organisation and adding value, but really we just love to find fault. The trouble is, those mean, tight-fisted bean counters won’t give us unlimited budget, the pinstriped suites in Corporate keep reminding us we have contractual dates to meet and those frilly young things in marketing have gone and told the world that ‘WonderWidget’ will be demonstrated at ‘such and such geek show’. Given the limitations above, a risk-based approach to testing can help ensure that we get the biggest bang for our buck. Risk-based testing is not new or difficult, it’s something we do all the time when writing tests, but often don’t realise. It’s a case of asking, “Where is the software most likely to fail, and what will be the consequence if it does?”

Time is a requirement too One of the big challenges for testers who adopt a risk-based approach is to accept that time to market is a requirement too. In fact it may be the most important

T.E.S.T | October 08

Risk-based testing | 7

Probability + Impact = Exposure, ie: Risk System won’t cope with number of users

Probability Impact Exposure =

Converting Name field from 15 characters to 25 may cause data loss 1 + 5 =

Transmitting confidential data between 16 different systems 5 +

requirement for a particular project. If we have to meet a deadline, it’s no good saying we still have 100 tests to run, we have to meet a deadline. We have to be able to say, “We have 100 tests left to run, however the risk associated with those 100 test is X, therefore if you release now the possible consequences are Y.” We have to have to be able to say, at any given stage, “These are the areas we identified as high-, medium- and low-risk, these are the tests we have run, these are the results, and based on these it suggests that, blah blah blah.”

Risk factor The key to running a successful risk-based test cycle is to correctly identify, assess and address the risks associated with the software. The process follows five stages:

1. Risk identification Understand the risks by meeting with the relevant experts from both the technical and the business communities and prepare a register of risks. Workshops, document reviews, past projects data can all be used to feed into this activity.

2. Risk analysis Not all risks are equal, to ensure that testing addresses the high exposure risks

first, relevant experts and stakeholders need to meet to discuss and analyse the risks. Each risk is assigned a probability and an impact, this activity involves all parties, as sometimes, for example, IT staff may not realise that a low technical risk, might represent a high business risk. For each risk identified ask the question, “How likely is this to happen?”, and assign each risk the appropriate score based in the agreed answer, ie: certain to happen, highly likely • A=lmost 5 his is probably going to happen, we • Tthink it is likely = 4 N ot sure • is 50:50 =this3 will happen, we think this his probably will not happen, • Tunlikely =2 he chances of this happening are a • Tmillion to one, this is very unlikely = 1

The trouble is, those mean, tight-fisted bean counters won’t give us unlimited budget, the pin-striped suites in Corporate keep reminding us we have contractual dates to meet and those frilly young things in marketing have gone and told the world that ‘WonderWidget’ will be demonstrated at ‘such and such geek show’.

Again, for each risk ask the question, “If this happens, what will be the impact on the business?” and assign the risk the appropriate score, ie: It would be critical, the business objective could not be achieved, we would suffer immense loss = 5 It would be severe, the business objective would be undermined, we would suffer significant loss = 4

• •

October 08 | T.E.S.T

8 | Risk-based Feature testing

t would be moderate, the business • Iobjective would be affected, we would suffer some loss = 3

t would be low, the business objective • Icould still be achieved, we would suffer minor loss = 2 It would be negligible, there would be no real impact on the business objective, we would suffer no loss = 1 Having given each risk a Probability and an Impact score out of five, the exposure that each risk represents can be clearly seen and agreed.

•

3. Risk response Agree the appropriate response to each risk. It may be that not all risks require a test written, there might be another mitigating action that is more appropriate. Formulate test criteria for each risk with clear objectives and pass/fail conditions. To help with this ask the following questions: What would cause this risk to materialise, what data, action, circumstance or event would need to be in play? In which area(s) of the system is this most likely to occur? Who would be the most likely to experience the consequence of this risk, and when and why?

• • •

T.E.S.T | October 08

types of testing, techniques, or • Wtoolshatwould be best to expose this risk? Document the requirements for each test, such as data, environments, time scales, business resource.

4. Test scope definition Agree the scope for the testing, which risks are in and which are out of scope, what the test schedule will be, who is responsible for what, and what the minimum success criteria is for each set of testing. It is very helpful to have agreed both generic quality or test objectives for the system (eg demonstrate that contractual requirements have been met) as well as specific risk-based quality or test objectives (eg demonstrate secure data transfer between system A and system B). Produce a test coverage matrix, based not on functional requirements, but on identified risks. Agree what reports will be produced, and what information should be included. Agree who these reports should go to and how often. The main purpose of risk-based test reporting is to be able to show which of the identified risks have been tested and which have not yet been addressed. It should provide the reader with the answer to the following

question: “If I release now, what is the danger of any of the identified risks becoming an actual issue?”

5. Test phase Testing will now follow a traditional test cycle, (preparation, execution, reporting) however the focus is now not on testing each individual function, but on running the tests that exercise the areas of risk identified, of course concentrating first on the high exposure risks.

Give it a go I believe that risk-based testing can offer significant return on investment for many organisations, directing the test effort to where the pain is most likely to be felt. I believe it can empower the business decision makers, giving them the information they really need. It tells them not how many tests have been run, and how many bugs have been found but what are the likely consequences to the business if the code is released in its present state. I would encourage anyone that finds themselves either with too little budget or two little time to do all the testing they want to do to consider piloting a risk-based test strategy to discover if it can benefit their organisation.

Tony Simms Principal consultant Roque Consulting www.roque.co.uk

10 | Sponsored feature Feature sponsored by

Riding the rollercoaster The planet and the economy are in perfect harmony – in meltdown together. The air that we breathe, the water that we drink, the homes that we treasure, that little treat, all essential for our wellbeing, all seemingly now in jeopardy. Angelina Samaroo asks what this means for software testers?

o attempt an answer to the question of how the turmoil in the world will affect software testers we must first find our place in the world. Many say that we work in information technology; many say (quite strongly) that we work in engineering. In schools, we talk about ICT, information communication technology. Venturing a little deeper, we find that we work in software and hardware. But we do not work alone. They can (and must!) hear us scream. They being the designers; the business analysts; the developers; the technical architects; the users; the management; to name but a few.

The development lifecycle In software development, testing is a major activity. To fully understand its reach, let us take a small step back, and look at a software development lifecycle. Each arrow represents a testing activity. Each design document is tested against its predecessor; the code is tested against the designs; the product is tested against the customer needs. This is a rather tall order for one community. We don’t all do it all the time, but we do know that it should

T.E.S.T | October 08

be done. In making all of these arrows a reality, we would put the engineering back into software. Note that we are not suggesting that all systems require all of these designs, but we should not wholly ignore the value of thinking (designing) before coding. However, despite many projects foregoing all that pomp and ceremony, the world of IT isn’t that bad. On balance, many more systems work than those that don’t. IT is here to stay, and its influence is growing. We want it, we need it. We work with it, we play with it, and we live by it. In these uncertain market conditions, the question of whether the butterfly’s flapping of its wings can cause a wind of change the other side of the world has been answered – yes it can, we have the technology – ICT – information communication technology. Your antics on Friday night in Amsterdam can cause you more than a headache at your interview in London on Monday morning. Reactions to the stock market can take place even quicker, with impressive results. The media can tell us as it happens, and we can feel gloomy then and there. We

The world of IT isn’t that bad. On balance, many more systems work than those that don’t. IT is here to stay, and its influence is growing. We want it, we need it. We work with it, we play with it, and we live by it.

Sponsored Feature feature | 11 Feature sponsored by

Customer need

t Formal capture of needs

t High level design

t Low level design

t Code production

t Product

October 08 | T.E.S.T

12 | Sponsored feature Feature sponsored by

can then not go out and not spend any money. Needing air, we can then take ourselves off to the bank, and withdraw our cash – all whilst the media is watching – then tomorrow, or even while standing in line (thanks to even more technology), we can see the effects of our actions, combine it with what the rest of the world is doing, then react some more, and hey presto, we feel even worse. The financial world is in recession – and they’re taking the rest of us with them. If they can’t trust each other from the inside, what chance do we have from the outside? The dramatic moves by governments across the world in pumping scary sums of money into the economies may well serve to prevent us all from plunging into recession, technically. Those who have already lost their jobs or feeling more than a pinch may take umbrage with the definition however.

Hanging on for a rough ride While the economy has caught a cold, it will recover, history tells us this. We seem to move in cycles, up, down, then up again, so we need to hold on while we ride the rollercoaster. Undoubtedly sectors will suffer; we already know about finance and housing and holidays and fancy cars seem to be too decadent, for now. Government projects may well take a hit, politicians must act quickly and not for tomorrow, for today, headlines matter, those costs must be seen to come down. Demand for jobs may well outstrip supply. Consultants and contractors may be looking for cover during the bumpier rides, increasing the supply of full time labour to companies. The good news is that business-as-usual will continue, customers may pull back, but they will not

T.E.S.T | October 08

become extinct. The trick will be to find them, and to offer them what they cannot do without. The Olympics will require a multitude of systems, from media, to transport, to registrations, to scoreboards, to security, amongst many others. Having said that history tells us that economies recover, it does not give us that much confidence in the planet. Will it recover? Has it caught a cold, or should it be on dialysis? Innovation in this area, green energy, increasing water supplies, enhancing food production, reducing green-house gases without taking cows out of the food chain will be essential. Investment here may take a tumble, but it is unlikely to be for too long. A continual issue reported in the UK professional magazines, such as those from the Institute of Engineers and Technicians (IET), the Royal Aeronautical Engineering Society (RAeS), and the British Computer Society (BCS), is the lack of suitably skilled resources for work in IT (and other engineering professions). The shortfall is unlikely to be the same as the reduction in the number of jobs. We will not have a zero deficit.

A call to arms Thus, the call to arms in IT will continue. If you’re already in it, then you need to find your X factor, to ride the waves. If a consultant can come in and do your job better than you can in a matter of days, fix it! Become a domain expert, look around you and work out how and where you fit into the overall business, and become indispensable, or at least give it your best shot. Become a testing expert (or seriously hot programmer, or keen business analyst, or project manager extraordinaire). Start

with your own processes. Are they good enough, or can they be improved? Search the net, buy a few books (or better yet, get your boss to buy them). Learn from others around you. Consider whether formal training, in-house or externally supplied can help. If you’re thinking of changing companies at any time, or looking for a promotion, then seeking a recognised qualification may be a wise investment. There are a few on offer. In the old days, professional qualifications were offered by professional bodies, with the necessary gravitas, governance and regulatory control – schools, universities and learned bodies. As part of the new economy, it seems that qualifications can also be offered from any organisation popping up, largely in cyberspace. Initial research into this phenomenon shows that some are notfor-profit, and have noble ambitions, but the questions of why and how they will make this happen are yet to be answered with any confidence. Provision of professional qualifications is a serious business. It cannot and must not be taken lightly. Neither can it be simply willed into existence by good intentions, or sustained hype. It has to be part of the learned fabric of society, like GCSEs, A-levels and university degrees. The UK’s IET, RAeS and BCS have all been in existence for decades (since the mid 50s), and have survived the economic cycles since inception. They are all appointed by Royal Charter, and ratified by the Engineering Council. Their membership is in the hundreds of thousands, their investment worldwide, their conferences and lectures held routinely. Their offices are representative of their standing, when you walk in you feel the weight of your

Sponsored feature | 13 Feature sponsored by

membership. Your qualifications from them are undisputed the world over. A qualification is like any other investment, what we need to consider is how long we need it to provide a return for us. Is it for life or for hype?

Tried and trusted As we head into difficult times, we could argue that the tried and trusted should become our trusted, we must now dispense with the trying. We all need the comfort of the big, the bricks and mortar, the known. Putting your money in an internet-only bank now would put you in the top percentile of risk takers today. Leaving your job in fear, before you know what is actually going on, could be like trying to navigate the seas by the stars that you do not see, not the ones that are actually there, the course plotted as a result could lead you to dark and lonely places. Jumping out of the front line of contracting back into a permanent job, may keep you warm for a little while, but it could mean that you lose your birdseye view. You may not spot the wave until it actually hits, then you’re in with the masses. Conversely, jumping into contracting to get that birds-eye view (and possibly increase your salary, very tempting right now) may mean that you fly too high too quickly and your tentative wings could let you down. The downturn could present an opportunity for you and your organisation to get your house in order, to work out root causes of previous issues, to spend time reflecting and talking to each other, and to build solid offerings for the upturn – when it comes, no excuses, you should be ready.

Angelina Samaroo MD Pinta Education www.pintaed.com

Thus, the call to arms in IT will continue. If you’re already in it, then you need to find your X factor, to ride the waves. If a consultant can come in and do your job better than you can in a matter of days, fix it! Become a domain expert, look around you and work out how and where you fit into the overall business, and become indispensable, or at least give it your best shot. October 08 | T.E.S.T

14 | Agile testing

Delivering with agility Stephen Walters, product manager LQM at Borland tackles the challenges of implementing agile processes in the software testing environment and how to integrate agile with automation.

T.E.S.T | October 08

Agile testing | 15

Those companies that have already deployed automated testing tools in traditional projects run into other challenges when making the agile transformation. For example, most software testing products force companies to operate in a specific way (such as waterfall) rather than an agile, companyspecific process.

ore and more companies are moving to agile software delivery approaches. But agile delivery brings with it a new set of challenges; among them, functional test automation (opinion is currently divided in the agile community on the value of automated testing). However the reality is that software teams must manage quality if they are to avoid operational risk. Businesses are encouraged to adopt an agile process that fits their unique needs. Yet, nothing about agile is pre-determined or dictated; so it’s no surprise that businesses find it challenging to implement as a trusted business process. And using tools to automate their agile delivery process is another variable that further complicates the decisions organisations must make. In the world of agile software delivery, requirements are raised, changed, and reprioritised in much shorter timescales. At the same time, the functionality that you deliver must be fully tested to ensure that it meets the needs of endusers and the business. As software volume and complexity increases, it’s clear that manual processes will fail to deliver fast and repeatable results that underpin the agile proposition. Inevitably, without automation, quality will suffer as testing is marginalised to achieve on-time delivery.

automated (and as reasonable) as possible because of regression burden.” The issues facing organisations implementing agile testing automation are: Operating agile and traditional testing methods side-by-side; Identifying tools that are usable and flexible; Allowing non-technical and non-testing specific members to actively contribute; Enabling open-source integration; Enabling test-driven, or model-driven, development.

A new way of working

•

Regardless of whether your development team is considering going agile, making the transition, or already operating in an agile mode, it’s critical to consider how technology can support this new way of working – particularly in the area of testing. According to independent research company Forrester; “there’s room for improvement in agile processes – for example, through better support for agile from mainstream tools for test automation, test management, and requirements management.” Forrester notes that, in an agile process, testing is continuous and imperative, as features are not ‘done’ until all associated test cases pass. Moreover, “Testing has to be as

• • • • •

The role of automated testing in agile development Let’s qualify why testing needs to be automated in an agile delivery environment. The people involved in testing are part of the delivery team – not an isolated group that developers hand code to as a final step to release. Ideally, they sit side-by-side with the developers, who, as they create code, pass it to testers early in the process for evaluation against acceptance criteria. Since capability is built iteratively, and the team needs to maintain velocity, the code assets need to be verified quickly. For agile to truly succeed, functional testing has to be quick, iterative, and responsive. Specifically, automated functional testing offers a number of efficiencies that cannot be achieved with manual testing. For example, it can: Greatly accelerate the code-and-test process by supporting fast, automated test scripts; Ensure the repeatability of tests, to ensure regression testing from sprintto-sprint, iteration-to-iteration; Enhance test efficiency further via robust, yet flexible, test management processes; Avoid the inherent inaccuracies that manual processes inject in to the process – particularly when time is tight.

• • •

The need for speed Automation enables testers to create simple, reusable scripts, which they can deploy to save time and increase the

October 08 | T.E.S.T

16 | Agile testing

consistency of testing across similar user stories, story points, or requirements within and across projects. They can be run rapidly and iteratively, developed from the user story to drive the functional capabilities, enabling test driven development. It significantly lightens the workload of testers and eliminates the need for late night and weekend testing marathons that can burn teams out.

The need for repeatability Regression testing requires that 1) you perform the same tests each time you test a particular piece of code and 2) that the test is scripted against the acceptance criteria of each respective user story. Whenever code changes, (or is extended to include new capability) you need to re-run all functional tests for all user stories, up to the latest change, to ensure other user stories weren’t impacted inadvertently. (With agile development, regression testing should typically be performed at the end of every iteration – in some cases this means daily). Repeatability is nearly impossible to achieve with manual testing due to human error, variability, and inconsistency. People simply can’t remember exactly which tests they performed for each piece of code for the last iterative cycle – and even one oversight can cause problems in final code. But with automated, repeatable functional testing and regression testing, you can execute tests consistently whenever you need to.

Choosing the right test automation tools We’ve proven through our own agile transformation experiences that it scales effectively when it’s supported by the right mix of people, process and tools. While many may argue that automation has no place in agile, the realities of geographic separation, multiple teams and limited resource dictate a more pragmatic approach. Automated functional testing and centralised test management play a vital role in helping development teams implement an agile delivery process. But the majority of businesses today continue to use manual processes, which can make it difficult, if not impossible, to implement and scale agile methods

T.E.S.T | October 08

We’ve proven through our own agile transformation experiences that it scales effectively when it’s supported by the right mix of people, process and tools. While many may argue that automation has no place in agile, the realities of geographic separation, multiple teams and limited resource dictate a more pragmatic approach.

FREE ANNUAL SUBSCRIPTION

To qualify for this offer please download a subscription form from www.customermagazine.net quoting reference CMFREE1108 or email subscriptions@31media.co.uk

INSPIRING CUSTOMER CENTRICITY www.customermagazine.net * This offer may be withdrawn at any time and free subscriptions are subject to our terms of control and are at the publisherâ&#x20AC;&#x2122;s sole discretion

XXX NFEJB DP VL

18 | Agile testing

As software volume and complexity increases, it’s clear that manual processes will fail to deliver fast and repeatable results that underpin the agile proposition. Inevitably, without automation, quality will suffer as testing is marginalised to achieve on-time delivery.

within an enterprise. Manual testing slows down production, and is error-prone – it does not support re-use and provides no predictable repeatability. Those companies that have already deployed automated testing tools in traditional projects run into other challenges when making the agile transformation. For example, most software testing products force companies to operate in a specific way (such as waterfall) rather than an agile, companyspecific process. Interestingly, the story comes full circle. As a vendor of software testing products who, two years ago, decided to undergo the enterprise agile transformation, and now carries out over 60 percent of software projects using agile methodologies; we needed to adopt some changes. It was imperative that our automated testing products were aligned to helping organisations implement and enhance an agile delivery process while ensuring quality, reducing risk, and lowering costs. Employed by thousands of enterprises worldwide, Borland’s SilkSuite has also transformed, and continues to develop, to best support development team’s preferred processes, whether waterfall, iterative or agile whilst enabling automated testing that is rapid, repeatable and accurate. We know this because, in our agile development environment, we run automated testing on our automated testing tools.

T.E.S.T | October 08

Stephen Walters Product Manager LQM Borland www.borland.com

Complete implementation of ITIL in 12 weeks?

Optimal service management with BMC Alignability Alignability is the newest innovation in service

for the employees. The Alignability model fills the

management systems, which may significantly reduce

gap between the ITIL framework and your service

the time needed for an ITIL implementation project from

management tool.

12-18 months to 12-14 weeks.

In the Alignability Process Model the experience gained

ITIL gives guidelines to organisations who want to

from over 150 companies in 30 countries, is distilled.

define their service management processes. However,

It is a practical instrument that has been developed

processes are not enough; they have to be completed

over the past 10 years. We can now safely say that

with the details of how to carry out those processes

when no changes are made to the processes they are

into work instructions to give a real practical advantage

guaranteed to work.

Interested? Contact InfraVision at: InfraVision Ltd Delegate House, 30A Hart Street, Henley-on-Thames, Oxon, RG9 2AL

T: + 44 (0)1491 635340, F: + 44 (0)1491 579835, info@infravision.com, www.infravision.com

20 | Event preview

EuroSTAR 2008 – The future of software testing on show Bob van de Burgt, programme chair of EuroSTAR 2008 tells us what’s in store at the EuroSTAR 2008 international conference that takes place November 10th-13th in The Hague, Netherlands.

his is the 16th year of EuroSTAR - Europe’s unique independent international conference. I am really excited about the programme for this year’s conference, which is themed: ‘The Future of Software Testing’. The conference will ask some challenging questions such as: How can we use our capabilities in such a way that we can flexibly respond to the context we are working in? How is business going to change over the years? How is the IT function going to respond, and what role can we as software testers play? Where do we go from here and where do we need to be in about five years? The programme covers a wide range of topics that should appeal to delegates at all levels of experience from novice to expert. We have presentations from introductory to advanced level

T.E.S.T | October 08

from industry experts, experienced practitioners, some new speakers and some old favourites. The conference programme will present a vision of the future for software testing that will be shaped by the experts and practitioners through five great worldclass keynote sessions. James Whittaker will present his vision for what testing will look like in five to ten years. The tools required to do it right, the technological innovation necessary to get there, and a sneak peek into Microsoft’s incubations in this area, which are the subject of this futuristic view of the discipline. Bharat Mediratta and Antoine Picard will present how Testing on the Toilet, a weekly flyer at Google that is posted in every Google restroom, can bring better testing to your organisation. Randall Rice will explore major trends that are expected to emerge

in business and information technology, and will discuss how they may impact software testing in the foreseeable future. Erik van Veenendaal will bring the TMMi Foundation to EuroSTAR and this is your chance to learn about the latest initiative in test process improvement. James Lyndsay will give a flavour of what it is like to bring a test perspective into an agile project - and what it is like to be on an agile project that has a sudden need for testing skills. Before the conference formally starts on Tuesday afternoon, we have seven fullday tutorials and seven half-day tutorials presented by professional trainers. The body of the conference is made up by some 37 track sessions with presenters from 14 different countries covering the complete range of software testing topics about the

Event preview | 21

The conference programme will present a vision of the future for software testing that will be shaped by the experts and practitioners through five great world-class keynote sessions. future of software testing, testing become involved in the international in agile environments, model based standardization process with the dual testing, outsourcing, case studies, test aims of both producing a better standard environments and many others. and gaining greater consensus within the I can guarantee there is something in testing community. A true chance for this programme for everyone. Within EuroSTAR delegates to shape the future these sessions are four mini-tracks, of testing. which give an opportunity for less During the conference three special experienced speakers to make their workshops will create a statement for first contribution to the profession by the testing community worldwide, about sharing their experiences. the future of software testing. This will In addition there are two participatory take the form of a Manifesto that gives workshops. Michael Bolton will report the major guidelines for our journey into on research into heuristic reasoning. the future. The workshops will investigate He'll show the ways in which heuristics what we can learn from past experiences are central to engineering, testing and that can benefit the future of software development and show how heuristic testing and will create route maps for the approaches are fundamentally aligned next five years. The manifesto workshops with agile principles. Stuart Reid will will be lead by two highly recognised present the current state of the new experts in our profession, Dorothy Graham ISO 29119 testing standard. This and Isabel Evans. Delegates can join them workshop will be a collaborative forum and be part of this ‘Future’. The Software target-ad-small:Layout 1 to4/9/08 16:25 Page providing an invaluable opportunity Testing Manifesto will 1be explained in

Bob van de Burgt Programme chair EuroSTAR qualtechconferences.com

more detail in the next issue of the STAR Tester. This programme is the result of 422 submissions from over 25 countries. I would like to thank all who have submitted and I’m sorry for those we had to turn down as we only had space for ten percent of the submissions. We tried to make the process as fair as possible and therefore, this year the programme review committee worked in ‘double blind’ mode so no-one knew who the application was from. Besides that we applied rules about company and country distribution. I would like to thank all members of the review committee for scoring all the proposals and the programme committee members, John Fodeh, Graham Thomas and Iris Pinkster for their help in creating this programme. Apart from the conference itself, this year EuroSTAR hosts the world’s largest Test Tools and Services Exhibition – a great opportunity for you to meet and quiz the vendors. On Thursday 13th November, EuroSTAR will host the Gala Awards Dinner in The Grote Kerk where the 10th European Testing Excellence Award will be presented. I look forward to welcoming you to my hometown, The Hague, in November!

Software Testing Professionals As a committed IT professional working in the rapidly expanding Software Testing industry, you know how important it is to keep up to speed with the latest developments. Keeping your career on the fast track is equally important. When you join BCS you’ll be doing both. Joining the UK’s leading body for IT professionals gives you the support of a wide network of like-minded individuals and immediate access to services and benefits keeping you ahead of the field at every stage in your career.

Achieve professional recognition and the rewards that go with it Visit www.bcs.org/softwaretesters or call us on 01793 417 424 BCS IS A REGISTERED CHARITY: NO. 292786

October 08 | T.E.S.T

22 | The future of testing

Predicting the future of software testing With changing trends in the industry, software testing has to change too. An increasing emphasis on the four Rs: repeatability, reliability, reuse and robustness, has opened a new path to testing, where it will need to change to accommodate trends and become a business-led activity. Ian Londesbrough, principal solutions engineer at the worldâ&#x20AC;&#x2122;s largest independent testing, quality management and certification solutions company, AppLabs, charts a course into the future.

T.E.S.T | October 08

The future of testing | 23

rends in the industry suggest that software testing in the future will look very different to how it does today. Businesses are leading change and transformation projects and application development is changing to support it. Current trends include: web based applications; service oriented architectures (SOA); software as a service (SaaS); wireless and mobile technologies. Each of these trends is further complicated by a more agile approach to software development and an increasing emphasis on the four Rs: repeatability, reliability, re-use and robustness. In order to drive these changes in effective and efficient application development of quality systems, more emphasis must be placed on improving the analysis and design phases and ensuring early engagement of testing in these areas. To date, analysis and design have been people-based activities with business analysts and systems architects producing a combination of natural language, models and diagramming techniques (such as data flow diagrams, logical data structures, logical data models and process data diagrams) and some structured language techniques (such as use cases and UML). This has led to business requirements and functional

designs which do not meet the four key criteria which make the resultant systems testable, clear, assumption-free, unambiguous and complete. In the future, if we are to enable the production of quality applications, in less time and at lower cost, we will see more structured and more automated analysis and design techniques, with static testing (reviews, inspections and walkthroughs) becoming ‘robotised’ and built in as part of the analysis and design requirements capture tools. This will result in clarity, the eradication of assumptions and ambiguity and a greater likelihood of completeness which will in turn enable: - Executable specifications - Auto generated code and systems configurations - Model-based test generation - Model-based bug prevention - System simulation - F ault tolerance developing into self-testing, self monitoring, self healing software - Performance optimisation ‘designed in’ and ‘on the fly’ So what will it mean for testing and QA? Well the products should be more reliable, reusable and robust. Automatically generated test scripts with full branch and decision tree coverage at the early test execution stages; unit test and

In the future, if we are to enable the production of quality applications, in less time and at lower cost, we will see more structured and more automated analysis and design techniques, with static testing (reviews, inspections and walkthroughs) becoming ‘robotised’ and built in as part of the analysis and design requirements capture tools. This will result in clarity, the eradication of assumptions and ambiguity and a greater likelihood of completeness. October 08 | T.E.S.T

24 | The future of testing

‘integration in the small’, combined with the development of fault tolerance into self-testing, self-monitoring, selfhealing software will result in defects being found and eradicated earlier in the lifecycle. As a result, fewer defects should be found at the system testing and ‘Integration in the large’ stages and businesses will be able to concentrate on acceptance testing (user, operational and performance) in a fully integrated target environment and do full end to end testing of business processes and model office scenarios. In this way, businesses will be able to leverage their investment in technology to realize true business benefits and competitive advantage through creative integration and deployment of reliable, re-usable, robust components. The way in which the components are integrated and deployed will be the differentiator, not the components/systems themselves. In turn, this will enable business to concentrate on business propositions, and go to market services and products, supported by the ability to deal with high volumes, provide excellent customer service and quickly enable high margin/low volume services etc.

Testing robots? What about professional testers? Software testing in the era of mainframe systems was merely executing a series of steps manually and punching various key strokes as an attempt to break the application. This was followed by the event-driven client-server environment in which applications were deployed on multiple hardware platforms in conjunction with various other applications. As a result, the scope of testing has increased as failures and unexpected behaviours have become more prominent. The recent development of web applications, where multilayered designs were architected in order to hold complex business logic, which are to be supported on numerous browser types and various platforms and different network structures, further increases the potential areas of failure. In addition, web applications say something about the integrity of business operations to a large audience of external users (ie customers and business partners) and can have a major impact on brand and reputation. New and emerging technologies, like SOA are significantly changing the

T.E.S.T | October 08

trends in the application testing. SOA in particular has major implications for how tests are designed and performed. SaaS also has a totally different view of software testing. We can conclude that new and more complex technology/ architecture creates greater risk. So, how will the new approaches postulated in the predictions in the previous section of this paper manifest themselves and what will it feel like to be a test professional in an increasingly automated and robot-driven software engineering environment?

Testing mitigates risk and despite automation and robot-driven testing techniques, the testing professional will continue to be pivotal in the software engineering lifecycle. Working in multidisciplinary teams alongside subject matter experts, business analysts, systems architects and end users they will continue to play the roles of; arbitrator, mediator, translator and negotiator between the other parties, ensuring that their ideas are not only effectively specified and designed, but that the criteria required for effective quality and testing are captured

The future of testing | 25

and articulated into the models and tools so that tests are accurately generated by the â&#x20AC;&#x2DC;robotsâ&#x20AC;&#x2122; to verify the design and validate the requirements. In this new world, the professional tester will concentrate on adding value to the business and the software development lifecycle and will operate in a new and elevated position where the combination of their structured processdriven approach, creativity, destructive inquisitiveness and ability to articulate and define quality and testing criteria into the models will be critical to the success of the business implementation of technology.

The growing importance of testing Change is a given and we can be sure that it will come in larger amounts and at increasing pace. No industry, least of all software engineering, is immune to change and therefore none is immune to risk.

The changes in software development will require software testing to develop in parallel. Of one thing we can be sure, the role of the professional tester will become more interesting and more essential to both the definition of model-based testing criteria and the facilitation of acceptance testing (user, operational and performance), testing of business processes and model office scenarios alongside subject matter experts and end users and ultimately realising business benefits. Testing and QA will become more important and add more value as we move into architectures and technologies that support businesses in their goals of bringing products and services to the market as rapidly as possible, with minimal risk. We will see a shift towards testing the transformed business operation, the business processes, the way people interact with the systems and processes and the information it provides, and therefore mitigating the risks and increasing the benefits of business change.

Ian Londesbrough Principal solutions engineer AppLabs www.applabs.com

In this new world, the professional tester will concentrate on adding value to the business and the software development lifecycle and will operate in a new and elevated position where the combination of their structured process-driven approach, creativity, destructive inquisitiveness and ability to articulate and define quality and testing criteria into the models will be critical to the success of the business implementation of technology.

October 08 | T.E.S.T

26 | Training

Can you afford not to train? In the current economic climate, when many companies are feeling increased budget pressures and organisations are naturally trying to reduce their spending, Tom Millichamp, company director of Edgewords, warns companies not to make the expensive mistake of freezing their training requirements. The question is not whether a company can afford to train, but rather can you afford not to?

T.E.S.T | October 08

Training | 27

ll too often businesses make the mistake of thinking that not spending, directly equates to saving money. However, a test team that is not properly trained to do its job is at best not working to its optimum capacity, and at worst a financial disaster waiting to happen. Within the testing industry, the cost of inadequate training can be felt in a variety of ways. It may be that a project simply overruns, incurring additional cost in manhours. Or, a lack of training may result in defects being missed in the software, exposing the business to the risk of software failure. This would invariably lead to escalating costs to fix these defects, either later in the development life-cycle, or worse still, post implementation. During the past decade, testing has matured into a recognised and respected profession. Rather than drafting in developers and end users, organisations recruit dedicated testers to form independent test teams. Testers are now an integrated part of the software development life-cycle, involved from the very beginning with the writing of software requirements. If the test teams have not had suitable training, how are they expected to understand what makes a good requirement or, if there are errors in these requirements? As these requirements are used as the foundation for developing test scripts, errors at this stage will result in errors in the tests. The cost of not detecting faulty requirements at this early stage becomes much more significant later on. For each software requirement there may be hundreds of lines of code, many tests cases, pages of user-help and other documentation; if one requirement has a defect, the impact can ripple out and affect numerous other entities, each carrying an escalating cost to fix.

The right skills If testers are not equipped with the right skill set they can easily miss defects, and this will be costly to the organisation. The further along the development life-cycle a defect is detected, the more expensive it becomes to fix. Worse still, if a defect

Within the testing industry, the cost of inadequate training can be felt in a variety of ways. It may be that a project simply overruns, incurring additional cost in manhours. Or, a lack of training may result in defects being missed in the software, exposing the business to the risk of software failure. This would invariably lead to escalating costs to fix these defects, either later in the development lifecycle, or worse still, post implementation. continues into the live application and the application then fails, the costs are even higher, as well as financial losses, there will be damage to the testing company’s reputation, maybe legal action, possibility even death. Companies should take warning from a case in 2006. A third party company’s software was used by the London Underground, and unfortunately a failure in this software caused the closure of the Central Line during rush-hour. This caused further knock-on delays throughout the week. This company was then threatened with a fine of over £1million from London Underground. In addition to this, numerous London businesses incurred costs from employees not arriving in to work on time. Combine this with the loss of customer confidence and the cost incurred having to fix the problems in a live system and it clearly shows what may begin as a relatively small software problem, can easily escalate into huge financial and reputational losses. Had the testers been well trained, the defects which caused the central line closure may have been detected earlier, preventing the subsequent problems from

arising. The final costs of the disaster are almost immeasurable; compare this to the modest cost of training the testers properly in the first place!

ISEB certification Thanks to schemes such as the ISEB certification, there is now an accepted formal training structure for learning the key foundation skills of testing. Training in automated testing is more specifically tailored to users’ requirements and training providers are fewer. Automated testing tools are not cheap: licences, maintenance agreements, external consultancy, the use of contractors and of course the training, all mean that automated testing is a major investment for any organisation. To ensure a return on this investment, these tools must be used correctly and to their maximum potential. If a test team is not properly trained in the tools they are using, they will quickly become frustrated and that expensive piece of software all too often becomes ‘shelfware’.

October 08 | T.E.S.T

28 | Training

Automated test engineers who do not fully understand the tools they use cannot develop good automated tests. Once again, defects will be missed, scripts will not be robust and will need a high amount of maintenance and re-work as time progresses, results reporting will be inaccurate, all of which will result in higher costs to the project. Many companies adopting automated testing tools also rely on external skills from consultants and contractors to give them a ‘quick start’. During times of recession, these external resources are often dramatically reduced, or removed, so it is imperative that the permanent test team can cope with the understanding and maintenance of the automated scripts. Training an in-house test team in the tools should be seen as an essential part of the entire automation package.

Training the managers A recent, and very positive industry trend, is to also train the test managers and project managers in the automated tools. This makes sound business sense, as without training, how can people in these positions effectively plan for test automation? How can they assess how long the automation will take, or understand what can and should be automated, as well as what testing should not be automated? All too often managers see automation as a black art and rely heavily on information from the external automated test ‘experts’ for planning. Now how accurate would that nformation be from external consultants and contractors who are looking to ensure their on-going daily rates? Some managers fear that as soon as they train their testers in test automation, these testers will leave to go contracting. This is an extremely short sighted and risky viewpoint. It must be remembered that employees see training as an essential part of their remuneration package. It is part of the company’s investment in that individual, and the more that is invested, the more empowered the individual feels and will then in turn reward the employer by being more energised, more productive as a result of their increased skill set, and offer a higher degree of loyalty to them. Research shows that a five percent rise in the training rate increases gross value added per employee by four percent and employee retention rates directly increase with training. Fully trained employees will always be much more productive for a business,

T.E.S.T | October 08

lowering a company’s overheads in the longer term, while also greatly enhancing its reputation. Professor Mike Campbell director of research and policy for the UK Commission for Employment and Skills, recently stated: “The failure rate among companies that never gave their staff any training was 2.5 times higher than those that did.”

Tackling the crunch When competition is at its fiercest, such as during a credit crunch, having the best and most productive employees gives a company that all important leading edge. The company which has the highest trained employees will undoubtedly be in a better position to meet the needs of its customers and provide the better product. This will increase reputation and, therefore, increase custom. As we continue to operate in a negative economic climate, it is more important than ever that organisations ensure their permanent testing teams have the capability to perform all testing requirements; this is particularly important for businesses that are trying to reduce their dependency on more costly contractors. Professional, expert training can at first appear to be an unwanted expense, but when considered against the cost of not training: • cost of external skills: consultancy, contractors • cost of missing defects early in the project • cost of failure or project delays due to under-trained staff • impact on customer service, both internal and external • cost of losing staff due to lack of skill & career progression • cost of tools being used incorrectly or becoming ‘shelfware’ it is a small investment for maintaining a solid business. To ensure that investment is well placed, and for training to be of real value, using the best, most suitable training provider is crucial. The following questions should always be asked to help achieve this: • Are the training provider’s trainers of the right quality? • Do the trainers have working backgrounds in what is being taught so that they can provide real-world examples and answer specific industry questions that delegates may have? • Can the training be provided on-site to reduce costs of staff expenses? So next time you question whether you can afford to train, ask yourself, whether you can afford not to train!

Tom Millichamp Company director Edgewords edgewordstraining.co.uk

Prevent Data Breaches from Lost or Stolen Computers with ComputraceOne

Absolute Software’s ComputraceOne Helps Fight Computer Crime “I was really impressed by the recovery process – I didn’t have to chase Absolute Software at all. They informed me when the laptop started ‘calling in’. Not long after, I had a call from the police telling me the laptop had been recovered. All we had to do was let Absolute know when one of our computers went missing.” – Adrian Marks IT Systems Administrator BRC Reinforcement

oday’s computing assets contain more sensitive and valuable information than ever before – making each computer a potential liability without proper protection. Organisations risk legal and public relations nightmares when even one notebook goes missing. Computrace®One™ notebook security and tracking software helps ensure regulatory compliance by protecting data, tracking hardware and users, providing auditing capabilities and acting as a historical record of computer assets and their use.

Download the free whitepaper: “Compliance. Protection. Recovery. A Layered Approach to Laptop Security” and learn how to: t Track & recover lost or stolen computers t Delete data remotely to protect confiden tial information t Detect unauthorised software & hardware t Accurately inventory computers regard less of their location

Download the essential laptop security whitepaper at: www.absolute.com/vital

www.absolute.com/emea | +44 (0) 1635 30424

30 | Quality asssurance

T.E.S.T | October 08

Quality assurance | 31

Anarchy in the QA There is unrest brewing in IT QA departments all over the globe and a revolution that is gaining momentum, and is forcing people to re-think their attitude towards software test automation. Colin Armitage, CEO of Original Software is storming the barricades.

here is unrest brewing in IT QA departments all over the globe. A frustration borne out of a lack of recognition within their own organisation as to the importance of software quality, the potential competitive advantage their company could enjoy if they get software quality right, and the major negative impact that can result from poor software quality processes. On top of this, the major proponents of software test automation have repeatedly failed to offer a reliable solution, thus those QA departments that have been lucky enough to secure funding to improve their software quality through automation, have been let down by their vendors (often multi-national giants that really should know better), and have often failed to deliver positive return on investment. More often than not, even after substantial investment in software test automation, these QA departments are no better off than they were before they spent the money. No wonder then, that a recent survey of CIOs found that only six percent were totally happy with their automation investment. These weren’t small town respondents, they were CIOs of corporations with

revenues totaling over $5 billion and IT budgets approaching a billion dollars. Not surprising then, that a recent IDC report showed more than 40 percent of all software applications are released with between one and ten critical defects, the management being fully aware of this at the time of issue.

Horror stories But why the disregard? The media are always reporting horror stories about the implications when things go wrong. Software errors can cost companies millions in lost revenue, companies have even been purported to go bankrupt on the strength of software failures (Like American LaFrance). When Sainsbury’s online store went down for a day because of a software failure, it reportedly cost the business over £1.5 million in lost revenue and compensations. But it is not just money that’s at stake here, reputations can be tarnished, costing companies untold amounts of business. The launch of Heathrow Terminal 5 earlier in the year was a PR disaster due to software failures – and giving BA a reputation that today is still slightly tarnished. It’s costing the business lots of marketing revenue to try and clean up Terminal 5’s image.

There is unrest brewing in IT QA departments all over the globe. A frustration borne out of a lack of recognition within their own organisation as to the importance of software quality, the potential competitive advantage their company could enjoy if they get software quality right, and the major negative impact that can result from poor software quality processes. October 08 | T.E.S.T

32 | Quality assurance

The fall guys

Time

Cost Resource

T.E.S.T | October 08

Risk

Testers know full well where the blame will lie if their companies are unfortunate enough to be caught out. But how can they shoulder the responsibility for unknown bugs when they are not given adequate time or resources to fully assure the quality of their applications? CIOs are well aware of the dangers, so why are they still despondent about doing something about it? QA need help. Application quality is no longer an activity within the software development lifecycle that IS teams can aspire to improve in their own time. It is a business imperative that demands a solution now. It’s enough to make you scream! The same survey found that of people that had invested in technology to aid the process, only six percent were totally happy with their automation investment. Why is this? Perhaps it has been the very limited success of the test automation solutions that came out of the last century which needed months to configure, required expensive and scarce specialist skills and simply could not survive in the face of rapidly changing applications. Many CIOs complained that they felt the process was a waste of time and resources as the level of script

maintenance needed rendered the process counter-effective. Whatever the reasons, people have been burnt in the past and a lot of CIOs don’t have high expectations of automation solutions. I’ve spoken to prospects that have spent millions, taken years (and vast amounts of resources) to implement their solutions and as a result are able to implement just five percent of their total testing efforts. This particular company was actually more than happy with their automation solution. They didn’t feel they could expect much more. This is an appalling state of affairs. Manufacturers of these old-style solutions are giving the industry a worse reputation than Heathrow T5! Solutions which at best will only address 20 percent of the problem only give ‘solutions’ a bad name.

The quality conumdrum Achieving quality takes time, effort and money, and in the current credit crunch constrained times one can understand a temptation to take short cuts, to try to avoid this fundamental discipline. But given the horror stories this cannot be right. You want better quality software? Right, well it will take more time to get right. You want the application live next week? OK, but we will either need to double the

Quality assurance | 33

team of testers or we will only complete 75 percent of testing and the quality will be compromised. We don't have any more budget for more resources? Well there is a decision to be made between an on-time application or a high quality application—you can’t have both. A familiar conversation? This is the quality conundrum. See Fig 1. It is this quality conundrum issue, along with perhaps a slightly blasé attitude to software quality from the relevant IT management, that has caused anarchy to manifest itself into the realms of QA departments across the world. QA teams are obviously frustrated by the situation – the lack of recognition within their own organisation as to the importance of software quality, the potential competitive advantage their company could enjoy if they get software quality right, and the major negative impact that can result from poor software quality processes. On top of this, the major proponents of software test automation have repeatedly failed to offer a reliable solution, thus those QA departments that have been lucky enough to secure funding to improve their software quality through automation, have been let down. More often than not, even after substantial investment in software test automation, these QA

departments are no better off than they were before they spent the money – hence the anarchy.

Time for a rethink It is time for a rethink. CIOs cannot forego basic IT fundamentals, but we do need cost effective technology in these cost constrained times. It has to make the entire test process more productive and be applicable in the fast-changing, business-critical areas of the applications. Automation vendors need to pick up their game and businesses need to attach more importance to this mission-critical, fundamental business process, otherwise, it doesn’t take a crystal ball to foresee an increasing number of horror stories in the press and QA teams everywhere dying their hair in frustration! There is unrest brewing in IT QA departments all over the globe and a revolution that is gaining momentum, and is forcing people to re-think their attitude towards software test automation. That revolution is the next generation of software test automation. Solutions that do not rely on scripting language, are quick and easy to implement, can update themselves when an application changes and offer a more broad testing scope compared to other tools. It’s time to join the revolution.

Automation vendors need to pick up their game and businesses need to attach more importance to this mission-critical, fundamental business process, otherwise, it doesn’t take a crystal ball to foresee an increasing number of horror stories in the press and QA teams everywhere dying their hair in frustration!

Colin Armitage CEO Original Software www.origsoft.com

October 08 | T.E.S.T

UKCMG EuroTEC 2009

Training Education Conference with Exhibition & Workshops

18th – 20th May 2009

Whittlebury Hall Hotel, Whittlebury, Northampton Performance Testing

Performance Engineering

Capacity Management Performance Management

Capacity Planning

SLM

Performance Assurance A must attend event with excellent papers from industry experts, US guest speakers, education specialists and end users. The three-day event will cover hot topics and issues in the Performance, Capacity Management, Service Management and Mainframe arenas, including beginner sessions and workshops. The event offers excellent education and training opportunities for attendees. Alongside the multi-tracked conference agenda is an exhibition hall with leading vendors.

Best End User Presentation Competition Why not submit a presentation for the main event agenda and have the chance to win the ‘Best End User Presentation Competition’ and win a trip to the USCMG event in Reno in December 2009. To submit a presentation please visit www.ukcmg.org.uk

10% Discount for EuroTEC 2009 Delegate Bookings Received Before 31st December!! Media Sponsor

More event details are available at www.ukcmg.org.uk T.E.S.T | October 08

Automated testing | 35

Teaching a new dog an

old trick By looking back at some old techniques and applying them to today’s systems we can shave weeks off the time taken to build an automated test system says Duncan Brigginshaw, director of Odin Technology.

utomating the testing of a typical application involves familiar tasks to many of us, defining test data, writing scripts and building object maps. For some modern applications however some of these tasks can be automated too. By borrowing some tricks first used back in the days of green screen applications we can make the path to automation a less painful one. First let us look at what we mean by an object map.

Object maps de-mystified Without trying to point out the obvious, testers and automation tools are very different and they see the world in

different ways. Most people would refer to the visual elements of an application using simple terms related to the business at hand, eg ‘The login window’ or the ‘OK button’. Test tools on the other hand use a variety of technical properties to identify and relate to the same things. Fig 1 shows a simple example of the way a tester would describe a UI element and the terms the average test tool would use. Simply put, an object map provides the mapping between the human terms used to describe the elements of an applications interface and the test tools’ terms. Most modern test tools provide some kind of object map facility, usually referred to in slightly different terms.

October 08 | T.E.S.T

36 | Automated testing

Fig. 1.

Fig. 2.

As with all automation efforts, an investigation of the time taken to create this bespoke application versus the time taken to manually build the object map would be sensible before diving in and creating it. If it will only take two days to build and save the team two weeks of object mapping time then it is the right thing to do. But remember to factor in the time taken to navigate to the objects as well.

Building an object map can be a laborious process. With the test tool and the application open, a tester would use the mouse to point to an element of the application. The tool would read the properties it requires to refer to the object, then prompting the user to type in a sensible business name for the same object. This would be stored and the user would move to the next object and so on.

Introducing the old trick – self describing applications Some of us still reminisce about using dumb terminals and how

T.E.S.T | October 08

things were simpler back then. Not surprisingly, similar application design to that used in the days of central servers and dumb terminals is back and being used in modern client server applications today. Your average ‘green screen’ user interface was made up from a set of regular-sized pages, with fixed rows and columns of text. Descriptions of these pages were held on the server and when required would be sent to the dumb terminal where they would be interpreted and presented in all their monochromatic glory.

Some of today’s bigger enterprise applications are borrowing these techniques, storing descriptions of the windows that make up the interface in a database. The description is interpreted and presented dynamically to the user when required, as shown in Fig.2. An example of the kind of description stored in the database is shown in Fig. 3. More and more applications are using similar ideas with different non-database architectures, opening themselves up through APIs and other self-describing techniques.

The advent of Microsoft .NET has taken this to a new level with the availability of a technique called ‘Reflection’ to provide information about an application. This however is a topic for an entirely different article. Using the old design techniques to our advantage we can make the path to automation a much easier one, let us explore how.

Creating object maps automatically We can see from the examples above that there are two important pieces

Automated testing | 37

of information stored in the application description held in the database: 1. A ‘LABEL’ that your average manual tester could use to describe the application elements ie Surname or Gender. 2. The technical properties that the test tool might use to refer to the same element, ie ‘TYPE’ and ‘SEQ’. We nearly have all the information we need to create an object map, giving both the human description of the application interface and the tools requirements. In practice a bespoke application to do the translation would need to be created. Fig 4 shows where this would fit.

Can this work in practice This all sounds excellent in theory but will it work in practice? We have used this technique on two automation projects in

WINDOW: CUSTOMERDET FIELD TYPE LABEL LENGTH SEQ

CUST_SNAME INPUT Surname 20 1

FIELD TYPE LABEL LENGTH SEQ

CUST_GEN OPTION Gender 1 2

Fig. 3.

October 08 | T.E.S.T

38 | Automated testing

In many areas of information technology, application design techniques are being recycled and re-applied in new and highly-effective ways. When embarking upon a new automation effort, it is often worth taking a look back at the past and remembering some of the old tricks, as they may apply to even greater effect today.

the last year to vastly improve the time to automation. The figures stand up for themselves. Project 1 – 1812 object map entries created automatically. By using a conservative figure of a minute to find and map a single object, nearly a week’s worth of solid mapping was completed in less than a day. Project 2 – 7537 object map entries created automatically. Again using a minute per object, over four weeks solid of object mapping time was saved using the object map generation techniques.

Recycle and reapply In many areas of information technology, application design techniques are being recycled and re-applied in new and highlyeffective ways. When embarking upon a new automation effort, it is often worth taking a look back at the past and remembering some of the old tricks, as they may apply to even greater effect today.

T.E.S.T | October 08

Duncan Brigginshaw Director Odin Technology www.odin.co.uk

Fig. 4.

'0$64 0/ :063 */%6453: " POF EBZ FWFOU GPS TFOJPS MFWFM QSPGFTTJPOBMT UP EJTDVTT EFCBUF BOE SFTPMWF UIFJS NPTU QSFTTJOH DIBMMFOHFT UISPVHI B TFSJFT PG QSF BSSBOHFE 'PDVT (SPVQT

UI 4FQUFNCFS 1BSL *OO )PUFM )FBUISPX s $EBATE 3ESSIONS s 0EER 4O 0EER .ETWORKING s $ISCUSS )NDUSTRY 7IDE )SSUES s -INI %XHIBITION s &REE !TTENDANCE

s +EYNOTE 3PEAKER s A-JNJUFE 1MBDFT "WBJMBCMF

'PS NPSF JOGPSNBUJPO

$POUBDU (SBOU 'BSSFMM PO &NBJM HSBOU GBSSFMM! NFEJB DP VL 4ELEPHONE &ACSIMILE %MAIL JOGP!WJUBMGPDVTHSPVQT DPN 7EBSITE XXX WJUBMGPDVTHSPVQT DPN

4HE 6IT!, &OCUS 'ROUPS ARE OPEN TO ALL INDIVIDUALS WITHIN THE )4 INDUSTRY ALTHOUGH EIGHTY COMPLIMENTARY PLACES ARE OFFERED TO -ANAGERS (EADS $IRECTORS AND #)/ S ON A FIRST COME FIRST SERVED BASIS 4ERMS AND #ONDITIONS APPLY

!N EVENT ORGANISED BY -EDIA PUBLISHERS OF 6IT!, -AGAZINE

40 | Testing Feature environment

Keeping it

real

Without a realistic environment, software testing is meaningless and with an everchanging array of different environments software testers need to ensure that the appropriate context is included in the testing regime. Itrinegy product director Frank Puranik explains.

oday, as a consequence of organisations seeking to reduce costs by implementing strategies such as data centre consolidation, virtualisation and increased use of home/remote working, more and more applications are required to work in a wide variety of different networks (WAN, wireless, satellite etc). Therefore, the professional test/QA specialist needs to ensure testing applications in these environments is included in their test

T.E.S.T | October 08

regime. After all, if your testing procedures donâ&#x20AC;&#x2122;t include checking how software functions and performs in the final production network how can you say that you are doing your job properly? Sadly though, a lot of software performance testing is still frequently conducted over the simple LAN of the test lab. These LANs are normally very fast and reliable but satisfactory application performance in these conditions is no guarantee of acceptable

Testing environment | 41

It should be no great surprise to find out that when a new application is rolled out in the production network it fails and is passed back to the test team or the application developers to be fixed or rewritten and retested. performance in non-LAN networks where characteristics such as restricted bandwidth, latency and packet loss (more on these later) can have a major impact on how an application behaves. Therefore, it should be no great surprise to find out that when a new application is rolled out in the production network it fails and is passed back to the test team or the application developers to be fixed or rewritten and retested. However, by incorporating realistic WAN environment testing in their test regime, the test/QA Specialist can enhance their perceived value by confirming that the application has been checked for ‘WAN-readiness’ before it is handed over for live implementation.

Why different networks influence how well applications work Local area networks (LAN) are big, fast and reliable networks normally found within an organization’s main site or building (or the test lab). They can be rightly viewed as being super fast highways. The amount of data they carry is going to be light, relative to their capacity, and the distances this data travels are relatively short. Wide area networks (WAN), in contrast, are geographically disbursed networks of frequently lower capacity which are required to carry a high volume of data, compared with their capacity, over much greater distances. Wireless, satellite and radio networks

typically have lower capacity, lower transfer rates and are susceptible to packet loss and reordering. The growing trend of accessing applications via PDAs, Blackberrys, iPhones and other mobile devices means testing across these types of network is becoming increasingly important. So, as you can already see, LANs, WANs and wireless networks are really very different environments and to claim that conducting testing in LAN conditions can give you a good indication of how the same application is going to perform when placed in a WAN/wireless network is plainly unrealistic. From here on, when we talk about WANs, we are also referring to wireless LANs, 3G, Satellite and similar networks.

Application performance The key network characteristics that influence application performance: a) Available bandwidth. In LANs, available bandwidth is rarely an issue, but as WANs have lower bandwidths and there are more applications competing for space, the amount of bandwidth available to a particular application will be much lower and this may have a significant impact on its performance. In addition, network administrators can set up their networks to favour certain applications like voice over IP (VoIP) over ‘conventional’ applications and as a result, the software you are testing may have very restricted available bandwidth when placed in the production environment. Therefore, you need to factor bandwidth restrictions into your testing.

b) Latency and jitter. Let’s consider running an application over a network between London and New York. Standard TCP/IP networks do not send data in a continuous stream. Instead, they break it down into packets (like envelopes in the post) and send it in batches. They then wait for confirmation that the packets have arrived safely before sending more, causing a delay. Also, the journey itself is not direct and various network devices will be encountered along the way, which all add their additional own delay. As a result, it can take 90ms to complete a round trip journey and an application transaction will consist of many such trips. This delay is commonly described as latency and it can be a real showstopper for some applications. So, if you are testing software that is going to be accessed remotely from say, an Indian data centre or a branch office in the USA then you must ensure your test environment reflects these conditions. In addition to latency, networks can exhibit variable latency more commonly known as jitter which can have a particularly bad impact of VoIP, video and other types of streaming application. c) Packet loss, error and reordering. As the packets of data travel over the network they can be lost, errored or reordered so that they arrive out of sequence, or don’t arrive at all. It’s like sending a bus down the road that either doesn’t arrive or doesn’t arrive entirely intact and is therefore probably useless when it reaches its destination. Wireless WAN, satellite and 3G/mobile

October 08 | T.E.S.T

42 | Testing environment

phone networks are generally subject to higher loss and error rates than wired networks. The increased use of wireless networks within buildings and as a way of allowing mobile ‘on-the-road’ members of the workforce to receive data means that applications need to be developed to cope with this potential for loss, erroring or reordering. Once again, reproducing these conditions in your test lab isn’t going to be possible if you confine your testing to using the internal LAN.

How do you test application performance over the WAN? OK, so now you’re (hopefully) convinced of the importance of testing in the WAN. However, a request to conduct testing of a new and untried application in your live WAN environment carrying businesscritical data is likely to be declined by your network manager. A possible alternative is to confine your testing over the live WAN to outof-office hours. The problem is that most live networks are actually busy (or even busier) at night as the company performs network back-ups etc. It is also going to be impractical to get the right people in place at the required locations in order to conduct the initial test and subsequent retesting. Another major downside of using a live network is the fact that you cannot guarantee that the same conditions will apply when you go to repeat your testing. A third way is to create a duplicate of the live network but when you consider the cost of doing this (including national or international circuits), the chances are that such a request will be turned down. Furthermore, a duplicate network will not have the competing traffic to create the necessary realism.

T.E.S.T | October 08

WAN emulation – the safe, controllable alternative WAN emulation/network simulation technology behaves like a real WAN or wireless environment but can be deployed in the same room as your normal test rig or even on your desktop. It allows you to recreate a wide variety of different WAN or wireless conditions and enables you to ensure that software can be tested during prototype, development, quality assurance and pre-deployment testing. By being able to manipulate the network characteristics such as bandwidth, latency, packet loss etc., it is possible to replicate a wide variety of network environments in which to conduct your tests and see how the application behaves. Typical network impairments that a network emulator should be able to produce include: • Bandwidth restrictions; • Delay (latency and jitter); • Packet re-ordering, packet error and packet loss; • Traffic shaping and traffic prioritisation (QoS); A good network emulator will also be able to recreate the following types of network: •H igh latency WANs (national, international and satellite); • Wireless networks (eg Wi-Fi, WiMAX and 3G); • J ittery networks – such as cause VoIP deployments a problem; • Networks that lose and/or damage traffic; •Q oS type networks, including MPLS, ATM and VLANs; It should also be possible to apply different impairments to different traffic as would happen in a real WAN. Network emulators can also work

Being able to spot and correct network-related problems earlier in the application lifecycle greatly enhances the opportunity for a successful roll-out into the production network as well as helping to save money and time, which in the current economic climate has to be a good thing. seamlessly with load generation and performance tools to further enhance the testing of applications.

The benefits Utilising WAN emulation technology as part of the testing programme enables the tester to conduct much more realistic pre-deployment checks on application behaviour. Being able to spot and correct network-related problems earlier in the application lifecycle greatly enhances the opportunity for a successful roll-out into the production network as well as helping to save money and time, which in the current economic climate has to be a good thing. If, through the deployment of this technology, the value of the testing team is seen to be enhanced as well, then that can’t be a bad thing either.

Frank Puranik Product director Itrinegy www.itrinegy.com

T.E.S.T company profile | 43

Seapine Software TM

With over 8,500 customers worldwide, Seapine Software Inc is the leading provider of quality-centric application lifecycle management (ALM) solutions. Headquartered in the United States, with offices in London, Melbourne, and Munich, Seapine solutions help companies reliably and efficiently develop quality software applications. Seapine’s products support best practices, integrate into all popular development environments, and run on Microsoft Windows, Linux, Sun Solaris, and Apple Macintosh platforms. Integrating quality assurance into the software development lifecycle is critical to ensuring successful product development. To stay competitive, organisations need to implement software quality assurance tools and processes earlier in the development lifecycle to help identify problems during development. Seapine’s crossplatform solutions help development organisations:

• Automate functional and regression testing; • Implement an Agile development methodology; • Track development-related issues and artifacts; • Control changes to test scripts; • Manage changes to source code and other digital assets; • Meet compliance and policy requirements; • Streamline software development and QA processes; • Facilitate team communication; • Generate quality metrics reports. From award-winning TestTrack Pro for issue management, advanced automated testing with QA Wizard Pro, flexible configuration management with Surround SCM, to our innovative solutions such as TestTrack TCM for test case management, Seapine’s ALM solutions keep the focus on quality in the development process.

Issue and Defect Management with TestTrack Pro TestTrack Pro, a scalable, cross-platform issue management solution, facilitates team collaboration across the enterprise. Simple to install, easy to use and maintain, TestTrack Pro features comprehensive workflow and process automation rules, powerful filters, escalation rules, email tracking, reports, and role-based security. It also includes extensive customizations, seamless integration with current development tools, and support for MS SQL Server, Oracle, and MySQL. Keep team members informed and on schedule with TestTrack Pro.

Test Case Management with TestTrack TCM Testing software applications requires hundreds to thousands of unique test cases, the time to execute them, and the ability to efficiently manage the results.

In cross-platform TestTrack TCM, you have the tool you need to write and manage thousands of test cases, select sets of tests to run against builds, and process pass/fail results using your development workflow. With TestTrack TCM driving your QA process, you’ll know what has been tested, what hasn’t, and how much effort remains to ship a quality product. When quality is important, TestTrack TCM is essential.

Test Planning and Tracking with TestTrack Studio From the creation of test cases through the resolution of defects, TestTrack Studio tracks and manages all the details of your testing effort. Featuring the awardwinning defect tracking features of TestTrack Pro and the timesaving test case management features of TestTrack TCM, TestTrack Studio helps companies deliver higher quality products by providing more control over the QA process, including knowing what is being tested, how much testing remains, and the status of issues discovered during development or testing. Traditional test planning tools track the process of testing, while defect tracking tools merely track the results. Only TestTrack Studio gives QA managers and project managers a complete view of the QA process, from planning to testing and fixing through shipping.

Software Configuration Management with Surround SCM Surround SCM, a scalable, cross-platform software configuration management solution, keeps source code and digital assets organized as it tracks change throughout your organization. Simple to configure and maintain, Surround SCM facilitates concurrent development with virtual branching, configurable workflows, custom metadata, in-application triggers, email notifications, and seamless IDE integration. It also includes Unicode and WebDAV support, secure remote access, and an optional proxy server for distributed teams.

Automated Testing with QA Wizard Pro QA Wizard Pro automates the functional and regression testing of Web and Windows applications, helping QA teams test more of an application in less time. Featuring a powerful scripting language and an intuitive recording interface, QA Wizard Pro includes advanced object searching, a global application repository, data-driven testing support, validation checkpoints, and built-in debugging. It also includes batch file support, a real-time status tool, and remote execution support. Focus on what you want to test, instead of how to test it, with Seapine QA Wizard Pro. With Seapine solutions, software quality assurance becomes part of the development lifecycle. Learn more about how you can leverage Seapine solutions.

www.seapine.com/test08 October 08 | T.E.S.T

44 | T.E.S.T company profile

Test early test often A major presence in the software testing arena, Borland is dedicated to helping IT organisations transform software delivery into a managed, efficient and predictable business process. T.E.S.T editor Matt Bailey spoke to Borland’s VP product marketing, Andy Seager. Borland uses a combination of technology and services to help IT organisations transform software delivery into a managed, efficient and predictable business process. It offers comprehensive, Open Application Lifecycle Management solutions – open to the user’s processes, tools and platforms. It also offers consulting and education services to ensure customers can leverage existing bestpractices and continuously improve their software delivery processes.

continuous integration. Rather than ‘specialisation’ our differentiation is LQM which changes the concept of testing quality from an ‘afterthought’ to incorporating it as a discipline throughout the lifecycle, starting with requirements. We support integrated testing for traditional and agile methods in software delivery. We see the adoption of a quality maturity process as key to scalable enterprise software delivery, where quality is a core value.

Andy Seager is the vice president of product marketing for Borland. He is responsible for leading all product and field marketing activity, worldwide, for the company’s Application Lifecycle Management (ALM) solutions that address the company’s vision for OpenALM. With over ten years experience in product marketing for the technology sector he has a proven track record in execution and management.

T.E.S.T: who are the company’s main customers today and in the future?

T.E.S.T: What are the origins of the company; how did it start and develop; how has it grown and how is it structured? Andy Seager: Borland has a 23 year history in software development, starting in the developer tools market and latterly as the leading supplier of Open Application Lifecycle Management (ALM) solutions. The acquisition of Segue added a suite of testing tools to our ALM portfolio that now represent the cornerstone of a fully integrated Lifecycle Quality Management (LQM) component to ALM. The company is headquartered in Austin Texas but and has international offices in Europe and Asia Pacific. T.E.S.T: Does the company have a specialisation within the testing industry? AS: We offer a full compliment of testing capabilitiesy including test management, automated functional and performance testing and

AS: Our customers are mostly enterprises that have large project portfolios, Borland helps customers to achieve scalability and predictability in software delivery across multiple projects and teams. T.E.S.T: What is the company’s commitment to corporate social responsibility, ie, ‘green’ issues? AS: As a supplier of automation systems, our purpose is to help our customers remove their old manual, paper-based processes. Process automation is one of the major ways clients can reduce their carbon footprints. And of course these days we never actuallyseldom physically ship any of our productsphysical media, with the exception of some highly regulated government bodies our customers download our products. T.E.S.T: What range of products and services does the company offer? AS: Borland’s OpenALM solutions include: Requirements Definition and Management (Borland Caliber); Software Change & Configuration Management (Borland StarTeam); Lifecycle Quality Management (Borland Silk); Borland Management Solutions; Team Demand, Team Focus and Team Analytics. We also offer a full range of consulting services across our entire ALM solutions

Borland www.borland.com 8 The Pavilions, Ruscombe Business Park, Berkshire RG 10 9NN, United Kingdom Phone: +44 (0) 118 924 1400 Fax: +44 (0) 118 924 1400

T.E.S.T | October 08

T.E.S.T: What is your view of the current state of the testing industry and how will the recent economic turbulence in the economy affect it. What are the challenges and the opportunities? AS: Software testing is still not a fully-automated discipline for the majority of software teams. Many still rely on outdated manual processes that are time consuming and error prone, this ultimately means that a lot of very important testing is either not done well or not done at all. Testing is still thought of as an exercise that is performed at the end of the delivery process – where defects cost significantly more to correct. Indeed, most defects found in the testing product phase are 50 to 100 times more expensive to fix, so ‘test early, test often’ is a moneysaving mantra. Software quality remains a significant challenge for the industry and teams need to adopt best practice frameworks, continually improve quality processes and isolate defects much earlier in software delivery cycles. Given that budgets are getting tighter – it makes sense to work smarter, remove defects that create costly rework and drive customer acceptance and satisfaction. Not investing in test automation is a false economy! T.E.S.T: What are the future plans for the business? AS: We will continue to deliver on our OpenALM theme: To transform software delivery in to managed and predictable business process. This means empowering teams with the right tooling and harvesting metrics that help them continuously improve – at a practitioner level this means integration between the different phases of the SDLC but the management layer, extracting relevant and timely information that supports decision making. This is relevant across ALM but also in the context of LQM. T.E.S.T: Andy Seagar, thank you very much.

T.E.S.T company profile | 45

31 Media 31 Media is a business to business media company that publishes high quality magazines and organises dynamic events across various market sectors. As a young, vibrant, and forward thinking company we are flexible, proactive, and responsive to our customers' needs.

T.E.S.T – The European Software Tester

Software testing is an emerging yet burgeoning market across the UK and the rest of Europe and with many businesses now realising the importance of this key function, a thirst is growing for a solid media platform that helps take the market to the next level. As this business-critical area expands it has becomes apparent that the professionals and suppliers in this space require additional avenues to air their thoughts and ideas, share their challenges and experience, and help grow the market in which they operate.

Customer Magazine

Customer Magazine was launched to address and assist with the various challenges senior professionals face when establishing a customer-centric business. Our editorial takes a pragmatic approach to what has become a series of complex issues and delivers dynamic, provocative, and insightful articles, case studies, opinion pieces, and news stories that not only challenge our readers but also bring clarity and vision to the many challenges they face.

VitAL Magazine

VitAL is a journal for directors and senior managers who are concerned about the business issues surrounding the implementation of IT and the impact it has on their customers. Today senior management are starting to realise that implementing IT effectively has a positive impact on the internal and external customer and it also influences profitability. VitAL magazine was launched to help ease the process.

VitAL Focus Groups

The VitAL Focus Groups are a specifically designed programme of Focus Groups that bring together senior decision makers for a series of well thought-out debates. In our role as information provider we spend a lot of time speaking and listening to our customers and then seeking out innovative ways to meet their needs. It has become apparent that senior decision makers wish to discuss their challenges in a structured manner with a view to finding pragmatic and workable solutions to what are invariably complex issues. So in September 2009 VitAL Magazine will be launching the inaugural VitAL Focus Groups. 31 Media www.31media.co.uk info@31media.co.uk Crawley Business Centre, Stephenson Way, Crawley, West Sussex, RH10 1TN, United Kingdom Phone: +44 (0) 870 863 6930 Fax: +44 (0) 870 085 8837

Original Software Original Software offers software testing and quality assurance products that deliver tangible benefits across a wide range of IT and application environments. As a recognised innovator, Original Software’s goal is to reduce business risk and improve application time to market for IT departments through the development of powerful, next generation automated software solutions. Over the last 10 years, more than 400 organisations operating in 25 countries have come to depend on Original Software for their software testing solutions. Current users range from small software development shops to major multinationals, including Debenhams, JPMorgan Chase, The New York Stock Exchange, Circuit City Stores, Pfizer Pharmaceutical (Ireland), DHL, Coca-Cola, Skandia and hundreds of others. Original Software operates central offices near Chicago and London. Their solutions can be obtained through these offices or through a network of qualified and knowledgeable business partners throughout Europe, the Middle East, Australasia and the Americas. The company’s TestDrive solution is ideally suited to complement the test-driven approach familiar to agile developers. Because of the code-free scripting and the self-healing script capabilities, companies are finding that TestDrive's tolerance to change enables it to be introduced far earlier in the delivery cycle than a traditional automation tool. TestDrive's assisted manual testing can be used by developers as soon as they have the user interface available.

The company's range of powerful solutions include: TestPlan – For effective planning, team communications, and accurate tracking of every testing activity. TestBench – As Original Software’s flagship solution, TestBench is the only solution that focuses on testing the "invisible" underlying systems database rather than just the "visible" system aspects. It addresses test verification, disk space, and data confidentiality issues. In addition, control of test data ensures that every test starts with a consistent data state, essential if the data is to be predictable at the end of testing. TestDrive – A full-featured automated testing solution that allows technicians to define and execute sophisticated tests without being hindered by complex programming languages. State of the art self updating technology automatically adapts tests to new software releases and upgrades. TestDrive Assist – A new concept in testing that delivers active support for manual testing and compiles history on recent testing, making it easy to recreate and isolate software defects. Its ease-of-use makes it ideal for user acceptance testing. Original Software www.origsoft.com Web for manual testers: www.manualtesting.com solutions@origsoft.com Grove House, Chineham Court, Basingstoke, RG24 8AG, United Kingdom Phone: +44 (0)1256 338666 Fax: +44 (0)1256 338678

October 08 | T.E.S.T

46 | T.E.S.T company profile

iTrinegy Network emulation & application testing tools iTrinegy is Europe’s leading producer of network emulator technology which enables Testers and QA Specialists to conduct realistic pre-deployment testing in order to confirm that an application is going to behave satisfactorily when placed in the final production network.

Delivering more realistic testing As more and more applications are being delivered over wide area networks (WANs), wireless LANs, GPRS, 3G, satellite networks etc, there is a growing need to test application performance in these environments. iTrinegy Network Emulators enable you to faithfully recreate these types of network environments for testing applications, including VoIP, in the test lab or even at your desktop.

Ease of use Our network emulators do not require you to be a network expertise in order to use them. They let you quickly and easily create and control network characteristics such as bandwidth, latency, jitter and packet error or loss in order to test software in the conditions that will be encountered in a live production environment or in a proposed new network environment. Any test scenarios you create can be saved and quickly retrieved for subsequent reuse. We also provide you with an extensive range of predefined network scenarios to get you started. Our products also work seamlessly with load generation and performance tools to further enhance software testing.

A comprehensive range to suit your needs iTrinegy’s comprehensive range of network emulators is designed to suit your needs and budget. It includes: – Software for installation on your own desktop or laptop (trial copies available); – Small, portable inline emulators that seat silently on the desktop and can be shared amongst the test team; – Larger portable units capable of easily recreating complex multi-path, multi-site, multi-user networks for full enterprise testing; – High performance rack-mount units designed to be installed in dedicated test labs; – Very high performance units capable of replicating high-speed, high-volume networks making them ideal for testing applications in converged environments. If you would like more information on how our technology can help you ensure the software you are testing is ‘WAN-ready’ and going to work in the field, please contact iTrinegy using the details below. iTrinegy www.itrinegy.com info@itrinegy.com Bulse Grange, Norton End, Wendens Ambo, Saffron Walden, CB11 4JT, United Kingdom Phone: +44 (0)1799 543 345 Fax: +44 (0)845 226 0607

Edgewords Make your training investment count We know that training budgets are being squeezed; we know the value that high-quality training adds to a test team’s productivity; but more importantly, our expert trainers, will give you the knowledge to maximise your capability. Edgewords specialises in delivering expert training to professional testers. We are one of only three companies within the UK certified by HP to deliver HP Mercury Training. Our trainers have all worked in the industry, so they will give you expert advice using real-world examples and best practices. We don't expect you to just take our word for this; Edgewords has trained many of the UK's leading test consultants, we have also trained HP Mercury's own in-house trainers. Over 50 percent of our business is repeat business or customer referrals. Our customers include leading organisations such as E.ON, Barclays, Fujitsu, Cap Gemini, LloydsTSB and SkyBSB. We deliver courses in HP’s LoadRunner, QTP and Quality Center, as well as Edgewords’ own software testing courses, and can also provide the ISEB certification training. We know that only the best quality trainers can provide the best quality training, so when you choose your training provider – make sure you choose the best. To help your training budget go further, and ensure that you can still get the best, Edgewords quality training, we have two amazing offers available until December 31 2008. Free accommodation and food included with all public courses!

Free accommodation & food included with all public courses! If you book on any public course before the end of the year, we will include your accommodation & food at the Swallow Buckatree Hall Hotel in Telford. All for no extra charge on the standard course price!

Book five delegates on a course and receive the fifth place free of charge! To celebrate Edgewords being five years old, we are offering a fifth place free! Book five delegates on any public course between now and the end of the year and the fifth delegate is absolutely free.

Edgewords www.edgewordstraining.co.uk training@edgewords.co.uk Tel: +44 (0)1746 766246 Fax: +44 (0)1746 766541

T.E.S.T | October 08

T.E.S.T company profile | 47

ISEB

ISEB is a leading awarding body for professional qualifications in computing, IT and communications. We provide a wide range of qualifications internationally with almost 350,000 exams being delivered worldwide, software testing being amongst the most popular. ISEB qualifications are internationally recognised – ISEB has been active throughout the world with exams taken in over 50 countries including Australia, Japan, South Africa, USA, Brazil and many others. ISEB qualifications add value to professional careers by providing both the means and platform for recognition and enhanced career development. ISEB is part of BCS (British Computer Society). BCS is the leading professional body for those working in IT and communications. Established in 1957and now with over 66,000 members in more than 100 countries, BCS aims to lead the development of the IT profession and make IT the profession of the 21st century. BCS is the industry body for IT Professionals and Chartered Engineering Institute for IT. We are responsible for setting standards for the IT profession. Our vision is to see the IT profession recognised as being a profession of the highest integrity and competence.

BCS membership for software testers BCS membership gives you an important edge; it shows you are serious about your career in IT and are committed to your own professional development, confirming your status as an IT practitioner of the highest integrity. Our growing range of services and benefits is designed to be directly relevant at every stage of your career.

Industry recognition Post-nominals - AMBCS, MBCS, FBCS and CITP - are recognised worldwide, giving you industry status and setting you apart from your peers. BCS received its Royal Charter in 1984 and is currently the only awarding body for Chartered IT Professional (CITP) status, also offering a route to related Chartered registrations, CEng and CSci.

Best practice By signing up to our Code of Conduct and Code of Good Practice, you declare your concern for public interest and your commitment to keeping pace with the increasing expectations and requirements of your profession.

Networking opportunities Our 44 branches, 16 international sections and over 40 specialist groups including Software Testing (SIGIST) and Methods & Tools, provide access to a wealth of experience and expertise. These unrivalled networking opportunities help you to keep abreast of current developments, discuss topical issues and make useful contacts.

Membership grades Professional membership (MBCS) is our main professional entry grade and the route to Chartered (CITP) status. Professional membership is for competent IT practitioners who typically have five or more years of IT work experience. Relevant qualifications, eg a computing related degree, reduce this requirement to two or three years of experience. Associate membership (AMBCS) is available for those just beginning their career in IT, requiring just one year’s experience. Joining is straightforward – for more information visit www.bcs.org/membership where you can apply online or download an application form.

Specialist Group in Software Testing (SIGIST) With over 2,500 members SIGIST is the largest specialist group in the BCS. Objectives of the group include promoting the importance of software testing, developing the awareness of the industry’s best practice and promoting and developing high standards and professionalism in software testing. For more information please visit www.sigist.org.uk

Information services The BCS online library is another invaluable resource for IT professionals, comprising over 200 e-books plus Forrester reports and EBSCO databases. BCS members also receive a 20 percent discount on all BCS book publications. This includes Software Testing, an ISEB Foundation. As well as explaining the basic steps of the testing process and how to perform effective tests, this book provides an overview of different techniques, both dynamic and static, and how to apply them.

Career development A host of career development tools are available through BCS including full access to SFIA (the Skills Framework for the Information Age) which details the necessary skills and training required to progress your career.

isebenq@hq.bcs.org.uk www.iseb-exams.com BCS, First Floor, Block D, North Star House, North Star Avenue, Swindon, SN2 1FA, United Kingdom Tel: +44 (0)1793 417530

October 08 | T.E.S.T

48 | T.E.S.T company profile

Odin Technology

Approaching its tenth anniversary, Odin has been providing the platform for successful test automation projects with clients across a broad spectrum of industries, including both the Private and Public sectors in the UK, mainland Europe, the US and Australia. Odin’s unique product, Axe, provides a consistent, structured approach to test automation that ensures the successful implementation of automation tools projects. Integration is provided for the market leading commercial automation tools and leading tools from the open source community. Axe has been developed and refined over ten years to incorporate industry best practices. Axe embodies and supports leading testing techniques such as Business Process Driven Testing (BPDT), data driven testing, agile testing and Test Driven Development (TDD). Odin Technology is one of the leaders of the ‘new generation’ of automated software that makes test automation easier, quicker and more cost effective to implement and maintain. Perhaps more importantly, Axe is aimed at increasing testing productivity and the simplification of what was previously a ‘difficult’ skill to learn. Test automation is quickly becoming an essential tool in the IT department’s ‘tool box’. Given the rapid and persistent rate of change that all businesses must cope with, testing can

no longer be allowed to be the ‘bottleneck’ that stops businesses making changes to gain competitive advantage. Axe is positioned and proven to ensure organisations can respond to constantly changing business environments. We know that, with a consistent and structured approach, any organisation will benefit from test automation and show significant return on investment. The ‘Axe’ difference is that this can be achieved in the shortest time possible and without the need to ‘reskill’ the testing team. Previous generations of automation approaches have been more akin to software development meaning that the test automation produced was primarily owned by ‘developers’ rather than the subject area specialists… the tester. Axe rectifies this problem by giving the tester the interface to manage and grow large numbers of test cases whilst “managing” the interface to complex and powerful test execution tools. Another area where Axe helps is that it produces not only the technical ‘script’ for the test execution tool to use, it also produces the documented test cases to give full auditability from the test requirements through to the actual test results. The test assets are primarily held in Microsoft Excel spreadsheets, used in anger by 97 percent of the testing population. The functionality of Excel can thus be combined with the tester’s knowledge to give a large number of scalable and re-usable component based test cases.

Odin Technology www.odin.co.uk info@odin.co.uk Atlantic House, Imperial Way, Reading, Berkshire, RG2 0TD, United Kingdom Tel: +44 (0)118 903 6101 Fax: +44 (0)118 903 6100

T.E.S.T | October 08

Business Analysis Project Management Information and Security

P Man roject age men t

IT Assets and Infrastructure

Soft Test ware ing

ITIL / IT Service Management

ITIL Man/ IT Ser age vice men t

s tem t Sys opmen el Dev

s& sset re IT A structu Infra

Systems Development

urit

Sec

Law Software Testing

Law

s ines Bus lysis Ana

ISEB offers a variety of internationally recognised qualifications, with 8 major subject areas in:

ISEB qualifications come in foundation, practitioner and higher level to suit each individual candidate. ISEB qualifications allow you to learn new skills in specific business and IT areas which measure competence, ability and performance. This helps to promote career development and provide a competitive edge for employees.

MTG/AD/472/1008

: tion a m or r inf e h t iseb Fur org/ . s c .b www