TEST Magazine - August-September 2013 by 31 Media

INNOVATION FOR SOFTWARE QUALITY VOLUME 5: ISSUE 4 AUGUST 2013 THE EUROPEAN SOFTWARE TESTER

www.testmagazine.co.uk

IS IT TIME FOR A RETHINK? QUALITY AND RAPID DEVELOPMENT MUST GO HAND-IN-HAND

INSIDE: WORLD NEWS: EDWARD SNOWDEN FEARS FOR HIS LIFE INTERVIEWS FOCUS ON: CAREER

CONTENTS

INSIDE THIS ISSUE

NEWS 6 British youth can’t work at Facebook?

WORLD NEWS 9

10.

NSA whistleblower fears for his life

NEWS FEATURE 10 Will tough penalties really deter cyber criminals?

WILL TOUGH PENALTIES REALLY DETER CYBER CRIMINALS?

In light of a potential new European directive outlining tougher penalties for cyber criminals, Sophie-Marie Odum looks at how effective these will be…

INTERVIEW

More haste, less speed

Paco Hope explains why it’s so important to ensure security is built in at the earliest stages of development…

14.

Breaking barriers 14

Sophie-Marie Odum asks Eyal Maor, what’s the best approach for game development, for multiple platforms, in a single process?…

BREAKING BARRIERS

18. IS IT TIME TO RETHINK YOUR APPROACH TO SOFTWARE TESTING?

COVER STORY 18 Is it time to rethink your approach to software testing? Ziaan Hattingh explains why last-minute, resource-intensive manual testing is increasingly inappropriate…

MANUFACTURING IT

IT: A driver for quality manufacturing David Rigler discusses the industrialisation of IT quality in the manufacturing sector…

TESTA 24

A word from the judges…

TESTA judges share what they are looking for from entries…

AUGUST 2013 | www.testmagazine.co.uk

PAGE 1

CONTENTS

26.

TEST DATA 26

Linking test data management and quality software

LINKING TEST DATA MANAGEMENT AND QUALITY SOFTWARE

Srikanth Dora Karam and Narayana Maruvada discuss how affordable test data management solutions can enhance test efficiency…

MOBILE APP TESTING

Mobile app testing in the cloud

Martin Wrigley looks at mobile app testing in the cloud…

FOCUS ON: CAREER

Celebrating the industry

Sophie-Marie Odum gives software testers a reason to be cheerful…

Are you really a tester?

32. CELEBRATING THE INDUSTRY

Ben Williams looks at how the advancement of tools in the industry challenges the idea of being a software tester…

Which path will you take?

How can you build your career in software testing? Brian Hambling and Pauline van Goethem offer some advice…

36. WHICH PATH WILL YOU TAKE?

39. SNEAK PREVIEW

LAST WORD 38

You can never go home again

Returning to his previous place of employment wasn’t what Dave Whalen thought it would be…

SNEAK PREVIEW 39

AUGUST 2013 | www.testmagazine.co.uk

head of October issue’s 20 Leading Testing A Providers supplement, TEST offers you a sneak preview of who will be featured…

PAGE 3

LEADER

A WELCOME FROM THE NEW EDITOR Hello and welcome to the August issue of TEST.

am delighted to introduce myself as the new editor. I look forward to introducing many new and exciting ideas over the coming months to ensure TEST remains the voice of the software testing industry. Through the delivery of high quality, thought-provoking content, you can be confident that you’ll continue to gain a true insight into the issues affecting the market, and benefit from engaging, informative and cutting-edge editorial that reflects new thinking and trends in a constantly evolving, fast-pace industry. We’ve introduced a new “Focus” section – a relaxed, more personal read where you can acquire tips and advice that aim to enhance your professional life and feed your creativity. This month, we focus on careers. Please turn to page 32 for a variety of articles that offer helpful training advice and discuss the rewarding role of being a software tester. Not long now until the first ever European Software Testing Awards (TESTA), which is taking place on 20th November. A celebration of the industry, the Awards offer teams and individuals an excellent opportunity to showcase their best projects and innovative solutions to a wider audience and get the recognition they deserve, whilst sharing novel ideas and techniques. As we countdown to the grand finale, I catch up with some of the judges to find out their definition of an “award-winning” entry, please see pages 24 and 25. Still haven’t entered? There’s still time, please visit www.softwaretestingawards.com for more information. All that is left for me to say is… I hope you enjoy this issue!

Do you want to write for TEST magazine? Please email sophie. odum@31media.co.uk Sophie-Marie Odum Editor

© 2013 31 Media Limited. All rights reserved. TEST Magazine is edited, designed, and published by 31 Media Limited. No part of TEST Magazine may be reproduced, transmitted, stored electronically, distributed, or copied, in whole or part without the prior written consent of the publisher. A reprint service is available. Opinions expressed in this journal do not necessarily reflect those of the editor or TEST Magazine or its publisher, 31 Media Limited. ISSN 2040-01-60 T H I R T YO N E

AUGUST 2013 | www.testmagazine.co.uk

EDITOR Sophie-Marie Odum sophie.odum@31media.co.uk Tel: +44 (0)203 056 4599 TO ADVERTISE CONTACT: Sarah Walsh sarah.walsh@31media.co.uk Tel: +44(0)203 668 6945 PRODUCTION & DESIGN Tina Harris tina.harris@31media.co.uk

EDITORIAL & ADVERTISING ENQUIRIES 31 Media Ltd, 41-42 Daisy Business Park, 19-35 Sylvan Grove, London, SE15 1PD Tel: +44 (0) 870 863 6930 Email: info@31media.co.uk Web: www.testmagazine.co.uk PRINTED BY Pensord, Tram Road, Pontllanfraith, Blackwood, NP12 2YA

PAGE 5

NEWS BRITISH YOUTH CAN’T WORK AT FACEBOOK? Working for Facebook would be the ideal job for many youngsters in Britain, however, the popular social networking site is struggling to recruit young people in the UK, and finding that those from other countries are often better qualified. Facebook operates its only software engineering base outside the US in London, and says that there is a lack of suitably qualified candidates. Simon Milner, Facebook’s Head of Policy said, “It’s really not easy. We don’t tend to find a lot of British young people who are ready to come and work at Facebook.”

THE POPULAR SOCIAL NETWORKING SITE IS STRUGGLING TO RECRUIT YOUNG PEOPLE IN THE UK, AND FINDING THAT THOSE FROM OTHER COUNTRIES ARE OFTEN BETTER QUALIFIED

secret that the technology industry finds it difficult to hire people with the right skills, as highlighted by Facebook and Google. SAS increasingly finds the lack of available talent in areas such as maths, science and statistics to be prevalent and it is often the case that people from other nations such as China and India are better qualified, posing a serious threat to the growth of the UK economy.”

Intellect, the UK’s technology trade association, estimates that 100,000 people are needed to enter the digital sector each year, but despite generous salaries and benefits, many firms say they are finding that schoolor university-leavers are not ready for work.

Earlier this year, the Government announced changes to the school curriculum to better prepare young people for jobs in the digital sector as it recognises that IT employment is set to grow. A spokesperson for the Department for Business, Innovation and Skills publically expressed that changes to ICT classes would put “a greater emphasis on computational thinking and programming”.

Geoffrey Taylor, Head of Academic Programme, SAS UK & Ireland, shared his thoughts. He said, “It’s no

Taylor added, “With IT employment set to grow by around 2.5% per annum on average over the next five

ARMY RESERVES TO BECOME CYBER SECURITY SPECIALISTS

years, the big data market has been identified as a particular area of opportunity. There is proven demand for staff with skills in big data, and as predicted in a recent report SAS conducted with e-skills UK, demand for this type of talent will increase by 18% per year on average between 2012 and 2017. “It’s imperative that the technology industry collaborates with government to ensure that the national curriculum is equipping the future generation with the skills required by UK businesses today, or the UK will lose out. This will be crucial to the UK’s ability to compete on the international stage.”

RE TRAIN IN CYBER SECURITY Commenting on the shortage of cyber security skills in the UK, Bill Walker, security analyst and technical director at QA, said, “There is clearly a critical and growing need for companies to plug the cyber skills gap and graduates are a great starting point. Longer term, organisations need to take big steps to attract and retain more women in the cyber security sector, in which they are currently hugely under-represented.

Recent news reports revealed that some reservists in the British Army will become specialists in cyber security under reforms to transform the force in preparation for future conflicts. The Territorial Army will have a much more integrated role to counter the new threats presented by technology and WMDs. Other army reserves will specialise in chemical-biological warfare and intelligence. Ross Parsell, director of Cyber Security at Thales UK, believes this is a positive step. He said, “By re-skilling its existing force in cyber security, the British Army is addressing the blurring of the lines between physical and virtual defence, which has becoming prevalent over the past decade. With the advent of cyber espionage and attacks, which threaten national critical infrastructure, the need for a holistic approach to national security is long overdue. It’s great to see the Army taking its share of responsibility for this alongside its traditional physical defence remit. “In addition, and just as importantly, this move will help enormously in positioning public sector cyber security as an attractive career prospect for the next generation.”

PAGE 6

BAE Systems has announced it’s taking steps to help plug the cyber skills gap with nearly half of its graduate recruits this year going into its cyber security business. A lack of skilled workers is hampering the UK’s fight against cyber crime, costing the UK public an estimated £18 to £27 billion annually, according to National Audit Office (NAO) report released earlier this year. According to this report, the number of IT and cyber security professionals in the UK has not increased in line with the growth of the Internet, and this skills gap will take years to close.

“This is a wise step by BAE, but will only address a fraction of the problem. Our records show that cyber security training business has more than doubled in the past year, but even we see huge scope for further growth in this area if we are to even start to close the skills gap. “A career in cyber security looks set to have a very robust future. Regardless of the tough economy, cyber security is going to grow and grow. There is an ideal opportunity here for anyone in IT to retrain and specialise in cyber security and it should also give any graduate a long and highly desirable career path.”

AUGUST 2013 | www.testmagazine.co.uk

NEWS LOOKS AREN’T EVERYTHING The new design of the Sony PlayStation 4 has been unveiled and while the design of the new console is important, it’s really the network that is the driving force behind the ultimate success of the modern games console. The multipurpose games console is challenging existing network like never before, and these ‘vital organs’ must be continually refreshed in order to constantly deal with consumer demand and expectation, explains Mervyn Kelly, EMEA marketing director at Ciena: “The gaming space has grown increasingly mainstream and along with the recently announced Xbox 360, the PS4 stands at the pinnacle of this evolution. As such, the device will be as much of a vital organ for the modern connected home – encouraging real-time sharing, social integration and online streaming – as it is a device that can offer best in class gaming experience. The growing importance of powerful game consoles to the modern home is evident... This growth in demand for online television and rich content sourced from the Internet, all over a computer console, will undoubtedly make an

MILLION POUND START-UP COMPETITION London’s Million Pound Start-up competition has brought in a huge wave of innovative technology as the hunt for the next high-growth company is gathering momentum. Applications have been recieved from Canada, US, South America, Africa, New Zealand, Australia, Asia, India, Middle East, and Europe. Million Pound Start-up is said to be the first global contest offering a £1 million equity investment and is supported by Seedrs, KPMG, Ketchum, Taylor Wessing, Digital Shoreditch, Playgen, City of London, London & Partners, City University London, School for Startups and London First. Headquartered in London, the goal is to propel the winner into a

Kam Star, founder of Digital Shoreditch and leading the Million Pound Start-up competition, commented, “No matter where they are in the world, the competition will help the winning company to establish their headquarters in London and give them the support they need to grow exponentially.”

“Operators must ensure that existing and future networks are ‘smarter’ than ever before – providing the scalability and programmability needed to deal with the mountain of data associated to new technologies like the PS4. There’s also a requirement for these networks to be able to differentiate between high and low priority traffic and adjust dynamically, on-demand and in real-time, according to changes in capacity demand. As the launch of the PS4 and its Xbox rival draw closer, the time is now for network providers to consider the implications of this shift in online behaviour on their network strategy.”

THE CRUCIAL ROLE OF TECHNOLOGY COMMERICALISATION

£100 million company within five years. The partners and investors are looking for more than just financial return. They are pooling resources of creativity and innovation, industry expertise and contacts to accelerate the success of the winning business. The closing deadline for Million Pound Start-up entries is August 31st 2013.

impact on networks and data centres, something that will require greater capacity in order to avoid damaging both service quality and the trust of end users.

Over two thirds of UK private sector productivity growth between 2000 and 2007 was the direct result of innovation, according to figures from the NESTA Innovation Index 2009. But according to UK consultancy ExeTec, for the UK to remain at the top table, the ability to innovate needs to focus upon the crucial role that technology commericalisation has on company profits. Gordon Young, ExeTec Consulting, said, “Product development has a reputation for high failure rates; research indicates that only a quarter of products that begin the development cycle see it through to the market, and even then a third fail once launched. This is a hugely uncertain and risky venture.” The last available figures on Gross Domestic Expenditure on R&D in the UK are from 2011 when it stood at £27.4 billion which, although up from 2010 in real terms by 2% year-on-year, only represents 1.79% of GDP. “This is the real challenge for technology businesses whether they are a start-up or an established player. Companies simply cannot afford to fail to successfully monetise the heavy development costs involved in product and solution development,” added Gordon.

A STATUS OF IT EXCELLENCE The Institution of Engineering and Technology (IET) is now offering Chartered IT Professional (CITP) status for experienced IT specialists. IET members can achieve CITP status through a partnership with BCS – the Chartered Institute for IT. CITP is aimed at those wanting to demonstrate competency in IT. Nigel Fine, IET chief executive, said, “Through this partnership, we are delighted to be able to offer AUGUST 2013 | www.testmagazine.co.uk

experienced specialists this highly regarded status of IT excellence.” Aimed at experienced IT professionals, the CITP Certificate of Current Competence (CoCC) is proof of an individual’s in-depth knowledge within their chosen field. With revalidation every five years, CITP is of real value and relevance to employers and provides a recognised benchmark across the IT profession. PAGE 7

NEWS AGILE WORKING OFFERS UK BUSINESSES COST SAVINGS OF OVER 10% Workforce agility has enabled companies to enjoy benefits equivalent to 3-13% of workforce costs, according to a report. In addition, there is scope to increase those cost-savings by a further 3-7% and, in some cases, sales uplift up to 11%. The Agile Future Forum’s (AFF) report, Understanding the economic benefits of workforce agility identifies how traditional models of work are coming under strain and that new agile models of work will be needed to deal with the challenges and opportunities created by technology, changing customer demands, globalisation and demographic shifts.

NEW TRAINING PORTFOLIO FOR THE TESTING INDUSTRY The SQS Academy, which offers delegates a full range of testing training; new role-based courses; and the 2012 Syllabus ISTQB Certified Tester: Advanced Level Test Manager training, has been launched. The training portfolio promises to deliver learning paths aimed at providing the rounded skill set required to be successful within testing and quality management. SQS Academy is said to offer a core set of standard courses and also runs customised training to help an individual or organisation achieve specific training goals. Both are available as public sessions or closed private events. SQS Academy director, Steve Dennis, said, “Whether you are responsible for the quality policy in your company, have a leadership role in large test projects or are tasked with ensuring the delivery of a software system within budget, time and quality constraints, our new look training portfolio has courses to suit all levels.”

practices, starting with a clear The AFF aims to maximise understanding of the needs the competitiveness of of the business and their UK businesses in the WORKFORCE workforce. global marketplace. AGILITY HAS ENABLED 22 businesses AFF Chair, Sir Win Bischoff, COMPANIES TO ENJOY who currently Chairman, Lloyds Banking BENEFITS EQUIVALENT TO realise financial Group, said, “Creating 3-13% OF WORKFORCE benefits through agile workforces has COSTS using workforce helped AFF companies agility have joined to compete in the global together to help market. We believe agile other UK businesses working practices could help to do the same. The Forum other UK businesses too. I believe the believes that agile practices can be economic benefits outlined in our established that benefit business and report demonstrate that it is possible employees, but a new approach to develop a way of operating that is needed where business leaders, is valuable for businesses, employees rather than simply HR, should and customers and ultimately the lead the development of agile UK economy.”

BELGIAN ENTERPRISES GAIN CONFIDENCE IN ICT Amidst the Eurozone crisis and budget cuts, Belgian enterprises have gained confidence in their ICT investments, according to a recent survey from Kable. 36% of Belgian enterprises are planning to increase their ICT budgets in 2013, compared to 33% in 2012. Encouragingly for ICT providers, the number of enterprises who intend to decrease their ICT budgets has reduced from 40% in 2012 to 28% in 2013. In addition, the survey indicates that Belgian investments in core technologies will continue at a steady pace. However, with at

least 59% of respondents planning investments in technologies such as business intelligence, green IT and virtualisation, and mobility through to the end of 2014, it looks like organisations are increasingly investing in advanced technologies to improve their productivity and operational efficiency. Piyush Sharma, a Kable analyst, said, “With increasing efficiency and cost cutting high on their agendas, enterprises in Belgium are focusing their attention on hosting services, especially for advanced technologies.”

ZERO-DAYS SOLD FOR SEVEN FIGURE SUMS Recent news, which has examined how governments, including Britain, Brazil, India and North Korea pay hackers hundreds of thousands of dollars to exploit “zero-day” coding flaws in software, raises the question, will this tempt rogue developers to purposely implant bugs in the software supply chain? Just a few years ago, hackers would have sold the knowledge of coding flaws to companies like Microsoft and Apple, which would fix them. But, now hackers sell technical details of vulnerabilities to countries that want to break into the computer systems of foreign adversaries for up to seven-figure sums. Jeremiah Grossman, founder and CTO of WhiteHat Security, commented, “As zero-days go for six to seven figures, imagine the temptation for rogue developers to surreptitiously implant bugs in the software supply chain? It’s hard enough to find vulnerabilities in source code when developers are not purposely trying to hide them.”

For the latest news visit, testmagazine.co.uk and follow us @testmagazine

PAGE 8

AUGUST 2013 | www.testmagazine.co.uk

WORLD NEWS NSA WHISTLEBLOWER FEARS FOR HIS LIFE At the time of going to press, National Security Agency (NSA) whistleblower, Edward Snowden was in Moscow – three weeks after landing in Sheremetyevo airport – awaiting confirmation of Russia’s offer of asylum. He is said to fear for his life. Snowden revealed that European privacy protections are a delusion: under Prism and other programmes, the US NSA and Britain’s GCHQ can, without much legal interference, gather any electronic communication whenever one of 70,000 “keywords” or “search terms” are mentioned. The former CIA technical worker, described as a master on computers, fled to Hong Kong from the US in May after revealing this extensive Internet and phone surveillance by US intelligence. Explaining why he decided to leave the US, he recently told The Guardian newspaper, “I don’t want to live in a society that does these sort of things… I do not want to live in a world where everything I

do and say is recorded.”

“I DO NOT WANT TO LIVE IN A WORLD WHERE EVERYTHING I DO AND SAY IS RECORDED.”

CYBERCRIMINALS EXPLOIT WORLD TRAGEDIES

Although these revelations are of obvious public interest – even President Barack Obama has acknowledged that they invite a necessary debate – the US treats Snowden as a spy and has charged him under the Espionage Act. Snowden has said that he would not leak any information that would harm Americans. He worked on IT security at the CIA in Geneva and expressed to The Guardian, “Much of what I saw in Geneva really disillusioned me about how my government functions and what its impact is in the world. I realised that I was part of something that was doing far more harm than good.” Snowden said he considered going public earlier, but waited to see whether President Obama’s election in 2008 would change the US’ approach.

US IT GIANT TO BUILD TRAINING CENTRES IN CHINA Hewlett-Packard (HP) is to build international software training centres in East China. With an investment of $2 billion, a software talent training centre, software testing outsourcing service facility and an IT resource service base will be built in the Jining National High-tech Industrial Development Zon, in Jining city, Shandong province. The talent training centre will train up to 10,000 IT professionals each year, and will equip students with skills and training of HP software.

SOFTWARE COMPANIES IN INDIA “UP IN ARMS” Many software companies in India are up in arms against a Defence Ministry memo which does not allow them to include research, design and software testing as part of the offset requirement for the AgustaWestland chopper, according to reports. The memo follows allegations that monetary bribes were routed

AUGUST 2013 | www.testmagazine.co.uk

through two software companies under the guise of software contracts. NASSCOM, the organisation that represents and sets the tone for public policy for the Indian software industry, is now working with the authorities to develop systems to track subterfuge.

Cybercriminals have used RedKit to exploit recent world tragedies, according to AppRiver, LLC. Using intelligence collected via its SecureSurf Web Filtering Engine, the provider of email messaging and web security solutions, identified three main campaigns aimed at exploiting the Boston Bombings and the huge fire that ripped through a Texas fertiliser factory. Two of the campaigns enticed victims to a landing page where fake news video images were stacked two over two over two. In the bottom right corner, one of the videos appeared to be suffering from a load error, but in actuality, this is where the first exploit was hidden. The third campaign included graphics to make it appear as though it was from CNN. Fred Touchette, security analyst for AppRiver, said, “It no longer comes as a surprise when we see malware campaigns destined for inboxes that pretend to be news stories or videos about recent tragedies that become world topics. “It’s an appalling technique that these unscrupulous cybercriminals found to be highly effective. In May, RedKit came in as number five in our top 10 web threats with nearly 20,000 occurrences last month alone. This type of despicable activity not only hurts the user duped into following the link, but also potentially the aid agencies desperately trying to raise awareness, and donations, in the wake of these disasters. “Even brands, such as CNN, are being tarnished by these exploits. Adopting a layered approach to security, adding spam and virus filters to email, using web protection services or devices, employing endpoint anti-virus software and encrypting sensitive messages all help deflect this type of attack. But, at the end of the day, vigilance is key.”

PAGE 9

NEWS FEATURE

WILL TOUGHER PENALTIES REALLY DETER CYBER CRIMINALS? Whilst it’s positive that a new European directive will get tougher on cyber criminals, how effective will the harsher proposed penalties be? Sophie-Marie Odum investigates…

arlier last month, the European parliament drafted a directive outlining tougher penalties for cyber criminals, which states that those found guilty of running a botnet of hijacked home computers should serve at least three years in jail. It also suggested that those who commit serious damage to systems or attack computers controlling a nation’s critical infrastructure should serve five years. This directive builds on Europe-wide rules, which have been in force since 2005 but introduces new offences that cover use of a botnet, which have become a staple in the cybercrime underworld as they are used by many criminal hackers to send spam, attack websites or as a resource to steal saleable data. It is helpful in its aim to improve co-operation between member states to investigate crimes and prosecute offenders, and is expected to be formally adopted into national law soon.

EASIER SAID THAN DONE? However, while the new directive aims to crackdown on cybercrime culprits, it seems that the general thought amongst the profession is that enforcing new laws will be easier said than done. For example, Etay Maor, fraud prevention manager at Trusteer, recently expressed: “While any news of progress in cybercrime laws are encouraging, we have to keep in mind that, in many cases, the people running the botnets and hijacked computers do not reside at the place where the crime takes place. Unfortunately, in most cases the people who get caught are the money mules (that may not even be aware they are committing a crime) and not the bot masters or ring leaders.” With this being said, how easy will it be to catch such criminals if law officials don’t know where they are? A tight-knit group approach is needed to tackle these “masterminds”, said Maor.

THIS IS ANOTHER EXAMPLE OF ADDING TO THE EVER GROWING PATCHWORK OF CYBER RISK LAWS. THE MEMO FROM THE EUROPEAN PARLIAMENT REGARDING THE DIRECTIVE APPEARS CONFUSED

PAGE 10

He added, “To apprehend these masterminds, law enforcement agencies will need to have cooperation with local agencies all around the world. This is not an easy task, and cyber criminals know this. This is why they usually reside in a country where they will stay safe from most western governments.

“Until the day that TO we see tight APPREHEND THESE cooperation between LEAs and criminals MASTERMINDS, brought to justice it is LAW ENFORCEMENT up to organisations AGENCIES WILL NEED TO and users to prevent HAVE COOPERATION fraud. We have WITH LOCAL AGENCIES to make sure that users’ devices stay ALL AROUND THE malware-free and that WORLD organisations worldwide have a clear picture of what is targeting them and how they can mitigate the threat quickly and effectively.”

CONFUSED DIRECTIVE? Headlines have been awash with the topic of cyber security lately; from cyber-espionage attacks to attacks on cyber defences, which all need to be taken tackled. According to John Yeo, Trustwave EMEA director, this overarching approach to cyber risk laws leaves the directive open to various interpretations and needs more of a direct focus on other types of computer crimes too. He said, “This is another example of adding to the ever growing patchwork of cyber risk laws. The memo from the European parliament regarding the directive appears confused. It addresses primarily a subset of computer crimes – focussing on ‘botnet’ related activity – however you don’t need a ‘botnet’ to carry out attacks of the targeted variety. “The language is sufficiently unclear to remain open to various interpretations, even by industry experts, which presumably was not the intention.”

AUGUST 2013 | www.testmagazine.co.uk

Burn Your Spreadsheets

Manage product testing with TestTrack TCM TestTrack TCM tracks all details of product testing for you, including test cases, suites, runs, and results. Now you can have real-time visibility over your testing effort and more time to plan and manage. • Achieve real-time visibility into testing progress. • Reduce risk and achieve greater confidence in the release. • Gain more time to plan projects and manage your team.

INTERVIEW

MORE HASTE, LESS SPEED... With software development cycles getting faster, Sophie-Marie Odum speaks to Paco Hope, principal consultant for Cigital, about why it’s more important than ever to ensure security is built in at the earliest stages of development, before code is committed, rather than face the costly consequences of flaws and bugs discovered in the testing stages…

oftware security is a primary concern for all industries that are entrusted with an increasing volume of sensitive data. Whether you are a traditional independent software vendor or a “software as a service” company providing customers with online solutions, you must maintain a high level of reliability and availability, which includes ensuring that your software solutions can withstand the most challenging attacks. So with this being said, why do many companies risk securing the integrity, availability and confidentiality of information by not building security in to the development cycle early? According to Paco Hope, principal consultant for Cigital, he believes that the rapid sprint through the development process, especially in light of the app explosion, can mean that security is sometimes forgotten in the early stages. He said, “Nowadays, we sprint through the software development process so fast that at the end we look back and say, ‘where was security supposed to go?’ but, ideally, wherever you were going to build performance, you should build security too.” Hope explains how Cigital – a consulting firm specialising in software security, organisation design, build and secure software maintenance – helps clients regard non-functional requirements as important as functional requirements. “When clients approach us for assistance with their functional requirements, we ask them, what about confidentiality, integrity, availability, accountability, etc? So that as we do their stories, we are covering the important security requirements they need,” said Hope. “Similarly, those that do Agile development will pick up ‘technical debt’, because as they build at such speed, there will be issues that will need resolving such as security, and they will work this technical debt into each sprint.”

AN EXPENSIVE APPROACH Not building security into at the development cycle early as early as possible can come at a high price as Hope explains, “The later you find a security defect, or any defect in software, the more expensive it is to fix. “Think about it, when you find defects in production software, and it shows up in the field, a root cause analysis is needed to discover there is a flaw in the design, which then needs changing, implementing and then testing. Then, companies must consider how they will patch the software and distribute it to users. This is the worst possible way to find a bug. A security problem is just a particular class of bugs and defects generally.

PAGE 12

AUGUST 2013 | www.testmagazine.co.uk

INTERVIEW

NOWADAYS, WE SPRINT THROUGH THE SOFTWARE DEVELOPMENT PROCESS SO FAST THAT AT THE END WE LOOK BACK AND SAY, ‘WHERE WAS SECURITY SUPPOSED TO GO?’

“We believe the earlier that you find a flaw, the better. If the flaw is in the requirements, such as the decision to require a password, for example, you can change this. However, if you wait until it’s in the field, this is when problems can occur and reputations are at stake. The need to go all the way back to the beginning will be necessary.”

So it seems that including security at the beginning of the lifecycle makes economic sense, and, although a non-functional requirement, security requires specialised skills and expert knowledge.

FLAWS AND BUGS There is a fundamental difference between bugs and flaws and if companies want to produce software that is secure, they need activities built in to the lifecycle that cover both, according to Hope. “A flaw is a problem in the design, whereas a bug is an implementation error,” explained Hope. “Bugs are local and, with recoding, the problem generally goes away. However, because flaws are design failures, they usually require big, costly changes. It’s important to get the design correct at the beginning. This then ensures everything that follows goes in the right direction. “The actions required to find a bug are different from what’s needed to find a flaw. But in regards to security, hackers don’t care; they will just go in and do what they want to do. So, as a company who creates software, we have to make sure our activities in securing our software cover both flaws and bugs.”

MAKING SOFTWARE SECURE Making software secure doesn’t happen by magic, it requires the expertise of software testers, explains Hope, “If I had a message to every tester in the world, it would be security is not black magic and voodoo. It’s actually just another thing that you are already doing. “I frequently talk to software testers about simple testing, such as boundary value testing and equivalence class partitioning, but what about all the possible inputs that might come to the program and break in to the database if it worked? So if you’re a software tester, there is all this low-hanging security fruit. You could be throwing a few extra test cases at it, which will help develop durable and secure software.” “Ultimately if you have dirty pipes, you can’t produce clean water,” said Hope. The BSIMM (Building Security In Maturity Model) Assessment is data Cigital has gathered from leading corporate software security groups to provide their clients with a clear understanding of how their practices compare to others. “The BISMM can help measure the cleanliness of your pipes. If you have poor quality in your life cycle, the chances of the software coming out secure are much lower.”

AUGUST 2013 | www.testmagazine.co.uk

PAGE 13

INTERVIEW

BREAKING BARRIERS What’s the best approach for game development for multiple platforms in a single process? Sophie-Marie Odum speaks to Eyal Maor, CEO of IncrediBuild, to find out how today’s technology can accelerate the entire game development process for all consoles and PCs...

n recent years, the number of lines of code has increased dramatically within game development, making build times unbearably long. However, with Christmas less than six months away, game developers are looking for solutions to shorten build time and speed up development lifecycles to ensure products are available on time. In addition, the time to develop one game for many different consoles or platforms, such as PlayStation 3, Xbox 360 and PCs to meet customer demands, needs to be factored in.

PAGE 14

Software testers have a huge role to play here as Eyal Maor, CEO of IncrediBuild, explains, “The game developers think they’ve got a really cool game that works great. It’s the testers’ jobs to prove them wrong. By creating a game to work on every system simultaneously, it ups the odds in the developer vs. tester competition.” Today, the total size of source code for one single project can easily exceed 2 million lines of code. As such, every build can often last anywhere between several tens of minutes and several hours. This includes a lengthy wait for compilation to complete. Daily builds can take 10 -15

AUGUST 2013 | www.testmagazine.co.uk

V ITAL INSPIRATION FOR THE MODERN BUSINESS

Subsribe FREE! News, views, strategy, management, case studies and opinion pieces

www.vital-mag.net/subscribe VitAL: INSPIRATION FOR THE MODERN BUSINESS

on vitAL : Inspirati busines s for the modern

vital

sU ser pp vic or e t De sh sK ow & is it sU e

VOLUME 6 | ISSUE 5 | MAY-JUNE 2013

V ITAL

sid Vit e: AL 16 Dig -pa est ge

INSPIRATION FOR THE MODERN BUSINESS

VOLUM E 6 | ISSUE 6 | JULY-A UGUST 2013

V ITAL INSPIR ATION FOR THE MODE RN BUSIN ESS

ss der n bus ine for the mo Ins pira tion 2013 4 : March / April volume 6 : Issue

De-risking change Transforming desktop delivery

Intelligent communication in a social world

IT projects Successful focussed

Exploring the evolution of the intranet

all, Keeping it sm derstood and easily un

Unlocking security for IT

IT agility

Tackling the global cyber security threat

arer path finding a cle ility to greater ag

VOLUME 6| ISSUE 5 | MAY-JUNE 2013

2013 March / April : Issue 4 : volume 6

partnership cloud in thfoe ship’s lewis partner hn Jo r

The quantiﬁed self

The implications of wearab le technology for business

Mass customisation

ud life in the clo agement team service man n: A TAlE oF lEMEnTATIo US: CMDB IMP FEATUrE FoC

Applying manufacturing principl

es to IT

Feature Focus: Doing more with less: 34-37

ES: 54-57 Two EXTrEM cover_with spine.indd 1

Published by

T H I R T YO N E

www.31media.co.uk www.31media.co.uk

Telephone: +44 (0) 870 863 6930 Email: info@31media.co.uk Website: www.31media.co.uk

30/04/2013 16:56

The order behind creative chaos Rolling out ITSM at the world ’s biggest physics experiment

INTERVIEW

minutes, but when you need to stop working several times a day and wait 15 minutes every time, it’s cumbersome. If you add other tasks such as shading, conversion, packaging, rendering, and more, you spend more time waiting. Testing adds another factor of timeconsuming tasks. “Testing is critical,” said Maor. QA testing and regression is a vital part of the development process, and a single QA cycle can take hours. But just one compilation or link error would lead to a failure to run the QA tests, effectively resulting in 24 hours lost against a project schedule. Maor added, “Bottlenecks in any one part of ALM THE GAME can significantly slow the DEVELOPERS entire testing process. THINK THEY’VE GOT

A REALLY COOL GAME THAT WORKS GREAT. IT’S THE TESTERS’ JOBS TO PROVE THEM WRONG. BY CREATING A GAME TO WORK ON EVERY SYSTEM SIMULTANEOUSLY, IT UPS THE ODDS IN THE DEVELOPER VS. TESTER COMPETITION

“Like everyone in the software development process, testers are forced to suffer from the ‘hurry up and wait’ syndrome,” continued Maor. “Streamlining the entire development process

eliminates the disease, allowing people to focus on the creativity of their jobs instead of their java intake.” Therefore, to speed up unit testing, it makes sense to spread the workload and for compilation, and many other development tasks, to simultaneously take place on more than one machine. In doing so, this accelerates other processes in the development cycle. For example, the automation of QA scripts to run in a parallel manner can reduce the development cycle overall; and running code analysis faster would obviously help. Accelerating other heavy duty jobs such as rendering, lighting, shading, and conversion, which are compute intensive processes, could significantly reduce the overall application lifecycle management (ALM) process. Maor said, “Including all your platforms, instead of building one for PC, then one for PlayStation and Xbox, and compiling them separately, saves time, allowing the team to do more, and encourages software testing to be completed straightaway, creating a more efficient team. In the lead-up time to Christmas, a busy time for many game companies, such a solution for the development process would prove invaluable.”

CASE STUDY Electronic Arts INC and IncrediBuild drastically improved Autodesk Maya runtimes, which allowed the company to focus on product development. Electronic Arts Inc. (EA) is a developer, marketer, publisher and distributor of interactive entertainment software. It creates games for a variety of video game systems, personal computers, wireless devices and the Internet. EA’s bestselling games include FIFA 10, Madden NFL 10, Need for Speed SHIFT, The SIMS 3 and Battleﬁeld: Bad Company 2. Autodesk Maya 3D animation software, a key element of EA’s game development toolkit, is used to process most of EA’s game assets, including animations, models and textures used to support game play. By integrating Maya with IncrediBuild, EA was able to dramatically enhance Maya’s performance. This case study explores how EA used IncrediBuild to accelerate the development of the SIMS 3 game.

THE CHALLENGE For the development of the new SIMS 3 game, EA used Maya to convert asset files into their game-ready formats. SIMS 3 has tens of thousands of asset files and required building an enormous amount of data files. Rendering all these asset files took several hours each. EA’s build system is set up as follows: • A Python script launches runs of mayabatch.exe. Each run of mayabatch outputs a single independent asset file (either animation or geometry). • All mayabatch runs take place on a single machine, and all output files are saved to a folder on the machine.

PAGE 16

• The Python script then processes all these ﬁles and stores them in a single data pack accessed by the game. Typically, the runs of all mayabatch calls on a single 16 CPU machine took about four hours. EA was looking for ways to improve performance and reduce runtime. By reducing the time of the Maya runs, EA could signiﬁcantly expedite the entire application development cycle.

THE SOLUTION EA has been using IncrediBuild’s Xoreax build acceleration product as it desired a similar way to accelerate Maya processes. As opposed to running Maya on a single machine, the product distributed Maya’s subtasks (“mayabatch.exe”) to unused resources on the network, making use of idle CPU cycles on those machines. Agents were installed on seven machines (user workstations – no dedicated servers were required) for a total of approximately 108 CPUs. Each machine was utilised without impacting local user performance. The output ﬁles from these remote runs were copied by IncrediBuild back to the host machine. Finally, a Python script processed all output ﬁles to create a single data pack. Running processes on remote machines in parallel allowed Maya to dramatically reduce its overall runtime from four hours to only 40 minutes, resulting in 6x faster processing results, which has maximised productivity and enabled EA to meet deadlines. In addition, EA used time saved to add enhanced features to the SIMS 3 game.

AUGUST 2013 | www.testmagazine.co.uk

COVER STORY

ZIAAN HATTINGH MANAGING DIRECTOR INDIGOCUBE

IS IT TIME TO RETHINK YOUR APPROACH TO SOFTWARE TESTING? As the outsourced offshore software testing debate continues, Ziaan Hattingh, managing director of IndigoCube, gives more reasons as to why last-minute, resource-intensive manual testing, which is often outsourced offshore, is increasingly inappropriate given software’s growing importance as an enabler of business success…

n the past, software testing was typically something that occurred late in the development cycle. The typical waterfall process culminated in a big-bang implementation. Testing was largely manual and resource requirements fluctuated substantially. In addition, it was seen as a discrete process, one that certainly was not core. It therefore often made sense for testing to be outsourced to an offshore destination where cheaper resources would test manually, and the factory configuration could accommodate the shifting resource demands. However, the results were often not that satisfactory. It’s hard for a manual process to cover all the possible contingencies, no matter how many resources are applied to it. More importantly, the overarching development process itself was found to be less than ideal. The big-bang approach at the end of the

PAGE 18

traditional waterfall process means that defects are only evident at the last minute, which can come with grave consequences. Deadlines are often not met and, worse, users have to contend with sub-standard software. Fixing bugs right at the end of the development cycle is also extremely expensive.

POOR QUALITY SOFTWARE Poor software quality is estimated to cost business more than $500 billion worldwide per year, according to Capers Jones research.1 In today’s tight business conditions and with governance controls ever more stringent, budget overruns, traditionally associated with software development, are simply unacceptable. Most persuasive of all, it has become clear that a business’s ability to achieve sustained success is now very

AUGUST 2013 | www.testmagazine.co.uk

COVER STORY much dependent on its software. Software development has become strategic; a core competence for many companies. Indeed, in many industries, excellence in software development is becoming a competitive differentiator. Research by the IBM Institute for Business Value, suggests that almost 70 per cent of companies using software development for competitive advantage outperform their peers in profitability.2 Better testing does not only save money, it produces software that works better and is more closely aligned with what the business requires.

AGILE APPROACH In response, the process of software development has been undergoing substantial refinement. Today’s Agile method has created an iterative development process that produces results quickly – and testing is integral to these rapid, iterative cycles. It can no longer be relegated to a separate process right at the end of the development cycle, but must become part of the process itself. Agile-style development relies on closely knit teams. Aside from considerations of efficiency and cost, to have testing done outside of the team runs contrary to the spirit of Agile, and would compromise the whole process. Not coincidentally, technological advancements have made this kind of rapid, integrated, repetitive testing much more feasible. One element is the huge advancement in virtualisation, which makes it possible to test early even in the most complex, interconnected environments. Virtualisation also makes it possible to predict what resources would be needed more accurately, thus removing the need for the scalability that offshore testing factories can provide. Another important advance has been the growing sophistication of testing tools. These tools automate some of the testing process and, when used correctly, reduce the need for resource-intensive manual testing. Rather than many low-skilled resources, in fact, new best practices require fewer, more highly skilled resources. At the same time, wages in offshore locations like India have been on the rise, removing the cost advantage of manual testing. And as software development has grown in strategic value, the attractiveness of outsourcing its testing has waned. Testing is now seen as integral to what has become a core process on which the company’s future success depends.

NEW DEMANDS IN SOFTWARE DEVELOPMENT The tight integration between development and testing, a hallmark of the Agile approach, is emphasised as companies respond to the startling growth in mobility. The world of the mobile app – especially when it comes to apps destined for customer use – is placing new demands on software development and thus on testing. In the cutthroat world of the app store, speed to market is vital. But, so is quality. No longer are the frustrations of poor software masked by the corporate firewall – the app is running on the customer’s device. The reputational damage of a poor app can be substantial. Apple’s

On the contrary… Indian offshore testing remains strong, according to Hardeep Singh Garewal, President - European operations, ITC Infotech. While Hardeep Singh Garewal doesn’t deny that onshore testing in the UK may be growing, he doesn’t see a decline in offshore outsourcing testing: “We actually inaugurated a new testing centre of excellence in Bangalore earlier this month, which specialises in mobile and cloud performance testing. Given the way performance testing is being positioned, we see very clearly that if you have the right tools deployed as a service, and the original things that made offshore so strong: scalability plus the skills and the time zone and cost benefits, then the case for offshore testing remains strong. “We have been strong in testing for a long time, particularly in the banking and financial services sectors and especially using offshore test factories. Recently, a large bank in the Middle East moved its testing teams and service from on site to Bangalore. Offshore is still a very powerful proposition. “As far as Agile development is concerned, what is extremely important is the articulation of requirements and user stories. Secondly, in terms of the testing team itself, it needs to be involved right from the beginning. We use an offshore/onsite model which we have deployed successfully many times, but people are sceptical. “We have a concept called the ‘extended enterprise’ where the point is not about whether you are sitting in front of the customer or not. When we work with clients, we become an extension of their team and we spend a lot of time building these relationships, so all our people don’t have to be sitting next to their clients on site. “If you invest enough time and are flexible, agile and adaptive then this is perfect for managing an Agile project. If you employ things like cloud deployment and agent-less tools, and have the client’s trust, then as far as the technology is concerned, you can do the testing from anywhere. “The same model works very well in the mobile sector, and we have spent a lot of time investing in this area. There is significant growth in testing, and because of the need for speed to market, especially in the mobile sector, testing is becoming more important. There is a very big market out there.”

poorly-received map app is a prime example. Mobile users will drop a poorly performing app very rapidly, and are unlikely to be tempted by future apps from the same source. Apps also need to be updated frequently. In short, quality and rapid development must go hand-inhand. The case for building a sophisticated, in-house testing capability is now strong even though it creates the need to invest in training. Correspondingly, the business case for manual testing, particularly when outsourced offshore, has become much weaker. If software is, or could become a source of competitive differentiation, perhaps the time is ripe to rethink your approach to testing.

References 1. Capers Jones, Software quality in 2010: A survey of the state of the art (November 2, 1010 [sic – presume 2010] 2. Mark Albrecht, Eric Lesser and Linda Ban, The software edge: How effective software development and delivery drives competitive advantage (IBM Institute for Business Value, 2013)

AUGUST 2013 | www.testmagazine.co.uk

PAGE 19

INDUSTRIALISED IT DAVID RIGLER DIRECTOR SQS, CENTRAL AND NORTHERN REGION

IT: A DRIVER FOR QUALITY MANUFACTURING

David Rigler, director, SQS, Central and Northern Region, discusses how the industrialisation of IT quality in the manufacturing sector will be the norm in the next five years, and why itâ&#x20AC;&#x2122;s important for manufacturing companies to embrace this trend that is here to stayâ&#x20AC;Ś

anufacturing organisations already live, eat and breathe quality practices and processes, including lean manufacturing, six sigma and metrology. They know that poor quality products can cause massive reputational and financial damage, with expensive recalls and reduced order numbers impacting profitability. Manufacturers also recognise that high quality is an enabler of business agility. Quality driven manufacturers can respond more readily to changing customer demand and technical advances, and consequently maximise the opportunity to increase revenue and beat the competition. But it seems odd that often the quality of the IT systems underpinning the whole business, such as ERP, PLM, MES or CAD, is not given the same priority.

FEAST AND FAMINE APPROACH As more of the manufacturing sector becomes dependent on IT, it is worth reflecting on what improving IT quality means. Entry-level IT quality is focused on detecting high risk defects before they cause damage. In the past, IT quality has relied on the manual testing efforts of large teams often drawn from several departments across the business. This manual testing effort is time consuming and reduces overall productivity of all teams during the testing phase. The result is that any retesting following changes can be very expensive. However, if the testing is not conducted, the risk of introducing a regression into the software is high.

famine approach to IT change, the IT department often becomes the bottleneck to change within a business. Higher levels of IT quality call for the prevention of defects. This can be achieved through the introduction of an effective quality management regime covering the entire software lifecycle; from requirements validation to production monitoring. A key part of this process involves significant amounts of regression testing, so that every time a change is made, it can be demonstrated that the change has not impacted the key business processes. Improving quality costs money, and spending more money on quality eventually results in diminishing returns. Therefore it is necessary to control the investment in quality based on an assessment of risks to the business. However, having found a defect in production that was not found in testing, it is important to ensure that it does not reoccur. Consequently, quality is also about effective reporting and root cause analysis so that an organisation has the ability to learn and respond quickly to an IT production issue with exactly what occurred; how it managed to get to production; how it was/will be fixed; and how reoccurrence will be prevented in the future. The problems with manufacturing IT quality are only set to increase as more organisations attempt to integrate their core IT applications; doing so can provide significant return on investment but can also substantially increase the risks posed by changing an application.

Therefore an entry-level approach to IT quality leads to a state where a change of IT systems is deferred until it becomes unavoidable through obsolescence or lack of competitiveness. With this feast and

LARGE SCALE AUTOMATION UNDERPINS THE BUSINESS AGILITY OF ORGANISATIONS...

PAGE 20

AUGUST 2013 | www.testmagazine.co.uk

Event Loading...

London Autumn 2013

Transforming Testing: Identifying Solutions that Work This highly anticipated event now returns to London on 24th October 2013. To register your place today just visit: testexpo.co.uk

INDUSTRIALISED IT For example, an isolated ERP application that fails can only impact the business processes using that system. However, if the ERP system is connected to other parts of the IT estate, one failure could cascade to other parts of the estate or cause extensive data corruption. Only an effective quality regime will produce the business confidence to apply patches, hotfixes, upgrades and new functionality to different parts of the interconnected whole.

The techniques used include:

In essence, quality enables change and change enables business agility. The amount of IT change that is available in the manufacturing domain is increasing and stems from a number of factors including:

• Code scanning.

• The need for software to control manufacturing advances: For example, additive manufacturing and new materials. • Greater flexibility offered by IT advances: For example, mobile and cloud. • The desire to take advantage of advances in newly applied technology: For example, virtual manufacturing and virtual commissioning. It is also clear that many manufacturing organisations around the world need to improve profitability levels. This can often be achieved by greater standardisation through the use of IT solutions. These standards need to be kept up-to-date as improvements or regulatory changes are identified which, in turn, lead to additional, essential changes.

LARGE SCALE AUTOMATION Rather like manufacturing, IT is becoming all pervasive in a number of other industries. For example, banking is an unsustainable business without high-availability reliable software systems; retailers are increasingly relying on IT to ensure that their complex multi-channel, multi-priced, stock optimised businesses are as profitable as possible; and gambling companies are now competing by adding features to ingame betting software that can be delivered through every available type of console or mobile device. These business verticals and the manufacturing sector are increasingly using software quality to support successful launches of new products or services, and neutralising the competition as swiftly as possible if they find themselves outmanoeuvred.

• Outsourcing quality management and testing to specialist testing providers. • Setting up test centres of excellence. • Large scale automation covering unit, component, service and user interface testing. Large scale automation underpins the business agility of organisations including Google, Facebook, Twitter and Amazon. Their ability to innovate has seen a meteoric rise in company fortunes. Powered by automation, this new breed of organisation is able to put several new releases of code into production within a week. With adoption rates growing in the manufacturing sector, industrialised IT quality is an emerging trend among some of the more forward thinking software quality and testing specialists.

WHAT NEXT? The rate of change in manufacturing IT will not slow down for several years. Organisations that are embracing change will thrive, while companies that choose to rely on their present approach to IT quality will struggle to compete. For those organisations that have large order books, it may seem that the need to compete is not as strong; however, even in these cases, higher quality IT will allow them to improve profits and ready themselves for the next challenge. Until recently, one of the primary reasons for a manufacturing company to industrialise IT quality was to provide evidence of testing, for example, process manufacturing companies that have to provide evidence for GxP or other standards such as J-SOX. Some of these companies have embraced automation and managed testing services to provide this evidence. However, over the next few years, all manufacturing companies should carefully consider the benefits provided by industrialised IT quality. Not only in terms of evidence for reporting, but also to provide greater business agility to enhance revenues and ensure standardised and crossapplication processes to improve profits.

Industrialisation of IT quality is being driven by the factors shown in the table below: Factor

Rationale

Large scale

Most large businesses have a huge number of test cases that ideally need to be run when there is a change that could regress the affected area.

Continuous improvement

Companies are under pressure to increase the efficiency and effectiveness of their quality and testing processes.

Shared risk

It is not possible to carry out all of the testing that is required for every change. Therefore there is always some risk that defects will appear in production. Sharing that risk allows an organisation to confidently state to the wider stakeholders in the business how they are mitigating the overall risk.

Focus on core business

Quality management and testing are disciplines in their own right and can distract a company from focusing on their primary goals.

These factors are driving the uptake of managed testing services; shared risk/reward models based on output- or outcomebased pricing; and test centres of excellence tasked with driving continuous improvement, such as SQS’ Test Automation FaQtory, for example.

PAGE 22

AUGUST 2013 | www.testmagazine.co.uk

TESTA

A WORD FROM THE JUDGES… THE EUROPEAN SOFTWARE TESTING AWARDS

CELEBRATING TECHNICAL EXCELLENCE

In the lead up to the first ever European Software Testing Awards (TESTA), Sophie-Marie Odum catches up with some of the judges to find out what they are looking for from entries and why they are looking forward to the Headline Sponsor Awards ceremony…

Something special and innovative, as well as something that is effective and of high quality is what I will be looking for from entries.

I will be THE EUROPEAN SOFTWARE TESTING AWARDS

looking for innovation in the

I'm excited to be a judge as I can find out what the latest issues are; new techniques; new problems being faced by testers; as well as how the testing community is developing in terms of professional status. I'm also looking forward to meeting the best practitioners and see what is happening in the industry.

CELEBRATING TECHNICAL wayEXCELLENCE things have been tested,

the product and its challenges, and something that will inspire others to do things differently.

Being a judge for TESTA offers an amazing opportunity to look at what the industry is achieving, and I’m really looking forward to the awards ceremony and celebrating our vocation with peers.

KAREN THOMAS, SENIOR PRACTICE MANAGER, BARCLAYCARD CHANGE PERFORMANCE AND CONTROL

MIKE HOLCOMBE FOUNDER AND DIRECTOR, epiGenesys

In order to win one of the TESTA awards, entries need to stand out by evidencing that they have pushed the boundaries, evolved testing methodologies whilst working against the usual demands testing faces such as challenging timescales, limited resources and, more often than not, changing requirements.

Being involved in the first ever TESTA is a huge compliment. Having been in the testing industry for over eight years, this is the first time I have witnessed the testing community getting the recognition they deserve. I’m also excited to have the privilege of seeing all of the applicants and, no doubt, the amazing dedication and commitment shown.

LISA DONOVAN PROGRAMME & QA MANAGER, PROXAMA

PAGE 24

The 20th November will offer a great opportunity to meet lots of like-minded test individuals and teams, and celebrate the success that our industry truly deserves.

AUGUST 2013 | www.testmagazine.co.uk

TESTA I am proud to be chosen as a judge for the first ever TESTA awards and will represent both the UK Testing Board and the BCS. I am excited about the number of submissions we will receive; and the ability of the entrants – both teams and individuals – in developing and delivering innovative and agile testing solutions and services in the real world to real clients.

When judging the entries, I will be looking for “Passion”: passion for testing; passion for making a difference; passion for innovation; passion for the clients; and passion for the industry. I will also be looking for originality – looking at something old in a new way; or finding a new way to test something; or a fresh application for a product; or presenting a novel way to process data, will stand out. A clear submission is also important. Keeping it simple will ensure the judges can understand the submission.

I am really looking forward to reading all the submissions about how individuals and teams have made a real difference to the client, pulled out all the stops, and developed an original idea, process, tool or accelerator.

IAN HOWLES SENIOR DIRECTOR QUALITY ENGINEERING AND ASSURANCE SERVICES UK&CE

The award ceremony will be a culmination of a lot of hard work for the organisers, the judges, but most of all the entrants. The ceremony event is an opportunity to meet all the people who are making a difference in the industry and celebrate the best of the best.

This is an exciting time for the software testing industry with the introduction of innovative resource models, testing tools, and cloud-based technologies to drive the overall efficiency and effectiveness of QA organisations. The TESTA awards provide a real opportunity to showcase these innovations to a worldwide audience and help our industry demonstrate the ability of testers to add genuine value in the development of IT solutions. As a judge for this year’s awards, I’m excited by the opportunity to find out more about the latest challenges presented to our industry and, more importantly, how testing companies and individuals are responding to these challenges. I’m really looking forward to the awards ceremony in terms of celebrating testing innovations and having the opportunity to come together with cutting-edge solution providers with a view to exchange ideas on testing best practice.

AUGUST 2013 | www.testmagazine.co.uk

The key things I’ll be looking for from entrants are demonstrable evidence of: • Consistent delivery and successful engagements with clients. • A skill to respond creatively to common testing challenges. • Capability to create innovative tooling solutions to both common and new testing challenges.

SIMON JONES TEST MANAGER (GLOBAL BUSINESS SYSTEMS), AMEC

• An ability to shape testing solutions for clients, pulling on a wide range of resourcing models, tools and methodologies with a view to providing bespoke ‘tailored’ approaches.

PAGE 25

TEST DATA

SRIKANTH DORA KARAM TECHNICAL LEAD VALUELABS

NARAYANA MARUVADA SYSTEM ANALYST - QA VALUELABS

LINKING TEST DATA MANAGEMENT AND QUALITY SOFTWARE With the increasing complexity and scaling of IT applications, together with the growth in end-user demands and expectations over quality attributes, it can be difficult to carry out the typical verification and validations activities for assessing software quality. Srikanth Dora Karam and Narayana Maruvada discuss how affordable test data management solutions can support test efficiency and streamline efforts…

enerally, test activities are aimed and designed to simulate the real world usage of the system, as well as to detect problems before they affect the end users. Therefore a comprehensive and most realistic test effort is planned and implemented to ensure reliable testing which predicts and provides an opportunity to correct any erroneous behaviour. As the majority of today’s business applications are datacentric, testers need to understand and appreciate the fact that data plays an important role in the test efforts. In fact, test data development and its management activities constitute a major part of any testing effort. So, test teams need to ensure that test data, with characteristics close to real production data, should be considered to help properly test and evaluate the system’s behaviour.

Quality test data will truly reflect the characteristics of your production data set. But, just replicating production data is not a realistic or cost-effective option considering the risk, data security and compliance measures associated with it. Furthermore, using production data sets might set up data storage and maintenance costs which indirectly bog down the test efforts, reducing the overall test efficiency. So with this being said, “What are the basic attributes that define good test data?” Good test data correctly represents the full range of production or real world data, and is sized appropriately to support or suit the testing needs. Improper test data demands longer test execution times, which proportionately increases the test cycle times and eventually impacts the overall test inefficiency; leads to maintenance overheads in terms of storage and cost; increases analysis and debugging efforts; and, more importantly, inaccurate test data increases the business risk due to incomplete or very unreliable test results. As quality test data should be a representative of production data, certain critical activities such as data profiling and AS THE MAJORITY discussions with business OF TODAY’S BUSINESS users etc. are carried APPLICATIONS ARE out to help the team DATA-CENTRIC, TESTERS understand what

NEED TO UNDERSTAND AND APPRECIATE THE FACT THAT DATA PLAYS AN IMPORTANT ROLE IN THE TEST EFFORTS

PAGE 26

AUGUST 2013 | www.testmagazine.co.uk

www.softwaretestingawards.com THE EUROPEAN SOFTWARE TESTING AWARDS

CELEBRATING TECHNICAL EXCELLENCE

Headline Sponsor

THE EUROPEAN SOFTWARE TESTING AWARDS

CELEBRATING TECHNICAL EXCELLENCE

An independent awards programme designed to celebrate and promote excellence, best practice and innovation in the software testing and QA community.

If you would like to enter the awards please contact the team on: +44 (0) 870 863 6930 or email awards@softwaretestingawards.com Headline Sponsor

Supported by

Category Sponsors:

TEST DATA

makes the production data interesting. As part of this exercise, the team might come across the following questions and look for suitable answers: • Are there any data quality related concerns? i.e. is the data well organised, complete, reliable and timely?

ABOVE ALL, IT IS IMPORTANT TO CONSIDER AND UNDERSTAND THE OVERALL CONTEXT IN WHICH THE TEST DATA WILL BE USED

• What is the importance or significance of specific test data? • What data or its combinations are most commonly used? • Which data or its combinations tend to be problematic? etc. Now, having answered all these questions, the testing team will be provided with the requisite information to help them get through the data selection process and prioritise their testing efforts. Understanding key pieces of information related to data ranges and its limits, data relationships, data dependencies etc; and understanding the business users and using the appropriate data profiling tools, are preferably the best ways to develop an insight about the test data. Next, ensuring an optimised data set is another quality of good test data. This optimisation can be achieved by following some established testing practises, where certain test cases are formulated using a variety of standard test techniques such as boundary value analysis, equivalence class partitioning, etc. Above all, it is important to consider and understand the overall context in which the test data will be used. This can be achieved by having a comprehensive questionnaire that the testing team use to come up with questions specific to applications and/or environments. This should be included as part of the test planning itself to ensure that a complete set of test data requirements are ready before actual testing commences. Nevertheless, the test team should ensure to document both the questions and the answers so that they can be reused and revised over time.

A METHODICAL APPROACH Once the test team has developed the test data requirements and/or a questionnaire to answer, a methodical approach for organising and analysing the information is needed. This can be achieved by formulating a suitable test data management framework to help organise the information and maintain the information about test data over time. So, as the testing team attempts to build this framework, it is essential for them to provide a thoughtful consideration on the following data characteristics and/or attributes:

1. Classification of test data There are many different ways to classify the test data, but when it comes to test data management, it is important to consider these three fundamental classifications: • Environmental data defines the overall application’s operational environment and forms the crux of the

PAGE 28

complete test effort, as it establishes and/ or forms the actual test execution context. Environmental data typically includes the data and details pertaining to system configuration (i.e the operating system, databases, application servers etc.); and user authentication and authorisation (system access levels for generic and role-based or tester specific account, user-ids, passwords etc.). Generally, this data is identified and prepared at the very initial stage of the project and is maintained thereafter.

• Baseline data establishes a meaningful starting point for testing and establishes a set of expected results. This data is recognised by just looking at the test-case prerequisites and it typically includes meaningful business data. • Normal input data is typically entered into the system under test (SUT) to evaluate how it responds to the provided input. The observed behaviour establishes the actual results, which must then be compared to the expected results to determine the correctness of the behaviour.

2. Sources of test data Data comes from a variety of sources and can be found in almost any format. However, the major sources of test data can be: • Simulated Data which is highly useful where the production data sets may not contain values that fit for the test. Examples include unit testing, error checking etc. The testing team would need simulated data when working with new functionalities or when there are no historical data or production data equivalents. • Replicate of production data. This forms the major part of test data that is used. The production data is the best source for obtaining test data to simulate production like characteristics. However, the team should avoid the use of complete production data copies and if used, the data set should be properly sanitised to avoid the risk of data security breaches.

3. Selection rules for test data Generally, the outcome of the test data requirements process is a detailed understanding of the data and a set of test specific needs. Now, as the team builds the test base, they develop a long list of test data criteria that accounts for default conditions, redundancies, positive and negative testing scenarios, cross-project dependencies and other statistical characteristics of data, etc. Eventually, this exercise leads to developing a most comprehensive set of rules which would be very helpful to the team for selecting the most suitable test data. Testing efforts may have increased over recent years and, as a result, degraded the test efficiency by some numbers. Although many solutions and/or factors were introspected and implemented to counteract such situations, they have been proven ineffective and costlier later. However, the data management solutions offered above can help optimise test efforts and add value to the business by retaining quality.

AUGUST 2013 | www.testmagazine.co.uk

TestPlant has emerged as one of Europe’s largest specialist tool vendors Source: PAC 2013

Real user experience

= To find out more visit testplant.com

THE QUEEN’S AWARDS FOR ENTERPRISE INTERNATIONAL TRADE 2013

MOBILE APP TESTING MARTIN WRIGLEY CHAIRMAN, App Quality Alliance (AQuA)

MOBILE APP TESTING IN THE CLOUD Martin Wrigley, Chairman of the App Quality Alliance (AQuA) looks at mobile app testing and what impact the trend of testing in the cloud might have… recent report from Research and Markets, Global Software Testing System Integrator Market 2012-2016, discussed the trend of vendor offerings geared towards cloud-based testing solutions. Now this may be all well and good for cloud-based enterprise systems, but what about mobile apps?

Once you agree that you need to test a mobile app (and frighteningly many available apps are totally untested until some poor unsuspecting user loads it on their device), your next question might be how to go about it and what tools are there to help and automate that testing. Mobile app testing is a tricky topic, full of traps ready to catch the uninitiated. Basic functional testing is relatively straightforward, as it uses a single device and controlled conditions. But the reality of deployment is that there are many different devices that behave in different ways with different network connections, battery and performance characteristics. The traditional approach was that a developer needed to physically functionally test their app on every device that they targeted. Clearly this was uneconomical. Add to this the need for speed of development and updates of mobile apps, and testing more than the basic function of the app starts to become a seemingly more complicated process. In the ideal world, a developer would rely on the accuracy of a device emulator, and easily port an app across platforms (from Apple iOS to Android to Microsoft Windows Phone to Blackberry) but that is very rarely a reality. In some tightly controlled closed platforms, the issue is reduced as there is a single version of the hardware and OS from a single unified manufacturer. However, this is just one part of the market and the rest is still as fragmented and confused as ever. With no magical solution in place, we are starting to see developers trending towards cloud-based solutions, which are becoming part of their answer. To begin with, there is no substitute to having a real target device in the development process. The app speed and feel is vital, and the emulator can only get you so far. Functional testing by the developers and then QA testing by an independent test team using a functional test script, combined with specified testing criteria, can get you to a good working app on one device. Many apps have a continuous improvement cycle in place, and new releases are put out to the users every few weeks or months. One critical aspect to support this is the ability to re-run the functional testing scripts and check that only the

PAGE 30

intended changes have happened. Such script automation is feasible on a physical device, but if the original functional test script is also implemented in a cloud-based tool then this can also be very effective. Having achieved a well-behaved app on one device locally in the lab, there are a few other variables in the reallife scenario that need testing, to answer: • Does the zz run well on other devices? • Does the app work in other geographies? • Does the app perform well in variable network conditions, for example changing from Wi-Fi to 3G to 2G to losing the network altogether in a railway tunnel? • Can cloud testing help here? Cloud-testing really comes into its own when testing for the app’s compatibility on a wide range of devices. Having a bank of devices tethered in the cloud and available for a brief period of time at a fraction of the cost of owning the device is a great boon and clearly a sensible way to share such resources. Such cloud-based solutions can really help by allowing a set script to be run against a large range of devices to find any discrepancies. This is a major task that many developers face without such a tool. However one aspect that is much tougher in the cloud is geography and location. The testing of, for example, a money transfer system on specific handsets overseas can really only be done with a real device on the ground in the right geography. Moving from location to location can be simulated in the cloud, but it can’t test the apps behavior as a GPS signal is gained or lost as the user moves indoors. And then of course there is the network connection. Many apps go awry as the connection gets chopped and changed by the device. Whilst this can be simulated, the bank of devices is actually in a fixed datacenter somewhere. They’re not truly floating in the clouds so can’t really experience the everyday phenomena of losing signal. There is no doubt that cloud tools for mobile testing are increasingly getting better. And although many challenges remain within mobile testing that cloud solutions inherently cannot address, a developer and test team armed with the right tools can successfully overcome such hurdles.

AUGUST 2013 | www.testmagazine.co.uk

FOCUS ON: CAREER

CELEBRATING THE INDUSTRY Many different surveys suggest that UK workers are stressed and unsatisfied but, on the contrary, others state that employees are happy with their work-life balance. With the average person in the UK is estimated to spend about 99,117 hours at work – that’s over 4,100 days or 11.3 whole years – it’s only right that you enjoy what you do. Sophie-Marie Odum reports…

espite working longer hours than most other sectors, 64% of IT and telecoms staff reported that they were happy with their work-life balance in a recent survey, which is well above the national average of 59%. Although this figure is not specific to the software testing profession, it wouldn’t be wrong to assume that the level of job satisfaction amongst software testers is high. Typically, they are last stop before software goes live, indicating that software testers are an integral member of the team. Samant Katyal, senior consultant at CapTech Consulting, proudly raves, “Software testing is that one unique field which bridges the gap between business requirements and technical specifications.”

RECESSION-PROOF INDUSTRY Nowadays everything we do is predicated on IT. Add to this the increase in mobile apps and an enormous and almost constant demand for software testers is created, making it a recessionproof industry. It was recently reported in the June 2013 Adzuna Job Market Report, that the average salary of an IT and technology professional in the UK now stands at £43,127, an increase of 5% over the past six months. This bucks the UK trend, contrasting to a fall in average salaries across most sectors and demonstrates the high salaries that those with specialist knowledge and relevant experience can command. Likewise, it’s become very important for software testers to have comprehensive knowledge of the business domain and an understanding of their clients’ businesses. However, becoming a software tester is not a spare of the moment decision; it’s a “calling”. Samant added, “It’s funny how often those of us in the domain of software testing and quality assurance often consider it a ‘calling’. Almost as if you have to be wired in a particular way which makes you not only well-qualified, but a natural at identifying defects. “Think back to childhood, when we tapped into our basic instincts of curiosity and ingenuity. There were some of us who found enjoyment in taking things apart, and then figuring out how to put them back together (the engineers). Then there were those of us whom focused on the sheer enjoyment of simply figuring out how to take the things apart (the testers).”

RISING TO THE CHALLENGE Explaining why he enjoys being a software tester, Samant continued, “Testing software is like a puzzle, better yet, a challenge! I became a software tester as a teenager, working a retail job selling popcorn

PAGE 32

AUGUST 2013 | www.testmagazine.co.uk

FOCUS ON: CAREER SOFTWARE TESTING IS THAT ONE UNIQUE FIELD WHICH BRIDGES THE GAP BETWEEN BUSINESS REQUIREMENTS AND TECHNICAL SPECIFICATIONS at the local movie theatre. Six years into the job as a ‘supervisor of concession operations’, I was presented with a unique opportunity. It was user acceptance testing of a cutting-edge, stateof-the-art, touch-screen till... or, sorry, a ‘retail point-of-sales system!’ “This was no ordinary cash register! They were going to replace our old-fashioned cash registers with these new computerised systems, and asked me to figure out ways to break it? My first question, ‘Do you mean I can intentionally spill soda on the machine as to simulate a potential accident?’ I was formulating all kinds of scenarios to simulate (little did I know I was compiling a list of test cases which their own internal teams hadn't thought of). However, they steered me away from testing the hardware. They wanted to me to focus on all the various types of transactions and interactions with the system itself. They wanted me to test the software. “How can I crash their system by virtue of executing typical and atypical transactions? I had one shift to come up with these test scenarios, and a second shift to execute them. By the end of both shifts, time had flown by so fast. I was in sheer enjoyment of the temporary role and honour that was bestowed up on me. It was then that it clicked – how do I become a full time software tester, so I can get paid to do something I enjoy.”

GOING GLOBAL Looking beyond software testing in the UK, there is a growing need for skilled professionals in this field as outsourcing becomes more popular (debatable). However, a recent report by Infiniti Research showed that there is a shortage of such skilled professionals to meet the demand. The report cited that India, for example, was expected to face a shortage of 39,000 to 41,000 skilled workers in the next few years. As such, the Malaysian Software Testing Board (MSTB) is spearheading the planning and implementation of the Malaysia Software Testing Hub (MSTH) initiative. The Board has embarked on a programme to help the adoption of industry-developed software testing skills and promote software testing as a viable career among existing software professionals as well as university students in the software engineering and other IT fields.

GATEKEEPERS TO QUALITY As organisations generally will not allow software to be released until it has been fully tested and approved by their software quality assurance group, software testers are the gatekeepers to releasing high quality software products. “Software testers have the most in-depth knowledge and holistic view of the end-to-end system/solution being implemented,” said Samant. “Others may not have had the same opportunity to dissect, detect and decipher at the same granular level.” If you needed another reason to rejoice in your chosen career, feel safe in the knowledge that you didn’t choose to become an accountant. The same study cited earlier found that this group of professionals was least happy with their work-life balance (42%), even though they have a shorter average working week than the UK average! References available on request.

AUGUST 2013 | www.testmagazine.co.uk

PAGE 33

FOCUS ON: CAREER

BEN WILLIAMS SOFTWARE TESTING PROFESSIONAL

ARE YOU REALLY A TESTER? Does the advancement of tools in the software testing arena challenge the idea of being a software tester? Software testing professional, Ben Williams, answers… was once told testers are not created, but they are born. As a tester, you must have the right attitude and inquisitive mind to dive into something and try and break it. However, because today’s tools challenge the idea of a tester, we, as a testing community, have to clearly define what testing is.

TESTING VS CHECKING •

Testing is the process of evaluating a product by learning about it through experimentation, which includes to some degree: questioning, study, modelling, observation and inference.

I recently came across a great blog called, Testing and Checking Refined1 by James Bach and Michael Bolton, and they raised the valid point that with today’s fast pace demand to get products to market or tight deadlines to hit “go live” dates, there is a growing tendency to say “Let’s use tools”, or “Let’s build our own tools” for testing. Yes, test tools have been around as long as testing, but the key difference today is time, or lack of it. Projects and products run out of this precious commodity before they even start. There is a an ever increasing demand for instant results and one of the ways to get this is to go and buy or build tools to do the job of the tester quicker.

•

Checking is the process of making evaluations by applying algorithmic decision rules to specific observations of a product.

James and Michael write, “We believe that skilled cognitive work is not factory work. That’s why it’s more important than ever to understand what testing is and how tools can support it.” I tend to agree. We are in a world where ‘testing’ presents itself in many different forms and we are in danger of watering down what it is and the benefits it provides, which may eventually dilute the skills of a tester.

EVERYONE IS A TESTER! One of the issues we have in the test industry is that everyone is a tester... For example when you’re searching for a new car, you go for a test drive (you guessed it) you’re testing; when you trial a product, you’re testing; when you invite a few friends over to sample a new wine or cocktail, and you gather their feedback, you’re effectively a test manager. This is loosely speaking of course, but the list goes on. Now, because everyone is a “tester” this gives society preconceived ideas about what testing is and what is needed. Therefore, they tend to drive testing in ways it shouldn’t go or suggest the use of tools at the wrong time. Why? Well because mainly we are human and we form opinions based on what we already know, however little. Therefore we need to differentiate between the everyday tester and the skilled tester with the use of tools.

The use of tools in testing is what James and Michael say should be defined as “checking”. To differentiate:

Checking is broadly defined as the use of tools in testing, although it could also be humans following the same algorithms and rules on a script. Tools are a must in testing today and need to be developed to allow better and faster turnaround in patches, upgrades and getting products to market. We use pre-set algorithms and rules to produce an evaluation of the product, which cannot be passed or failed until a human has analysed the results. This can cut days or weeks out of a test plan compared to human checking, so in terms of money, resource and time, it makes cost effective sense to have tools to do the checking. But this is not testing… Testing is what us as skilled testers provide! A true tester never stops with what is exactly written in a script. Instead, we are looking at what’s on screen, assessing the usability, exploring and always asking “what if I just...?” to see what happens. Tools cannot reproduce this human element of testing, at least not yet anyway, 2029 a.k.a The Terminator days are not that far away! It is having the skill of knowing when to ask the questions and when to push the right button (or the wrong one) that is important. So as much as tools are needed to provide checking of code, skilled testers are needed to go on bug hunts, to provide that element that algorithm and rules miss. As James and Michael say, we need to come together as a community and define testing and checking for the future to give ourselves clear direction and allow project teams, businesses and developers to understand where we add benefit. When to use testing and when to use checking needs to be clearly defined. Testing now is more important than ever. With most things going digital and cloud-based, skilled testers are going to be needed; we just need to let everyone else know what we do and how we do it.

References 1. Testing and Checking Refined: www.satisfice.com/blog

PAGE 34

AUGUST 2013 | www.testmagazine.co.uk

FOCUS ON: CAREER BRIAN HAMBLING CHAIR OF THE SOFTWARE TESTING EXAMINATION BOARD AT ISEB, AND AN ISTQB EXAMINER

WHICH PATH WILL YOU TAKE? At whatever stage you are in your career, Brian Hambling and Pauline van Goethem offer some helpful advice as to how you can build your ideal career in software testing…

oftware testing is a career that offers the ability to manage your own progression. As it’s a profession based on a certification scheme, which is internationally-recognised, you can take advantage of career building opportunities all around the globe. There are four layers of certification; from Foundation level to Expert level, and each step of the qualification ladder, along with the relevant experience, makes an individual more attractive to employers. There are three steps involved in planning your testing career. Firstly, it’s important to understand the different levels of certification; secondly it’s worthwhile understanding the experience needed for the next level of certification; and finally, how will you find your ideal job?

LEVELS OF CERTIFICATION: • Foundation level The Foundation level is open to novice testers, or testers who want to obtain the higher levels of certification for which the Foundation course is a prerequisite. Many testing related vacancies require applicants to have the ISTQB Foundation Certificate. In addition to certification, it is important to gain relevant experience and, if you can, gain experience in a particular

PAGE 36

application area or market sector where you can begin to focus your specialism. At the Foundation level, you may already be able to create, run and analyse tests from requirements, but aim to get experience working in, or leading a small test team in order to progress to the Intermediate level. You may also find it useful to get some experience of testing in an Agile environment.

• Intermediate or Advanced? From the Foundation level, it is possible to move onto the Intermediate level or progress straight to the Advanced level. However, the gap between the two is so significant that many testers choose to tackle the Intermediate certification first, which is based on the practical application of what is covered at Foundation level. In order to progress to the Advanced level, you should aim to get experience managing a team of testers or test functionality on a complex project. The Advanced level consists of three streams, aimed at three distinct testing roles: • • •

Test manager Test analyst Technical test analyst

For the advanced practitioner, it’s worthwhile looking for ways to increase responsibility and authority to

AUGUST 2013 | www.testmagazine.co.uk

FOCUS ON: CAREER PAULINE VAN GOETHEM EXAMINATION WRITER ISTQB ADVANCED SYLLABUS

manage a test centre and expand project management experience.

• Expert level To move to the Expert level, you must have at least two years’ experience of the specific Expert level topic. Currently the Expert level exams are: • •

WHEN APPLYING FOR THE JOBS YOU ARE QUALIFIED FOR, BE SURE TO LOOK ABROAD TOO – THINK OF PLACES YOU WOULD LIKE TO GO TO BUILD AN INTERNATIONAL CAREER

T est Manager: Improving the Test Process, Test Management Test Analyst: Test Automation, Security Testing

Following this there are no further levels, but the Expert level is renewed every five years. Renewal at Expert level may be achieved by re-sitting the exam; through other practical work such as teaching relevant training courses; or by writing a book or article.

FINDING THE PERFECT JOB These different levels provide some rough guidelines as to whether certification is relevant and appropriate to your current role and experience. But, it is possible to take a much more active approach to your testing career. Begin by constructing your ideal CV. This will help to gauge how much you already have of what you need – even if you have little knowledge or experience, you

AUGUST 2013 | www.testmagazine.co.uk

will have a starting point. Also, it might be helpful to search for your ideal job on job sites to see what experience and qualifications employers are looking for.

As you gain each level of qualification, revisit your ideal CV and re-evaluate where you are and where you want to go. When applying for jobs be sure to look abroad too – think of places you would like to go to build an international career. Remember, jobs could be based anywhere in the world so you may not want to specify a location. To get some inspiration, it may also be a good idea to look for different job titles that require the same level of certification you need. Do you want to be a proficient generalist? A specialised test manager? Or a testing expert in a specialist area? Your choice may change as you progress so always keep it under review. Nevertheless, whatever your chosen career choice, it is within reach if you are equipped with the right knowledge and sufficient experience.

For more information, BCS, The Chartered Institute for IT has a list of accredited training providers available on its website. Also, the ISTQB has more information about the Foundation, Advanced and Expert level syllabus and exams on its website.

PAGE 37

LAST WORD DAVE WHALEN PRESIDENT AND SENIOR SOFTWARE ENTOMOLOGIST WHALEN TECHNOLOGIES HTTP://SOFTWAREENTOMOLOGIST.WORDPRESS.COM

YOU CAN NEVER GO HOME AGAIN By Dave Whalen

o quote Rodney Dangerfield, “I don’t get no respect”. I have sailed on to my latest consulting gig after the last one ended somewhat prematurely (that’s another story for another day) and, even though it’s for a company where I worked as a permanent employee about 10 years ago, Cap’n Dave isn’t feeling the love. It was a start-up company when I first worked there. During my initial tenure, the company grew and life was good! But, unfortunately, when the economy tanked, there were a number of “restructurings” and I became a victim along with the majority of the test team. 10 years later, the company has grown substantially and now occupies its own building. There has been significant employee turnover, but there are still a few folks who remember the Cap’n. Fondly, I hope. It was going to be like a homecoming. But maybe I should have stayed away? Because as Thomas Wolfe wrote, “You Can Never Go Home Again”. As I sailed back into familiar, friendly waters some things had changed. Others had not. Before I was made redundant, one of my final tasks was to completely revamp the testing processes and methodology, so I looked forward to seeing how “my baby” had grown and matured. I was like a proud parent about to be reunited with my long lost child after many years at sea. How have they grown? How have they matured? Will I recognise them? It turns out that they had literally thrown out the baby with the bath water. There was nothing left! I tried to keep the hurt feelings hidden. I swallowed my pride, hoisted the Jolly Roger and got to work. But, as the days went on, the frustration built until finally I had to say something. So I pulled my WAIT – boss aside to share my sad tale and my ideas for returning us DON’T YOU to glory. He was enthusiastic KNOW WHO I AM? but advised me that it I’M CAP’N DAVE! I would be best to approach WRITE FOR A HUGELY it slow – baby steps. I wasn’t POPULAR SOFTWARE hired as a team lead or test manager and shouldn’t TESTING MAGAZINE force my ideas on them, but AND I HAVE A

instead make it a “collaborative effort.” Apparently my reputation had preceded me. I wrote an email to the team telling them how happy I was to be back and working with such a talented team of testers. It read, “I have sailed the vast Software Testing Seas, gone to many strange new lands, and learned many new things. Now I want to share my vast experiences with you, my new crew… Oh, and by the way, here’s a list of things that I think we should do... What do you think?” (Or something to that effect!) The silence was deafening. Nothing. Crickets. I thought, “Wait – don’t you know who I am? I’m Cap’n Dave! I write for a hugely popular software testing magazine and I have a blog! People love me! You too must love me!” My delicate ego was destroyed. It was like a child’s balloon in a cactus field. Pop! What to do? I went home for yet another soul searching weekend. My wife hit the problem right between the eyes, “Suck it up and play nice!” Or as she likes to put it, “Shut up and test!” So with my tail firmly between my legs and ego in check, I went back to work on Monday. I grabbed some free time with my boss and once again humbly offered my services to him and the team, with “How can I make us succeed?” We reset some expectations and we have our ship once again sailing in the right direction and towards a common destination. This has definitely been an eye-opening year for the ol’ Cap’n and although there is promise on the horizon, I have to agree with Thomas Wolfe: You really can’t go home again. To be honest, I really don’t want to. There are still too many things to see and do!

BLOG!

PAGE 38

AUGUST 2013 | www.testmagazine.co.uk

Ahead of next issue â&#x20AC;&#x2122;s 20 L eading Pro viders, here â&#x20AC;&#x2122;s a sneak pre view of a few compan ies being fea tured. . .

THE EUROPEAN SOFTWARE TESTER INNOVATION FOR SOFTWARE QUALITY

Subscribe to TEST free!

INNOVATION FOR SOFTWAR E QUALITY THE EUROPE AN SOFTWARE

TESTER

VOLUME 5: ISSU E4 AUGUST 2013 www.testmaga zine.co.uk

IS IT TIME FOR A RETHINK? QUALITY AND RAPID DEVELOPMENT MUST GO HAND-IN -HAND

INSIDE: WORLD NEW S: EDWARD SNO WDEN FEARS FOR HIS LIFE INTERVIEWS FOCUS ON: CAREER

FOR EXCLUSIVE NEWS, FEATURES, OPINION, COMMENT, DIRECTORY, DIGITAL AND MUCH MORE VISIT: testmagazine.co.uk

Published by T H I R T YO N E

www.31media.co.uk

Telephone: +44 (0) 870 863 6930 Email: info@31media.co.uk Website: www.31media.co.uk

SNEAK PREVIEW

SUCCESSFUL TEST CASE MANAGEMENT:

IT TAKES MORE THAN A SPREADSHEET Many QA teams start out using spreadsheets to track test cases and results. But spreadsheets aren’t the best approach as teams and project complexity both grow. Instead, teams need a test case management solution, like Seapine Software’s TestTrack TCM, that provides easy sharing, auditing, and linking between test cases and other artifacts... SPREADSHEETS DOCUMENT BUT DON’T TRACK AND NOTIFY Using a spreadsheet helps testers keep test cases and testing data in a single location. Part of the problem with that, though, is it’s not really a single location, but one or more files for every tester. Testers save the spreadsheets on their local computers. Some testers have multiple spreadsheets, organising them in folders based on the functionality or feature they test. Unfortunately, not everyone uses the same method of organisation, and it is often a challenge to find the test case files and status.

Figure 1. Test case management tools link test cases and test runs to create relationships and dependencies

Testers also have to type everything into the spreadsheet, from case descriptions to test run results, and they can’t easily share that information. Usually, the test manager or project manager must aggregate the data from multiple spreadsheets into a master file. The master file can’t be opened and modified by more than one person at a time, and there is also no way to track who made changes and when.

BEYOND THE SPREADSHEET Centralising data in a test case management solution improves the consistency of test cases and streamlines reporting. More importantly, it allows teams to do things that would be nearly impossible to do with spreadsheets, such as easily viewing all tasks involved in the testing effort. Many companies use Seapine Software’s TestTrack TCM to centralise test case management. One such company, Integra Telecom, realised over 600 hours of savings in four testing cycles on just one team. According to Julie Rouzee, CIO, Integra Telecom, “We are now able to perform complete product testing that covers areas that would be easy to overlook otherwise.” Automated test management tools automatically track the changes made to a test case, making everyone accountable for their work. These tools also enable users to create links from requirements to test cases to defects, and even to source code, helping ensure that downstream artifacts are related to actual requirements. A test case management tool that automatically links test cases to related requirements gives testers visibility

AUGUST 2013 | WWW.TESTMAGAZINE.CO.UK

into which requirements they are testing. If a defect is reported, it is automatically linked to the failed test case, which is linked back to the requirement. Overall, the entire team knows the state of testing and the quality of the application. The team also knows how much more testing is required, and can easily estimate the time needed to ready the application for delivery. Not long ago, spreadsheets were the most convenient way to organise the testing effort. Now there are tools designed specifically for test case management, and there is no reason to use an inferior approach. These tools enable teams to easily set up and share test cases and related information, track changes, and link artifacts to make test management easy and effective.

About Seapine Software Seapine Software is the leading provider of processcentric product development and quality assurance solutions. Headquartered in Cincinnati, Ohio, with offices in Europe and Asia-Pacific, Seapine’s development and QA tools help organisations of all sizes streamline communication, improve traceability, and deliver quality products on time and within budget. www.seapine.com

PAGE 41

SNEAK PREVIEW

ENHANCE THE USER EXPERIENCE THROUGH FUNCTIONAL AND PERFORMANCE TESTING eggPlant is a range of test tools that helps teams get software products to market faster, with higher quality, less effort, and clearer traceability... Through unique technology, eggPlant tools combine power and flexibility with a simplicity that means any tester can be productive in hours. From mobile to desktop to mainframe, from functional testing to performance testing to test environment management, from QA test automation to live monitoring, from media to defence to financial services, eggPlant test tools can help any team improve their productivity and quality. There are two key elements within the eggPlant product range: eggPlant Functional helps users to test better, faster and with less effort by automating the execution of their functional testing. eggPlant Functional has revolutionised functional test automation with its patented image-based approach to GUI testing. eggPlant Functional can interact with any device or application in the same way a user does, by looking at the screen. eggPlant uses sophisticated image and text search algorithms to locate objects on the screen in a completely technology agnostic manner and then drive the device. eggPlant’s user-centric test automation approach also makes it incredibly intuitive to write tests, so anyone can be productive with eggPlant within a few hours. Key features of eggPlant Functional: •

Test any device, any operating system, any technology from mobile to mainframe.

•

Test from any host operating system – Windows, Mac OSX, or Linux.

•

Test many languages, including Chinese, Russian, and Japanese with full optical character recognition.

•

Test scenarios involving multiple devices.

•

Create tests quickly with the intuitive test definition language “SenseTalk”; or use any language via the eggPlant Drive interface.

•

Integrate eggPlant into popular ALM tools such as Jenkins, IBM Rational Quality Manager, and HP Quality Centre for end-to-end QA test automation.

•

Test millions of combinations of input with datadriven testing.

•

Create reliable functional testing quickly with eggPlant’s rich test creation GUI and debugging environment.

PAGE 42

eggPlant Performance ensures that cloud- and serverbased applications will continue to work and perform with hundreds or thousands of simultaneous users. eggPlant Performance provides sophisticated performance and load testing tools that can test the widest range of technology, can scale up to simulate any load you need, but are still easy-to-use. The fundamental principles of software performance testing are deceptively simple – to simulate multiple users, to measure response times, to verify responses, and to monitor the use of resources. In practice performance testing tools and load testing tools require breadth and depth to manage the technical challenges that are intrinsic to testing all software and protocols. eggPlant Performance has been designed to easily manage the difficult elements so testers can spend more of their time looking at what really matters. Key features of eggPlant Performance: • Comprehensive technology support, including full web, client-side .NET, client-side Java, and TCP/UDP. • Higher-level virtual users for richer simulation and validation including eggPlant Functional, Citrix, and Selenium. Also includes eggPlant Mobile for mobile performance testing. • Scalable to the number of users you want to simulate with a low overhead. •

Intelligent test script creation tools.

•

Clear runtime control and monitoring.

•

Effective analysis and reporting.

THE EGGPLANT UX When used together, the two products above provide testing departments with the ability to understand how an application will behave in the real world in the hands of real users. We call this the eggPlant user experience or “UX”. Most software products are now distributed systems, with some components running in the cloud; some components running on a desktop; some components running within a web browser; and some components running on a mobile device. This distribution and diversity makes products much more difficult to test, especially when many test tools can only test one component in

AUGUST 2013 | www.testmagazine.co.uk

SNEAK PREVIEW

isolation, like the web browser, for example, so testing takes longer and too many faults are only found postrelease by users. The result is a bad user experience. eggPlant Functional makes it easy to test distributed products because it can test any computer from mobile, to cloud, to mainframe, in a consistent and intuitive way. eggPlant Functional can connect to all the components of an application and drive them in synchronisation by testing the GUI to give true end-to-end product testing.

REAL WORLD SIMULATION But to really understand how a distributed software product is going to behave in the real world, testers need to be able to simulate a large number of concurrent users accessing the server or cloud over an imperfect network. This is exactly what eggPlant Performance does. eggPlant Performance can simulate thousands of users accessing a server over any protocol from HTTP, to .NET, to customer UDP protocols.

The eggPlant UX is the combination of eggPlant Functional and eggPlant Performance and provides real user experience testing. eggPlant Performance puts server components and databases under real-world loads while eggPlant Functional validates that all the client components from mobile to web are behaving correctly and delivering a quality user experience, in terms of responsiveness, for example. eggPlant is the only range of test tools that combines protocol-level load injection with application-level user experience validation across all platforms to give true user experience testing.

About TestPlant TestPlant is an international software business based in London. It has development centers in the USA and the UK with sales and support centers in the USA, the UK and Asia, as well as an extensive network of business partners. The company was founded in 2008 by George Mackintosh and Jon Richards with venture capital backing and it has shown rapid revenue growth organically and through the acquisitions of Redstone Software and Facilita. TestPlant’s products are used in over 30 countries by well over 300 enterprise customers in sectors which include Financial Services, Automotive, Healthcare and Life Sciences, Media and Entertainment, Retail and Defense and Aerospace. The eggPlant range is a set of tools which supports the design, development, test and management of software

AUGUST 2013 | WWW.TESTMAGAZINE.CO.UK

applications for mainframe, desktop and mobile use in any technology platform environment. The tools are relevant in agile, mobile, web and DevOps deployments. eggPlant use improves and reports on the quality and responsiveness of software systems, reduces time to market and lowers costs by introducing process automation. The eggPlant tools have been granted patents with several applications pending. The company’s achievements have been recognised within the software industry and by the venture capital community. TestPlant is also a Red Herring Global Top 100 company and was awarded a Queen’s Award for Enterprise in 2013. www.testplant.com

PAGE 43

SNEAK PREVIEW

NEW VERSION OF DEVELOPMENT TESTING PLATFORM RELEASED Innovations in test prioritisation take the guesswork out of unit testing, enabling development to meet increasing time to market pressures while reducing software risks... Coverity, Inc. recently announced the release of version 6.6 of the Coverity Development Testing Platform, the company’s integrated suite of software testing technologies for identifying and remediating critical quality and security issues during development. The Coverity 6.6 platform offers a full range of new features and capabilities, including a patent-pending innovation in Coverity Test Advisor that helps developers prioritise which automated tests need to be run, along with the order they should be run, based on code criticality and risk, change impact and project priorities. This innovation enables development teams to use their testing time wisely – aligned to where the risk is in the code. “Time to market pressure results in continually shrinking software delivery windows, while the business risks associated with software defects have never been greater,” said Jennifer Johnson, chief marketing officer for Coverity. “Development teams don’t have the luxury of time, so wasting precious cycles running 100s or 1000s of tests that don’t move the needle is no longer feasible. Test prioritisation is a significant step forward in making unit testing more effective and efficient. By providing critical intelligence into code – to ensure the TEST right tests are run, in the right PRIORITISATION IS order – development teams A SIGNIFICANT STEP can meet rapidly increasing FORWARD IN MAKING business demands for highquality and secure software UNIT TESTING MORE with confidence.” EFFECTIVE AND

EFFICIENT

“Software drives innovation and competitive position, and visibility into and velocity

of software releases has never been greater even as deployment complexity across mobile, social and cloud platforms has increased dramatically. Companies are scrambling with constrained staff to crank out high performing software, so intelligent, prioritised decisionmaking about what and when to test can drive effective execution for software delivery,” added Melinda Ballou, program director for IDC’s Application Lifecycle Management and Executive Strategies service. Additional enhancements in Coverity 6.6 include: • Policy management: Define and enforce code quality and security policies, tailored to specific business requirements and risk by component, development team or third party supplier. Customisable heat maps provide a more granular view of software risks through early warning of policy violations, so issues can be eliminated before they result in costlier, more time consuming problems further along in the Software Development Lifecycle (SDLC). • Breadth and depth of code analysis: 10 new and enhanced analysis algorithms for finding critical quality and security defects in C/C++ and Java codebases with even greater accuracy. New algorithms include expanded coverage for OWASP Top 10 and CWE Top 25 issue detection in Java web applications. • Platform and language support: Expanded support for popular languages and platforms including C++ 11 and MAC OS X version 10.8. • C overity Certified integrations: Integrations from new Coverity Certified Partners Black Duck Software and PRQA | Programming Research enable development teams to view their best of breed, third party analysis results alongside defects identified by the Coverity platform, in a unified SDLC workflow.

About Coverity Coverity, Inc., the leader in development testing, is the trusted standard for companies that need to protect their brands and bottom lines from software failures. More than 1,100 Coverity customers use Coverity’s development testing platform to automatically test source code for software defects that could lead to

PAGE 44

product crashes, unexpected behavior, security breaches or catastrophic failure. Coverity is a privately held company headquartered in San Francisco. Coverity is funded by Foundation Capital and Benchmark Capital. www.coverity.com

AUGUST 2013 | www.testmagazine.co.uk

ONLINE THE EUROPEAN SOFTWARE TESTER INNOVATION FOR SOFTWARE QUALITY

The Whole Story Print Digital Online

FOR EXCLUSIVE NEWS, FEATURES, OPINION, COMMENT, DIRECTORY, DIGITAL AND MUCH MORE VISIT: testmagazine.co.uk

Published by T H I R T YO N E

www.31media.co.uk

Telephone: +44 (0) 870 863 6930 Email: info@31media.co.uk Website: www.31media.co.uk