TEST Magazine - August-September 2014 by 31 Media

INNOVATION FOR SOFTWARE QUALITY VOLUME 6: ISSUE 4 AUGUST 2014 THE EUROPEAN SOFTWARE TESTER

www.testmagazine.co.uk

NEW TECHNOLOGY BRINGS EMOTIONS TO LIFE

INSIDE: DAY IN THE LIFE OF... AN SDET INTERVIEW: TURN OFF SPYING, TURN ON FREEDOM

CONTENTS

INSIDE THIS ISSUE

NEWS 8 IOs outsource to overcome C compliance challenges

14.

12 ICT investments in Singapore and Malaysia rise

EDITORIAL ADVISORY BOARD 14

COMMUNITY MANAGEMENT IN THE GAMING INDUSTRY

Community management in the gaming industry Jim Woods discusses the important role of the community management team within the gaming industry...

T HE EUROPEAN SOFTWARE TESTING AWARDS 2014

TESTA supports the fight against dementia

The European Software Testing Awards announces which charity it supports this year...

INTERVIEW 18

Turn off spying, turn on freedom

Sophie-Marie Odum speaks to Martijn de Vrieze about the role software testers play in the Reset the Net campaign...

DAY IN THE LIFE OF…

18. TURN OFF SPYING, TURN ON FREEDOM

22.

DAY IN THE LIFE OF A SOFTWARE DEVELOPMENT ENGINEER IN TEST

A software development engineer in test

Shane Kelly interviews Ben Kirby, an SDET (Software Development Engineer in Test) for an online gaming company, about what his job involves and the challenges he faces on a daily basis...

24. BRINGING EMOTIONS TO LIFE

COVER STORY 24

Bringing emotions to life

Following recent news that the BBC is testing a new facial coding technology, called CrowdEmotion, which investigates how viewers react and behave towards its TV shows, SophieMarie Odum speaks to CrowdEmotion’s cofounder, Daniel Jabry, about the development of this innovative technology...

AUGUST 2014 | www.testmagazine.co.uk

PAGE 3

Assurance is the science of optimization. In today’s overly complex technology world, testing and QA functions must balance the art of perfection with the science of optimization. There exists a way: Tata Consultancy Services (TCS). With TCS’ independent enterprise testing arm, Assurance Services Unit (ASU), you can balance your testing needs and business goals with market-proven, world-class experience, expertise and guidance. Visit tcs.com/assurance and you’re certain to learn more. Or write to us at: global.assurance@tcs.com

IT Services Business Solutions Consulting Scan the code to know about TCS Assurance Services

CONTENTS

INSIDE THIS ISSUE

TEST AUTOMATION

26.

AUTOMATION – THE PATH TO TESTING EFFICIENCY

Automation – the path to testing efficiency

Automation is the bedrock of the modern software development quality operation. How does a major testing organisation deploy automation and what are the benefits and challenges? Vijay Balasubramaniam explains...

34 An IT team walks in to a bar... Elizabeth Winspear explains how a project with an independent brewing and retailing business changed from a manual world into automation...

CROWD TESTING

36.

Testing from Timbuktu

Hitesh Joshi discusses crowd sourcing in the testing industry, and asks, can a team of crowd-testers match the service levels of corporate professional testing service providers?...

TESTING FROM TIMBUKTU

38. DRIVING CHANGE: WHY SOFTWARE SAFETY MATTERS IN THE AUTOMOTIVE INDUSTRY

SOFTWARE DEVELOPMENT

Driving change: Why software safety matters in the automotive industry

42. FROM HM ARMED FORCES TO A TESTING TEAM

David Gil looks at how software is assured in markets such as the aviation industry to see what lessons can be applied to the automotive industry, arguing that as software and onboard systems increase in their complexity, the need for Independent Verification and Validation increases...

FOCUS ON: CAREER 42

From HM armed forces to a testing team

Hiring ex-Forces personnel is vital. Col (Retd) Stewart Sharman explains how their skills, gravitas and experience can boost your testing team...

LAST WORD 46 Who tested this?! Dave Whalen shares his “challenging” experience of using a website that wasn’t sufficiently tested...

AUGUST 2014 | www.testmagazine.co.uk

PAGE 5

www.neotys.com

LEADER

WHAT DOES THE RISE IN MOBILE DEVICES MEAN FOR TESTERS? Hello, and welcome to the August issue of TEST Magazine.

ccording to new figures recently published by the Office of National Statistics, over threequarters of adults (76%) accessed the Internet daily in 2014. This has more than doubled from the 35% recorded in 2006.

The Internet Access - Households and Individuals 2014 report illustrated that 84% of households have some kind of Internet connection, up from 9% in 1998 when this series began. Over half of all adults (55%) used the Internet to read or download the news, newspapers or magazines in 2014, compared with only 20% of adults in 2007. In 2014, 68% of adults accessed the Internet using a mobile device, with this number rising to 96% for those aged 16 to 24. With more individuals accessing the Internet using a mobile device, tablets are projected to overtake the sale of traditional personal computers by 2015. According to research analyst, Gartner, smartphone sales have already overtaken feature phones for the first time last year. The vast number of new mobile devices on the market has led to a huge increase in app downloads — an industry predicted to reach $267 billion by 2017. This news calls for mobile developers to ensure they constantly update their testing techniques, as standard

tests like load and performance tests are no longer good enough for mobile apps. Much more comprehensive testing is now required before products are launched to the market.

Do you want to write for TEST magazine? Please email sophie. odum@31media. co.uk

Vijay Balasubramaniam, Global Head of Testing Services at ITC Infotech, commented, “Long gone are the days of developing an app for one device, browser or carrier. With the advent of the ‘Internet of Things’ and so many products and services now communicating with mobile devices, apps must now be much more flexible if they are to meet demand.” The number of specialist tools and techniques required for mobile testing have increased in order to meet the challenge. However, this begs the question, as testers demand more out of the tools they use, does the industry face pressure to constantly better itself in order to meet this demand, and, if so, how can it improve?

Until next time

Sophie-Marie Odum Editor

© 2014 31 Media Limited. All rights reserved. TEST Magazine is edited, designed, and published by 31 Media Limited. No part of TEST Magazine may be reproduced, transmitted, stored electronically, distributed, or copied, in whole or part without the prior written consent of the publisher. A reprint service is available. Opinions expressed in this journal do not necessarily reflect those of the editor or TEST Magazine or its publisher, 31 Media Limited. ISSN 2040-01-60 T H I R T YO N E

AUGUST 2014 | www.testmagazine.co.uk

EDITOR Sophie-Marie Odum sophie.odum@31media.co.uk Tel: +44 (0)203 056 4599 TO ADVERTISE CONTACT: Sarah Walsh sarah.walsh@31media.co.uk Tel: +44(0)203 668 6945 PRODUCTION & DESIGN Tina Harris tina.harris@31media.co.uk

EDITORIAL & ADVERTISING ENQUIRIES 31 Media Ltd, 41-42 Daisy Business Park, 19-35 Sylvan Grove, London, SE15 1PD Tel: +44 (0) 870 863 6930 Email: info@31media.co.uk Web: www.testmagazine.co.uk PRINTED BY Pensord, Tram Road, Pontllanfraith, Blackwood, NP12 2YA

PAGE 7

NEWS

CIOS OUTSOURCE TO OVERCOME COMPLIANCE CHALLENGES 39% of CIOs, on average, choose to outsource application development and testing when new legislation or industry regulation requires compliance changes to be made to an organisation’s mainframe applications. According to an independent global research study undertaken by Vanson Bourne and commissioned by Micro Focus, the US leads the charge, outsourcing an average of 62% of testing and development projects. Germany outsources the least at 26%, while the UK sits at 40%. The poll surveyed 590 CIOs and IT directors from nine countries around the world. Explaining the compliance challenge, more than half of IT leaders (55%) say that it is highly likely or certain that the original knowledge of

their mainframe applications and supporting data structure is no longer in the organisation. Similarly, nearly three quarters (73%) confirm that their organisation’s documentation is incomplete. This lack of clear or upto-date records poses a problem in identifying and making compliance changes to the right applications. In fact, 44% of CIOs confirm they lack the capability to do application compliance change work in-house. Derek Britton, director of product marketing at Micro Focus, said, “On-going legislative changes have resulted in an array of new compliance measures such as ISO27002, Basel III, FACTA and SEPA. In order to support compliance requirements, organisations need to change and update their core business applications. The complexities

EU TESTING SOFTWARE TO CUT AIRPORT ENERGY COSTS A new software and sensor system, named CASCADE, promises to reduce EU airports’ carbon emissions and energy costs by 20%, the European Commission recently announced. Commission funding supported its development, and pilot testing is now taking place in major Rome and Milan airports, with the system expected to save at least 6,000 MWh, equal to 42,000 tons of CO2 and saving $1.1 million annually. Partners in Germany, Italy, Ireland, and Serbia are working on the new system, and the Airports Council International Europe – representing

more than 450 airports in Europe – has committed its support, so the system will enter wider use beginning in 2015. Nicolas Réhault, coordinator of the CASCADE project at the Fraunhofer Institute for Solar Energy Systems in Freiburg, Germany, said, “Innovative software can detect faults, for example fans operating when they are not required, simultaneous heating and cooling, control errors, and so on. It can then suggest corrective actions to the energy management and maintenance teams, like resetting controls or replacing faulty detectors.”

Sales

POLL RESULTS Last month we asked, "HOW DO YOU TRAIN YOUR TESTING TEAM?". Answer this month's poll at: www.testingmagazine.com

IN-HOUSE

35% 61% 4%

In-‐house

courses UUniversity NIVERSITY Outsourced training COURSES

OUTSOURCED TRAINING

of missing code documentation, constrained resource pools and data privacy risks mean companies are frequently turning to the outsource market for this work.” Whether they outsource or not, almost all of respondents (98%) believe that the ability to demonstrate the resilience of their current mainframe environment is important, with 91% believing this ability will only increase in the future. When outsourcing mainframe application development and testing to comply with new legislation, 60% of CIOs contractually pass the legal responsibility for data protection and privacy requirements to their outsourcing partner. A further 16% expressed a desire to do so.

NASA OVERSPENDS NASA is spending more money than it should to maintain one of its software verification and validation facilities, leaving less funding to evaluate the software, according to a report by the space agency’s inspector general. The site in question is NASA’s independent validation and verification (IV&V) facility in Fairmont, where testing and validation on software is performed before it’s loaded onto NASA spacecraft. The inspector general found that NASA could save up to $9.7 million between 2015 and 2018 if the IV&V program took steps to reduce operational and maintenance costs. Between October 2008 and September 2013, the report said that NASA spent about $36.3 million for costs associated with the building. “In our judgment, continuing this arrangement does not make fiscal sense for NASA, particularly when the agency has more projects needing IV&V services than the current budget can accommodate,” the report said. The report provided several options to cut such costs.

For the latest news, visit softwaretestingnews.co.uk and follow us @testmagazine

PAGE 8

AUGUST 2014 | www.testmagazine.co.uk

NEWS TFL TRIALS NEW SOFTWARE Transport for London (TfL) has confirmed that trials of detection software, to enhance bus driver awareness of pedestrians and cyclists, will start later this August as part of a continued drive to improve road safety in the capital. TfL is trialling two systems, ‘CycleEye’ from Fusion Processing Limited and ‘Cycle Safety Shield’ from Safety Shield Systems Limited. The technology directly alerts bus drivers when pedestrians and cyclists are moving close to their vehicles, helping to reduce collisions. Four buses will be fitted with brand new pedestrian and cyclist detection software as part of a six-week trial. CycleEye is advanced cyclist detection technology, which uses both radar and optical technology to detect cyclists in close proximity to vehicles, and the system audibly alerts the bus driver to their presence. Whereas Cycle Safety Shield is able to detect pedestrians, cyclists or motorcyclists in close proximity to vehicles, giving a visual warning and then an audible alert to the driver. One of TfL’s top priorities is to reduce the number of people killed or seriously injured on London’s roads by 40% by 2020.

EUROPEAN PROVIDERS STILL TESTING THE WATERS OF BIG DATA A new survey by IDC Health Insights shows that European healthcare providers are still uncertain about the benefits of investing in Big Data and analytics (BDA) technologies. The survey shows that 16% of non-hospital healthcare provider respondents plan to invest in BDA solutions in the next 24 months, compared with 6% of hospital respondents. European healthcare providers are currently focusing their investments on more familiar tools, such as business intelligence and analytics (BI/A). Greater familiarity with these solutions and use cases has led to wider adoption and stronger investment plans, but with healthcare providers increasingly required to provide more insights into their clinical outcomes and the appropriateness of their services, the more traditional BI tools are no longer able to support healthcare providers’ information insight needs. The survey also highlights that BDA is strategic to support the evolution of healthcare services delivery and the adoption of integrated care models — the objectives of all healthcare system reforms across Europe. Silvia Piai, EMEA research manager, IDC Health Insights, said, “European healthcare providers need to investigate current best practices and start building the business case for a BDA proof of concept pilot, by identifying areas where the implications of operating without access to timely, relevant, and actionable information have a bigger impact on the organisation’s performance. “In order to ensure the success of the pilot and to futureproof their BDA investments, healthcare providers should pay particular attention to aligning incentives and ensuring cooperation between the IT department and line-ofbusiness executives, merging BDA investments with the broader enterprise data management strategy, and leveraging the interdependencies with other 3rd Platform technologies.”

AUGUST 2014 | www.testmagazine.co.uk

Isabel Dedring, Deputy Mayor for Transport, said, “This is another great example of London leading the way by using the latest innovative technology to further improve safety for every road user. We’ve seen real improvements in reducing the number of accidents on our streets involving buses and if this trial proves successful we’ll look to roll it out further across London’s fleet.” The new trial is the latest in a long line of technological advances being trialled by TfL on London’s buses. Recently, innovative pollution reducing equipment was installed on the exhaust systems of over 1,000 buses operating on more than 50 routes across London. Combined with the roll out of new hybrid buses (including the New Routemaster) and the early introduction the ultra-low emission Euro VI engine to new buses, the retrofit programme is said to have led to significant improvements in emissions throughout London.

FIRST COMPANY TO EXCLUSIVELY RECRUIT EMPLOYEES WITH AUTISM Auticon, a software testing company headquartered in Berlin, is the first of its kind to exclusively recruit employees on the autism spectrum. The company was founded in November 2011, with a 500,000€ grant from the Social Venture Fund. Dirk Muller-Remus, who started out as a project manager in the commercial software development of Siemens and has four children, one of whom has been diagnosed with Asperger’s syndrome, founded the company. His experiences in the IT world, along with the personal responsibilities of raising a child with autism, gave him the idea to start a company that would train individuals on the autism spectrum for employment, while also educating clients regarding ways to maximise the unique talents of these individuals. Auticon seeks to offer individuals with autism and Aspergers syndrome an environment that is suitable to their needs, while educating and supporting clients on the strengths they can bring to the job with the right accommodations. Auticon also employs job coaches to help the individual communicate with supervisors and co-workers, and to guide the staff and management on suitable accommodations that would help the employee be successful. The goal is to help more talented individuals on the autism spectrum use their skills to contribute in meaningful, full-time employment. Skills offered by Auticon employees include analysis, checking and testing of highly complex systems specification, static code analysis of programs written in C, C++, C# and other languages, specification and modeling of complex work flows, migration of content-management systems, analysis and cleansing of large amounts of data by visual comparison, draft of technical manuals and documentation, and highquality data-recording and indexing.

PAGE 9

NEWS IS SUMMER THE BEST TIME TO RECRUIT? SHARE BUTTON MAY ALSO SHARE YOUR BROWSING HISTORY

land their dream job – the research showed that, on average, there are only 1.2 new job seekers going after every new vacancy in January.

1 in 18 of the world’s top 100,000 websites track users without their consent using a previously undetected cookie-like tracking mechanism embedded in "share" buttons. A new study by researchers at KU Leuven and Princeton University provides the first largescale investigation of the mechanism and is the first to confirm its use on actual websites.

With senior staff on holiday and potential candidates out of the country, it may appear at first glance that August is the worst time of year to try to hire new staff. But research from specialist IT recruiter, Randstad Technologies suggests the opposite.

The mechanism, called “canvas fingerprinting”, uses special scripts – the coded instructions that tell your browser how to render a website – to exploit the browser’s so-called "canvas", a browser functionality that can be used to draw images and render text.

An analysis of the ratio of job seekers to vacancies shows that, on average, there are 3.1 candidates for every new job in August – compared to an average of 2.3 candidates over the course of the year.

Mike Beresford, MD of Randstad Technologies, said, “If you want the best odds on scooping up high-flyers, August is the month to hire. These figures show the competition for the best talent is less intense. With the ratio of candidates to potential vacancies so advantageous to the hiring organisation, the summer is an ideal time to recruit."

Gunes Acar, the first author of the study, said, “This is an advanced tracking mechanism that misuses browser features to enable the circumvention of users’ tracking preferences. We hope that our results will lead to better defenses, increase accountability for companies deploying sticky tracking techniques and an invigorated and informed public and regulatory debate on increasingly resilient tracking techniques.”

Randstad Technologies also found that whereas August was the best month for employers to stand out and fill vacancies, January is the best month for IT and digital professionals to try to

From an employer’s point of view. Winter (December, January, and February) is by far the worst time of year to recruit with just 1.3 new job seekers for every new role.

STILL USING XP? NOT TO WORRY, SECURITY VENDORS WILL CONTINUE TO SUPPORT YOU

When a user visits a website with canvas fingerprinting software, a first script tells the user’s browser to print an invisible string of text on the browser’s canvas. Another script then instructs the browser to read back data about the pixels in the (invisibly) rendered image.

In an independent analysis of over 20 antivirus protection offerings by AV-Test, it found that many security vendors are still providing protected from online threats, even though Microsoft has ended its support.

This data contains important information about the user’s browser type, graphics card, system fonts and even display properties. Because this grouping of data is highly likely to be unique for each user, it can be reliably associated to individual users, like a fingerprint.

For the protection portion, software is loaded up onto a clean machine and AV-Test uses malware made of zero-day malware attacks, classic known attacks and dangerous websites are visited.

Once a website has determined a device’s fingerprint, it can easily recognise the user on subsequent site visits, much in the same way cookies do. But while unwanted cookies can be flagged or blocked to enhance a user’s online privacy, there is no available solution for doing so with fingerprints. In this study, the researchers used automated ‘crawlers’ to scan the world’s top 100,000 websites for canvas fingerprinting scripts. They found canvas fingerprinting scripts on 5,542 of the Internet’s top 100,000 websites, a prevalence of 5.5%. This is the first time canvas fingerprinting has been observed on real websites and traced back to specific provider domains. Analyses of the real-world scripts reveal that fingerprinters are going beyond the techniques known by the academic research community.

PAGE 10

AV-Test scores software on three criteria: protection, performance, and usability. Each section can score a maximum of six points and a perfect test would result in a score of 18.

Even though it’s been more than three months since support for XP has ended, 13 of the tested software were able to block everything thrown at them and scored a six. This includes Kaspersky Internet Security 2014, McAfee Internet Security 2014, Norton Internet Security 2014 and F-Secure Internet Security 2014. Not all fared so well. Microsoft’s free Security Essentials 4.5 only scored half a point and was the lowest scoring of the batch, according to the report. However, Microsoft did announce at the beginning of the year that it would no longer support the antivirus software once XP support ended in April. Performance was also tested, clean systems with the antivirus software installed completed 13 real-world computing tasks, including downloading files from the Internet, installing programs and running Microsoft Office, to see if the security software affected the speed and performance of the machines. The final test measured usability by rating the distractions caused by warning messages appearing due to false positives and how often safe applications were flagged as malicious. All tested software performed well for this round, with the lowest score being awarded to Comodo Internet Security Premium 7.0 with a score of four. Overall, AV-Test’s latest round of analysis shows that if you still want to keep your XP system, security vendors will continue to support you without the help of Microsoft.

AUGUST 2014 | www.testmagazine.co.uk

THE EUROPEAN SOFTWARE TESTING AWARDS

2 0 1 4

RECOGNISING AND CELEBRATING TECHNICAL EXCELLENCE

ENTRY DEADLINE EXTENDED! DEADLINE EXTENDED UNTIL 1ST SEPTEMBER 2014, 5PM PLEASE ENTER VIA www.softwaretestingawards.com

The European Software Testing Awards is an independent awards programme designed to celebrate and promote excellence, best practice and innovation in the software testing and QA community

Why Enter? • An opportunity to be acknowledged and rewarded • W ith such recognition comes self belief, the breeding of success, and the attraction of fresh and bright talent • H igh levels of exposure and acclaim for each individual, team, and organisation, as well as the profession as a whole • Impress partners, clients and investors • Network, interact and learn from Europe’s software testing elite.

Headline Sponsor

Sponsors

MAGAZINE

Headline Sponsor

THE EUROPEAN SOFTWARE TESTER

PAGE 16

AUGUST 2014 | www.testmagazine.co.uk

INTERVIEW

MARTIJN DE VRIEZE, SENIOR TEST CONSULTANT

TURN OFF SPYING, TURN ON FREEDOM Sophie-Marie Odum speaks to Martijn de Vrieze, a senior test consultant, about the role software testers play in the Reset the Net campaign…

new fight against mass Internet surveillance recently hit the headlines. The campaign, called Reset the Net, literally reset the net on June 5th 2014, the anniversary of the date the first Edward Snowden story broke, detailing the Government’s PRISM program. As part of the campaign, which aims to turn off government spying, organisers called on developers to add at least one NSA resistant feature to mobile apps, and on websites to add security features. Sophie-Marie

PAGE 18

Odum speaks to Martijn de Vrieze to learn more about the role software testers play here…

Sophie-Marie Odum: As a tester, please explain why you are supporting Reset The Net? Martijn de Vrieze: It is not so much as a tester as it is as an IT professional and a citizen. I believe that governments and other agencies do not have the basic right to sniff in my affairs, whether they are online or offline. In the current world, it is incredibly

I BELIEVE THAT GOVERNMENTS AND OTHER AGENCIES DO NOT HAVE THE BASIC RIGHT TO SNIFF IN MY AFFAIRS, WHETHER THEY ARE ONLINE OR OFFLINE

AUGUST 2014 | www.testmagazine.co.uk

INTERVIEW

difficult to do anything without leaving a substantial digital footprint. Many, many people do not realise the risks they run when using everyday services online, whether it is Facebook or Dropbox. People often do not realize how easily accessible their data is. When thinking about privacy and data security, as a tester, I am more concerned with how data is transferred and stored, and whether different passwords and accounts are needed for read versus write or update actions in databases, etc. Moving more into the realms of Reset the Net movement, I am thinking more about encryption and specifically exploits of these methods. Take, for example, backdoors into SSL; this is a security risk. However, since SSL was thought of as safe and nobody knew about these backdoors since NSA and other agencies kept these backdoors secret, how would you test for them? Can we make sure that developers, product managers/owners and testers (also the traditional functional testers) are more aware of the risks of bad encryption?

SMO: What will be encrypted? How will encryption happen? And what role will testers have here? MV: Theoretically anything and everything can, and probably should, be encrypted. However, thinking logically and practicall,y not everything will be encrypted since it takes a lot more processing power and, of course, basic effort to do it properly. When not done properly, encryption is just a means to annoy people rather than obfuscate your data properly. When talking about encrypting data, such as cloud storage, I trust client-side encryption most. This means the password and the client only knows encryption keys. Resulting in a server (and thus the admins of the servers and with them snooping parties like NSA) have no knowledge nor easy access to the data. Encryption of data-traffic should be done with proper SSL tokens, ensuring the identity of the party you are communicating with. For testers this will mean paying a lot of attention to the details of the SSL signatures, testing for possible cross-site access, validating the validity of the duration of an SSL certificate etc. Testers will need to be very much aware of the possibilities of certificates and also of the dangers certificates can pose. For example, a certificate on a wildcard subdomain opens up a lot of possible risks if the subdomains are hijacked. So testers will have, in my view, the same role as they have now. However, they will need to learn some new stuff again, around encryption, certificates, handshakes etc.

SMO: Is your company getting involved? If so, how and why? MV: Our company is involved, but mostly in an advisory role. Besides the “regular” functional testing, we also provide security testing to our customers. During our security tests, we pay a lot of attention to the different business risks involved with security and privacy, both on a technical level, as well as on the human level. The human side of the safety of data should not be

AUGUST 2014 | www.testmagazine.co.uk

IF A COMPANY PLANS TO ACTIVELY WORK ON PREVENTING SURVEILLANCE, THE WORK OF A TESTER MIGHT BE IMPACTED QUITE HEAVILY. TESTING IN AN ENCRYPTED ENVIRONMENT CAN BE A LOT MORE CHALLENGING

forgotten. The fact that one person managed to get so much information as Snowden did also shows a huge security gap in the NSA ways of working of course. It quite often is very easy to gain access to data without involving any technology, but by working the humans controlling the data.

SMO: In your opinion, how does surveillance activities impact software testing professionals? MV: I believe it opened many peoples’ eyes, including testers. It might impact them in their daily lives, however I am not yet convinced it will directly impact their work or the way they work. That all depends on what their employer plans to do with (or against) surveillance activities. If a company plans to actively work on preventing surveillance, the work of a tester might be impacted quite heavily. Testing in an encrypted environment can be a lot more challenging. User data will not just be obfuscated, but also encrypted, resulting in no easy re-use of production data in their work. Validating data stored in the databases might become more difficult as well, thus the testing of business logic of for example business intelligence systems will increase in complexity. Overall however, I believe that when a company actively starts acting on, or against, surveillance and relative ease of access to their data, this will add a new technological level to master for both testers and developers alike.

SMO: It’s been said that “tech corporations did not join the Reset the Net campaign out of magnanimity, but because Snowden’s whistleblowing negatively impacts their interests”, please explain your thoughts MV: Obviously not all companies joined Reset the Net because they truly believe it is the best thing to do. For quite a few of them it is a headache to add these extra layers of security. Dropbox, for example, actively uses the fact that their data is not stored fully protected. When you upload a large file to your Dropbox storage, they check that file to see if it is already stored somewhere in their cloud, thus limiting the bandwidth and storage space they need. When all their clients are using proper client-side encryption, this will be near impossible for them to do. This will result in them needing far more storage space than they currently have, increasing their running costs and therefore decreasing their profits. In other words, for some companies, it is expensive to make this move and they now see they have to hop on the train because otherwise they will loose customers fast. This is of course not true for all companies joining and supporting the movement, and in the end, even the ones that hopped on because they “have to” will come out better in the end! They actually provide better service to their users/customers, increasing satisfaction and thus quite likely the loyalty of their users.

PAGE 19

VIEWPOINT SOWMYA RAMESH, CONSULTANT, SOPRA

BREAKING BARRIERS, TESTING STYLE Given the Government’s new strategy for digital delivery and its focus on inclusive design, Sopra consultant, Sowmya Ramesh provides tips to help learn more about web accessibility, together with a top 10 checklist of how to run a condensed version of Web Content Accessabillity Guidelines (WCAG) manually…

am a new mother and would like to share how this experience has got me thinking about testing (and by that I don’t mean the baby ‘testing’ my patience!) Any guesses? It is not about my curious experiments with baby food, nothing to do with the heroic tales of multitasking either. It is actually taking the baby around in a pram that has made me look at a form of non-functional testing in a different light. When I am out with a buggy, there is a constant hunt for ramps and lifts in public places. And where I don’t find any, I get frustrated to say the least. It has made me wonder what life is like for someone who faces such situations for a lifetime? What about people who face barriers all the time? Are those barriers limited to the physical world or is it rampant in the virtual world? What about the disabled and their right to access information? In this day and age, can technology be a true leveller? The sad truth is that the dream is far from being achieved. When it comes to web accessibility, awareness seems

limited and consideration inadequate – in business and technical domains alike. So, why should any business take this seriously? As many of you might know there is tough legislation around web accessibility, which makes it mandatory for websites to cater to those with different abilities. With millions of endusers from this group in the consumer market, there are crucial commercial opportunities to increase the reach and hence the revenue for any business. There are many hidden advantages as well in ensuring web accessibility like improved search engine ranking, etc.

WHAT IS MISSING IN OUR DELIVERY CYCLE? On the technical side, how about a bit of retrospection into what is missing in our delivery cycle – how do projects treat web accessibility? How about the situation in your work place? Has accessibility testing been set aside for a specialist to look at? Or worse, has it been postponed to be considered in a future release (in other words, never to be looked at unless it becomes a burning issue)?

WHAT ABOUT PEOPLE WHO FACE BARRIERS ALL THE TIME? ARE THOSE BARRIERS LIMITED TO THE PHYSICAL WORLD OR IS IT RAMPANT IN THE VIRTUAL WORLD?

PAGE 20

AUGUST 2014 | www.testmagazine.co.uk

VIEWPOINT

From my personal experience, it seems like the easiest decision is to set it aside conveniently as a non-functional requirement. But is it not really a key quality issue related to functionality in that a website might not be available to a part of the end user group? Are we, as testers, doing our bit in enabling the transformation of the web resources to be more inclusive – by making a strong case for it to be taken as a priority; by influencing the decisions about quality assurance activities for accessibility at test strategy level; and by being bothered to reserve some time in our estimates for accessibility checks? From experiencing temporary conditions like injuries to having ageing grandparents, we have all witnessed the effects of disability, either directly or indirectly. However, does the empathy reflect in our everyday work? It is worth asking that question, even more so, if you are working on a public-facing website.

THE REQUIRED KNOWLEDGE In my opinion, it makes sense for this to be every tester’s remit – irrespective of whether you are a specialist in accessibility testing. This brings us to the question of how easy is it for you to pick up the required knowledge, if you are a novice. Here are a few tips from my personal experience of learning more on this topic. 1) Internationally accepted guidelines – Web Content Accessabillity Guidelines (WCAG) are a logical place to start. The associated website www.w3.org is a highly recommended source of authentic information. 2) Familiarisation with assistive technologies – From using keyboard short cuts to magnifiers, it is important to get comfortable browsing in the accessibility mode (including default options provided by operating systems or any specific applications). Getting accustomed to a screen reader in particular involves a learning curve. 3) Getting acquainted with the tools – There are many useful tools available, which can make life easier. For example, a colour contrast analyser can assess precisely whether the given content is suitable for those with colour blindness. 4) Online forums and blogs – this is perhaps too obvious as a source, but there are some specific online forums that hold invaluable information. When should you get involved? At any point in the project, but the sooner the better. From reviewing the web design prototypes or wire frames, all the way through to auditing existing websites, a tester can be crucial in uncovering accessibility issues at any stage. However, early involvement can also ensure that there is significantly less risk of design changes late in a project life cycle. Are there other reasons that would deter a tester from delving deep in this aspect? Here are some arguments I have come across as to why testers hesitate to take this up and my response to them: • “Usage of too many tools, I prefer a manual, exploratory approach.” You would be surprised to find that the most effective of the accessibility tests are manually executed.

AUGUST 2014 | www.testmagazine.co.uk

•

“I like tools and this is not an exciting area of work.”

Again you would be surprised about the number of tools available and the innovation that you can bring in testing. Although human intervention might not be completely ruled out, there is a range of software available to make that effort minimal. • “WCAG checklists are massive and impractical to complete verifying them in a short span of time.” It is possible to achieve the same coverage through a concise approach. • “I can’t emulate the experience of a person with an actual disability.” While it is true that the ideal way to test a website for accessibility cannot be achieved without users with disability, it is not impossible to anticipate and avoid issues they are most likely to face. Removing the most rudimentary and obvious barriers can make a huge difference.

A USEFUL CHECKLIST In the spirit of simplifying these tests, how about a quick top 10 checklist you could run manually, a condensed version of WCAG if you like, to ensure some important aspects of accessibility. 1) Reading order – Disable CSS on the browser to check if reading order is logical and if there is appropriate usage of headings. 2) Focus order – Keyboard only access on all key parts of the web page, ensuring sequential navigation to the website which is logical in order. 3) Screen reader friendly – Access through a screen reader to check if all visible content is meaningfully conveyed. 4) Meaningful URL hints – Link descriptions are appropriate. 5) Alt text usage – Important images have appropriate alt text specified. 6) Colour cautious – colour contrast is carefully chosen in terms of fonts and colour alone is not used to convey any crucial information. 7) Timing out – Any time limited action gives the user enough time. 8) Instruction clarity – Forms are clearly labelled and have clear instructions. 9) Magnifier friendly – Content can be resized easily. 10) Easy navigation – Skipping sections is easily possible. I have personally found that going through a website with a screen reader and keyboard can be the quickest way to unearth the most significant of the issues. There is more to accessibility testing should you have a need and time to investigate further. For example, setting up user groups, establishing end user profile etc. I hope the information above will encourage you to be an ambassador for web accessibility. Why wait for a specialist, when you can be one. Go on and be the torchbearer for a cause that ensures an equitable resource, which is for everyone.

PAGE 21

DAY IN THE LIFE OF… BEN KIRBY, SOFTWARE DEVELOPMENT ENGINEER IN TEST

A DAY IN THE LIFE OF...A SOFTWARE DEVELOPMENT ENGINEER IN TEST Editorial advisory board member, and Head of QA and Test at William Hill, Shane Kelly, interviews Ben Kirby, an SDET (Software Development Engineer in Test) for an online gaming company, about what his job involves and the challenges he faces on a daily basis…

think that since I read the book How Google Test Software, by James Whittaker et al, I have achieved a better understanding of the testing disciplines and how we define the engineering roles that exist in our community. Many test practices have defined the roles of test engineer and development engineer in test to both promote the technical aspect of testing, and also to ensure the development and business functions have a better understanding of what they can expect from a tester. One of the more common acronyms for the developer in a test role is that of SDET (Software Development Engineer in Test), often pronounced as “ess-det”. It is now more widely used on job sites and in user group discussions where people feel they have an understanding of what

PAGE 22

an SDET does, and what are their responsibilities within a test team. To try and help explain this role and what they do from day-to-day, I chatted with Ben Kirby, an SDET working for an online gaming company, on what his job involves and the challenges he faces on a daily basis. I met Ben at the National Software Testing Conference and asked him a few questions on what it is to be an SDET in today’s software development environment.

Shane Kelly: Ben, what gets you out of bed in the morning and drives you for your job as an SDET? Ben Kirby: I like the challenge of a constantly changing environment where each day seems to bring up something different. I get to work with new technologies and work on ways on how we can best use them to meet the needs of the company.

AUGUST 2014 | www.testmagazine.co.uk

DAY IN THE LIFE OF…

SK: When you arrive in the office what is the first task that you set into? BK: I try to work in an agile manner, even outside of projects, where I look at the tasks ahead and see how best to handle them. Like anyone, I check through emails to see if there’s anything else I need to re-prioritise and, contrary to what others may think, I don’t just jump into writing code. It may be that we’ve decided to refactor some of the framework or write test cases against the latest code drop. I even get to work on manual testing when required. It can be quite varied, but mainly it’s about building the test framework for the team I work in.

SK: What is your test framework built on, and how is it maintained? BK: The framework was initially built in Java, using Selenium web driver, but more recently we have started to expand this and use http to make direct calls for data generation etc. We also use third-party tools for tasks like image recognition or load and performance testing. We’ve worked hard to ensure they complement each other well and are used for the jobs they were built for.

SK: One of the biggest challenges of using opensourced software is set-up and general support. When you need help on either installation setup or problems within your code, how do you go about getting the support required? BK: Java has a huge community of people that are always willing to help. Most of the time, an issue that I face will most likely have been faced and solved by someone else already. But, if not, then people in the Java community are very willing to help you find that solution.

SK: How is the framework managed; who decides on what changes can be made; and how are they controlled? BK: The full team of SDETs within the company work together to decide on what’s included in the framework. It is separated out into five core modules with people responsible for each area. But, as with any application, we try to follow good coding practices on when we need to change something or when it needs to be deprecated. I try to be aware that with a large framework, if I change something it may have an effect on someone else’s test.

SK: Being an SDET is not just about automation and writing Java, there are many other NFRs (Nonfunctional Requirements) that need your skills to test out. What are some of these NFRs that you’re involved in on a regular basis? BK: I have worked on load testing the live site, which was

AUGUST 2014 | www.testmagazine.co.uk

a great experience. We utilised a tool to generate site navigation and transactional volumes above the levels of the UK stock exchange. The tool initially wasn’t able to handle these volumes, but, working close with the vendor, we discovered the root cause of the problem and they provided a solution very quickly. We also use our framework to generate large data loads and provide the complex data required for certain tests.

SK: A key to the success of a good framework is that not only the SDETs use it, but also the STEs (Software Test Engineers) have an understanding of its main classes and methods, and can create their own tests utilising this framework. How have you helped the STE in achieving this goal? BK: I work very close with our STEs and I’m lucky that the full team collocates in the same area. We review the test cases together and I explain how I am asserting that the test has past, gaining feedback on the assertions validity. I also help provide training to STEs on the framework and Object Orientated Programming in general, of course focusing mainly on Java. I’ve explained to them how we use Jenkins to execute the test packs and that we aim for Green Dot Addiction.

SK: What is “Green Dot Addiction”? BK: *Laughs*, I forget others don’t use this term all the time. Green Dot Addiction is the term we use to highlight that the automation scripts should always execute, without failing, in a test environment. Jenkins uses green dots to highlight that the scripts have passed correctly and we then try to ensure that the whole team signs up to keeping these ‘dots’ green.

SK: Another important relationship is that of the SDET and the product owner, who represents the business within the product development team. How do you explain more technical concepts to the business and how do you ensure that what you are building is as they expect? BK: Currently I actually work in a waterfall type project (‘wagile’ as some people call it). But I still have good interaction with our business, and I will run demos on the work I’ve done so that they can actually see the test coverage and get a better understanding of the automation approach.

SK: And finally Ben, what gets you go home in the evening? What do you do to take your mind away from being a busy SDET? BK: I like socialising outside of work and play a bit of football, but I actually do also like doing programming and honing my skills as an SDET.

PAGE 23

COVER STORY DANIEL JABRY, CO-FOUNDER, CrowdEmotion

BRINGING EMOTIONS TO LIFE Following recent news that the BBC is testing a new facial coding technology, called CrowdEmotion, which investigates how viewers react and behave towards its TV shows, Sophie-Marie Odum speaks to CrowdEmotion’s co-founder, Daniel Jabry, about the development of this innovative technology…

umans are often categorised by physical traits and past behaviours to determine their future actions, yet there is still a massive gap in predicting human behavior,” said Daniel Jabry, co-founder of CrowdEmotion, as he explains the idea behind CrowdEmotion, which is said to be the world’s first cloud-based, facial coding technology to record facial expressions accurately, simply using a webcam.

“The emotional artificial intelligence (AI) technology captures, understands and links emotional signals to behaviours.” Daniel continued, “Emotions are an important contributor, but people cannot articulate them well. By blending machine-learning technology with 20 years of neuropsychology, CrowdEmotion captures and patterns human emotional signals to behaviour. This way, humans can continue acting the way they do, but the technology is now able to observe emotional signals using the devices we already have. “The software then takes that messy emotional data and cleans it up to enable comparisons with previous behaviour and ultimately enable better decision making.”

WILL THE TV SOON WATCH YOU? The BBC is testing this new technology developed by the British digital start-up to investigate how viewers react and behave towards its TV shows. The pilot study with BBC Worldwide has started with 200 participants in the UK, measuring their happiness, surprise, anger, fear, disgust and sadness. David Boyle, Executive Vice President of BBC Worldwide Insight, recently said that CrowdEmotion’s ability to capture, record and quantify the BBC audience’s emotional attachment and engagement to its TV shows places BBC Worldwide at the forefront of global audience research and ultimately determines what the BBC’s fans love to watch. Further explaining the relationship with the BBC, Daniel said, “CrowdEmotion gives businesses a cost-effective way to integrate emotional analytics into existing processes. There are multiple applications for emotional data, but regardless of context, it immediately answers two important questions for the BBC: “1. Creation: How do audiences engage with my content? The raw data can be processed to offer clients a scene-by-scene breakdown of emotional response to each stimulus, such as a video/advertisement, etc. By knowing how people engage with the content, the client can reverse-engineer products that are tailored to creating particular emotional states which are conducive to the firm’s aim, i.e. certain moods that are good for purchasing, analysing intent of people based on their emotional categorisation. “2. Planning: Which audiences are aligned with my content? For example, traditionally demographic data has been heavily relied on, however we have found that emotional data has a significantly higher correlation with audience engagement. This means that targeting efforts for media buying can be significantly bolstered.” “Any company with repeat customer experiences could benefit,” continued Daniel. “Retailers can capture the emotion journey to a path to purchase; video gaming can understand areas of boredom; and Border Control could potentially understand drug smugglers. The advertisment industry is particularly keen – we see this being a new metric to better understand the emotional driver behind engagement.”

PAGE 24

AUGUST 2014 | www.testmagazine.co.uk

COVER STORY DEVELOPMENT AND TESTING CrowdEmotion began a year ago and most of its projects started less than six months ago, “it’s been wild!” said Daniel. The software testing team is currently made up of three developers who also write/run the tests, using an agile approach as this allows the company to continually adapt to new opportunities and customer ideas. Emotional analysis is an extremely new field and the CrowdEmotion team is still exploring all the possibilities available. Discussing the development and testing of CrowdEmotion software, Daniel explained, “I’ve seen massive changes in the way that software is developed over the last 30 years – namely the wide adoption of test-driven development (TDD). These changes have allowed tiny companies such as ours to develop disruptive software on a shoestring budget whilst retaining reliability.

“I think our work is at the bleeding edge of a few software and computer vision areas, but our testing efforts are fairly standard. There’s some really cool testing technology out there that we could adopt in order to take us closer to the edge on that front, this desire will play a large role in our criteria for the next dev hires. “A new use case for emotion recognition technology is proposed in almost every conversation we have. We realised quite early on that we needed to build a highly flexible and ‘horizontal’ system with specific applications layered on top in order to serve our customers. “The layers we currently have are: A core recognition library; a publicly-accessible RESTful API to wrap the recognition library; and several API clients for visualising and reporting results. Testing this system is an immense challenge for us and we consider the problem to be composed of two main parts: Testing the layers separately and in various combinations. We have automated tests for the former, but need to do more to address the latter. “More specifically, the various projects we have created so far, and the technology we have used, are: A GPUoptimised C++ based set of libraries and tools for analysing videos; A realtime visualisation demo written in JavaFX/Spring; A RESTful API written in Spring for allowing customers to integrate our emotion engine into their systems; A server-based video analysis workflow engine for processing customer videos (also written in Spring); A web app written in PHP for reporting emotional results. “We run a continuous integration environment to manage multiple daily changes, making use of Jenkins and Maven for automated building, testing and deployment of all the projects mentioned. We make use of Spring profiles to manage test cycles – using a ‘dev’ profile for unit tests, followed by deployment to a 'staging' environment for further manual tests before manual deployment to production. We would like to automate more of the manual tests we run in the staging environment – most of the work involves writing more API client tests. “We have tried to keep the integration times low; from the git-push to the delivery alert in production. The time required can be less than five minutes.”

THE FUTURE As a company specialising in emerging technology, Daniel believes that the future of software testing will have a challenge in testing distributed systems, but cloud ecosystems could be the answer. He said, “Testing distributed systems is a huge challenge and I imagine that cloud ecosystems will take on a growing role in that space by providing some really smart testing services. “Combining the current generation of testing tools and best practices with DevOps could help close the loop that currently starts with user needs and ends in satisfaction in an automated way.”

I THINK OUR WORK “We have the luxury of being able to ask all the IS AT THE BLEEDING members of our company to test the software EDGE OF A FEW SOFTWARE we produce while we’re still very small,” added AND COMPUTER VISION Daniel. “Along with the aim of adopting more AREAS, BUT OUR TESTING innovative software testing methods, we’d like to retain the inclusive testing culture as we EFFORTS ARE FAIRLY STANDARD. grow. THERE’S SOME REALLY COOL “We’d love to hear some suggestions from TESTING TECHNOLOGY OUT readers of TEST Magazine about some of the THERE THAT WE COULD ADOPT technology they currently use and think would IN ORDER TO TAKE US work for us in achieving either of those aims.” CLOSER TO THE EDGE ON THAT FRONT

AUGUST 2014 | www.testmagazine.co.uk

PAGE 25

TEST AUTOMATION VIJAY BALASUBRAMANIAM, GLOBAL HEAD OF TESTING PRACTICE, ITC INFOTECH

AUTOMATION – THE PATH TO TESTING EFFICIENCY Automation is the bedrock of the modern software development quality operation. How does a major testing organisation deploy automation and what are the benefits and challenges? Vijay Balasubramaniam, global head of testing practice at ITC Infotech, explains…

n the Agile software development world, automation has evolved with astonishing speed. From the days when design, scripting, testing and implementation were distinct separate processes, we have moved on to an era of “dev/ops” where development takes place sideby-side with testing, in a contiguous process and with fast implementation cycles. In this environment, any changes to an application will require extensive testing; not only of functionality, but also of the impact the change brings to the software as a whole. Automation is crucial to making this possible.

A high degree of automation is now essential as an efficiency strategy to reduce the timeline of testing; provide a gate to check on whether the drop is ready to be manually tested; and to prove that the software has passed the basic build verification (BVD) tests and basic functionality tests. While the majority of functionality testing can be automated, there will always be some elements that require manual intervention, and automation is usually only carried out for those parts of the application which are the standard functional requirements for the application across all releases.

UNDER AUTOMATION, THERE ARE OPPORTUNITIES TO PERFORM NIGHTLY RUNS WHERE HUNDREDS OF THOUSANDS OF TEST CASES CAN BE EXECUTED AFTER “LIGHTS OUT”. THE NEXT MORNING WHEN THE TESTERS COME IN TO WORK THEY CAN UNDERTAKE THE ANALYSIS OF THE DEFECTS THAT HAVE BEEN COLLECTED

PAGE 26

AUGUST 2014 | www.testmagazine.co.uk

DIGITAL THE EUROPEAN SOFTWARE TESTER INNOVATION FOR SOFTWARE QUALITY

The Whole Story Print Digital Online

FOR DIGITAL ISSUES, DIRECTORY, COVER STORIES, POLLS AND MORE, VISIT testingmagazine.com

Published by T H I R T YO N E

www.31media.co.uk

Telephone: +44 (0) 870 863 6930 Email: info@31media.co.uk Website: www.31media.co.uk

TEST AUTOMATION

that would have otherwise had to be deployed. Applications are now built in a way which ensures that functionality is added in every release, a In test automation there is a law of diminishing principle of the Agile methodology. Some returns. The possibility of catching a defect WHEN AN parts of the functionality will remain pretty with an automation script is highest in the first much unchanged throughout all the APPLICATION NEEDS cycle. Subsequent cycles yield no defects iterations and updates of the software TO BE TESTED OVER because the limitations of automation from its first to its latest release. These A NUMBER OF RELEASE dictate that the script can only do exactly parts are the most suitable to have what it has been programmed to do. It CYCLES AND THERE ARE their testing automated. Obviously is very difficult to predict exactly what A NUMBER OF TEST CASES there will be also be some elements effect changes in the application will of the functional requirements THAT NEED TO BE TESTED IN have on the way it functions. Automation which do not lend themselves well EACH CYCLE, THERE COMES is extremely useful for things like build to automation because they do not A POINT QUITE EARLY ON verification and basic functionality, but need to be tested in every release. beyond that where there is an element WHERE AUTOMATION of complexity a more human approach is Manual testing also takes place STARTS TO PAY FOR required. in cases where the time and labour ITSELF needed to automate is not worth the The beauty of automation however is its ability effort. It is also advisable to rely on a manual to execute a large number of test cases quickly. approach when the technical feasibility dictates But the true efficacy of testing is in the number of defects that it may not be practical to automate because of found and, for that, human intelligence is also required the sheer complexity of the automation necessary. This because many of the glitches in software are caused by is where human intervention comes in. Here, we have to defects of omission, which automation won’t capture rely on the skills of the testers to decide what can and because it blindly follows the script put in place. what should be automated. Often specialist manual domain skills and experience is a necessity to get this The main challenge for organisations increasing right. their automation is in understanding what level of automation is sensible. The process of building the Clearly, automation adds tremendous value through application itself is very complex so the job of working the efficiencies gained. Under automation, there are out which is the stable part of the software in every opportunities to perform nightly runs where hundreds of release is in itself quite complex. thousands of test cases can be executed after “lights out”. The next morning when the testers come in to work they can undertake the analysis of the defects that have been collected. A typical tester can work an eight-hour shift, but automated testing can keep going around the clock. Automation does of course have costs attached, such as license fees and tool costs, as well as the investment needed for building and maintaining the automation script. However, when an application needs to be tested over a number of release cycles and there are a number of test cases that need to be tested in each cycle, there comes a point quite early on where automation starts to pay for itself. When there are maybe 15, 20 or 30 cycles, or perhaps daily releases, the costs incurred by automation are offset very quickly by the manual effort PAGE 28

You have to avoid automating something that you may end up scrapping altogether. It has its own costs and requires some effort to put it in place, and until we clearly understand what is required of the application and what each release must deliver and then understand what part of the functionality is common across all releases, automation could lead to a lot of wasted effort from focussing on the wrong things. The challenge is in knowing where it can be used most effectively. The biggest challenge however will remain the decision about where, when and how you automate. When the application itself is constantly evolving it is still very difficult to pinpoint exactly where automation makes the most sense.

AUGUST 2014 | www.testmagazine.co.uk

Don’t miss TestExpo this October! 20% discount for TEST magazine readers! Just quote ‘TMEXPO20’ in the comments ﬁeld on our online registration form. Register today: testexpo.co.uk

Discount valid on full price bookings made between 15th August and 30th September 2014 only. Not to be used in conjunction with other offers.

TEST AUTOMATION

BIRTHS, DEATHS AND MARRIAGES… ALL AUTOMATED TEST Magazine looks at how the Office of National Statistics (ONS) overcame challenges to create custom process configurations and export scripts…

NS statistics like those on aspects of the economy and life events such as births, deaths and marriages have been collected for many years, and ONS software requires ongoing management and development to ensure it remains fit-for-purpose. The responsible department uses IBM tools to manage the software lifecycle, and upgraded to the latest IBM Rational suite for collaborative lifecycle management (CLM) in 2012. The ONS was then faced with two challenges. Firstly, with 15 or more concurrent projects, it needed to find a way to configure the Rational tools quickly, in particular the agile application lifecycle management solution, Rational Team Concert (RTC), to suit the workflow style of the ONS. The large number of projects gave rise to the risk that each development team would implement its own customised workflows in RTC, which would have required ONS’s Rational support to maintain dozens of different process configurations. The second challenge involved migrating projects from the old Rational ClearQuest software to the new Rational tools. While a migration utility exists, the ONS was struggling to get it to work. Most organisations are able to start the software lifecycle management process afresh with the latest tools when they move on to new projects. The ONS, however, needed to maintain and update long-lived applications. It therefore had to find an effective way to export data and workflows from the old Rational software.

PROVIDING SOLUTIONS The ONS was introduced to Automation Consultants in 2012, who helped the government body implement a Rational Functional Tester test automation framework, and helped them re-write HP LoadRunner scripts for IBM’s Rational Performance Tester. The engineers were then faced with the Rational ClearQuest migration challenge. Company director, Francis Miers, said, “Migrating development data from Rational ClearQuest to RTC was just as challenging as implementing appropriate workflows. “IBM has only one consultant in the UK who can do this, and his time was split between many different IBM customers. We also have these skills, and thanks to our in-house development capability we were able to write scripts to edit the export records automatically so they could be easily uploaded to RTC.”

LOOKING FORWARD... ONS head of test, Andy King, explained the difference the fine-tuning made to ONS software development. He

ONS SOFTWARE REQUIRES ONGOING MANAGEMENT AND DEVELOPMENT TO ENSURE IT REMAINS FIT-FOR-PURPOSE

said, “The improvement was immediately obvious. Instead of building a process configuration for every application, all we had to do was share the master process before starting a new project. This saved us considerable time and resource as we migrated with speed and efficiency.

“We’re using the savings to up-skill our development workforce – we’ve already invested in additional Rational training “I now share their [Automation Consultants] belief that automating the software lifecycle is the key to quicker and more cost effective IT project delivery – you could say I’m a convert! Their consultants’ technical knowledge and understanding of the Rational toolset, combined with the excellent training they provide, has meant we can forge ahead with application development.” Miers continued, “Ultimately we want the ONS to automate as much of the software lifecycle as possible to ensure optimal return on investment and increase development and deployment speed. We’ll be making recommendations for continuous improvement based on these principles.”

About the Office for National Statistics The Office for National Statistics (ONS) is the UK’s largest independent producer of official statistics and is the recognised national statistical institute for the UK. It is responsible for collecting and publishing statistics related to the economy, population and society at national, regional and local levels. In order to do this effectively, the ONS is continually developing and deploying new applications to assist in its day-to-day operations.

PAGE 30

AUGUST 2014 | www.testmagazine.co.uk

Software

When it comes to crash testing We’re no dummies

Comprehensive Software QA With access to digital content so readily available to your customers with the likes of Android, Apple and Microsoft putting instant access into the hands of more and more users across a huge range of devices every day, it has never been more important that your software works without failing. We’ve years of expertise ready and waiting to give you the simple and effective testing solutions you need to make sure your software and content delivers a bug-free, seamless and intuitive experience for your customers. For further information and to download our complementary white paper on software testing please visit:

www.testroniclabs.com

Manual and Automated Testing • Test Strategy • Compatibility • Back Office Validation • Mobile Testing • Website and Browser • HTML • Exploratory and Technical Testing • App Store Compliance • Load and Performance • Accessibility • User Experience

TEST AUTOMATION

DMITRY TISHCHENKO A1QA

TO USE KEYWORD-DRIVEN TESTING OR NOT? THAT IS THE QUESTION Dmitry Tishchenko, from A1QA, explains if and how using a keyword-driven testing approach is suitable for your business model…

utomation is a topic that is under constant discussion and improvement today. Its consistency has been already proved. But, for the IT industry in general, this practice tends to be more and more effective and applicable to fit QA process needs better. One of its means is cost effective usage of testing – just imagine that you can double return on investment KPI. On the one hand, you save money, and on the other, you may increase quality using the same budget, or mitigate QA budget risks on stabilisation and delivery stage.

WHAT IS KEYWORD-DRIVEN TESTING?

prepares description of test using already implemented keywords creates automated tests at the same time. 3) Last, but not least, it maintains the relations between test developers and testers themselves, regarding automated tests, which positively impacts QA budgets and overall processes. Of course, there are patterns for using this approach, such as for data driven or others. For instance, you may apply a scripting technique, which requires less money for initial test development if your test process assumes coverage of stable critical functionality (meaning smoke test) the frequency of builds is closer to daily basis, and testing is required for more than four to six upcoming months.

There are many interpretations of what a Thinking of the keyword-driven approach, keyword-driven approach in the world of you should ensure that you have a lot of THINKING test automation is. Different tools allow you business logic. There are similar actions, OF THE KEYWORDto use keywords, but not all cases are which can be used in different test DRIVEN APPROACH, truly keyword driven. The way I used cases (potential keywords). The to describe it simply was, “manual YOU SHOULD ENSURE THAT business logic is too complex for testers mostly create test cases using data-driven approach, which may YOU HAVE A LOT OF BUSINESS keywords and test developers create be extremely effective in some LOGIC. THERE ARE SIMILAR code which implements particular other particular cases. ACTIONS, WHICH CAN BE USED IN keywords via available interfaces”. DIFFERENT TEST CASES (POTENTIAL What do you get from this meaning?

GIVING IT A TRY KEYWORDS). THE BUSINESS LOGIC So, you never know without IS TOO COMPLEX FOR DATAtrying! We at A1QA decided it’s DRIVEN APPROACH, WHICH worth a try. I’d like to share some MAY BE EXTREMELY EFFECTIVE of the most important findings. One IN SOME OTHER of the crucial requirements here is PARTICULAR CASES that special description format should

1) The keyword-driven approach can help to achieve better ROI in the long term as it saves money on support activities and it works better if you have more test cases. It is also worth noting that such an approach requires more efforts at the beginning of test development process.

2) It optimises technical threshold for those who may be involved in automated tests development processes i.e. testers who prepare the description of tests, which is pretty much the same with test automation in terms of keyword driven. We may say that a person who

PAGE 32

be applied from the very beginning of testing activities. Taking into account this fact, and the absence of experience with tools that are specially designed for keyword-driven approach implementation, we decided to split our first keyword-driven practice in two steps.

Step 1 – Proof of Concept (POC). This is the basis for the

AUGUST 2014 | www.testmagazine.co.uk

TEST AUTOMATION

article you are reading. Try out a particular tool; check basic automation indicators like execution/support efforts; and check if the manual tester might be able to handle test case description. Step 2 – Using the keyword-driven approach. Apply this approach from the very beginning of the particular project and cover the whole QA cycle using it. Thus, we have found a “guinea pig” for the first step and here is a description:

SOLUTIONS COMPARISON: Existent solution

Keyword driven POC

Automation approach

Scripting

Keyword driven

Programming Language

Python

Test Automation framework

A1QA SMART framework

Robot framework

Automation tool

Selenium

Test execution system

TFS

Jenkins

Basically, we have selected quite a stable scope in terms of business logic and user interface changes. It was decided to select a rather large scope, which guarantees that there will be enough keywords to make measurements and to check test development abilities using keywords only. As a matter of fact, 200 test cases were selected for automation. It should be mentioned that the selected project was quite dynamic and required daily test executions. It was necessary to speed up the solution comparison process. One of the required factors was to get a situation when test development efforts on the keyword-driven test framework were less labor-intensive in comparison with actual automation for the existent solution. We achieved the mentioned result just after over 150 cases were developed.

LEARNING LESSONS Some good lessons have been also learned about keyword-driven while tests were developing. There are three top issues you should consider applying keyword driven practices to the project: • TBD 1 First tests on RobotFramework KDT require more than average efforts (five to six hours when scripts with DDTregular approach requires three to four hours). Efforts decrease with a list of base keywords being completed and available for re-use (down to one to two hours). • TBD 2

keywords for creating the customised keywords. It takes some time to get familiar with framework specifics on tests development (IDE, libraries, etc.) • TBD 3 RobotFramework execution engine doesn’t contain default mechanisms, implementing itself to solve parallel test execution problems. It was quite a big challenge to implement this mechanism. We tried to run scripts simultaneously on a single VM by using several jobs in Jenkins, but it failed because of AUT UI specific interactions: some UI elements were available only in the currently available browser instance. After a necessary number of tests were ready, we started actual comparison. The first comparison had respect to test execution time. Keyword-driven assumes that some additional logic of parsing keywords works at the background. The impact on test execution duration was mainly under our focus. The results indicated that test execution time is comparable and we may apply keyword-driven in future without additional efforts/hardware for optimising test duration. Efforts required for analysing test results became another comparison point here. In fact, it shows the similar picture. Despite the keyword-driven approach requiring a little bit more efforts to support tests, we got the same trend there. The reason may lay in the sphere of business logic understanding (engineers developing actual solution and brand new one). This part of measurements should be developed further. Another point is test support time. We see that both solutions are comparable by this indicator as well. As for the test development efforts – the manual QA engineer was able to develop a test case description using existent keywords. Taking this into consideration, we may consider spending to be effective, compared to the current solution.

SUMMING UP The first conclusion we are able to mention is that the keyword-driven approach has no contraindications for use and the POC stage showed positive results. On the other hand, based on our measurements we see no extraordinary results or substantial breakthrough. We started the measurements after test development efforts of the new cases became comparable and we’ve got pretty much the same characteristics for critical automation process on both solutions. So, based on this research, one can suggest using the keyword-driven approach when you are going to cover hundreds of test cases in long term projects with a business logic, assuming a high level of keyword re-usage. Having this as a consequence, we decided to apply the keyworddriven approach to several upcoming projects.

The tester is required to learn several special formats and

AUGUST 2014 | www.testmagazine.co.uk

PAGE 33

TEST AUTOMATION ELIZABETH WINSPEAR, CONSULTING TEST MANAGER, MARSTON’S PLC, AND CAPITA ITPS

AN IT TEAM WALKS IN TO A BAR… Elizabeth Winspear, consulting test manager at Marston’s PLC, and Capita ITPS, was originally engaged with Marston’s, an independent brewing and retailing business, 18 months ago to provide manual test leadership for a technical SAP upgrade project. Here, Elizabeth explains how the project changed from a manual world into automation…

obody relishes the prospect of a major IT upgrade, especially not one that impacts several business-critical functions, and has a mountain of custom development thrown into the mix. This is the challenge that the Marston’s IT team faced when we undertook a tricky upgrade of a complex SAP environment, comprising the finance, sales, production planning and warehouse management modules. As a major independent brewing and retailing business with around 1,800 pubs and bars across Great Britain, the top priority was to ensure that any upgrade would be very low risk and cause minimal disruption to the overall business.

CAPTURING INFORMATION The biggest hurdle to overcome was that Marston’s had no test scripts that encapsulated the entire business process, and the IT team didn’t have a full understanding of how the business actually used the applications day-

PAGE 34

to-day. Engaging with the business in terms of getting information and time from them would have also proved to be operationally impractical. So it was agreed that the traditional way of capturing the information needed to define testing scope and to create test scripts would be inadequate. Deploying business analysts across Marston’s 15 UK-wide sites to record the business processes, would simply be too expensive, too time consuming and far too disruptive to users. Marston’s solution was to provide a test management framework to capture business processes, creating high quality tests to reduce the risk during the SAP upgrade. As there were no formal test scripts, and essentially starting from scratch, the SAP upgrade project provided Marston’s with the opportunity to build the foundation for a robust test library.

QUICK AND EASY The Marston’s IT team has years of experience with SAP

AUGUST 2014 | www.testmagazine.co.uk

TEST AUTOMATION

WE WERE ABLE TO CREATE THE CASES BY USING THE NEW STEP-BY-STEP and has therefore worked with most of FROM MANUAL TO GUIDES THAT DEFINED HOW the industry’s leading software testing AUTOMATED TESTING THE BUSINESS USED SAP AND tools before. However the problem with I’m a manual tester by heart, most test tools is that you have to employ RECORDED SCREEN SHOTS but after analysing the situation, specialist programmers, which Marston’s OF ALL THE CUSTOMISED tools and scope of the testing simply couldn’t justify. So the decision was SCREENS. IT TOOK A LOT project I knew the solution was to taken to evaluate a code-free solution OF GUESSWORK OUT OF go for an automated regression from Original Software using its Qualify and WRITING THE EXPECTED suite. With three days of training TestDrive-Assist products. we were able to begin turning the RESULTS FOR THE TEST Members of the team brought in to evaluate business-recorded results into the CASES the proof of concept were impressed with the regression pack. test management software, which appeared to be very quick and easy to map project methodologies into the system. It was intuitive to use, and the ability to hold all documents in a single area was a major benefit. It was decided that it was exactly what was needed to manage testing effectively.

Marston’s was hopeful that by using the selected test tools, the business wouldn’t have to take time out of their day to evaluate and document the business processes required – they could simple record them. They could carry on working as normal, whilst their SAP processes were captured. It meant that IT had a perfect understanding of what business users required the system to do before testing any changes or upgrades.

POTENTIAL TO REALITY The IT team at Marston’s PLC is relatively small, therefore attempting to turn these plans into a working process required additional resource. Capita IT Professional Services was contracted to scope, build and execute the testing for the SAP upgrade and to implement a regression pack to test the impending SAP ECC6 upgrade. An additional Capita resource with experience of automation was brought in to aid the scoping and management process. This specialist and I needed to understand the abilities of the test tools, and the complexity and coverage of Marston’s business requirement and find a solution to bring the two together. As we weren’t able to be deployed across all of the Marston’s sites, we needed a way to gather details of the transactions to be tested, so Marston’s immediately set to work using the recording capabilities of the test automation solution. One hundred carefully chosen users helped us to capture all the day-to-day, period-end and year-end processes that are used across the Marston’s Group. The software was remotely installed on each user’s PC overnight. With an online training guide the business users found it easy to record their processes, with only six queries raised. The recordings made by the users were logged in the test management system, where the Marston’s and Capita team were able to see all the screens, inputs and sequences, with relevant descriptions of the business scenarios to keep everything in context. After a few weeks, Marston’s had over 1,000 transactions fully recorded and ready for the testing work to begin. As a result, we were able to create the cases by using the new step-by-step guides that defined how the business used SAP and recorded screen shots of all the customised screens. It took a lot of guesswork out of writing the expected results for the test cases.

AUGUST 2014 | www.testmagazine.co.uk

Over the course of a month, our team took the 900 plus manual test scripts, which were being executed through the course of the SAP upgrade project, and were able to build automated versions for 650 of them. The whole exercise went incredibly smoothly. We had all that we needed to understand the requirement of the user, without having pages of documentation to analyse and review. Though we were informed that specialists weren’t needed to use the automation tool, we were cautious and resourced a Capita test automation engineer in India to lead the other members of the team. However, the technology was user-friendly enough that soon our manual testers were creating the automation cases with ease. The detailed process of visual recorded results also helped remove any ambiguity over what was required. We were able to run the regression pack numerous times during the upgrade project with the result that we experienced no issues at go-live, so now I have a pack that can be run as often as required. It now only takes two days to execute and saves approximately two weeks of users’ time compared to performing regression testing manually. Maintenance is still important as with any automation suite. The upgrade project the regression pack was originally created for was a technical upgrade, and therefore didn’t change the user interfaces, meaning the "before" and "after" picture from the users’ perceptive shouldn’t change. However as more projects progress through their lifecycle, it is important that the IT team and development specialist maintain their strong communication so that the regression can be reviewed and updated appropriately. Getting 650 automated scripts so quickly was fantastic, especially as automating SAP is notoriously difficult, and by creating a dedicated regression test environment we hope to get that number to over 900. We are even starting to look at what other SAP and non-SAP applications we can automate.

COST REDUCTION By selecting a code-free software testing solution and partnering with testing specialists, Marston’s was able to accelerate its upgrade. The approach kept costs down, enabled business and project people to spend less time on manual validation, and focused effort on areas of change. It also reduced risk and disruption, and the icing on the cake was the project winning a prestigious industry award in recognition of our hard work!

PAGE 35

CROWD TESTING HITESH JOSHI, PRESALES MANAGER, AMDOCS

TESTING FROM TIMBUKTU Hitesh Joshi, presales manager for Amdocs, discusses crowd sourcing in the testing industry, and asks can a team of crowd-testers match the service levels of corporate professional testing service providers?...

ne day I was quite intrigued to see if there is such a place called Timbuktu, as, until then, I thought it was only a common adjective for a faraway place. I decided to search for the term "Timbuktu" in a well-known search engine and, to my amazement, I found out that yes, it not only exists but it’s a historical place in Mali, North Africa. There are certain credible sources of information such as Wikipedia and Wiki Travel, which became the source of my information. But who ensures that what I read is accurate and credible? Well there are more than 50,000 registered users of Wikipedia ensuring the content is correct and is an accurate reflection of the facts, thus ensuring the quality.

I WONDERED TO MYSELF IF I COULD FIND A TEAM OF CROWD-TESTERS WHO COULD NOT ONLY TEST, BUT MATCH THE SERVICE LEVELS OF CORPORATE PROFESSIONAL TESTING SERVICE PROVIDERS

One hundred hours of videos are uploaded onto YouTube every minute from its user-base of many millions of users. YouTube is a prime example of the crowd-sourced content generation. The website Harrypotterfanfiction. com hosts an astounding 82,000 stories written by legions of fans who have converted their passion for the venerable children’s franchise into a penchant for creative fiction. Orienting the power of crowds towards

PAGE 36

AUGUST 2014 | www.testmagazine.co.uk

CROWD TESTING

such an activity is now an established phenomenon that has disrupted many business models in the market.

IF CROWDTESTERS JOIN HANDS ONLINE WITH MILLIONS OF SIMILARLY MINDED TESTERS IN DISPARATE LOCATIONS AROUND THE GLOBE (DARE I SAY – EVEN IN TIMBUKTU!), THE ECONOMIES OF SCALE ACHIEVABLE COULD CREATE A FORCE FOR of defects by distributing the testing DISRUPTING THE workload efficiently to meet short deadlines. INDUSTRY

Such a trend has also gained some traction in the arena of the testing industry. Using the power of crowds to carry out testing on software provides excellent benefits in terms of turnaround times – directly impacting on the time-to-market and the quality of output, leading to improved cost efficiency. The concept of CrowdTesting as a Service (CTaaS) adds an interesting new welcome dimension to the industry.

CROWD TESTERS VS CORPORATE PROFESSIONAL TESTING SERVICE PROVIDERS After all these phenomenal success stories, I wondered to myself if I could find a team of crowd-testers who could not only test, but match the service levels of corporate professional testing service providers? Corporates have large offices and large teams of testers with a deep and concentrated knowledge base from where they can draw their knowledge. They have onshore, offshore and near shore facilities to gain cost and time efficiencies using a “follow the sun” approach. The comparison against a bunch of crowd-testers seems unrealistic. Fair enough, but then can they partner with these giants and find common cause with them? What value would they be able to bring to the table? We’re not accustomed to thinking of communities as economic entities. A company clearly offers advantages when productivity is weighed through a concentrated infrastructure – you’ll always need a factory to produce steel. But in the realm of information production, the community is beginning to rival the corporation for primacy. Hence if crowd-testers join hands online with millions of similarly minded testers in disparate locations around the globe (dare I say – even in Timbuktu!), the economies of scale achievable could create a force for disrupting the industry. It is the evolution of online communities with their ability to efficiently organise people into economically productive units that will drive changes in the marketplace. What's more, crowd testers do not necessarily require any technical background in testing. Anyone can join these communities and become productive members with full support given to them remotely.

CROWD-TESTING AS A SERVICE What areas of testing could be typically covered by CTaaS players? If I built a mobile app, would they be able to test it? Indeed they could cover that – not only by testing on multiple platforms such as Android and iOS, but also by being able to quickly detect a great number

AUGUST 2014 | www.testmagazine.co.uk

If I give them the game which I recently had built for testing – could they surprise me with their ability to find defects? What if I got greedier and gave them a web portal and security testing work of a web application? What if I turn to enterprise systems and look at very complex applications and systems, which were built and tested either by in-house teams or by external vendors? I asked myself, will my CTaaS testers from remote geographies ever be able to test this and not compete but co-exist alongside the world of corporate testing vendors.

This interesting story gives some insights on the future of crowd-sourcing as a service strategy. TopCoder.com is an online crowd sourcing company and the world’s largest crowdsourcing development, design, and data science platform. The TopCoder community gathers the world’s leading experts in design, development and data science to work on interesting and challenging problems for fun and reward. The community helps members improve their skills, demonstrate and gain reward for their expertise, and provide the industry with objective insight on new and emerging technologies. Today they are around 663,000 members strong. In 2006, Sree Kotay the senior VP of software development at AOL, called TopCoder. During a meeting, they showed him a live competition of application development which was going on at the TopCoders’ site. “That’s not a contest,” the TopCoder CEO Morris told Kotay. “That’s the future of software development.” AOL Commissioned TopCoder to write three programs: an enhancement to AOL’s email system; a content syndication system; and an ambitious back-end system that would allow AOL’s instant-messenger application to work seamlessly with other IM clients, such as Google Talk. This was the tipping point for TopCoder. AOL was so happy with the final product that they’ve since commissioned TopCoder to work on about a dozen more programs. TopCoder’s programs had an average of .98 bugs per thousand lines of code beating the industry standard. The CEO, Morris notes with pride. “A traditional firm might have assigned six or seven developers to the project, and it would have taken well over a year to complete. It took us a little over five months.” Considering the “corporate world” example, the software application development has done it, but can crowd-testing communities achieve similar results too? Or are these coding communities a threat to the crowd testing communities? These are the questions which still remain unanswered, though I can surely see a tester from Timbuktu benefitting and having a bright future.

PAGE 37

SOFTWARE DEVELOPMENT DAVID GIL, CONSULTANT SOFTWARE ENGINEER, CRITICAL SOFTWARE TECHNOLOGIES

DRIVING CHANGE: WHY SOFTWARE SAFETY MATTERS IN THE AUTOMOTIVE INDUSTRY David Gil, consultant software engineer, Critical Software Technologies, looks at how software is assured in markets such as the aviation industry to see what lessons can be applied to the automotive industry, arguing that, as software and onboard systems increase in their complexity, the need for Independent Verification and Validation increases…

t is fair to say that the automotive industry has been very successful in integrating software systems for many decades. In fact, systems like traction control, ESP and ABS have dramatically improved the performance and safety of vehicles over many years. However, recently, on quite a few occasions, safety issues in the software of road vehicles have made the news headlines, resulting in vehicle recalls that have damaged the industry’s reputation and cost it millions of pounds. This trend is not entirely unpredictable. The automotive industry is extremely competitive compared to other industries that include software systems in their products. Compared with, for example, the aerospace, nuclear THE and medical industries, the automotive industry AVERAGE operates in a state LUXURY CAR HAS of exacerbated ABOUT 100 MILLION competition, in which

LINES OF CODE, WHILE THE RIGOROUSLY TESTED BOEING 787, ONE OF THE MOST MODERN AIRCRAFTS IN THE WORLD, HAS ONLY 7 MILLION LINES OF CODE

PAGE 38

innovative products are often rushed to market in order to compete aggressively with competitors who are also driving the next big thing. Entertainment systems, head-up displays, hybrid propulsion and parking aids are some of the best examples of modern automotive innovations that require the use of complex software. For example, the type of system that now commonly controls hybrid propulsion integrates several functions that were previously scattered across different systems and adds new functions related to other, new sub-systems, such as battery and hydrogen storage. Of course, every new sub-systems introduces new safety issues of its own.

DRIVING INNOVATION The automotive industry has been very successful in driving innovation for many years. However, unlike its previous software systems, today complex systems, driver inputs and actuators are being integrated, especially in hybrid cars. These new systems present entirely new

AUGUST 2014 | www.testmagazine.co.uk

SOFTWARE DEVELOPMENT

safety issues, with a wide range of new technologies pushing the boundaries of automotive development.

RECENTLY, ON QUITE A FEW OCCASIONS, SAFETY independent persons not involved in ISSUES IN THE SOFTWARE the original software development activities. The process of ISVV OF ROAD VEHICLES HAVE identifies suitable design assurance MADE THE NEWS HEADLINES, levels across a process to achieve RESULTING IN VEHICLE compliance with a chosen RECALLS THAT HAVE functional safety standard, thereby DAMAGED THE INDUSTRY’S increasing confidence in the chosen solution. REPUTATION AND COST IT MILLIONS OF “Validation” ensures the system’s POUNDS requirements are fit-for-purpose by

Further complexity is added when we consider that vehicles are operated on streets, driven by ordinary people and repaired in everyday high-street garages. While this sounds rather obvious, it presents a rather uncontrolled environment when compared with the aerospace industry, for example, where aircrafts are operated over controlled airspace, flown by trained pilots and maintained by trained engineers, adhering to highly-controlled maintenance and operation procedures.

SYSTEM ENGINEERING CHALLENGES For the automotive industry, these factors present systems engineering challenges, the outcome of which is complex software with different fail safe paths and increased logic in Failure Detection, Isolation and Recovery algorithms (FDIR). It also means that the complexity of the software that is present in cars is growing exponentially. The average luxury car has about 100 million lines of code, while the rigorously tested Boeing 787, one of the most modern aircrafts in the world, has only 7 million lines of code. Complexity aside, an increasing number of lines of code often correlates with an increasing number of software problems. On average, before validation activities, 100,000 bugs will exist per million lines of code. Despite the fact that systems engineering and software standards in the automotive industry are on a par with those in the aerospace industry, road vehicles have been much more prone to prominent software issues than western aircraft and space systems. The painful lessons that come from integrating complex software systems badly is a lesson that the aerospace industry learned long ago. The defence industry too, was among the first to adopt a widespread mission and safety-critical approach to the development of their systems, equipment and operational procedures involving software.

INDEPENDENT SOFTWARE VERIFICATION AND VALIDATION As with other industries, when dealing with systems that have high levels of safety-criticality, Independent Software Verification and Validation (ISVV) should also be prescribed within the automotive industry to verify the operational safety of systems. This would involve project documentation, design/code verification and software testing to be performed by

AUGUST 2014 | www.testmagazine.co.uk

determining whether they fulfil the key performance and safety obligations. “Verification” ensures that the system’s design is faithful to these requirements, and that the software artefacts are faithful to the system design. Throughout, the goal of ISVV is to improve a system’s reliability, availability and safety performance, which is one of the reasons it is undertaken by an independent entity. To date, however, within the automotive industry, software verification and validation activities are often awarded to specialised departments within the company or group producing the system in the first place, removing independence. Although this conforms with written standards requirements, this can create a conflict of interest, as the testing department are often sensitive to the same competitive financial constraints of the automotive manufacturer. By contrast, common practice in the aerospace industry goes beyond written standards. Often at the insistence of the prime contractors who provide the aircraft or the satellite as a whole, ISVV activities are awarded to companies with no technical, financial or shareholder links with the system manufacturer. This is because manufacturers of complex aerospace systems recognise the reputational, financial and human damage system failures can bring. As an indication of where the automotive industry may be heading, these standards were enforced on the aerospace industry by governments, through certification authorities and prime contractors who feared their latest assets were too unreliable, expensive and dangerous. At the time, other industries with critical systems and assets overlooked the approach, deeming it “unnecessary”, “uncompetitive”, “slow-moving”, “not applicable” and “too expensive”. Those industries have since paid for that approach, with significant embarrassment, economic loss and, most regrettably, human lives. The question is, what is the automotive industry waiting for? With more and more industries relying on ISVV to sustain confidence and assurance in their mission, safety and business-critical processes, it is time that the automotive industry joined them.

PAGE 39

SOFTWARE DEVELOPMENT DMITRIY RADCHENKO, QUALITY ASSURANCE TEAM LEAD, PROJECT MANAGER, QATESTLAB

THE IMPORTANCE OF MOBILE SECURITY TESTING Dmitriy Radchenko, QATestLab's quality assurance team lead, project manager, discusses the two most important aspects of mobile security testing…

ince software products are used in many business dimensions and deal with different data, security has become of vital importance for most of them.

Not only must top secret government data be well protected. Various banking applications, programs for managing military projects, satellites, producing goods and others that deal with finances, weapon, production, personal and other sensitive user data must guaranty data protection from outside interference. Even computer games should be protected from unauthorised usage. There are security aspects in each application where the user has to log in. The range and quality of online services are still growing. More and more people begin to use desktop and mobile programs for different purposes, including financial operations. However, online activities cannot be based on pure trust, as there are some ill-minded people who want to steal sensitive data. Unfortunately, some mobile application developers neglect security issues due to tight time and budget constraints of the projects or other reasons. Poor security is not as obvious as performance or usability defects are. It does not affect functioning of the program, but it can result in severe consequences for the users. Security aspects often receive less attention and care on a mobile project than they do during construction of a desktop software product. That is why security issues must always be properly verified in course of mobile application testing. Security testing is a complex task; it includes checking of many aspects, and automated test engineers with special skills and knowledge mostly perform it. Many different instruments and equipment can be involved in this kind of activity. However, some obvious aspects of mobile program security can be checked during manual testing by a test engineer, who is not involved in test automation.

MOBILE APPLICATION SECURITY ASPECTS THAT SHOULD ALWAYS BE CHECKED: • Whether the sensitive user data is transferred by secure communication protocols; • Whether it is possible to get to the pages that must be restricted from unauthorised access trying various URLs. Secure communication protocols should be used for all the sensitive information, even for usernames and passwords. If no supplementary instruments and devices are available, the tester can pay attention to visual signs of using the protocols. For example, there may be a padlock in the

PAGE 40

browser and so on. It is possible to capture the traffic and verify if it is properly coded and cannot be read by people with the help of a personal computer, an emulator of the smartphone or tablet and an instrument analysing network protocols. To do this, one needs some technical knowledge or the aid of a programmer or automated test engineer. One can do the same by installing an instrument for network protocols analysing on the mobile device. But, experts in mobile software testing claim that it is often hard to find a suitable instrument for a tablet or smartphone. Another way of checking security of the traffic is accessing a personal computer with an installed instrument for analysing network protocols through the mobile device by means of a virtual network computing application. This enables the tester to interact with the tested software on a tablet or smartphone and watch the traffic on the computer. It may sound weird, but sometimes hackers, and just curious users, can access restricted data by just trying a different URL in the location bar. It happens when data, which must be protected, is in plain text in the URL and is not coded. There are programs where it is possible to get to the personal account of another user by entering your account and changing some words or numbers in the location bar. URL may cause problems to software engineers. On the one hand, usability requires that URLs should be clear, understandable and user friendly. On the other hand, it mustn’t cause security flaws. Therefore, it is necessary to change the URL to get to the pages with restricted access during the program testing. The software must display an error message and do not give any information that can be used for evil purposes. Security measures become more and more complex and reliable along with information technology evolution. Unfortunately, hackers also become more skilled and learn how to overpass the best security measures. But it’s a shame when people with evil intentions can easily steal somebody’s sensitive data just because elementary software security rules and precautions are neglected or overlooked. This often refers to mobile software products and their usage. With regards to public WiFi hotspots, ill-minded people often connect their instruments to them and gather unsecured traffic, namely logins, passwords, personal and financial data, and then they use them for criminal purposes. Mobile software security comprises a lot of other aspects. Test engineers with good programming skills may work as hackers and try to get to the protected pages and data. For this, it is wise to utilise special tools imitating hacker attacks. In all eventualities, all security defects should receive high priority and be corrected before the product release.

AUGUST 2014 | www.testmagazine.co.uk

Software Testing Network Strength in numbers www.softwaretestingnetwork.com

Membership benefits include: Series of one day debate sessions High-brow webinar streams Research & industry findings Exclusive product discounts Peer-to-peer networking Annual gala dinner And so much more...

Becoming a member of the Software Testing Network joins you together with like-minded professionals that are all striving for technical excellence and championing best practice and process

Focus on: Career COL (RETD) STEWART SHARMAN HEAD OF EX-FORCES PROGRAMME, FDM GROUP

FROM HM ARMED FORCES TO A TESTING TEAM Hiring ex-Forces personnel is vital. Head of FDM’s Ex-Forces Programme, Col (Retd) Stewart Sharman, explains how their skills, gravitas and experience can boost your testing team…

aving left the Army in December 2013 after nearly 30 years in the Royal Signals, I am well versed in the importance of a smooth transition from the Forces into a commercial working environment. Through leading FDM’s Ex-Forces Programme, I have seen personnel adapt to a range of business situations, and many have become excellent testers. I am often asked, “How did the Forces prepare you for a new career in the commercial world?” Quite simply, there are a range of skills and competencies such as leadership and initiative that are integral to employers – and being able to think on your feet becomes second nature after working in the Services. If you stop and think about the capabilities required for a certain role, you will realise that many of these are deep-rooted in ex-Forces personnel, and they are prepared above and beyond those early on in their career.

WHERE THE FORCES AND TESTING COLLIDE: Employment in the Armed Forces develops fundamental skills that undeniably suit a testing team. The challenge of starting a new job is magnified for ex-Forces personnel starting a new career in the civilian environment; yet don’t think for a minute that this does any disservice. Commitment, team building and maturity are just some of the attributes required on both sides of the career spectrum – civilian life and HM Armed Forces – and can

PAGE 42

be moulded and developed alongside technical ability.

EMPLOYMENT IN THE ARMED FORCES DEVELOPS FUNDAMENTAL SKILLS THAT UNDENIABLY SUIT A TESTING TEAM

Leadership: A leader is someone who can direct a project or people with care; appreciate all inputs; and make measured decisions. Creating clear test plans to outline an agreed approach to testing activity will no doubt feature at the early stages of a project. This will require a vigilant strategy to establish goals, timescales, responsibilities and potential issues. Being able to analyse and lead situations is a skill found across all ranks of the army, and the transference of these skills to the workplace should be recognised and facilitated.

1. Maturity: A high level of commitment and dedication is demonstrated during a career in the army and thus, a level of maturity as a result of experience working with a range of individuals. Testers are required to identify and log defects and work with the development team to resolve these issues; a sometimes laborious and time-consuming process that requires excellent people management skills. 2. “Can do” attitude: Having the confidence to get on with a project without negative thinking is demonstrable amongst the ex-Forces community and essential when persevering with a problematic defect as a tester.

AUGUST 2014 | www.testmagazine.co.uk

Focus on: Career 3. Team building: Working across different teams at various stages of a testing project is a daily task, particularly via the developer to tester channel that flows back and forth. The Armed Forces appreciate camaraderie and are trained to apply a collaborative approach; a skill that is integral to the smooth running of a project where the developers and testers must work towards their common goal. 4. Reliability: Forging a career in the Forces shows the upmost dedication to your country and a fearless drive to not give up on challenging training programmes and operations. Testers are required to produce reports of progress, success and failure and reliably documenting these stages is essential to a successful project. 5. Strong communication: Due to the continual review of a product at testing stage, an ability to communicate with various levels of seniority and across departments is key to translating any issues and moving the project forward. In high-pressured situations that often arise in the Forces, an ability to explain a situation clearly will portray a confident persona and transmit that confidence to the whole team. 6. Problem solving mentality: A skill that comes naturally in the Forces and a daily task as a tester. Testing is analytically, intellectually and creatively challenging, as it requires continuous assessment to ensure software is of high quality and effectively meets the user’s needs. With that in mind, a clear vision of the end product is required, accompanied by finite attention to detail to meticulously iron out bugs.

7. Analytical thinking: To be THERE certain of objectives, ARE A RANGE it is important to OF SKILLS AND review business COMPETENCIES SUCH AS (user) requirements and ensure these LEADERSHIP AND INITIATIVE requirements are THAT ARE INTEGRAL TO sufficiently detailed, EMPLOYERS – AND BEING unambiguous, and ABLE TO THINK ON YOUR ultimately “testable”. FEET BECOMES SECOND Expert analysis is an ability drawn on NATURE AFTER both during military WORKING IN THE missions and a civilian SERVICES environment.

EMPLOYING EX-FORCES PERSONNEL FDM launched its Ex-Forces Programme earlier this year, which offers Service leavers a new career in software testing. We recognise all these skills in ex-Forces personnel and our Ex-Forces Programme is imperative to facilitating their civilian career. By building on skills they already have, our ex-Forces consultants can quickly adapt to new environments and apply their knowledge. The skills, gravitas and experience I gained during my time in the Royal Signals have most definitely played an important role in my transition from the Army to Head of FDM’s ExForces Programme, and it is fantastic that companies like FDM are recognising the value ex-Forces personnel can bring to their workforce.

“The Armed Forces equipped me with skills that enable me to carry out my daily tasks in a proactive, productive manner by making the most use of my time through effective time management and also how to effectively work within a team. “I learnt how to be highly flexible and adaptable, something that is sought after in the workplace. I think the Armed Forces also taught me how to handle constructive criticism effectively, especially when starting in a new career.” Jarryd Aubert, FDM Test Analyst at Barclays “I am delighted with the relationship Virgin Money has developed with FDM. Securing talent in our business is critical and I’m excited to explore opportunities beyond the realms of IT and Change and also initiatives that support exMilitary. This is a lasting partnership that I really value.” – Chief Operating Officer, Virgin Money

AUGUST 2014 | www.testmagazine.co.uk

PAGE 43

TESTEXPO 2014 PREVIEW

TESTEXPO 2014: DEFINING THE DIGITAL TESTING STRATEGY Following a hugely successful 2013 event, TestExpo will be returning to London on 21st October. With 20% off for TEST Magazine readers and insightful presentations from some of testing’s greatest, this is one event not to be missed…

Each year, the TestExpo programme is updated and refreshed with exciting new themes and topics that impact today’s business. For 2014, the key theme addressed will be “Defining the Digital Testing Strategy”. This very current topic, one that is close to the heart of TEST Magazine, will aim to address how technology and software are moving towards an increasingly digital landscape; how this is impacting testing; and what needs to change in order to keep up with the pace of innovation. Join us to explore the following topics: • Defining the digital testing strategy and considering which methods and tools will provide the right mechanisms for success: Including agile, cloud, SOA, automation and standardised processes. • Incorporating new technologies and devices into testing plans: Such as feedback from social media and Big Data/analytics, and channels such as mobile, smart tv, and the Internet of Things • Budgets: How much of the testing budget should go into new channels and the need to prove the value of QA through ROI. • The changing role of the tester: From technical testers to new skills, the role of new testers coming up into the market, and do testing certifications cater for new waves of applications and devices? Differentiating between “testing” and “checking” and bug hunter vs. bug preventer.

PAGE 44

What sets TestExpo apart is its emphasis on knowledge sharing and “real life” experience, providing participants with insight into how other organisations are leading the field in testing and quality assurance. Lively discussion sessions on a range of topics will return in 2014, alongside the inspiring keynotes and presentations delivered by thought leaders and experts from leading organisations in the software testing arena. In addition, exciting case studies from Experian Data Quality and BBC Future Media will respectively cover how the role of the tester has evolved, and how testing for mobile platforms has developed to keep up with the breakneck pace of mobile application development and delivery. TestExpo is structured to maximise opportunities for visitors and exhibitors to network, interact and learn about new opportunities, with the option to view useful demos, and learn about valuable innovative ideas and techniques. As Platinum sponsor, Borland headline this year’s event. They are already joined by Zephyr, Prolifics, Seapine Software, TestPlant, Tricentis and PlanIT. TEST Magazine offer media support for the event for the second year running and look forward to meeting delegates on their stand on the day. For any organisations thinking about sponsoring TestExpo – it’s not too late. Please contact the team via email at testexpo@unicom.co.uk.

AUGUST 2014 | www.testmagazine.co.uk

TESTEXPO 2014 PREVIEW

Sponsors:

TestExpo 2014 will take place at the Hotel Russell in Londonâ&#x20AC;&#x2122;s Russell Square on Tuesday 21st October, and will start at 8.30am. To claim your 20% discount off full price tickets and guarantee your place at the event, please visit www.testexpo.co.uk and quote 'TMEXPO20' in the comments field on our online registration form. AUGUST 2014 | www.testmagazine.co.uk

PAGE 45

LAST WORD DAVE WHALEN PRESIDENT AND SENIOR SOFTWARE ENTOMOLOGIST WHALEN TECHNOLOGIES HTTP://SOFTWAREENTOMOLOGIST.WORDPRESS.COM

WHO TESTED THIS?! By Dave Whalen

tend to be somewhat obsessed with applications that are obviously not tested. I don’t mean not well tested. I’m talking about the ones with no testing whatsoever! It’s rare that I come across one these days, but they’re still out there and they drive me completely insane! I recently had the need to visit the mobile application of a major financial institution to conduct some rather important financial transactions. And because they had a mobile application, I thought these guys are obviously “in tune” technologically. So I download the application, launch it, and try to create my user account. Here’s where the fun begins. A standard user account creation page is displayed, but I don’t see anything indicating any of the fields are required. It’s not a big deal – it’s something only a tester would notice. I enter most of the information, intentionally leaving out my phone number and email address. I tap “save” and I get an error message that I’m missing required fields, but it doesn’t tell me which ones, so I’m left to guess. I’m guessing it’s my phone number, so I add it and tap “save” again. Same message. Being the smart tester that I am, I quickly deduce that phone and email must be required. So I add my email address and tap “save” again. Boom! Another error. Once again, I’m missing required fields. I look at the form and notice both the phone number and email fields are empty. I repopulate the fields, and tap “save” again. I should be golden this time right? Nope. Apparently my phone number isn’t “correctly formatted”. Does it tell me the proper format? No! After three tries I get the correct format and clear that error. Phew – finally done. Nope! My frustration is building but I’m still relatively calm. This time I’m getting an invalid password error. Does it tell me what a valid password is? No! So I try passwords of various lengths, using upper- and lower-case letters, numbers, and special characters. On my fifth try, I hit the magic combination and that error clears. Now I’m getting a “user already” exists error. Ugh! I feel my head beginning to explode as my blood pressure rises. OK – I try a new username. Finally I successfully create an account. I get a nice welcome message, followed by an email with my account information. Great, my username and password were just sent via email. Seriously! Back to the business at hand. I tap on the “View My Account” button. Thinking. Thinking. Thinking. Finally a response! The system is down for “periodic maintenance, please try again later.” AHHHHHHH! Audibly this time, scaring my wife, children, and about 12 jittery Starbucks customers half to death. Sorry.

PAGE 46

BY NOW, I’M LIVID! THIS TIME I’M CALLING THEM. I STEP OUTSIDE AND CALL THE CUSTOMER SUPPORT By now, I’m livid! This LINE. I’M SURE YOU CAN time I’m calling them. ALREADY GUESS WHAT I step outside and call HAPPENED NEXT the customer support line.

I’m sure you can already guess what happened next. Yep – after about 10 rings I get – “I’m sorry, we are unable to assist you at this time. Please call back during normal business hours.” Great! When are normal business hours? Which time zone? I’m in Colorado. Where are you? “Press one to leave a message.” Oh, I thought about it! But common sense took over. The next morning, after I had calmed down, I called customer support. 10 rings, and then an answer. He said his name was Bob, and he was actually a pretty nice guy. We took care of business and he recommended that I write an email to explain my frustrations with the mobile application, and then gave me the appropriate email address. Awesome! That’s exactly what I did. Of course I started by explaining that I’m a professional software tester and that I write an article for a major European software testing magazine. I was very nice. Not only did I explain the situation, I made suggestions on possible usability improvements. Very cordial. I even had my wife and daughter proofread it to make sure I was behaving. In addition, I praised Bob for his assistance. Pretty good email if you ask me. Well it’s been about 60 days. I’ve yet to receive a response. The mobile application hasn’t changed. What to do? Maybe I’ll write an article about my experience?...

AUGUST 2014 | www.testmagazine.co.uk

THE EUROPEAN SOFTWARE TESTER INNOVATION FOR SOFTWARE QUALITY

Subscribe to TEST free! TEST: THE EUROPE AN

VOLUME 6: ISSUE 2 APRIL 2014

SOFTWARE

THE EUROPEAN SOFTWARE TESTER

TESTER

.uk agazine.co www.testm

INNOVATION FOR SOFTWAR E QUALITY

SOFTWARE TESTER

EUROPEAN

5: ISSUE 4 VOLUME 13 AUGUST 20

INNOVATION FOR SOFTWARE QUALITY

TEST: THE EUROPEAN SOFTWARE TESTER

TEST: THE

N INNOVATIO E WAR FOR SOFT ITY QUAL

www.testmagazine.co.uk

THE EUROPEA N SOFTWARE

VOLUME 6: ISSU E4 AUGUST 2014

CASH IN HAND?

R IS IT TIME FO ? K IN TH E R A

NEW TECHNO LOGY BRINGS EMO TIONS TO LIFE

CONSUMERS CAN SOON SECURELY LINK THEIR CREDIT CARDS TO THEIR PALMS FOR FASTER PAYMENTS

D RAPID QUALITY AN T MUST EN DEVELOPM AND -H -IN GO HAND

INSIDE:

5: ISSUE 4:

MOBILE TESTING:

AUGUST

BATTERY LIFE: FROM 100% : CAREER

2013

TO 0% IN 60 MINUTES

4: AUGUST 2014

VOLUME

WS: WORLD NE INTERVIEW: N SNOWDE THE EVOLUTION OF EDWARD E R HIS LIFSOFTWARE TESTING FEARS FO WS VIE ER INT FOCUS ON

VOLUME 6: ISSUE

VOLUME 6: ISSUE 2: APRIL 2014

INSIDE:

INSIDE: DAY IN THE LIFE OF... AN SDET

NATIONAL SOFTWARE TESTING CONFERENCE 2014INTERVIEW: PROGRAMME: PAGE 27 - 46TURN OFF SPYING

cover_with_spine_april_2014.indd 1

, EDOM

TURN ON FRE 23/04/2014 16:13

FOR EXCLUSIVE NEWS, FEATURES, OPINION, COMMENT, DIRECTORY, DIGITAL AND MUCH MORE VISIT: testmagazine.co.uk

Published by T H I R T YO N E

www.31media.co.uk

TESTER

www.testmaga zine.co.uk

E TESTER SOFTWAR OPEAN THE EUR

Telephone: +44 (0) 870 863 6930 Email: info@31media.co.uk Website: www.31media.co.uk

SUCCESSFUL SOFTWARE DELIVERY

DO YOU WORK WITH THE SAME VISION? Create the software your business really needs Requirements. It’s surprising how often a single link can break the software supply chain. But it’s where most defects occur. Removing them later in the lifecycle is costly, impacts delivery schedules and drains resources. That’s why errors and rework so frequently undermine project success. While complexity has increased exponentially, managing requirements hasn’t evolved at the same pace. Borland tools can improve collaboration. Our technical solution keeps stakeholders in sync throughout the project lifecycle. Better input means better products. So join the thousands of Borland customers who already tackle requirements the right way ... precisely. See more at www.borland.com/connect