TEST Magazine - July 2015

TEST: THE EUROPEAN SOFTWARE TESTER

INNOVATION FOR SOFTWARE QUALITY VOLUME 7: ISSUE 3 JULY 2015 www.testmagazine.co.uk

INSIDE: BIG CHANGES AHEAD FOR THE SMALL SCREEN GETTING IT FIRST TIME RIGHT TESTING TOOLS: AIDING BUT NOT REPLACING

VOLUME 7: ISSUE 3: JULY 2015

WILL THE RISK OF FAILURE ONLY INCREASE?

TESTING TIMES FOR THE BANKING SECTOR

CONTENTS

INSIDE THIS ISSUE

24.

Software localisation: challenging, but comes with promise

6.

Getting it first time right

20.

Testing times for the banking sector

THOUGHT LEADERSHIP

6

Getting it First Time Right

Siva Ganesan, Vice President and Global Head of Assurance Services at Tata Consultancy Services, contemplates – what the advantages of creating a product that is first-time-right.

COVER STORY 20

NEWS 9

OPINION

Why is the IT Sector not on the government’s agenda?

THIRD DIGITAL AGE

SMART TVS 22

Culture mindset changes needed to attract the new generation of software testers

JULY 2015 | www.testmagazine.co.uk

Big changes ahead for the small screen Daniel Box, Technical Client Services Manager, BugFinders, discusses how QA departments can prepare for the rise of smart TVs.

16

Taken from her keynote speech at the National Software Testing Conference in May, Helen Byrne, CIO Testing Services, Direct Line, addresses the need for a culture of change to ensure continued successful operations in the future.

Testing times for the banking sector In January this year, Sainsbury’s Bank suffered a widespread system failure, leaving customers unable to use their credit cards. They were quick to voice their anger on Twitter and the story was picked up by the mainstream press.

14

The IT sector seems to have disappeared completely off the political agenda, Daniel Beazer laments the missed opportunity.

SOFTWARE LOCALISATION

24

Software localisation: challenging, but comes with promise

Quality assurance is a huge obstacle in the way of smooth software localisation. Ofer Shoshan, CEO, One Hour Translation, explains.

PAGE 1

CONTENTS

INSIDE THIS ISSUE

16.

Culture mindset changes needed to attract the new generation of software testers

26.

Testing tools may find bugs, but they don’t improve quality

TESTING TOOLS 26

Testing tools may find bugs, but they don’t improve quality

Noel Wurst at Skytap considers the questions developers and testers should ask about their tools.

LOAD TESTING

CONFERENCE 2015

The National Software Testing Conference 2015

On the 19th of May 2015, the halls of the world famous British Museum, known for its impressive collections of the past, were filled with discussion about technologies of the future.

THE TESTING CAREER CORNER 38

30

At your service

Interview tips: true and tested Richard Akrofi from TEST Associates shares his best interview tips.

The challenges of load testing can be reduced if you take an open-source tool and convert it into a service. Gérald Pereira, CTO, Jellly.io, explains.

TEST AUTOMATION

32

Automated software testing: the where & why

36

TEST AUTOMATION 40 Tackling the larger scale

Hung Nguyen, CEO, LogiGear, explores largescale test automation and its potentials.

Arupratan Santra, Sr. Project Manager, Infosys, illustrates a case study in automated testing.

PROCESS AUTOMATION DEVOPS

35

Defining DevOps

Carl Bruiners, Head of DevOps, MagenTys, highlights some common mistakes people make when implementing DevOps.

JULY 2015 | www.testmagazine.co.uk

42

The collaboration crossroads

Neil Kinson, Vice President, EMEA, Redwood Software, explains why collaboration of IT and business processes is critical to the success of any business.

PAGE 3

Get the UX factor Put the user at the heart of your testing Ensure the customer remains at the heart of your software development, and your organisation stays ahead of the competition.

Our UX certification combines an exam and portfolio assessment to deliver valuable, practical skills.

Š BCS, The Chartered Institute for IT, is the business name of The British Computer Society (Registered charity no. 292786) 2015

BC1245/LD/AD/0615

bcs.org/ux

LEADER

THE BANK OF ENGLAND INVITES THE ETHICAL HACKERS Hello, and welcome to the JULY 2015 issue of TEST Magazine.

Speaking earlier this July, Bank of England Governor Mark Carney, noted that less than a third of financial institutions deemed ‘critical’ to the UK economy have carried out cyber security testing under the Bank’s CBEST framework.

Launched last year, the CBEST is a set of guidelines and tailored tests for banks to see how vulnerable they are to cyber attacks. The voluntary tests are designed with input from the GCHQ, the UK’s cyber intelligence and defence agency. Ethical hackers try to break into bank computer systems, while frontline cyber security staff is unaware of the exercise. The hackers then report any security gaps or flaws that they encounter.

Reportedly, only 11 out of 35 institutions deemed ‘critical’ to the UK’s economy have participated in the tests.

However, when addressing the Treasury Select Committee in Parliament, Carney said that he wasn’t worried about the low uptake of the CBEST testing, as many institutions are foreign and are probably working with other regulators on cyber security.

Carney was quoted as saying, “The importance of cyber risk is very high, as high as it’s ever been.” Indeed, a survey from PwC published last month highlighted that 90% of large organisations suffered a cyber security breach in 2015, costing £1.46 - 3.14 million on average.1 Since 2011, the Bank of England has run two industry-wide cyber security tests, nicknamed ‘Waking Shark.’ After the most recent test in 2013, the Bank reported “considerable progress” in cyber attack responses from participants

1.

‘2015 Information Security Breaches Survey’, PwC, http://www.pwc.co.uk/assets/pdf/2015-isbs-technical-report-blue-03.pdf.

including investment banks, financial market infrastructure, the financial authorities and relevant government agencies.

Do you want Then next year, we will see to write for the first transatlantic cyber TEST Magazine? exercise, organised by UK and Email cecilia.rehn US authorities. This will test the @31media.co.uk vulnerabilities of large financial institutions in a co-ordinated manner. It will be very interesting to track the outcome from this project, to see how such collaborative efforts can translate into other sectors.

This July issue covers the financial sector, with Richard Lowe from SQS discussing how the testing industry should respond to the system failures at Sainsbury’s Bank earlier this year, to avoid a repeat. We’ve also got a host of other articles on topics from test automation to load testing, as well as a discussion on how to inspire and employ Generation Y (the Millennials) from Helen Byrne at Direct Line.

I hope you enjoy this issue, and look out for our new redesign in September!

Cecilia Rehn Editor

EDITOR Cecilia Rehn cecilia.rehn@31media.co.uk Tel: +44 (0)203 056 4599 THE EUROPEAN SOFTWARE TESTER

© 2015 31 Media Limited. All rights reserved.

TEST Magazine is edited, designed, and published by 31 Media Limited. No part of TEST Magazine may be reproduced, transmitted, stored electronically, distributed, or copied, in whole or part without the prior written consent of the publisher. A reprint service is available.

Opinions expressed in this journal do not necessarily reflect those of the editor or TEST Magazine or its publisher, 31 Media Limited.

ISSN 2040-01-60

JULY 2015 | www.testmagazine.co.uk

TO ADVERTISE CONTACT: Anna Chubb anna.chubb@31media.co.uk Tel: +44(0)203 056 4945 Magda Urbanska magda@31media.co.uk Tel: +44(0)203 668 6941 PRODUCTION & DESIGN JJ Jordan jj@31media.co.uk

EDITORIAL & ADVERTISING ENQUIRIES 31 Media Ltd, 41-42 Daisy Business Park, 19-35 Sylvan Grove, London, SE15 1PD Tel: +44 (0) 870 863 6930 Email: info@31media.co.uk Web: www.testmagazine.co.uk PRINTED BY Pensord, Tram Road, Pontllanfraith, Blackwood, NP12 2YA T H I R T YO N E

PAGE 5

THOUGHT LEADERSHIP SIVA GANESAN VICE PRESIDENT AND GLOBAL HEAD, ASSURANCE SERVICES, TATA CONSULTANCY SERVICES (TCS)

GETTING IT FIRST TIME RIGHT

We are living in a digital deluge – now, default is digital.

Technology is advancing at a never-before-imaginable pace. To maintain advantage in the fiercely competitive environment, companies not only need to deliver products that can boast of technological excellence, but also accelerate the delivery of such products at optimised cost. Given this scenario, the advantages of creating a product that is ‘first-time-right’ are undeniable.

PAGE 6

JULY 2015 | www.testmagazine.co.uk

THOUGHT LEADERSHIP

A

first-time-right product reduces cycle time and cost. Faster delivery with the assurance of product quality provides superior consumer experience and helps businesses build credibility. By making the shift from reactive to proactive, businesses can achieve predictive supremacy.

However, the move towards acceleration in the era of continuous delivery has its risks – the risks of overlooking mistakes – mistakes that could have catastrophic consequences such as loss of credibility, severe damage to the brand, financial penalties and even lawsuits.

Therefore, it is crucial for assurance to not only interlock with all the nodes of the software development lifecycle, but to also understand the industry needs, market drivers and core business processes. But it is not as easy as it sounds. There are several challenges – evolving business requirements, changing engineering needs and designs, iterative code builds, incremental releases, agile methods, distributed teams, disaggregated architectures, and finally, ‘yesterday’ deadlines – all of this, amidst the rapidly transforming market realities.

‘Culture eats strategy for breakfast’ as they say. As organisations and companies enter the ‘default is digital’ era, it will mark a critical time for directors and managers to set out and promote a corporate culture befitting the times. Policies to ensure quality assurance and a first-time-right mentality must come from the top down, or else chaos could ensue. Imagine high-risk undertakings, such as flying and landing an airplane, or semiconductor chips that control amusement park rides. You may not get a second chance to make it right. In the airlines sector, pilots undertake continuous training, and there is a series of measures in place to make sure the whole flight ‘lifecycle’ (take-off, flight and landing) goes smoothly. The importance of getting it first time right underpins the aviation training and management culture, lending credence to the idea that to prevent a failure, it needs to be constantly expected and meticulously prepared for.

Rather than acting as the 'last line of defence', assurance plays a proactive role

PREVENTION IS BETTER THAN CURE The entire process can be envisaged as a supply chain, assurance has to be engineered as the protective umbrella enveloping the supply chain. This holistic product and solution development approach will ensure error-free products that conform to requirements as they progress to the production-ready stage. In this approach, rather than acting as the 'last line of defence', assurance plays a proactive role. Assurance provides continuous visibility and feedback on the quality, accuracy and conformity of not just the business requirements, product and solution design, but also the development process. This paradigm shift requires a phased approach – a move from the traditional design-develop-test approach to concurrent testing integrated with all phases of the lifecycle. With a focus on business integration as well.

When inputs and feedback are conveyed early, errors can be detected and fixed early in the process, and not during production, where they are the most expensive to fix. Also, continuous assurance can shorten feedback loops, ultimately enabling businesses to accelerate market reach. This can be further enhanced if the assurance team are also domain experts.

In the context of manual, exploratory and regression testing methods, which take longer to execute, the right combination of automation for the unit testing of individual functions and end-to-end acceptance testing of the system ensures rapid feedback. Automation also enables early detection of any deviance of the product from its designated course, ensuring that the final product is devoid of defects, and compliant with business requirements. Simply put, accelerated and automated assurance acts as a defensive shield over the complete lifecycle of the product, assimilating any requirement changes or other challenges that occur through completion, and accelerating the product’s go to market.

JULY 2015 | www.testmagazine.co.uk

CHANGING THE CULTURAL MINDSET

Similarly, in the entertainment industry, intense quality assurance is the norm during the entire value chain, from chip fabrication to equipment construction and all the way to safety protocols during the rides. There is a culture of no-shortcuts as a tiny glitch or non-adherence to protocol can have great impact on business reputation.

Of course, most everyday work rarely involves life or death risk. However, when you’re faced with a new generation of users and consumers who expect immediate delivery, it could be tempting to strive to meet demand by pushing out content at record (or reckless) pace. This is where clear guidelines from the top down are needed to make sure costly mistakes are avoided, and quality assurance is preserved.

DEFAULT IS DIGITAL The optimal path for such acceleration (and excellence!), and continuous high quality delivery, can be charted out by determining the optimal interlock of assurance and software development lifecycle, while keeping in mind constraints and challenges. An assurance specialist must be involved from the outset, engage developer teams, make optimum use of automation tools and assure the accuracy of the product at every phase. The assurance specialist must also develop domain expertise and track business trends. With accelerated business-centric assurance, QA folks can become first-time-right facilitators for flawless products, rapid go to market, and customer delight. ✦

PAGE 7

NEWS WHAT APPLE’S CARPLAY MEANS FOR THE FUTURE OF CONNECTED CARS As Apple’s CarPlay software takes the stage this summer, being introduced in Hyundai and Chevrolet models, the automotive industry is taking note and wondering what’s ahead for connected cars in the future.

Intended to streamline the jumble of apps from smartphones that play on some dashboards into a more organised – and less distracting – arrangement, CarPlay makes used of iPhone-based programs, including the familiar maps, messages and music, into a single interface. Users should recognise familiar Siri voice commands and touch controls.

However, the software cannot control usual car functions, such as checking the engine status or even switching FM radio stations. Drivers will have to switch out of CarPlay in order to access these functions. This has led to some expressing concerns, as switching between different sets of controls – digital and physical – could be distracting to drivers.

Earlier this summer, at its developers’ conference, Apple set out a plan whereby instead of technology companies creating apps for cars, automakers should write apps for the technology company’s software.

This would mean that automakers, such as Ford, would write an app to measure engine status, which will run under CarPlay. Along with essentially turning automakers into Apple developers (or Android Auto developers), it is raising issues about safety and privacy.

Addressing the privacy issue, Apple has said it will not collect or use tracking information from the proposed auto company apps.

However, experts have warned that if automakers were to begin

TRICENTIS AND TASKTOP IN NEW OEM PARTNERSHIP Signalling an expansion of their relationship, Tricentis and Tasktop have announced an OEM partnership.

Tricentis will offer an OEM solution called Tosca Connect, which is powered by Tasktop Sync, Tasktop’s SLI solution, and allows customers to connect automated testing with existing development processes to lower business risk and accelerate product development.

JULY 2015 | www.testmagazine.co.uk

developing apps for CarPlay, it could open a new avenue for hackers.

“There are absolutely security issues,” said Robert Clyde, International Vice President at Isaca, a nonprofit organisation focused on information technology security. “Could a bug in iOS or in Android give someone access to critical systems?”

A software bug, if discovered, would cause many practical issues.

As extensive testing is carried out and updates are developed, would drivers need to return to the dealership for a software update?

There’s also the added factor that most automakers have already invested considerable time and effort in developing their own connected car systems.

For example, Ford is introducing the latest version, Sync 3, of its infotainment system this summer.

Sync 3 already supports a multitude of smartphone apps, taking the more traditional approach of having companies develop apps for its system, rather than the other way around. Ford also promotes SmartDeviceLink to make the process easier for developers, as this gives app creators a way to write a program once that could then run in any compatible car. Toyota has said it is studying SmartDeviceLink as well. ✦

The new solution removes barriers to test automation adoption and enables standardisation of Tosca Testsuite as a seamlessly integrated component of the software development tool stack.

Customers can extend their benefits of test automation by also automating the collaboration between test and other disciplines such as requirements management and defect management. This removes costly and error-prone manual processes while enabling the short test sprints required by agile teams. Tasktop Sync integrates Tosca with Rally, Atlassian JIRA, HP QC,

TOYOTA AND LAND ROVER SEE CAR RECALLS DUE TO UNRELATED SOFTWARE BUGS Both Toyota and Land Rover announced separately this July that they were recalling vehicles due to software bugs.

Land Rover is recalling more than 65,000 cars to fix a software bug that can ‘unlatch’ the vehicles' doors. According to the company, drivers would get no dashboard warning that the doors of their car had been unlocked. The software glitch affects Range Rover and Range Rover Sport vehicles sold between 2013 and now. In a statement, Land Rover said no accidents or injuries were reported to have occurred as a result of the bug.

Meanwhile, Toyota is recalling approximately 625,000 hybrid cars around

the world. The firm said that a fault in the software could lead it to shut down the hybrid system while the car is being driven.

Models affected by the recall include Prius V minivans, called Prius Alpa and Prius+ in some markets, that were built between May 2010 and November 2014.

In the current recall, Toyota said that Japan accounted for 340,000 of the cars, while 160,000 were in Europe and 120,000 were in North America.

The car giant said that there had been no reports of crashes or injuries due to the software glitch.✦

HP ALM, Jama, IBM RTC, Microsoft TFS, IBM DOORS, IBM DOORS Next Gen/ Rational Requirements Composer, ServiceNow, VersionOne and others, resulting in strong collaboration across software development and delivery teams.

Todd DeLaughter, Automic’s CEO, added that the DevOps space was changing rapidly, but that automated testing remained essential and that partnering with Tricentis ensured Automic’s testing remained robust and thorough. ✦

PAGE 9

NEWS UK ENERGY INDUSTRY CRITICISES GOVERNMENT PROPOSALS FOR SMART METER TESTING As we move towards the UK government’s goal of all households having smart meters installed by 2012, the recently proposed testing methods have come under fire from the main energy suppliers, including British Gas, E.ON, Utilita, SSE and nPower.

The recent report by the Data Communications Company, the organisation created by the government for the purpose of deploying the smart meters, was criticised for being out of touch with industry standards.

The energy companies reject suggestions such as paying for a network connection six months in advance for testing, money that project managers said is wasted if they are not planning to take part in pre UIT testing.

utility E.ON highlighted concerns over clarity. “We do not agree with a number of the proposals in the Interface Test Approach document. In addition there are a number of proposals with a lack of clarity and detail to enable a firm statement of agreement to be made."

The government’s proposed testing approach does not include test stubs, a move many have labelled ‘high risk.’

“We are particularly concerned that the overall testing timeline has been condensed as far as possible with the overlap of user acceptance testing and interface testing. The further overlapping of solution testing with interface testing and user acceptance testing, we believe adds too much risk,” E.On said.

“We are also concerned at the proposals to introduce real devices into the user entry process tests part way through testing. From our foundation experience we are concerned that introducing different devices part way through a test phase could cause undue delays.”

Also, in a letter to the Department for Energy and Climate Change,

AUSTRALIAN GOVERNMENT FINANCES AUTISTIC PROJECT AT HEWLETT-PACKARD Hewlett-Packard’s latest recruitment drive, a project in which people on the autistic spectrum will be employed as software testers, is being funded by the Australian government’s Department for Human Services.

PAGE 10

The project forms part of a co-operation known as the Dandelion Project, which launched in at the beginning of 2015, and sees the Danish Specialist People Foundation (SPF) select and assess autistic Australians who are then hired by Hewlett-Packard to work as software testers. According to local news reports, the trial programme is under way in Adelaide and more than

Utility EDF Energy also weighed in: “We do not agree with the current approach to potentially compress and parallel run testing phases. In any testing environment such an approach is not good industry practice and introduces a high risk. Should the risk become issues, all concerned, including consumers, pay in terms of time and financial costs.

“The use of test stubs will always be a contentious issue, but we understand the position the DCC is in with circumstances beyond its control. However some mitigation towards the test stub approach would be for the DCC to provide its evidence that test stubs will deliver to the desired levels of assurance as required in the Smart Energy Code [the guidelines for the smart meter initiative] as soon as possible, rather than waiting until the decision to use stubs has to be made.”

The latest set of proposals from the government were published in July 2015.

Energy companies will have until 1 September to raise any concerns with the guidelines, before they become law in six months’ time. ✦

10 people have been employed since January. SPF is an organisation that provides staff, most of whom have a diagnosis on the autism spectrum, to work as consultants in sectors with technically orientated tasks and jobs, such as data management, software testing and quality control.

A similar project with Microsoft in the US was announced in April. ✦

JULY 2015 | www.testmagazine.co.uk

NEWS with 150 startup accelerators in 47 countries to deploy this programme globally. India is recognised as a key country, thanks to its vibrant startup culture. In addition to Azure services, startups here will receive the full suite of development and test software and tools such as Visual Studio, Windows and Office.

MICROSOFT CONTINUES TO SUPPORT INDIAN STARTUPS As part of Microsoft’s recently launched BizSpark Plus program, the company will offer free Azure Cloud services to qualified startups (worth US$120,000 per year). The startups

MIT RESEARCHERS PRESENT AUTOMATIC BUG REPAIR At the Association for Computing Machinery’s Programming Language Design and Implementation conference, MIT researchers presented a new system that repairs software bugs by automatically importing functionality from other, more secure applications.

Without requiring access to the source code of applications from whose functionality it is borrowing, MIT’s system, CodePhage, analyses the applications’ execution and characterises the types of security checks they perform. As a consequence, it can import checks from applications written in programming languages other than the one in which the program it is repairing was written.

Once it has imported code into a vulnerable application, CodePhage can provide a further layer of analysis that guarantees that the bug has been repaired.

In the MIT terminology, CodePhage works by repairing the ‘recipient’ program by borrowing functionality

PAGE 12

can use US$10,000 worth of credits monthly, to invest in Microsoft services to power their business. In addition to open-source friendly Azure, Microsoft is making available free software, developer tools, and technical support to help startups be successful. Microsoft is partnering

“Startups need all the help they can to get access to the right tools, technology and guidance,” Rajinish Menon, Director, Microsoft Ventures India, said. “At Microsoft, we are committed to supporting these startups and through the BizSpark Plus programme we want to support India’s upcoming entrepreneurs and the government’s Digital India vision. Azure provides the productivity, security, analytics and scalability that startups need to quickly build and grow their businesses.”

To qualify for the BizSpark Plus programme, startups must be privately held; less than 5 years old; have a less than US$1million revenue annually and be member of a select Accelerator or VC firm. In India, the programme has limited seats for other startups as well. ✦

from a ‘donor’ program. To begin its analysis, CodePhage requires two sample inputs: one that causes the recipient to crash and one that doesn’t. Earlier this year, the same group announced DIODE, a buglocating program that generates crash-inducing inputs automatically.

The researchers tested CodePhage on seven common open-source programs in which DIODE had found bugs, importing repairs from between two and four donors for each. In all instances, CodePhage was able to patch up the vulnerable code, and it generally took between two and 10 minutes per repair.

As the researchers explain, in modern commercial software, security checks can take up 80% of the code – or even more. One of their hopes is that future versions of CodePhage could drastically reduce the time that software developers spend on grunt work, by automating those checks’ insertion.

“The longerterm vision is that you never have

to write a piece of code that somebody else has written before,” Martin Rinard, Professor of Computer Science and Engineering at MIT said. “The system finds that piece of code and automatically puts it together with whatever pieces of code you need to make your program work.” ✦

JULY 2015 | www.testmagazine.co.uk

OPINION DANIEL BEAZER SENIOR CONSULTING ANALYST PEER 1 HOSTING

WHY IS THE IT SECTOR NOT ON THE GOVERNMENT’S AGENDA? The IT sector seems to have disappeared completely off the political agenda, Daniel Beazer, Senior Consulting Analyst, Peer 1 Hosting, laments the missed opportunity.

PAGE 14

I

f you compare the speech made around the pre-election budget back in March to what was announced recently in the July budget, IT seems to have disappeared completely off the political agenda. This is despite the fact that IT has played a central role in the UK’s economic recovery. But, where are the initiatives that reflect this?

JULY 2015 | www.testmagazine.co.uk

OPINION

Currently, it’s just too easy for UK talent to leave for Silicon Valley or the US in general

A quick way to demonstrate this de-prioritisation is to CTRL+F the Chancellor’s July Budget speech with the word ‘technology’; you’ll see it is mentioned once - in the context of vehicle exhausts. That’s better than the internet though, there was no mention of it. Nothing!

INCREASING INTERNET SPEED AND ACCESS In March there were a few initiatives that gave the industry some hope, including the government’s announcement that it will invest £40 million in developing Internet of Things technologies through large-scale demonstrator programmes, a business incubator space and a research centre. In particular, the funding will focus on projects in the healthcare, social care and smart city industries.

It also announced it would provide £7.4 million in funding to support libraries in England so that they can provide internet access and WiFi to the community.

However, neither of those projects were mentioned in the July Budget speech.

INVESTMENT IN IT INFRASTRUCTURE Going back to the pre-election Budget, in relation to technology the government made a promise to be “much more ambitious” in its use, and to provide funding to develop a financial technology incubator in Leeds.

Again, there was no mention of these projects in July’s budget.

There was also quite a bit of discussion in March around infrastructure spending, which is encouraging.

One of the goals was to be “bold in delivering infrastructure”. But the infrastructure projects that were listed are decidedly old school. For example, the government has promised to improve the M4 in Wales and the Great Western Line. This is great and much needed, but what about plans to bring 100 Mpbs broadband to Britain?

Here is the Chancellor only a few months ago on this subject: “To further enhance the UK’s digital communications infrastructure, the government is announcing a new ambition that ultrafast broadband of at least 100 Megabits per second (Mbps) should be available to nearly all UK premises.”

JULY 2015 | www.testmagazine.co.uk

I like the use of the word ‘ambition’ there, rather than ‘commitment’ or ‘promise’, what exactly does that mean? Once again, there was no mention of the highspeed broadband plan in the most recent budget.

PROVIDING THE RIGHT LEVEL OF SUPPORT Despite the lack of visibility in the most recent budget announcement, the health of the IT industry has never been better, and it really doesn’t need another subsidy or tax break. However, further support is required in order to continue the impressive growth of the last couple of years.

This includes continuing funding for the creation of digital skills, from young people to those who are already within the IT industry today. The government must also address brain drain. Currently, it’s just too easy for UK talent to leave for Silicon Valley or the US in general. However, tech has always been international in nature and there is a plus side to sharing the intellectual wealth with other countries. One of the founding partners of Sequoia is Welsh, and it’s great to see UK talent influencing a number of US tech powerhouses. A way to solve brain drain might be to create some proper competition in banking so salaries come down, making jobs in tech and startups more attractive for skilled workers.

In addition, better infrastructure and rates relief for smaller IT companies will only help to further the tech economy we are slowly building. The government has previously promised that 25% of all government IT contracts will be awarded to SMEs by 2015, but the National Audit Office has direct awards currently sitting at around 10%. It would great if the government could focus on hitting this target.

MAKING AN IMPACT The theme is clear: lots of promises before the election, which mysteriously disappear afterwards. Nothing new in that, but what is deflating is the missed opportunity.

The first budget of a parliament is the one chance a new government has to do something radical, because anything radical will inevitably be unpopular somewhere, and will need at least four years for the country to get used to it. ✦

PAGE 15

THIRD DIGITAL AGE HELEN BYRNE CIO TESTING SERVICES DIRECT LINE

More and more enterprises will need to adopt tactics normally associated with startups (e.g. continuous integration, deployment/DevOps, behaviour-driven testing) in order to handle the need to support ever-changing digital fields

CULTURE MINDSET CHANGES NEEDED TO ATTRACT THE NEW GENERATION OF SOFTWARE TESTERS

Taken from her keynote speech at the National Software Testing Conference in May, Helen Byrne, CIO Testing Services, Direct Line, addresses the need for a culture of change to ensure continued successful operations in the future.

PAGE 16

JULY 2015 | www.testmagazine.co.uk

THIRD DIGITAL AGE

T

oday’s leaders must instil a culture of change in order to remain relevant in the third Digital Age. It is clear that the increasing pace of the online world is moving a lot faster than most organisations can keep up with and there needs to be a step change in behaviours in order to adapt to the changing landscape.

LIFE AT A SLOWER PACE But first, let’s travel back to the mid-1990s. I had just started on my career path and was happy to be working in a city where I felt that anything was possible.

Although it didn’t feel like it at the time, life moved at a slower pace. News reached people via the TV, radio and newspapers. Newspapers that you actually had to pay for (remember when the Evening Standard cost 40p?).

Agile didn’t mean anything to IT workers, only yoga teachers and dog trainers.

‘Testing’ was a fairly new career option and being completely honest we made it up as we went along. We followed a kind of waterfall methodology although we didn’t recognise it as such back then. Testers were chosen because of their business experience of the application under test rather than any formal testing qualifications. Scripts and defects were logged on excel spreadsheets and were managed manually.

Computers were something to be afraid of rather than revered – remember when the world started to get worked up about ‘The Millennium Bug’?

This was a turning point where things started to get more serious – we invested in tools. We went through ISEB – following the successful answering of 25 out of 40 multiple choice questions you could come out a certified professional tester!

One element of the course work which will always stay with me was the section that confirmed what characteristics you need to make a good tester – it wasn’t rocket science – persistence in trying to ‘break’ the application, attention to detail, and a pessimistic attitude (that things will not work).

I also remember a lot of ‘down time.’ Waiting for code, waiting for environments, waiting for defect fixes, waiting for requirements. The bits in between the actual testing itself could be quite boring.

THE TURNING POINT Then 1998 marked a turning point in terms of cultural shift. The internet was really starting to gather pace. The Monica Lewinsky scandal was the first real news item to break on the internet and it changed the way we devour news and interact with it. Whilst we may have embraced these changes on a social level, I don’t think most workplaces were ready for this cultural shift.

Most of my peers, myself included, could be categorised as Generation Xers. Generation X was born during the

JULY 2015 | www.testmagazine.co.uk

years 1965 to 1980. This generation has been defined by the following characteristics: •

Sceptical of authority.

•

Tends to not respect hierarchy, status or title.

•

Seeks work-life balance and prefers an informal, fun workplace.

Typical characteristics of the Generation X workplace would focus on self-reliance, individual projects and minimal supervision. This generation is not interested in spending hours in meetings, instead the Xer demands high productivity and prefers to complete tasks as quickly as possible to free up more personal time.

On face value, it sounds like this generation would suit itself quite well with more agile methodologies – there was only one problem. Most of my generation, myself included, had progressed up the career ladder and were now in management roles.

Typically as managers we wanted people to work for us who respected authority, hierarchy, status and title!

THE NEXT GENERATIONS We found these hard working employees; we just had to look a bit further offshore. Post 2008 and the recession, businesses needed to rationalise cost and create efficiencies, and with the world now seeming a lot more accessible and, crucially, cheaper. The offshore support model became the norm and the cultural traits, particularly of the Indian employees worked well within the waterfall environment. The industry found offshore workers to be stable, loyal, hardworking and respectful of management decisions.

However, the Indian culture values indirect communication and workers do their best to avoid personal confrontation. They are sensitive to criticism and to losing face in public.

This is not as fitting for agile projects. However, there is a new global generation of potential testers, who are exhibiting differing characteristics simply because they have grown up with a steady stream of information that is independent of culture.

They’ve grown up online.

Also known as the Millennials, Generation Y was born between 1981 and 2006. Now well-established within organisations, this generation is the most diverse of all the generational groups – one in three is a minority.

Those in Generation Y are optimistic, confident, civic-minded and fully committed to moral and ethical principles.

This generation expects full communication, speedy decision-making and requires information to be available immediately.

Characteristics of a Generation Y workplace include constant communication, multitasking and a recognition that work is a means to an end. ➤

PAGE 17

THIRD DIGITAL AGE The Generation Y QA sits well in an agile environment because it is a generation that has grown up as part of the digital revolution.

While this working practice should promote collaboration and a more open way of working, it requires like-minded people.

Everyone now is an IT geek and this generation are used to multi-platform, multi-device, multi-application and multi-interface products that are constantly evolving and adapting to the marketplace.

A CULTURE OF SHARING

A PEOPLE PROBLEM So why is it that over 20% of IT projects still fail on time, cost, quality or all three? My belief is that this is due to the largely understandable immaturity of the professional services organisations due to the scale of agile adoption seen so far – it is a people problem.

More and more enterprises will need to adopt tactics normally associated with start-ups (e.g. continuous integration, deployment/DevOps, behaviour-driven testing) in order to handle the need to support ever-changing digital fields such as mobile application development, web analytics, and social media.

Agile is all about the people: Agile only works if you have the right people, skills and resources with the right understanding, and the right mindset. At the end of the day it is a people problem – when people understand it, it works; if they don't understand it, it fails.

If there are a lot of people with no experience of agile there will be problems, as it is very easy for agile to be used without discipline and I’m sure I am not the only one to experience first hand the problems that can bring.

Investing in training and cross-skilling/ knowledge transfer is critical.

Agile requires strong project management, which can be surprisingly hard to find.

Strong leadership is a prerequisite, although not necessarily leadership in the traditional authoritative sense.

CREATION OF CROSS-FUNCTIONAL TEAMS IN THE SAME LOCATION AND TIME ZONE Research suggests that agile works best in the same location and the same time zone. Many organisations feel that it is essential that developers and business users work side by side with constant communication to move forward, step by step, with software releases.

Earlier this year I read an alarming statistic, only 13% of the UK workforce are fully engaged in their job.

I believe there is direct correlation between this statistic and the need to break down the outdated hierarchical structures that have been in place since the 1990s.

This has been in force for decades since it was believed that the ability to make decisions came from the top – but in the digital age this is no longer the case.

We need engagement, innovation and speed more than we need efficiency, and the decision-making needs to sit at all levels. People at the edges need to take responsibility rather than waiting for decision-making to go up the food chain for someone else to make a plan of action and they need to be able to have the honest non-hierarchical conversations that facilitate that decision-making.

Leaders need to take that big step and trust their subordinates to make the right decisions in the same way that parents eventually learn to trust their teenager to go out into the world and make their own decisions.

An example of this working really well can be found in the American food chain Red Robin. Some 87% of its workforce are millenials. The company brought out a burger called the ‘Pig out style burger’ and then asked for feedback via social media.

The responses that came back were not positive so the managers then asked for suggestions on their own internal network on how to respond. As a result, changes were implemented into their kitchens within 30 days of receiving the feedback rather than 6 - 12 months, which is normal in that industry.

Managers tend to abhor this style of working as it means senior managers at executive level are talking to the grassroots staff, and they feel loss of control. We need to get the managers to understand that they are a crucial part of solution.

It is advisable to dip your toe in the water with small projects. It takes time to build up the capabilities, knowledge, and experience needed for these ways of working to be successful in large programmes which often have a high degree of complexity.

They are the ones that can create that culture of sharing if they can buy into the notion that being influential is not about how much you know or what you job title is, it is about the size and quality of your network – and that when you share information you develop a relationship and engage with people on a level that transcends hierarchies. In its ultimate form a job title itself becomes irrelevant.

Agile is not a new concept, but it is fair to say that the coming of the third platform and the digital explosion of web-based apps promoting direct interaction with business users has really focused the software development world towards faster ways of working without increasing quality, risk, and cost.

If on a small scale, organisations start to see change driven through networks rather that CEO/CIO directives then I believe this culture can be created and harnessed to move us forward. I believe this is the only way we can truly harness the power of the new millenials and give us the skills needed to operate in the new digital era. ✦

PAGE 18

JULY 2015 | www.testmagazine.co.uk

COVER STORY RICHARD LOWE ACCOUNT DIRECTOR SQS

TESTING TIMES FOR THE BANKING SECTOR In January this year, Sainsbury’s Bank suffered a widespread system failure, leaving customers unable to use their credit cards. They were quick to voice their anger on Twitter and the story was picked up by the mainstream press.

T

he issue at Sainsbury’s Bank came just two months after RBS was fined some £56 million for a similar IT failure in 2012 that saw 6.5 million customers of Natwest, RBS and Ulster Bank unable to make payments for as long as three weeks. The fault, related to a botched software upgrade, prevented customers from using online banking, properly checking their account details, making mortgage payments and receiving salaries.

The failures are a worrying sign of things to come, and prove that the growing pressure on complex IT systems can and will affect consumers. Banks desperately want to avoid this at a time when customer confidence is already at a low point, and industry regulation is increasing.

It is easy to point the finger at legacy systems that have been adapted and patched up over decades of mergers and acquisitions (M&A). But is it time for the industry to start looking at the problem from a new angle? Just because something is old it doesn’t mean it can’t be effective; what is required is a deeper understanding of its capabilities and performance under differing levels of load, stress and circumstance. By having an integrated quality assurance approach, businesses can weigh up the benefits of implementing new systems over functionally rich ones that have been developed over many years of IT change. The gaps identified can then be assessed against the business process requirements in order to make the right decision to create a fit-for-purpose platform that benefits both the bank and its customers.

HOW TO PREVENT A REPETITION While the exact cause of the system failures at the two banks above is unknown, it is possible they could have been prevented by a more thorough and substantial quality assurance process. A change of culture and mind set is needed. Every individual team needs to tick its own boxes and only pass it on to the next when they

PAGE 20

JULY 2015 | www.testmagazine.co.uk

COVER STORY are satisfied the code is working correctly and secure. This should happen throughout the software development lifecycle (SDLC) all the way through to completion. There seems to be little end-to-end ownership from conception to production across not just the financial services industry but throughout the wider business society, yet the problem is often accentuated in businesses that have seen substantial M&A activity.

The business objective of speed-to-market is often incorrectly prioritised to the detriment of the IT department’s ability to undertake the appropriate due diligence and testing. A stringent quality regime needs to be at the heart of this process, to ensure that any system changes happen for the right reasons and with little or no risk to the business.

Quality gates need to be tightened, implemented and adhered to, over and above the pressures of meeting deadlines to the detriment of a product’s reliability. This also requires those quality gates to be drawn in accordance with the needs of the business (i.e. with the end goal in mind), rather than adopting an ‘ivory tower’ approach, which opens them to being bypassed for a greater business imperative. All of this can be best achieved by aligning quality assurance with the needs of the business.

TESTING TIMES

Testing should happen from the very start of any new project and by sector specific testers – beginning with static testing of business requirements. Not only will the testers have an impact on the overall build by understanding thoroughly what needs to be tested, but any faults found at this stage will be much more easily fixed, resulting in cost savings and avoiding unnecessary rework. This is known in the industry as ‘shift left’. The practice requires accountability and increased communication with all stakeholders in the process to ensure the project is staying in-scope. In our experience of testing in the financial services market, the typical cost savings from the shift left approach are in the 20% - 25% range.

Testing needs to begin before development, right at the inception stage of a product, and requires board-level buy-in from the start. However, this will often ignore the risks posed by integration with other systems. A thorough change impact assessment should be part of any early testing effort, to understand the risks to other legacy systems, as well as the risks created by the integration with those other systems. For example, a core online banking solution may have been thoroughly tested in isolation, but unless the risks of connecting it to the institution’s payment gateways are considered, the business processes that the solution serves are still at risk.

THE PROBLEMS WITH LEGACY SYSTEMS

Where testing verifies that specific requirements are met, quality assurance seeks to implement a fit-for-purpose solution, which involves ensuring that the requirements are correct to begin with, and working with the project team to build a solution that will meet its requirements.

As quality assurance is part of the implementation and testing phase it needs to be independent of the testing performed throughout the development phase. Ideally, it will be carried out by different people, with specific skillsets. In a way, quality assurance aims to pre-empt testing, and testing aims to prove whether quality assurance was effective.

THE RISING COST OF COMPLIANCE AND RISK MITIGATION Higher standards are now required. Unless there is a holistic quality assurance approach, then the risk of failure will only increase. With the cost of non-compliance in the form of regulatory fines running into the millions, and the cost of having to undertake corrective measures potentially millions more, effective quality assurance has to be a focus for all financial services organisations. The intangible detrimental effects that IT failures can have on consumer confidence also need to be considered.

With the increase in internet-facing functionality and mobility features, legacy systems require frequent code changes and upgrades to implement security measures and render them compliant with regulatory requirements. However, the legacy nature of the application can make these changes complicated to implement and test when compared to a newer platform. Legacy kit needs to receive continual security updates from the vendor, with these then needing to be applied in a controlled manner, including regression testing, by the organisation to ensure no breaches occur. The issue tends to be the downtime often required for security updates and whether the business can accommodate this, a difficulty when access to banking systems is required 24/7 via a plethora of devices.

This brings us to the danger of having an unfounded loyalty to legacy IT systems. This inbuilt fear of change is having a huge impact upon businesses’ ability to keep up with the pace of IT change. Technology The issue is now considered to be the backbone of a successful business strategy in today’s tends to be fast-paced, competitive environments.

The banking industry should guard against the downtime often The IT department may in fact be the tendency to reduce an objective or a the bottleneck for change when it requirement to a list of checkboxes and bear required for security comes to keeping up with consumer the original objective of the regulation in and market demand. Whilst we updates and whether the mind. This is best carried out with a testing don’t advocate ripping out legacy that not only has the skills to assess a business can accommodate team systems and simply replacing them system for compliance, but also the domain with a shinier modern alternative this, a difficulty when access knowledge required to understand the without consideration, it is important regulation and its impact on the business. to banking systems is to truly understand older IT systems While it is impossible to get away from the to provide an accurate answer as to required 24/7 via a issues that banks are currently facing, it is whether there is a genuine need to to identify one of the root causes modernise. plethora of devices. forpossible these problems as a lack of understanding For years now, the IT department has when it comes to legacy IT systems. Customers had a bad reputation when it comes to start to feel the pain when organisations work outside collaborating with the rest of the business, and the the parameters of their systems and start to add extra recent high-profile software failures at Sainsbury’s Bank and RBS components without investigating the potential pitfalls in have done little to alter this opinion. The reality is that the sheer performance. Expectations of legacy IT systems need to be pace of IT development has meant that they are struggling managed as they are unlikely to be able to cope with the new to keep up, and resort to upgrading legacy systems in a bid customer demands and new product diversification that is to adapt and adhere to budget and time constraints. This needed for banks to remain competitive in a highly regulated hand-to-mouth approach means that cracks start to appear. market. ✦

JULY 2015 | www.testmagazine.co.uk

PAGE 21

SMART ???

TVS DANIEL BOX TECHNICAL CLIENT SERVICES MANAGER BUGFINDERS

BIG CHANGES AHEAD FOR THE SMALL SCREEN Daniel Box, Technical Client Services Manager, BugFinders, discusses how QA departments can prepare for the rise of smart TVs. Software testers will need to test apps on TVs made by many different manufacturers, as well as on every big platform in the smart TV space

PAGE 22

JULY 2015 | www.testmagazine.co.uk

SMART TVS

B

ack in June, Apple announced that it would be giving developers the same number of test slots for Apple TV as for the iPhone and iPad. For some online commentators, this was an indication that Apple is beginning to place as much emphasis on smart TV apps as on mobile apps, having long allowed developer accounts to be linked to the new platform.

suffers. As many different devices as possible need to be tested on, often in a matter of days. As a result, testing becomes rushed and bugs on certain devices are missed.

There is a precedent when it comes to developers being encouraged to work on TV apps. At Google’s I/O developer conference last year, Android Engineering Director Dave Burke described the smart TV space as “not dissimilar to the mobile space in 2006.”

It’s also worth noting that storage is another challenge posed to QA departments by the rise of smart TVs. Mobile devices are easy to store in a test lab, even if they are expensive to purchase for testing purposes. Storing Sony, Sharp and TP Vision TVs, however, is both expensive and impractical. This problem will only get worse as more connected TVs are brought to market.

DEVELOPERS ACTIVELY ENCOURAGED

So how do QA departments incorporate smart TV testing in a way that doesn’t harm the quality of TV and mobile apps?

This implies that smart TVs are now where mobile devices were in 2007, and moving ever closer to becoming as popular with internet users as smartphones and tablets.

No wonder that Google, too, is actively encouraging developers to start developing apps for Android TV, its own smart TV platform.

The prospect of TV imminently becoming a popular new platform for apps is good news for companies with customer-facing software. It suggests the medium will be liberated from the expensive and limited format of traditional TV advertising, providing greater opportunity for customer engagement and a brand new platform for E-commerce.

ENTER THE QA DEPARTMENTS However, the rise of smart TV may also create a new headache for QA departments. This is because new apps will need to function correctly not only on a broad range of mobile devices, but on multiple smart TV platforms and models too.

Android TV, for example, runs on TVs manufactured by Sony, Sharp and TP Vision. It is also the case that Google and Apple aren’t the only players in the game – Amazon Fire TV is also a competitor, for instance. This means software testers will need to test apps on TVs made by many different manufacturers, as well as on every big platform in the smart TV space.

CHALLENGES For many QA departments, adding another slew of platforms and devices to test cycles will prove a significant challenge. Test teams are already struggling to cope with testing apps on an adequate number of smartphones and tablets. This is because new mobile devices are being taken to market all the time, while consumers are continuing to use older devices and operating systems.

The net effect of this on-going expansion of the mobile device market is often that the thoroughness of testing

JULY 2015 | www.testmagazine.co.uk

Therefore, there is a real danger that adding smart TVs to device lists will put even more pressure on test teams that are already stretched. This could mean that overall app quality suffers, on both TV and mobile.

SOLUTIONS The way I see it, there are several options. One is always going to be difficult to get sign-off for: hiring more testers to cope with the extra capacity. Another option is to start using test automation, or increase the level that’s already in use. This is a good way to speed up testing, although it does often lack the rigour of manual testing, which can result in missed bugs.

A third option is to start working with a crowdsourced testing provider. Crowdsourcing companies maintain large communities of testers, often numbering thousands.

This enables them to assist QA departments by helping with device coverage and carrying out specific types of testing.

For example, crowdsourced testing partners could be used to perform TV app testing, while an in-house test team concentrates on testing mobile apps. Or wide-ranging exploratory testing on a range of TV and mobile platforms could be carried out by a crowdsourcing company, while very specific scripted testing could be performed internally.

Crowdsourcing also removes the need for businesses to buy and store expensive new connected TVs. Crowdsourced testers own the devices and platforms they test on, giving QA departments access to testing on the latest platforms without the expense of purchasing them.

THE FUTURE IS NOW Google and Apple’s encouragement of app development for smart TVs shows how important the platform is likely to become very soon. The businesses that begin releasing TV apps quickly will be the ones to gain a competitive advantage. This means that QA departments need to start preparing for the extra testing this will entail, whatever strategy they may end up choosing. ✦

PAGE 23

SOFTWARE LOCALISATION

SOFTWARE LOCALISATION: CHALLENGING, BUT COMES WITH PROMISE Quality assurance is a huge obstacle in the way of smooth software localisation. Ofer Shoshan, CEO, One Hour Translation, explains.

PAGE 24

JULY 2015 | www.testmagazine.co.uk

SOFTWARE LOCALISATION OFER SHOSHAN CEO ONE HOUR TRANSLATION

L

ocalising software is one of the simplest, surest and low cost ways to boost software sales around the globe. Common Sense Advisory, which surveyed 156 global product developers, concluded that most global software developers generate up to 50% of their total corporate revenue by spending a miniscule 1% to 2% of their development budgets on localisation.

Software localisation takes place in order to gain revenue in a specific country and is often triggered by an important customer that demands to localise the software.

Some products cannot penetrate specific foreign markets without localisation. Mobile device providers such as Samsung, for example, cannot bypass the necessity to offer Chinese language devices and documentation in order to penetrate China.

The third difficulty in localising English-language software results from the nature of the English language.

Translating the English content of the original software will cause an increase of 20 - 30% in the number of words in the localised software or application. As a result, changes must be made in the graphic design of the localised software.

How can a QA professional who checked a software program in English check The fourth challenge pertains to localisation into right-to-left (RTL) the program after it languages, such as Arabic, which has been localised to represent huge market potential for localised software. The support needed Chinese or even to

by RTL languages practically requires Spanish? the replacement of the entire software

CHALLENGES As a company which performs thousands of software localisation projects each year, One Hour Translation has been tackling and handling the multiple difficulties involved in software localisation on a daily basis. A few key challenges include: First, to present content in a foreign language, a software program needs to have suitable infrastructure. Some programs have more advanced infrastructures geared to these needs, while others do not even have the basic capability to present foreign language contents, and the necessary infrastructure has to be laid down.

For example, some accounting software programs should be physically installed on the computer, rather than being used as a cloud-based service. Many such programs have been written on old infrastructures that often lack multi-lingual capability. Localising them becomes questionable, since it might involve time consuming handling of both the development language and the development environment.

Second, software programs require adaptation to the specific locale. Localisation goes far beyond translation of the language.

The vendors must adapt the software to a comprehensive array of cultural and other contexts. Among other things, the process requires adjustments in the currency values and symbols, in the structure of the hours and dates and in various symbols. For example, the localised software has to ‘know’ whether the currency symbol should be written on the left side or on the right side of the currency value. Software is everywhere, even in our air conditioning systems and cars, and localisation should be ubiquitous as well.

JULY 2015 | www.testmagazine.co.uk

For example, car clocks in Japan should present the date in a different format compared to car clocks in the US.

interface, a process which consumes much time, effort and money.

QA-RELATED LOCALISATION The fifth challenge is perhaps the biggest obstacle to smooth software localisation: quality assurance. The usual QA staff employed by companies cannot really check a localised software. How can a QA professional who checked a software program in English check the program after it has been localised to Chinese or even to Spanish? The program has been developed in English. If its interface is migrated into Japanese or Chinese, the mere fact that there are no spaces between the words makes the English QA team's task impossible.

QA-related localisation sometimes results in long ‘localisation cycles’. The content has to be taken out from the software, localised and then integrated again back into the software and the QA. Such a cycle might be needed even if only one word changed in the software.

To sum up, localising a software program or application is a great global revenue enabler.

Nevertheless, in order to succeed in this process, software vendors must be able to understand and effectively tackle the sometimes surprising difficulties it presents. ✦

PAGE 25

TESTING ???

TOOLS NOEL WURST MANAGING EDITOR SKYTAP

TESTING TOOLS MAY FIND BUGS, BUT THEY DON’T IMPROVE QUALITY Noel Wurst at Skytap considers the questions developers and testers should ask about their tools.

PAGE 26

JULY 2015 | www.testmagazine.co.uk

TESTING TOOLS

I

have always rooted for software testers. Their job is difficult; but in software, from the smallest start-up to the enterprise – nobody’s job is easy. I’ve always rooted for testers because along with being responsible for making sure that users will a) actually enjoy using your software, and b) that when they do use it, it doesn’t expose their private data (or yours!) – testers still have to constantly justify the importance of their work. And oftentimes, even if their work is viewed as valuable, they’re not given adequate time to test, and everyone wonders why the last release had those bugs that made it into production.

While this can quickly devolve into a battle of semantics: what testers are saying here, and they’re correct, is that automation tools on their own do not move the needle on quality. In fact, if you’re writing ‘bad’ automation scripts, these tools can quickly do more harm than good. On the contrary, skilled automation experts can leverage quality tools to replace time-consuming and error-prone manual processes, thus leaving testers with more time to do what they do best – finding bugs with creativity and business domain expertise.

If your software testers are spending their days writing test automation scripts, guess what they’re not doing: Testing

When bugs are found in production, someone has the nerve to say, “it worked on my machine”, and work on the next release grinds to a halt as the scramble begins to recreate the bug so it can be eliminated or patched. Trust and relationships between departments start to erode – and guess what – the next release ships with the same lack of adequate testing, but there’s not as much of a rush to fix the new bugs because the customers have all left, or deleted your app because they can no longer risk it continuing to cost them their own customers.

These highly-skilled testers can then begin testing far sooner in the SDLC, and in continuous integration with smaller iterations of new code instead of in a single timeboxed instance. ➤

Unfortunately, breaking this cycle of poor quality software is not easy (it’s still software) but there are solutions that remove a large number of constraints and bottlenecks that give testers more time to test, and the ability to test earlier in the SDLC.

AIDING BUT NOT REPLACING Cloud-based testing tools offer a number of advantages, and some challenges, for companies struggling to keep up with the speed and quality at which business and consumerisation demand.

Many of these tools focus on test automation. With agile development’s continued popularity and the rise of DevOps, automation is certainly the goal for many organisations, though automation on its own is in no way a “silver bullet’ to achieving higher software quality. If your software testers are spending their days writing test automation scripts, guess what they’re not doing: Testing.

On a related note, many testers are quick to point out that these tools don’t actually automate ‘testing’; they automate ‘checking’ and that actual testing will always be carried out by testers – not their tools.

JULY 2015 | www.testmagazine.co.uk

PAGE 27

TESTING TOOLS

LOOKING TO THE CLOUDS

To reach that nirvana of ‘better software faster’ organisations are breaking down silos, implementing continuous integration, parallel development/testing, and then perhaps even continuous deployment

There are many considerations to look at when selecting cloud-based testing tools: paid versus open-source, will you need to add automation expert/team headcount, and a long look will obviously need to be taken at cloud security.

Another consideration that I’d recommend highly valuing is if this new tool and your test strategy as a whole will enable rampant collaboration – especially between globally dispersed teams. Let’s look again at methodologies and cultures such as agile and DevOps. To reach that nirvana of ‘better software faster’ organisations are breaking down silos, implementing continuous integration, parallel development/testing, and then perhaps even continuous deployment. Each of these things requires not just trust between teams who are willing to work seamlessly together. They have to have the technology that enables them to work that way.

This technology is not summed up in tools alone (they’re really not much good to anyone on their own, are they?) – development and test teams must have the ability to provision the environments they

PAGE 28

need to continuously improve the quality of their work. And these environments must be complete, complex, and collaborative. I know we usually reserve ‘collaborative’ for describing us humans, but environments that can be provisioned with the click of a button, frozen in-state, cloned, and shared globally are just as collaborative as we’ll ever be.

In closing, with all of the above considerations you’ll give serious thought to, there is one more that should absolutely be asked, and by everyone from the developers and testers up to the manager or executive who is cutting the check. Instead of simply asking, “Which tool should we use?” Ask instead, “What business problems are we trying to solve?” Remember, the tool itself will not be solving the problems. That’s what the testers, developers, IT professionals, and everyone else in the SDLC is there for.

If the tool you’re considering doesn’t help your dev/test teams deliver the solutions you and your customers need – you might want to put it back on the shelf. ✦

JULY 2015 | www.testmagazine.co.uk

LOAD ???

TESTING GÉRALD PEREIRA CTO JELLLY.IO

AT YOUR SERVICE The challenges of load testing can be reduced if you take an open-source tool and convert it into a service.

Gérald Pereira, CTO, Jellly.io, explains.

L

oad testing is essential to assess the behaviour of a system under a specific load. Its purpose is to allow you to identify and correct potential performance bottlenecks.

The most common example of a load testing use case is in preparation of the sales period. E­-commerce websites will face a sudden increase of their visitors number before Christmas or during Black Fridays. The challenges are to: • Be sure the application won’t fall apart.

• Know if the performances are acceptable, even under load.

• Check how many simultaneous users the website can handle.

There are multiple benefits of such tests; the primary is securing company's targeted revenues. It is essential to bear in mind that response times are important for user satisfaction. It helps to improve the conversion rate and thereby the revenues in the long run. Indeed, as everything moves increasingly faster, we are more and more used to swiftness and fast response times. Studies have demonstrated that 43% of web users give up after 3 seconds. And in any case, a website outage is always bad for a company's brand image, especially when it follows a large marketing campaign.

But of course load testing does not only profit online stores. Any website, web or mobile application should have its performances checked under load. It is even more important if they intend to welcome many simultaneous visitors.

Whatever your business is, you probably have several competitors who will get an advantage over you if their application is faster.

OPEN-SOURCE SOLUTIONS Performance problems are common and range from software configuration issues to insufficient hardware resources. The only way to highlight these performance problems is by simulating a large number of simultaneous users. Several open-source tools are available to help perform load tests.

PAGE 30

JULY 2015 | www.testmagazine.co.uk

LOAD TESTING

Gatling The Gatling tool is a newcomer compared to JMeter.

It's maintained by Stephane Landelle. The first version was released in 2011. It supports HTTP, WebSocket and JMS protocol. Gatling simulations must be written in Scala, but their documentation helps ensure a quick start.

JMeter JMeter is a mature desktop application written in Java (first release in 1998!). It comes with a graphical interface and provides both load testing and functional testing capabilities. It supports a wide range of protocols such as HTTP, JMS, WebSocket, SMTP and more. As a proven industry standard, JMeter is widely used by performance testers. It can be considered one of the most complete free solutions available.

Although it is a good tool, JMeter user experience could be better.

Some limitations include: • Both Java and JMeter must be downloaded and installed on a tester’s computer.

• JMeter has a complex graphical interface. It takes some time to learn how to use it.

It is more profitable to rent than to invest in machines for a one-time usage. This allows QA engineers to free themselves from JMeter installation. As a complete SaaS solution, it also comes with a friendly online graphical interface. It permits easy and quick design of realistic test plans via variable extraction tools and correlation rules.

The goal is to soften the learning curve of stress testing, by introducing a practical load testing solution using the power and completeness of JMeter.

CASE STUDY Tapvalue has developed a breakthrough technology to give retailers a holistic understanding of the omnichannel journey of their customers and prospects, and allow them to interact with them in real-­time and in a personalised way.

It permits easy and quick design of realistic test plans via variable extraction tools and correlation rules

• To simulate realistic virtual users, you may extract variables from server responses and inject them into subsequent requests. This process must be carried out manually.

• You will probably need several servers to simulate heavy loads of virtual users. JMeter cluster configuration can be tedious. Plus you will have to size each machine and JMeter instance.

• Load testing results analysis does not come out of the box with JMeter. You either have to use external plugins or external tools to visualise your performance metrics.

The company collects all existing technology and siloed data (instore, website, mobile app, marketing campaigns, CRM, etc.) within a retailer’s organisation and connects all the dots to create the ultimate and omnichannel unified customer view.

The Tapvalue Omnichannel Platform is a SaaS intuitive platform that provides data visualisation and analysis, omnichannel segmentation and marketing activation.

The infrastructure may handle data coming from thousands of external servers. The requests received are various, from small ones to 1 MB. Load testing is needed to validate that the infrastructure has proper response times, regardless of how many external servers are sending requests. It is also used to validate performances that could be increased using different technologies.

Tapvalue successfully used Jellly’s solution to quickly check that it can process huge amount of streaming data to deliver real-­time analytics dashboards and super­reactive marketing solutions. ✦

JMETER AS A SERVICE To improve JMeter usage, it has been propelled into the cloud. Not only the execution of tests, but also its design and analysis. JMeter has a plugin architecture, which allows its features to be extended easily. Everything has been designed for extension and improvement. Thus at Jellly, we created a Listener plugin that sends the hit statistics to our server. Then they are sampled and stored, to be displayed in our online web application thereafter.

It allows us to offer reports with all performance metrics required, from user load to response time. The reports include: • Line charts to display performance evolution over time.

• Pie charts to let you know the repartition of resource types.

• Top charts to pinpoint performance bottleneck at a glance.

These improvements mean the company does not need to use the JMeter cluster mechanism when running load tests from several geographical locations. And load testers do not need to bother with machines anymore. Jelly takes care of the servers deployment and configuration.

JULY 2015 | www.testmagazine.co.uk

PAGE 31

TEST AUTOMATION ???

ARUPRATAN SANTRA SR. PROJECT MANAGER INFOSYS

AUTOMATED SOFTWARE TESTING: THE WHERE & WHY Arupratan Santra, Sr. Project Manager, Infosys, illustrates a case study in automated testing.

M

ost of the software development lifecycle1 (SDLC) models have some deficiencies with respect to test activities because testing activity starts after the coding phase. The V-model2 and W-model3 have defined the different levels of testing in the development lifecycle. Continuous testing involves testing in every phase of the lifecycle.4,5 It has been observed that the 30 - 40% defect6 will be covered in the early testing involvement in SDLC. The remaining ~60% of defects are detected in the testing phase through manual, as well as automation. Automation tools can be used to identify some of the defects in the unit, integration, system and acceptance testing phases in V-models. Benefits of the automation testing include the ability to streamline a) testing process and documentation; b) execution of tests; c) collection of metrics; d) cost and time; and e) quality of software. Automation is implemented through various framework/approaches7 e.g. modular framework, BDD approach, data driven, library framework, etc.

PAGE 32

The acceptance testing process is not only a sequence of action but is also an interactive continuous process

JULY 2015 | www.testmagazine.co.uk

TEST AUTOMATION

For example, actual function: Public int add (int x,int y) { int z=0;z=x+y; return z;} An automated unit test case for above function is below: Public void testAdd () { int z=add (2,3); assertEquals (5,z);} A unique environment will be used by which all the unit test cases can be run and reports will be generated at one chance. GUI validation (special characters, length, alpha numeric, etc.) can be validated by any functional testing tool.

FIGURE 1. QTP SCRIPT

Even with so many benefits and framework, automated testing cannot be implemented in every part of an application. Selection of the application part/code for the automated testing and selection of automation test suite are not easy activities for the testing professionals.

This article describes where to automate and where not to automate an application with some practical scenarios.

CASE STUDY This case study was performed for a migration application from FORTRAN to .Net. The testing team had to compare the output of the .Net application with the existing FORTRAN application. The output is in Excel data format.

There are 1890 output data sets from each version for comparison. If a test engineer takes a minimum 30 seconds to verify, compare, calculate % variation and report one set of data, then total time required for one cycle is: 30 x 1890 = 56,700 sec. = 15.75 hrs. For three cycles of testing: total time = 47.25 hrs = ~ 6 working days.

So, one test engineer needs 6 days to test before release.

But in reality there are only 1.5 days allocated for testing.

This situation was anticipated by the testing team upfront and hence testing was automated using QTP8 from the FORTRAN output template.

The result is discussed later in this article. This study helped to depict suitable/non-suitable automation scenarios.

Automation scenarios are not only limited to these as mentioned below.

Advantages Unit testing reduces the defect seepage ~50%, which reduces the cost of defect detection in the SDLC in the later phase.

Scenario 2 If the application has multiple interfaces then the implementation of automated integration testing is beneficial for early detection of defects. These will be carried out by developing ‘stub’ and ‘drivers’. These are developed by any programming language or XML/WSDL based on the output value. For example, we have the modules: login, master, and student module. The login module is ready for test, but we call functions from master and student (which is not ready) or we need to independently test the login and other modules. To test at a selective module, we write dynamic automated script called stubs for top down integration and drivers for bottom up integration.

Advantages Automated integration testing reduces the interface defect and reduces the cost of defect.

Scenario 3 System testing meets 100% test coverage so there is a 0% chance to get the new defects unless a new defect is introduced into the system. The testing should be repeated over and over if new lines of codes are incorporated for defect fixing or for requirement change. One defect is incorporated for every six lines of code change. For this type of repetitive/ regression testing, automation approach is the best option.

Types of regression testing are1: • Unit regression testing: Retesting of a single program/ component after modification. o Real time mission critical system uses this approach.

WHERE TO USE TEST AUTOMATION Scenario 1 As per the V-Model, testing has shifted left or upwards.

Accordingly, automated testing was also shifted to the early stage. It has been proved that automated unit testing tools can carry out effective unit testing (e.g. JUnit, VBUnit, CPPUnit, etc.). Test code should be designed in parallel to code design or prior to code development. A unit testing tool can test control flow error, boundary value error, calculation error, validation error, etc. Automated unit test cases should be written for the public method, which returns any value.

JULY 2015 | www.testmagazine.co.uk

• Regional regression testing: Retesting of modules connected to the program/component. o Smoke regression testing for different version release. o Large data comparison through export or import from multiple/single system.

• Full regression testing: Retesting the entire application after a change has been made. o Regression testing for concurrent users. o Regression testing for role based application. o E2E functional test cases for repeated releases. ➤

PAGE 33

TEST AUTOMATION Regression testing is one of the expensive activities in the SDLC. Automation is a suitable and sustainable technique for regression testing. Automation improves cost and effectiveness of the retest/regression test.

Advantages Automated testing is more beneficial for regression testing than manual testing. The complete study is described in the cost benefit section later.

Scenario 4

Advantages Early identification of these scenarios will save time for test strategy preparation.

Result The QTP test result has helped to zero down the various automation scenarios as above. As per the result, each cycle has taken ~30 minutes for execution. There are 1005 failed data sets in the output file. Figure 3 describes the effort savings trend for automation.

The acceptance testing process is not only a sequence of action but is also an interactive continuous process. That is why automation is best applied only to a narrow path of testing, not to the broader path of the test process as suited for acceptance testing from the second release. Automation is best suited for use in a sequence of action from GUI level for UAT. Advantages Major advantages include test case coverage and the cost-effectiveness due to saved efforts.

WHERE NOT TO AUTOMATE Variability in test cases is one of the great advantages of manual over automation. Manual testing is best for complex business process, as it is more variable and more directed at new features and specific areas where defects are more likely to be found.

Manual testing is advisable for test cases in backward flow and is useful for an application where design changes frequently. Automation test is inappropriate for batch process, recovery, ad hoc testing, etc.

Validation of dynamic report and graph by using automation tools is possible but not advisable due to technical complexity. If the GUI properties are changing through the external file (e.g. .init, .txt, etc.) then it is very difficult to handle the validation process by automation tool.

FIGURE 3. EFFORT BENEFIT CHART.

We can infer that initial cost is higher for automated testing with respect to manual testing but after three cycles of regression testing, it is more cost-effective than manual.

Conclusion This article discussed different scenarios with practical examples for automated testing. Some of the scenarios where automated testing are not possible, are also highlighted. By implementing automated testing quality will improve but it cannot eliminate the manual testing.

The fact is that manual testing and automated testing are two different processes and should not be considered as two different ways to execute the same process. So it’s recommended treating test automation as one part of a test process in the test strategy. ✦

REFERENCES

FIGURE 2. QTP RESULT.

PAGE 34

1.

Marick, B., ‘The Craft of Software Testing’, pp 50-102 Prentice-Hall (1995).

5.

Perry, W.E., ‘Effective Methods for Software Testing’, pp 50-55 John Wiley & Sons, 2nd Edition, (1999).

2.

Kaner, C., Falk, J. and Nguyen, H.Q., ‘Testing Computer’ Software, 2nd Edition, pp 11-91 John Wiley & Sons, New York, (1993).

6.

3.

Myers, G.J., ‘The Art of Software Testing’, pp 5-50 John Wiley & Sons, New York, (1976).

Spillner, A., ‘From V-Model to W-Model – Establishing the Whole Test Process, Proceedings Conquest’ - 4th Conf. on Quality Engineering in Software Technology, (2000).

7.

4.

Jalote, P., ‘An Integrated Approach to Software Engineering’, pp 101-200 Narosa Publishing House, 2nd Edition, (1998).

Penchansky, ‘The concept of access: definition and relation ship to consumer satisfaction.’

Medical Care 19(2), (1981), pp 127-140.

8.

HP QuickTest Professional software - HP - BTO Software, www.hp.com.

JULY 2015 | www.testmagazine.co.uk

DEVOPS CARL BRUINERS HEAD OF DEVOPS MAGENTYS

DEFINING DEVOPS Carl Bruiners, Head of DevOps, MagenTys, highlights some common mistakes people make when implementing DevOps.

evOps is a marriage between three strands within IT; development, technical operations and quality assurance. The purpose of this marriage is to enable the ability of a company to be able to rapidly deploy software (note: environments are software) through a highly governed pipeline that is focussed on giving the business the confidence that the software is fit for purpose and passes various quality gates. This coupled with agile development gives companies additional assurances such as business continuity/disaster recovery as well as the ability to become provider independent.

D

IN LAYMAN'S TERMS DevOps is a capability of being able to deploy both software and its underlying environments through a managed pipeline that offers rapid deployments through a high quality model that we call a pipeline.

Operationally environments are software and these environments along with the environment configurations this enables rapid deployment of the underlying infrastructure and to wrap around these environments repeatable tests that ensure we have suitable environments to deploy our solutions onto.

• Overloading an existing role with the addition of being ‘DevOps’ for the team instead of having a separate team or individual within a team tasked with pipeline/build management.

• Not embedding monitoring early on, pipelines with little or no monitoring capabilities offer no benefits i.e. you're rapidly deploying code without knowing it or what the outcome of the tests throughout the pipeline are doing or if they are failing.

DevOps, alongside quality assurance, is a guardian of quality

WHAT ARE THE COMMON MISTAKES PEOPLE MAKE WHEN IMPLEMENTING DEVOPS? There are a number of potential caveats when implementing DevOps: • Separating out the Dev and the Ops parts within team.

• Not being automation centric.

• Acting as a Tiger team who operate in a JFDI model at all times.

JULY 2015 | www.testmagazine.co.uk

• Not acknowledging the cultural, behavioural and belief issues that will be encountered when implementing a DevOps function.

KEY BENEFITS THAT DEVOPS OFFERS TO TESTERS AND ORGANISATIONS DevOps, alongside quality assurance, is a guardian of quality.

Automation of repeatable testing actions frees up testers to focus on exploratory testing. Quality assurance works with the DevOps function on ‘baking’ in various test types into the DevOps pipeline.

DevOps enables an organisation the capability to delivery both environments and software into production more quickly with a high degree of quality through a vastly reduced number of defects escaping into ‘live’. In addition to this, the capability to ‘lift and shift’ the entire infrastructure and software estate gives the organisation the competence to become supplier agnostic as well as providing the reassurance of business continuity in the event of a disaster. ✦

PAGE 35

On the 19th of May 2015, the halls of the world famous British Museum, known for its impressive collections of the past, were filled with discussion about technologies of the future.

Over 270 attendees from scores of different businesses that included easyJet, Credit Suisse, Direct Line, Mail Newspapers and Sony, congregated to partake in marvellous speaker sessions, interactive workshops, and enthralling Q&A debates.

We heard from speakers covering various pressing topics ranging from smart cities, to the legal liabilities within software development right through to the arguments for and against standardisation.

Our exhibition floor was busy, with attendees viewing the latest products and services from some market leading providers. In addition, Test Associates hosted a popular recruitment clinic, tailored for the software testing community.

And once we’d had our fill of knowledge and networking, we ate like kings and were entertained at the Gala Dinner, held at the fabulous 5* Russell Hotel, just around the corner from the British Museum.

Thanks again to everyone who came, shared their knowledge, networked, and enjoyed themselves! Thanks also to all of our sponsors, especially our headline sponsor, Borland for all of their support over the years. Also Gold sponsor Birlasoft and our silver sponsor IBM. PAGE 36

CONFERENCE FEEDBACK*

84%

SAID THE CONTENT WAS EITHER “GOOD” OR “FANTASTIC”

94%

THOUGHT THAT THE CONFERENCE WAS “EXTREMELY WELL-ORGANISED AND EXECUTED”

95% 91%

OF ATTENDEES SAID THE KEYNOTE SPEAKERS WERE “GOOD“ OR FANTASTIC

72%

ARE INTERESTED IN ATTENDING THE NEXT EVENT

FELT THAT THE CONFERENCE WAS “GOOD VALUE FOR MONEY”

*DATA GATHERED FROM THE NATIONAL SOFTWARE TESTING CONFERENCE 2014/5 FEEDBACK

WHO WAS THERE? 285 DELEGATES

49% 33% 18%

• TEAM LEADERS • MANAGERS • SENIOR MANAGERS • HEADS • DIRECTORS • C-LEVEL • TESTERS • ARCHITECTS • CONSULTANTS • ENGINEERS

JULY 2015 | www.testmagazine.co.uk

JULY 2015 | www.testmagazine.co.uk

PAGE 37

THE TESTING CAREER CORNER RICHARD AKROFI PRINCIPAL CONSULTANT TEST ASSOCIATES richard.akrofi@testassociates.co.uk

INTERVIEW TIPS: TRUE AND TESTED Whether applying for the first big career job, or transitioning from one role to another, you’re likely to have to step up to the plate and prove yourself in a series of interviews.

I

have compiled some of my true and tested interview tips. It’s important to not short-change yourself and come prepared, as unfortunately you may only have that short timeframe to impress a future employer and stand out from the crowd.

FIRST IMPRESSIONS COUNT To be invited for an interview, you first need to get your resume in front of the hiring manager/recruitment consultant.

Clear communication is key. You need to produce a well-written (and proof-read) and well-presented resume and cover letter – outlining relevant experience and skills sets.

There is no set rule for what turns off hiring managers when looking at resumes, but it’s safe to say that you should not

PAGE 38

JULY 2015 | www.testmagazine.co.uk

THE TESTING CAREER CORNER give them an easy excuse to discard your application. And by easy excuse I mean a poor presentation, messy layout or bad spelling and grammar.

If you are communicating with the hiring manager or recruitment consultant prior to an interview, make sure you sound professional and serious on the phone and over email.

Finally, show up to the interview looking neat and well presented, proving you are taking the meeting seriously.

BE CONFIDENT IN YOUR SKILLS BUT SHOW HUMILITY It can be tempting to embellish skills and experiences, and while it’s important to promote yourself, I can’t stress enough that you should not lie on your resume. Chances are, if you’ve lied to match the skills/experience outlined in the job posting, you will be asked to demonstrate these in detail leading to mild embarrassment, or even a prompt dismissal for timewasting.

If you can demonstrate a willingness and enthusiasm for learning new things, then this will always be in your favour. Few jobs expect you to come in knowing everything, and hiring managers will be looking for candidates both capable and interested in learning and further developing their skillsets. A little humility and recognition that you might still have some things left to learn will go a long way.

A young, vibrant, specialist IT recruitment partner to the Computing Software sector; finding and placing market-leading Software Testers, Software Developers and Quality Assurance talent around the world. We assist organisations to attract suitable passive talent in the market, rather than being limited to a list of candidates that are actively seeking jobs.

What we do?

We recruit for permanent and contract positions, as well as offering a bespoke recruitment service, and a specialist search and selection process.

Our experienced consultants bring a wealth of knowledge and experience, and our database of Software Testers, Software Developers and Quality Assurance professionals provides us with the tools to ensure we find the right candidate for your vacancy.

We operate with absolute discretion, integrity and professionalism.

HIGHLIGHTED VACANCY

BE PREPARED FOR AN ASSESSMENT

Programme Test Manager

If the job has outlined specific skills or experience, it’s likely that there will be a form of assessment during the interview. You could be asked to speak in detail about a hypothetical test case or you may be in front of a computer screen asked to go through it.

JOB TYPE: Permanent JOB STATUS: Full Time LOCATION: Central London SALARY: Very competitive

It is important to remember that the test serves several functions – they are not just looking to see that you’re doing it ‘correctly’, but also how you work. Can you explain your thinking/process?

It all comes down to communication again; if you can explain your conclusions and ideally give a step-by-step process, then the hiring manager will be able to understand if how you work fits their expectations.

JOB BRIEF: Fantastic opportunity for an experienced Programme Test Manager (18 months plus) to move into the early stages of a major project with a client in Central London. This particular project involves managing Test Teams, Test Managers and multiple projects simultaneously to ensure successful implementation of a new global ERP system.

So take a deep breath, go through the task slowly and carefully and focus on presenting your best work self.

RESPONSIBILITIES & EXPERIENCE REQUIRED • Setting priorities that balance desirable against practical testing strategies • Definition of strategies to ensure the most effective and efficient testing is carried out, including the use of specialised testing tools and techniques • Consulting on all aspects of software testing and give advice to both users and IT management to determine the most appropriate testing strategies for projects • Managing test improvement activities, including automation implementation • Managing test plans • Management of multiple projects simultaneously • Full understanding of the testing life-cycles • Stakeholder management • Managing Test Managers and test teams of various sizes • Owning the gating process to ensure smooth transition from one phase to another • Manage 3rd party through System Integration Testing and System Acceptance Testing • Managing UAT cycles through to go-live • Analysing project documentation to extract requirements, system functions and other features to be tested • Manage, motivate and lead project teams demonstrating effective delegation techniques.

ASK QUESTIONS You should turn the interview on its head and ask yourself the question, “do I want to work here?” and make sure you come prepared with some questions about the company culture.

This will serve you well in two ways, firstly you’ll find out if this is the kind of company where you’d like to work. Secondly, it sends the signal to the hiring managers that you are serious about your future, and that you care about fitting in. Similarly, you are giving them a chance to talk about their workplace and its positive attributes. This will take the focus off of you, whilst ensuring that the hiring managers remember you as the candidate who asked a lot of questions and is really interested in working here.

MAKE THEM LAUGH And finally, make them laugh if you can. Not at you, of course, but if you can fit in a small joke somewhere, then that could never hurt. You’ll be remembered positively, which could help you stand out when a final decision is being made. ✦

JULY 2015 | www.testmagazine.co.uk

For more visit: www.TESTassociates.co.uk

PAGE 39

TEST AUTOMATION ???

HUNG NGUYEN CEO LOGIGEAR

TACKLING THE LARGER SCALE Hung Nguyen, CEO, LogiGear explores large-scale test automation and its potentials.

PAGE 40

JULY 2015 | www.testmagazine.co.uk

TEST AUTOMATION

L

arge-scale test automation is difficult, but done right, is achievable and manageable. It requires architectural and framework design, comprehensive resource planning and the implementation of lean processes to be successful. What constitutes large-scale is different for different organisations. It could be testing 1000 web pages on all different browser versions, or it could involve distributed testing on 100 or more machines at the same time. It’s not uncommon for testing to quickly grow to several thousand test cases with heavy usage of data variation (or data-driven tests).

ACTION BASED TESTING In ABT, tests are kept as a series of actions in a test module (which looks essentially like a spreadsheet).

Each action is written as a line in the sheet, starting with an action keyword followed by arguments.

Modules make it possible to organise tests much like chapters in a book. Test modules provide a breakdown of tests so that each test module has a clearly defined and well-focused scope and is Whatever the application, manual testing at scale is differentiated from the other modules. The scope of impractical if not impossible and test automation a test module subsequently determines what its test is a necessity to execute the required testing, cases should look like. The scope of a test module particularly in agile. Successful large-scale determines what approach to take to develop test automation requires that production the test module. This means the choice of throughput must be high enough It’s worth testing techniques used to build the test to rapidly reach critical mass, and cases (such as boundary analysis, decision mentioning that the provide the ability to rapidly tables, etc.), and who should get involved author automated tests, keep up to create and/or assess the tests. For the BDD and keywordwith fixing broken tests when the example, a test module aimed at testing application changes, and rapidly the premium calculation of insurance driven method can work triage and resolve failed and policies might need the involvement of false-positive results. an actuarial department. well together to achieve

SELECTING AN AUTOMATED TEST FRAMEWORK

good test design

All of this requires a framework that provides flexibility, manageability, and a team collaboration facility. This type of framework allows programmers and testers to work from test development through test execution, and test maintenance (or unbreaking tests) primarily by solving test specification and authoring problems. It also simplifies turning test specifications, test cases, or user stories into executable automated tests. In addition it provides the flexibility to enable teams to customise as special situations arise and handle changes in the application under test, its configurations or environments, transparently.

Test design – that is how teams design or author tests for maintainability, scalability and execution performance optimisation – is an additional major contributor to efficiency and good coverage. Regardless of all claims, this can be difficult. The principle of test design should be ‘lean and mean.’

The tests should be of a manageable size, and at the same time complete and aggressive enough to find bugs before a system or system update is released.

In selecting an automated test framework and test methods that support the lean and mean test design, there are a number of options.

• OpenSource: oo BDD (behaviour driven development method) coupled with Cucumber and Selenium, SpecFlow, RSpect, etc. oo Keyword-driven method with FITnesse.

• Third-party COTS with keyword-driven support – some examples of the third-party tools with keyword support include HP UFT, SmartBear Test Complete and TestArchitect.

It’s worth mentioning that the BDD and keyword-driven method can work well together to achieve good test design.

Over time, the keyword-driven method was enhanced, becoming ABT (action based testing).

JULY 2015 | www.testmagazine.co.uk

Long-term test maintainability is achieved at the test case level by specifying only the high-level details that are relevant for the test. For example, from the end-user perspective ‘login’ or ‘change customer phone number’ is one action; it is not necessary to specify any low-level details such as clicks and inputs. The low-level details should be ‘hidden’ in separate, reusable automation functions common to all tests.

This makes a test more concise and readable and cuts maintenance costs since low-level details will not have to be changed one-by-one in every single test when the underlying system undergoes changes. The low-level details can then be re-specified (or have their automation revised) only once and reused many times in all tests.

Embedded in TestArchitect, the ABT method places the focus on how keyword-driven tests can best be organised and designed to accommodate their effective automation.

SUMMARY The similarities of these frameworks are that they utilise test methods built around a plain text, business-readable, domain specific language which optimises test automation production.

Both BDD and keywords contribute to speeding test production, and it is the technology platform that will produce the ultimate efficiency. The upside of open source tools is that they are free, but a potential downside can be, while a solution such as Selenium or Cucumber reduces the need of coding, it’s still a solution for programmers, and some coding is still required for automated test production, and additional in-house software development time is required to handle the automation engineering implementation.

When it comes to addressing the needs of large-scale testing, the method is as important as the technology. Thinking that technology itself will solve the problem at hand will negatively and significantly affect the ability of the organisation to optimally use its selected tools and achieve large-scale test success. The good news is that testing teams have options on both methods and technology. ✦

PAGE 41

PROCESS ???AUTOMATION NEIL KINSON VICE PRESIDENT, EMEA REDWOOD SOFTWARE

THE COLLABORATION CROSSROADS Neil Kinson, Vice President, EMEA, Redwood Software, explains why collaboration of IT and business processes is critical to the success of any business.

PAGE 42

JULY 2015 | www.testmagazine.co.uk

PROCESS AUTOMATION

T

echnology is evolving at an astonishing rate, making it difficult to think of a time without smartphones, let alone the days before widespread email and social media use. In the business world, work has changed profoundly over the last five years with the increased adoption of business, marketing and IT technologies ranging from communication tools such as web-conferencing or social networks through to automation software and data management platforms.

visibility and gain complete control across all of its supply chain activities. RS Components brought together all the company’s end-to-end supply chain tasks under a single platform. Using a standard global template, the organisation maintains control of activities, minimises risk, and can scale to fit the ambitions of the business by quickly rolling out changes or adding new regions.

When businesses can see the bigger picture across their IT and business processes in this way, they can turn their attention to achieving their goals and delivering positive results to their customers instead of spending time getting their own house in order.

While these reliable mainframe and similar technologies are worthy allies in the management of central processes, they are all too often surrounded by multiple layers of cloud-based, distributed and virtual technologies. With siloed operations and systems latched on to the mainframe HARNESSING TECHNOLOGY TO IMPROVE it can be hard to maintain thorough EFFICIENCY AND ACCURACY Automating process visibility and fully co-ordinated cross-system processes which can So, how can IT professionals accomplish their business processes lead to inefficiency or error. This goals without working around the clock? presents a significant challenge to The trick is to find repeatable processes and end-to-end makes organisations which are looking automate them. That way, each task will completing hundreds – to reduce complexity and ensure complete with absolute consistency, speed collaboration across both IT and and visibility – and organisations don't have or hundreds of thousands business processes. to spend hours examining results and asking questions.

– of repeatable tasks By 2016, Gartner predicts that “70% of the most profitable Automating business processes end-to-end manageable companies will manage their makes completing hundreds – or hundreds of processes using real-time predictive thousands – of repeatable tasks manageable. By analytics or extreme collaboration.”1 removing manual activities, organisations reduce the For this vision to become a reality, many opportunity for human error and improve their visibility businesses will find themselves needing to lift over business operations. In addition, when IT professionals the bonnet on their existing operations in order to can automatically document steps, they can audit regular realise how they can deliver the most value from their existing checks of business operations. ➤ and future IT investments. With the growing business need for in-depth analytics and the efficient processing of data to support competitive advantage and customer service, the priority for businesses should be finding a way to co-ordinate the process and maintain business transparency.

BRINGING SEEMINGLY DISPARATE ELEMENTS TOGETHER In order to co-ordinate operations and improve visibility, top performing organisations must better engineer the processes that support their business. This can be achieved by bringing seemingly disparate elements together. To get the most from a complicated IT environment, companies need platform-agnostic automation.

With process automation, companies eliminate manual effort and simplify the monitoring, definition and analysis of the completion of business processes such as order-to-cash, procure-to-pay, and supply chain management to name but a few. By harnessing control over operations, companies achieve the detailed, real-time status visibility needed to swiftly pinpoint, analyse, and ultimately eliminate process bottlenecks before they happen.

For example, when it comes to assets, many organisations struggle to unite a range of supply chain tasks – including inventory processes, order-taking, order-to-cash and delivery fulfilment. For RS Components, a company with operations in 32 countries and shipments of more than 44,000 parcels every business day, detailed analytics and fast processing of data is central to supporting competitive advantage and customer service and the same will be true for the vast majority of businesses.

The global distributor for engineers worked with Redwood Software and used process automation to enhance

JULY 2015 | www.testmagazine.co.uk

PAGE 43

PROCESS ???AUTOMATION Promoting high standards of efficiency and accuracy whilst keeping a keen eye on improvement in any organisation requires constant vigilance. Automation is becoming an obvious means to achieve this greater efficiency and reduce costs, whilst enabling organisations to maintain full control of business processes. By streamlining processes to achieve greater efficiency across the business model, employee time can be used more resourcefully, freeing them up to learn new skills.

True process automation isn't just a planning chart or ‘to do list’ style task dashboard. With process automation, organisations can build the logical steps that actually execute business operations with reliable precision. Once the right processes are automated correctly, they occur exactly the same way every time. Automating business-critical processes enables organisations to increase the value of existing information technology systems to eliminate costly and unnecessary time, effort and energy.

For the retail industry which hinges on the delivery of excellent customer service, streamlined business processes are especially important and many retailers are investing in tools to meet customer demands. For example, with numerous streams of new products and information from suppliers, Bol.com a large online retailer in the Netherlands, needed to efficiently handle a complex inventory, accommodate a steady influx of new suppliers, manage downstream logistics and fulfilment, ensure next day delivery, enable customers to track their orders, and accurately forecast.

The retailer turned to automation to expand its business, and used Redwood Cronacle® to eliminate manual activities, standardise tasks and co-ordinate all of its core supply chain processes. With a complex array of daily activities, the company now maintains an accurate catalogue of content, can quickly onboard new partners, and ensures its logistics, self-service portal and forecasting operates seamlessly. This is essential to reassure Bol.com that any problems will be easily detected and resolved promptly to produce an accurate end-result.

Other retailers and businesses can benefit from this approach; co-ordinating business operations efficiently to boost profits, brand value and ultimately customer loyalty. There is also a second major benefit for organisations as, when businesses use technology to break down massive projects, they also unlock the innovative power of their workforce. This means that businesses have an opportunity to transform their IT team of detail-watchers into real analysts that can deliver business insights.

There is more work to be done to free up IT departments to focus on wider business innovation and this is where business technologies can help. Not only can streamlining processes manage complex environments with ease, but it will ensure that businesses are getting the most from their technology investments, wherever they reside.

BUILDING A BETTER BUSINESS With automation in place it becomes much simpler for business and IT stakeholders to collaborate in the management and notable improvement of essential processes. Automated processes ensure complete co-ordination, accuracy and consistency across diverse technologies, locations and platforms. Tamed with automation, even the most complex IT landscape becomes manageable.

Of course, it can be difficult to see the how collaboration across many siloed activities is achievable, but it is possible by seamlessly integrating IT and business processes. If Gartner’s collaborative vision is to become a reality then organisations will need to make sure their processes are fit for purpose in 2015 and beyond – winning companies will be those that collaborate and maintain alignment going forward. ✦ REFERENCES 1.

‘Gartner Says by 2016, 70 Percent of the Most Profitable Companies Will Manage Their Business Processes Using Real-Time Predictive Analytics or Extreme Collaboration’, http://www.gartner.com/newsroom/id/2349215.

AUTOMATE TO UNLOCK THE POTENTIAL OF A WORKFORCE If the economic downturn taught us one thing, it is the need for businesses to deliver results with scarce resources. The ‘cult of overworking’ is well and truly upon us, but working long hours doesn’t necessarily mean that quality and productivity is guaranteed. In fact, working overtime can significantly increase the rate of mistakes and reduce the ability for staff to perform at their highest level. Rather than asking people to complete more tasks in shorter time frames, companies should consider how they can better utilise their current workforce, and therefore deliver more, with less.

Today's process automation technology frees IT professionals from demanding, repetitive work. With process automation, there is no need to compromise on speed or quality for efficiency. They can have it all, and they can even have a day off now and then!

PAGE 44

JULY 2015 | www.testmagazine.co.uk

SUCCESSFUL SOFTWARE DELIVERY

BUILD ON YOUR MANUAL TESTING QUALITY WITH AUTOMATION Manual testing is great, so far as it goes. The problem is, it can’t scale to keep pace with development and rapid Agile release cycles. So what if you extend the reach of human testing through automation?

With a hybrid environment from Borland, you integrate requirements, manual and automated testing. So when manual testing struggles to keep up, you get a helping hand to increase testing speed while maintaining quality – across all of your different devices, platforms and OS versions. It’s automation at its best, with Borland’s human touch.

‘The human side of test automation’– download your FREE White Paper at www.borland.com/testautomation

CONNECT YOUR PEOPLE FOR SUCCESSFUL SOFTWARE DELIVERY Copyright© 2015 Micro Focus. All Rights Reserved.

Portions Copyright © 1994-2009 Borland Software Corporation (a Micro Focus company).