TEST Magazine - June-July 2014

VOLUME 6: ISSUE 3 JUNE 2014 www.testmagazine.co.uk

THE EUROPEAN SOFTWARE TESTER

INNOVATION FOR SOFTWARE QUALITY

HIGHLIGHTS: NSTC 2014 VERY PROFESSIONAL, GREAT VENUE AND FANTASTIC NETWORKING OPPORTUNITY

CONTENTS

INSIDE THIS ISSUE

NEWS 8

Professionals to map their career to get ahead

11.

Plans for Tablet TV Freeview delayed

THE EUROPEAN SOFTWARE TESTING AWARDS 2014 11

WHAT ARE JUDGES LOOKING FOR?

What are judges looking for?

As The European Software Testing Awards 2014 opens for entries, TEST Magazine finds out what the judges will be looking for this year…

THOUGHT LEADERSHIP

14

Enabling “assembly line” assurance

Siva Ganesan writes about the role of automation and instrumentation to enable “assembly line” assurance, which ensures efficiency in software development and value for the end consumer…

AUTOMATED TESTING

19

Is automation testing the future or a fad?

22.

BARRIERS TO TEST AUTOMATION

Sean Rand asks, does automation have a place in every test team therefore being the future of testing? Or is it for a fad?...

22 Barriers to test automation

24.

Amir Ghahrai examines some of the most common barriers to test automation, and how these challenges prevent organisations from succeeding in this area of testing…

EVM IN AGILE: WHAT’S IN IT FOR SOFTWARE TESTING?

19.

IS AUTOMATION TESTING THE FUTURE OR A FAD?

AGILE PROCESSES 24

EVM in agile: What’s in it for software testing?

Mukesh Sharma discusses Evidence Based Management (EVM) in agile processes, and how this next major evolution in software testing brings freedom and accountability…

JUNE 2014 | www.testmagazine.co.uk

PAGE 3

Assurance is the science of simplification.

In today’s overly complex technology world, testing and QA functions must balance the art of perfection with the science of simplification.

There exists a way: Tata Consultancy Services (TCS). With TCS’ independent enterprise testing arm, Assurance Services Unit (ASU), you can balance your testing needs and business goals with market-proven, world-class experience, expertise and guidance. Visit tcs.com/assurance and you’re certain to learn more. Or write to us at: global.assurance@tcs.com

IT Services Business Solutions Consulting Scan the code to know about TCS Assurance Services

CONTENTS

INSIDE THIS ISSUE

30.

FOCUS ON: THE WORKPLACE 30

NEW LEGISLATION: FLEXIBLE WORKING

New legislation: Flexible working

July welcomes a new legislation that will extend the right to request flexible working to all eligible employees. Emma Clark explains how these laws will impact your organisation and how can you best prepare for the change…

32 10 skills your employees will thank you for Whether you employ one person or 1,000, there are 10 skills that no employer or manager should be without…

COVER STORY 39 National Software Testing Conference 2014 Review TEST Magazine looks back at the first ever National Software Testing Conference, which saw much of the software testing community descend into London...

32.

10 SKILLS YOUR EMPLOYEES WILL THANK YOU FOR

LAST WORD 46

I’ll just work from home!

Is working from home a good thing or a bad thing? Dave Whalen explains why he believes it’s a positive...

39.

NATIONAL SOFTWARE TESTING CONFERENCE 2014 REVIEW

JUNE 2014 | www.testmagazine.co.uk

PAGE 5

www.neotys.com

LEADER

STRENGTH IN NUMBERS Hello, and welcome to the June issue of TEST Magazine.

A

nd just like that, the first ever National Software Testing Conference is over, and what a huge success it was! The atmosphere was vibrant; the presentations were enlightening; and the feedback was tremendously positive. We feel extremely proud to have brought together the software testing industry, and provided a great platform to learn, network and debate all within majestic surroundings... For a full review, please turn to page 35.

Our aim, via the National Software Testing Conference, TEST Magazine and The European Software Testing Awards, is to raise the profile of software testing, and help improve best practice and process by supporting and bringing together senior professionals.

To further promote this unity, we are please to bring you the Software Testing Network, which was launched at the National Software Testing Conference. It is our belief that the software testing profession must continually join together and promote all that is good about an under-recognised industry. There is more “strength in numbers”.

The Software Testing Network helps members strengthen their business performance Do you want by assisting in the to write for improvement of TEST magazine? their testing practices Please email sophie. and processes, while odum@31media. providing opportunities for co.uk new relations with individuals across various vertical sectors. This encourages cross-industry knowledge sharing, bringing uniformity to the industry.

The Network offers members exclusive access to independent research; valuable market insights; and high quality content and literature, as well as highbrow discussion forums, awards, conferences and gala dinners, plus lots more. For more information, please visit www.softwaretestingnetwork.com. Check it out, there are various levels of membership to suit different needs!

All that is left for me to say is, I hope you enjoy this issue...

Sophie-Marie Odum Editor

© 2014 31 Media Limited.

All rights reserved.

TEST Magazine is edited, designed, and published by 31 Media Limited. No part of TEST Magazine may be reproduced, transmitted, stored electronically, distributed, or copied, in whole or part without the prior written consent of the publisher. A reprint service is available.

Opinions expressed in this journal do not necessarily reflect those of the editor or TEST Magazine or its publisher, 31 Media Limited.

ISSN 2040-01-60

T H I R T YO N E

EDITOR Sophie-Marie Odum sophie.odum@31media.co.uk Tel: +44 (0)203 056 4599 ADVERTISING MANAGER Nick Hayward nick.hayward@31media.co.uk Tel: +44(0)203 668 6949 ADVERTISING EXECUTIVE Sarah Walsh sarah.walsh@31media.co.uk Tel: +44(0)203 668 6945

PRODUCTION & DESIGN Tina Harris tina.harris@31media.co.uk ENQUIRIES 31 Media Ltd, 41-42 Daisy Business Park, 19-35 Sylvan Grove, London, SE15 1PD Tel: +44 (0) 870 863 6930 Email: info@31media.co.uk Web: www.testmagazine.co.uk

PRINTED BY Pensord, Tram Road, Pontllanfraith, Blackwood, NP12 2YA

JUNE 2014 | www.testmagazine.co.uk

PAGE 7

NEWS

PROFESSIONALS TO MAP THEIR CAREER TO GET AHEAD Whose responsibility is career development, and how are businesses and individuals tackling the challenge of personal development? These are just some of the questions that BCS, The Chartered Institute for IT, is asking as part of its new initiative, Map Your Career.

The three-month, online initiative follows the Institute’s recent study, which found that although 90% of employers and 91% of individuals believe that continued professional development (CPD) is valuable, businesses are devoting less resource to it than ever before.

68% of employees have seen a drop in the amount of funding made available for professional

development at work, and 78% of UK firms stated their workforce committed less to CPD than they would like.

David Evans, membership director at the Institute, said, “This gap between what employees and employers expect – with the responsibility for professional development becoming blurred – is an issue we need to seriously look at.

“However, it’s not just an issue that employees and employers face; it is mirrored in the freelance world, where businesses often now ask for contractors to have specific professional qualifications as well as relevant experience.”

Speaking about the IT profession in particular, Evans added, “The IT

GOOGLE TO ENCRYPT EMAILS Google is testing a new browser extension that will be able to encrypt Gmail messages sent to and from Google Chrome.

While email encryption software isn’t new, and Google already offers an encrypted connection for Gmail, the new service would encrypt the message content. The company said it hoped the plugin, called End-to-End, would make the process of encryption more accessible and therefore more widely used. End-to-End is said to promise uninterrupted protection of data travelling between two parties.

According to a recent Google Transparency Report, 40% to 50% of emails sent from within their hosted

accounts are not encrypted.

A statement from Google said, “We recognise that this sort of encryption will probably only be used for very sensitive messages or by those who need added protection. But we hope that the End-to-End extension will make it quicker and easier for people to get that extra layer of security should they need it.”

The company has released the source code to the tech community to check for bugs and get feedback before a public launch.

This move follows Yahoo’s announcement in April that it was moving towards a platform where all emails would be encrypted by default.

POLL RESULTS Last month we asked, "WHAT AUTOMATION FRAMEWORK DO YOU USE?". Answer this month's poll at: www.testingmagazine.com

16% 17% 69%

BESPOKE VENDOR OPEN SOURCE

profession is facing a huge challenge as short-term goals have driven a longer-term change in development behaviours.

“Traditionally companies have given their employees time and resources to undertake CPD-related activities, but, this year, 61% of employees have experienced a drop in the amount of time made available to them to devote to it.

“Meanwhile the number of highly skilled professionals entering the freelance arena is growing. There is a widening gap between those who set goals and plan their careers and those who find themselves trapped in jobs because of short-term circumstance – this is something that we want to talk about.”

UPDATE FOR SAMSUNG DEVICES Samsung is currently prepping to release the new OS v4.4.3 update to smartphones, according to reports.

It’s thought that Samsung’s Galaxy S5 models (SM-G900H & SM-G900F) and Galaxy S4 LTE (GT-I9506) are being tested with new software build version KTU70, believed to be the un-released Android v4.4.3 firmware. The report does not specify when the OS update will get released to these devices, but if the report is to be believed, the firmware might not be made available until another month or so as it is being integrated with Samsung’s custom interface.

The report also revealed that the Samsung’s three mid-range devices – Galaxy Grand 2 (SMG7102), Mega 5.8 (GT-I9152) and Mega 6.3 (GT-19200) are under final phase of the v4.4.2 KitKat (build version: KOT49H) testing, and is said to be rolled out this month.

For the latest news, visit softwaretestingnews.co.uk and follow us @testmagazine

PAGE 8

JUNE 2014 | www.testmagazine.co.uk

NEWS PLANS FOR TABLET TV FREEVIEW DELAYED Testing of Motive Television’s technology for Tablet TV Freeview, is taking longer than anticipated. According to the company, once the hardware and software tests by the Digital Television Group have been completed, this will be the first and only technology of its kind to have been certified for Freeview viewing on tablets.

The company was hoping to have the testing and debugging completed in time for a UK launch ahead the World Cup. The app will allow viewers to have a full Freeview experience, including “red button” interactive capabilities, plus additional Motive features to watch and record all Freeview channels.

Leonard Fertig, chief executive officer of Motive, said,

A WORKFORCE THAT MEETS TODAY’S DEMANDS Following recent news that dozens of GCSEs, AS- and A-levels are to be scrapped in the latest stage of a major shakeup of exams, it is hoped this is practically driven and in line with the increasing need for analytics talent, according to SAS UK. In a consultation paper published, Ofqual will call for comments on existing exams to be reformed along the lines of English, maths and science papers at GCSE and A-level, in order to make them more exam-based and rigorous.

Laurie Miles, Head of Analytics at SAS UK & Ireland, said, “The youth of tomorrow is expected to build applications and analyse data, not recite times tables. The UK needs a workforce that can meet today’s business needs to drive innovation and the economy. It is imperative this curriculum change in maths and science is practically driven and in line with industry demands.

“The explosion of big data has created an unprecedented demand for analytics talent, with around 70,000 more big data specialists needed by 2017, according to research by SAS and e-skills UK. “Jobs requiring analytic skills are in high demand, but there simply isn’t enough talent to fill those jobs. There is an emerging industry effort to arm the next generation with the right skills for tomorrow’s jobs.

“This curriculum change, if applied in the right way could well be the magic pill that will embed industry skills across a much broader population. But it cannot stop there. A concerted effort from industry and academia is required to help the next generation develop the right skills for the economy.”

JUNE 2014 | www.testmagazine.co.uk

“Although we had hoped to complete the testing and certification process sooner, Motive believes that the novel and superior features of the Freeview Tablet TV product will set new standards for convenience and capabilities for a viewer.

“The Motive team is excited that testing is underway and that the expected launch of this innovative product in the UK will take place shortly,” he added.

NEW QUALIFICATIONS, AVAILABLE WORLDWIDE The explosive growth of Android applications has created a boom in demand for developers, but employers still need to assess their competency, which is why Android ATC has designed official courses and, in partnership with Pearson VUE, computerbased assessments, will provide a range of new qualifications.

Under the new partnership, the companies will deliver four official exams globally in four languages (English, Chinese, Portuguese and Spanish). IT professionals can sit the four exams – Android Application Development, Android Security Essentials, Monetize Android Applications and Training Skills for Android ATC Certified Trainer – at over 5,000 test centres in 178 countries.

Adnan Ibrahim, CEO of Android ATC, said, “This new partnership with Pearson VUE means we will have a secure, highly stable and global test platform that gives thousands of IT professionals around the world the opportunity to provide proof of their Android abilities. Our training courses have been developed and calibrated by a team of Android trainers and experts with several years of experience.”

Bob Whelan, President and CEO, Pearson VUE, added, “We are proud to be entering into this new partnership with Android ATC to deliver tests that will ultimately allow employers to differentiate between Android developers based on their certified abilities.”

ADDITION TO NUMERICAL LIBRARY The Numerical Algorithms Group (NAG) announces new functionality added to its numerical library for C and C++ programmers.

The new functionality included at Mark 24 of the NAG C Library brings the number of available functions to over 1,500, all of which are expertly documented and includes extensions in the areas of optimisation, wavelet transforms, time series analysis, random number generators, correlation and regression analysis, statistics and hypergeometric functions.

The new NAG C Library contains additional functions that have been added in response to customer requests, and further enhancements contributed by NAG’s expert developers and collaborators.

The inherent flexibility of the mathematical and statistical functions in the NAG C Library enable it to be used across multiple programming languages, environments and operating systems including Excel, Java, Microsoft .NET, Python, Visual Basic and many more.

PAGE 9

DIGITAL THE EUROPEAN SOFTWARE TESTER INNOVATION FOR SOFTWARE QUALITY

The Whole Story Print Digital Online

FOR DIGITAL ISSUES, DIRECTORY, COVER STORIES, POLLS AND MORE, VISIT testingmagazine.com

Published by T H I R T YO N E

www.31media.co.uk

Telephone: +44 (0) 870 863 6930 Email: info@31media.co.uk Website: www.31media.co.uk

THE EUROPEAN SOFTWARE TESTING AWARDS 2014

2 0 1 4

RECOGNISING AND CELEBRATING TECHNICAL EXCELLENCE

Sponsors

WHAT ARE JUDGES LOOKING FOR?

Supported by

As The European Software Testing Awards 2014 opens for entries, TEST Magazine finds out what some of the judges will be looking for this year…

CHRIS LIVESEY, GM OF PRODUCTS AND CHIEF MARKETING OFFICER, MICRO FOCUS “If last year’s awards were all about how our industry was dealing with continually increasing levels of governance and regulation, then this year is all about developing innovative ways of managing complexity. While the two issues are not entirely separate, they are different. Because to meet these challenges, we’re looking at the more tangible results of that innovation. I’m thinking about things like operating systems, platforms and browsers, here. How are the entrants using innovation to test in agile or mixed development environments? I’d also be interested to see entrants’ strategies around DevOps – how are they balancing this with the twin demands of process and control?

"This is the premier awards programme for this industry and the bar continues to climb. It’s not enough to ‘just’ have a great idea or solution. We’re in a fast-paced environment that shows no sign of deceleration, so winners must be able to collaborate with those around them. Show me the project or solution, fine, but if you want to win then contenders should be able to show how they communicate with multiple stakeholders, and truly get the business on board.

“Focus on the practises and the small, simple things that make a difference – things that others can learn from. Large complicated solutions don’t often lend themselves well to helping others learn. In other words, the big idea must be communicable to everyone. It’s this level of ‘deep dive’ knowledge that sets aside the winners from the worthy entrants. Like any good idea, it must bear close analysis, so pull it apart and try and break it. If you can’t, then you may well be on to something and we should be hearing from you.”

MAGAZINE

THE EUROPEAN

2 0 THE EUROPEAN SOFTWARE1TESTING AWARDS RECOGNISING AND CELEBRATING 4 TECHNICAL EXCELLENCE SOFTWARE TESTING AWARDS

Headline Sponsor

GWEN STEWART, QA MANAGER, AMAZON “I am hoping to see examples of holistic thinking that tackle quality for the end product and focus on customer experience.

Across the categories I’m hoping to see new ways of thinking and engaging with other teams across the whole lifecycle. As well as use of technology to support project deliveries; either through automation or using technology to facilitate communications, etc.

“For me, a clear example of a positive impact of change on the organisation or for the customer is what makes an award-winning entry. It’s always good to see lessons learnt along the way and that things were tried and then tweaked to improve things further. I would also like to see examples of innovation to solve problems that the team was facing.

"FOR ME, A CLEAR EXAMPLE OF A POSITIVE IMPACT OF CHANGE ON THE ORGANISATION OR FOR THE CUSTOMER IS WHAT MAKES AN AWARD-WINNING ENTRY." “This will be my first time judging The European Software Testing Awards and I am looking forward to seeing examples of how other people deal with the common issues that everyone in testing faces, as well as how the industry is being moved forward by specific individuals with good ideas.”

"FOCUS ON THE PRACTISES AND THE SMALL, SIMPLE THINGS THAT MAKE A DIFFERENCE – THINGS THAT OTHERS CAN LEARN FROM." JUNE 2014 | www.testmagazine.co.uk

PAGE 11

THE EUROPEAN SOFTWARE TESTING AWARDS 2014 MARK GALVIN, SYSTEMS ASSURANCE MANAGER, UNIVERSITY OF CAMBRIDGE “Personally, I feel that there should be a clear link between the entry and the category. This should be immediately apparent from the start. I expect there will be some incredible entries, but it is still important to correlate the entry to the category and demonstrate what defines it as unique and special.

"SOMETIMES IT IS USEFUL TO TELL IT LIKE A STORY REMEMBERING THAT THERE IS A BEGINNING, A MIDDLE AND AN END." “For me, it would be good to see entries that not only advance the originating product, team or organisation, but entries that are pushing the boundaries of testing – trying to shape and mould the future, enhancing the reputation of the wider testing community. The software testing profession is one we should be proud of and actively promote, so if there are examples of how entries have enabled this while delivering for the category itself, then they stand a good chance of adding something extra.

“Clear background information is important, sometimes it is useful to tell it like a story remembering that there is a beginning (the problem and the background), a middle (how you resolved the problem and the challenges faced) and an end (the outcome and how it will be improved over time).

"There is so much that we can share across the profession that it is genuinely exciting. I know that reading the entries will inspire me and the eventual winners will inspire the wider testing community.”

PETER HYAMS, TECHNICAL ASSURANCE, GLOBAL TECHNOLOGY PRODUCTION, DEUTSCHE BANK AG “When judging this year, I’ll be looking for something ‘different’ that stands out from the crowd. This could be a new testing technique, or a fresh way of using a particular tool.

“It helps if the entry is written up clearly and is easy to understand; many entries may represent something truly innovative – but it is lost in pages of description. I recommend keeping an entry broadly to three sections (the problem, the approach, the outcome).

"I’LL BE LOOKING FOR SOMETHING ‘DIFFERENT’ THAT STANDS OUT FROM THE CROWD." “Finally, be optimistic – you may well win! Our industry is wider than our own roles within it. TESTA is a reminder of the bigger picture.”

ROD ARMSTRONG, SENIOR DIRECTOR OF QA, EXPEDIA “TESTA 2014 will build on the success of the 2013 awards, highlighting the great achievements and exceptional efforts that teams/individuals have made in bringing our profession to its status of key partner in SW delivery, regardless of methodology employed.

“I am looking for entrants who have demonstrated application of the virtuous cycle: ‘Work Problem - Self Study - Knowledge Gained - Knowledge Sharing’. This will show how software testers can add real value to the delivery process and are not simply an obstacle to be overcome. The software industry is evolving at breakneck speed and entrants should show how they too are revolutionising what they do.

“A winning entry will show genuine creativity and industry awareness in solving a work problem, leading to not only individual success, but success of the entire team’s delivery. My advice is to keep your entries clear and simple, ‘what was the problem? What did you do? What were the results?’

“I am very excited about being part of the awards as I see them as a showcase of how far our competency has come, thus providing an inspirational and motivational message to those that engage. Best of luck to all who enter and remember, ‘Keep your Saw Sharp’.”

"MY ADVICE IS TO KEEP YOUR ENTRIES CLEAR AND SIMPLE, ‘WHAT WAS THE PROBLEM? WHAT DID YOU DO? WHAT WERE THE RESULTS?"

SHANE KELLY, HEAD OF QA AND TEST, WILLIAM HILL “I’ll be looking for entries from those who are passionate for making improvements in the field of testing.

People who have done research outside their current role and shown how they’ve applied what they have learned to make marked advances in how they work.

“In my opinion, a passion for testing makes an awardwinning entry. Having worked in the field for over 15 years, you can really spot those people whose core DNA is a passion for test; people who are fully committed to the role and how it can be continuously improved.

“Using real life examples to highlight accolades, rather than theoretical ideas, would be great to include in entries. Although the theory is very much important, I think showing how one has actually gone about affecting change (and the challenges therein) can really show a lot.

“I think it’s great to have a gathering such as TESTA, where people who excel in the field of testing can be recognised by their peers for the great work they do. Let’s celebrate achievement in the testing practice and how we all work together to continuously improve.”

"IN MY OPINION, A PASSION FOR TESTING MAKES AN AWARD-WINNING ENTRY"

PAGE 12

JUNE 2014 | www.testmagazine.co.uk

THE EUROPEAN SOFTWARE TESTING AWARDS 2014 PETER FRANCOME, HEAD OF TEST AND QUALITY, VIRGIN MEDIA “For the individual entries, I’ll be looking for passion and originality to really make a difference. For tools and innovation, I’ll be looking for the ability to really solve a business problem; and for the testing project entries, I’ll be looking at how the project was led, how the team worked together and with other stakeholders, and how the successful implementation helped deliver the business case and the contribution of test and quality was realised.

"MY ADVICE TO ENTRANTS IS DON’T RAMBLE! IF YOU THINK YOU HAVE A GOOD ENTRY – DO IT JUSTICE!" “Originality, passion to succeed, great team working and really furthering test and quality as a significant enabler in meeting project and organisation goals, is what makes an award-winning entry.

“My advice to entrants is don’t ramble! If you think you have a good entry – do it justice! Concentrate on those elements that will make your submission great (e.g. innovation, leadership, team working, communication). Accentuate the positives and the results achieved, and make sure it reads well; with a beginning, middle and end.

“It’s my first time as a judge and I’m really looking forward to reading about what’s great in our industry right now and what immense things people have achieved. I believe as test and quality professionals our contribution to organisation success is not always fully recognised, and I believe TESTA 2014 is a great opportunity to bring the spotlight on to those massively talented people who make up our profession.”

JAMES MURPHY, HEAD OF QA AT GUARDIAN NEWS & MEDIA “Entrants who have played key testing roles in teams that have successfully achieved an organisational goal or objective would certainly be well placed to contend for an award. These achievements could take the form of a successful release of a new software product, or the increase in business revenues through the consistent delivery of high quality product features, or even technical challenges such as the successful implementation of a continuous integration process.

“Take the time to define what the business objective or challenge you were trying to solve was. When dealing with complex testing solutions, it is quite easy to become so involved in the detail, that the problem is not clearly stated.

Clearly describing the problem, helps provide the context as to why the solution was valuable.

“TESTA is a great opportunity to recognise some of the excellent work that is being done within the industry across a variety of market sectors. I’m really looking forward to seeing about how different teams are taking on some of the toughest testing challenges in the industry, and the approaches and solutions they are implementing to solve them.”

"TAKE THE TIME TO DEFINE WHAT THE BUSINESS OBJECTIVE OR CHALLENGE YOU WERE TRYING TO SOLVE WAS."

JUNE 2014 | www.testmagazine.co.uk

ROB HORNBY, TEST MANAGER, JOHN LEWIS IT TESTA WINNER: THE BEST OVERALL TESTING PROJECT – RETAIL SECTOR & THE BORLAND EUROPEAN SOFTWARE TESTING AWARD “It’s interesting being on the other side of the fence, having won in 2013 but not seen the entries. I’m interested to see what everyone else is up to and also what I can learn. It’s going to be fun judging.

“I’ll be looking for innovation and simplicity and people doing this at scale. It is easier to do things on a small scale on small projects, but true skill comes in making the simple work on big projects and not losing sight of the simple in big teams. I believe that we need to be making testing less about the art and more about the science; utilise best practise from other industries and science, and apply to testing. Anything that you can do to make testing more visual appealing, is also something I will be looking for.

“In my opinion, something new, something simple while delivering a high degree of quality makes an awardwinning entry. A winning entry is something I want to take away and apply myself.

“My advice is to be bold, nothing is ever perfect and sometimes the learning from being innovative are as important as the innovation.”

"BE BOLD, NOTHING IS EVER PERFECT AND SOMETIMES THE LEARNING FROM BEING INNOVATIVE ARE AS IMPORTANT AS THE INNOVATION."

CHRIS AMBLER, DIRECTOR OF QUALITY ASSURANCE, TCSJOHNHUXLEY “I am very excited about the opportunity to judge TESTA 2014. I think recognition in our industry is well overdue and the stars of our craft should be rewarded by their peers.

“I would like to see well-structured explanations that tell a story and are easy to follow along with results that are successful to business delivery not testing process.

Innovation is paramount to a winning entry, either as an individual or a group. Testing needs to evolve with the times and cross new boundaries. Testers need to drive the business as a profit center, not a cost center. Entries that show these things will go a long way with me.

“My advice is to go for it! Publicising what you do will always provoke feedback at worst and you never know, meet the above criteria and get you the recognition you deserve. I really look forward to reading your entries and learning about other people’s experiences.”

"I WOULD LIKE TO SEE WELL STRUCTURED EXPLANATIONS THAT TELL A STORY AND ARE EASY TO FOLLOW ALONG WITH RESULTS THAT ARE SUCCESSFUL TO BUSINESS DELIVERY NOT TESTING PROCESS."

PAGE 13

THOUGHT LEADERSHIP SIVA GANESAN VICE PRESIDENT AND GLOBAL HEAD, ASSURANCE SERVICES, TATA CONSULTANCY SERVICES (TCS)

ENABLING “ASSEMBLY LINE” ASSURANCE Siva Ganesan, Vice President and Global Head, Assurance Services, Tata Consultancy Services, writes about the role of automation and instrumentation to enable “assembly line” assurance, which ensures efficiency in software development and value for the end consumer…

A

fter World War II, Toyota transformed car manufacturing by creating and applying a new production methodology, called “Lean” to its assembly lines. As you know, Lean is about creating value for the end consumer by cutting out all avoidable waste in the process and following principles of continuous improvement.

With many firms looking for ways to cut costs since the global recession, Lean has crossed the boundary from manufacturing to service industries. In Healthcare, for example, Lean is being used to deliver seamless patient experience, and to reduce length of stay and waiting time for patients. In banking and financial services too, Lean is being used to deliver high quality, customised customer service across different channels, and to control operational costs.

So how does any of this apply to software development and to assurance in particular? Well, though not entirely new to development (agile development already draws upon Lean principles), its application to assurance is relatively new. The key aspect is to be able to identify the “waste”. In software terms, waste refers to practices such as: unnecessary code, delay in the software development process and insufficient testing, leading to avoidable process repetition.

WHAT CAN BE ACHIEVED THROUGH “ASSEMBLY LINE” ASSURANCE? Understanding the waste we want to avoid, we can now apply automation and instrumentation, which are the key to enabling efficient “assembly line” assurance. But before we apply them, let’s see what we can achieve through assembly line assurance.

Using instrumentation and automation in an assembly line manner will help inject efficiencies into the way we conduct assurance, and will deliver: • Speed: Reduction in test cycle times, with quicker cycles of testing.

• Cost-effectiveness: Reduction in overall test effort using a right-first-time approach, which can reduce waste of time, effort and cost.

• Improved time-to-market: First-mover advantage to business, which helps enterprises in securing an increased market share.

AUTOMATION AND ASSURANCE In an ideal world, it is good to automate everything in the assurance space, but typically automation tends to be a lot lower due to a few perceived inhibitors:

PAGE 14

• Configuration of tools: Existing tools may not fit the need and so it is assumed that an area is not worth pursuing for automation.

• Adaptation of tools to context: Some phases of the test lifecycle lend themselves more to automation such as test execution – where currently, automation is at its highest. However, automation can also be applied to other areas, such as test data synthesis, environment creation and provisioning, and even metrics at an executive level.

• Cost of automation: As well as the cost of sustaining the currency of automation.

In all these cases, it is possible to customise a suite of tools for a specific situation, thus unlocking the value in the process and delivering best-in-class testing and validation.

However to justify this, we need to create a business case which demonstrates a decent ROI, typically over a two to three year horizon (in most cases, where there is a lot of manual and ad-hoc testing, there is usually a good business case). This is often the biggest inhibitor to automation, not because it isn’t possible, but because the opportunities to automate have not been understood.

WHAT ABOUT INSTRUMENTATION? Where does instrumentation apply, after one has identified the automation opportunities? It applies to all aspects of the assurance lifecycle, as “the all-in-one enabler”, bringing in efficiency, repeatability, reusability and quality all in one go.

Instrumentation is a deliberate attempt to ensure that there is algorithmic intent over and above human efforts in rendering an assurance task. Naturally, as in the real world, this then allows us to design for and predicate outcomes as opposed to stumbling upon them.

There are a multitude of technologies today that can intelligently manage and automate multiple processes to deliver a superior consumer experience, whilst minimising effort and cost. What we need to do is to identify the “waste”, uncover the opportunities for automation, and couple them with the benefits of instrumentation.

The bottom line being: the more you automate, the fewer chances of error, shorter waiting times, less rework and hence, more benefits; the more you practise instrumentation, the faster you can unlock the desired efficiencies in a predictable manner, provided, if both, automation and instrumentation are applied with a thorough business understanding of where and how they bring the most benefit. Only then will you have true endto-end assurance.

JUNE 2014 | www.testmagazine.co.uk

NEWS

SOFTWARE

THE GATEWAY TO SOFTWARE EXCELLENCE

FOR EXCLUSIVE NEWS, FEATURES, OPINION, PRODUCT NEWS, EVENTS, DIGITAL AND MUCH MORE VISIT: www.softwaretestingnews.co.uk

Published by T H I R T YO N E

www.31media.co.uk

Telephone: +44 (0) 870 863 6930 Email: info@31media.co.uk Website: www.31media.co.uk

THOUGHT LEADERSHIP P. VENKATESH DIRECTOR, PRODUCT DIVISION MAVERIC SYSTEMS

QTE – DON’T JUST SHIFT LEFT. START LEFT...

Research shows that eight out of 10 technology absorption programs don’t meet quality, cost and time goals “right first time”. But how does one improve quality without increasing budgets? In an interview with P. Venkatesh, director, Product Division, Maveric Systems, he says that a change in approach is all that is required to deliver critical quality goals…

W

hat are the obstacles faced by an organisation while trying to achieve quality?

P. Venkatesh: With software testing becoming more complex, organisations are taking more time to complete the process of testing in order to maintain high quality. This unfortunately can turn out to be extremely expensive, in terms of the time spent and the use of resources.

A recent industry survey that intended to understand the evolving role of testing within financial services, points out that 50% of the respondents felt that the time taken to complete system testing has increased in the last two years. Over 66% stated that they have spent about 25% of their project costs on system testing.

Organisations need a change in strategy and approach to create this balance between high quality and lesser

time-to-market; the answer to this a new paradigm – QTE.

What according to you is the new quality paradigm?

Why is now the right time to make this shift?

PV: The QTE (Quality, Time, Economy) paradigm is a framework to achieve high quality and reduced timeto-market. The dot com crisis moved software testing several notches up higher in the priority list. The financial crisis in 2009 moved the situation to the next gear; quality adherence became critical in technology programs in the CIO’s agenda. Today, quality as a function is no more pushed down the line.

Defect prevention rather than defect detection is the need of the hour. With a constantly evolving environment, organisations are under extraordinary pressure to make technology changes seamlessly.

Furthermore, with the old quality paradigm, there is an acceptance rate of 5% defect. However, organisations have now started to realise that these defects could cost them dearly and are starting to demand a 0% defect rate.

Therefore, I feel this is the right time to make the shift towards QTE.

How is the old quality paradigm different from QTE? Firstly, the old quality paradigm is such that it applies to specific projects and not to the entire enterprise.

Secondly, the old paradigm does not address the stakeholders involved in the entire ecosystem. Thirdly, there is no long-term investment (since it is within the context of a specific project, one cannot expect ROI within a short period). And finally, there is insufficient time to make necessary changes, which leads to a dilution of results.

To meet these challenges, there needs to be a change in the mind set; this is where QTE comes in. The first advantage of QTE is that it applies to the entire enterprise and therefore every project. The second advantage is that every internal and external stakeholder is involved in the quality assurance process. Lastly, QTE is a long-term investment; the organisation should recognise the fact that there will be no immediate ROI.

PAGE 16

JUNE 2014 | www.testmagazine.co.uk

THOUGHT LEADERSHIP

What are the elements that make up the QTE paradigm? Please elaborate.

PV: The elements of the QTE paradigm are Quality, Time and Economy. To elaborate: Quality “starts left” i.e. unlike traditional software testing where software testing vendors get involved right before the application goes live, the QTE paradigm addresses historical issues and ties it up with necessary requirements, both functional and technical. Here, each specification is reliable and detailed so that improved coverage can result in assuring quality. Short-term outcomes are established to enable committed focus and investments in the long-term.

Time implies faster time-to-market. This means cutting down on time spent at every stage of testing, and automating functions that consume the most time. Such a mechanism would result in maximum benefit and lessen the overall timelines required for quality assurance at all stages – requirements, program and application, hence reducing costs and use of resources.

Finally, Economy is at the enterprise level and more longterm focused as compared to “cost”. Accountability has now shifted to the vendor therefore the stakeholders from the customers’ organisation are involved only for validation and decision making. They are instead enabled to focus on their core business priorities.

What is required by an organisation to adopt a “start left” approach?

QTE is a new mindset and the shift to a new paradigm.

This means organisations need to change the way they look at testing. This includes: • Identifying testing/quality as a critical strategic objective.

• Allocating sufficient budgets.

• Involving all critical stakeholders – both internal and external; business heads, C-level executives such as the CIO/CTO/COO and heads of various silos in technology, program, infrastructure, development and testing etc. Also, external stakeholders such as vendors and support partners should be involved In addition to this, the organisation should take a leap of faith and understand that it would take 24 to 36 months to get the required results depending on the issues and investments involved. It is always advisable to have short-term results planned so that people can see the difference early. Finally, the organisation needs to have a stable, comprehensive and detailed framework in place to achieve quality.

What would be some of the key drivers of QTE? While deploying QTE, an organisation should have these key drivers in place: • Secure the future – The organisation should start by detailing requirements and have specification and

JUNE 2014 | www.testmagazine.co.uk

test cases at every level as acceptance criteria. This way, issues are eradicated at every stage instead of being accumulated, just before the system goes live.

• Address current issues – Once defects are identified, it is wise to determine where the issues are – at the requirements, test design, development, data or infrastructure stages.

• Reduce time – It is preferable to automate functions that consume a lot of time. This helps in minimising effort and time, and thereby allows comprehensive testing as well as faster time-to-market.

With regard to quality, time and economy, the organisation should prioritise them in the same order.

Quality will help set the ball rolling and therefore achieve the ultimate objective. Automation of some processes will enable faster time-to-market and, economy is at the enterprise level and more long-term; reduction of resources and automation help optimise spending.

What is the market acceptance for QTE today?

Customers are looking for a result-based model that ensures quality assurance right from the requirements stage to the time of release. Organisations today have begun to realise that to get ROI from a model like QTE is quite long-term; this shows that they are willing to step out of the conventional paradigm. A recent study on the testing industry also shows that more than 70% of the respondents would like to change the way they conduct testing. We definitely believe that the market is ready to accept the QTE paradigm.

Who can offer QTE? What are the prerequisites to become a QTE vendor?

A vendor who is a domain expert along with consulting capabilities and can partner with a client right from identifying historical issues and requirements, until the “time-to-market” phase would be an ideal partner to provide QTE. Deploying QTE involves “early technology lifecycle” services relating to requirement, program and process assurance.

For instance, quality issues in an IT system do not occur right before it goes live, but occur at every stage.

Maveric’s data over the last 10 years of our assurance journey indicates that in 50% of the projects up to 35% of the IT budget is wasted due to rework caused by poor definition and management of requirements.

Therefore it is important to maintain quality right from the requirements stage. In an ideal situation, 25% - 28% of the IT project time should be spent on writing requirements i.e. technical and functional specifications. However, in reality, less than 10% of the total time is dedicated to this exercise.

Until now, quality assurance predominantly came under the purview of software testing companies that did “defect detection”. Today, clients have begun to understand the scope for “defect prevention” to achieve QTE.

PAGE 17

Software

When it comes to crash testing We’re no dummies

Comprehensive Software QA With access to digital content so readily available to your customers with the likes of Android, Apple and Microsoft putting instant access into the hands of more and more users across a huge range of devices every day, it has never been more important that your software works without failing. We’ve years of expertise ready and waiting to give you the simple and effective testing solutions you need to make sure your software and content delivers a bug-free, seamless and intuitive experience for your customers.

For further information and to download our complementary white paper on software testing please visit:

www.testroniclabs.com

Manual and Automated Testing • Test Strategy • Compatibility • Back Office Validation • Mobile Testing • Website and Browser • HTML • Exploratory and Technical Testing • App Store Compliance • Load and Performance • Accessibility • User Experience

AUTOMATED TESTING

SEAN RAND SENIOR TEST ANALYST, TH_NK

IS AUTOMATION TESTING THE FUTURE OR A FAD? Does automation have a place in every test team and is therefore the future of testing?

Or is it for the technically more advanced testers to help add to the traditional manual test process, therefore being a fad? Asks Sean Rand, senior test analyst at TH_NK…

D

igital changes testing workflows continually, but the desire to achieve quality, in a testable and quantifiable fashion, remains constant.

Every company wants to hit its “quality gates” quickly and effectively to maximise output and reduce costs, and this drives management decisions towards automation.

So is automation the future or a fad? Writing test cases in MS Word proved to be a fad, falling off the radar many a year ago (thankfully), so will automation go the same way?

Many developers use automation testing on a daily basis.

Most organisations shipping software already use it as standard. Look to the field of OAT testing where load, performance, penetration and web service testing are all automated to attain quality asserts – all utilising the power of repeatability at a machine level. Yet automation is being dressed up as a revolution in testing. Automation is nothing new and, as testers, we shouldn’t be surprised or alarmed by it.

Certainly boundaries are being pushed – with testers entering into a testable world of development to manipulate page object models from a more user-based test approach – but we’re not talking about a new discipline within test.

NO SUBSTITUTE FOR MANUAL TESTING Manual testers with no coding skills or automation experience could be forgiven for their scepticism.

Automation can’t carry out an exploratory test, where manual testing excels. Exploratory testing needs a combination of experience, attention to detail, understating of current standards and the overall solutions brief. It’s a powerful method, which shouldn’t be overlooked in QA. Equally there will always be elements of manual testing within any web-based solution, which holds a degree of user experience – as there’s no substitute for the human eye when looking at behaviours and the intricacies of web solutions. I call this extremely skilled area of testing

WITH AUTOMATION, SOLUTION TESTS CAN BE EXECUTED IN A LINEAR FASHION OR IN PARALLEL, WITH A 24-HOUR TEST WINDOW EACH DAY IN WHICH TO TEST. LENGTHY AND LABORIOUS DATA-DRIVEN TESTS CAN HARNESS THE POWER AND SPEED OF AUTOMATED TESTING TO FREE UP TESTERS

JUNE 2014 | www.testmagazine.co.uk

PAGE 19

AUTOMATED TESTING

EXPLORATORY TESTING NEEDS A COMBINATION OF EXPERIENCE, ATTENTION TO DETAIL, UNDERSTATING OF the “Clever Silly Users” scenario, where CURRENT STANDARDS AND THE hardware overhead housing multiple automation can’t adequately replace OSs and browsers, plus the ongoing OVERALL SOLUTIONS BRIEF. manual QA testing. continual upgrades needed and IT’S A POWERFUL METHOD, provided by an Ops team.

Natural behaviours and interactions are WHICH SHOULDN’T BE difficult to code for, since they unearth the Automation allows tests to be executed OVERLOOKED IN QA most notorious and difficult bugs that are hard in the cloud on actual designated virtual to replicate quickly in automation. Effective automated exploratory testing would be near on impossible to code without infinite time.

But manual testing is getting more automated, and manual toolsets can be configured for semi-automated solutions, driven by the execution of manual scripts.

Microsoft Test Professional is a prime example – it can be configured to record and create action scripts, which can then be automated. And it’s being pushed subtly into toolsets which you might not expect a manual tester to use.

machines, housing real browser versions and OS versions – without any of the associated hardware implications, constraints or internal Ops team involved.

Profit margins and test time can make or break a project’s profitability at times, and here’s an example of how we weighed up the cost implications of manual vs automation testing on a recent project.

Plus, discrete behaviour-driven tests can be built through frameworks such as Selenium, which allow powerful automation test suites to work even in an agile environment. This allows the tester to link business requirements through to the actual test source code.

We allowed for five days’ worth of test time for one tester, on a £500 day rate. We needed to create and execute 20 test cases in three cycles. Working on the basis that a manual test case takes 30 minutes to script and a further 15 minutes to execute, and an automation case takes 45 minutes to script and two minutes to execute, the below demonstrates the cost benefits of automated vs. manual testing:

BDD TEST AUTOMATION

Manual testing

With automation, solution tests can be executed in a linear fashion, or in parallel, with a 24-hour test window each day in which to test. Lengthy and laborious data-driven tests can harness the power and speed of automated testing to free up testers. Manual tests are prone to user error and become more so the more you repeat, so if you have to repeat a set of test steps more than twice, it makes sense to automate it.

With a team of experienced and mature testers with automation experience – plus good training – BDD scripts can be quicker to script up (depending on the framework) than manual tests, which means you can produce the same script in the same or less time than manually, and reap the rewards that automation brings to both the QA team and the wider business.

Each can be carried out in a repeatable way alongside the start-up and tear down extensions, ensuring a test finishes in exactly the same state it begun. This is a major advantage of automation since the tear down of a test rarely happens with manual testing: so automation helps extend best practices (such as tear downs) throughout an organisation.

COST BENEFITS OF AUTOMATION Testing against multiple operating systems, browser versions, mobiles and tablets can create a large

PAGE 20

Time to script Time to execute three test cycles Total time Allocated time (five days, 5hr in a day) True cost of the tester

= 10 hours = 15 hours = 25 hours = 25 hours = £2,500

Automation testing Time to script Time to execute three test cycles Total time Allocated time (five days, 5hr in a day) True cost of the tester

= 15 hours = 2 hours = 17 hours = 25 hours = £1,700

This points directly to a future of automated testing, but I think elements of testing will always remain manual.

Testers can make better use of automation – when the tasks are repetitive, the burden on hardware is heavy, and the cloud makes economic sense – but manual makes more sense in exploratory testing and when human interaction is important, such as in user-experience behaviours.

Automation is the future and no fad: like any new digital approach, the concern will always be that the machines will take over, but I don’t see the manual approach disappearing altogether anytime soon.

JUNE 2014 | www.testmagazine.co.uk

THE EUROPEAN SOFTWARE TESTING AWARDS

2 0 1 4

RECOGNISING AND CELEBRATING TECHNICAL EXCELLENCE

ENTRIES ARE NOW OPEN VISIT WWW.SOFTWARETESTINGAWARDS.COM The European Software Testing Awards is an independent awards programme designed to celebrate and promote excellence, best practice and innovation in the software testing and QA community Why Enter?

• An opportunity to be acknowledged and rewarded • With such recognition comes self belief, the breeding of success, and the attraction of fresh and bright talent • High levels of exposure and acclaim for each individual, team, and organisation, as well as the profession as a whole • Impress partners, clients and investors • Network, interact and learn from Europe’s software testing elite.

Here are some of the 2013 winners: •

John Lewis IT

•

Close Premium Finance

•

Lloyds Banking Group in partnership with Cognizant Technology Solutions

•

Jaspersoft

•

TechMahindra

LMAX Exchange

•

Knowit Oy

•

Could you be a winner this year? Enter now!

Sponsors

Supported by MAGAZINE

Headline Sponsor

TEST AUTOMATION AMIR GHAHRAI TEST AUTOMATION MANAGER HOTELS.COM

BARRIERS TO TEST AUTOMATION Amir Ghahrai, test automation manager at Hotels.com, examines some of the most common barriers to test automation and how these challenges prevent organisations from succeeding in this area of testing…

I

t is not difficult to imagine the benefits of having automated testing alongside product development – faster releases, increased test coverage, frequent test execution, faster feedback to development team, just to name a few, yet many organisations have not made the move or are resistant in investing in test automation.

Possibly the most difficult and challenging aspect of any test automation endeavour is to understand the limitations of automated testing and setting realistic goals and expectations to avoid disappointments. With that in mind, let’s see some of the most common misunderstandings about automated testing:

AUTOMATED TESTING IS BETTER THAN MANUAL TESTING Referring to Michal Bolton’s blog post Testing vs.

Checking, automated testing is not really testing. It is checking of facts. When we have an understanding of the system, we can enforce that understanding in the form of checks, and then by running the automated checks we confirm our understanding.

Testing, on the other hand, is an investigation exercise where we aim to obtain new information about the system under test through exploration. Testing requires a human to make a sound judgment on the usability of the system. We can spot anomalies when we were not anticipating.

We should not be leaning entirely towards one or the other, as both methods are required to get insight to the quality of the application.

ACHIEVING 100% AUTOMATED TESTING Just as there is no practical way of achieving 100% test coverage (due to endless possible permutations), the same applies to test automation. We can increase test coverage by running automated tests with more data, more configurations, covering a whole variety of operating systems, browsers, but achieving 100% is still an unrealistic goal.

When it comes to test automation, more tests do not necessarily mean better quality or better confidence. It all depends on how good a test is designed. Instead of aiming for a full coverage, instead focus on the most important area of functionality, which is crucial to the business.

ALTHOUGH MANY OF THE VENDORSUPPLIED AND HOMEBREWED TEST AUTOMATION SOLUTIONS ARE VERY SOPHISTICATED AND HIGHLY CAPABLE IN PERFORMING COMPLEX OPERATIONS, THEY WILL NEVER BE ABLE TO COMPETE WITH THE INTELLIGENCE OF A HUMAN TESTER

PAGE 22

JUNE 2014 | www.testmagazine.co.uk

TEST AUTOMATION QUICK ROI When implementing a test automation solution, there are more interrelated development activities rather than simply scripting test cases. Normally a framework needs to be developed that can support bespoke operations, which are useful and meaningful for the business, such as test case selection, reporting, data driven, etc.

The development of the framework is a project on its own, and requires skilled developers and takes time to build. Even when a fully functional framework is in place, scripting automated checks initially takes longer than executing the same test manually. Therefore, when we require a quick feedback on the new feature that’s just developed, checking it manually is usually quicker than automating the test.

However, the ROI is returned in the long run when we need to execute the same tests at regular intervals.

HIGHER RATE OF DEFECT DETECTION THROUGH AUTOMATED CHECKS Although many of the vendor-supplied and homebrewed test automation solutions are very sophisticated and highly capable in performing complex operations, they will never be able to compete with the intelligence of a human tester, who can spot unexpected anomalies in the application while exploring or executing a set of scripted tests against the system under test.

Ironically, people expect automated testing to find lots of bugs because of allegedly increased test coverage, but, in reality, this is not the case. True, automated tests are good at catching regression issues – after a new feature has been added to existing code base, we need to ensure that we haven’t broken current functionality and we need that information fast – however, the number of regression issues, in most cases, tends to be far less than issues in the new functionality being developed.

Another point to bear in mind is that the automated checks only check what they have been programmed to check by the person who wrote the script. The scripts are as good as the person who wrote them.

All automated checks could happily pass, but major flaws can go unnoticed, which can give a false impression of the quality of the product. In essence, checking can prove existence of defects, but it cannot prove their absence.

WE ONLY REQUIRE UNIT TEST AUTOMATION So, if the likelihood of finding defects is greater in testing new features, why are we not running our automated tests against the new functionality as it is being developed?

Well, this is somewhat the case for teams that practice TDD. The developers write a unit test first, watch it fail and then write enough code to get the unit test passed, and the cycle is repeated until the intended functionality is delivered. In essence, these automated unit tests are checking new functionality and overtime they form the unit regression pack, which is executed repeatedly as new functionality is delivered.

There is a caveat to this, TDD is highly encouraged and a strong development practice in building quality from the grounds up, whilst unit tests are only good at finding programmer errors, not failures.

There is a much larger aspect of testing that happens

JUNE 2014 | www.testmagazine.co.uk

when all the components are tied together and form a system. Many organisations have the majority of their automated checks at the system UI layer. However, scripting automated checks for the UI or system layer, while the features are being developed is a daunting task, the new functionality tends to be subject to many changes and effort maybe wasted.

WE ONLY REQUIRE SYSTEM UI AUTOMATION There are values in running automated checks at system level. We get to see what the user experiences when interacting with the application; we can test end-to-end flows and third-party integration, which we could not test otherwise; we can also demo the tests to clients and endusers so they can get a feel of test coverage.

Relying solely on automated checks at UI layer has limitations. UI is constantly changing to enhance visual design and usability, having automated checks failing due to the UI changes and not changes in the functionality can give a false impression of the state of application.

UI automated checks are also much slower in speed of execution than at unit or API layer and because of this, the feedback loop to the team is slow. It might take a few hours before a defect is spotted and reported back to the developers.

Understanding the context of each test and at which layer the test should be automated is important. Test automation should be part of the development activity, so the whole team is responsible for test automation, with developers writing and executing unit tests, software developers in test writing, executing and maintaining acceptance tests at API and/or UI.

LOSING FAITH AND TRUST IN TEST AUTOMATION You spend many hours developing a perfect test automation solution, using best tools and best practices, but if the automated checks do not help the team it is worthless. If the team has no visibility or knowledge on what is automated and executing, they either release with fear of unknown or duplicate their regression testing efforts. If the automated checks are flakey, slow or give intermittent results, then it can confuse the team more than providing confidence of the health of the system under development.

Don’t be afraid of removing automated checks that are always failing or give inconsistent results. Instead aim for a clean and reliable suite of tests that can give correct indications of the health of the application.

Test automation is a long-term investment. It will take time and expertise in developing and maintaining their frameworks and scripts. It is not a one-off effort where you deliver a solution and let it run; it needs constant monitoring and updating. Rather than aiming to replace manual testers or expecting the automated checks to find lots of defects, we should instead embrace the advantages test automation brings to the team, such as liberating testers time for more exploratory testing where chances of revealing defects are maximised, or using automated scripts to create test data that can be used for manual testing.

Understanding the limitations, and setting realistic expectations is important in overcoming the barriers and challenges of test automation.

PAGE 23

AGILE PROCESSES MUKESH SHARMA CEO QA INFOTECH

EVM IN AGILE: WHAT’S IN IT FOR SOFTWARE TESTING? Mukesh Sharma, CEO of QA InfoTech, discusses Evidence Based Management (EVM) in agile processes, and how this next major evolution in software testing brings freedom and accountability…

T

he industry at large has accepted and embraced agile development methodologies well for more than a decade.

When Ken Schwaber, one of the co-developers of the Scrum process and a signatory of the Agile manifesto, was asked in a recent interview on what next to expect in the Scrum framework, he talks about Evidence Based Management (EVM) – a management process that lends itself to metrics-driven operations to measure and prove what teams have been working on.

While the faster time-to-market and product acceptance in the marketplace are solid outcomes of the team’s effort, he sees EVM as the next major evolution to bring freedom and accountability amongst product team members – freedom from having to base decisions on intuitions and accountability in one’s actions shown through measurable results, bringing in overall business value.

THE RIGHT METRICS While the value of such an EVM effort is undoubted, the challenge would be around picking the right set of metrics for every discipline. This is to ensure they are adequate to showcase the team’s value, yet are not a huge overhead taking away a chunk of the team’s time in measuring outcomes as opposed to delivering outcomes.

Also, these are metrics that not just showcase value, but also ones that can highlight any off-track team performance to help them correct their course of action at the appropriate time. And while a core framework can be put in place to determine what those metrics to measure on an ongoing basis are, they are far from being static.

They need to be a set of metrics that can be revisited time and again to see if they still align to the larger goal of providing “evidence”, and more importantly help take

METRICS IS NOT A NEW PHENOMENON TO PRODUCT TEAMS, ESPECIALLY TEST TEAMS. WHAT IS NOW BECOMING IMPORTANT IS HOW TO BALANCE THEM WITH THE CORE PRODUCT TASKS WHEN THEY ARE ALREADY WORKING WITHIN TIGHT TIMELINES

PAGE 24

JUNE 2014 | www.testmagazine.co.uk

THE EUROPEAN SOFTWARE TESTER

EXECUTIVE DEBATES

Offering you the key to successful solutions

• One-day event • Monthly • Lunch & refreshments provided • Central London venue • Network with like-minded individuals • Cutting edge content

For more information, contact Swati Bali on +44 (0) 203 668 6946 or email: swati. bali@31media.co.uk

Organised by 31 Media, Publishers of VitAL Magazine www.31media.co.uk

T H I R T YO N E

AGILE PROCESSES

EVIDENCE BASED MANAGEMENT (EVM) – A MANAGEMENT and that the testing team is expending actionable steps based on the values PROCESS THAT LENDS the right effort in building “automation of they provide. So, what do we gather ITSELF TO METRICS-DRIVEN true value”. Such ongoing measurements, from all of this? We understand these also come in handy to correct course metrics need to be actionable; are OPERATIONS TO MEASURE of action in case of deviations from the more dynamic than static; and need AND PROVE WHAT “value path”, making corrections easy an engine for an optimal ongoing TEAMS HAVE BEEN and affordable. maintenance. Now, what are these WORKING ON metrics, specific to the testing discipline that we as testers need to care about?

At a broad level, test effort metrics and product quality metrics, are two broad heads that directly map to the testing team’s work – one that directly relates to the quality of the effort they have been putting in; and, the other ,the quality of the product that they have been testing.

Again the goal is to keep both these categories simple, manageable yet powerful in helping bring out the “freedom and accountability” EVM strives for. While these two heads by themselves are not very new to testing groups, what values we measure under each of them is what is changing over time. For example, under test effort metrics, values such as code coverage, defect removal and containment, test automation effectiveness, manual vs. test automation value comparison are all metrics that are becoming inevitable by the day.

CODE COVERAGE Let’s take the case of code coverage. A decade back, code coverage test run was an optional test run typically taken up at the end of the testing lifecycle (by which time the full test pass, along with the automation suite have taken shape). This run, time permitting, when taken up, showed the overall test coverage percentages and also details around dead code, which most often was punted for analysis until next release for want of time.

PRODUCT QUALITY

Moving to the next head, the head of product quality, where we measure outcomes to map to the product quality’s exit goals defined in the test strategy. Herein, some age old metrics never lose value – ones such as defect analysis by module, priority, severity, type, how found etc. Defects and the data they provide are such valuable indicators of the product health that they have been, and will continue to be, used in any effort that attempts to objectively understand product quality.

Along with these traditional metrics, some external facing metrics are becoming a need of the day to differentiate a product in the marketplace. These are metrics around how one’s product is faring against competition (on performance, feature richness, security, usability grounds, to name some main ones) to give the product a fair chance to better compete amongst other players.

Finally, metrics that measure end-user satisfaction, asking for their direct inputs around varied aspects of the product, including the product’s quality. These together are metrics that show the team has done its due diligence and homework of developing the product not just with an internal focus, but also looking at what the external drivers are, which together help improve its chances of success in the marketplace.

But with the agile cycles, most teams are encouraged to take on code coverage-based runs from the initial testing days, which helps not just testers beef up their coverage in specific areas, but also provide an opportunity to help developers make their code more robust and clean.

Similarly, test automation is no more a number game that asks what percetage of tests were automated. It is a very specific and focused test effort that now asks what value I provide in improving product quality, saving manual test cycles and bringing in more repeatability in the test effort.

Metrics is not a new phenomenon to product teams, especially test teams. What is now becoming important is how to balance them with the core product tasks when they are already working within tight timelines, and how to map them to actionable evidence that the management can relate to. Sooner or later, these metrics could become the drivers for organisations that want to implement “service level agreements”; both internally for their own management reasons and externally to their user base to bring in a “commitment based delivery,” which will help define user expectations for the product.

To this effect, metrics that measure the value of test automation results, the kind of bugs reported by automated runs, how they fare in comparison with manual test results are all taken into account in determining the “evidence” that automation is valuable,

With all of these drivers slowly but deeply entrenching the development world, now is the time for test teams to revisit our “evidence” lists (aka our testing metrics) to ensure we gear for the upcoming days of “Evidence Based Management” within the Scrum framework.

PAGE 26

JUNE 2014 | www.testmagazine.co.uk

TESTING & PERFORMANCE. CONNECTED.

As our world connects itself, we embrace a new identity in the form of our brand and purpose - inspired by the past and motivated by the future.

Tech Mahindra's Global Test Factory is the next generation concept in software testing, providing a superior and flexible alternative to standard offshoring model. It is a multi-location based testing center of excellence that combines best-in-class test processes, people, techniques, facilities and methodologies. It also offers a full range of scalable testing services, cutting-edge innovation along with optimum unit cost of testing..

Welcome to a connected world of connected solutions to connect better with your customers, improve your customer experience and optimize operations.

Connect with us: http://www.techmahindra.com or write to connect@techmahindra.com

AGILE PROCESSES VIJAY BALASUBRAMANIAM GLOBAL HEAD OF TESTING PRACTICE ITC INFOTECH

HARDEEP GAREWAL PRESIDENT, EMEA OPERATIONS ITC INFOTECH

TESTING: OFFSHORE AND AGILE? If offshore, outsourced testing organisations want to win and retain business they need to be working closer with their clients, explains Vijay Balasubramaniam, global head of testing practice, and Hardeep Singh Garewal, President, EMEA operations, at ITC Infotech...

AGILE CUTS ACROSS ALL THE LAYERS IN THE DEVELOPMENT PROCESS. IN ORDER TO BE TRULY AGILE, YOU WILL NEED TO HAVE THE WHOLE METHODOLOGY OF DEVELOPMENT AND TESTING IN THE AGILE FORMAT

PAGE 28

JUNE 2014 | www.testmagazine.co.uk

AGILE PROCESSES

S

ince its launch in 2001, the Agile Manifesto (AM) has taken the software development industry and testing organisations by storm. The Manifesto was set up by 17 developers to define an approach for working software, customer collaboration and responding to change.

WITH THE MANTRA OF “TEST EARLY, TEST OFTEN” HOWEVER, IT hardware devices that need to be IS DIFFICULT TO SEE HOW taken into account. And beyond this paradigm shift, the complexity ANY OFFSHORE TESTING is also ramping up because of the ORGANISATION CAN EVER emergence of challenges like the TRULY ADOPT THE AGILE need for reduced lead times, where MANIFESTO WHOLESALE. agile has played a major role in the IS THAT EVEN A SENSIBLE last 13 or so years.

EXPECTATION? In amongst all this complexity and

With the parallel growth of the Internet and external, offshore software testing services, suppliers are now offering 24/7 follow-the-sun capabilities and highly efficient customised testing services to their partners.

With the mantra of “test early, test often” however, it is difficult to see how any offshore testing organisation can ever truly adopt the Agile Manifesto wholesale. Is that even a sensible expectation?

BREAKING DOWN THE BARRIERS The kind of changes that the agile model has brought to the software development lifecycle (SDLC) has broken down the walls between all the stakeholders and processes involved in development.

Previously, agile testers used to be involved after development, whereas now a true agile project does not have testers, developers and requirements managers in separate teams, it has them all co-located and closely associated with no lines of demarcation. It is going to be difficult for an offshore organisation to adapt to this way of working.

What has not changed from an outsourced business perspective, however, is the non-functional testing aspects: reliability testing, security testing and user acceptance testing. All of these require a substantial amount of functionality to already be in place before they can be undertaken, and are still needed towards the end of the development process.

So now all the non-functional aspects of testing get outsourced. In practise, every few sprints of all the nonfunctional aspects can still be taken care of in their own sprint by offshore third-party organisations. But clearly the closer in culture, approach and knowledge-base these organisations are to the client, the more productive and agile the relationship will be.

Agile cuts across all the layers in the development process. In order to be truly agile, you will need to have the whole methodology of development and testing in the agile format.

CHANGING WORLD There is an irony for testers in this rapidly changing and highly fluid industry that as a lot of the software development work becomes relatively easier because of its commoditisation – as it shifts to a Software as a Service (SaaS) or cloud-based delivery model – the testing becomes ever more complex. This is due to the proliferation of platforms, operating systems and

JUNE 2014 | www.testmagazine.co.uk

change, one of the most challenging and dynamic markets is the banking, financial services and insurance (BFSI) sector. This is one of the prime areas that ITC Infotech has been effective in deploying agile techniques. We have recently been involved in a complex agile project for a UK-based Internet bank with over one million customers and a range of financial products.

The bank was suffering from the effects of slow testing and development cycles, which had in turn slowed down its time-to-market with the kind of new products and services that customers are now increasingly demanding.

The bank desperately needed to get its products to market quicker and this is where an agile approach came into play.

We worked with the client for over five years on an endto-end testing exercise for one of its savings account products. Today we are the client’s sole testing partner, as well as being a trusted development partner on several projects (including many complex areas such as mobile services and faster payments and current accounts).

We set up a dedicated offshore Testing Centre of Excellence (TCoE) to handle all of the bank’s product lines, as well as its day-to-day banking activities. Apart from pure-technology product development and testing, we also conducted a marketing-driven initiative intended to improve the web service user-experience. We leveraged agile methodologies to build a web-service platform that is user-friendly and intuitive, and is also a powerful marketing tool through which the client can cross-sell its range of products and services to the existing customer base. This also saw an enhanced and richer interface to appeal to its customers.

THE GOOD NEWS FOR TESTERS The good news is that in this ever more complex software development environment demand for testers, agile or otherwise, will increase. New breeds of testers, who understand technology and development, will be highly sought after. While automation will increase in importance and push way beyond its present boundaries, it can never take the place of the human touch because the systems we test are designed by humans to be used by humans.

That’s why it stands to reason that you will need more human intelligence to test these aspects alongside the increases in efficiency brought by increased automation.

These testers will have to work more closely with their clients, making closer, more agile partnerships an increasingly prevalent part of the software development world.

PAGE 29

FOCUS ON: THE WORKPLACE EMMA CLARK PARTNER ABBISS CADRES

NEW LEGISLATION: FLEXIBLE WORKING At the end of this month, a new legislation will extend the right to request flexible working to all eligible employees. But how will these laws impact your organisation and how can you best prepare for the change?

Emma Clark explains…

I

n a couple of weeks, new laws will permit any employee with over 26 weeks’ service to make an application to their employer to work flexibly for any reason. The Government hopes it will encourage more employees to be able to work around their civic responsibilities, and an employee can make the request for any reason.

DEALING WITH THE REQUESTS Having reflected on the viability of the request, employers may reject the flexible working request, as well as the likely risk of a potential sex discrimination claim or a breach of the Flexible Working Regulations 2002.

Following receipt of a flexible working request, an employer must currently follow a tight 28-day timeframe in which to consider the request; meet with the employee; and then have a further 14 days in which to make a decision.

From next year, employers only need to respond to the requests in a “reasonable manner” and within a “reasonable” period of time. Whilst seemingly helpful for employers, it does introduce uncertainty by comparison with the current procedure, which may result in more, and not less, litigation.

The Government has said it will introduce guidance on this point, which may go some way towards providing a format understood by both employees and employers.

THE CHALLENGES • Competing requests from team members As an employer, what if you receive one flexible working request from a working mother and the other from a woman who wants to work flexibly in order to write a novel, or a man who wants to coach the school rugby team?

Realistically, against the backdrop of a potential sex and disability discrimination claim from some groups of employees, value judgments will be made by employers when they face competing requests from team members. Together with the fact that the European Parental Leave Directive 2010 confers the right to request flexible working to parents returning from parental leave, employers do need to consider giving priority to some classes of employees.

This is not the Government’s intention: it expressly wants to avoid a tiered right. It believes any prioritisation would be counter-intuitive as it would simply reinforce

PAGE 30

the view that flexible working is primarily for parents and carers.

AS AN EMPLOYER, WHAT IF YOU RECEIVE ONE FLEXIBLE WORKING REQUEST FROM A WORKING MOTHER AND THE OTHER FROM A WOMAN WHO WANTS TO WORK FLEXIBLY...

AN AGILE APPROACH Now may be the right time for organisations to consider the creation of an agile workforce – is it not time we looked beyond the wording of the flexible working legislation?

The current law on flexible working is seen by many employers as a benefit to working parents and carers, and not about a better way to do business. Yet, many of those organisations who have embraced and promoted flexible/ agile working report higher levels of productivity, improved retention rates and better employee engagement.

Working life has changed. Against the backdrop of everevolving advances in IT systems and the global race for talent, many employers are seeking to retain talented people no matter where they are located or how they wish to balance or "merge" their professional and personal lives.

WHAT’S YOUR STRATEGY? Organisations should focus on their flexible working policies and strategies in preparation for the imminent changes, if they haven’t done so already. Businesses may wish to scope out the impact of a move into a smarter way of working.

If you’re considering making changes to your business model, it’s worthwhile examining a number of factors, including: • The employment law implications.

• The most effective method in which to communicate the change to staff.

• How best to manage any barriers to change.

• Whether the appropriate models are in place to ensure and constantly monitor success.

JUNE 2014 | www.testmagazine.co.uk

Software Testing Network Strength in numbers www.softwaretestingnetwork.com

Membership beneďŹ ts include: Series of one day debate sessions High-brow webinar streams Research & industry ďŹ ndings Exclusive product discounts Peer-to-peer networking Annual gala dinner And so much more...

Becoming a member of the Software Testing Network joins you together with like-minded professionals that are all striving for technical excellence and championing best practice and process

FOCUS ON: THE WORKPLACE

10 SKILLS YOUR EMPLOYEES WILL THANK YOU FOR Whether you employ one person or 1,000, there are 10 skills that no employer or manager should be without. If used appropriately, they will enhance your work environment and make your team happy, says Christina Hession, from Toastmasters International…

1.

Critical thinking: A successful critical thinker gathers information before interpreting it, analysing it and using logical reasoning to reach conclusions. To become a better critical thinker, you need to be informed, avoid making decisions too early, have an open mind, don’t be afraid to ask questions, weigh opinions against facts and consider and analyse all options.

2. Listening: By acquiring good listening skills you’ll receive more and better, information. This will help you to identify and clarify issues, make decisions, resolve conflict and be creative.

3. Time management: Budget your time and accomplish tasks and projects efficiently by: identifying short- and long-term goals, making a daily to-do list, prioritising the most important items on the list, making a schedule, delegating where possible, leaving time for unexpected tasks and managing interruptions.

4. Giving feedback: It is vitally important for team members to know what they are doing well, what they are not doing well and what they need to do to improve. Allowing a team member, who is not pulling his/her weight, to continue on regardless, can prevent a team from reaching its goals and cause resentment.

Likewise, recognising the efforts of high-performing team members can motivate them and enhance the performance of the whole team.

5. Planning: This involves goal setting, establishing strategies, setting a timetable, assigning responsibilities and anticipating obstacles.

6. Organising and delegating skills: Organisation involves dividing the work into logical assignments, providing the resources, determining the lines of responsibility and authority, and establishing a communication structure for co-ordinating efforts and providing feedback.

PAGE 32

How to delegate effectively? Choose the right person, get their agreement, make expectations clear, establish how and when feedback should be given and give the appropriate authority.

7. Team building: Ideally, choose team members who already have the skills and knowledge needed for the chosen task. Consider colleagues who are reliable, hard working, motivated and who work well with others. What goals will your team be working towards?

Agree on the procedures and rules to be followed.

Develop a plan. How will the team’s performance be monitored? How will you build the team’s trust?

8. Facilitation skills: A facilitator establishes the structure a team needs to function effectively, ensures that structure is viable and removes obstacles that may impede progress. They empower the team, encouraging it to take control and assume responsibility for proceeding with its projects. Assisting the team to clarify tasks, define roles within the team, plan meetings and projects, make decisions, resolve conflicts and identify processes does this.

9. Motivate others: A good motivator creates and maintains an environment, where team members are likely to become motivated. Different things motivate people. Find out what motivates individual team members, then develop rewards that match what members value.

10. Become a mentor: A mentor recognises someone who is less experienced, but cultivates his/her potential and helps them succeed. Be a mentor by being a role model, offering advice, encouraging independent thinking, helping the mentee recognise areas that need improvement and offering opportunities for skill development.

These 10 skills will be an asset to employers and employees alike.

JUNE 2014 | www.testmagazine.co.uk

Burn Your Spreadsheets

Manage product testing with TestTrack TCM TestTrack TCM tracks all details of product testing for you, including test cases, suites, runs, and results. Now you can have real-time visibility over your testing effort and more time to plan and manage.

• Achieve real-time visibility into testing progress.

• Reduce risk and achieve greater confidence in the release.

• Gain more time to plan projects and manage your team.

Download a free, 30-day trial of TestTrack TCM at www.seapine.com/testuk © 2013 Seapine Software, Inc. All rights reserved.

WCSQ SHOW PREVIEW

SHIFT LEFT – INSPIRATION AND INNOVATION FOR SOFTWARE QUALITY Just a taste of what’s to come at the 6th WCSQ, which is taking place on 1st to 3rd July 2014, at the Hilton Metropole Hotel, London.

T

he table below just shows a selection of sessions. WCSQ is a partnership of the Software Division of the American Society for Quality, the Software Group of the European Organisation for Quality – since 2005 represented by iSQI (International Software Quality Institute), and the Union of Japanese Scientists and Engineers.

BCS, The Chartered Institute for IT, and the UK Testing Board, will host the 6th WCSQ. The “Shift Left” theme concerns the fundamental theory that we shift tasks from the right-hand-side of a plan to the left-hand-side to increase the quality. It’s about building it right first time, and not just testing it right later.

Time 8:00 Morning

Tuesday 1st July Tutorials Registration

Wednesday 2nd July Conference Registration

Thursday 3rd July Conference Registration

Quantifying all qualities Tom Gilb, Norway/UK

Keynote: Andy Green

Keynote: Tom Gilb, Norway/UK

RDTA – Repository Driven Test Automation Danny Almog and Yaron Tsubery, Ben Gurion University, Ness Technologies, Israel

Kanban at scale – A Siemens success story Bennet Vallet, Siemens Healthcare

Building better software with business stories Paul Gerrard, Gerrard Consulting Improving end-user satisfaction via applying design thinking Martina Luenzmann and Martina Wroblewski, SAP Germany

Afternoon

Transformational quality management Ben Fry and David Rigler, SQS

High performance software engineering Processes Bernd Hindel, Method Park AG

Still “testing on the toilet”; 8 years later Antoine Picard, Google USA

Assess, prioritise and do parallel and step-by-step improvement Rik Marselis, Sogeti, The Netherlands

ISTQB Career – Expert Level Graham Bath, T-Systems, Germany

Testing in an agile environment – how did our role change?

Alon Linetski, Best- Testing, Israel Test automation patterns Dorothy Graham, Serreta Gamba

Shift to the right side of your brain: Diversifying the ways you test Jan Jaap Cannegeiter, SYSQA B.V. Mobile testing – the relation between business goals, design considerations and testing Derk-Jan de Grood, Valori Keynote: BJ Rollinson, Microsoft USA

Testing software the crowdsourced way – How to enhance software quality by utilising real people and real devices George Hansbauer, Testbirds Gmbh IDBS: Statistical process control and metrication – A case study of quality and development process improvement Mike Jarred and Ilca Croufer, IDBS Excellence in test data management for improved testing quality Karthikeyan Murugesan, Cognizant Technology Solutions UK Stories of my life Vipul Kocher, SALT India Framework for sustainable reuse of function specification Noriko Iizumi, Tomoko Tomiyama and Atsuko Koizumi, Hitachi, Japan Keynote: Paul Gerrard, Gerrard Consulting: Why shift left is more important than agile Keynote: Sasumu Sasabe, JUSE, Japan

Evening

Networking Evening in the WCSQ Exhibition

Social Event at Madame Tussauds

For the full conference programme go to www.wcsq.org

PAGE 34

JUNE 2014 | www.testmagazine.co.uk

SUPPLIER PROFILE CHAKRAM VANKATACHARI NARAYANAN HEAD OF QUALITY ASSURANCE AND TESTING SERVICES SYSTEMS PLUS

26 YEARS AND COUNTING… TEST Magazine speaks with Chakram Vankatachari Narayanan, Head of quality assurance and testing services at Systems Plus, about how the company has grown over the past 26 years…

U

nder the current economic climate and market conditions, enterprises need to be agile to thrive and flourish, believes Chakram Vankatachari Narayanan, Head of quality assurance and testing services at Systems Plus.

The generational shift is bringing along with it a shift in user-base, which is more technologically savvy, more inclusive and always-on technologies. From restricted form factors of devices, BYOD is becoming order of the day in enterprises.

The question it creates in an enterprise's approach to adoption of technologies is: • With interconnected partners, devices and applications, will the business processes sustain?

• Can I build out the application, which can grow with users or build out the best for future today?

• Will my customers get the response on time in the interconnected world?

• How do I secure application?

• How do I make the access inclusive?

PAGE 36

• How quickly can I react to the changes in my industry landscape (e.g., M&A) and be relevant and profitable?

The answers to the above questions not only remain in the realm of how applications are built, but also in the way the applications are profiled. This is the change which will alter the future of software testing.

From remediating a software to see whether it meets the requirements of today, software testing is moving towards how it will work in future. This calls for new age testing, which includes quantum of NFR (nonfunctional requirements) testing such as: Services testing; security testing; automation testing; accessibility testing; performance testing; big data testing; device compatibility testing; and the list goes on.

All these testing services require human capital and resources, which are transient in nature. Software testing service providers who are agile and understand this will enable the enterprises to traverse the path to future, believes Chakram Vankatachari Narayanan.

JUNE 2014 | www.testmagazine.co.uk

SUPPLIER PROFILE TEST Magazine: What are the origins of the business?

restricted by the presence of existing competition.

Chakram Vankatachari Narayanan: Systems Plus has been successfully operating in the UK since 2005 and, overall, is a 26-year-old niche business solutions provider, enabling companies to embrace technology at the right value.

The changing dynamics of the enterprise presents myriad sets of issues, including:

We are an 800+ strong company with global delivery centres in Pune, Mumbai and Ahmedabad serving clients across UK, USA, APAC and Continental Europe.

We began our journey with offering strategic consulting to large enterprises to adopt offshore services and, over a period of time, started offering focused solution services to extend our customers’ portfolio to realise their total cost of ownership (TCO). Some of our enterprise customers include Pepsico, Shipco, Tropicana, Payless, Family Dollar, Schnieder, Frito Lays, Orangina Schweppes, just to name a few.

TM: What do you offer the market?

CVN: We offer specialised solutions, namely technology consulting services, ADM (Applications Development and Maintenance) services and managed testing and automation services. These solutions are leveraged by frameworks and methodologies, which improve the productivity and reduce the TCO.

TM: Do you specialise in any particular areas?

TM: So, where do your testing services fit in and how do you help?

CVN: With the proliferation of new applications and the challenges faced by the enterprises for retaining business process continuity, making sure the applications are alive 24/7 is extremely FROM critical. This is where Sysplus’ Managed REMEDIATING Testing Service acts as a centralised and A SOFTWARE TO independent verification and validation arm for all application streams, SEE WHETHER IT MEETS providing end-to-end testing services; THE REQUIREMENTS OF right from test strategy, test planning TODAY, SOFTWARE TESTING to test execution, automation, IS MOVING TOWARDS HOW reporting and optimisation.

IT WILL WORK IN FUTURE. The endeavour of our testing services THIS CALLS FOR NEW is to ensure the applications that are AGE TESTING WHICH developed get released to production faster, better and cost effective. We INCLUDES QUANTUM have the following assets under the OF NFR TESTING

CVN: In ADM services, we have an application solution framework, which provides elastic provisioning of services in the cloud using .net/ Java and non-SQL databases. In managed testing services, we have branded frameworks under the brand TestPlus to address test automation, security and accessibility testing.

Consulting services built on ROI also supports the TestPlus frameworks. This basically provides a transparent approach to the services and expectation setting with the customer. Contractually the niche services are supported by SLAs to validate the service offering.

Through these niche services we engage with more than 30 clients worldwide in multi-year services provisioning.

TM: Have you made any recent acquisitions?

CVN: Our growth has been organic and we have successfully enlarged our footprints with our existing customers and won new ones. As a part of enriching customer experience, if there are any opportunities for providing additional services we will look at acquisitions.

TM: What plans for growth do you have?

CVN: Existing customers drive our growth. We will expand our offerings to our existing customers by cross-selling new services. To attract new customers, we want to create a strong brand recall in our niche offerings and also through referrals. Increased SGA expenses in a measured way will help focus our objectives.

TM: What is your current view of the market in which you operate?

CVN: We understand the market is saturated with service providers. However with the economy opening up, user profiles changing and edge of enterprise expanding, many businesses have to think of different ways they interact with their customers. These provide new opportunities for niche service providers and are not

JUNE 2014 | www.testmagazine.co.uk

• Time-to-market.

• Reduction in cost to quality.

• Proliferation of devices to access content.

• Business process down time reduction.

• Spikes in demands for new application development and testing.

TestPlus umbrella, to help in quick release management, such as:

• Keyword driven automation framework to deliver automation of test cases faster and cheaper and hence the entire testing is carried out faster.

• Optimised performance testing model, which converts the capex model into opex model through which the customer cuts upfront deployment of capital for testing. By providing a mixture of open source licenses and branded tools, we help in reducing the coq of the software.

• Security and accessibility test cases that help in addressing the usability and security of web applications based on OWASP and WCAG 2.0 standards.

• These assets are supported by consulting for managed testing services built on TMMi model.

• The test solutions also have interesting outcome oriented test engagement to bind contractually with the customer.

TM: Based on your experience, how has testing changed over the years?

CVN: In the earlier stages, testing was considered a remediation activity to uncover the bugs in a software.

This led to models which were isolated and diffused approach to a solutions delivery.

With the emergence of global enterprises, time-to-market and 24/7 availability of business processes testing has become an enabler of an enterprise aspiration. This has completely altered the mind-set of the testing teams to be more focused on business, value addition and improved productivity, leading to agile teams.

Companies, which are nimble and can plan for future, will survive, and we are extremely happy to be in a space that is demanding.

PAGE 37

National Software Testing Conference

2014

NATIONAL SOFTWARE TESTING CONFERENCE 2014 REVIEW

Breaking today’s boundaries to shape tomorrow

AN “INSPIRATIONAL” TWO DAYS Sophie-Marie Odum shares her thoughts on the first ever National Software Testing Conference, held in May at the British Museum…

"I

nformative, inspirational and great value for money,” “A great networking event for the testing industry”, and “A conference that puts the focus where it matters,” are just a few comments from delegates of the first ever National Software Testing Conference… and what a huge success it was!

was inspirational not only for the venue, but also for the large contingent of like-minded individuals gathered together, with the goal of sharing their knowledge and learning from one another.

“The organisation was faultless, with great presentations from a cross section of individuals and companies, happily passing on their own experiences of the world of testing today. In the gaps between presentations, we were provided with great catering services and a wide range of vendor exhibits.

WAS

Speakers from many well-known companies, including William Hill, Expedia, Virgin Media and Waitrose, just to name a few, offered I delegates a new way of thinking by GENUINELY demonstrating how cross-industry IMPRESSED WITH THE collaboration can help breakthrough LEVEL OF ENGAGEMENT today’s boundaries to shape FROM THE DELEGATES AND tomorrow.

WITH THE INCISIVENESS OF

“As a speaker myself, I was genuinely impressed with the level of engagement from the delegates and with the incisiveness of their questions. Looking forward to the next conference already, the only shame is that it happens but once a year. Thanks you to all involved in putting this together.

Congratulations on an excellent event.”

Held at the grand British Museum, THEIR QUESTIONS. LOOKING which is famed for human history and FORWARD TO THE NEXT culture, and illustrates how today’s society is built upon yesteryear; this CONFERENCE ALREADY, THE was the ideal setting for the National ONLY SHAME IS THAT IT As well as a wide selection of Software Testing Conference. The HAPPENS BUT ONCE A presentations, covering the core areas Conference demonstrated how software of software testing, including strategy, YEAR testing has evolved over the years and how management, process and tools, the National now is the right time for the industry to work Software Testing Conference also hosted a series together, irrespective of sector, and elevate the of Executive Debate Sessions, facilitated and led by software testing industry as a whole. key figures, and an exhibition, which saw companies such as Borland Software, IBM and Sogeti, plus many more, To further bring the industry together and promote unity, showcase their latest products and services available. the National Software Testing Conference was used as a platform to launch the Software Testing Network. This The National Software Testing Conference 2014 received Network provides a unique opportunity for learning via fantastic feedback, and the team behind TEST Magazine access to exclusive information, research, events, services feels very proud to have hosted an event that brought and lots more. It was encouraging to see many delegates the industry together to network and swap and share take an interest in this new network and we look forward ideas, allowing delegates to head back to their offices to growing its membership. and implement change with immediate effect. We are really looking forward to next year!

Keynote speaker, Rod Armstrong, senior director QA, Expedia, said, “The National Software Testing Conference All presentations are available at www.softwaretestingconference.com for you to view, download and refer back to at your leisure.

NSTC14 in numbers* - 84% said the NSTC’s content was either “Good” or “Fantastic” - 86% of attendees gained a lot of value from attending - 91% felt that NSTC14 was “good value for money”

- 94% thought that the NSTC14 was “extremely well organised and executed” - 72% are interested in attending NSTC 2015

*Data gathered from the National Software Testing Conference 2014 feedback form

JUNE 2014 | www.testmagazine.co.uk

PAGE 39

NATIONAL SOFTWARE TESTING CONFERENCE 2014 REVIEW

“Sogeti had a great time at the National Software Testing Conference, and feel that it was executed well considering it was TEST Magazine’s first branch out into the conference space.”

- Samantha Mills, Sogeti UK

“This conference was a revelation for me, I’m fully energised and motivated to apply a lot of the ideas within my organisation.”

– Gabriela Pinder, test/quality assurance lead, Impellam Group”

PAGE 40

JUNE 2014 | www.testmagazine.co.uk

NATIONAL SOFTWARE TESTING CONFERENCE 2014 REVIEW

“Passion and enthusiasm from all delegates and speakers. Fantastic job, well done.”

- Rod Armstrong, senior director of QA, Expedia

“It was an amazing event! Like with the European Software Testing Awards earlier, the team pulled out all the stops. Very professional, great venue and fantastic networking opportunity. Thank you.”

- Karen Thomas, practice manager, Barclaycard

“Congratulations on a superb event! It was well worth the trip to London.”

– Collette Haworth, test manager, BAE System

JUNE 2014 | www.testmagazine.co.uk

PAGE 41

NATIONAL SOFTWARE TESTING CONFERENCE 2014 REVIEW

“Great buzz, great team, great venue, great passion.

Really amazing atmosphere to meet like-minded people.”

- Asia Shahzad, test manager, Expedia

“Fantastic conference focused at testing professionals. Good opportunity to network, new cutting-edge products and ideas. Looking forward to next year.”

– Nadine Abley, GTF test manager, JP Morgan

“I really enjoyed the conference/ exhibition and would attend again in the future.”

– Jill Armstrong, software test team lead, Almac Group

“The advice and guidance provided by the keynote speakers gave me the most value.

The venue was lovely and ideal, and the event was very well organized and structured.”

- Jennifer Flowers, QA test manager, Wheatley Associates

PAGE 42

JUNE 2014 | www.testmagazine.co.uk

NATIONAL SOFTWARE TESTING CONFERENCE 2014 REVIEW

“As a leading provider of industry recognised software testing and agile certifications, iSQI were very warmly welcomed by delegates at the the NSTC. We look forward to next year.”

– Kyle Siemens and Debbie Archer, iSQI Gmbh

To view more photos and download all the presentations, please visit: www.softwaretestingconference.com

JUNE 2014 | www.testmagazine.co.uk

PAGE 43

NATIONAL SOFTWARE TESTING CONFERENCE 2014 REVIEW

“THE EVENT HAD THE PERFECT BLEND OF DELEGATES” Exhibitors and event partners at the National Software Testing Conference, Cigniti Technologies, share their thoughts about the event…

T

EST Magazine: How valuable was the National Software Testing Conference, and what were the key takeaways for you?

Cigniti Technologies: This was a great event for us. It happened at the right time; Cigniti opened its London operations and the event was an apt platform for branding.

The event had the perfect blend of delegates from the testing community, right from test architects to decision makers and influencers. Cigniti is the world’s third largest independent software testing services company with a focus on helping organisations build better quality software. The UK is our second largest market in terms of customers and prospect base, being locally present helps us sever our customers better. The messaging at the event was a great start for Cigniti.

TM: Did you find that the delegates at the conference were the correct target audience for Cigniti Technologies?

CT: The delegates at the conference were the perfect blend of audience Cigniti was looking for. We had testers, architects, decision makers and influencers visit us. The event also had an array of testing industry thought leaders. The speaking opportunity at the conference also gave us wide exposure to the audience expectations in the UK.

PAGE 44

TM: Would you recommend others to be a part of the conference next year?

CT: Yes, organisations with an interest in software testing, this will be the tailor made event.

TM: How did Cigniti Technologies begin?

CT: Cigniti Technologies started the journey with a focus on helping organisations build quality software. Cigniti Technologies today is the world’s third largest independent software testing services company, headquartered in Dallas, TX. Cigniti’s test offerings include TCoE, enterprise mobility testing, testing on cloud, big data testing, functional, automation, security and performance testing services.

Over the last 15 years, Cigniti has helped enterprises and ISVs, across verticals, build quality software while improving time-to-market and reducing cost of quality.

Cigniti has translated its R&D into Cigniti SMART Tools that accelerate testing and help improve the quality of services delivered to clients.

Recognised as a leader in IP led testing services by industry analysts like Nelson Hall, Cigniti is also the world’s first independent software testing services company rated at CMMI SVC1.3 maturity level 3 and is an ISO 27001:2005 certified organisation.

With a team of over 800 people, Cigniti operates out of 95,000 sq ft of office space spread across Hyderabad,

JUNE 2014 | www.testmagazine.co.uk

NATIONAL SOFTWARE TESTING CONFERENCE 2014 REVIEW

Dallas and Toronto. Cigniti has India’s first of its kind Robotics Test Labs, HP co-branded mobile test lab and a world class performance test lab. Cigniti has strategic alliances with the leading product, tool and technology vendors. Cigniti’s CSR initiative Project Cignificance aims to impact 1 million+ lives through education as an enabler.

and inefficiencies in test life cycle. Cigniti translated its experience and expertise into tools, point solutions and frameworks to jump start QA efforts and realise quicker ROI. Cigniti SMART Tools are cloud enabled with Pay As you Go model service options.

We specialise in following IP led testing services:

TM: How was Cigniti Technologies developed?

CT: We took a step-by-step firm and consistent approach to grow from being a specialised testing services company to the world’s third largest. Some of the key milestones in the journey include: • Premier independent testing services organisation • Corporate headquarters in Irving, Texas • Global delivery team with a vision to be your local partner • Dedicated TCoE’s with “cloud enabled” test labs • Cigniti Smart Tools for test acceleration and quicker ROI • Public listed and ISO 27001 certified • Strategic alliances with product, tool and technology vendors • Trusted partner for several Fortune 500 companies

TM: What do you offer the market?

CT: Cigniti delivers independent quality assurance services backed by its proprietary IP with career testers who are passionate about testing. Cigniti believes in delivering custom solutions to address nuances of specific problem statement. As a strategic partner, Cigniti brings comprehensive service offerings in the quality assurance space, which accelerate overall test efforts for its clients.

TM: What are the plans for Cigniti for the next five years?

CT: We plan to become a 2,500 people company by 2015. We are working to become the world’s largest and most respected independent software testing services company. We will continue to invest in R&D to offer IPled testing services by building new tools, frameworks that accelerate the entire software test life cycle. We will continue to invest in building world-class test labs and look to partner with leading tool vendors across technologies and verticals.

TM: What is your current view of the market in which you operate?

CT: The market in particular is looking for innovation in the software testing space. The key drivers are efficiency, innovation and delivery excellence. Clients need disruptive solutions that can accelerate the entire software test life cycle. The clients are looking for career testers with a passion to understand the functionality of software from an end-user perspective along with the ability to automate end-to-end scenarios.

TM: What is your current view of the software testing market as a whole?

CT: Software testing market as a whole is bound to grow further. In fact, independent software testing is expected to grow by a run rate of 10% in 2014, $12,300 million, as per the report from NelsonHall; most spending seems to be in financial services in telecom in the year ahead.

Also the need to have specialised career testers is expected to grow by at least 25,000 every year and, as a whole, 230,000 career specialist testers are required.

As the enterprises become more digital and connected, the need for specialist software testing providers with global test program management skills is the need for the hour. Specialist skills like security, performance, enterprise mobility and test automation across technologies will stay as drivers for successful QA strategies.

TM: Which particular areas do you specialise in?

CT: With over a decade of experience serving Fortune 500 companies, Cigniti has witnessed several redundancies

JUNE 2014 | www.testmagazine.co.uk

The ability to provide superior software testing services, backed by IP-led testing services, world class labs, executive management with proven test thought leadership will drive the software testing market.

PAGE 45

LAST WORD DAVE WHALEN PRESIDENT AND SENIOR SOFTWARE ENTOMOLOGIST WHALEN TECHNOLOGIES HTTP://SOFTWAREENTOMOLOGIST.WORDPRESS.COM

I’LL JUST WORK FROM HOME!

By Dave Whalen

T

hanks to modern technology, co-workers no longer need to coexist in the same workspace – I can work just as easily sitting in a cubicle as I can sitting at home on the couch in my underwear. I’ll give you a minute to get that visual out of your head.

We can connect virtually via the Internet using teleconferencing or virtual meetings (but if using video, you may want to get dressed). We can share what is on our screen with the virtual attendees; and we can email, or chat. As long as I have a laptop and Internet connection, I can work from anywhere.

I don’t have to deal with traffic and so I’m less stressed.

For me, on a good day, it’s approximately a 30-40 minute drive to come in to the office. But, I live in Colorado where it tends to snow, and when it does, my drive is well over an hour.

Since I can work virtually from anywhere I don’t even need to be in the same city, county, state, province, country, or continent. Many companies are taking advantage of this by hiring offshore employees, geographically separated employees, moving to smaller facilities, etc. One major technology company in California, Yahoo, did just that. Sound’s like a win-win situation for both employers and employees right?

However, Yahoo recently ended their work from home policy. The CEO felt that working from home hindered communication and collaboration. Other companies are initially adopted working from home are following suit.

Having worked in a couple of companies that allowed working from home, I can see their point. Personally, I like having the work from home option, but I only use it in emergencies or bad weather.

Like anything, there are pros and cons. Here are some of the problems you may need to deal with: • Distractions: I’m easily distracted. If the television is on, I’m worthless. It’s hard to stay focused sometimes.

• Other’s perceptions: Just because you’re home, it doesn’t mean you are available to do non-work stuff.

It took my wife and kids a while to figure out that I’m working! I’m not here to wash dishes, make dinner, bathe the cats, do laundry, run errands or any other items on your list.

• Time: I’m in Colorado. We have team members in Indiana, Russia, and India. Getting us all together is by far our biggest challenge. Someone is going to have to be available after hours or in the wee hours of the morning. It can sometimes take two or three days to get an answer to a question or solve a problem, compared to 15 minutes if the developer, for example,

PAGE 46

had been in the cubicle next door.

YAHOO RECENTLY ENDED THEIR WORK FROM HOME POLICY. THE CEO FELT THAT WORKING FROM HOME HINDERED COMMUNICATION AND COLLABORATION. OTHER COMPANIES ARE INITIALLY ADOPTED WORKING FROM HOME ARE FOLLOWING SUIT

• Cultural issues: Anything from different holidays to language. But then again, language can be an issue even if you are all speaking the same language!

However, luckily, none of these are huge issues and are easily resolved.

• Distractions: Tune ‘em out. Don’t work on the couch in your pajamas. Turn off the television. I created a home office in our spare bedroom. When I work from home, I get dressed as if I’m going to the office. It helps with the mental perception that I’m “at work”.

• Perceptions: Deal with them. I tell the family that just because I’m home I am not their maid. I’m working. I had to stand my ground a few times here.

• Time: This is a tough one to overcome, but try to be fair.

Everyone should share the pain of coming in early or staying late. Consider working a different shift or split shift.

• Cultural issues: Be aware of other cultures. Learn about their customs. You may be the life of the party at your local pub, like me, but your humor may not translate well. Take it from me – it usually doesn’t.

Is it a challenge? You bet. Insurmountable? Not really. Plus you get to learn about other cultures!

JUNE 2014 | www.testmagazine.co.uk

THE EUROPEAN SOFTWARE TESTER INNOVATION FOR SOFTWARE QUALITY

Subscribe to TEST free!

TEST: THE EUROPE

VOLUME 6: ISSUE 1 FEBRUARY 2014

www.testmagazine.co.uk

TESTER

E TESTER SOFTWAR OPEAN THE EUR

AUTOMOTIVE APPLICATIONS KEEP CHALLENGING SOFTWARE TESTING

D RAPID QUALITY AN T MUST EN DEVELOPM AND -H -IN GO HAND

3: JUNE 2014

VOLUME 5: ISSUE 4: AUGUST 2013

WO N SNOWDE THE NEW EDITORIAL EDWARD R HIS LIFE ADVISORY BOARD FEARS FO WS INTERVIEW: VIE ER INT THE CROSS-POLLINATION : CAREEROF TESTING PRACTICES FOCUS ON

VOLUME 6: ISSUE

VOLUME 6: ISSUE 1: FEBRUARY 2014

INSIDE:

NEWS: : TRENDS FOR THE YEAR AHEAD RLD NEWS

HIGHLIGHTS: NSTC 2014

VERY PROFES SIONAL, GREAT FANTASTIC NET VENUE AND WORKING OP PORTUNITY

FOR EXCLUSIVE NEWS, FEATURES, OPINION, COMMENT, DIRECTORY, DIGITAL AND MUCH MORE VISIT: testmagazine.co.uk

Published by T H I R T YO N E

www.31media.co.uk

THE EUROPEA N SOFTWARE TESTER INNOVATION FOR SOFTWAR E QUALITY

CONNECTED CARS: RACE TO THE FUTURE

R IS IT TIME FO ?

K IN TH E R A

INSIDE:

www.testmaga zine.co.uk

TESTER

SOFTWARE

THE EUROPEAN SOFTWARE TESTER

.uk agazine.co www.testm

VOLUME 6: ISSU E3 JUNE 2014

AN SOFTWARE

EUROPEAN

5: ISSUE 4 VOLUME 13 AUGUST 20

INNOVATION FOR SOFTWARE QUALITY

TEST: THE EUROPEAN SOFTWARE TESTER

TEST: THE

N INNOVATIO E WAR FOR SOFT ITY QUAL

Telephone: +44 (0) 870 863 6930 Email: info@31media.co.uk Website: www.31media.co.uk

SUCCESSFUL SOFTWARE DELIVERY

DO YOU WORK WITH THE SAME VISION? Create the software your business really needs Requirements. It’s surprising how often a single link can break the software supply chain. But it’s where most defects occur. Removing them later in the lifecycle is costly, impacts delivery schedules and drains resources. That’s why errors and rework so frequently undermine project success.

While complexity has increased exponentially, managing requirements hasn’t evolved at the same pace. Borland tools can improve collaboration. Our technical solution keeps stakeholders in sync throughout the project lifecycle. Better input means better products.

So join the thousands of Borland customers who already tackle requirements the right way ... precisely.

See more at www.borland.com/connect

Copyright© 2014 Micro Focus. All Rights Reserved.

Portions Copyright © 1994-2009 Borland Software Corporation (a Micro Focus company).