TEST – March 2018 by 31 Media

MARCH 2018

02:23

MOBILE TESTING TECHNIQUES IN THIS ISSUE IMPROVE MOBILE DEVOPS WITH AGILE TESTING

IN THIS ISSUE CLOSING THE DIGITAL DIVIDE

IN THIS ISSUE LOOKING TO ACCELERATE MOBILE APP DELIVERY?

NEWS | VIDEOS | PRODUCT NEWS | WHITE PAPERS

Industry Leading Portal

DevOps Awards

DevOps Online is the premium online destination for news, reports, whitepapers, research and more relating to the DevOps movement. Covering all aspects of IT transformation you can be sure that DevOps Online will keep you informed and up to date.

www.devopsonline.co.uk T E S T M a g a z i n e | M a r c h 2 01 8

C O N T E N T S

THE national T E S T COVER

STORY:

M A G A Z I N E MANUAL

TESTING:

M A R C H WANTED

2 0 1 8

DEAD

ALIVE

MOBILE & APPLICATION TESTING

Closing The Digital Divide ................................ 4 Improve Mobile DevOps With Agile Testing .... 6

THE GOVERNMENT ADOPTS AGILE

Creating A Modern Social Referral Platform With Devops 2.0 ............................................... 8 Open Source Governance Accelerates Innovation ...................................................... 10 Deploying IoT Solutions ................................. 12 Next Gen Data Mart ....................................... 14

DevOps Conference NORT

Accelerate Mobile App Delivery? .................. 18 DEVOPS & CONTINUOUS DELIVERY

A Developer's Playpen.................................... 20 From Testers To Quality Engineers ................ 22 Technical And Cultural Building Blocks .......... 26

Ship Safely To Sea And Start Cursing At Maximum Speeds ............................................................ 30 Manual Testing: Wanted Dead Or Alive ........... 34 Tcoe’s Existence In Today’s Devops World ..... 38 Staying On Top Of The Competition .............. 42 AGILE METHODOLOGIES

Allowing Change From The Ground Up ........ 46 The Government Adopts Agile ...................... 50

TRANSFORMING FROM TESTERS TO QUALITY 22 ENGINEERS STAYING ON TOP OF THE COMPETITION

Moving Away From Waterfall Methods ......... 54 EVENTS

National Software Testing Conference .......... 56

T E S T M a g a z i n e | M a r c h 2 01 8

E D I T O R ' S

C O M M E N T

TEST MAGAZINE MARCH 2018 VOLUME 10 ISSUE 1

MARCH 2018

02:23

MOBILE TESTING TECHNIQUES IN THIS ISSUE IMPROVE MOBILE DEVOPS WITH AGILE TESTING

IN THIS ISSUE OPEN SOURCE GOVERNANCE ACCELERATES INNOVATION

IN THIS ISSUE WWW.TESTINGMAGAZINE.COM

EMBRACING A TECHNOLOGICAL REVOLUTION

LOOKING TO ACCELERATE MOBILE APP DELIVERY?

MARCH 2018 | VOLUME 10 | ISSUE 1 LEAH ALGER JOURNALIST

he “death of manual testing” has been a topic raised for years’. This is because of some testers believing that automation is “taking over” the way a manual tester works. Despite this, both types of testing have different aims and restrictions – automation is typically used in stress and load testing; big projects, with different releases and builds; and regression testing, where you would usually expect to find bugs. Differently, manual testing is usually used when testing a new functionality; specific programmes and devices; and to document user requirements. DevOps, which automates as much as possible and removes wasted effort from development processes, appears to be changing the scope of software testing, accelerating application delivery through continuous integration, continuous feedback and real-time app updates. This software engineering culture and practice has been a game changer in not only mobile application testing, but also a variety of sectors, wiping out traditional waterfall methods. It also appears that waterfall methods are rapidly declining in popularity as more DevOps and agile methodologies are increasingly adopted. So, if your company hasn’t come to terms with these new approaches already, why not stay ahead of the game and embrace this technological revolution? This magazine issue primarily touches

upon DevOps, continuous delivery, agile methodologies and mobile and application testing, to help you grasp current testing principles and to open up your mind via a journal of state-of-the-art information. We at 31 Media support the software testing community all year round, by supplying you with TEST Magazine; daily news via www.SoftwareTestingNews. co.uk and www.DevOpsOnline.co.uk; as well as hosting a variety of events throughout the year, covering the most pressing industry issues and topics (www. softwaretestingnews.co.uk/event). New for 2018, we have created our own job board, where you can advertise jobs with Software Testing News; the perfect place to find quality candidates, while reducing hiring time via a large network with followers around the globe (www.softwaretestingnews.co.uk/jobs). Success depends on the individual, so prove you are valuable by networking, training and up-skilling.

© 2018 31 Media Limited. All rights reserved. TEST Magazine is edited, designed, and published by 31 Media Limited. No part of TEST Magazine may be reproduced, transmitted, stored electronically, distributed, or copied, in whole or part without the prior written consent of the publisher. A reprint service is available. Opinions expressed in this journal do not necessarily reflect those of the editor of TEST Magazine or its publisher, 31 Media Limited. ISSN 2040‑01‑60 EDITORIAL DEPARTMENT editor@31media.co.uk +44 (0)203 056 4599 JOURNALIST Leah Alger leah.alger@31media.co.uk +44 (0)203 668 6948 ADVERTISING ENQUIRIES Shivanni Sohal shivanni.sohal@31media.co.uk +44 (0)203 668 6945 PRODUCTION & DESIGN Ivan Boyanov ivan.boyanov@31media.co.uk 31 Media Ltd, 41‑42 Daisy Business Park 19‑35 Sylvan Grove London, SE15 1PD +44 (0)870 863 6930 info@31media.co.uk www.testingmagazine.com PRINTED BY Pensord, Tram Road, Pontllanfraith, Blackwood, NP12 2YA  softwaretestingnews  @testmagazine  TEST Magazine Group

T E S T M a g a z i n e | M a r c h 2 01 8

CLOSING THE DIGITAL DIVIDE Sky and Space Global, CEO, Meir Moalem, reveals how he's helping rural areas connect via narrowband ssatellites

T E S T M a g a z i n e | M a r c h 2 01 8

M O B I L E

A P P L I C A T I O N

eir Moalem, CEO and Managing Director of Sky and Space Global, co-founded the company with Meidad Pariente and Yonatan Shrama at the end of 2015, before being acquired by an Australian company. The firm is now traded at the Australian Stock Exchange (ASX) and has been for more than a year and a half. Now it’s working on a constellation of 200 nano-satellites in equatorial low earth orbit for narrowband communications, which is expected to cost up to US$150million in total. “We are a business that provides narrow communication services through the use of a disruptive technology based on nano-satellites. Three of them are in space and our journey will lead us to the constellation of over 200 satellites, which will be launched and deployed by the end of the decade – so within two years from now,” revealed Moalem.

T E S T I N G

COMMUNICATING

Narrowband satellites

'HASSLE FREE'

are the only way

WORLD-FIRST

"Narrowband satellites are the only way rural countries can communicate hassle free. This is where we come in, providing affordable services and satellites for whoever and whatever enquires communication, such as electricity, oil, schools, and clinics. Everyone/everything needs to communicate, and our aim is to help.” Sky and Space Global provides WeFarm with communication infrastructure services. If you’re a farmer having issues with your crops, you can text Wefarm and its experts will help you. Despite this, the problem with rural areas is the communication infrastructure. A farmer has to typically wait until he/ she has signal to make a phone call or send a text message. Moalem continued: “Imagine what it would do for farmers living in remote areas if they could communicate from the fields.

ACHIEVEMENTS

HELPING

Being the first ever company to consider using nano-satellites for communication, it has recently successfully achieved world-first achievements. “Satellite communication was never done before with nanosatellites. We demonstrated the capability of using a standard smartphone and a small narrowband satellite terminal to send a text message, image, voice recording and to do a phone call between two users last September. The satellites used were about 30cm in length and 500km over the ground,” he added. Affordable satellite communications appear to drive digital inclusion into emerging markets by revolutionising the way people look at communication, helping close the gap between the digital divide. Moalem commented: “Today phoning is expensive and people who live in undeveloped areas where there is no infrastructure such as Africa, Latin America, and South East Asia need this type of connectivity.

THE THIRD-WORLD "Imagine if you had a river and there was water meters and monitors showing whether it's drinkable or not, the difference it would make. We want to help people through communication to live a better life, because sometimes we underestimate how crucial it is." Sky and Space Global enables people to use their phones by giving them a small satellite terminal which acts the same way your internet browser works through Wi-Fi and Bluetooth. "Everyone that connects can send messages, pictures, voice recordings and phone calls, in a bid to help the third-world through better communication systems," he added. He also noted that his company can have a tremendous impact on the economy, closing the gap between certain countries, and that the scope of testing for Sky and Space Global is to deploy more and more satellites to help improve people’s day-to-day lives.

rural countries can communicate hassle free. This is where we come in, providing affordable services and satellites for whoever enquires communication

MEIR MOALEM CEO AND MANAGING DIRECTOR SKY AND SPACE GLOBAL

Meir has over 20 years’ of experience in management, R&D and operation of state-of-the-art projects in space and unmanned aerial systems. Currently he is working on his PhD in National Security and Space Programmes at Tel Aviv University, Israel.

T E S T M a g a z i n e | M a r c h 2 01 8

CEO of Mobile Labs, Dan McFall, explains how test automation and continuous delivery enables enterprise mobility professionals to boost mobile DevOps

T E S T M a g a z i n e | M a r c h 2 01 8

M O B I L E

A P P L I C A T I O N

here never seems to be a dull moment in enterprise mobility. Digital transformation has resulted in the continuous output of mobile devices and operating systems that offer new challenges for both application development and testing teams. Depending on available resources and cost, enterprise mobility teams might struggle to keep up with demand and to continue delivering high-quality mobile apps and mobile websites on time. But how can enterprise mobility teams respond to the inherent challenges posed by constant innovation, so they do not fall behind? The answer lies in improving mobile DevOps processes.

MOBILE DEVOPS IS HERE While DevOps is not a new concept, many enterprise mobility teams are still struggling to fully implement a DevOps approach in their mobile testing labs. As more mobile developers, testers and quality assurance professionals continue to feel the pressure of developing, testing and releasing better apps and mobile websites faster than before, it will be crucial to streamline all processes and become more nimble and efficient. Two ways to accomplish mobile DevOps goals and initiatives is through agile testing and continuous delivery. By leveraging mobile test automation in a continuous delivery framework, testers and quality assurance team members will be better aligned and have the right tools to do the right job at the right time. On its own, mobile test automation is not enough for true agile testing. All automated tests must be able to be accessed and executed by continuous delivery tools. It is the continuous delivery process that makes mobile DevOps happen.

AUTOMATED TESTING FOR AGILE TESTING The concept of agile testing fits in with the overall ‘agile manifesto’ by enabling mobile testers to test apps and mobile websites quickly and efficiently. A familiar mandate of agile development tasks teams to build, test and release software or in this case, mobile apps, frequently. But, releasing frequently is not enough to be agile. The entire process from development through testing and release must be a straight-forward process that can be repeatedly executed. Processes are then streamlined and run in a cohesive manner that results in continuously faster development, testing and delivery. While automated testing is the ideal solution for testing mobile apps and mobile websites with a lot of releases, there are several factors to consider when incorporating test automation. When getting started building a test automation strategy, consider the type of tests that

T E S T I N G

are required before deployment. Whether unit, functional, and/or performance, it is important to not only factor in the type of tests, but also the volume of tests. Depending on the number of apps, operating systems and devices that must be tested, a large volume of tests is mandatory before an app can be released to the consumer. In order to really leverage agile testing to its fullest potential, it is important to choose the right testing tools. By using the tools that best compliment and utilise the skill sets of the entire team, all members will become more productive, especially when leveraging a solution that promotes collaboration and productivity. Whether choosing an open source testing framework, such as appium, or by choosing a commercial tool like MicroFocus’ UFT, pairing the right tools with your testers will enable them to succeed and to stay on track to release often and on time. But, automated testing is only one piece of the puzzle to successful build a continuous delivery pipeline to increase mobile DevOps.

During implementation it's important to keep the process well-defined and simple, providing development, testers and quality assurance the right tools to accomplish their goals

POWER MOBILE DEVOPS WITH CONTINUOUS DELIVERY When building a continuous delivery pipeline all members of the team have separate roles to play. Although development, testers and quality assurance professionals are all responsible for different aspects of building and releasing a mobile app, the power of constructing a continuous delivery process enables collaboration and feedback at all stages. During implementation, it is important to keep the process well-defined and simple. Provide development, testers and quality assurance the right tools to accomplish their goals. After release, use monitoring tools to communicate any issues and feedback to fellow team members so bugs can be quickly repaired. When collaboration is possible, innovation and mobile DevOps can thrive.

PREPARE FOR FUTURE

DAN MCFALL CEO MOBILE LABS

INNOVATION By being proactive and establishing agile testing and continuous delivery as part of mobile DevOps initiatives, organisations can ensure that they are ready for the future. No matter what new mobile trends and technologies may emerge in the future, a team that is prepared can meet challenges effectively. By establishing streamlined DevOps processes today, mobile developers, testers and quality assurance will have the building blocks in place to continue to refine mobile DevOps to account for change and innovation.

Dan McFall has more than 19 years’ of professional and executive experience in the B2B technology sector. He is currently the CEO of Atlanta-based Mobile Labs, a leading provider of mobile testing and device management solutions.

T E S T M a g a z i n e | M a r c h 2 01 8

CREATING A MODERN

SOCIAL REFERRAL PLATFORM WITH DEVOPS 2.0 DevOps Manager and Canlead Founder advises start-ups and enterprises wanting to create a modern social referral platform to adopt DevOps 2.0 T E S T M a g a z i n e | M a r c h 2 01 8

M O B I L E

A P P L I C A T I O N

T E S T I N G

ichael Quan began his career as an IT Service Management (ITSM) Consultant before working at a fintech company where he had a devised strategy and roadmap to build DevOps in order to improve the software delivery life cycle (SDLC). IT operations management tooling enables the creation of business value by taking control of the IT estate, which helped Quan improve on operational efficiency and optimise business opportunity, as well as enable an enterprise to understand and predict how technology impacts the business, and how business impacts the IT infrastructure.

‘Feedback Inbox’ and a card with a red tag is added to the planning tool. Product managers and developers can discuss and prioritise this software defect immediately. Similarly, when the user selects ‘Request a Feature’ an email is sent to the Inbox and a card is added to the planning tool. A fast feedback loop is critical to the success of the user retention and conversion programme that a product defect can be resolved or a feature can be deployed into a controlled cloud hosting environment in realtime.

CAPABILITY MATURITY

Quan revealed: “My team and I have built a development operations capability using modern cloud services to help collaborate, plan, refine and improve. We practise DevOps culturally and technically, which led to the faster release of software features, shorter development time, lower development and operations cost – reducing rework from 40% to less than 10%.” Large enterprises wanting to create application services to start with DevOps at a scale framework like SAFE must create a DevOps engineering function to provide internal consultancy, do experiments, build solutions and support the development community, while aiming towards team selforganisation, reducing project management staff and training functional managers to become effective product managers, according to Quan. Differently, he believes small startups should find a qualified DevOps 2.0 professional to provide advice unless they have DevOps 2.0 experience, and cloud services should be used when they can. Although DevOps 2.0 is a culture shift, he advises the following elements to “shift it”:

MODEL (CMM) Here is a typical method and sounding he often used during his time as an ITSM consultant:

After several years of technology consultancy, Quan conceptualised a new way for people to monetise their social network. One where people refer opportunities to people over incentives, other than favours; challenging the traditional social enterprise referral methods. As a result, he managed to build a social referral platform using the DevOps 2.0 capability, integrating marketing and product management processes to development and operational capabilities. With DevOps 2.0 you can see the emergence of adaptive feature delivery as a component for software releases. This tool kit allowed him to align technology to business using cloud services, helping process automation and containerisation with a near term plan to complete microservice architecture and automated testing. As an example, his app allows users to select from a list of defects or features: • Give Product Feedback • Report a Bug • Request a Feature When the user selects ‘Report a Bug’ -> ‘Bug Type’ an email is automatically sent to the

EVANGELISE REGULARLY

• Start small • Set up a DevOps working group • Start with a core set of great DevOps engineers • Evangelise regularly • Attend meet-ups to learn and share – be part of the community • Automation first strategy • Push for cloud technology and services (SaaS) • Bring like-minded engineers together Quan also noted that he listens and works with his teams to ensure that the best advice and solutions are implemented.

Quan conceptualised a new way for people to monetise their social network. One where people refer opportunities to people over incentives, other than favours; challenging traditional social enterprise referral methods

MICHAEL QUAN PRODUCT MANAGER AT CANLEAD DEVOPS MANAGER AT WORLDPAY

Michael has spent more than 20 years' in information technology management and software and electronics engineering. As an Entrepreneur from an early age, he began his career in audio electronics product development, launching two products in the 1990s, followed by an enterprise software solution sold in 2009.

T E S T M a g a z i n e | M a r c h 2 01 8

OPEN SOURCE GOVERNANCE ACCELERATES INNOVATION Sonatype's Vice President, Derek Weeks, exposes how open source governance accelerates software innovation

T E S T M a g a z i n e | M a r c h 2 01 8

M O B I L E

A P P L I C A T I O N

“

e have a responsibility to build software that’s secure,” noted Derek Weeks, VP and DevOps Advocate at Sonatype. But despite this responsibility, companies throughout the UK are continuing to build unsafe components into their applications. Weeks explained that this has been driven by dramatic changes in software development. During the past five to six years, businesses have come under increasing pressure to move faster and be more agile, which has caused usage of open source components to surge. Today, “80 to 90% of an application is assembled from open source components”, as they can be quickly and easily downloaded in seconds. However, this widespread use of components from unknown sources also poses challenges. Weeks revealed that one in eight open source components downloaded to repository managers by developers in the UK contained unknown security vulnerabilities up 120% from the previous year, presenting huge risks to businesses. Over the course of the year, UK developers also unwittingly downloaded 68,000 known vulnerable versions of Bouncy Castle components relied upon for cryptography and 40,000 vulnerable versions of Apache Struts components implicated in the breach at Equifax. In all instances, safe versions of these same components were available for developers to download. Weeks advised: “Organisations need to have an understanding of security defects. If you’re aware of what components you’re using, and the quality of its parts, then you can make decisions on what to use in a product.”

CYBER HYGIENE For companies looking to develop safe and secure applications, Weeks offered further guidance: "You have to think about how to invest in better cyber hygiene from the beginning. You need to choose what's acceptable and put practices in place via open source governance policies - you can do this by taking steps such as downloading a component with the correct license type, and ensuring that it’s the latest versions of a component, not a version that’s nine to ten years old. "The average component or open source project is released 14 times a year, so every year there are 14 new versions; use any component that is three years old or less to give a developer access to 42 versions of an open source project, on average, without eliminating productivity." When you look at open source capabilities

T E S T I N G

it’s “pretty simple” to evaluate them and decide whether they are suitable. However, Weeks stresses that automation is critical. “We have seen organisations that use between 5,000 to 10,000 components; from a manual review cycle they are tracking up to 50,000 to 100,000 manual hours, which you can’t physically keep up with.” This is why more automated tools are being used. “For example, one customer in the financial services sector had 800 components, which were approved through manual processes and highly regulated. They spent 56,000 hours a year to ensure security practices, taking 6 to 12 weeks to get an answer back to developers. In the development community, most sprints take two weeks. The manual process was working for those with patience, but the majority didn’t have any patience because of deadlines."

Organisations need to have an understanding of security defects. If you’re aware of what components you’re using, and the quality of its parts, then you can make decisions on what to use in a product

COLLABORATE TO LEARN In addition to continually reviewing components and stopping flawed components from being designed in the first place, Weeks recommended addressing company culture to help promote security. "You have to create a culture to improve performance and look for opportunities to improve security and quality – everyone in the team needs to seek opportunities to improve the overall system and how it’s working. Some people can be too comfortable with their ways of doing things, but despite this, they need to collaborate to learn continuously and seek opportunities to improve the overall system.” For Weeks, this is part of adopting a DevOps mentality, which strives to remove siloes to reduce friction and enable rapid innovation. He has seen first hand how this approach has benefited Sonatype. “When I first joined the company we had an activity where we were able to release new versions of software every three to six months, now our sprints are every couple of weeks. We have taken automation and focused on faster releases, seeing the endto-end pipeline of what’s been achieved and collaborated better in smaller teams – allowing us to release to our customers quicker, making our employees a lot happier. "For large organisations wanting to move to a DevOps culture, I would advise top leadership to understand what DevOps can bring while being competitive and innovative – you need to be open to risks while being relentless,” he added. Weeks also noted that testers must ensure high quality and secure components to prevent clever hackers from being malicious – which is more than possible!

DEREK WEEKS VICE PRESIDENT SONATYPE

Derek is a huge advocate of applying proven supply chain management principles into DevSecOps practices, to improve efficiencies and sustain long-lasting competitive advantages. He currently serves as Vice President and DevOps Advocate at Sonatype.

T E S T M a g a z i n e | M a r c h 2 01 8

DEPLOYING IOT SOLUTIONS CTO at Testplant, Antony Edwards, warns that IoT systems are becoming increasingly complex, which includes a huge variety of apps, data systems and devices

YOUR PHONE TOLD YOUR FITBIT THAT TOLD YOUR NEST THAT TOLD YOUR SONOS THAT TOLD ME THAT YOU OWE YOUR WIFE AN APOLOGY.

TWO DOZEN RED ROSES ARE ONLY £19.99 FOR A LIMITED TIME.

s a result, testing procedures and processes are not quite ready to test Smart IoT, and, if no change occurs, it’s likely to result in delays in deployment, updates, and user acceptance. For example, how would you test this? There are so many things here most testing teams couldn’t deal with – the distributed application, the 3rd party interfaces, the non-PCs, the non-form interfaces, the validation of the result. Functional testing today is based on the ‘golden oracle’ principle – a known state, where you execute some defined steps, and you compare the output to an expected result. But if you think about typical IoT usecases, it’s very unlikely that you can set up the perfect ‘known state’, or that there’s only one right answer. The good news is that there are things DevOps teams can do today to prepare for Smart IoT testing, and they don’t rely on some magical technology that doesn’t exist or getting 100x the budget you have today. Here’s what you can do: Today

T E S T M a g a z i n e | M a r c h 2 01 8

M O B I L E

A P P L I C A T I O N

our entire testing approach focuses a lot on testing the code we produce, and not the product, or even the user experience that we deliver. We approach testing as a compliance function checking code-to-spec; that’s what all our wonderful traceability matrixes are about. And all this made sense in the old world of monolithic software. But in the new world, especially in the IoT world, products are composed of technologies from lots of different vendors, and user experiences are based on products from different vendors. So, if you only test the code you write, you’re probably only testing 5% of what’s impacting the user experience. Smart IoT really exacerbates this gap between testing code components and testing the product or the user-experience; your DevOps team needs to make sure you’re focussed on the product and the UX.

TEST CHANNEL CONSISTENCY Currently, there are multiple interfaces for accessing the core service or data – such as through a mobile interface, a web interface, a voice interface, or an on-device screen. The product may even be interacting with plenty of other products via APIs. Users should ideally be receiving a consistent view of the service or data regardless of the interface they are coming through. If two people are getting different information about where an item is located in a warehouse, this will create a great deal of confusion and errors. So, DevOps teams testing Smart IoT systems need to implement channel consistency.

AUTOMATE TESTING Currently, there’s still a huge amount of manual activity both in the process of creating tests and in the process of reviewing results and doing analysis. The complexity, the variation, and the robustness requirements of most IoT products mean that a largely manual process of test automation just won’t work. We now need to expand automation across the entire testing process to include the creation of tests, the optimisation of test execution, and the analysis of results. This can be achieved using intelligent models to autogenerate tests, with AI, ML, and analytics allowing DevOps teams to analyse data from testing, and to identify the patterns with bugs. This optimises testing to significantly improve user experience.

T E S T I N G

CONVERGE TESTING AND MONITORING Combining testing and monitoring has an efficiency benefit in that operations aren’t duplicating work that has already been tested. By converging testing and monitoring, DevOps teams can really bring the user into the automated testing process, so testing is not just product-focused, but equally userfocused. Teams observe what users really care about, what impacts their productivity, what impacts their effectiveness, and what impacts their sentiment. They can then use this to determine whether the test was a pass or failure for a Smart IoT product.

Combining testing and monitoring has an efficiency benefit in that operations aren't duplicating work that has already been tested

GET READY FOR LOAD TESTING All DevOps teams involved in deploying IoT solutions need to develop competence in load testing. Given a lot of people add IoT solutions to their existing IT infrastructure, more data may be entering your server than ever before, and very few companies are ensuring that their infrastructure can handle that much data. For example, one company had a legacy warehouse management system to manage their inventory of 400 warehouses. This involved a single PC in each warehouse, so they had roughly 400 PCs sending information to a server every hour. Load testing enabled them to discover their infrastructure couldn’t in fact handle that much data to ensure a functioning Smart IoT network, and they have since taken steps to update their infrastructure. So, for DevOps teams deploying IoT solutions, ensure that you include load testing to identify whether your Smart network can handle vast amounts of data. By incorporating all these new testing availabilities, DevOps teams can ensure their processing is ready for Smart IoT. Given that consumers are expecting more and more from their home technology, consumer tech suppliers should be increasingly keen to adopt Smart IoT networks as a basic component. But even if your technology is in place to create this network, DevOps teams and testers still need to focus on testing whether their data can be correctly processed and stored and that bugs can be identified easily and efficiently in any user environment.

ANTONY EDWARDS CTO TESTPLANT

Antony studied Computer Engineering at the University of South Wales, Australia. He worked as a developer in Sydney before joining IBM Research in New York. Moving to London, Antony joined the mobile operating system builder Symbian, moving from system architecture to eventually becoming a Vice President and member of the executive team. Prior to joining TestPlant, he held the position of CTO with a major US online entertainment company.

T E S T M a g a z i n e | M a r c h 2 01 8

NEXT GEN DATA MART: A TEST MANAGERS DREAM Wouldnâ&#x20AC;&#x2122;t it be nice if we could re-create our enterprise organisation from the ground up using technology that connected together seamlessly?

T E S T M a g a z i n e | M a r c h 2 01 8

M O B I L E

A P P L I C A T I O N

ll the systems could use the same network, tools and data. What if every team in the enterprise were on the same time zone, spoke the same language, used common processes, methodologies, and so on. Of course, this is not the reality we live in. An enterprise organisation is made by decades of patchwork technologies, methodologies, tools, practices, traditions, time zones, languages, etc. They are added layer upon layer to fit the need of the moment. This makes for a very diverse technological ecosystem. This diversity results in thousands of environmental artifacts including systems, servers, networks, versions, hardware types, and networks. Identifying these variations can be a daunting task. A few years ago, one mortgage company was working to expedite their mortgage application approval process. In the discovery phase, it was discovered that the application process required input from 64 different servers. Each of these had what was perceived as a “critical” role in the mortgage application process. The issue was further compounded by the fact that the IT Network Manager knew only a dozen of those servers. The rest were hidden in closets, behind furniture, or under desks of people whom had long since left the company. Subsequent desk residents were simply told, “Don’t touch that computer. We don’t know what it does.” Efforts to improve application efficiency became nearly impossible until the environment and workflow was addressed. Sure, you can make changes to code, but how do you test it against an unknown environment? I’d be willing to bet there are several of you who have horror stories far worse.

T E S T I N G

THE IMPACT TO TEST

A centralised test-focused

MANAGEMENT

data mart will provide

Enterprise test management is more complex than ever. It requires advanced collaboration and orchestration across a plethora of teams, environments, tools, projects, methodology, workflow, languages, time zones and more. Making sense of the myriad of different tools and spreadsheets, to track and manage integration tests, load tests API tests, CI/CD, code quality across an enterprise is daunting. Managing test coverage is another challenge. How can you manage your risk, if you don’t know where you’re exposed? To further complicate things, the pace of product delivery continues to increase. The condensing lifecycle means test teams often have to be selective about which tests they have time to run. Testing is quickly becoming a bottleneck and pain-point.

the ability to recognise, understand and respond to trends. This will enable stakeholders to identify issues that have the greatest impact

A GLANCE AT THE FUTURE Development lifecycles are only going to get faster. The current average ratio of developers to testers is 4.5:1 and it’s only expected to increase. This pressure on test teams will only continue to build. If this building pressure isn’t addressed within the enterprise, quality will continue to slip and it’s only a matter of time before disastrous results. We see these examples regularly in the news, regarding various companies who have lost millions or even billions of dollars due to an issue with their software. Many of these can trace the cause back to insufficient, or un-organised testing. As enterprises grow, the challenges of test management also grow. Coordinating and tracking geo-diverse teams, with a variety of workflows and environmental artifacts is a challenge that needs a solution.

THE SOLUTION: PROACTIVE TEST

JEFF KEYES DIRECTOR OF PRODUCT MARKETING PLUTORA

MANAGEMENT

Jeff has spent his career writing code,

The key to successful test management is being proactive. It’s not enough to just reactively test code as it’s tossed over the developers’ wall. Test managers need to centralise test data and results. They need to coordinate and automate from a centralised

running dev and test teams, consulting,

designing software features and UI, and evangelising product messaging. Outside of 6 years at Microsoft, he has primarily focused on growing startup companies.

T E S T M a g a z i n e | M a r c h 2 01 8

The value of this data and its corresponding analytics is highlighted in the developers 80:20 rule, that says 80% of the reported issues come from 20% of the code. If those trends are never identified and addressed, your development and test team will always be chasing an unattainable goal

T E S T M a g a z i n e | M a r c h 2 01 8

M O B I L E

A P P L I C A T I O N

management tool. A centralised test-focused data mart will provide the ability to recognise, understand and respond to trends. This will enable stakeholders to identify issues that have the greatest impact. Analytic tools like Tableau, Qlik Sense, R, Splunk, Power BI, Spotfire and others are great for visualising and analysing data, but what if you could also apply the same level of dynamic and predictive analytics to your test data. This would be a game changer to improve quality and manage the risks of every product release. In order for this to work, there are a few things that need to occur. The first need is to collect real-time test data from all of the different sources, with their associated user stories, sprints, version, workflows, test results, and much more. And all this has to happen without taking the tester away from testing. Secondly, that collected data would need to be collated into a centralised data mart. This would be challenging enough in a small business. However, like we said earlier, enterprise environments are anything but simplified. The value of this data and its corresponding analytics is highlighted in the developers 80:20 rule, that says 80% of the reported issues come from 20% of the code. If those trends are never identified and addressed, your development and test team will always be chasing an unattainable goal. In addition to tracking status, coverage, and knowing the risks of iterative releases, the benefits of having a data mart of test data can be realized in many different scenarios. A few years ago, analytics like this would have proven immensely useful to one particular legacy telecom company. They had outsourced development of their customer account management system to a third-party contractor. The problem was that while the company leadership was being told that the contractors were making great progress, their team of 12 contractors were actually spending over 70% of every work week fixing system crashes and recovering loss data. After a little while, one of their newly hired testers began to notice patterns in the issues being reported. It was only after a detailed analysis that the company leadership was able to see the problem and make necessary corrections. Creating a data mart from test data is not an easy thing. However, Plutora Test does this out of the box. Itâ&#x20AC;&#x2122;s an enterprise test management solution that does much more than just collect data. Plutora integrates with

T E S T I N G

tools like Selenium, Puppeteer, Maven, Jira, VersionOne VSTS and many more. Plutora unifies these tools into a single cohesive centralised test management solution. Plutora can capture data from these integrated tools and through its built-in data mart, it can show real time progress, and results. A Requirements Traceability Matrix, provides a view of test coverage allowing test managers to intelligently determine

product risk levels. Collaboration, planning, test execution, orchestration, and the ability for individual teams to manage their own custom workflows, are all native features. The ability to kick off a batch of automated Selenium test scripts, and automatically log defects into tools like Jira are all part of what the next generation of intelligent data marts can do. Of course, those features are in addition to having a full suite of built in configurable reporting tools through a partnership and powerful integration with Tableau, allowing you to perform descriptive, predictive and prescriptive analytics. The real value of a data mart is realised in its ability to be accessed and drilled into via reports and dashboards from any geographic location by any stakeholder throughout the enterprise and on a variety of devices. While data marts for enterprise test management are relatively new, the application are endless. Test management solutions like Plutora allow you to personalise your enterprise solution around your data mart. This ensures that you and your stakeholders will always know testing status, results coverage and where you need to be in the future. What does your ultimate test data mart look like?

NEWs Videos Product News Jobs white papers

bookmark now

www.softwaretestingnews.co.uk T E S T M a g a z i n e | M a r c h 2 01 8

LOOKING TO ACCELERATE MOBILE APP DELIVERY? AGILE ISNT ENOUGH Creator Project Lead at Ionic, Matt Kremer, focuses on how IT leaders need to embrace agile and DevOps as part of their mobile app strategy

n the earliest days of app development, most teams used what is called the “waterfall” method – a rigid sequencing of activities from requirements gathering to development, testing and production. The problem with waterfall is that it took too long to get a new product or feature in the hands of users. As someone once said, “waterfall is great for delivering the product you needed a year ago.” Agile development emerged as a welcome alternative to waterfall. The agile method focused on breaking down requirements into

T E S T M a g a z i n e | M a r c h 2 01 8

much smaller chunks of work (perhaps a single feature) that could be developed iteratively over time, based on user input and changing market dynamics. It also got teams to work in very short intervals, or “sprints,” of typically one to two weeks. The agile approach is a better fit for the software-as-a-service (SaaS) era and has been widely adopted by app dev teams. But, there’s a catch. While agile has become widespread, the frequency of production releases hasn’t changed much since the waterfall days. Many teams are still

M O B I L E

A P P L I C A T I O N

bundling features and shipping major releases just a few times a year. A recent survey found that half of app dev teams released applications four times a year or less. As one Forrester research article puts it: "Agile product management tools improve reporting and visibility, but they don’t improve other essential activities in the delivery pipeline, like building, testing and deploying software.” Effectively, the bottleneck has shifted further down the delivery chain – from planning and development to building, testing and deployment.

WHY SPEED IS SO IMPORTANT TO MOBILE Thanks to the high standards set by popular consumer apps like Twitter, Pinterest and Sworkit, mobile users have become conditioned to expect regular app updates, new features and rapid fixes. While infrequent updates might be okay for some enterprise use cases, it doesn’t work well in mobile. Instead, user expectations put pressure on enterprise teams building mobile apps – whether the app is for employees, partners or customers – to deliver new releases on a nearly continuous basis. In a few cases, the business model requires teams to ship production releases in days or weeks. Acker Wines is a highly successful wine merchant that auctions off premium wines to discerning buyers, and their auction cycle runs every two weeks. Napa Group, a digital consultancy responsible for building the Acker Wines mobile app, has just two weeks to build, test and release new features before the next auction. Bundling features and shipping semiannually, or even monthly, is not an option.

SHIPPING CODE FASTER To fulfill the vision of agile development, we need to look beyond agile. One avenue is DevOps. DevOps is a squishy and often overused term, but it’s an important concept for app dev leaders. Essentially, DevOps extends the different agile tools and methodologies beyond the development team to include the operations and QA teams that are responsible for putting production code in the hands of users. In the simplest terms, DevOps "ensures that the working software sitting in a developer's laptop reaches the production phase easily and quickly." The dream of DevOps is to move at the speed of development. That means being able to build, test and release working code as soon as it’s ready.

T E S T I N G

THREE WAYS DEVOPS ACCELERATES APP DELIVERY With that in mind, here are three concrete ways that DevOps tools and practices help development teams be more successful. 1. CONTINUOUS INTEGRATION: When a developer commits a code change to a source code repository, CI automatically builds and integrates the changed code and prepares a new version of the app for testing or release. CI accelerates the pace of delivery by streamlining the build process and by providing a shared environment where multiple developers can contribute code changes at the same time. 2. CONTINUOUS FEEDBACK: Automated testing, real-time error monitoring and user testing tools provide continuous feedback for product teams to improve app quality and fix problems at their source. Continuous feedback helps to make sure teams are building the right product and increases the pace of updates and fixes to address UX issues. 3. REAL-TIME APP UPDATES: In the mobile space, app developers are often at the mercy of the app stores to review, approve and publish production releases. Even when teams are moving fast to fix a bug or push a key feature, they often have little control over when a user gets the update. How can you get around this? Some mobile DevOps tools let app publishers send changes directly to users in real-time, without going through the app stores. This puts control back in the hands of the publishers and can drastically reduce the time it takes to get a production release in the hands of users.

AGILE AND DEVOPS = HARMONY App dev leaders need to look critically at their culture, tools and processes to see if they’re fulfilling the vision of agile, or if they’ve just found a more efficient way to deliver the same user experience as waterfall. To keep pace with user expectations, IT leaders should look beyond agile and embrace DevOps practices to accelerate app delivery. Of course, the best approach is to combine agile with DevOps. Putting them together creates a powerful combination that should set the standard for how mobile apps – and software in general – should be delivered in 2018 and beyond.

In the mobile space, app developers are often at the mercy of the app stores to review, approve and public production releases

MATT KREMER CREATOR PROJECT LEAD IONIC

Matt Kremer is a developer, product owner and evangelist at Ionic, as well as the maker of the popular open source Ionic Framework. He is currently the Product Lead for Ionic Pro, a mobile DevOps solution that helps developers and teams design, build, test and ship app updates at the speed of development.

T E S T M a g a z i n e | M a r c h 2 01 8

A DEVELOPER'S PLAYPEN

How is the DevOps revolution affecting the testing community, and how should the testing community embrace DevOps?

T E S T M a g a z i n e | M a r c h 2 01 8

D E V O P S

C O N T I N U O U S

he rise of DevOps has changed the face of systems development in what seems to be a remarkably short space of time. If structured methods and waterfall approaches managed by Prince 2 were systems development version 2.0, DevOps is definitely a reboot to version 3.0. DevOps is quite literally the integration of development and operations, traditionally two opposite ends of a spectrum. Development implies change and operations require stability. The promise of DevOps is to bridge this gap and supply good quality software with shorter release cycles and be reactive to a business’s rapidly changing needs. A company can react to a competitor in days rather than months. The reality is the seeds of DevOps, which has been maturing for many years, however, in the last couple of years, these seeds have blossomed into maturity to enable a revolution in systems development. Agile methodologies, which always required continuous integration and supported continuous delivery, are now coupled with a growing range of rapidly evolving open source tools that are powering the DevOps movement forward. The open source movement has united developers and is providing development tools that are perfectly tailored to the DevOps approach. Cloud providers such as Amazon can now supply a working environment on demand; it is a developer’s playpen.

QUICK MOVING PROJECTS I don’t want to write this article about the increasingly rich toolkit of DevOps; instead, this article is about how the DevOps revolution is affecting the tester community and how the tester community should embrace DevOps. The role of the tester is changing, but test skills are still required. DevOps places a large dependence on testing, high quality, repeatable automated testing. Applications cannot be released without being tested. So how does the tester fit into the DevOps world, or perhaps it’s better to ask how testing is performed, as the definition of a tester is now possibly changing. Fundamentally, I am a developer at heart and I love quick moving projects with quick delivery, so I’ve embraced agile and the DevOps world. The tools I’m seeing now are what we dreamed of even a few years ago. But what is the tester in the face of this revolution, and what is the role of the specialist tester as distinct from the test function?

D E L I V E R Y

Well, developers have always done unit testing. This doesn’t change in the DevOps world. However, because of the use of automated testing tools, continuous integration and automation servers, unit testing is a lot better in the DevOps world. A good developer will be expected to produce high-quality code and an associated unit test suite as a standard. Unit test level bugs should be reduced dramatically. Also because the unit tests are now tightly coupled to the code, as re-running the automated test suite can carry out regression testing.

The reality is the seed of DevOps, which has been maturing for years, however, in the last couple of years, these seeds have blossomed into maturity to enable a revolution in system development

PASSING A TEST The use of techniques such as BDD, (Behavioural Driven Development), TDD, (Test Driven Development) and ATDD, (Acceptance Test Driven Development), in the DevOps world show the importance of testing to the success of DevOps approaches. Continuous testing is the key to the success of DevOps. For example, making use of TDD, a java developer will probably produce a set of Junit tests, which will define the definition of done for the code they are about to produce. They will then produce code and run it against the Junit tests, fix any failures, then keep repeating the process until all test has passed. At this point, the code will be considered done and it can be integrated into the master branch of code. The Junit tests will also integrate so they can be run whenever regression testing is required. So, not only has the code been developed and successfully unit tested, it can be regression tested at any point in time in the future without any additional effort or time. Unit testing tends to be quite specific to low-level details. BDD and ACDD approaches tend to address the user behaviour level and the user acceptance criteria level. Essentially this means the use of business scenarios that will reflect the behaviour of the function in operation. An example of this would be the use of Cucumber/Gherkin in which the tester can test business scenarios and create an automated suite of tests that can test the code against the acceptance criteria. To write tests in Cucumber/Gherkin requires minimal technical skill and in some cases can be done by the intended users of the system, if requirements were defined in Cucumber/Gherkin then they would produce a testable set of requirements at the outset of any development. However, running and maintaining these tests does require technical skill that end users and manual testers will not possess.

WAYNE TOFTEROO THE HEAD OF TEST, COMMON SERVICES DVLA

Wayne has provided test leadership for enterprise wide transformations and strategic projects in telecoms and financial service sectors. He is a proven innovator and founder of several innovative start-ups, including ID business solutions and drug discovery software.

T E S T M a g a z i n e | M a r c h 2 01 8

D E V O P S

DevOps is still a powerful

DEVELOPMENT

enabler, and its power to

LANGUAGE

transform will make it a more powerful influence as companies transform

C O N T I N U O U S

It is clear that in order to operate as a specialist tester in a DevOps squad, the tester should have good test automation skills and it would be useful if the tester had some knowledge of the development language and the environments being used. The tools mentioned above, as well as the many tools that also support TDD, BDD and ACDD require technical skills to be used to their maximum potential. Automated regression testing eliminates any errors that can be introduced in a manual test phase and it can be run very quickly, within the time span of the actual change. DevOps has formalised the need for a new role called Software Development Engineer in Test, an SDET. An SDET will be required to perform test automation, creating test infrastructure that can support those tests and potentially support different environments and different languages. This is a role that primarily supports the test function and enables tests to be automated quickly and efficiently by the use of development engineer skills. Automated test tools and SDETs does not minimise the need for good testing skills, the quality of the product depends more than ever on the ability of the DevOps squad to put in place a good set of tests that will ensure the product meets the expectations of the end users, itâ&#x20AC;&#x2122;s no good being fast and wrong! Testing tools will not tell you how or what to test. Skilled testers and developers will need to do that.

MOBILE MARKETPLACE In a competitive market online, or a mobile marketplace, the need to react quickly to a competitor is essential to success, being first is important but not being too long getting to second place is potentially about survival. The ability to make a change, create a new customer-facing feature or integrate a new supplier, in a short space of time and be able to automatically regression test your platform without any additional effort may enable you to get that change in a few days rather than weeks. We know that companies, with a good DevOps approach to testing, supported by a well built and managed continuous integration and continuous delivery pipeline, can delegate change to individual DevOps squads who can operate autonomously and manage rapid changes to defined areas of functionality. This can be done because a full regression set can be run whenever a change is implemented.

T E S T M a g a z i n e | M a r c h 2 01 8

D E L I V E R Y

Where a business is able to exploit this is where change is not centrally managed but can be delegated to the appropriate business unit, in this case, the path for a change going from perceived business need to implemented, as software supporting customer growth can be very short. So are we talking about tester version 3.0, or a reboot of the tester for the DevOps world? I think we probably are, specialist testers will need to learn automation and appropriate technical skills. Testers canâ&#x20AC;&#x2122;t be insulated from having to deal with tools such as Docker and cloud environments. However, at the moment, the reality is that very few DevOps projects operate in a totally green field. Most projects have to deal with integrations into existing legacy systems, adhere to corporate and regulatory frameworks, be developed to in-house processes and procedures and be subject to the corporate political influences. DevOps teams may well need to co-exist with existing operational and support teams as well as manage requirements from business change teams. All these external stakeholders will have test requirements that need to be met.

TRANSFORMING SYSTEMS There is also the need for system integration testing and in large complex systems with legacy components and that will not be easy. Specialist integration testers may well be required. Automation of integration tests in these complex hybrid environments is difficult. New approaches may well need to be found. The need to integrate specialised legacy integration environments to support the endto-end functionality will place challenges to the automation process. This is a more complicated environment and the role of the tester will obviously be wider than meeting the development demands of the DevOps team. A strategy often employed is to roll out the DevOps approach to the more dynamic areas of an enterprise, such the online and mobile retail offerings, while change in other areas of the enterprise is implemented and managed by development and operational teams. Even in this environment, DevOps is still a powerful enabler, and its power to transform will make it a more powerful influence as companies transform, their IT operations and transform their systems. But it will happen over time, for the moment, nearly all new tester roles will require automation skills as companies will at the very least look to leverage the benefits of test automation even if they do not have a full blown DevOps implementation.

froglogic Squish

GUI Test Automation Code Coverage Analysis

cross platform. multi language. cross design. Learn more and get in touch: www.froglogic.com T E S T M a g a z i n e | M a r c h 2 01 8

TRANSFORMING FROM TESTERS TO QUALITY ENGINEERS How should we become champions and advocates for testing and quality activities across the lifecycle?

he new world of modern software delivery, ushered in by the impact of digital disruption, offers tremendous opportunities both to improve solution quality at the desired speed and to develop new capabilities. However, as testers, we need to acknowledge that in order to ensure that software quality remains at the forefront of the mindset of everyone involved throughout the development lifecycle, our role and approach will have to change. CULTURAL CHANGE IS REQUIRED There is general recognition that the cultural changes required for an organisation to successfully move into the world of DevOps/ CI/CD can prove a much bigger barrier than the technical challenges. To survive and thrive in this new world we need to develop a continuous testing

T E S T M a g a z i n e | M a r c h 2 01 8

culture across our function, complimenting and supporting a wider continuous delivery culture. We must become the champions and advocates for testing and quality activities across the lifecycle. This should bring what we have long sought – involvement right from the outset. Rather than managing towards ‘hand-offs’ into segregated testing phases, we need to foster the approach of collaborative work and joint problem solving. The wider culture, typically adopted in this modern delivery world, of trying new approaches and tools and discarding those things that don’t work/add value, can help us in testing to also be more innovative. We can, and perhaps should, become more comfortable in experimenting with new techniques and ways of working in the quest to improve quality.

D E V O P S

C O N T I N U O U S

D E L I V E R Y

Rather than managing

NEW FRIENDS AND ALLIES The advent of DevOps also brings the opportunity to work more closely with our colleagues in operations. We share goals of protecting production and the customer from poor quality and resulting outages. Often those areas that the operations team are concerned about and want to see closely monitored are areas that we should be focussing our quality lens on too. On the flip side, the instrumenting and tools we utilise in testing may also deliver benefit to our operations colleagues. SINGLE VIEW OF THE RISK In this world of modern software delivery and improved collaboration, we have an opportunity to develop a single, joint view of the level of risk and where this lies across the solution landscape. When we have achieved this single view, there should be much greater buy-in from our technical colleagues and senior stakeholders into our proposed approach to testing, which should be closely aligned to that same view of risk. In turn, this should avoid any suggestion that the scope of testing/QA is inappropriate and might even ‘kill the business case’. ESTABLISHING A ‘RUTHLESS AUTOMATION’ MINDSET The challenge in this new world for all disciplines including testing/QA is to think about how to automate all tasks as far as possible almost as soon as they are identified. This has been referred to by Forrester as ‘ruthless automation’. This approach of maintaining a constant eye on the challenge of frequent perhaps daily or even more frequent releases is an important shift in mindset. TRANSFORMATION FROM TESTERS TO QUALITY ENGINEERS It has been suggested that what is required to deliver solutions is actually ‘T-shaped technologists’, individuals with a dual focus. Firstly, they have their specialism, perhaps in development or more likely in our case test automation, exploratory testing or test data management, but then secondly, they must have the overall end-to-end delivery focus required. This means we will have the opportunity to develop new skills, become more technical if we have the aptitude/ desire, but certainly move closer to the other functions. We need to add value by advocating and driving quality across all of the different delivery functions and activities. From the initial story definition, through to the method

towards ‘hand-offs’ into segregated testing phases, we need to foster the approach of collaborative work and joint problem solving of deployment, we can champion quality. This must include ensuring that the final delivery we are working towards meets all the necessary quality criteria. We have the opportunity now to re-invent ourselves and become quality engineers rather than merely testers. AVOIDING THE PERILS OF BURN-OUT When we establish what constitutes a sustainable pace of delivery, we need to consider not just technical considerations but our engineers and avoiding them becoming stretched beyond reasonable work commitments. As managers, we need to ensure our quality engineers have a work/life balance that is appropriate. The benefits of any improved delivery speed could be very short lived if in the process our talented engineers suffer ‘burn-out’ and consequently become less productive. EMBRACING THE OPPORTUNITIES The new era should ultimately (appreciating this won’t happen overnight!) bring with it a clearing of the technical debt that previously meant being stuck with troubleshooting problematic legacy systems. This in turn should mean greater opportunity for our teams to work on more exciting new work. Professional testers and those working in QA share a genuine passion for delivering quality solutions. The new world of modern software development offers us opportunities to develop new skills and have a greater influence, and ultimately play a pivotal role in delivering better quality at the speed demanded by today's demanding and perhaps more discerning customers. Rather than seeing the new delivery methods of DevOps/CI/CD as a threat to our existence, we need to evolve our teams and individuals to meet the challenge. Ultimately, we need to transform ourselves to become fully fledged quality engineers and in the process, earn enhanced trust and respect from our colleagues.

RICHARD SIMMS TEST ARCHITECT ROQ

Richard is responsible for designing test strategies and approaches to testing as well as managing the transition process for new clients at ROQ. His role as Test Architect involves working with a wide variety of clients across a range of industries and gaining a deep insight into their testing and quality assurance challenges. Before joining ROQ, over a career in IT spanning 30 years, Richard has worked in the roles of Developer, Team Lead, Project Manager and Test Manager before taking on the role of Quality Manager for a major retailer.

T E S T M a g a z i n e | M a r c h 2 01 8

TECHNICAL AND CULTURAL BUILDING BLOCKS Director of DevOps at ADP UK, Keith Watson, plots a course from a monolithic product design with legacy build and deployment processes to a continuous delivery process into the cloud for API micro-services using DevOps tools and techniques

T E S T M a g a z i n e | M a r c h 2 01 8

D E V O P S

C O N T I N U O U S

D E L I V E R Y

CONTINUOUS

PLAN DEVOPS AS A

Starting with a clear

DELIVERY THROUGH

BUSINESS CHANGE

vision and strategy for

DEVOPS: WHERE TO

PROGRAMME

the specific outcomes we

START?

Tools are necessary but not a sufficient condition for DevOps to succeed. As well as new tools and methods, it is important to understand that business processes and cultural habits will also need to change. Changes will often experience resistance unless benefits when implementing DevOps can be demonstrated. Because of previous deployment issues, most companies will have additional governance processes and separation of duties to reduce the risk of production issues. This will naturally create silos, which are anti-patterns to DevOps success. Breaking down these barriers must be done in a sensitive way and not imply lack of commitment or professionalism from either side. Building relationships and demonstrating deployment competence of the development teams are often the only way to change opinions. This takes time and effort. John Kotter’s 8 stage change process is a useful tool to manage business change (see diagramme on next page). The key to this is building a guiding coalition. Initially, it is important to spend time building relationships across the different disciplines and silos. This helps determine whom the key decision makers are, what strategies and tools are already being used, and what degrees of freedom there are to make changes. Finding ways to demonstrate business value (shortterm wins) are an excellent way to gain trust and respect across the various stakeholders and between silos.

Unless you are a greenfield start-up, most companies will have a number of IT products which are built on code developed and deployed using processes created from days well before DevOps.

HAVE A CLEAR STRATEGY FOR CONTINUOUS DELIVERY THROUGH DEVOPS Starting with a clear vision and strategy for the specific outcomes we desire has allowed us to put in place the technical and cultural building blocks in our product for the permanent transformation of our product delivery pipeline. We have a simple but clear vision to: • Have a holistic approach that optimise the efficiency of the whole system, not just some parts • Re-engineer the product architecture and the release processes to improve deployment frequency and deployment lag • Demonstrate improved behaviour of the product in production to the operations teams • Conform to all appropriate company compliance standards • Ensure all stakeholders in the delivery pipeline are involved in creating the solution. Our aim is to deliver this vision using the "configuration as code" DevOps model with standard software engineering principles. The outcome we desire is to reduce the time from code commit to the deployment of high quality, highly secure small functional artefacts into production.

MAKE AUTOMATED TESTING PART OF YOUR DEVOPS STRATEGY It is also important to understand potential objections and address those objections in any DevOps strategy. For example, it is vitally important to prove that the new processes deliver higher quality and more secure artefacts than previous deployment pipelines. This means implementing a “shift-

desire has allowed us to put in place the technical and cultural building blocks

KEITH WATSON DIRECTOR OF DEVOPS ADP UK

Keith is responsible for the promotion and adoption of continuous delivery and DevOps principles, patterns and tools within the development and operations teams for one of ADP's flagship products. Before joining ADP, Keith has led various software development and operations teams in IBM and Ordnance Survey.

T E S T M a g a z i n e | M a r c h 2 01 8

D E V O P S

C O N T I N U O U S

D E L I V E R Y

Using automated testing enables us to provide quality gates earlier in the continuous delivery pipeline. This ensures greater confidence in deployments into test environments and eventually into production

Source: https://www.kotterinc.com/8-steps-process-for-leading-change/

leftâ&#x20AC;? testing policy particularly by using automated testing earlier in the software development cycle. In many organisations, testing is often performed late and usually includes manual or semiautomated testing. Using automated testing enables us to provide quality gates earlier in the continuous delivery pipeline. This ensures greater confidence of deployments into test environments and eventually into production. It also enables regression testing to be performed when modules are changed reducing the risk for production deployments. This does, however, have implications for the new skills and behaviours required in both software development teams and testing teams.

appropriate project to demonstrate business benefit quickly is often a better approach than company-wide initiatives over a longer period. Which tools are chosen are usually less important than adopting DevOps policies and principles such as using recipes, adopting version control for all code (not just source code) and using appropriate pipeline orchestration. As with any coding discipline, our knowledge of how to build pipelines grows as we gain experience writing more and more complex pipelines.

USE APPROPRIATE

DELIVERY AS A WAY OF

WORK IN PROGRESS â&#x20AC;&#x201C; CONTINUOUS

TOOLS AND RECIPES TO LIFE VISIBLY DEMONSTRATE BENEFITS Continuous delivery is such a different paradigm to the usual deployment methods that stakeholders, particularly budget holders, need to be shown why investment in tooling is important. Adopting some tools and choosing an

T E S T M a g a z i n e | M a r c h 2 01 8

While there is still work to be done, the early results we have seen in our product have demonstrated the business value to various stakeholders and allowed us to gain credibility within the development and operations teams. Each new delivery gives us more confidence and enables us to grow our knowledge in building relationships across teams and using DevOps techniques.

registrations are now open KEY DATES 24 August – Booking & Submission Deadline 12 September – Finalists announced 16 October – Awards Gala Dinner

he DevOps Industry Awards celebrate companies and individuals who have accomplished significant achievements when incorporating and adopting DevOps practices. The DevOps Industry Awards have been launched to recognise the tremendous efforts of individuals and teams when undergoing digital transformation projects – whether they are small and bespoke, or large complex initiatives. Enter The DevOps Industry Awards and start on a journey of anticipation and excitement leading up to the awards night – it could be you and your team collecting one of the highly coveted awards.

Be a winner - REGISTER TODAY

 www.devopsindustryawards.com T E S T M a g a z i n e | M a r c h 2 01 8

SHIP SAFELY TO SEA AND START CURSING AT MAXIMUM SPEEDS How can you utilise test automation in a more “fruitful way”, with simple tweaks to test automation processes?

T E S T M a g a z i n e | M a r c h 2 01 8

D E V O P S

C O N T I N U O U S

evOps is one of the buzzing words in the software development and delivery industry. Everyone talks about DevOps and has put these words into action in most development and product delivery organisations. To put it short, DevOps is a set of practices that we will undertake to minimise the time taken to develop a change and deliver it to the final customer with high-quality. DevOps goes hand in hand with other well-known practices, such as agile and continuous delivery. The need for DevOps arose from the increasing success of agile software development, as it led organisations to want to release their software faster and more frequently. DevOps and continuous delivery share a common background in agile methods and lean thinking: small and frequent changes with focused value to the end customer. To facilitate DevOps we use automated tools for each and every part of the product development stages (code, build, test, package, release, configure and monitoring). To help the DevOps pipeline in an organisation, many software delivery institutions have now embarked on test automation. With proper test automation tools and practices implemented we are able to test and deliver software with high-quality. At the same time, there are a lot of repercussion and nightmares that organisations will face when undertaking the wrong test automation strategy.

UTILISE TEST AUTOMATION We all deliver web solutions to our customers. Facebook, Google, Amazon and Microsoft are some of the leading software giants who deliver web-based solutions to billions of users worldwide. These solutions utilise web services and sometimes microservices architectures. Therefore, we embark first on the development of these web services before putting the final UI touches on the system. To facilitate faster delivery we should first automate the testing of such web services rather than waiting for the UI to be integrated into the delivery feature. We should use more web services, test automation tools and test frameworks to extensively test web services and microservices architecture. It is a good practice to have more test coverage on web services rather than with a UI based functional tests. The more test

D E L I V E R Y

issues we see at the foundation level, the more that are better off in the final stages of testing. These tests should look at testing web services alone, by integrating with other web services and testing against the contracts established. Web service testing can be automated with tools like RestAssured, Kataloon Studio, Postman and Karate. These are well-known tools in the industry. Web service testing should not limit to functional aspects. It should also be used for performance and security aspects; tools like JMeter, Load UI and TestingWhiz can be used to test the performance of a web service.

FACILITATE CONTINUOUS DELIVERY In DevOps, we emphasise on continuous delivery. To facilitate continuous delivery we use continuous integration. Most of us do wrong when using continuous integration for test automation. We only push UI functional automation suites to continuous integration, so automated regression has to wait until the delivery of a new UI option. We should move away with such practices. The best way is to push our service level automated test into the continuous integration. This allows maximum leverage of our test automation practice. The automated security test at services level should also move into the CI process, which allows us to identify any security risks at earlier stages of development. Use of the ‘Behavioral Driven Development’ (BDD) approach is another area that also helps application development and automated test development. BDD facilitates creating development code, as well as automated code, to what is actually required by the customer, thus eliminating any requirements gaps in delivery and also reworking. We should have a practice where we write automation scripts based on the scenarios (features) accepted by the customers. There are so many add-ons we can utilise for our test automation framework to facilitate BDD. Cucumber for Java and JavaScript, RSpec for Ruby and SpecFlow for .net are some of the add-ons that help this integration.

CODELESS TESTING We should have a test automation strategy to automate test quickly to keep in phase with the delivery pipeline. Rather than

With proper test automation tools and practices implemented we are able to test and deliver high quality software. At the same time, there are a lot of repercussion and nightmares that organisations face when undertaking a wrong test automation strategy

KUSHAN SHALINDRA AMARASIRI SENIOR QA CONSULTANT VIRTUSA

Kushan Amarasiri is a Test Automation Professional with more than 14 years’ of experience in technical QA, who has worked in agile and DevOps environments. Currently he is serving Code House group, a UK owned software delivery organisation in Sri Lanka.

T E S T M a g a z i n e | M a r c h 2 01 8

D E V O P S

opting for tools that need extensive automation coding, it’s a good practice that we use tools that needs less coding or are simply codeless. This helps us to reduce our efforts on test automation, and instead, speed up the test process. Codeless test automation also helps non-technical quality assurance teams to get involved in test automation with no background in coding knowledge. If they know how to test, they can automate. To put codeless test automation into practice we can use test automation tools like Leap Test, Testim.io or Test Craft. Machine learning and the use of artificial intelligence (AI) is also an area of focus today, as it will help teams to bring down testing efforts and speed up delivery with high quality. An AI-based test tool should be capable to come up with the test scenarios for a feature, with maximum coverage and design to test automation scripts, while having the ability to execute when necessary. There are two great AI automation tools available in the industry. Namely, Appvance IQ and Mabl. With Appvance you can create 1500 test cases in just 5 seconds; a dramatic improvement in testing.

T E S T M a g a z i n e | M a r c h 2 01 8

C O N T I N U O U S

OPERATING SYSTEMS To enhance our DevOps pipeline, and to leverage the maximum of test automation, we should also utilise the cloud-based testing solutions. These solutions enable us to test against a wider range of operating systems, mobile devices and even web browsers, which we cannot have in-house due to high investment and maintenance costs. Automated testing in such solutions helps to assure that the application delivered works on different platforms and environments. The solutions can utilise our BrowserStack, CrossBrowser Testing, Sauce Labs and Perfecto Cloud. Another way we can help faster testing, and high-quality delivery, is to use tools to help manual testers do more in-depth debugging, helping hunt down more hidden bugs. Tools like Postman and Fiddler helps us to debug web services more in a 'drill down' fashion. We should also keep in mind that our test automation suite should have the maximum code coverage of the application being tested. There should

D E L I V E R Y

be more emphasis on code coverage tools like NCover, dotCover, Atlassian Clover and Cobertura. Looking at test automation code coverage helps us to ensure that our test automation code validates most parts of the application.

DEVOPS SUCCESS With DevOps, a test automation engineer’s role expands as they perform more of a developer role. They will have to create automated unit tests to test the developer code as they perform a test drive approach. Automated unit testing is another area where test engineers can embark upon. Therefore, the quality engineering team should work on TestNG, Junit, Mockito and Powermock. So, adding to the final points, we need test automation to facilitate DevOps, as it’s a major contributor to DevOps success. So these tweaks that I have highlighted in this article serves a lot of DevOps practitioners around the world to put the ship safely to the sea and start cursing at maximum speeds.

12-13 June 2018 The Principal York, York

The first ever DevOps Conference North will take place at the luxurious Principal York Hotel on 12th and 13th June 2018. The DevOps Conference North is produced by the same organisers as The National DevOps Conference, which has been running in London since 2016, after it became apparent that a replica event was required in the north of England to cater for those in our northern cities, as well as our friends and colleagues in Ireland and Scotland. The DevOps Conference North is a UK-based conference that provides the IT community with invaluable content from revered industry speakers; practical presentations; roundtable discussion forums that are facilitated and led by key figures; as well as a market leading exhibition, which will enable delegates to view the latest products and services available to them. In 2017, the National DevOps Conference featured speakers from many well-known companies, including NHS Digital, ITV, Tesco and HSBC to name but a few; as well as a wide selection of presentations and a major exhibition, which saw companies such as Sogeti, Contino, Interoute and CA Technologies showcasing their latest products and services. DevOps Conference NORTH has its finger on the pulse as it is owned and supported by the industry leading journal TEST Magazine and is produced by the organisers of The DevOps Industry Awards. No other DevOps conference in the UK or Europe can boast these credentials. It’s here that you get access to the authoritative analyses and ground-breaking research that will enable you to keep one step ahead of market trends.

£525.00

Day 1 Only*

£325.00

Day 2 Only*

£325.00

*Includes entry to the Evening Drinks Reception on the first day of the Conference. All prices are subject to VAT.

 0203 931 5827  info@devopsevent.com  http://north.devopsevent.com T E S T M a g a z i n e | M a r c h 2 01 8

MANUAL TESTING:

WANTED DEAD OR ALIVE

TEST Magazine Journalist, Leah Alger, approaches a number of software testing assets to find that software testing is “not dead – it’s alive with new approaches”

T E S T M a g a z i n e | M a r c h 2 01 8

D E V O P S

C O N T I N U O U S

s we all know, the “death of manual testing” has been a key topic discussed at meet up’s, conferences, and in blogs and magazines for years’, and with Microsoft’s Director James Whittakes proclaiming “testing is dead,” you would think this type of testing really has popped its clogs. Gone are the days when a software tester would ‘test’ his/her own code and send it to production; monitoring systems, delivery systems, artificial intelligence, test automation, continuous integration, microservices and rollback capabilities have taken the spotlight. But does this mean we don’t need, nor want, the helping hand of a manual tester anymore? “Manual testing covers the most complex scenarios that involve a lot of human judgment and decision making. Machines add value where completing repetitive tasks such as regression, and humans excel in tasks involving experiment, judgment, creativity and wit,” said Dileep Singh Marway, QA Manager at The Economist. It can be prohibitively expensive to design test automation scripts for new products that are still in development, therefore it’s constantly changing. Keeping automation scripts current could end up costing more than the software is worth, so the only way to get the application stable enough for automation is through manual testing, according to Mario Mathee, COO at DVT in a blog post.

TECHNOLOGICAL REVOLUTION “Testing is completely changing, making us more efficient. We have the opportunity to bring new levels of automation to functional and non-functional testing. World-class performance testers who used to spend countless hours manually assessing code with a few tests can now focus more time on automating hundreds of tests that will results in higher quality applications being released,” shared Derek Weeks, Vice President at Sonatype. With Google’s CEO Eric Schmidt saying the key for humans who want to succeed in the future will observe this “separation of powers by specialising in what we do best, while collaborating with computers”, it appears that we need to take into consideration how to deal with this technological revolution.

D E L I V E R Y

To stay “ahead of the game”, testers must abide by the following: • Stay up-to-date with technology trends by locating and studying publications written for workers in your field • Attend workshops and training sessions, including outside of work • Streamline your technological experience by making a list of all the programmes you use • Eliminate technological programmes that you do not need to help to stay informed about remaining applications • Schedule regular meetings with technology support.

Modern software developments mean testers need to be adaptable and willing to learn new skills

THE CREATIVITY OF A HUMAN “Modern software developments mean that testers need to be adaptable and willing to learn new skills. It is important testing is done from a different point of view as it’s going to get more and more technology driven as time goes on,” added Marway. In the early stages of this tech-driven revolution, artificial intelligence requires the creativity of a human to drive the test strategy, according to Marway. “Humans are still far superior than computers when it comes to tasks that involve the utilisation of intelligence. A person can write a poem, sing, dance and play music; although a machine can be programmed to do such tasks, they do not possess the ability to create or have intuition like us humans – we are far more fallible,” he continued. Another modern software development practice evolving the role of software testers is DevOps. This is because of its goal to automate as much as possible and to remove wasted effort from development processes. “DevOps has absolutely changed the way software testing works. Traditional practices see codes thrown back and forth over the wall between development and test that force code changes and repeated testing. In DevOps, test experts and automated testing can be embedded earlier into the development lifecycle resulting in improved efficiencies,” said Weeks.

DEVOPS PRINCIPLES For a tester to be successful in a DevOps environment, Marway recommends the

DILEEP MARWAY QA MANAGER THE ECONOMIST

Dileep has more than 10 years’ experience as a Software Testing Professional and is extremely passionate about QA, with his key principle being “putting customers first”.

T E S T M a g a z i n e | M a r c h 2 01 8

D E V O P S

Keeping automation scripts current could end up costing more than the software is worth, so the only way to get the application stable enough for automation is through manual testing

C O N T I N U O U S

following: • Provide training to DevOps engineers on testing methodologies • Become more involved in ensuring quality in the test environment deployment, packaging and other components of the application lifecycle • Discuss code quality for deployment tools, not just core applications. The objective of DevOps is to make software delivery quick, while ensuring that it’s delivered to high-quality outputs. All processes touch upon aspects of building and developing software as part of the latter stage: • • • • • • •

Code – manage, develop and review Build – continuous integration Test – continuous testing Package – artefact repository Release – release automation Monitor – end-user experience Configure – infrastructure configuration

Marway believes that, despite DevOps principles and practices helping drive quality assurance and testing by improving communication and collaboration between all essential stakeholders, it is eating up part of the manual testing quota, due to the fact it encourages automation.

ENTERPRISE IT It’s important for DevOps engineers and their teams to always be on the lookout for tools in order to improve productivity and efficiency. Some key DevOps tools include: Jenkins, SolarWindsLog & Event Manager, Vagrant, PagerDuty, Prometheus, Ganglia, Snort, Splunk, Nagios, Chef and SummoLogic, according to Stackify. Nelson Ajoku, Scrum Master at HM Land Registry, stated: “As long as tests are still running, the scope of software testing is by far not dying because of manual testing being automated. If you’re deploying, as you should be, manual testing should still be needed. As for DevOps, you still need to confirm congregations; it’s just a way of testing your code after testing it manually.” Furthermore, machines/artificial intelligence (AI) is becoming more agile due to its ability to adapt to real time, without the need to be programmed in advance to tasks. With the likes of Amazon,

T E S T M a g a z i n e | M a r c h 2 01 8

D E L I V E R Y

IBM and Google embracing artificial intelligence, some have wrote online that robots are more than capable to deliver faster, more accurate analyses than humans when analysing data sets. “IBM and Google aren’t the only companies applying AI techniques. Within the past year, AI in software testing has become feasible. Software testing must evolve in response to the shift to agile and DevOps. No matter how many testers you employ, it’s simply not possible for manual testing to provide agile developers immediate feedback on whether any of their constant changes impacted the existing user experience,” wrote Tricentis in a blog post.

REMAINING RELEVANT “It’s not possible for machines to become agile enough to not require any creative input from a human being. The stunted social skills of AI highlight to me that people's jobs are safe for now, though this is dependent on the type of role and notably the creativity of a role,” believes Marway. “From a testing point of view, machines and AI will compliment creativity and not eradicate the need for skilled creative test professionals. The testing world is an exclusive arena where people are willing to learn new skills to stay relevant; therefore, my advice would be to keep your skills up-to-date,” he added. AI algorithms can be immensely helpful in the testing industry in making smarter and more productive software for the end-user. It is, however, important to decipher how to use AI intelligently. “So far, virtually all machine learning and AI algorithms are only as good as the humans who design, train and improve them,” commented Ilia Kolochenko, CEO at High-Tech Bridge “The developments of AI technologies usually require expensive long term investments that people typically cannot afford. Therefore, I don’t see substantial risks or revolutions that may happen in the digital space because of AI – not within the next five years at least.” It appears that, as long as testers are willing to make the investment in remaining relevant and “ahead of the game” by recieving technological support and using useful tools to help support a project, long live software testing!

WHEN IT COMES TO SOFTWARE TESTING...

...DO YOU HAVE AN EYE FOR DETAIL? To support our continued expansion and client wins, ROQ is expanding our test team and is actively looking for experienced Test Consultants across all levels. If you are looking for your next career move, relish new challenges and are eager to collaborate and develop alongside like-minded test professionals, ROQ wants to hear from you. Permanent and Contract opportunities available across the North West and London. Ready to take your next step? Please get in touch with Katie Franklin on 01257 208890 or email careers@roq.co.uk.

www.roq.co.uk

MAKING T E SOFTWARE S T M a g a z i n e | MWORK a r c h 2 01 8

TCOE’S EXISTENCE IN TODAY’S DEVOPS WORLD DevOps is not a methodology; it is a notion to help cut down the barriers between Dev and Ops, says Project Lead QA at Value Labs, Narayana Maruvada

T E S T M a g a z i n e | M a r c h 2 01 8

D E V O P S

C O N T I N U O U S

t is always an acceptable and/or argumentative fact that the perceptions towards ascertaining the QA/testing needs (encompassing techniques, tools, practices, and strategies etc.) has underwent many ramifications in the recent past, and more predominantly when organisations started to incline more towards adopting today's agile and DevOps mode of project delivery, which totally changed the very approach for availing the QA and testing services. One major and notable change was organisations interest in promoting, establishing and investing in TCoE (dedicated centralised testing teams). Although, it is very evident that the agile and DevOps approach was instrumental in expediting the time to market situations, meeting the shorter or frequent release cycles and facilitating continuous delivery. But if organisations are equally looking out to establish a highly standardised QA and testing practice (either for a specific or a complex business need) and aims to build a centralised repository of reusable test assets to deliver quality at optimal cost through optimum utilisation of resources, then TCoE is still the ‘go-to’ solution.

DEVOPS – THE KNOWN AND UNKNOWN It is very well understood that DevOps is not a methodology, instead it is a notion to help cut down the barriers between Dev and Ops, and purely intended to meet and expedite the needs pertaining to shorter and more frequent release cycles to promote the ‘agile’ mode of delivery with ease. The said notion is achieved through seamless integration and effective collaboration between the assorted teams viz. ‘Dev’ and ‘Ops’ teams, and together, they function as a single entity to accomplish the following activities as a continuous process viz. • • • •

Continuous testing Continuous integration Continuous delivery Continuous monitoring

Although application developers and system engineers bring correct things into the right environments, the actual crux lies with Q&A teams since they need to continuously stay focused to align their test deliverables, and ensure that every minute and/or potential code changes work as intended without actually breaking anything with the application and/or product, regardless of frequent build requests being encouraged. The reason as to why Q&A teams are considered to be crucial in the DevOps ecosystem is for the fact that they meticulously take care of following key activities continually and quickly

D E L I V E R Y

in the same sprint (or on demand basis) for the application under test viz. a) b) c) d) e) f)

Identify the critical test scenarios Develop and automate test cases Organise test cases into respective test suites Outline the execution order of test cases Schedule the execution as part of C.I Generate and/or share the test execution results to requisite stakeholders

DevOps is not a methodology, instead it is a notion to help cut down the barriers between Dev and Ops

From aforementioned, it is understood that the test approach automation adopted by the Q&A team appears to be very systematic with a cohesive test design, which together helps cut down the challenges with test planning and test management. However, the said approach with Q&A in DevOps is definitely not going to be straightforward and result oriented if the applications/products are of intricate nature. For example, applications and products are designed and developed for BFIS, utility and healthcare (US demographic) and domains are considered to be most complex. Especially with healthcare domain (US demographics) – it is very exhaustive as it encompasses the most critical pieces of business functions pertaining to • Payroll / invoice Generation • Benefits enrollment and disenrollment and their administration • Payments and remittances • Processing of claims Besides, EDI (stands for Electronic Data Interchange) forms the major crux of the healthcare domain (specifically for US demographics, as mandated by HIPPA) since it forms the basis and/or enables the transaction processing through the exchange of sensitive data in an acceptable standardised file format between different heterogeneous systems. Again, it’s not just one file format, but there are several file formats available today each of which is intended to standardise the transactions pertaining to a specific business function. So, to build such a typical domain centric application or product it not only involves the simulation of bundle of critical functionalities using contemporary technology stack, it also requires extensive domain knowledge to validate. Especially from a QA standpoint, a robust team encompassing of an exploratory, functional, automation and security testers are very much needed in order to assess the said applications/products thoroughly. However, availability and/or building such QA teams on need basis are one of the major gaps with DevOps.

NARAYANA MARUVADA PROJECT LEAD QA VALUE LABS

Narayana Maruvada is a Computer Science and Engineering Graduate who has more than 11 years’ of experience working in the QA vertical. His major area of work and expertise is validating applications and products that are built in open source technology stacks for different business domains.

T E S T M a g a z i n e | M a r c h 2 01 8

One of the major objectives with DevOps is the early detection of defects in the development cycle

D E V O P S

C O N T I N U O U S

CONTEMPORARY CHALLENGES FACED BY Q&A TEAM’S IN DEVOPS Following are some of the key challenges that QA teams tend to experience as part of working in the DevOps ecosystem: EFFECTIVE COLLABORATION: Apparently, it’s the QA team, which acts as a crucial interface between the product development team and operations. So, it’s imperative that QA teams should be made part to witness the entire project/ product life cycle and its associated discussions, rather than having them involved during the testing phase alone. But this collaboration is of very less possibility looking at how DevOps operates, and hence there will be potential gaps to both understand and align with the testing needs and expectations. TEST ENABLEMENT: It is very critical that QA teams understand the business (and underlying critical functionalities) for which the application/ product is built and verified. So, in order to achieve this QA team should team-up and has to work closely with business experts / SME’s to understand how the system supports the business, based on which QA team can ascertain the testing needs and enable the QA services accordingly. But in DevOps, it is seemingly hard to find and/or facilitate this channel of association. TEST COVERAGE: In DevOps, one cannot guarantee 100% test coverage of the application/ product for two fundamental reasons – the rush to deliver the software quickly to meet the release cycles and secondly there are high chances of overlooking from validating critical functionalities due to dynamically changing requirements, which can attribute to defect leakage. FACILITATING EARLY TESTING: One of the major objectives with DevOps is the early detection of defects in the development cycle, and this is possible only when testing is planned to begin during the early stages. But it is more likely achieved when there is requisite test

D E L I V E R Y

documentation (and other supporting test templates) in place to outline, organize and prioritize the user-stories that can be readily tested without any dependencies. TESTING MATURITY: Regardless of any and every approach with software delivery, the keydifferentiating factor from a QA standpoint is the test maturity. Generally, it is one such attribute that cannot be quantified all the time, but it is derived taking certain key factors into consideration such as approach, experience, skill-set, ability to orchestrate and automate etc. So, if the Q.A teams fail to possess the requisite test maturity, then it is inevitable to see challenges and failures while taking projects through completion.

CENTRALISED TESTING SERVICE (AKA TCOE) - STILL THE RELIABLE AND ‘GO-TO’ APPROACH The idea behind having TCoE is to establish a highly standardised QA and testing practice besides building reusable test assets and repositories that can deliver quality at optimal cost by optimum utilisation of resources. It helps bring people, process and technology together, and likewise promote the requisite collaboration across teams to improve the effectiveness of testing. It is a comprehensive structure in itself and is driven by independent and self-sustaining CoE’s outlined below, backed by a strong core team (for governance, facilitation and coordination services), which together act as key accelerators and differentiators to meet any challenging/ complex and/or evolving business needs (from QA standpoint) regardless of time to market situations or delivery patterns. • • • •

Functional test competency team Domain competency team Automation competency team Non-functional test (performance, security etc.) team

The proposition is not all about building big centralised testing teams, or TCoE’s, but to ascertain the possibilities of integrating or embedding already existing centralised testing services into agile or DevOps and maintaining them, to maximie ROI and deliver high customer satisfaction and service standards.

T E S T M a g a z i n e | M a r c h 2 01 8

STAYING ON TOP OF THE COMPETITION Test Magazine Journalist, Leah Alger, interviewed the Head of Quality Assurance at Third Bridge, Niranjalee Rajaratne, and the Head of Testing at Bank of America Merill Lynch, Sudeep Chatterjee, to find out their experiences in the world of testing and how to stay on top of the “competition”

T E S T M a g a z i n e | M a r c h 2 01 8

D E V O P S

C O N T I N U O U S

TELL ME ABOUT YOURSELF AND YOUR JOB ROLE: SUDEEP CHATTERJEE: I am a Senior Technology Manager who has more than 19 years’ experience with top-tier investment banks, fintech and consulting firms, managing testing globally for enterprise-wide change programmes. I am currently consulting as the Head of Testing at Bank of America Merrill Lynch within FICC - Global FX Technology group. Prior to the Bank of America Merrill Lynch, I worked as the Head of Testing with Lombard Risk, Barclays, UBS, GE and Accenture, while primarily focusing on building high performing multi-disciplinary testing teams and delivering testing for complex technology-driven business transformation initiatives. NIRANJALEE RAJARATNE: I am an IT professional with over 12 years’ experience who has worked in various verticals such as telecom, financial services, publishing and e-commerce. Currently I am the Head of Quality Assurance at Third Bridge. Third Bridge is a leading independent financial research company that provides private equity firms, hedge funds and strategy consultants with the information they need to make an informed decision about investment opportunities.

WHAT IS THE MOST COMPLEX SYSTEM ARCHITECTURE YOU HAVE MANAGED? SUDEEP CHATTERJEE: I have managed testing on many complex system architectures starting from PoS systems for one of the UK’s largest retail company, complex integrated voice recognition systems for one of UK’s largest telecommunication company, complex banking systems built over quants and big data solutions, as well recently worked on the high frequency low latency electronic trading platform for FX.

NIRANJALEE RAJARATNE: The most complex systems architectures that I have managed testing on have been in the financial services and telecommunication sectors. Due to the scale of these projects, and the demand for new features and changes, the development was happening on unstable grounds often. The systems were built and managed in an ad-hoc manner. It was challenging to manage testing to achieve quality and take proactive measurements to risk management. ARE YOUR CHANGE MANAGEMENT PROCESSES MODERNISED AND AGILE

D E L I V E R Y

FRIENDLY? SUDEEP CHATTERJEE: Change management process is agile and focuses on ensuring SDLC is built over robust processes, which allows for faster delivery with high quality. NIRANJALEE RAJARATNE: Yes, up to some extent. We keep evolving. We try to iterate the change management process so that it enables us to achieve so the business is faster, better and within budget. As an outcome, we try to do early engagements with business units, improve communication to keep it consistent and get the buy-in from senior management as early as possible. The team is given autonomy and time to try new things and adapt to change as new and brilliant ideas come to light.

Adopting a modern digital transformation strategy is part of the technology roadmap, which includes bestin-class delivery for mobile platforms and cloud solutions for end users

HOW DO YOU STAY AHEAD OF THE COMPETITION WHEN IT COMES TO MOBILE SYSTEMS, CLOUD COMPUTING AND APIS? SUDEEP CHATTERJEE: Adopting a modern digital transformation strategy is part of the technology roadmap, which includes best-inclass delivery for mobile platforms and cloud solutions for end users. NIRANJALEE RAJARATNE: Research and assessments helps us to continuously refine and adopt the right level of technology to achieve business objectives. We assess our current workflows and software to identify gaps in efficiency that technology can bridge.

CAN YOU GIVE ME AN EXAMPLE OF A CI/CD FAILURE YOU’VE EXPERIENCED, AND WHAT YOU LEARNED FROM IT?

NIRANJALEE RAJARATNE HEAD OF QUALITY ASSURANCE THIRD BRIDGE

Niranjalee has spent large parts of her career focusing on testing and software quality assurance. Currently, she is responsible for implementing delivery assurance across software development. With the focus on software quality, test automation and DevOps, she is passionate about developing proactive approaches to quality management, continuous integration and delivery.

SUDEEP CHATTERJEE: CI/CD has gone wrong mostly in my experience, especially when dev teams are used to implementing plans and runbooks for deployments in non-production and production environments. When teams start using CI/CD tools without proper training there are times when an issue comes up, particularly around configuration setups. NIRANJALEE RAJARATNE: I can give many, such as the wrong selection of CI servers, inefficient build infrastructure, uncorded and unstructured pipelines, incompetent skill set and lack of management support. It resulted in delivering business value to the end user albeit on broken builds. CDs cannot be performed and organisations cannot be productive with an inconsistent CI. To make CI/CD work, it was important that the business and technology

SUDEEP CHATTERJEE HEAD OF TESTING BANK OF AMERICA MERRILL LYNCH

Sudeep Chatterjee is a Senior Technology Leader with 19 years’ experience in top tier Investment banks, fintech and consulting firms, managing testing globally for enterprise-wide change programmes.

T E S T M a g a z i n e | M a r c h 2 01 8

D E V O P S

Implementing DevOps without organisational culture change can cause conflicts between teams. For successful DevOps teams, it is important the change management process complements the ethos

C O N T I N U O U S

management understood the value of an efficient build pipeline, and supports it without seeing the effort as pure technical work.

learn to work with DevOps tools including the continuous integration and continuous delivery de-facto standard.

HOW HAS DEVOPS CHANGED YOUR WAY OF WORKING?

NIRANJALEE RAJARATNE: It will further support the team’s autonomy to be selforganised by removing dependencies, further breaking silos. It will continue to improve inter-team collaboration, empowering people to learn new skills and share knowledge. It will bring users' and technology teams closer to achieve a symbiotic relationship. If iteratively improved, this will enable organisations to take a paradigm shift in how they deliver software.

SUDEEP CHATTERJEE: DevOps has helped organisations to reduce the time to deliver software to production with continuous testing embedded in the process. It has also strengthened testing frameworks like behaviour driven development. NIRANJALEE RAJARATNE: DevOps improved infrastructure and created efficient build pipelines to help get new products and services to the users quickly. It created a shift in testing and quality assurance by enabling automation of repetitive tasks such as regression and smoke testing. The development teams were empowered to follow test first development practices such as TDD and BDD, helping them towards an efficient build pipeline, and to receive fast feedback.

WHAT DEVOPS TOOLS DO YOU USE? SUDEEP CHATTERJEE: Jenkins, Git, BitBucket, Maven and Ansible. NIRANJALEE RAJARATNE: Github, Ansible, Docker, Jenkins, Elasticsearch, Monit.

HAVE YOU FACED ANY BAD EXPERIENCES WHEN IMPLEMENTING DEVOPS? SUDEEP CHATTERJEE: Implementing DevOps without organisational culture change can cause conflicts between teams. For successful DevOps teams, it is important the change management process complements the ethos, and all team members from BA, developers, architect, environment management, QA and application support must understand their role in the DevOps world. NIRANJALEE RAJARATNE: We certainly did have challenges and we still have some to this day, but I would not say they were bad experiences. Instead they were learning opportunities for the team to perfect the delivery pipeline.

HOW WILL DEVOPS CONTINUE TO IMPROVE? SUDEEP CHATTERJEE: DevOps will continue to improve and will be ‘the norm’ for all software delivery. Dev and QA teams will

T E S T M a g a z i n e | M a r c h 2 01 8

D E L I V E R Y

DO YOU BELIEVE MANUAL TESTING WILL COME TO A CLOSE BECAUSE OF AUTOMATION AND DEVOPS? SUDEEP CHATTERJEE: There will always be a requirement for manual exploratory testing by domain experts though this may not just be a QA professional but can be anyone in the team with strong domain knowledge like BA, product owner, or users. NIRANJALEE RAJARATNE: The purpose of manual testing and the value it brings cannot be fully replaced by automated testing. Whilst automation testing is an integral part of DevOps, manual testing should continue to function in order to help achieve optimum levels of quality in software.

ANYTHING ELSE YOU WOULD LIKE TO ADD? SUDEEP CHATTERJEE: One of the challenges in the agile and DevOps world will be how organisations measure product quality and compensate its employees for performance management. You must make sure the HR process matures to be to able to provide compensation to the entire team, or you will have to still rely on individual feedbacks and do performance benchmarking between team members. NIRANJALEE RAJARATNE: If you look at the trend towards DevOps, it is quite clear that organisations wants to adopt a model that encourages a structure which removes obstacles in order to get high-quality software to the customers better and faster. I believe DevOps cannot be perfect without a strong testing process in place. In DevOps agile, testers should play a major role in taking the ownership of continuous improvement and quality tracking. To do this, it’s essential that they continue to develop their skills and knowledge.

STAY ON TOP OF THE COMPETITION WITH

registrations are now open KEY DATES 14 September – Submission Deadline 4 October – Finalists announced 21 November – Awards Gala Dinner

Now in its sixth year The European Software Testing Awards celebrate companies and individuals who have accomplished significant achievements in the software testing and quality assurance market. Enter The European Software Testing Awards and start on a journey of anticipation and excitement leading up to the awards night – it could be you and your team collecting one of the highly coveted awards.

Be a winner - REGISTER TODAY

 www.softwaretestingawards.com T E S T M a g a z i n e | M a r c h 2 01 8

ALLOWING CHANGE FROM THE GROUND UP By having responsibilites in place from start to finish, agile decreases time to market, as well as helps deliver the right product/features to its customers, according to Ranrorex's CCO

T E S T M a g a z i n e | M a r c h 2 01 8

A G I L E

M E T H O D O L O G I E S

TELL ME ABOUT YOURSELF AND YOUR JOURNEY WITH RANOREX: I have been with the firm since the very beginning. Prior to working at Ranorex, I did software testing and software development on the team which was led by the future founder of Ranorex, Jenö Herget. This was 15 years ago. Back then Mr. Herget was in charge of the company-wide continuous integration/ build system. He was also in charge of selecting an appropriate test automation tool. Mercury/HP was already under evaluation, but it was not affordable – so he decided to develop a free test automation library testers and developers could use to create automated tests. Before then, I was just a Junior Developer and did not feel confident enough handing over my code to the testing department. Using this free test automation library helped me ensure that my components had the expected level of quality. That was my first experience with automated testing. I realised pretty quickly that the earlier you have automation in place, the faster you are at delivering software. Back in 2005, the free Ranorex automation library became available online, and many testers and developers started using it. Mr. Herget then decided to make a business out of it. He asked me whether I’d like to join the team in September 2007 as a sales engineer, helping customers understanding the concept and idea behind Ranorex. As is common in start-ups, everyone is doing a little of everything, but I always wanted to stay involved with customers. So, I wrote documentation, developed the first Ranorex training concept, helped marketing with positioning our tools, and was also responsible for demonstrating and selling our solution. Today, I’m responsible for the organisation at our European Headquarters of over 80 employees, with a strong focus on the sales department. HOW DOES RANOREX ENSURE TEST AUTOMATION ANYWHERE, AT ANY TIME? Ranorex supports building automated tests for multiple platforms including desktop, web and mobile environments. That’s easy for test automation vendors to claim, of course, but much more challenging in practice. But our focus has

always been to support our customers with their toughest automation challenges, whether they are testing legacy systems or apps built with the latest technologies. Many of our customers have to automate testing for a wide variety of technologies. From a tester’s perspective, you might not have a clue what type of UI technology is used in the background. That’s why we include support for all technologies in Ranorex Studio – testers have a single, familiar interface regardless of the development technology used for the application under test. The automation of regression tests is usually the first step for our customers. With Ranorex, these tests can easily be triggered by any continuous integration/build system overnight. In addition, Ranorex supports executing tests in cloud environments. This is especially needed for customers testing their web applications on multiple browsers in parallel. Ranorex supports any type of WebDriver-based automation infrastructure like SauceLabs, BrowserStack or SeleniumBox.

The automation of regression tests is usually the first step for our customers. With Ranorex, these tests can easily be integrated by any continuous integration/build system over night

HOW DOES THE FIRM DRIVE INTEREST IN AGILE? Increasing agility is very hard to achieve without automation. The more agility you want the more automation is needed. This is not only related to test automation. From a developer’s perspective, nothing else drives you crazier than waiting feedback about your latest commit/check-in to the code base on the server. Unit testing meets the need for faster feedback and is increasingly recognised as a QA standard in every agile development team. Unit tests by themselves would never be enough to ensure adequate review of the application. In order to make sure that component A also works in combination with component B, tests on a system level are needed. Having at least a minimum set of smoke tests configured to execute overnight and maybe during lunch would be a good combination. On the other hand, there are organisations that are still in the process of transforming from waterfall to agile; especially in the mid to enterprisesized companies. Ranorex helps these customers to incrementally achieve a higher agility level with automated testing of legacy applications at the UI

CHRISTOPH PRESCHERN GENERAL MANAGER AND COO RANOREX

Christoph Preschern is the General Manager and Chief Commercial Officer at Ranorex, and has been with the company since its founding in 2007.

T E S T M a g a z i n e | M a r c h 2 01 8

Agile is a goal, not a destination. You can never say that you’ve "arrived" because there is always opportunity for improvement

A G I L E

M E T H O D O L O G I E S

level. For these customers, unit testing does not work because the system under test was not designed for that. Or even worse, the knowledge about the code base is no longer in-house, perhaps because developers have left the company, or components have been bought instead of being developed inhouse. WHAT STRATEGIC OUTCOMES DOES THE FIRM WANT FROM IMPLEMENTING AGILE? Ranorex is a unique company in many ways, but when it comes to implementing agile, our goals are really the same as most organisations. We want to decrease time to market, deliver the right product/ features to our customers, get faster feedback from our customers and deliver better quality by having responsibilities right at the start of development. I suppose that one of the things which makes us unique is that we are our OWN customer – we use our test automation solution to test our own product. HOW WILL YOU KNOW WHEN YOU'VE ACCOMPLISHED AN AGILE PROCESS? Agile is, of course, a goal, not a destination. You can never say

that you’ve “arrived,” because there is always an opportunity for improvement. But some of the most significant markers of what we could call a “mature” agile process are a development team that is empowered to make decisions about how to implement a certain product requirement on their own, and an organization that is confident enough to present the outcome/product of every sprint in front of the customer. WHAT EVIDENCE DO YOU HAVE THAT AGILE HELPS RANOREX BE MORE EFFICIENT? Our agile processes enable us to consistently deliver our releases on time and at a more frequent interval than in the past. But most importantly, we have improved quality of these releases as measured by the number of issues reported after release. HOW DOES AGILE TRANSFORMATION TIE IN WITH YOUR OWN OBJECTIVES OR STRATEGIC DIRECTION? One of the key benefits of an agile approach is reduced management overhead. Our teams are able to spend more time discussing how a feature should look instead of trying to plan a large project. Of course, our internal test automation and continuous integration processes allows our QA personnel to conduct less manual testing and focus more on ensuring the quality of our product. HOW WILL YOU HANDLE THE TYPICAL ORGANISATIONAL CHANGES NEEDED TO MAKE A TRANSFORMATION SUCCESSFUL, SUCH AS RESTRUCTURING CENTRALISED FUNCTIONS (E.G., GOVERNANCE, TESTING, RELEASE, AND DEPLOYMENT) THAT NEED TO BECOME SERVICES? We worked closely with an expert to help us manage the transition to agile processes. If you don’t have the expertise in-

T E S T M a g a z i n e | M a r c h 2 01 8

A G I L E

M E T H O D O L O G I E S

house, then I highly recommend getting assistance from outside. You may want an external consultant since it can be challenging for team members to have the necessary perspective to restructure existing roles and procedures. It’s also useful to specify certain roles during the transition. For instance, you might need more scrum master skills at the beginning while your teams are adjusting to the new processes. Once those processes are running smoothly, continuing them day-to-day might not require the same intensity. Get buy-in from all parties that change is necessary. Allow change to happen from the ground up, and avoid forcing change from above. Let internal and external people demonstrate the advantages. Once adopted by few, the agile virus spreads quickly. When you hit stumbling blocks, such as people defending out-of-date approaches with “we’ve always done it like that,” encourage them to reflect on the reasons those approaches developed, and how they could be improved. Persuade them to try out alternatives. Provide plenty of time to experiment. Most importantly, try to tear down the mental barriers in your own head (“what if that goes wrong?”) before tearing down another’s.

have goals that contribute to the team’s goals. Don’t just evaluate how much code someone produces, but how they help to let everyone else produce great code. HOW WILL YOU COMMUNICATE YOUR GOALS, APPROACH, AND EXPECTED OUTCOMES FOR TRANSFORMATION? Well, of course, it is important that the organisation’s leadership has a true open-door and open-ears policy. In addition to presenting the goals and outcomes for improvement, leaders should always be role models for the change they expect to see in their teams.

If a team does hit a stumbling block, simply re-explain the goal and hold a retrospective on the improvements already achieved

HOW WILL YOU ENSURE DIRECT TWOWAY COMMUNICATION TO FOSTER THE IDENTIFICATION AND RESOLUTION OF

HOW WILL YOU ENSURE THAT THE LAYERS OF MANAGEMENT FROM THE TEAM TO THE TOP ARE PREPARED TO DEVELOP AND DEMONSTRATE AGILE BEHAVIORS? Occasionally, management should participate in reviews and retrospectives. This would not be from a stakeholder perspective but as an observer; to see how agile is working in a team. Remove all but the critical management positions and make people part of the teams as leaders. Team leads should ensure that team can work without disruptions. Make it clear that management is not there to tell the people what to do, but to ensure that people work most efficiently together. HOW WILL YOU ENSURE THAT YOUR PERFORMANCE SYSTEM IS ADJUSTED TO ACCOUNT FOR TEAM VERSUS INDIVIDUAL ACCOMPLISHMENT SUCH AS SHARED GOALS? Each individual team member needs to

IMPLEMENTATION CHALLENGES? The key to communication is to put team members as close to each other as possible – ideally in the same workspace. Formal reporting lines, processes or directions on how to communicate should not be the focus. HOW WILL YOU SUSTAIN MOTIVATION AND FOCUS ON THE WHOLE TRANSFORMATION LIFECYCLE? In general, motivation is not a problem when team members have sufficient responsibility and authority to do their job. If a team does hit a stumbling block, simply re-explain the goal and hold a retrospective on the improvements already achieved.

T E S T M a g a z i n e | M a r c h 2 01 8

THE GOVERNMENT ADOPTS AGILE Scrum Master at HM Land Registry, Nelson Ajoku, reveals how the Non-Ministerial Department of the Government masters agility

T E S T M a g a z i n e | M a r c h 2 01 8

A G I L E

M E T H O D O L O G I E S

crum Master at HM Land Registry, Nelson Ajoku, reveals how the Non-Ministerial Department of the Government masters agility. Working with the WebOps team at HM Land Registry (HMLR), Ajoku handles people management by making sure sprints go to plan. He has two daily stand-ups – one for the release team, which supports app development teams, and one with the service improvement team, which looks to constantly improve DevOps practices. According to Ajoku, the main challenges he faces are anticipating the nature of BAU tasks by viewing the product backlogs of development teams, while completing backlog items that the team has committed to for a sprint. This challenge determined the approach towards grooming its product backlogs; goals are set for 12 weeks, with the commitment to improve the WebOps service. In his sprint planning meetings, HMLR pulls work from the team's product backlogs, while bearing in mind that regular urgent BAU tasks will be coming from app development teams during the process. The sprint backlog is a list of tasks identified by the scrum team to be completed during a sprint. Several product backlog items, which are usually in the form of epics, are then broken down into user stories. Estimation and planning poker sessions then take place, while teams discuss how much effort each user story requires. “Strategy implementations depend on how the results are being received. We view activities on the infrastructure side of things, and then flag whatever it is that needs to be improved,” revealed Ajoku. In scrum, new features are delivered in sprints, so the value of a product is being realised from an early stage. According to Ajoku, scrum promotes continuous improvement of a product, because as long as a product exists, so does its backlog.

THE BENEFITS OF SCRUM Other benefits of scrum include: • More transparency and project visibility through daily stand-up meetings in order to eliminate

misunderstandings and confusion • Increased team accountability to improve collaboration, empowering each team member to be independent • Accommodates change with short sprints and constant feedback • Increased cost savings to help lower expenses and increase quality.

HMLR’s sprint team

Agile is open to changing requirements over time and encourages constant feedback from end users. Using agile in scrum means that, instead of finished, detailed descriptions of how the project has been completed, it is left to the scrum software development team. According to Ajoku, the difference between agile and traditional project management is that agile is very transparent and communication is exaggerated frequently, to ensure the tester is testing and delivering correctly. As for with the traditional management, such as waterfall, a lot of people are blinded about if they are still on track or not. "Obviously, in agile, you have ceremonies and ideologies that help you achieve your goals, such as sprint planning, reviews and retrospectives etc, which can be a combat card for agile," added Ajoku. Ajoku believes agile has taken over the waterfall approach: "With the waterfall approach you need to record everything, test everything, quality assure everything, and then deploy everything, so you have a very long feedback loop. As for when using agile you must start with the most valid aspect of the product, record it, deploy it, receive feedback, and keep going back; releasing and fixing issues/ bugs sooner, before severe damage is done,” he added.

necessary tasks to

selects a number of product backlog items, usually in the form of user stories, while identifying complete each user story

NELSON AJOKU SCRUM MASTER HM LAND REGISTRY

THE AGILE MANIFESTO Ajoku's key agile principles include: • Satisfying customers through early, continuous delivery of valuable software • Welcoming changing requirements, even if they’re late in development • Delivering safe, working software frequently • Working with developers throughout the project, as well as business people daily • Building projects around motivated individuals by giving them support • Discussing issues and ideas face-to-

Nelson kicked off his IT professional career as a Java Software Developer Trainee at FDM Group PLC, before moving on to HSBC Sheffield (Griffin House) as an Analyst Developer supporting the rollout of technologies used for the Customer Due Diligence Programme. He is now currently a Scrum Master at HM Land Registry Plymouth, supporting DevOps/WebOps of new digital services technologies.

T E S T M a g a z i n e | M a r c h 2 01 8

When using agile you have to start with the most valid aspect of the product, record it, deploy it, receive feedback, and keep going back

A G I L E

M E T H O D O L O G I E S

face • Showing continuous attention to technical excellence • Reflecting on how to become more effective. There is an array of project management frameworks, from agile (scrum, kanban etc), to v-model, to waterfall. They all have different pros and cons, so it can be tricky for software testing teams to decide what ones to choose from based on the nature of their projects, according to Ajoku.

TOP ADVANTAGES Here are Ajoku's top agile advantages: • It's simple to accept and accomodate changes during a project • There's opportunity to refine and reprioritise the backlog, allowing teams to introduce changes in a matter of weeks • It's beneficial for projects where the end-goal is not defined clearly • Goals come to light and developments can adapt to evolving requirements • Conducting tests during iteration

T E S T M a g a z i n e | M a r c h 2 01 8

• •

•

means bugs can be solved and identified more quickly High-quality software can be delivered more quickly Agile shows strong team interaction by highlighting the importance of frequent interactions Customers have the chance to see the work being delivered, allowing them to share their input, further impacting on the final product Encourages feedback from team members and users' throughout the whole of a project Lessons are learned and used to improve future iterations.

As an ex-HSBC employee, Ajoku noted that HMLR works differently, because the bank includes DevOps SMEs in application development teams, ensuring they handle any DevOps testing, while HMLR has a separate WebOps team that supports all application development teams. HSBC also has a lot of dependent teams where some still practice the waterfall approach, making it difficult to align teams’ roadmaps, while HMLR application teams are more standalone in an agile perspective.

T E S T M a g a z i n e | M a r c h 2 01 8

MOVING AWAY FROM TRADITIONAL WATERFALL METHODS DevOps Manager at KPMG, Adnan Rashid, explains how the firm has transformed away from traditional waterfall models to a collaborative, customer-focused DevOps approach, while encompassing agile methodologies

T E S T M a g a z i n e | M a r c h 2 01 8

A G I L E

M E T H O D O L O G I E S

PMG is a global network of independent member firms in 154 countries, which is most commonly known for providing audit, tax and advisory services. In recent years, many IT departments have found themselves under increased pressure to keep up-to-date with the rapid pace of innovation and change within technology along with increased demand from customers for new features and services. In order to address these challenges, KPMG in the UK has been going through a steady transformation by moving away from traditional waterfall models to a collaborative, customerfocused DevOps approach which encompasses agile methodologies and cloud technologies such as AWS, Azure and GCP. Due to the nature of the services KPMG provides, there are considerable regulatory and governance considerations to comply with in order to protect our clientâ&#x20AC;&#x2122;s data and ourselves. With this in mind, in 2015 KPMG launched a pilot project with the vision to deliver services efficiently, reliably and securely to our customers. We started off by creating a small businesscentric team to deliver an application in the cloud with the vision to become the benchmark of future projects and evidence to the business that it is possible to do DevOps in the enterprise whilst adopting the necessary security and governance requirements. The application was to be hosted in AWS and utilised infrastructure as code, allowing deployments into multiple environments in a consistent, repeatable manner. It also allowed deployments without having to be concerned around configuration drifts and with the added benefit of rapid deployment speeds.

THE INFRASTRUCTURE STACK We also began to slowly develop a pipeline which would allow us to release software code from development through to production consistently and also allow controls and checks in between. The infrastructure stack consisted of Atlassian Stash, now known as Bitbucket, which is our secure, and private source code repository, TeamCity for testing and building the application package and finally to be handed to Octopus Deploy which manages the deployments into a number of environments. The project was an immediate success, as the application team now had the ability to quickly accommodate customer requests, and ensure the stability of the environment without having to rebuild any component manually. It is a common misconception that large enterprises cannot adopt DevOps and Agile practices, as it is often believed it is suited to smaller start-ups due to the number of processes within a large organisation. We had started our automation journey, however, we now needed

to address stringent security requirements, service management considerations and ensuring the ongoing maintenance of the application. Instead of attempting to change an entire department process and method of working, we started having discussions and listening to the various concerns whilst keeping at the forefront Cloud best practices such as utilising disposable resources, infrastructure as code, automation, loosely coupled components and asynchronous integration. Each team had built up their own set of considerations and best practices, therefore when attempting to adopt a new method of working and deliver value, it was extremely important to try to find a middle ground and compromise. Once there was a clear understanding of the various controls, due to our agile way of working, it was easy for us to change the architecture design whilst minimising both downtime and development impact. We ran two-week sprints and began to work through the deliverables in order to present back to the relevant teams and put in place the necessary documentation.

Due to the nature of the services KPMG provides, there are considerable regulatory and governance considerations to comply with in order to protect our clientâ&#x20AC;&#x2122;s data and ourselves

UTILISING SERVICES As the business saw how quickly we were able to deliver value, the word spread around the organisation and the team found themselves going from a single project almost 100 within the space of a year. In order to accommodate for such growth, the team rapidly increased the number of engineers and also began working more closely with service management continuously improving to ensure incidents, changes and problems were handled efficiently and in a timely manner. We saw improvements in change management as we developed all infrastructure as code, it was easy for us to track, review and manage changes. At a high level, everything was committed into source control, reviewed by senior engineers and evaluated by the product owner to validate there were no business risks. Today we have matured our offering in a variety of ways from our pipeline by utilising services like Jenkins to handle orchestration, Terraform for infrastructure, Vault for security, Trend Micro for anti-malware and anti-virus along with our cloud transformation professional services helping some of the biggest companies in the world adopt DevOps and move towards working in a more agile manner to ultimately improve customer experience. Adopting DevOps and cloud initiatives involves changes from all parts of the organisation and at all levels including culture, process and technology. Organisational change is a journey which takes time, understanding and collaboration to succeed.

ADNAN RASHID DEVOPS MANAGER KPMG

Adnan is a Cloud Architect with a passion for building secure and reliable solutions. He has experience building architectures on premise and in the cloud, helping some of the biggest companies in the world transform their IT departments.

T E S T M a g a z i n e | M a r c h 2 01 8

P R E V I E W

he National Software Testing Conference will take place on the 22-23 May 2018, Millennium Gloucester Hotel, London, for the fifth year running. Taking place over two packed days, the National Software Testing Conference is ideal for all professionals aligned with software testing, and is a fantastic opportunity for testers to network, learn, share advice, and keep up-to-date with the latest industry trends. Providing the software testing community with invaluable content, this conference will consist of presentations from revered industry speakers, including the Winners of The European Software Testing Awards; roundtable discussion forums that are facilitated and led by key figures; as well as a market leading exhibition, which will enable delegates to view the latest products and services available to them. The National Software Testing Conference is open to all, but is aimed and produced for those professionals that recognise the crucial importance of software testing within the software development lifecycle. Therefore the content is geared towards C level IT executives, QA Directors, Heads of Testing, Test Managers, Senior Engineers and Test Professionals. Last year, we was lucky enough to have professionals from the likes of DVT, Hotels.com, AstraZeneca and Sony Interactive Entertainment Worldwide Studios host key notes at the conference; inspiring presentations from Banco Santander, ThirdBridge, Gamesys, Infostretch, Tricentis, Tandem Bank, University of Derby, BBC, RBI Technology, 10 Digital Media Limited and Credit Suisse; as well as useful executive workshops from Testbirds and DVT.

T E S T M a g a z i n e | M a r c h 2 01 8

TWO FULL DAYS OF PRESENTATIONS FROM SENIOR SPEAKERS The National Software Testing Conference speaker line-up includes: • Adrian Brown, Head of Fusion QA, TP ICAP • Rajdeep Varma, Senior Automation Engineer, Badoo • Felicity Lord, Quality Assurance Manager, Wyndham Vacation Rentals UK • Steve Watson, Former Director Quality Engineering, Reed Business Information • Nicky Watson, IT Director, Credit Suisse • Natalie Wood, Director of Quality Engineering, Elsevier • Niranjalee Rajaratne, Head of Quality Assurance, Third Bridge • Suresh Chandrasekeran, Senior Test Delivery Manager, Credit Suisse (More speakers to be announced)

BOOK YOUR PLACE To book your place today please contact us:  www.softwaretestingconference.com  info@softwaretestingconference.com  +44 (0)203 931 5827

22-23 May 2018 Millennium Gloucester Hotel London

EXHIBITORS

T E S T M a g a z i n e | M a r c h 2 01 8

BE INSPIRED

on 22-23 May Millennium Gloucester Hotel London

• Meet the experts • See the future of testing • Collaborate with industry peers

Register today The National Software Testing Conference is a UK‑based conference that provides the software testing community at home and abroad with invaluable content from revered industry speakers; practical presentations from the winners of The European Software Testing Awards; roundtable discussion forums that are facilitated and led by key figures; as well as a market leading exhibition, which will enable delegates to view the latest products and services available to them.

End User Conference Registration 2 Days*

£735.00

Day 1 Only*

£525.00

Day 2 Only*

£525.00

* All registrations include entry to the evening drinks reception on the first day of the conference. All prices are subject to VAT.

 softwaretestingconference.com  0203 931 5827  info@softwaretestingconference.com

T E S T M a g a z i n e | M a r c h 2 01 8