TEST Magazine September 2021 by Tami.Citroen

Volume 13 Issue 2 https://www.softwaretestingnews.co.uk/products/test-magazine/digital-issues/

Automation in Motion Manual and Automated Testing Breaking the Myths on QA Automation The Role of AI and ML in Automation The Future of Test Automation

Sponsored by De

Now in its eighth year the European Software Testing Awards celebrates companies and individuals who have accomplished significant achievements in the software testing and quality assurance market. The awards ceremony will be held at the Old Billingsgate, London in November 2021 Enter the European Software Testing Awards and start on a journey of anticipation and excitement leading up to the awards night – it could be you and your team collecting one of the highly coveted awards!

ENTRIES NOW OPEN ATTEND THE PROGRAMME >> SoftwareTestingAwards.co.uk

September 2021 CONTENTS

testmagazine.co.uk

Sept 2021

What's inside

11 The role of AI and ML in Automation

Automation in Notion: by Matthew Clarke

14 Interview: Venkataraman Purushothaman

18 Demystifying QA Automation: by Priya Ghandi

Manual vs. Automated Testing: by Nuria Manuel

The Future of Test Automation

Interview: Wojciech Olchawa

Test Automation in Agile Teams: by Gayathry Adhavan 3

12th & 13th October 2021 at The British Museum, London

September 2021 EDITOR'S LETTER

testmagazine.co.uk

Editor's Note Volume 13 Issue 2 Welcome to the 2nd issue of Test Magazine 2021! We are delighted to present to you the next issue of Test Magazine and wanted to thank you for your endless support after the re-launch in July! It means everything to us, and you are the reason we are continuing to do what we do so we can provide you with the best content in Software Testing and DevOps. This month’s issue is all about Automation! In the wake of the pandemic, we have witnessed automation slowly becoming the norm and, for certain industries, taking over completely. According to several studies, the UK factory automation market is expected to grow significantly by 2026. This doesn’t come as a surprise as automation is said to bring more productivity and quality to the work, processes, and tools. We are then thrilled to make this rising topic the heart of our second issue of Test Magazine!

From learning about Automation in Performance Testing to Breaking the myths around QA Automation, Test Magazine takes you on a formidable journey within the software testing world. Throughout the issue, you will have the opportunity to discover some of the best processes, advice, and great management content from professionals in the industry. Moreover, Test Magazine features a variety of Automation leaders, managers, and architects from a wide range of organisations. They have shared their knowledge and expertise with us so that it can inspire and enrich a new generation of testers, developers, and IT experts. We are then thrilled to be able to share this issue with you and celebrate one of the most interesting and riveting topics in the software testing world. We hope you will enjoy this new publication of Test Magazine! Happy reading!

Constance Drugeot Editor

© 2021 31 Media Limited. All rights reserved. TEST Magazine is edited, designed, and published by 31 Media Limited. No part of TEST Magazine may be reproduced, transmitted, stored electronically, distributed, or copied, in whole or part without the prior written consent of the publisher. A reprint service is available. Opinions expressed in this journal do not necessarily reflect those of the editor of TEST Magazine or its publisher, 31 Media Limited. ISSN 2040‑01‑60

EDITOR Constance Drugeot constance.drugeot@31media.co.uk +44 (0)203 931 5826 ADVERTISING ENQUIRIES Moeez Ali moeez.ali@31media.co.uk +44 (0)203 668 6947 DESIGN & PRODUCTION Tamara Citroen tamara.citroen@31media.co.uk

PUBLISHED BY 31 Media Limited Fleet House 8-12 New Bridge Street London, EC4V 6AL +44 (0)203 931 5827 hello@31media.co.uk testingmagazine.com PRINTED BY MIXAM, 6 Hercules Way, Watford, Hertfordshire, WD25 7GS, UK

 softwaretestingnews @testmagazine  TEST Magazine Group

September 2021 PROCESS & DEVELOPMENT

testmagazine.co.uk

Automation in motion - UK Census Utilising plurality of vision when implementing vastly divergent system coverage, allows consistent assurance across a service regardless of the funding of individual projects

Introduction Today I’d like to welcome you into the world of enterprise automation for large scale government programmes, or at least this engineers’ implementation of it. Something of an idiosyncratic introduction, as I have the pleasure of introducing the Office for National Statistics (ONS) heuristic assurance framework and service to the light of the world for the first time. At the heart of current events, we have the UK Census, recently concluded, stretching the length and breadth of British society. At the time, one of the largest public-sector digital programmes. We will focus on a small aspect of that programme, the implementation and execution of Assurance Automation and the process automation that facilities its operation. Having over 72,000 possible input parameters creating an almost endless number of unique possible journeys, almost all of which are important to understanding the demographics and distribution of people across the nation. For now, I’ll leave you guessing, as to how exactly.

Matthew Clarke Currently the Lead Assurance Engineer for the UK Census Programme at the ONS but moving on to become the Head of Engineering for Eurofins Digital Testing at 25 years old, Matthew has a deep love for engineering and assurance.

September 2021 PROCESS & DEVELOPMENT

testmagazine.co.uk

Methodology Rather than adopting the ONS existing hybrid-waterfall model or the hybridagile model that had been utilised for the Census & Data Collection programme. The Assurance engineering capability chose the informal ARPA Project management approach. This allowed the rapid materialisation of enterprise assurance ecosystems with extremely limited resources and a highly truncated time scale in which to implement any assurance framework. Utilising plurality of vision when implementing vastly divergent system coverage, allows consistent assurance across a service regardless of the funding of individual projects. A blend of long-standing theoretical principles coupled with contemporary technologies. It may seem anachronistic or unviable to use abstracted and dense mathematical formulas or computer science principles when engineering a ‘real’ solution. Utilising plurality of vision when implementing vastly divergent system coverage, allows consistent assurance across a service regardless of the funding of individual projects.

Test Coverage & Efficacy Principles & Theory Test coverage for the heterogeneous services and components of the Census and Data Collection programme is vast, integrated and divergent. Creating a comprehensive model of coverage that is both effective and expedient requires moving away from basic testing techniques and utilising more sophisticated methods. The Engineering team is a small group with a large test as a result efficient method are also required in contrast to methods that a simple effective as there is a limited time frame in which it executes a complex criterion. Algorithms have been written and deploy to determine optimal rapid test coverage. You may be familiar with pairwise testing or otherwise known as 2-way testing. This is a basic but moderately effective method of software testing and depending on the software captures up to 90% of defects. But it can be as low as 60%. This is insufficient. Conventional orthodox has event lower defect capture rates, around 40-60% on average. As such we utilised combinatorial

“Creating a comprehensive model of coverage that is both effective and expedient requires moving away from basic testing techniques and utilising more sophisticated methods” testing, this is the more developed implementation of pairwise. Utilising 3-way, 4-way and greater. This requires the use of sophisticated and discretionary algorithms. However, a 4-way test is sufficient in scope to capture all possible conditional values for most systems and may even be redundant. The detection rate of defects increases exponentially with the strength of interactions. In other words, as we increase t-way interactions we significantly increase our defect identification. Below is a binominal coefficient, this coefficient is utilised within a script to determine the smallest viable number of test cases required for 100% condition coverage of all services within the programme. It’s worth understanding conditional coverage juxtaposed against decision coverage. 100% decision coverage would be exponential in scale and create coverage for outcomes that wouldn’t occur in the wild, so to speak. Conditional coverage however will cover almost all practical eventualities. Creating a comprehensive model of coverage that is both effective and expedient requires moving away from basic testing techniques and utilising more sophisticated methods.

Maths, run for your lives! Utilising formulas like these allow the effortless implementation of combinatorial combination:

Equation 1 - Binomial Function

Rather than leave you hanging, assuming as many mathematicians do that you will simply understanding this barrage of nonsense, I will briefly break it down. Brace yourself! Let’s take a simple abstract system, then let’s assume we have a website that hosts

a survey of 30 pages, and each of those pages ask us a simple question, where we can respond with; Yes, No Hopefully, your imagination can picture that nicely. That gives us 230 possible inputs, or in other words, 1,073,741,824 combinations. This is obviously a ludicrous number and is a perfect example of why conventional orthodoxy warns against exhaustive testing. However, all 3-way combinations can be covered 29 tests,

Equation 2 - Binomial Function Example

This means for every 3-way test we cover some unique 4,060 conditions and across all 29 tests every possible point of conditional interaction will be covered. There is a general rule of thumb for working out the number of tests you need, most combinatorial functions with Python, Java etc. will provide you with an output array of the number of tests needed. But if you’d like to do it yourself vt log n where v is the number of values (i.e., On, Off) and t is the number of ways 2-ways, 3-ways and n being the number of parameters. It is possible to test every component interacting with every other component at least once. Practical considerations like redundant pairs, combinations that must happen in sequence such as OS>Browser>Region can’t create a pair that revises that order thus creating many additional coverage permutations is avoided. The ultimate output is an orthogonal coverage array and matrix that can be used to create the configuration for either test script or reports. Fault Localisation: An effort had been made to develop fault localisation, were to rapidly increase the root cause analysis of a given defect we look at t-way

September 2021 PROCESS & DEVELOPMENT combinations that trigger a failure, fault or error, this is easily done at an abstracted level, utilising test execution matrix’s for Automation test cases that are executed in CICD pipeline, all test case are executed with a matrix of inputs at a high-level it is a scenario/workflow, i.e. A specific user journey, with a set of input parameters;

testmagazine.co.uk

for example, I may need to test a regionalspecific function for one product but not for another, therefore I would create a condition that prevents a t-way interaction forming.

Figure 2 - N-Way Combinations

Figure 1 - Fault Localisation

This diagram and set shows how failing combinations are identified by excluding passing combinations within the various subsets of executions, P combinations in passing test, F combinations in failing tests, C fault triggering combinations, F\P combinations in failing tests that are not in any passing tests, must contain the faulttriggering combinations since if they were in C the passing test would fail, which is how we arrive at

Next, we have a basic example of an orthogonal array, where we can see how input combinations interact and how a smaller number of tests can achieve full coverage. For example, when we execute the test that have the A+C pair we don’t then need to execute a C+A pair later as the coverage has already been achieved. This continues until our last row consists of only one pair. This type of matrix works perfectly up to 6-way interactions.

We host all of our services internally within our Google Cloud Platform solution, rather than having a centrally managed DevOps function, assurance operates independently of the mainline operation. This allows greater flexibility, control and deviations. Not to mention more independent assurance.

C F \ P this is the basics in addition to this core exclusion we also use: u Eliminations and exclusions u Interaction’s continuity u Redundancy u Value dependence u Out of scope FRs, NFRs For example, we can immediately identify a localised fault in a combination of configuration inputs a basic example of this in Java and Python showing parametrizable config, when a failure occurs a failure is written at multiple levels throughout the framework and reports back to a matrix of results, the speed at which a root cause failure is identified is instant with the human resource being the limiting factor. Here I’ve provided some high-level diagrams to explain what’s actually happening without all that unhelpful maths getting in the way. We have a set number of inputs and a set number of script configurations, some of them will interact and some of them won’t,

“Due to some of the extremes required in full-spectrum assurance, we do have some custom solutions for our network address translation that allow us to operate truly vast number of concurrent connections across multiple networks and regions”

Figure 3 - Orthogonal array

Automation Framework – Practical Automation exists for us within two vertical domains, service automation; Test Suites and process automation; Reporting, Releases Management, post and pre-processing. Our automation of both domains exists within a larger heuristic framework that exists across a wide range of responsories, platforms and services, but all integrate and flow into a cohesive assurance capability. As is typical with counterparty implementations of assurance, all the familiar staples are present from Terraform, Helm, Kubernetes – to Message Brokers, Virtual Private Network and all flavours of the Cloud.

Due to some of the extremes required in full-spectrum assurance, we do have some custom solutions for our network address translation that allow us to operate truly vast number of concurrent connections across multiple networks and regions. For the Census, a peak concurrency rate of over 250,000,000 requests per hour with a mean response time of 7ms. However, maintaining that sort of scale requires a degree of sophistication. We have a very positive working relationship with our Cloud Provider Google that allowed us to push the platform in certain areas. More focused on automation specifically, Gradle is used as the dependency management tool, and as general automation task manager for both Automation, Security and Unit testing in multiple languages. We use Gradle to create consistent structure and ephemeral outputs, we always want our builds and results to be

September 2021 PROCESS & DEVELOPMENT

testmagazine.co.uk

index position of the value located with a 2-dimensional array:

produced to temporary directories that exist no longer than our Kubernetes deployments and can be easily processed by are reporting and file management automation. Due to some of the extremes required in full-spectrum assurance, we do have some custom solutions for our network address translation that allow us to operate truly vast number of concurrent connections across multiple networks and regions.

Central-Test-Framework – Programme u System Automation – Project u Website Automation u Welsh Translation – Work package u Performance u Security & Compliance u Smoke & Unit u Configuration The Framework is built to integrating into the service horizontals and verticals of the business. As well as the Functional and Non-functional requirements. u Programme u Project u Work package This allows the seamless reporting and coordination of the technical into the business and vice versa. Rather than utilising Gradle’s built in Test function, we extend its task function and create targeted suites, within nested projects. Gradle wraps around the framework then each A RegEx expression is used to identify the runner file for each sub-project, this allows much more relative pathing, and allows the use of N+1 runners, I.E you can use any number of runners within the same directory if your so wish.

Figure 5 - Gradle Tasks

We utilised multiple cross-device and browser tools, from our internally managed Seleniod instances to externally managed BrowserStack. Rather than having to rotate through various remote drivers and configuration packages, we simple extended an additional Gradle task. This significantly increases framework stability and maintenance, a core principle of our assurance capability. By creating clear, extensible code. Allowed the framework to mature rapidly while maintaining internal consistency. This also feeds in nicely to the creation of pipeline jobs and creating a considerable diversity within regression suites. Within our Kubernetes deployments, the appropriate tag is passed. We currently execute against 10 services every 2 hours across for 100% conditional coverage.

Summary A fully independent assurance platform, with dedicated services hosted on identity-aware proxies across the internet and integrated into solutions across each programme’s estate. Backed up by deep engineering and a robust foundation. Allow large scale, Service and Process automation to be carried out to their maximum potential, some highlights: u Operationally effective within 2 months u Fully operational within 10 months

Implementing COMBINATORIAL

u 26,000+ End to End Automation Tests executed in 10 months

For demonstration purposes I have used a basic slimmed down version of our input parameters, as a public service organisation, we are required to meet certain digital accessibility standards, least of which is browser accessibility. We must assure that both the latest and previous versions of a wide number of browsers across devices function as required.

u 500,000 effective operational hours

The below list doesn’t even begin to starch that, but shows a basic set of parameters that you might use:

Figure 4 - Runner ReGex

Here we have tasks that define variations on a suit, this is the website sub-project the collection of landing pages a user can access before authentication, it consists of around 10 proto suites that collectively make up one full suite.

Any number of input parameters and conditions can be applied, when operating at scale using database, or files are far more practical to lift and manage the array of input parameters that might exist for a given programme or project. As briefly mentioned, this produces an output, a 2d array. This array can really be quite large. That basic set of inputs creates 18,144 combinations, if we didn’t apply our efficient little coefficient, but with it the number of test cases drops to around 42.

u 0 Production incidents or Defect for areas assured u 3 minor workplaces accidents (Stubbed toes & bruised egos) Finally, you may ask why you’ve never seen assurance like this? Simple truth, is that it has a high barrier to entry and the number of resources who are full stack engineers, mathematicians & working in assurance is regrettably quite low. Assurance engineering when done right is an advance engineering discipline.

As discussed above, we want to eliminate redundant or invalid pairs. To do this we create conditional relationships between values. The integer row is an

The Performance engineering for Census was the real star of our framework and I hope one day I get to share more of what that service achieved.

These values come from the National Institute of Standards & Technology

The framework is deceptively simple:

July 2021 testmagazine.co.uk

5th - 6th October 2021 at The British Museum, London

DevOps is a culture, movement or practice that emphasises the collaboration and communication of both software developers and other IT professionals, while automating the process of software delivery and infrastructure changes. It aims to establish a culture and environment where building, testing, and releasing software can happen rapidly, frequently, and more reliably. Implementing the culture and practices of DevOps can be easier in small, lean organisations where engineers, developers, and IT operations all operate from the same office. However, it is likely to be a more difficult and complicated transition in large, established organisations with complex legacy systems. Here, organisations can be plagued by siloed business functions, with little or no communication across different silos. If an organisation wants to change critical IT infrastructure, achieving compliance and ‘buy-in’ from all departments can prove challenging. With DevOps it can be different – when development and operations collaborate and work together towards the same goal, the results can be extraordinary. Organisations regularly report increased deployment speed and frequency, higher quality product output and lower failure rates.

devopsonline.co.uk/national-devops-conference

September 2021 PROCESS & DEVELOPMENT

testmagazine.co.uk

The Role of AI and ML in Automation AI/ML will pace up the development cycle with processes like automated code review, automated code generation, automated software development planning, etc.

While Artificial Intelligence (AI) and Machine Learning (ML) are rising topics in the software testing world, it is becoming more widely associated with Automation. By having automated tests, businesses have been able to improve the testing process as well as achieve better quality and productivity. Using AI and machine learning in software testing allows the delivery of better and more efficient automation, which enables testing teams to use their time more effectively. Hence, there is a clear trend towards AI-powered test automation. We then tried to understand what role do AI and ML play in automation and to answer this, we have asked experts in the industry to share their insights.

Sourik Mukherjee

Sourik Mukherjee is a passionate Engineer with around 10 years of experience in Machine Learning, Computer Vision, Data Analytics, Development and Automation Engineering space in service and R&D areas of Banking and Automative domain. Sourik is a avid reader and he always loves to enrich himself with latest technologies and ideas across different fields.

Hesham Allam

Hesham Allam is an expert in technology filled with +10 years of experience and currently holding a position of AI & Automation lead in one of a leading international company to achieve digitalization and transformation objectives through Automation using RPA techniques.

Artificial Intelligence & Machine Learning in Automation According to Hesham Allam, Automation is not new terminology, but it is generally considered an innovative way of working to reduce the number of repetitive activities. In the same way, while

September 2021 PROCESS & DEVELOPMENT

“This enables leading companies to trust and count on intelligent automation to increase its value and gain a much higher makert share” automation required automation, a new innovative idea emerged, which is now known as machine learning (ML). Artificial Intelligence (AI) is the way to achieve the best solution. So, both ML and AI combined have introduced intelligent automation, which can be considered an industry revolution. Regarding automation in the software testing area, AI/ML are thus able to improve the capability of testing different types of applications, removing the limitations of current automation tools and frameworks, Sourik Mukherjee continues. Indeed, similarly to the UI level automation, machine learning will help in finding the elements on the pages accurately with no dependencies on development. Regarding automation in other software areas, AI/ML will pace up the

testmagazine.co.uk

development cycle with processes like automated code review, automated code generation, automated software development planning, etc. AI/ML will pace up the development cycle with processes like automated code review, automated code generation, automated software development planning, etc.

AI & ML role ML and AI play the most critical role in the Automation process, Hesham underlines. By taking accurate and fast decisions and transforming complicated situations into easier ones, AI and ML bring more confidence into the complete automation process. This enables leading companies to trust and count on intelligent automation to increase its value and gain a much higher market share. Sourik notes that AI/ML is more of an advanced version of traditional automation. So, with the release of different open-source accelerators and tools in AI/ML, automation as a whole in the software industry will be more efficient and cost-effective. Hence, AI/ML will bring different aspects to automation which are all rule-defined until now. Therefore, using AI and ML within

automation provides higher production rates, increased productivity, more efficient use of materials, better product quality, improved safety, shorter workweeks for labour, and reduced factory lead times. This enables leading companies to trust and count on intelligent automation to increase its value and gain a much higher market share.

Benefits & Challenges According to Sourik, the main benefit is that AI and ML will enhance the undermined automation scopes in the software industry in every area of a software development lifecycle. Indeed, due to the availability of machine learning models of different domains in open-source areas, automating different applications of different domains will be more effective and will increase the quality. For Hesham, AI & ML are bringing many benefits for Automation, such as: u High efficiency u Accuracy u Increase productivity u Availing more opportunities for creativity and innovation Automating different applications of different domains will be more effective and will increase the quality. There are, however, still challenges to overcome before adopting AI and ML safely within automation. The main challenge, Hesham notes, that intelligent automation is currently facing is humans themselves. For instance, you might hear that intelligent automation will lead to an increased unemployment rate! Such a statement can be clearly seen during the planning and implementation phase of the transformation process to intelligent automation. The second challenge, which is again related to the human aspect, is the lack of good resources and researchers in the field. Even though AI & ML is getting much more attention, there is still a vital need for more experts and tools. Moreover, Sourik notes that defining machine learning engineers' role will become a challenge as well as finding dataset availability and allocating a budget for doing research and innovation in ML areas, specifically in automation.

September 2021 testmagazine.co.uk

PROCESS & DEVELOPMENT

There is less clarity at the management level on this area as the mindset is tilted towards the traditional concept of delivery and, according to him, automation is still wrongly being used in the industry as of now. Even though AI & ML is getting much more attention, there is still a vital need for more experts and tools.

The future of AI and ML in Automation Hesham believes that AI and ML will continue to grow. Indeed, he points out that this is not just a trendy technology, but we are in the era of intelligent automation, which invades almost everything in our modern lifestyle. AI & ML both are playing an important role to make human life easier, and, in the near future, both core technologies will help to offer the best choices for each individual by availing dozens of options to customize what you need from what is available. On the other hand, Sourik doesn’t think that there will be any specific automation role in the future as AI/ML will take over that area and people involved in AI/ML will then be utilized in building accelerators for those specific automation tasks. Yet, he still believes that with a good fundamental idea of traditional automation and good programming knowledge, we can have a smooth transition to ML. There is a high possibility that five years from now or even less, traditional automation will rise again to automate the process of AI/ML application development. Besides, along with the onset of AI/ML, different aspects will be coming up in security, ethical AI areas, which will then need automation to tame those threats. Sourik also thinks that the use of automation in AI/ML applications of core engineering areas will become a big trend in the coming years as well. Thus, he expects automation to grow hugely but in a refined way and, with the right mindset and continuous study, it will yield fantastic results in the future for people pursuing in this field. There is a high possibility that five years from now or even less, traditional automation will rise again to automate the process of AI/ML application.

September 2021 INTERVIEWS

Venkataraman Purushothaman Interview Test Magazine Venkataraman Purushothaman is an Automation Architect with eleven years of experience Venkat specialises in automation frameworks and is responsible for developing / enhancements frameworks based on client requirements. Venkat is a powerful force in the workplace and uses his positive attitude and tireless energy to encourage others to work hard and succeed. In his free time, likes to listen to music, build Lego’s and play games with his daughter.

What is your current role and responsibilities? I’m currently an Automation Architect at Centriva Hive in London with more exposure towards Microservices, Mobile Applications, Cloud Orchestration, Robotic Process Automation and Chatbots. Most of my work involves interaction with various clients across domains for implementing proprietary tools/solution accelerators and also design, development or customization of test framekworks.

What inspired you to get involved in the Testing industry? In software testing every feature and every project provides different problems, there is never a straightforward, “one size fits all” testing solution. Some of the most fun we have in software testing is in finding the best possible testing strategy, tool and execution. Software testing is about looking at the product from different angles, with different perspectives and testing it with different expectations. It is not always easy to develop the right mind-set and test the product from different aspects. This challenging atmosphere inspired me to pursue my testing career. In software testing every feature and every project provides different problems, there is never a straightforward, “one size fits all” testing solution.

Why did you choose to work in automation? Many types of tests exist to help an organization to assess and improve the quality of its product. Automation testing is the most important type of test to maintain the level of quality for your product. By automating your testing process, your organization will not only save time and money but also deliver higher quality products. Manual testing is an essential part of the testing process. Especially, functional exploratory testing needs to remain manual. However, more

testmagazine.co.uk

September 2021 INTERVIEWS

testmagazine.co.uk

and more, the part of automation will grow. The regression testing will need to be automated, as well as performance, load tests, and so on. As the automation tools evolve, more and more tests will be able to be automated. So, the demand for good automation testers will continue to grow. People working in test automation required to constantly improve, to keep up with the evolving technology. Software testers are always interested to learn and solve interesting challenges, so all these motivates me to choose automation from SDLC. Automation testing is the most important type of test to maintain the level of quality for your product.

Can you tell me about your journey and how you got where you are now? Coming from an Electronics background and chosen IT as a career. Later, I have done my master’s degree in software development & Management to stay relevant to the field. Started with Cognizant as In-House / Proprietary automation solutions developer and later did consulting, proof of concepts and RFPs for multiple clients across continents. Currently working on implementation and enhancement of test frameworks based on business requirements / latest technology trends.

Who do you look up to for inspiration or mentorship? People in the current organization: Seniors in the current organization could be your first go-to persons especially because they will be available to help you and most importantly, you have access to them. Moreover, they might have already faced the challenge you have today. Interacting with people higher in designation gives you a bigger picture and helps you to change your perspective.

Follow Key people: There are many software testing leaders worldwide like James Bach, Dave Haeffner, Michael Bolton etc. They share the latest updates and provide thoughts on testing trends across social media, blog posts. Also, podcasts like Test Talks, Ministry of Testing etc. to stay tuned. Books: Most influential medium to gain knowledge about software testing. Books like Agile Testing, Lessons learned in Software Testing, Agile Testing Techniques

etc. provide you insights from the experience of the authors. How do you keep your team motivated despite conflicts and obstacles? In order to achieve great results, you should be constantly thinking about ways you can motivate and inspire your team. Below are few ways to motivate the team in the right direction and achieve their goals: 1. Share your vision and set clear goals You can only motivate and inspire your team if they know what they are working towards. Make sure your team members are aware of your vision and what your ultimate goals are for the business. Set clear and measurable goals that are framed by this vision so that you and your teams can track progress and they are able to see their success in a tangible way. Supporting alignment within

In order to achieve great results, you should be constantly thinking about ways you can motivate and inspire your team and across teams cultivates increased productivity and can help employees to feel valued and motivated. 2. Encourage teamwork Encouraging and promoting teamwork boosts productivity because it makes team members feel less isolated and helps them to keep more engaged with their tasks. You can do this by regularly holding teambuilding exercises and opportunities for your team members to bond and get to know one another. 3. A healthy office environment Our environment has a significant impact on our productivity, contentment and creativity. Healthy and happy team members are more likely to feel motivated and engaged. Create a space that is enjoyable to work in and an office where your team members want to spend their time. Make sure to run regular Pulse Surveys for remote employees to find out what they are struggling with so you can support them better and help them create a positive and productive work environment, regardless of where they're located. 4. G ive positive feedback and reward your team

The power of positive praise is sometimes overlooked but recognizing and applauding achievement inspires team members as they can see themselves progressing towards the goals of the company. When team members achieve results, put in extra effort or do outstanding work make sure to tell them that you’re grateful and be specific in your praise. 5. Provide opportunities for development Team members feel more valuable when they are learning and enhancing their skills. To motivate and inspire your team to achieve great results you should provide your team members with opportunities for growth and development. In order to achieve great results, you should be constantly thinking about ways you can motivate and inspire your team

What are your current goals? Below are the niche areas I’m trying to explore during this year as part of my goal:

Learning Blockchain Techniques Blockchain technology challenges companies with the highest adoption costs, privacy and compliance issues, and legacy system integration inconsistencies, to name a few. The complicated transaction procedure includes encryption, validation, transmission, decryption, and a solo hitch that can cause the system to discontinue working. Thus, comprehensive tests become business-critical.

Explore Codeless Automated Testing The higher adoption of code-less testing tools will be the main software testing trend in 2021. The 2021 State of Test Automation Report explains Script-less testing is top of mind in 2021. Over 40% of respondents have an interest in using script-less/intelligent test automation solutions this year.

AI & ML Test frameworks The rising demand for Artificial Intelligence continues to grow due to the growing number of apps we use in our linked world. Teams are investing in AI and ML-based testing tools and practices, over 60% of respondents are looking to invest in AI & ML in codeless testing.

What are you the proudest of in your career so far? I have managed to implement the E2E automated flow start from when the developer checks in the code till it goes

September 2021 INTERVIEWS

testmagazine.co.uk

sometimes more, for weeks at a time. This is why, as a tester, it’s important to know how to keep yourself interested in the task at hand and to find enjoyment in the work. Doing so will allow you to specialize in certain aspects of the job and, more than anything, stay motivated. That being said, if you are able to overcome the challenges, the rest of the job is much easier!

Do you have a memorable story or an anecdote from your experience you’d like to tell? I was mapped for a project as an interim candidate to perform automation tasks. The initial task is all about creating some acceptance tests and integrating them as part of build pipelines. After few weeks, the existing team member is not able to return, and they asked me to continue till the end of the project since its business-critical.

into the production layer. Basically, when developer changes are made build will be triggered and automation scripts will run. Once they are successful it will move to the next stage looking at Change Request details or else its rollback to the previous version of the code. If the Change Request is approved, then it will deploy to specific servers and notify the business users about the feature that went into production through email. This implementation was well received by the customer and they started rolling out across the organization for low impact areas.

What has been your greatest challenge? As development cycles have become quicker and shorter, the definition of software quality has been evolving. Comprehending and testing products, while considering factors like functionality, performance, security and many others, is a multi-faceted and complex task. Furthermore, the intricacies due to mobile applications have exponentially complicated the testing of software. One of a software tester’s biggest challenges resides in understanding the vast range of devices available and checking the application’s behaviour in terms of response time and usability.

What is the favourite part of your job? 16

Software testing experience challenges every day, we have to stretch your mind to understand a product’s functionality and general behaviour, to improve analysis power, to learn new tools and to implement the learning in real life. What is even more critical than understanding is innovation, the generation of new ideas, and being able to comprehend the big picture. Software testing experience challenges every day, we have to stretch your mind to understand a product’s functionality and general behaviour, to improve analysis power, to learn new tools and to implement the learning in real life.

What have you learned from your experience so far? The biggest challenge QA testers face is the repetitive nature of the work. Being a professional tester is like running a marathon: you have to test the same application for 8 hours a day and

When I looked into Test strategy there are several phases of testing that need to be done before shipping this code to production. After automation, I had to create performance tests and ran with different loads across multiple servers. Next step I did Operational Acceptance testing along with operations team support. Later, we promoted this to the staging layer to perform accreditation testing and get certified by various payment providers. Finally, it made into production and able to achieve the peak TPS during the Black Friday load without any outrages. Something started as few weeks of work and ended up into seven months of steep learning curve covering niche areas. Always be prepared to explore unknown test territories with the necessary support and guidance.

Finally, do you have any advice for aspiring testers who want to grow in the tech industry? 1. Have communication with your team. This first point is essential to always keep

“Software testing experience challenges every day, we have to stretch your mind to understand a product’s functionality and general behaviour, to improve analysis power, to learn new tools and to implement the learning in real life”

testmagazine.co.uk

in mind; having great relations and getting along with PM’s, designers and especially with developers is very useful because they can help you a lot when you have questions or doubts, which is something that makes things way easier as a tester. All the members are your allies, in the end, you are part of a team searching for the same goal: your project's success. 2. Put yourself in the user position. It’s very easy to get carried away with the everyday workload of a project but it’s vital to always take into account the experience the user will have. As a tester, you need to become the user when testing the system as a whole, which means that you should try to recreate all the flows the user can think of, not only the happy paths. 3. Prioritize your work. Imagine you are assigned to a very big project, the development of it is still going, you (the QA tester) still have a lot to test in the system, the project has a very tight budget which means no other QA resources will be assigned and your deadline is coming up fast. This leaves you with only one option: distinguish which things are a priority and test these first to make sure the most critical user stories work fine. After that, you can pass to the non-critical user stories if you still have time. 4. Be creative and develop your attention to detail. The software can never be 100% bug-free, according to the seven general principles of testing; therefore, you always need to be on the lookout for new ways to test your system. If you’re not finding issues on your site anymore, it might be time to think even more outside of the box and try to find the most unusual scenarios the user could follow in order to continue identifying issues. 5. Document. This point is super important especially when you’re involved in a very complex and large project. Creating test cases will give you the opportunity of track your test coverage and they also can be the head start for test automation. The documentation can be very useful for the team and for the testers that will come to the project after you and trust me, they will definitely thank you for it. 6. Bonus! Have at least one mentor. This is one of the most valuable lessons I’ve learned during my time here. I'm very glad and lucky to have a supportive QA team because they are always willing to help you, answer your questions and clear any doubts that you might have. My advice is to try to surround yourself with people who know more things than you and whom you can learn from, you'll grow and your mentors as well. My advice is to try to surround yourself with people who know more things than you and whom you can learn from, you'll grow and your mentors as well.

September 2021 PEOPLE & LEADERSHIP

testmagazine.co.uk

Demystifying QA Automation Both manual and automation testing has their own value and having both these types of testing are mandatory to bring the desired software quality Myth 1: QA, QC, and Testing are the same. Fact: Quality Assurance, Quality Control, and Testing are always interchangeably used, and even though they are related, they are different. u QA, QC, and Testing have the same goal i.e., to ensure the development and delivery of a high-quality product for the customers. u Quality Assurance is a much wider topic than Quality Control and Testing because it covers the holistic quality of the product from how the product is being developed i.e., the input through the end product i.e., to the outputs of software. u QA includes activities that ensure the verification of developed software with respect to the defined requirements. This includes mapping out the testing activities against the requirement matrix and mainly focuses on processes and procedures rather than conducting actual testing on the system. QC, a subset of QA, focuses on actual testing by executing the software with an aim to identify bugs/defects through the implementation of procedures and processes. Testing, the subset of Quality Control focuses on actual hands-on testing.

Priya Gandhi Priya is the Global Director of Automation and QA Engineering at Hogarth Worldwide, a WPP subsidiary company. Holding degrees in Engineering and IT Marketing, she uses her knowledge and expertise in Automation and Emerging Tech to inspire her team and improve their digital processes.

September 2021 PEOPLE & LEADERSHIP

testmagazine.co.uk

Myth 2: Automation is a replacement for manual testing. Fact: Both manual and automation testing has their own value and having both these types of testing are mandatory to bring the desired software quality. u Any tests that are repetitive, mundane, and are required to run several times in the same or different environment, then tests should be automated. u Manual testing brings a more human approach to QA by focusing on user empathy and testing the application like how a customer would perceive it. This cannot and should never be replaced by automation. u Both manual and automated testing should be strategically structured in a way that complements each other. Both manual and automation testing has their own value and having both these types of testing are mandatory to bring the desired software quality.

Myth 3: Automation will give 100% test coverage. Fact: It is impossible to automate every test and it isn’t an efficient testing solution. u There are many test scenarios

“One of the main benefits of automated testing is that it frees up valuable time for the development team who can focus on other critical features” not worth automating owing to the software and system dependencies, the complexity of the system, flakiness of the tests, dynamic contents of the application, differing display on different screen sizes, etc. and these scenarios are easier to be tested manually. u Knowing how to prioritise tests and when to automate is equally important as knowing how to automate. u Combining manual with automated testing offers high-standard software quality and will increase the number of tests you’re able to run.

Myth 4: A universal automation tool is possible. Fact: It is not possible to develop one automation tool that addresses all of the Quality Assurance team’s requirements. u Having a realistic expectation of the automation tool, knowing the clarity on the problems that need to be addressed,

analysing the cost vs value of the tool, and agreeing on the end goal to measure automation effectiveness can help in choosing the right automation tool(s). u The best approach is to build a flexible automation framework by integrating different test scripts, tools, and frameworks together. The tools can include automated visual testing, content testing, load testing, API testing, functional testing of backend and front-end tests, etc., and can be a combination of both in-built and licensed tools. u Maintaining the tests to be run periodically based on the different phases of the development process or chosen development methodology will set the automation testing for success.

Myth 5: Automated Testing is an easy fix. Fact: Creating automated testing isn’t a breeze. u An efficient automation framework requires dedicated resources to maintain and keep it up to date. u Writing test scripts, scenarios, and acceptance criteria to work with the deployment team is equally important to eliminating flaky tests, building trusted test scripts, providing maintenance to keep the tests up to date, and providing support for test failures. u It’s not easy to keep scaling up the resources to expand on the automation scripts as this tends to divert the team’s focus from more strategic tasks.

Myth 6: Automation Testing Comes with a Heavy Price tag. Fact: This may be true for some small-sized companies but, the value the automation testing adds makes it all worth it. u One of the main benefits of automated testing is that it frees up

September 2021 PEOPLE & LEADERSHIP

valuable time for the development team who can focus on other critical features like security, customer satisfaction, and core functional features. u Automation testing handles the repetition of tests with ease that is time-consuming and expensive. Automated tests are more reliable for repetitive tests as there is no testing fatigue. u The initial investment in automated testing will be higher but the Return on investment (ROI) is better in the long run compared to Manual testing.

testmagazine.co.uk

“AI can also add intelligence to the automated test execution to make them more stable, be agile, to self-heal by detecting and fixing threats even before they occur as the system is continuously learning using the data”

u One of the main benefits of automated testing is that it frees up valuable time for the development team who can focus on other critical features.

Myth 7: Automation Testing is QA’s job. Fact: Quality is not solely the responsibility of the QA team and should be everyone’s responsibility. So, automation testing should be the team’s responsibility too. u Developers need to be accountable for the quality of code they write. Unit tests make sure the software programming is fundamentally working and these tests are especially critical in CI (continuous integration) where the automated tests make sure the builds are stable and bug-free. So white-box unit testing automation should always be the responsibility of the developer. u QA team has the mindset, training, and specialized skills for testing. So black-box functional tests like end-to-end test automation, content, visual tests should be the responsibility

of the QA/ testing team. u CI/CD is dependent on automated tests and is critical to deploy reliable and robust builds. So, integration, Performance, Release, and Post-Live testing should be the combined responsibility of the development, testing, and the dev-ops team.

Myth 8: Introducing Automation Testing into a manual team isn’t easy. Fact: On-boarding a manual development team into automation testing just requires strategic planning. u Knowing how you currently test to having a clear end goal on automation, to deciding which test cases to automate, identifying tests that are resistant to changes in the UI, and having clarity on the automation mindset is the key to transform the manual testing team into automation.

September 2021 testmagazine.co.uk

PEOPLE & LEADERSHIP

u Then selecting the right automated testing tool, dividing your automated testing efforts between the development, testing, and DevOps team, and selecting the suitable automation development language for different automation testing is crucial. u You don’t always have to develop the automation testing tool. Partnering with an automation vendor(s) who have vast amounts of experience tackling problems similar to yours, will work to your best advantage and achieve the automation transformation easily and efficiently.

Myth 9: Scriptless automation will completely eliminate coding test scripts. Fact: Scriptless automation only simplifies the process. u The Scriptless automation framework is set up in a way that the testers are only required to indicate the steps instead of writing an actual code after which the framework will translate those steps into test cases. So having the test scripts is crucial. u Scriptless test automation optimises the test process validation and execution, so the end-to-end quality assurance approach will be more robust, scalable, reusable, and easy to maintain. u As the coding aspect is minimum, business stakeholders can also participate in the QA process involve in the business logic validation process better and can also engage more in reviewing to enhance the quality of the end product.

Myth 10: AI will make Automation Testing Irrelevant. Fact: If the testing practices are structured well to facilitate intelligent testing, then AI can help with automated testing to its fullest potential. u Test Automation has always decreased the extent of manual testing activities and AI can augment with human intelligence to help automate and speed up activities that weren't possible only with test automation. u As mentioned in Myth 1, Quality Assurance is not just about testing and AI can help automate all if not most of the quality assurance process. Generating test cases, engage in data, estimating execution based on the brief to achieving test coverage by extracting keywords from Requirements Traceability Matrix, prioritizing regression test cases based on the high-risk areas of the application, automating unit testing, helping with brand compliance checks, improving security checks and vulnerability assessments, delivering real-time feedback on the application’s performance are few areas where AI can come in handy. u AI can also add intelligence to the automated test execution to make them more stable, be agile, to self-heal by detecting and fixing threats even before they occur as the system is continuously learning using the data. AI with its Machine Learning and data analytics abilities when combined with automated testing will spearhead the quality assurance functions. u AI can also add intelligence to the automated test execution to make them more stable, be agile, to self-heal by detecting and fixing threats even before they occur as the system is continuously learning using the data.

July 2021 testmagazine.co.uk

In its third year the North American Software Testing Awards celebrate companies and individuals who have accomplished significant achievements in the software testing and quality assurance market. Enter The North American Software Testing Awards and start on a journey of anticipation and excitement leading up to the awards night – it could be you and your team collecting one of the highly coveted awards.

ENTER NOW softwaretestingawards.ca

September 2021 PROCESS & DEVELOPMENT

testmagazine.co.uk

Manual vs. Automated Testing Both manual and automated testing strategies are vital in ensuring the longterm efficiency and quality of a software application, as well as ensuring effective communication between testers and developers by reducing the feedback cycle Manual vs automated testing is a topic continuously cycling the tech industry - should you adopt a completely automated testing strategy to ensure that your application is defect-free? Or should you manually test only? The answer to this is - Both. Agile software development puts a high emphasis on quicker processes and higher quality outputs, but teams may use a variety of approaches to achieve these objectives. Both manual and automated testing strategies are vital in ensuring the long-term efficiency and quality of a software application, as well as ensuring effective communication between testers and developers by reducing the feedback cycle. If you devote all of your QA time to only

one of these methods, you'll miss out on a lot of opportunities to improve quality, the combination of both manual and automated testing would result in a higherquality, more reliable application. Within manual and automated testing there are two categories of testing: u Functional u Non-functional

Nuria Manuel

Functional testing involves testing the real-world business application of a software product. For example, a food delivery app like Deliveroo must be able to connect end-users to the restaurants in their local area.

Nuria is a Software QA Lead who is passionate about tech, engineering, and software quality. She’s an advocate for implementing quality processes at the very start of a products development lifecycle to minimise defects and maximise efficiency, and an advocate for all aspects of diversity in Engineering and Technology and loves to encourage students and to share her experience.

While non-functional testing involves assessing the readiness

September 2021 PROCESS & DEVELOPMENT of the application based on nonfunctional requirements (performance, security, reliability, scalability) that are not covered by functional testing. In this article, we will be going through a generic view of manual and automated testing, and the best practices of implementing both into your testing strategy. Both manual and automated testing strategies are vital in ensuring the longterm efficiency and quality of a software application, as well as ensuring effective communication between testers and developers by reducing the feedback cycle.

Manual testing in an automated world Manual testing is a method of testing and validating your application using test cases without the support of coded scripts and tools. For any new application to be automated, it is important that it is first manually reviewed to understand the end-to-

testmagazine.co.uk

will analyse the application for defects. Manual testers, with the human eye, are able to detect if a part of the application seems visually incorrect, in a better and more effective way compared to automated test scripts. When a tester interacts with software in the same way as a user does, they will find usability problems and user interface flaws, which are aspects that are challenging to replicate with automated testing scripts alone. As well as the above, there are several scenarios that are either technically impossible to automate or would be prohibitively expensive - as part of the software development lifecycle, it is important to assess the optimal strategy to follow for your application. There are several different types of manual testing methods which are beneficial in understanding the software being tested, the end-user, and continuous improvement requirements, such as exploratory testing. “While automated tests focus on codifying

“While automated tests focus on codifying knowledge we have today, exploratory testing helps us discover and understand stuff we need tomorrow” manual testing that is all about inquiry, exploration, and understanding. It involves testers exploring the application to find possible edge cases, as part of this, it is critical that testers comprehend the various personas who are likely to use the application. By exploring the software, testers will discover rare and hard-to-find bugs, that were previously unnoticed by automated testing, which follows predefined guidelines. It’s easy to miss something that you might not have been looking for in the first place. Aside from exploratory testing, there is a multitude of manual testing methods that can be implemented into a combined automated testing strategy, for example: u Smoke testing u A functional testing technique involves verifying whether a build is stable enough to proceed with further testing and that all the key features are working as expected. u Acceptance testing u A functional testing technique performed by users or a user acceptance testing team, to ensure that the software and application have met the requirements specification. u Usability testing

end flow and to determine whether automation is possible. When an application is complete and ready for production, real-world users are the ones that interact with it. The way in which a user operates with an application is very different from how an automated script

knowledge we have today, exploratory testing helps us discover and understand stuff we need tomorrow”. u Markus Gärtner (Gärtner, 2013) Exploratory testing is a method of

u A non-functional testing technique performed with real users. During a test, real users will typically attempt to complete standard tasks while observers observe, listen, and take notes. u Negative testing u A functional testing technique involves checking that the application behaves as expected with invalid or unwanted inputs. The purpose of this technique is

September 2021 PROCESS & DEVELOPMENT

testmagazine.co.uk

to ensure that the application remains stable with negative inputs. u Cross-browser testing u A non-functional compatibility testing technique used to verify if the application behaves as expected across different browsers and devices.

“Manual testing is a very important process to include in your testing strategy, however, this is not to say that the solution is therefore only to run manual tests”

At the start of a project, it is important to have a clear understanding of the business requirements for the application being built, for example: u Who are the target market users? u How are they expected to interact with the application? u How many browser types, devices, or screen sizes is the application expected to be compatible and responsive with? u What happens if I take the incorrect step or input invalid values in the application? This information is required to have a clear understanding of the most applicable and effective manual test methods and to support the development team in understanding the customer and user-like perspective of the application. Manual testing is a very important process to include in your testing strategy, however, this is not to say that the solution is therefore only to run manual tests. Manually testing an application is beneficial in adding more coverage to your testing strategy and understanding how your target market users interact with your application, however, it can require more effort and time on its own for a complete application, hence why automation is key in supercharging your product quality lifecycle. Manual testing is a very important process to include in your testing strategy, however, this is not to say that the solution is therefore only to run manual tests.

Automated testing As highlighted manual testing, such as exploratory and usability is extremely beneficial, however, there are other forms of testing such as regression and functional testing that although can be

completed manually, require a significant amount of repetition which is ideal for test automation. Automated testing is the process of automatically running tests on, for example, repetitive user flows, managing test data, and analysing the results in order to enhance software quality. Test automation success necessitates meticulous preparation and design work. As part of the automated strategy, it is important to define your objective and, before defining the tests to automate and the tools required, assess the feasibility of automation for the application being built. Without taking this into account, automation can end up increasing overall testing costs instead of saving them. Below are a few criteria to consider

when implementing an automated testing strategy: u Are there repetitive flows? u Are there multiple different data sets that need to be verified? u Does the application need to run on several different configurations and platforms? u Are there complex functionalities that introduce high-risk conditions? The results of a successful automated test must be repeatable. This ensures that if the application being tested does not change, each test run would produce the same results every time. If the application is not stable enough i.e., it is subject to frequent changes or significant unplanned changes

September 2021 PROCESS & DEVELOPMENT throughout the development lifecycle, this can impact your ability to automate. There are several different types of automated tests, and as previously mentioned, some of which can also be tested manually, these include: u Unit testing u Also known as component testing, is a kind of functional testing, carried out by the developers, where individual units of code/ components are tested to ensure that it works as designed. u Integration testing u A functional testing technique, where integrated units of code are verified as a group, to expose potential merge conflicts that could occur. u Regression testing u A functional testing technique, that is used to verify that a new code or feature implemented does not result in any unintended consequences. This usually involves testing the application end to end flow. u Performance testing u A non-functional testing technique used to measure an application's speed, response time, consistency, durability, scalability, and resource consumption under a specific workload. The right test for your application is very much dependent on your objective, the application you are testing against and the amount of coverage you require. After deciding your test objectives, the types of tests to automate, and the types of automated testing to include, you should determine what actions your automated tests will take as well as the best tools to use in the process. When creating tests, it’s important to consider the following: u When creating tests, keep them simple and based on a single goal, this makes tests more reusable. u Ensuring that your automated tests are broken down into smaller logical chunks rather than creating much larger tests that test various aspects of the behaviour of the application at once. By adding small tests that fix new features, you will be able to maintain your automated tests more frequently and easily. After creating a few automated tests, these can then be combined into a single, larger automated test. u When testing an application, rather than waiting until an entire feature is introduced, test the functionality of your application as you integrate it. This will allow you to effectively validate sections of the feature and quick feedback to the developers. These best practices should also be considered when choosing the right automated testing tool that best meets your overall project requirements. Consider these key points when choosing an automated tool: u What is your allocated budget? u Do you have enough professional personnel to dedicate to automation tasks?

testmagazine.co.uk

“It should therefore not be a question of whether either should be implemented, but instead of how both manual and automated testing can be implemented in the best way possible to maximise efficiency in ensuring product quality throughout the process ” u What are the systems and technologies that the automated tool should support? u Is the tool scalable as your application grows? u What testing types does the tool support? u Does it integrate well with your existing ecosystems, such as TestRail for test management or JIRA?

Conclusion Both manual and automated testing are strategies that are effective in ensuring maximum end-to-end coverage for your application. By integrating both you have the ability to explore better methods that suit your development lifecycle, your application needs, and your business needs. It should therefore not be a question of whether either should be implemented, but instead of how both can be implemented in the best way possible to maximise efficiency in ensuring product quality throughout the process. No one test fits every scenario, be sure to plan, assess and analyse the best strategy to implement. It should therefore not be a question of whether either should be implemented, but instead of how both manual and automated testing can be implemented in the best way possible to maximise efficiency in ensuring product quality throughout the process.

September 2021 PROCESS & DEVELOPMENT

testmagazine.co.uk

The Future of Test Automation Test automation is all about performing a series of checks in an automated manner on a product and gathering information to gauge its functional correctness, performance, security, and usability

As we are witnessing a drive to release better and faster software, automating testing is becoming a vital part of the development process. Software testing is now adopted by many enterprises to guarantee better quality, productivity, and customer experiences. It is then expected that testing and automation will only grow in importance in the coming years, so we have asked experts in the industry to reflect on the future of test automation.

What is test automation? Testing involves gathering information about a product to evaluate it, Ashish Narmen starts. Test automation, however, is all about performing a series of checks in an automated manner on a product and gathering information to gauge its functional correctness, performance, security, and usability.

Steve Mellor

Steve Mellor has been involved in softwre testing for over 20 years, working on desktop, server, mobile and web applications using a variety of platforms and technologies. When he’s not in front of a screen, he’s often out on the hills of Yorkshire and Derbyshire with a map, compass and sturdy boots.

Ashish Narmen

Ashish works as an Automation Test Lead at Beam Mobility. He has performed techno-functional roles in global teams and has career expertise in test engineering and test automation. He has worked in various business domains like ride aggregation, software licensing, logistics and service management; and in technical areas like mobile applications, web applications and web services, test infrastructure planning, provisioning and deployment.

Test automation is all about performing a series of checks in an automated manner on a product and gathering information to gauge its functional correctness, performance, security, and usability.

September 2021 PEOPLE & LEADERSHIP

testmagazine.co.uk

Steve Mellor adds that Test Automation has the tooling and techniques to perform repetitive tasks, so that humans can focus on what they do best - asking questions, designing tests and spotting mistakes.

The problems of test automation… According to Steve, in some organisations, there is too much focus on automating everything. This then leads to a maintenance burden as, as some things that shouldn’t be automated (complex, front end heavy tests) are being automated, they become unreliable and occasionally fail. They produce false positives and thus, need updating. Ashish states that test automation is slowed down by untestable application source code structure, tightly coupled software components, and software architecture that causes the product to behave inconsistently and unpredictably.

Changes are underway Thankfully, there is a lot more choice in tooling compared to 10+ years ago. Rather than one or two vendors offering paid for test tools, there is a wide range of paid for, free and open-source tools, Steven points out. However, the choice is not an easy one to make, and there is a risk of selecting a tool that changes its license terms or stops being supported. In addition, even the most popular free & open-sourced tools have different capabilities, further complicating tool selection. Moreover, Ashish emphasizes that software components and solutions for automating tests are changing to be more extensible and integrate better with existing tools. Engineering teams are also becoming more quality-conscious, encouraging everyone to contribute to testing (and to writing automated tests). Software components and solutions for automating tests are changing to be more extensible and integrate better with existing tools. Engineering teams are becoming more quality-conscious, encouraging everyone to contribute to testing. Some tools and processes can help produce better automated testing such as:

“Software components and solutions for automating tests are changing to be more extensible and integrate better with existing tools. Engineering teams are becoming more qualityconscious, encouraging everyone to contribute to testing” u Processes to release with high quality ensures that application code and components are easy to test. u Tools like Cypress have better support for mocking external dependencies, and the tests can focus better on the component/subject under test. u Application development frameworks like Flutter have better support for test automation at different levels. u Easier to spin containers to provision dedicated test environments for executing

automated tests. u Reporting tools can leverage analytics to analyse test failure causes. A lot of improvements in the UI automation space have been witnessed over the past few years in order to help make tests more reliable by using more advanced locator strategies that are less prone to errors due to code changes, rather than just failing. However, it’s important to focus on testing at the appropriate levels - picking a BDD based tool and building a large number of end-to-end tests will still take up a lot of time and result in a big, difficult to maintain codebase of flaky tests.

Benefits & Challenges The new benefits of automated testing should be time saved in building and executing tests, as well as allowing products to be out to market faster, with the appropriate amount of testing performed. Steve underlines that there is no point in releasing a perfect product later than your competitors, or quickly releasing a defective product that loses you customers.

September 2021 testmagazine.co.uk

PEOPLE & LEADERSHIP

Ashish adds that the advantages of test automation would be for QA to focus on exploratory testing, as well as have faster releases by reducing the time taken to execute regression tests. Yet, there are still some drawbacks to test automation. Indeed, there are still engineering teams that refuse to invest in quality processes and develop and deploy untestable (or difficult to test) software components. Similarly, application architecture decisions, at times, do not take testability and the ability to add tests easily into account. As with all ‘new’ things, you need to consider if they do deliver on the promises and if that help in the context of your organisation and product?

What future for test automation? Some aspects of test automation are bound to change in the future. Steve believes that test automation will see the emergence of tooling that uses machine learning. For example, there will be advances in visual testing to compare rendering on different browsers and devices promises in order to make cross browser testing easier. It remains to be seen what other use cases for ML can be found in software testing. Ashish believes that, in the future, automated tests for regression are likely to become the norm. There will be more investment in test impact analysis tools to shorten the time taken to execute automated tests as well as more integrated tools to view the quality posture of a specific release. Besides, he thinks that levels will get more granular, and the focus will shift to testing re-usable components instead of the entire products. ‘Economists prize efficiency, but nature favours resilience’ For Steve, when you are embarking on a test automation project, you must design and build them to be maintainable and extensible. More than once, he has seen an ill-thought-out test framework be scrapped and re-implemented… ‘Start small with reliable tests, iterate and expand, and be smart enough to realise when there is sufficient automation. Human testers are great at spotting patterns, finding mistakes and asking great questions you can’t automate everything!’

5th - 6th October 2021 at The British Museum, London

September 2021 testmagazine.co.uk

INTERVIEWS

Wojciech Olchawa Interview Test Magazine Wojciech Olchawa is the Test Automation Program Manager at Tecan Wojciech is an experienced test engineer with passion for test automation. He specializes in ensuring that test automation is applied wherever applicable as well as for educating other test engineers on how to automate in the most efficient way. Wojciech is a powerful force in the workplace and uses his positive attitude and tireless energy to encourage others to work hard and succeed. In his free time, he enjoys watching martial arts and cooking pizza.

September 2021 INTERVIEWS What is your current role and responsibilities? I work as a Test Automation Program Manager at Tecan Schweiz AG. I am responsible for driving the Test Automation activities. That includes (but is not limited to) test automation framework development, training for verification engineers, cooperation with the Development Teams, DevOps teams etc.

testmagazine.co.uk

“On the other side, I really enjoyed documenting bugs/trying to reproduce them/checking if they have been fixed. That is how I discovered Testing”

What inspired you to get involved in the Testing industry?

that I could participate in an exchange program between the Lufthansa System offices in Gdansk and Zurich. That is how I got to live and work in Switzerland.

Believe it or not I started off as a Core JavaScript developer. It was a part time job, however after a couple of months it made me realize that I don’t have a great talent for coding. On the other side, I really enjoyed documenting bugs/trying to reproduce them/checking if they have been fixed. That is how I discovered Testing.

After over a year of testing in-flight software solutions, I started to work as an external employee (consultant) at Tecan Schweiz AG. My activities back then included mainly software tests, however for the first time in my life, I was testing software which controls instruments. It turned out to be an extremely interesting topic, especially given the fact that the SW development process is highly regulated.

On the other side, I really enjoyed documenting bugs/trying to reproduce them/checking if they have been fixed. That is how I discovered Testing.

Why did you choose to work in test automation? I believe it’s a natural development of your journey as Test Engineer. We live in times where the expectations on delivery to market are very demanding. Given that, it would be extremely hard to maintain the delivery pace when performing the tests manually.

Once the project at Tecan was over, I’ve joined a start-up company – Progile – were I was a part of a team which has developed a test automation platform called TestResults.io (test automation based on Image Recognition and OCR). After little more than 4 year I decided to leave the company and came back to Tecan but this time as an internal employee. And that is how I became the Test Automation Program Manager.

Who do you look up to for inspiration or mentorship? I don’t have a specific person in mind. I rather look up to what are the latest trends by going through forums and checking what can be done better. I also value the input I get from not only Software Testers, but also Software Developers and DevOps engineers. In the end, we all have the same goal – better, faster, more reliable SW deliveries. I also value the input I get from not only Software Testers, but also Software Developers and DevOps engineers. In the end, we all have the same goal – better, faster, more reliable SW deliveries. How do you keep your team motivated despite conflicts and obstacles? Maybe that will sound as cliché, but I strongly believe in positive attitude. Whenever we face obstacles or feel frustrated, a good,

Aside from that, I was tired of the repetitive work, which could have been easily solved with test automation.

Can you tell me about journey and how you got where you are now? That is quite a long story! I started off as a JavaScript developer. Afterwards came my first testing job at Lufthansa System Poland as a Junior Test Engineer. I’ve learned most of the test fundamentals there and thanks to

“I also value the input I get from not only Software Testers, but also Software Developers and DevOps engineers. In the end, we all have the same goal — better, faster, more reliable SW deliveries”

September 2021 INTERVIEWS

testmagazine.co.uk

sarcastic comment usually releases the tension. As it comes for motivation – this is quite simple: Shaping the test automation at a cool company, working with latest technologies, pushing the limits of what can be automated – this is a great motivator for the whole team already.

What are your current goals? Workwise: Each single new SW project automates tests from Day-1. Test Automation coverage – over 90% of tests. Life-wise: loose the Lockdown weight. What are you the proudest of in your career so far? 1. Having the guts to leave my comfort zone and move to a different country. 2. Transitioning from manual to automated testing

What has been your greatest challenge? Learning what Object-Oriented programming actually means. Although, I did code back at the university, I mainly did in a scripted fashion. In order to benefit from all the test automation “goodies”, I really had to understand what the good coding practices are and what you should avoid. In order to benefit from all the test automation “goodies”, I really had to understand what the good coding practices are and what you should avoid.

What is the favourite part of your job? Working with talented people full of ideas. What have you learned from your experience so far? I would say the biggest lesson I’ve learned is that you should never think that you know everything there is to know about testing/test automation etc. Don’t get me wrong, you should be aware of your knowledge and know-how, however there is always something more you can learn.

Finally, do you have any advice to aspiring testers who want to grow in the tech industry? Don’t stop learning and don’t be afraid of trying out new things. Look-up to people who are better than you. Get involved in testing communities and benefit from their know-how. Don’t stop learning and don’t be afraid of trying out new things!

September 2021 PEOPLE & LEADERSHIP

testmagazine.co.uk

Test Automation in Agile Teams There are different types of agile methodologies but whatever the methodology is, the role of QA and automation remain the same

Any project that’s active, runs with a certain methodology for delivery – whether it is waterfall model, iterative development model, any type of agile model. The team is set up and the tasks are based on the model. The quality and time to production also vary with the model. Before we jump into understanding how test automation is done in Agile teams, we will see how the transformation to agile has transformed testing. In waterfall or iterative development, the testing team is not involved until the final development is done. This would mean the testing is done towards the end and at the far right of the cycle. In agile, development and testing happen in parallel. There is also a concept of Development in the test which is the first step of transformation to agile. A project that moves from a waterfall into agile model is not an overnight change. Instead, the projects will move into sprint-based testing and then transform to complete agile eventually. There are different types of agile methodologies but whatever the methodology is, the role of QA and automation remain the same.

Gayathry Adhavan Gayathry is an engineering graduate, who started her career in 2007, as a manual tester. She self-taught automation and moved to an automation tester role. After a couple of years, she became a non-functional tester — inclusive of Load, resiliency, and accessibility testing. When the digital transformation of platforms happened from legacy to cloud, she became a cloud non-functional test architect. She is a certified scrum master as well and worked as an agile coach for teams who wish to move to agile technology. Apart from being a full time NFT architect, Gayathry loves travelling around, having been to 13 countries in Europe, and quite a few Asian countries as well.

September 2021 PEOPLE & LEADERSHIP

testmagazine.co.uk

The 5C’s of DevOps, which is common knowledge now is: Continuous Integration -> Continuous Testing -> Continuous Delivery -> Continuous development -> Continuous Feedback

“Automation is the driving force behind continuous delivery in agile practices”

What is Test Automation? It is the process of testing with less or no human interaction, to increase test execution efficiency, coverage, and effectiveness. Automation is the driving force behind continuous delivery in agile practices. In order to get the most out of automation, all teams from BA to DevOps are involved. Although there are examples like adhoc testing needs, complex business logic that may require the need for manual execution, automation doesn’t fully replace the need for manual testing. GUI, API, Integration, component, and unit tests – all form a part of a complete automation pack. Test automation is done for both functional and non-functional testing. Although the process of identifying test cases for both differ, and all test cases for non-functional testing are automated, a QA with the right skill set in the team, makes test automation easier. Automation is the driving force behind continuous delivery in agile practices.

u Test Coverage u Continuous Testing u Faster feedback u Shift testing to left

Why Test Automation?

u Improved test accuracy

A few of the many reasons for preferring automation in testing are:

u Faster regression cycles

u Capacity planning, Identifying Scalability

Benefits to the business: u Quality Product

analyse before jumping into automating everything. So, what should, and shouldn’t I automate? To answer this, below are some tips. Tests that should be automated: u Business critical flows u Repetitive tests that run for multiple builds u Tests that require multiple data sets u Frequently used functionality that introduces high-risk conditions

u Incremental development

u Customer satisfaction

u Frequent Changes

u Cost-efficient

u Continuous Testing

u Higher confidence

u Quick review of code quality

u Faster time to Market

u All critical flows that warrant nonfunctional testing

u Increased Return of investment

Tests that should NOT be automated:

What are the Benefits of Automation? The benefits are twofold - Benefits the QA teams and Benefits business. Benefits to the teams:

Should I automate everything? Although the benefits of automation are higher and it’s tempting to automate all test scenarios, the wiser it is to

u Cross browser/platform tests u Time consuming tests

u Tests that will only be executed limited times u User experience tests for usability u Exploratory tests u Tests that require visual confirmation

September 2021 PEOPLE & LEADERSHIP

u Tests without predictable results u Tests that cannot be fully automated u Tests that don’t warrant non-functional testing and functional testing Although the benefits of automation are higher and it’s tempting to automate all test scenarios, the wiser it is to analyse before jumping into automating everything!

What would be an ideal approach for Test Automation in Agile? Identifying the regression cases in the previous sprint and incorporating the automated pack in the next sprint along with identifying new set of regression cases is the key to remember. This is an iterative process for every sprint and the automation pack is formed. For an easy understanding, the below depiction helps.

testmagazine.co.uk

Developers in test and Pair programming are also few approaches for projects that are in their first step of agile transformation. These methods bring development and testing teams together. The tester can indicate the developer on the functional test cases for a feature WHEN the developer starts developing. This results in fewer or 0 defects after development.

What are the tools that aid test automation in Agile? There is a wide range of tools available in the market that enables automation and continuous integration. I can list out a few which are commonly used for test automation and for continuous testing: u Selenium – the most popular tool, friendly to use for automation, supports functional testing majorly u Neoload, JMeter, VSTS (Enterprise version) – are some tools that support non-functional testing

Some pre-requisites to look at for using the above approach: u Automation enabled testers available in the team u Automation tasks to be a part of the sprint u Automation friendly design u Availability of prototypes/reference docs before the start of the sprint The above approach is majorly defined for functional testing, as all non-functional testing needs the stories to be automated.

“Although the benefits of automation are higher and it’s tempting to automate all test scenarios, the wiser it is to analyse before jumping into automating everything!”

u GitLab, Jenkins, Docker – are common tools that support continuous testing and continuous integration.

What do we infer? A fact to agree upon is, the cost of automation is higher initially - tool usage, script development, hardware requirement consumes most of it, especially when the application is hosted in the cloud. But once the test harness

“With the agile transformation and DevOps flaring these days, test automation is important to bring more features to the end-users and hence higher ROI” is ready, execution can be done a number of times, repeatedly (with minor maintenance), and acquire the same accuracy in the test results. QA needs to be involved in sprint planning to understand the story points and the testing requirements. With the agile transformation and DevOps flaring these days, test automation is important to bring more features to the end-users and hence higher ROI. With the agile transformation and DevOps flaring these days, test automation is important to bring more features to the end-users and hence higher ROI.

ENTRIES NOW OPEN ENTER NOW devopsindustryawards.com

We’re on the look out for you! Calling all Software testing, QA or DevOps professionals?

A full-service recruitment consultancy for Software Testing, QA, & DevOps Professionals

TestAssociates.co.uk