SD Times 2019 Software Testing Showcase

Page 1

S

oftware testing ain’t what it used to be. That’s for sure. The days of waiting for a software release incorporating new features and bug fixes to be locked down so it could be “thrown over the wall” to test/QA teams to validate are long gone. We now see organizations releasing software multiple times per day, all of which have to be tested for functionality, and to make sure they don’t break the application. Some changes could be in a monolithic application, some might be in microservices and still others are to applications running on mobile devices.

These changes in how we develop have led to a sea change in how we test. Today, automation is critical, and testing has “shifted left” to be done much earlier in the development process. It’s daunting, but with the right processes and tools, your testing practice can not only keep up with development but can also help you release better quality software at this quickened pace. This showcase is a guide to some of the companies providing testing tools, as each comes at the challenges from a different perspective. We hope you find it useful. i


Full Page Ads_SDT028.qxp_Layout 1 9/24/19 10:19 AM Page 30

The Cloud’s #1 Continuous Testing Platform Accelerate digital transformation across the enterprise with a comprehensive suite of software testing tools – from agile test management to automated continuous testing for enterprise architectures.


S OF T WARE TES TING S HOWC AS E

Test Automation at the Speed of DevOps By Lisa Morgan Remember the old adage: “Speed, Cost, Quality — pick any two?” This tradeoff is no longer an option. With digital transformation initiatives driving DevTest teams to scale Agile and adopt DevOps, all three vectors must be addressed. This requires a fundamental shift in how you approach testing. It’s not a matter of acquiring more or better tools; holistic process transformation is required. That’s why Tricentis defined “continuous testing” and developed the industry-leading continuous testing platform. Organizations turn to Tricentis to make a fundamental process change in their testing process while lowering business risks and accelerating software delivery. Tricentis’ intelligent modelbased approach ensures the reliability, flexibility and actionable insights companies need to accelerate and automate the most demanding delivery cycles. “Continuous testing ensures that the right stakeholders have access to the right information at the right time. You can execute automated testing as part of the software delivery pipeline and obtain immediate feedback on the business risks associated with a software release,” said Wayne Ariola, chief marketing officer at Tricentis. Tricentis is among the 2019 SD Times 100 for its leadership in the Testing category and has also been recognized as a “leader” by Forrester, Gartner and IDC. Its continuous testing platform enables organizations to achieve more than 90 percent test automation, 5X or better speed increases and more than 80 percent risk coverage. Tricentis’ successful enablement of continuous testing is further reflected in its 80 percent annual growth rate and over 1,600 global customers. In addition, Tricentis enjoys the largest software distribution among global systems integrators.

SCALED AGILE, DEVOPS NEED CONTINUOUS TESTING Agile success is easier to achieve with a small project that involves just a web interface. However, as Agile projects scale, the underlying architecture and the necessary testing strategies become more complex. While risk mitigation strategies tend to include shift-left and shift-right, what’s really needed is continuous testing throughout the life cycle, especially as teams advance to DevOps. “When you scale Agile, you recognize that you need to radically alter the traditional testing processes. When you move to DevOps, those processes must again scale to an entirely different level of automation,” said Ariola. “As you move down the DevOps path, eventually, you conclude that continuous testing must be executed within the contexts of business objectives and business risks — not just ‘siloed application test results.’ ”

Scaled Agile and DevOps break down when speed and quality imperatives are out of sync. A symptom of that is approaching continuous testing from a tool perspective rather than a process perspective. The former creates process friction while the latter alleviates it. “You need holistic process transformation to achieve business transformation and DevOps transformation goals,” said Ariola. “A ‘chasm’ that often plagues organizations is the gap between a small team of brilliant, technical people who succeeded with an Agile or DevOps pilot and the larger population necessary to affect scaled Agile or DevOps which is more diverse.” For a small project with a simple architecture, Selenium scripts or free Selenium-based tools work well. However, when Agile and DevOps scale, the underlying architecture becomes more complex necessitating intelligent, end-to-end continuous testing processes.

IMPROVE USER JOURNEYS AND LOWER BUSINESS RISKS Consistent user journeys have become more important than individual component outcomes, so software teams need a means of ensuring quality that doesn’t slow value delivery. “Transformation conversations are tightly aligned with business objectives because companies understand they’re competing with digital natives who are disrupting entire industries,” said Ariola. “Software teams adopted CI to execute builds faster and more consistently. More recently, they’re adopting CD to package and ship quickly and automatically. What’s missing is the governance that can ensure quality across the end-to-end user experience. This is enabled through continuous testing.” Instead of asking whether testing is “done” yet, organizations should ask whether a release reflects an acceptable level of risk. To achieve that, software teams need to understand what they’re going to release, when they’re going to release it, the level of acceptable business risk and how they’ll measure success. Using Tricentis, Agile and DevOps teams can measure and govern risk, as well as quickly define and update tests across the architecture. Its model-based testing capabilities propagate a change throughout an entire test suite automatically. “Tools don’t solve the problem of testing at scale; process transformation does,” said Ariola. “Tricentis allows you to focus on the holistic problem of end-to-end continuous testing using a business risk approach. We support over 160 different technologies so you can achieve the resiliency, speed and success you seek at scale.” Learn more at www.tricentis.com. i


S OF T WARE TES TING S HOWC AS E

Top UI Testing Tools for Selenium Users product manager, Parasoft UI testing is relatively straightforward, as long as nothing changes. But the problem is, web UIs change all the time. Many new UI testing tools have come to market in the last few years, but only a handful tackle this problem without forcing the user to abandon Selenium to adopt a new, proprietary platform. BY CHRIS COLOSIMO,

KATALON For a free tool, Katalon does quite a lot. Its recorder plugs into the Chrome browser so you can generate test cases, and then the tool builds them in the Katalon IDE using the page object model, so they are highly maintainable. Pros: • Recommends smart locators • Uses the page object model • Has a self-healing capability (paid plug-in) Cons: • Although you can export your tests as Selenium, once you’ve exported you lose all the usability of Katalon, and the export doesn’t include the page object model, so the tests become hard to reuse and maintain • Their expensive customer support is only available at the enterprise level • Doesn’t directly integrate into your existing execution framework (uses a proprietary framework, CI integrations are available as plugins) Key takeaways: Users can start using Katalon with limited technical knowledge, creating test cases quickly, thanks to the keywords in the program. It’s free to use Katalon Studio to test your Web and Mobile applications, but to get the benefit of some of those exciting innovative technologies, you’ll have to upgrade or purchase paid plugins.

PARASOFT SELENIC Parasoft Selenic enhances an existing Selenium testing practice by adding AI-powered locators and self-healing. Integrating directly into your existing CI/CD pipeline, Selenic analyzes the Selenium tests at runtime, and if a failure is detected (e.g. due to a bad locator), it will self-heal the test and provide recommendations on how to fix the test for future runs. Selenic’s recorder for Chrome generates pure Selenium tests that leverage the page object model for maximum maintainability. Pros: • Creates Selenium tests that use the page object model • Executes self-healing on Selenium tests at runtime • Integrates directly into your CI process Cons: • No free version (just a free trial) • The current version only supports Java, JUnit 4 and Eclipse Key takeaways: AI-powered recommendations on existing selenium tests make it easy to add this solution to your existing

Selenium practice and gain significant value. One of the only tools that comes with enterprise-level customer support.

RANOREX WEBTESTIT Webtestit is a new offering from Ranorex. It’s pure code, with a “design first” approach vs. record-and-playback. You can interact with your user interface and build your scripts through helper actions in their proprietary IDE. It's still a little unclear where Ranorex Studio stops and Webtestit begins, but this should become clearer over time. Pros: • Uses the page object model for test creation • Recommends locators through their Selocity chrome plugin which is available for free • Great helper actions during test creation within the IDE Cons: • No recording functionality • Proprietary IDE — developers have to ditch their IDE of choice • Just a solution for test creation — no analysis at runtime, or self-healing capabilities Key takeaways: Organizations looking to have an assistive tool for code creation will likely find this solution useful. Without record-and-playback, users are encouraged to build from the bottom-up rather than top-down. In combination with their Selocity Chrome plug-in, users can draw out pages as objects and import them into the webtestit tool, making it very friendly for those looking to use the page object model.

SELENIUM IDE Selenium IDE is an open-source project that harnesses the power of Selenium in a Chrome plugin. Getting started with Selenium IDE requires no additional setup other than installing the extension on your browser, aligning with the project’s driving philosophy of providing a tool that’s easy to use and gives instant feedback. Pros: • Users can export any created test into a script, making it a powerful tool for UI test creation • Offers multiple locator strategies for each element it records • Has a self-healing feature (but doesn’t work on Selenium scripts outside of the Selenium IDE) Cons: • Does not fit directly into your CI/CD process • A black box (when something doesn’t get recorded, you have to export to code to figure out what’s going on) • Doesn’t use the page object model when building tests Key takeaways: The Selenium IDE is simple to use, giving users the ability to rapidly create tests against their Web UIs. It will feel natural for those accustomed to the framework, but it’s still an evolving open-source project and not always reliable. i


Full Page Ads_SDT028.qxp_Layout 1 9/24/19 10:20 AM Page 33


Full Page Ads_SDT028.qxp_Layout 1 9/24/19 10:20 AM Page 34


S OF T WARE TES TING S HOWC AS E

Mobile Labs Right-Sizes Testing By Lisa Morgan When it comes to mobile app testing labs, one size doesn’t fit all. App team sizes differ, as do app requirements and team preferences. Recognizing that, Mobile Labs recently announced three different solutions for teams just getting started with testing, enterprise teams and teams that need real-time testing capabilities. “The biggest testing challenge in the mobile space is the pace of change across devices, operating systems and apps,” said Angela Culver, chief marketing officer at Mobile Labs. “Companies, particularly in retail, realize that mobile apps help them increase product-related revenue. The competitive environment has become so fierce, they’ve moved to agile and DevOps releases, but from a brand standpoint, they can’t sacrifice app quality for delivery speed.” Delivering apps with high confidence requires testing on actual devices, since simulators and emulators fall short. With Mobile Labs’ GigaFox mobile testing lab with hosted and onpremises offerings, Agile and DevOps teams can choose the option that best meets their needs while enjoying the freedom to use their choice of DevOps, build, automated testing and manual testing tools and processes. All GigaFox offerings centralize testing in a private hosted environment or behind a firewall, so distributed teams don’t have to buy devices for one location and ship them elsewhere.

provides instant, real-time access to devices to team members anywhere on the globe. “GigaFox Red is our main enterprise offering,” said Culver. “Customers use it for manual and/or automated testing, using built-in Appium or other commercial and open-source testing tools including Tricentis, Eggplant, Micro Focus, and Ranorex. It also supports Jenkins and other build systems.” GigaFox Red brings private cloud benefits to development, DevOps, and testing workflows. It integrates seamlessly with a customer’s network, giving teams complete control of performance, security and assets. Anchored behind corporate firewalls, GigaFox Red is a high-performance, small-footprint mobile device testing lab that houses iOS and/or Android smartphones and tablets in a single, lockable cabinet.

GIGAFOX STARTERKIT

GigaFox Silver can reside behind corporate firewalls or inside Mobile Labs’ secure data center. Either way, GigaFox Silver provides access to more iOS and Android smartphones and tablets than GigaFox Red. Like the smaller versions of GigaFox, GigaFox Silver includes Appium automated testing and supports all build and DevOps tools as well as manual and automated testing tools and processes. “One thing that differentiates GigaFox Silver is the Graphics Extension Manager (GEM). It can handle streaming games or other rich content experiences that need to be tested in a realtime environment,” said Culver. “Real-time speed anywhere in the world is another differentiator. We optimize bandwidth so team members regardless of their location can have the same experience. If there’s an absolute requirement for no lag time, GigaFox Silver is for you.” GEM adds new levels of real-time smoothness and responsiveness to hosted cloud devices, so teams can avoid the delays and choppiness that degrade the effectiveness of rich content testing. With GigaFox Silver, enterprise mobility teams are empowered to deliver superior user experiences that distinguish their apps, products and brand. For more information visit www.mobilelabsinc.com. i

GigaFox StarterKit is the most basic version of Mobile Labs’ GigaFox secure mobile device lab. Teams get instant access to five dedicated iOS and/or Android devices via a monthly or annual subscription. The service is also available as an onpremises solution for those who prefer to test behind a firewall. “GigaFox StarterKit offers many of the same benefits the higher GigaFox subscription levels deliver, but it’s geared toward smaller teams and teams that are just getting started with mobile testing,” said Culver. GigaFox StarterKit includes 24x7 access to private, dedicated devices, real-time manual testing and built-in Appium test automation, all of which are available in Mobile Labs’ enterpriselevel GigaFox Red, but in a smaller, nimbler footprint. GigaFox StarterKit also provides the speed and superior performance of GigaFox Red but with fewer devices available for testing.

GIGAFOX RED GigaFox Red, formerly called deviceConnect, is for enterprise teams that want to test multiple apps across multiple platforms, operating systems and device types. Available as an on-premises offering only, GigaFox Red is a secure mobile testing lab that

GIGAFOX SILVER GigaFox Silver is a secure mobile testing lab that provides team members anywhere in the world with instant, real-time access to devices. As a hosted or an on-premises solution,

“The biggest testing challenge in the mobile space is the pace of change across devices, operating systems and apps.” —Angela Culver


S OF T WARE TES TING S HOWC AS E

Eggplant Uses AI to Keep Users Happy, Drive Value By Christina Cardoza Everyone wants to test faster and cheaper, but they forget about testing smarter in the process. A recent report from the test automation company Eggplant found improving customer experience is the number one priority for organizations, but almost half of businesses are having trouble testing for real user experience. The research also found that business leaders and development teams are misaligned when it comes to testing strategies. For instance, teams prioritize the productivity of testing more than the business, and the business is more focused on improving the usability of the product than IT teams. “To accelerate digital transformation, business leaders must recognize that delivering the best customer experiences hinges on the quality of their software. The way organizations build and test software can improve, or hinder, customer experience and overall business outcomes,” the Eggplant report stated.

VERIFICATION VERSUS VALIDATION The traditional way of testing was to think of testing in terms of verification and validation, according to Antony Edwards, COO of Eggplant. Verification looked at whether or not the software did what it was supposed to do while validation looked at whether it was what users actually wanted. What happened was testers thought they could start with verification because that seemed easier to obtain, than once verification was in place they would move onto validation. However, as a result testers lost sight of user expectations and focused just on testing that the softare worked. “No one can write specifications of sufficient details that really predicts everything a user might care about, and as a result of that you have testing teams becoming more technical, which takes them away from the users and problem domain,” said Edwards. “Testers decided they would rather focus at a very low level of detail on technology matters, looking at verification and compliance” However, Edwards explained if you ask users what they look for in a product, they care about performance, usability, accessibility and design — and that is what testers should be worried about. “As software becomes a key part of businesses and as business almost becomes digital products themselves, all those factors around usability, performance, accessibility as well as functionality just massively drive that user experience,” said Edwards. “Companies need to understand that and test teams can’t just say we are just focusing on making sure the functionality compiles with the specification.”

WHERE AI AND EGGPLANT COMES INTO THE PICTURE An AI-enabled continuous test automation solution can help deliver higher quality software more quickly. Testing is full of data. Business have access to data on all the tests runs that were ever done, what the results were, what products shipped, and what defects were found after the testing — yet they aren’t properly utilizing any of this data. According to Edwards, machine learning is a great way of identifying how technology factors actually impact users, and that information can then be brought back into your development and testing. Eggplant is designed to auto generate tests, select which tests to run and decide what failure looks like.

“No one can write specifications of sufficient details that really predicts everything a user might care about, and as a result of that you have testing teams becoming more technical.” —Antony Edwards “When people say they have automated 100 percent of their testing, it is a bit of a misnomer because what they actually automated is test execution,” said Edwards. “If you still have people designing tests, writing test scripts, maintaining test scripts, setting up environments, creating test data, reading test results, debugging tests, and deciding what tests to run, then you really haven’t automated testing. You’ve automated test execution.” With AI, Eggplant is able to look at the application, code, APIs, user interface, user behavior, production data and start to understand how it all influences business aspects and what tests to run. “AI is great at taking a huge search space and then condensing that down and focusing on things you care about,” said Edwards. “That is what we do. The company can generate billions of test cases for users to can run, look at the code, decide where the risks are, look at the testing and results already in place, look at the changes, look at who made the changes and decide what is important. What users end up with is a continuous and automated approach to software that improves business outcomes, aligns with what the customer wants, and delivers continuous digital improvement. “People are making software for a purpose, and Eggplant is all about how do we help people make software that delights users and drivers the user outcomes,” Edwards added. Learn more at eggplant.io. i


Full Page Ads_SDT028.qxp_Layout 1 9/24/19 10:20 AM Page 37

Let’s rid the world of bad software.. Bad software is costing your business more than you think. ÀǘƊȁDzȺ ɈȌ ǞȁɈƵȲȁƊǶ ƊȁƮ ƵɮɈƵȲȁƊǶ ƮƵ˛ƧǞƵȁƧǞƵȺ‫ ة‬ǿƊȁƊǐƵǿƵȁɈ ǏƊǞǶɐȲƵȺ ƊȁƮ ɈƵƧǘȁǞƧƊǶ ƮƵƦɈ‫ ة‬ɈǘƵ ƧȌȺɈ ȌǏ ƦƊƮ ȺȌǏɈɩƊȲƵ Ǟȁ ɈǘƵ Ç‫خ‬²‫ خ‬ƊǶȌȁƵ ǶƊȺɈ ɯƵƊȲ ȲƵƊƧǘƵƮ ‫ ׄ׈خׂٽ‬ɈȲǞǶǶǞȌȁ‫ خ‬RȌɩ ǿɐƧǘ ǞȺ ǞɈ ƧȌȺɈǞȁǐ ɯȌɐ‫ د‬ We’re eggplant, and we’re on a mission.

eggplant.io

²ɈƊɈǞȺɈǞƧ ǏȲȌǿ‫ ب‬The Cost of Poor Quality Software in the US: A 2018 Report. !ȌȁȺȌȲɈǞɐǿ ǏȌȲ XÀ ²ȌǏɈɩƊȲƵ ©ɐƊǶǞɈɯ ‫!ـ‬X²©‫ خف‬RƵȲƦ jȲƊȺȁƵȲ‫ خ‬²ƵȯɈƵǿƦƵȲ ׂ‫خ׈ׁ׀ׂ ة׆‬


Full Page Ads_SDT028.qxp_Layout 1 9/24/19 10:22 AM Page 38


S OF T WARE TES TING S HOWC AS E

ProdPerfect Removes the Burden of QA Testing By Christina Cardoza There are typically three levels of quality assurance testing maturity. One is the classic waterfall approach where it takes weeks to get a deploy ready. Then, there is the continuous development and continuous delivery approach where QA engineers are put in place to handle automation. The most mature way of tackling QA is removing QA engineering as a separate practice, and making all your engineers responsible for the quality of features. The problem is that none of these levels of maturity seem to be able to get QA right. “No one has a good answer. Enterprise are failing in waterfall structures. Agile teams are failing or running into difficulty hiring and maintaining QA engineers. Silicon Valley is having to hire only the most senior folks, and even then it is through force of will and pain they are able to keep test suites to a point they are happy with,” said Dan Widing, founder and CEO of the automated QA testing provider ProdPerfect.

AUTOMATING QA There is a better way. ProdPerfect removes the struggle it takes to set up a QA engineering department, and automates QA testing using live user data. This is “dramatically cheaper, dramatically faster, gets you a result faster, [and is] going to nearly guarantee that you catch bugs as part of your process,” Widing explained. ProdPerfect is able to obtain live user data by analyzing web traffic and creating flows of common user behavior. That behavior is then built into an end-to-end testing suite that ProdPerfect maintains, expands and updates based on actual user data over time. According to Widing, QA testing is “incredibly difficult, painstaking work that almost tends to be underappreciated by the organization itself,” and the folks who are having to deal with this are just overburdened with work. “We have a mechanism that lets us shake out the environment the customer needs us to test against… and then we are using a testing framework that lets us plug in our learnings from these steps to produce an automatically updated test suite,” he continued. “The experience the customer gets is a black box QA engineering department... What you get at the end is an autoupdated test suite that can run continuously in your CI system that just tests your application.” ProdPerfect covers every core workflow with applications, provides 95 percent or more of test stability, less than fourhour regeneration of broken tests, and less than 48-hour test

coverage for new feature sets. “You don’t need to do anything to build, maintain, or expand the testing suite. We got it. You need to respond to bug reports, of course, and keep a stable testing environment up and running for us, but that’s all. Very frequently people call this ‘magic’ or ‘too good to be true,’” the company stated on its website.

GETTING THE RIGHT METRICS ProdPerfect not only works to ensure QA testing is covered, but also works to help teams understand what the right metrics to quantify success are. “That is something we put into our service every step of the way. What your browser automation should be doing is catching as many significant bugs as possible whatever stage it is testing at and then otherwise staying as much out of the way,” said Widing.

“If you don’t set up your design and data strategy or set up the right tooling, everything falls apart.” —Dan Widing

You will know you have a solid testing foundation in place when you don’t ship a fire drill-style bug and have to wake up in the middle of the night and figure out how to deal with it or who is on top of it, Widing explained. Since ProdPerfect is already analyzing what users are doing, it can project how things should be working and make sure they stay working. The solution tests features continuously, detects any significant bugs and verifies the feature set is actually working. “We aim to stay out of the way by crafting what are the other metrics that are important to make sure you are not slowing down the software team,” said Widing. Additionally, the solution will measure against minimumfrequency thresholds to confirm its performance. “If you don’t set up your design and data strategy or set up the right tooling, everything falls apart and you have to work particularly hard to make sure all the pieces work together otherwise any singular improvement is not going to help you at all,” Widing said. Learn more at ProdPerfect.com. i


S OF T WARE TES TING S HOWC AS E

Perfecto: Continuous Testing at Scale By Lisa Morgan DevOps teams need to deliver high-quality web and mobile software quickly to satisfy end customers and keep their employers’ businesses competitive. But it can be difficult to assure product quality when the target devices are undergoing software and browser updates themselves. Testing remains a DevOps bottleneck because there isn’t enough testing and quality assurance talent available to achieve quality at scale, especially when app updates are being delivered every week or every day. Using Perfecto’s cloud-based continuous testing service, software teams can achieve their release velocity and quality targets consistently using their existing talent. “One of the things customers like about Perfecto is its flexibility,” said Eran Kinsbruner, chief evangelist at Perfecto by Perforce. “You can write Selenium or Appium scripts to automate tests or you can use machine learning to automatically create tests based on smart and maintainable script recording and playback.”

“There are four pillars you need to be successful with continuous testing: Creation, Execution, Lab, and Analysis.” —Eran Kinsbrunner With Perfecto, DevOps teams that are building mobile and web apps can easily scale testing as necessary across real devices and desktop browsers they don’t have to buy, configure or maintain. When a coding error arises, Perfecto automatically identifies it so it can be resolved immediately. If a device loses a Wi-Fi connection or a screen locks, Perfecto restores it immediately so continuous testing remains continuous.

THE FOUR CONTINUOUS TESTING SUCCESS PILLARS The Google Cloud team recently released a report that explains the differences between elite DevOps teams and trailing DevOps teams. The trailing teams’ DevOps practices are negatively impacted by the number of manual activities required to achieve test automation and test coverage. They also have trouble identifying the root causes of issues because they lack visibility into the necessary quality artifacts. “There are four pillars you need to be successful with continuous testing: Creation, Execution, Lab, and Analysis, or CETA as we call it,” said Kinsbruner. “When you can accomplish that all in one place, software quality and delivery speed increase.” #1: Creation – DevOps teams should be able to create robust and maintainable test automation that is agnostic to the skillsets

in the organization. With Perfecto, anyone on the team can create automated tests. #2: Execution – Test execution should be orchestrated and able to scale across platforms, browsers and devices, as well as in burst mode. “When you have an important feature you want to release, and you want to identify issues in a very short amount of time or you have a patch for a large system you want to validate quickly, you need instant access to the high number of platforms you’re executing against,” said Kinsbruner. “People who want to test continuously and scale up or down dynamically need a solid execution layer. Perfecto gives you that.” #3: Lab – Where tests are executed matters. If in-house, hardware, software and browser-related issues arise because each target device manufacturer has its own associated DevOps cycle. Emulators and Simulators don’t help. Perfecto provides a cloud-based lab; there’s no software to install or test environments to set up and maintain. Its lab is also more secure than an in-house environment because test data doesn’t get lost, and there’s no possibility of a side-channel security issue, such as from a smartphone to a desktop. #4: Analysis – Making sense of all the test data that can be generated at scale is difficult without the right tools. Some Perfecto customers generate more than a million scripts in a full regression cycle — far too many for even a brilliant human to analyze. Perfecto uses machine learning to automatically surface the issues and related artifacts as necessary to resolve the issue. Perfecto also provides users with single-screen access to builds and the tests associated with them, a heat map of issues relating to specific platforms and pass/fail data based on the platforms that were executed. If developers want to drill down into a single test case, they get full visibility into all associated test artifacts including log files, crash reports, HTTP archives, screen shots and video recordings. “We use machine learning to filter the noise out of the test data, so engineers only have to deal with actual failures,” said Kinsbruner.

LOWER BUSINESS RISKS Software value and brand image are now synonymous, which is why organizations across industries are racing to deliver apps and features that differentiate them from their competitors. However, they often make tradeoffs between speed and quality that cause customers to complain or churn. Perfecto helps ensure that the user experiences DevOps teams intend to deliver to customers are delivered to customers. For more information, visit www.perfecto.io. i


Your Continuous Testing Checklist. CLOUD-BASED LAB REAL DEVICES & DESKTOP BROWSERS CODELESS & CODE-BASED SCRIPTING SCALABLE EXECUTIONS FOR WEB & MOBILE SMART REPORTING & ANALYTICS Perfecto checks all your boxes.

Try it today!


Featured Companies n Eggplant helps organizations put users at the center of software testing to create amazing digital experiences that drive user adoption, conversion, and retention. Our Digital Automation Intelligence Suite interacts with software exactly like a real user to test the true user experience, and auto-generates tests at the UI and API level for greater productivity. Eggplant solutions enable customers to test the full user experience, including performance and usability, managing the test environment and orchestrating large-scale test execution, and generating predictive analytics to understand the impact of a change on users across a wide range of operating systems and platforms.

n Mobile Labs remains the leading supplier of in-house mobile device clouds that connect remote, shared devices to Global 2000 mobile web, gaming, and app engineering teams. Its patented GigaFox is offered on-premises or hosted, and solves mobile device sharing and management challenges during development, debugging, manual testing, and automated testing. A pre-installed and pre-configured Appium server provides “instant on” Appium test automation.

n Parasoft’s software testing tool suite automates time-consuming testing tasks for developers and testers, and helps managers and team leaders pinpoint priorities. With solutions that are easy to use, adopt, and scale, Parasoft’s software testing tools fit right into your existing toolchain and shrink testing time with next-level efficiency, augmented with AI. Parasoft users are able to succeed in today’s most strategic development initiatives, to capture new growth opportunities and meet the growing expectations of consumer demands.

n Perfecto offers a cloud-based continuous testing platform that takes mobile and web testing to the next level. It features a continuous quality lab with smart self-healing capabilities; test authoring, management, validations and debugging of even advanced and hard-to-test businesses scenarios; text execution simulations; and smart analysis. For mobile testing, users can test against more than 3,000 real devices, and web developers can boost their test portfolio with cross-browser testing in the cloud. n ProdPerfect fully automates the development and maintenance of browser-level testing using live user data. ProdPerfect analyzes your web traffic to create aggregated flows of common user behavior, which we build into an end-to-end testing suite that we maintain and expand over time, which kicks off automatically from CI.

n Tricentis is recognized by both Forrester and Gartner as a leader in software test automation, functional testing, and continuous testing. Our integrated software testing solution, Tricentis Tosca, provides a unique Model-based Test Automation and Test Case Design approach to functional test automation – encompassing risk-based testing, test data management and provisioning, service virtualization, API testing and more.

n Applause is the worldwide leader in digital quality and crowdtesting. Software is at the heart of how all brands engage users, and digital experiences must work flawlessly everywhere. With highly-vetted testers available on-demand around the globe, Applause provides brands with a full suite of testing and feedback capabilities.

n Applitools is on a mission to help test automation, DevOps, and software engineering teams release mobile and web apps that are visually perfect. We provide the only commercial-grade, visual AI-based test cloud that instantly validates any application’s user interface in a fully automated manner, across all customer engagement points and digital platforms – using our groundbreaking image-processing stack, developed from scratch in-house. n AutonomIQ can discover, ingest, and transform English language artifacts into immediately executable, shareable and manageable Test Scripts. Using deep-learning and AI algorithms, AutonomIQ detects natural language documents and changes, automates and enables self-healing, and provides advanced diagnostics. In real world situations, AutonomIQ has been shown to provide ~90% improvement in speed and quality compared to existing tools and techniques.

n CA, a Broadcom Company: CA offers next-generation, integrated continuous testing solutions that automate the most difficult testing activities – from requirements engineering through test design automation, service virtualization and intelligent orchestration. Built on end-to-end integrations and open source, Broadcom’s comprehensive solutions help organizations eliminate testing bottlenecks impacting their DevOps and continuous delivery practices to test at the speed of agile, and build better apps, faster. n Froglogic is well-known for its automated testing suite Squish with its flagship product Squish GUI Tester, the market-leading automated testing tool for GUI applications based on a wide variety of languages, operating systems and web browsers. In addition, froglogic offers the professional, crossplatform C, C++, C# and Tcl code analysis tool Coco Code Coverage.

n Functionalize is a cloud-based autonomous testing solution uses AI and ML to provide intelligent test automation. Our Adaptive Language Processing (ALP) converts test plans written in plain English into fully functional test scripts. It can even use the output of your test management system. With autonomous testing, you now have an intelligent test agent (ITA) supercharging the work of your test and DevOps teams. This ITA is the perfect regression tester – focused, tireless, and driven, but still intelligent. Functionalize turns testing into a competitive advantage when it matters the most – getting to market faster while ensuring higher customer satisfaction. n HPE Software’s automated testing solutions simplify software testing within fast-moving agile teams and for Con-


tinuous Integration scenarios. Integrated with DevOps tools and ALM solutions, HPE automated testing solutions keep quality at the center of today’s modern applications and hybrid infrastructures.

n IBM: Quality is essential and the combination of automated testing and service virtualization from IBM Rational Test Workbench allows teams to assess their software throughout their delivery lifecycle. IBM has a market leading solution for the continuous testing of end-toend scenarios covering mobile, cloud, cognitive, mainframe and more.

n mabl is the most reliable codeless UI testing service available. mabl enables continuous testing with an auto-healing automation framework and maintenance-free test infrastructure. mabl advances traditional UI testing using proprietary machine learning models to automatically identify application issues, including JavaScript errors, visual regressions, broken links, increased latency, and more.

n Micro Focus is a leading global enterprise software company with a world-class testing portfolio that helps customers accelerate their application delivery and ensure quality and security at every stage of the application lifecycle – from the first backlog item to the user experience in production. Simplifying functional, mobile, performance and application security within fast-moving Agile teams and for DevOps, Micro Focus testing solutions keep quality at the center of today’s modern applications and hybrid infrastructures with an integrated end-toend application lifecycle management solution that is built for any methodology, technology and delivery model.

n Microsoft provides a specialized tool set for testers that delivers an integrated experience starting from agile planning to test and release management, on-premises or in the cloud.

n Now Secure is the mobile app security software company trusted by the world’s most demanding organizations. Only the NowSecure Platform delivers fully automated mobile app security and privacy testing with the speed, accuracy, and efficiency necessary for Agile and DevSecOps environments. Through the industry’s most advanced static, dynamic, behavioral and interactive mobile app security testing on real Android and iOS devices, NowSecure identifies the broadest array of security threats,

compliance gaps and privacy issues in custom-developed, commercial, and business-critical mobile apps. NowSecure customers can choose automated software on-premises or in the cloud, expert professional penetration testing and managed services, or a combination of all as needed. NowSecure offers the fastest path to deeper mobile app security and privacy testing and certification.

n Orasi is a leading provider of software testing services, utilizing test management, test automation, enterprise testing, Continuous Delivery, monitoring, and mobile testing technology. n Progress: Telerik Test Studio is a testautomation solution that helps teams be more efficient in functional, performance and load testing, improving test coverage and reducing the number of bugs that slip into production.

n QASymphony’s qTest is a Test Case Management solution that integrates with popular development tools. QASymphony offers qTest eXplorer for teams doing exploratory testing. n QMetry is a leader in test management, test automation, and test analytics products. QMetry Intelligent Digital Quality Platform is designed for Agile & DevOps teams to build, manage & deploy quality software faster & better. QMetry has the complete agile testing solution with test management, automation, & powerful quality analytics for digital enterprises.

n Rogue Wave is the largest independent provider of cross-platform software development tools and embedded components in the world. Rogue Wave Software’s Klocwork boosts software security and creates more reliable software. With Klocwork, analyze static code on-the-fly, simplify peer code reviews, and extend the life of complex software. Thousands of customers, including the biggest brands in the automotive, mobile device, consumer electronics, medical technologies, telecom, military and aerospace sectors, make Klocwork part of their software development process.

n Sauce Labs provides the world’s largest cloud-based platform for the continuous testing of web and mobile applications. Founded by the original creator of Selenium, Sauce Labs helps companies accelerate software development cycles, improve application quality, and deploy with confidence across hundreds of browser /

OS platforms, including Windows, Linux, iOS, Android & Mac OS X. Optimized for Continuous integration (CI), Continuous delivery (CD), and DevOps, the Sauce Labs platform is built to handle the most secure data from its customers.

n SmartBear provides a range of frictionless tools to help testers and developers deliver robust test automation strategies. With powerful test planning, test creation, test data management, test execution, and test environment solutions, SmartBear is paving the way for teams to deliver automated quality at both the UI and API layer. SmartBear automation tools ensure functional, performance, and security correctness within your deployment process, integrating with tools like Jenkins, TeamCity, and more.

n SOASTA’s Digital Performance Management (DPM) Platform enables measurement, testing and improvement of digital performance. It includes five technologies: mPulse real user monitoring (RUM); the CloudTest platform for continuous load testing; TouchTest mobile functional test automation; Digital Operation Center (DOC) for a unified view of contextual intelligence accessible from any device; and Data Science Workbench, simplifying analysis of current and historical web and mobile user performance data.

n Synopsys: Through its Software Integrity platform, Synopsys provides a comprehensive suite of testing solutions for rapidly finding and fixing critical security vulnerabilities, quality defects, and compliance issues throughout the SDLC. n TechExcel: DevTest is a sophisticated quality-management solution used by development and QA teams of all sizes to manage every aspect of their testing processes.

n TestRigor is an automated regression testing tool that allows VPs of Engineering and Directors of QA improve test coverage to 100%, speed up testing schedules by at least four weeks, and increase team productivity by up to 210% – all for less than their entire outsourced QA department. i


Turn static files into dynamic content formats.

Create a flipbook
Issuu converts static files into: digital portfolios, online yearbooks, online catalogs, digital photo albums and more. Sign up and create your flipbook.