FC_SDT046.qxp_Layout 1 3/26/21 5:38 PM Page 1
APRIL 2021 • VOL. 2, ISSUE 46 • $9.95 • www.sdtimes.com
003_SDT045.qxp_Layout 1 3/1/21 10:16 AM Page 2
Instantly Search Terabytes
www.sdtimes.com EDITORIAL EDITOR-IN-CHIEF David Rubinstein drubinstein@d2emerge.com NEWS EDITOR Christina Cardoza ccardoza@d2emerge.com
dtSearch’s document filters support: popular file types emails with multilevel attachments a wide variety of databases
SOCIAL MEDIA AND ONLINE EDITORS Jenna Sargent jsargent@d2emerge.com Jakub Lewkowicz jlwekowicz@d2emerge.com ART DIRECTOR Mara Leonardi mleonardi@d2emerge.com
web data
CONTRIBUTING WRITERS Jacqueline Emigh, Lisa Morgan, Jeffrey Schwartz, George Tillmann
2YHU VHDUFK RSWLRQV LQFOXGLQJ efficient multithreaded search
CONTRIBUTING ANALYSTS Enderle Group, Gartner, IDC, Intellyx
HDV\ PXOWLFRORU KLW KLJKOLJKWLQJ forensics options like credit card search
CUSTOMER SERVICE SUBSCRIPTIONS subscriptions@d2emerge.com ADVERTISING TRAFFIC Mara Leonardi mleonardi@d2emerge.com
Developers: 6'.V IRU :LQGRZV /LQX[ PDF26
LIST SERVICES Jessica Carroll jcarroll@d2emerge.com
&URVV SODWIRUP $3,V IRU & -DYD DQG NET with NET Standard / 1(7 &RUH
REPRINTS reprints@d2emerge.com
.
.
.
)$4V RQ IDFHWHG VHDUFK JUDQXODU GDWD FODVVLILFDWLRQ $]XUH $:6 DQG PRUH
ACCOUNTING accounting@d2emerge.com
ADVERTISING SALES PUBLISHER David Lyman 978-465-2351 dlyman@d2emerge.com
Visit dtSearch.com for KXQGUHGV RI UHYLHZV DQG FDVH VWXGLHV IXOO\ IXQFWLRQDO HQWHUSULVH DQG developer evaluations
SALES MANAGER Jon Sawyer 603-547-7695 jsawyer@d2emerge.com
The Smart Choice for Text Retrieval® since 1991
dtSearch.com 1-800-IT-FINDS
PRESIDENT & CEO David Lyman
D2 EMERGE LLC www.d2emerge.com
CHIEF OPERATING OFFICER David Rubinstein
003_SDT046.qxp_Layout 1 3/26/21 1:56 PM Page 3
Contents
VOLUME 2, ISSUE 46 • APRIL 2021
FEATURES
NEWS 4
News Watch
9
BizOps Coalition Fuels Digital Success
18
Observability: A process change, not a set of tools
DevOps Institute creates capability assessment model
page 6
COLUMNS 52
GUEST VIEW by Thomas Richter OLAP + OLTP = ...PostgreSQL?
53
ANALYST VIEW by Jason English Shift testing left, but bank right
54
How you organize your development teams matters
INDUSTRY WATCH by David Rubinstein Internet crime complaints rise
BUYERS GUIDE page 12
Automated testing is a must in CI/CD pipelines
Developers reflect on challenges, feelings about remote work
page 21
page 13
Software Development Times (ISSN 1528-1965) is published 12 times per year by D2 Emerge LLC, 2 Roberts Lane, Newburyport, MA 01950. Periodicals postage paid at Plainview, NY, and additional offices. SD Times is a registered trademark of D2 Emerge LLC. All contents © 2021 D2 Emerge LLC. All rights reserved. The price of a one-year subscription is US$179 for subscribers in the U.S., $189 in Canada, $229 elsewhere. POSTMASTER: Send address changes to SD Times, 2 Roberts Lane, Newburyport, MA 01950. SD Times subscriber services may be reached at subscriptions@d2emerge.com.
004,5_SDT046.qxp_Layout 1 3/26/21 11:15 AM Page 4
4
SD Times
April 2021
www.sdtimes.com
NEWS WATCH Richard Stallman returns to FSF Richard Stallman, the founder and former president of the Free Software Foundation (FSF), has announced he is returning to the foundation as a board member. Stallman founded the FSF in 1985 and was the acting president until about 18 months ago when he resigned over comments he made regarding Jeffrey Epstein and his alleged victims.
At the time, Stallman said he was resigning over “a series of misunderstandings and mischaracterizations” regarding comments he made regarding allegations that AI pioneer Marvin Minsky had sexually assaulted one of the victims in the Jeffrey Epstein scandal. The announcement was made in a talk at FSF’s annual LibrePlanet. “I’m now on Free Software Foundation’s board of directors once again,” he said. “Some of you will be happy at this, and some might be disappointed, but who knows.
People on the move
n Codefresh has tapped Dan Garfield as its new chief open source officer. Garfield helped launch Codefresh in 2016 and has previously held vice president of marketing and chief technology evangelist positions at the company. In his new role, he will help realign the company as an open-source company. n Mike Hanley has become GitHub’s first chief security officer. Hanley was previously the CISO at Cisco, and led the security program at Duo Security. At GitHub, he will play an integral role in expanding the company’s cybersecurity initiatives. n Justin Mclean is joining Instaclustr as its vice president of training services. Mclean is currently a board member at the Apache Software Foundation. At Instaclustr, he will lead the company’s training and develop education assets around its open-source, data-later technologies such as Apache Cassandra, Apache Kafka, Apache Spark, Kibana, Elasticsearch and Redis. n SmartBear has added Vineeta Puranik as its new senior vice president of engineering. Previously, Puranik was vice president of engineering and operations at Veracode. At SmartBear, she will work toward strengthening the company’s product portfolio, which spans test automation, API design, collaboration, performance testing, and test management. n The Defense Advanced Research Projects Agency (DARPA) appointed Stefani Tompkins as its 23rd director. Tompkins is a former military intelligence officer and previously worked with DARPA from 2007 to 2017 as a program manager and deputy director of the Strategic Technology Office, DARPA chief of staff, and director of the Defense Sciences Office. She returns to DARPA after spending time as vice president for research and technology transfer at Colorado School of Mines.
In any case, that is how it is. And I am not planning to resign a second time.”
ServiceNow platform updated with low-code tooling ServiceNow rolled out the Quebec release of its Now Platform, introducing Creator Workflows and App Engine Studio and Templates to provide organizations with low-code tooling and expanded AI capabilities to facilitate the speed and agility they need to speed their digital transformations. Creator Workflows, the company said in its announcement, was designed to enable workers at any technical level to quickly build the applications they need. Creator Workflows joins ServiceNow’s existing IT, Employee and Customer Workflow portfolios. Josh Kahn, SVP of Creator Workflow products at ServiceNow, told SD Times, “Our founder’s original vision was to allow everyday people to route work around the organization and we started by offering App Engine capabilities in 2003. Fast forward to 2021 and businesses today are faced with unrelenting pressure to digitize and transform business processes, partly in response to increased market competition and also in response to unanticipated situations like the COVID-19 pandemic.”
Report: Go critical to business success The 2020 Go survey results found that 76% of respondents are using Go at work, and 66% stated it’s becoming critical to their company’s success. Additionally, 91% of respondents would like to use Go for
their next new project, and 89% find it is working well for their team. The results are based on 9,648 developer responses. “Understanding developers’ experiences and challenges helps us measure our progress and directs the future of Go,” the team wrote in a post. The language’s top features, according to respondents, include build speed, reliability, using concurrency, CPU usage and editor support. VS Code is the preferred editor with GoLand following closely. Respondents are also having success with its cloud services, debugging and modules.
TypeScript 4.2 released Microsoft has announced the official release of TypeScript 4.2, the latest version of its JavaScript superset programming language. There are a number of new features in this release, including smarter type alias preservation, leading/middle rest elements in tuple types, and stricter checks for the “in” operator. Type aliases can be used to avoid having to repeat functions that work for a number of different types, such as strings, numbers, and booleans. An issue arises when TypeScript tries to normalize union types into a flattened union type, which results in information loss, thus forcing the typechecker to search for every combination of types to see what type aliases might have been used. This release improves the internals around type aliases to keep track of how types were constructed and enable types to be printed the way they were used in the code.
004,5_SDT046.qxp_Layout 1 3/26/21 11:15 AM Page 5
www.sdtimes.com
Dart 2.12 features sound null safety The latest version of the programming language Dart is now available. Dart 2.12 includes stable versions of sound null safety and Dart FFI. Null safety strengthens the language’s type system, and will enable developers to more easily catch null errors during development. Previously, it was difficult to tell the difference between code that anticipated being passed as null versus code that didn’t work with nulls. This resulted in a lot of bugs passing through, even after rigorous code reviews, and possibly going unnoticed for years, in some cases.
Google to fund Linux security developers Google and the Linux Foundation have announced plans to maintain and improve Linux’s long-term security. As part of the plan, the organizations will prioritize funds to underwrite long-time Linux kernel maintainers Gustavo Silva and Nathan Chancellor as full-time developers focused on Linux kernel security development. This decision follows a survey by the Linux Foundation’s Open Source Security Foundation (OpenSSF) and the Laboratory for Innovation Science at Harvard (LISH), which found a need for additional security work on the Linux operating system.
Google Jetpack Compose UI now in beta Google released its new UI toolkit Jetpack Compose,
April 2021
SD Times
Value Stream Management group launches A new association has been formed to help advance the adoption of value stream management (VSM). The Value Stream Management Consortium (VSMC) was established by Digital.ai, HCL Software, Plutora, ServiceNow and Tasktop. “At this stage in the development of value stream management it’s important that we create and foster a community of experts to further VSM practices, innovation, and adoption,” said Helen Beal, chair of the Value Stream Management Consortium. “By creating this community, we will increase and accelerate the use of VSM, while developing and instilling best practices and standards. And, ultimately, just like the practice of VSM itself, we will help deliver the utmost value to industry practitioners.” designed to make it easier to build native apps across all Android platforms. According to the team, Compose offers modern, declarative Kotlin APIs and is built to integrate with existing Android apps and Jetpack libraries. Google’s product manager Anna-Chiara Bellini and developer relations team member Nick Butcher explained that the beta release of Compose API currently has all the features necessary to build production-ready apps and that the release is API stable.
Mobile native dev gets new foundation The Linux Foundation has announced the formation of the Mobile Native Foundation (MNF) with a mission to foster collaboration and improve processes and technologies. MNF is a place where developers can collaborate on open source software, standards and best practices. The foundation hopes to enhance large-scale Android and iOS apps as well as provide common UI frameworks, architectural patterns, build systems and networking
stacks. The foundation is currently supported by Airbnb, Capital One, Corellium, Elotl, Flare.build, GitHub, GogoApps, Haystack, Line, LinkedIn, Lyft, Microsoft, Peloton, Robinhood, Sauce Labs, Screenplay.dev, Slack, Solid Software, Spotify, Square and Uber.
Checkmarx’s new KICS solution In an effort to better secure cloud-native apps, software security company Checkmarx has launched a new opensource static analysis solution. The new Keeping Infrastructure as Code Secure (KICS) solution enables developers to write secure infrastructure as code (IaC) by automatically detecting issues from the start. According to the company, as organizations move to the cloud they are utilizing IaC to provision infrastructure faster and provide scalability. However, developers are struggling to manage IaC’s security, compliance and configuration risks. KICS aims to address this by automatically detecting issues, hard-coded keys, passwords, compliance issues, and misconfigurations.
Microsoft open sources low-code language Microsoft has announced a new open-source programming language designed for business users and professional developers. Power Fx is designed to be a low-code language that anyone can use. It is based on Microsoft Excel and uses already familiar formulas, opening the language up to a broad range of users and skill sets, according to the company. “With Power Fx, we can amplify the impact of developers by many multiples over the same time horizon. By offering citizen developers a familiar and approachable way to express logic, we’re dramatically expanding who can build sophisticated solutions. By delivering Power Fx with the tools a professional expects, including the ability to directly edit apps in text editors like Visual Studio Code and use source control, we’re making it possible for developers to go faster and find common ground with millions of makers,” Ryan Cunningham, director PM of Power Apps at Microsoft, wrote in a post. z
5
006,7_SDT046.qxp_Layout 1 3/26/21 11:17 AM Page 6
6
SD Times
April 2021
www.sdtimes.com
Observability: BY JENNA SARGENT
I
f you do a Google search for the phrase “observability tools,” it’ll return about 3.3 million results. As observability is the hot thing right now, every vendor is trying to get aboard the observability train. But observability is not as simple as buying a tool; it’s more of a process change — a way of collecting data and using that data to provide better customer experiences. “Right now there’s a lot of buzz around observability, observability tools, but it’s not just the tool,” said Mehdi Daoudi, CEO of digital experience monitoring platform Catchpoint. “That’s the key message. It’s really about how can we combine all of these data streams to try to paint a picture.” If you go back to where observability came from — like many other processes, it originated at Google — its original definition was about measuring “how well internal states of a system can be inferred from knowledge of its external outputs,” said Daoudi. Daoudi shared an example of observability in action where one of Catchpoint’s customers was seeing a trend where customers complained a lot on Mondays and Tuesdays, but not on Sundays. The server load was the same, but the services were slower. Through observability, the company was able to determine that backup processes that only run on weekdays were the culprit and were impacting performance. “Observability is about triangulation,” said Daoudi. “It’s about being able to answer a very, very complex question, very, very quickly. There is a problem where is the problem? The reason why this is important is because things have gotten a lot more complex. You’re not dealing with one server anymore, you’re dealing with hundreds of thousands of servers, cloud, CDNs, a lot of moving parts where each one of them can break. And so not having observability
A process change, not a set of tools into the state of those systems, that makes your triangulation efforts a lot harder, and therefore longer, and therefore has an impact on the end users and your brand and revenue, etc.” This is why Daoudi firmly believes that observability isn’t just a set of tools. He sees it as a way of working as a company, being aligned, and being able to have a common way to collect data that is needed to answer questions. The industry has standardardized on OpenTelemetry as the common way of collecting telemetry data. OpenTelemetry is an open source tool used for gathering metrics, logs, and traces — often referred to as the three pillars of observability. The three pillars are often referenced in the industry when talking about observability, but Ben Sigelman, CEO and co-founder of monitoring company Lightstep, believes that observability needs to go beyond met-
rics, logs, and traces. He compared the three pillars to Steve Jobs announcing the first iPhone back in 2007. Jobs started off the presentation by announcing a widescreen iPod with touch controls, a “revolutionary” mobile phone, and a breakthrough internet communications device, making it seem as though they were three separate devices. “These are not three separate devices,” Jobs went on to clarify. “This is one device, and we are calling it iPhone.” Sigelman said the same is true of telemetry. Metrics, logs, and traces shouldn’t be known as the three pillars because you get all three at once and it’s one thing: telemetry. Michael Fisher, group product manager at AIOps company OpsRamp, broke observability data down further into two signals: symptomatic signals and causal signals. Symptomatic signals are what an end user is experiencing, such as page latency or a 500 Internal Server
006,7_SDT046.qxp_Layout 1 3/26/21 11:18 AM Page 7
www.sdtimes.com
Error on a website. Causal signals are what cause those symptomatic signals. Examples include CPU, network, and storage metrics, and “things that may be an issue, but you’re not sure because they’re not being tied to any symptom that an end user might be facing.” Monitoring tools tend to focus mostly on the causal signals, Fisher explained, but he recommends starting with symptomatic signals and working towards causal signals, with the end state being a unit of the two. “When something is going wrong [the developer] can search that log, they can search that trace and they can tie it back to the piece of code that’s having an issue,” said Fisher. “The operations team, they may just see the causal symptoms, or maybe there is no causal symptom. Maybe the application is running fine but users are still complaining. Tying those two together is kind of a key part of this shift towards observability. And that’s why I talk about observability as a development principle because I think starting with the symptomatic signals with the people who actually know is a huge paradigm shift for me because I think some of the people you talk to or ITOps teams you talk to is that monitoring is their wheelhouse, whereas many modern shops, OpsRamp included, much more monitoring actually happens on the development team side now.” Providing good end user experience is the ultimate goal of observability. With monitoring, you might only be focusing on those causal signals, which might mean you miss out on important symptomatic signals where the end user is experiencing some sort of service degradation or trouble accessing your application. “When I talk about using observability to drive end user outcomes, I’m really
talking about focusing on observing the things that would impact end users and taking action on them before they do because traditionally this focus on monitoring has been at a much lower level, layer 3; I care about my network, I care about my switches,” said Fisher. “I’ve talked to customers where that’s all they care about, which is fine but you start to realize those things really matter less once you move up the stack and you have a webpage or you have a SaaS application. The end user will never tell you that their CPU is high, but they will tell you that your webpage is taking 10 seconds to load and they couldn’t use your tool. If an end user can’t use your tool who gives a damn about anything else?” It’s important that observability not just stay in the hands of developers. In fact, Bernd Greifeneder, CTO of monitoring company Dynatrace, believes that if developers just do observability on their own, then it’s nothing more than a debugging tool. “The reason then for DevOps and SREs needs to come into play is to help with a more consistent approach because these days multiple teams create different microservices that are interconnected and have to interplay. This is sort of a complexity challenge and also a scale challenge that needs to be solved. This is where an SRE and Ops team have to help with standing up proper observability tooling or monitoring if you will, but making sure that all the observability data comes together in a holistic view,” he said. SRE and Ops teams can help make sure that the observability data that the developers are collecting has the proper analytics on top of it. This will enable them to gain insights from observability data and use those insights to drive automation and further investments into observability. “IT automation means higher availability, it means automatic remediation when services fail, and ultimately means better experiences for customers,” Greifeneder said. When looking into the tools to put on top of your observability data to do those analytics, Tyler McMullen, CTO of edge cloud platform Fastly recommends constantly experimenting to see what works for your team. He explained
April 2021
SD Times
that often these observability vendors charge a lot of money, and teams might fall into the trap of buying a solution, putting too much observability data into it, and being shocked when they’re charged a lot of money to do so. “Are the pieces of information that we’re plugging into our observability, are they actually working for us? If they’re not working for us, we definitely shouldn’t have them in there,” said McMullen. “On the other hand, you only really find out whether or not something is useful after it becomes useful. Figuring out what you need in advance is I think, one of the biggest problems with this thing. You don’t want to put too much in. On the other hand, if you put too little in you don’t know whether or not it is useful.” As a result, your team will need to do lots of experimenting to discover the right process and the right balance. Daoudi added that it’s also important to answer the question of why you’re doing observability before looking into products. “Like every new thing that when a company goes and decides to implement something, you start with why? Why do you need to implement observability? Why do you need to implement SREs? Why do you need to implement an HR system? If you don’t define the ‘why’ then what typically happens is first it’s a huge distraction to your company and also a lot of resources being wasted and then the end result might not be what you're looking for,” he said. And of course, it’s important to remember that observability is more of a process, so looking for a tool that will do observability for you won’t work. The tooling is really about analytics on the observability data you’ve gathered. “I really don’t think observability is a tool,” said Daoudi. “If there was such a thing as go to Best Buy, aisle 5, or Target, or Walmart and buy an observability tool for like $5 million, it ain’t going to work because if your company is not functioning and aligned, and your processes and everything isn’t aligned around what observability is supposed to do, then you’re just going to have shelfware in your company.” z
7
009,10_SDT046.qxp_Layout 1 3/26/21 5:46 PM Page 9
www.sdtimes.com
April 2021
SD Times
INDUSTRY SPOTLIGHT
BizOps Coalition Fuels Digital Success O
rganizations have learned the hard way that software development efforts don’t always reflect business priorities or result in the expected ROI. Agile, DevOps and CI/CD efforts have accelerated software delivery, but not necessarily the delivery of value. BizOps bridges the gap by providing visibility and insights across the entire value stream so companies can achieve the levels of agility and resilience that today’s business environment requires.
Introducing the BizOps Coalition
Digital Transformation and The Pandemic Necessitated BizOps Digital transformation is the direct result of digital disruption. Every industry has been upended by cloud-native companies that have changed the rules of the game. To survive and grow, the incumbent organizations replace waterfall ways of working with cross-functional agility so they can adapt dynamically to the accelerating pace of change happening in the global economy, among customers and within their own organizations. “The need for BizOps is forcing changes to organizations. I see tremendous progress being made to break down silos with the creation of leadership roles
The BizOps Coalition is a group of visionary tech and software thought leaders who are advancing the concept of BizOps. BizOps is an approach for optimizing software development to achieve business objectives. It utilizes data to provide common insights throughout the value chain. Like Agile and DevOps, BizOps doesn’t just happen. It’s a conscious endeavor that requires enterprise-wide change management, executive The thought leaders behind the BizOps Coalition meet. support, a value chain management solution and a framework that like the Chief Digital Officer,” said Lauenables successful implementation. reen Knudsen, author of “Modern BusiRealizing this, the BizOps Coalition ness Management” and co-author of the created the BizOps Manifesto which is BizOps Manifesto. “These new leaders a declaration of principles and values help create transformative processes and that helps software development and data models through the entire business operations better meet the needs of a including Finance, HR, Sales, Marketdigital business through a combination ing, Business Operations, and Legal. In of technology, culture and communica- addition to technology, they’re teaching tion. The four values underpinning the new ways of rapidly achieving outcomes 14 principles are: by reducing bottlenecks, creating faster • Business outcomes over individual feedback loops, and tightly aligning team products and proxy metrics efforts to customer needs.” Though many organizations have • Trust and collaboration over individbeen executing their digital transformaualism and hierarchy • Data-driven decisions over opinions, tion strategy for several years, even the most agile companies have been judgments and persuasion • Learning and pivoting over a rigid plan reshaped by the pandemic’s effects.
Before the pandemic hit, digital transformation was occurring in a slower, more intentional fashion than in 2020. When the pandemic hit, IT needed to help their organizations become digital overnight. The companies capable of adapting quickly to change fared better than competitors. In fact, BizOps was the secret of many of 2020’s “winners.”
Dual-Speed Enterprises Are Struggling In many companies, Agile and DevOps practices are mature, but their adoption across the enterprise is nascent. The result is that Agile and DevOps teams have accelerated their own work but work still flows in a traditional fashion from the business. “You’ve got organizations betting big on their Agile and DevOps transformations but when you look under the hood, they’re delivering value slower than they were before,” said Mik Kersten, founder and CEO of Tasktop and BizOps Manifesto co-author. “They’re adding developers, but those developers are increasingly frustrated and unproductive.” The organizations that have embraced BizOps have changed the way their business operates and they’re releasing software faster than their competitors. Another challenge is that software teams are under pressure to reduce the backlog and create new products simultaneously. However, they don’t have the time or resources to do both, even if the teams are practicing Agile, DevOps and CI/CD. “Enterprises move markets. Frameworks move organizations. You have to take these frameworks and apply them in small places so they can grow and scale,” said Patrick Tickle, chief product officer at portfolio and work management solutions provider Planview and continued on page 10 >
9
009,10_SDT046.qxp_Layout 1 3/26/21 5:46 PM Page 10
10
SD Times
April 2021
www.sdtimes.com
< continued from page 9
BizOps Manifesto co-author. “I’ve seen success patterns in terms of how you architect a value chain for product innovation and the customer. It’s not about aligning silos. It’s about restructuring the organization around value stream flow.” The reason some organizations take a waterfall approach to digital transformation is because their culture or change management practices haven’t changed. “Companies like SpaceX are leaping over businesses that have been around for decades, but there are some exceptions,” said Knudsen. “We’ve seen Disney pivoting in the last year and making up the revenue they desperately needed. In fact, Disney was the first to introduce a streaming service with a plus on the end. Now everyone is trying to jump on that bandwagon.” One symptom of digital disruption is the need to deliver value faster. In many organizations this has been a catalyst for Agile, DevOps and CI/CD adoption. However, faster software delivery has resulted in bimodal organizations in which the business and developers are moving at different speeds. Any organization undergoing digital transformation cannot afford bimodal operations. The business and developers need to work together as a cohesive team that understands how to define, measure and deliver value.
BizOps’ Impact Planview realized early on that BizOps wasn’t just about how to rebuild the company’s value proposition for its customers but more fundamentally how the company operates. Its first value stream focused on the company’s go-tomarket capability which required the involvement of sales and marketing, followed by a customer value stream and a product value stream. “It feels daunting and big, but there are a few small ingredients — like having a value stream mentality, implementing incremental planning, and having a visual planning environment to support the process — that if you actually commit to them, you can drive an immense amount of change very quickContent provided by SD Times and
ly,” said Tickle. Organizational leaders may be surprised by some BizOps impacts, especially when the data suggests the executive’s hypothesis is faulty. For example, Tasktop’s Kersten used to require every line of code to be reviewed and unit tested before it advanced to production. Then tracking flow metrics and visualizing bottlenecks challenged his thinking. “We had a culture of 100% code reviews because that I had put in place and never questions because I thought it was a best practice,” said Kersten. “Then
It’s not about aligning silos. It’s about restructuring the organization around value stream flow. —Patrick Tickle
an advisor of mine pointed out the bottleneck, suggested making code review voluntary measuring the results in timers of impact on quality and on flow.” The proposition seemed a bit risky, but his engineering leadership assured Kersten that the company was good at managing quality, so if the quality went down, that fact would be immediately obvious. In the end, software quality did not drop and feature velocity “accelerated like crazy.”
Getting to Value The definition of “value” can be subjective. However, companies cannot afford to assume that its perception of value and the customer’s perception of value are one. “Users are saying, ‘Make it easy for me. That’s the value you’re creating for me,” said Knudsen. “Value is a tricky
word to use because most companies interpret that as adding more features, when really, what the customer wants is ‘simplify this for me.’” One of the challenges organizations face when attempting to improve value delivery is adopting a BizOps mindset but failing to have an operating model that supports it adequately. To succeed with BizOps, a company’s operating model must change and the company must be able to measure its value streams. However, in many organizations, software and software portfolios have become so complex, it’s difficult to know whether a specific activity yields results. BizOps and Value Stream Management enable businesses to understand and measure value flow. Without that capability, businesses can spend millions of dollars on the wrong things. For example, a Broadcom customer spent over $100 million annually rolling up team-level data which had likely been sanitized twice before any of the business leaders saw it. Given the accelerating velocity of business, organizations can’t compete on that type of data because it costs too much and data may be stale by the time it’s used. “Data must be the natural outcome of the processes we create,” said Knudsen. “All departments need to understand how to read and respond to the data together. You can’t just optimize engineering or DevOps and expect positive business outcomes. You have to optimize the entire value stream and flow from idea to outcome.” BizOps combines data, technology and process so organizations can finally understand whether the value they think they’re providing has merit. They can also better understand how value flows and the obstacles to value flow. “The value stream becomes the fundamental construct to think about across the organization. It creates a shared perspective of outcomes,” said Planview’s Tickle. “You’re optimizing across the organization, not optimizing within a function. Putting the value stream operating model in place forces a cultural shift.” z
012-17_SDT046.qxp_Layout 1 3/26/21 1:57 PM Page 12
12
How you organize your development teams matters
SD Times
April 2021
www.sdtimes.com
BY CHRISTINA CARDOZA
A
mong the roles played by development managers is to serve as the middle man between the business and developers. They have the hard task of facilitating the needs and wants of the business and end users through their development projects. How they set up, organize and empower their teams can result in the success or failure of a project, solution, feature or even overall business. “We have business objectives and we need people to fulfill them. The goal of an organization design is to group the skills necessary to deliver value in the most efficient way for the business overall,” said Jeremiah Lee, an engineering manager at InVision, a digital product design, workflow and collaboration company. Add COVID-19 and remote work into the mix, and things get even more difficult for development managers.
Assembling your development teams There are different ways to organize and bring developers together on a project and work toward the business’
success. For instance, there is the traditional development team structure or a squad approach. A squad is a term that came from audio streaming company Spotify’s engineering culture. According to Spotify, it is a “small cross-functional, selforganizing team usually less than eight people” that “sit together and have endto-end responsibility for the stuff they build, design, commit, deploy, maintenance, operations, the whole thing.” The company explained they started as a Scrum company, but once they started to introduce more development teams within the organization, it found the Scrum practices were getting in the way. “We decided that Agile matters more than Scrum, and Agile principles matter more than any specific practice,” the company said in a video. Typically, a squad includes a squad leader that acts as a lead developer or Agile coach, and six to eight developers who are split up into pairs. There are also support squad roles such as a designer, product owner, and application architect who come in for a period
of time to help with the overall design or UX, explained Chinh Vo, CTO and practice leader for IBM Garage.
The good and bad of squads According to Ravi Lachhman, an evangelist at the CI/CD software company Harness, “squads are more organized around a problem set to deliver functionality on and members of a squad can rotate in and out much more frequently than changing teams. Squads are typically problem- or functionality-based, bringing together several members from different teams to solve and deliver on a particular problem or build specific functionality. When those goals are accomplished, squad members can realign to different squads while continuing to grow their skill sets.” Squads can, however, have a negative effect on the human aspect of development, according to Liran Haimovitch, CTO and co-founder of debugging company Rookout. He explained that in a squad model, developers are required to change teams frequently and work with different people, which can hinder the
012-17_SDT046.qxp_Layout 1 3/26/21 2:01 PM Page 13
www.sdtimes.com
April 2021
SD Times
13
Developers reflect on challenges, feelings about remote work BY JENNA SARGENT any companies have just surpassed the one-year anniversary of sending their employees home to work remotely as a safety measure for COVID-19. At the time, many thought this might be a temporary situation and folks would return to the office after a month or so, but one year later, many workers haven’t returned to the office. At the start, some developers struggled with remote work, while others thrived. Initial struggles included setting up and getting used to a distributed environment for the first time, feeling isolated from coworkers, and balancing work and home life — especially for those with young children when normal childcare options weren’t there or they had to help their kids with remote schooling alongside working their normal job. Benefits included the ones normally associated with working from home: increased productivity, more free time due to not having a commute, and the convenience of not having to go anywhere. One year later, the benefits might have remained the same, but the negatives have compounded themselves for some. Those feeling isolated from coworkers at the start of the pandemic are now dealing with the mental toll of
M
development project because developers have to take time to get to know one another and how they will work together. However, Harness’ Lachhman looks at changing teams as a good thing because the only way he says a software engineer can grow and expand their skill sets is to experience and embrace new challenges and changes. “Squads can regroup and spin up and down quicker than changes in the team/management structure, allowing for individuals to solve problems they are passionate about and the squad can focus on the goals set forth,” he said. “Modern teams should focus on skills and domains of the ‘bench strength’ of the development organization and then allow development members to work on different projects/squads. This allows for development members to rotate around and work on fresh problems while building domain skills in the firm/organization.” Lachhman explained that in order for a squad to work, there needs to be a strong platform engineering culture in place. “The biggest part of the learning continued on page 14 >
having been isolated not only from coworkers for a full year, but also from family and friends. “A couple days in a month or a week, no problem, but forever? Well, that just requires a lot more intention from yourself, your team, and your coworkers,” said Anthony Tran, software engineer at Rollbar, a company that provides a continuous improvement platform. In fact, a survey released by Harness in August — 5 months into remote working — revealed that 12% of developers were less happy in their roles than they were pre-pandemic. There are some who either didn’t like or struggled with working from home at the start, but have changed opinions over time as they’ve gotten more used to it and experimented and figured out things that worked for them. “During the beginning of the pandemic it was a struggle to stay motivated at home, there were so many distractions that it made it difficult work,” said Tyler Corwin, a developer at digital marketing company Figmints. “I was still able to hit all of my deadlines, but I didn’t get the same drive to get things done as I did while I was still in the office. After the first month things got
“It has been awesome to get as much time as I have had with my child at such a young age.” —Daniel Valdivia
continued on page 16 >
012-17_SDT046.qxp_Layout 1 3/26/21 1:57 PM Page 14
14
SD Times
April 2021
www.sdtimes.com
< continued from page 13
curve when switching team to team is that no one ever deploys software the same way,” he said. “Common engineering efficiency tools such as those that enable the CI/CD pipeline and similar confidence-building steps teams take to reach production are crucial for squads to work. IBM’s Vo recommends the following best practices: implementing pair programming, rotating programming pairs to spread knowledge, having standup meetings, and co-locating squads. According to Mark Cruth, enterprise solutions architect at the software development company Atlassian, the best practices engineering managers can take from the Spotify model are: • Not to copy the model, but to try to understand the structure, practices
and mindsets, and then tailor it to fit your own organization and needs. • Autonomy and trust to empower teams to pick their own tools and make their own decisions • Transparency with community by building trust, being transparent, providing inclusive ways to gain feedback and aligning with how your organization wants to work • Encourage mistakes to constantly be learning and improving “Although it may look like a matrix organization, the key cultural elements of the model need to be in place to allow the structure to thrive, such as trust and autonomy. If an organization doesn’t shift its behaviors (and ultimately its culture), the benefits of the Spotify model will never be realized. If you simply rename teams to Squads,
Top five challenges development managers are facing today Debugging company Rookout surveyed developer and DevOps managers in the cloud-native space to get a better view of how increased pressure to digitally transform is impacting their ability to form and maintain successful teams. According to the report, the five challenges managers are struggling with are: 1. Maintaining productivity and velocity. As more development moves toward microservices and serverless structures, development teams are struggling to make technologies visible and troubleshoot their applications. “The complexity they bring demands a steep learning curve, and one of their main value points — namely, the abstraction they provide between the application code and the hardware the application is running on — also raises new challenges when attempting to troubleshoot the application without access to the hardware they run on,” the report started. 2. Resolving customer issues in a timely manner. Engineering teams that are not properly equipped to handle customer issues become apathetic to problems. They need to clearly put a face and a name to an issue, see how much the issue is impacting the bottom line, and align with the business. “Once engineers are equipped to help the customer, they start seeing benefits to the business and end customers. It is becoming much more motivating to help resolve those issues and ties the loop back in to focus, understand and align with the business,” said Liran Haimovitch, CTO and co-founder of Rookout. 3. Balancing speed and quality. When working in a distributed development environment, it can be hard for developers to balance velocity and quality. Leaders are measuring development on their ability to meet deadlines, make customers happy, and find as many bugs as fast as possible. 4. Teamwork and collaboration in a distributed environment. Teams are having trouble debugging and developing efficient code in a distributed environment. They need tools where they can collaborate, have access to the same information, and share knowledge. 5. Remote debugging: Traditional debuggers don’t work in a distributed environment. “The solution is adopting the proper tool: a modern debugging solution that’s built for cloud-native applications. Developers will be able to get the data they need from their code, no matter where it’s running, and while their code is running live,” the report stated. z
you’re just putting lipstick on a pig,” Cruth wrote in a guide about squads. InVision’s Lee believes Spotify’s squad model falls short of its promises. He explained while Spotify’s idea of squads was to basically have teams working as autonomous mini-startups with all the skills necessary to do their job without having to rely on another team, matrix management solved the wrong problem, he said. It was too fixated on team autonomy, collaboration was an assumed competency, and mythology became difficult to change, he said.
Stream-aligned teams Instead, Lee recommends a streamaligned team approach, which is an evolution of full-stack, multi-discipline product engineering teams or Agile feature teams. “Stream-aligned teams perform better than teams organized by discipline because coordination effort is reduced to coordination within a single team instead of across team boundaries,” he explained. IBM’s Vo finds that while ideally you want to have full-stack team members, there really aren’t any true full-stack developers. They either have more expertise in the front end or back end of software development. He explained forming teams into squads helps because you can pair different types of programmers together to get them more up to speed on the technology or in an area where they may not be as strong. Stephen Deasy, head of cloud engineering at Atlassian, explained that what engineering managers should really be focused on is organizing teams around customer impact. “Our first frame of reference is always we should organize the team to be most effective to deliver that value and how you organize changes,” he said. “What we found was being able to connect to the end user and really understand the domain and problem space the end user is having, the team can really develop a deep understanding in there and build really good solutions.” The way you organize and execute that will be different depending on your project or team size, but Atlassian
012-17_SDT046.qxp_Layout 1 3/26/21 1:58 PM Page 15
www.sdtimes.com
believes in the triad model where you look at the development, market and operational aspects of a solution. “We think of products as really being the what and the why, building that roadmap and understanding the domain, what outcomes we are trying to drive. Then, the engineering teams are trying to drive how we will build it, owning the schedules, delivery, execution and operations,” he said. As far as how big the team should be, generally Deasy says you want to make sure you can control and understand the growth of a team at that size. “Really trying to connect the team to the what and why, building a balanced team so they are set up for success with skills, level and size, and try to get out of the way and let teams win,” he added. As you start to grow the amount of developers you have, you will have to revisit how you organize and execute your teams. Deasy explained what works for say 40 engineers may not work once you scale to 200+ engineers. “The most optimal combination of team types changes as the organization size changes and the business needs change. Many organizations today start with several stream-aligned teams and expand to include the other types as the organization grows” Lee said.
Lessons learned working in a remote environment Traditionally, you would form a development team and they’d have their own physical space to work together, but the COVID-19 pandemic really forced development teams to rethink how and where they work. A year later, many businesses are actually seeing increased productivity and happier employee work-life balance as a result of working remotely that they are starting to consider a remote-first approach. It wasn’t an easy year, but organizations have started to work through the kinks and find what really works for their business in a remote world. One of the biggest disadvantages development teams faced in the pandemic and in the new remote world was the camaraderie that came with being co-located, according to Harness’
April 2021
SD Times
Trello’s rules for a remote work culture The collaboration tool provider Trello had mostly been working remote before the pandemic hit, and has provided a few tips and tricks on how others can successfully be a productive and collaborative remote company. The top five rules for creating a remote culture include: 1. Empathy is everything: Always assume positive intent. Things can easily be misread and misinterpreted over chat. In order not to misunderstand intent, always assume your colleague is being positive. 2. Treat others with transparency: Important information should be accessible to everyone. 3. Asynchronous is A-Okay: It’s okay to have communication across distributed time zones and work schedules. If you plan ahead, things should run smoothly. 4. Expect structure: There should be a standard process, structure and agenda associated with meetings and updates so everyone, no matter their location, are on the same page. 5. Different yet equal: Everyone is going to have different experiences, but all members are equal. “When you think of a vibrant, self-sustaining culture, you might not think of rules. But in this case, rules are social norms that provide participants with an expected experience when they enter the proverbial office. The special thing about norms is that they are collectively agreed upon. With 100% buy-in, these rules build trust, understanding, and support,” the company wrote in a post. Trello’s 6 rules for teams and individuals working remote 1. Assume remote: If one person on your team is remote, the entire team should “assume remote,” meaning you take meetings at your desk and make sure all information from the meeting is written down and shared. If everyone on the team is remote, it’s even more important that you digitally share information. 2. Have a dedicated office space with a door that closes: Have a designated area that’s set up for work. It creates a mental space to focus. 3. Have the tools to do your job: Every team and individual needs a digital toolset and a strong Internet connection. Toolsets should include project management, collaboration, documents sharing and virtual meetings capabilities. 4. Communicate, communicate, communicate: According to the company, “people need to hear a message seven times before they’ll internalize it. If you feel like you’re over-communicating information, you’re probably communicating just the right amount.” 5. Schedule face time: If possible, meet and bond in person to foster human relationships. 6. Teams must have time overlap: If you are working in different time zones, have a shared time where everyone is available to collaborate or meet. z
Lachhman. Managers have the extra task in a remote setting to make team building and skill-sharing a priority. “Setting up co-working time, virtual lunches and happy hours are also important to foster ongoing collaboration,” he said. What Lachhman found was that while, even when employees were still in the office they would typically “ping” or message someone before getting up and going over to their desk to ask for help. “You can still collaborate this way virtually and instead of walking over to someone physically, you can start a screen share,” he explained. According to InVision’s Lee, the suc-
cess or failure of a team in a remote environment has more to do with team interaction than team structure. “Working remotely means embracing asynchronous work. There are a million ways to collaborate and almost as many tools to help. When you are remote, you can’t look around the office and infer norms. You also can’t get everyone in a room at the same time to build consensus every time. You have to find another way of getting alignment,” he said. “Companies have to increase intentionality to work effectively remotely. That means creating and explicitly communicating an continued on page 17 >
15
012-17_SDT046.qxp_Layout 1 3/26/21 2:02 PM Page 16
16
SD Times
April 2021
www.sdtimes.com
< continued from page 13
much better as my time management and organization got better.” For example, one thing he started doing was creating “fallback” tasks that he could work on while he waited on answers from his teammates on Slack or email. “This kept me working more efficiently and it's something that I’ll continue to do even after we resume work back at the office,” Corwin said. Corwin added that at the start he struggled with motivation, communication with team members, and keeping his kids from running into his workspace. And now that the vaccine is here, he finds himself not wanting to return to the office five days a week. Maxime Basque, a developer at Unito, said that working remotely has been more good than bad. “While I do miss the camaraderie and things like being able to just ask something to someone directly without going the async route, as a generally anxious person I feel a lot calmer these days; not wasting 1h+ in transport every day, being able to concentrate with no distractions when I need to, having almost full control over my schedule, not having to think about lunch, etc. Eliminating the small things that caused a lot of stress were really beneficial for me,” he said. Daniel Valdivia, an engineer at Kubernetes-native object storage company MinIO, appreciated the extra time
he was able to spend with his family. “As the father of a 2-year-old, it has been awesome to get as much time as I have had with my child at such a young age.” Sachin Goyal, a principal engineer at Rollbar, also has had mostly positive experiences with working remotely. “I was able to use my time much more efficiently. Cutting down commute, lunch, and room-hopping is a huge time saver. Apart from that, I spent much more time with my 2-year-old and my wife,” he said. The one complaint he has, like many, is not being able to see colleagues regularly.
Rico Pamplin maintains a healthy work/life balance by scheduling activities that require him to leave his workspace.
Flexible schedules a plus Goyal feels that his team and manager have been very accommodating throughout this time. For example, since his daughter’s daycare is closed, he and his wife plan their day and meetings around making sure one of them is always with their daughter, and his company allowed him to have a more flexible schedule. “The ability to work at flexible hours is a huge benefit for me. Open communication was really helpful. Clearly stating the accommodations I wanted from my team and my manager and working with them to create a win-win was actually a ‘win’ for all us,” Goyal said. Tran also noted that his managers have put in a lot of effort in trying to make remote work a positive experience, such as having lunch meetings on working efficiently and ergonomically,
Tyler Corwin found that communication between team members was challenging in the beginning.
Zoom hangouts with trivia, group yoga, or playing whatever the latest popular Internet game was. “Also, I’d like to emphasize being candid with my managers and coworkers at Rollbar and sharing that I was losing motivation and focus, and feeling distant from the company and team was very helpful because they related that this was a common symptom of working remote and being able to share that, we were able to put more events/meetings/activities in place to help mitigate this feeling,” Tran said. Rico Pamplin, a lead process engineer at Lincoln Financial Group, also sees positive steps being taken by management to ensure employees are doing okay. “My manager also heavily promotes maintaining a healthy work/life balance and we have scheduled 1:1 sessions to ensure our professional requirements aren't overstepping the personal ones.” He said that one way he ensures he’s maintaining his work/life balance is scheduling activities that require him to leave his workspace, because otherwise he’s found himself with days where he’s gotten super focused on a project and then suddenly realized it was 10 p.m. As more people get vaccinated, many companies are in the process of discussing what that means for future plans, whether that means fully reopen-
012-17_SDT046.qxp_Layout 1 3/26/21 1:58 PM Page 17
www.sdtimes.com
ing offices, staying fully remote, or adopting a hybrid model. Valdivia said that for most of his career he’s been in a physical office and preferred it—because he doesn’t feel that the collaborative process of problem solving on a whiteboard translates to Zoom meetings, and in-person conversations can help build relationships that advance your career—but now has begun to rethink his views and see the value in a hybrid model. “I think it can recharge you, allow for deep work and add a few hours a week of family time without negatively impacting your productivity or the culture.” Basque said his company, Unito, will be adopting a hybrid model once the pandemic ends, where employees will be able to work from home two to three days per week. “The company believes this will allow us to maintain our strong culture, foster collaboration, but also adapt to the new reality and new needs of the team.” Pamplin also sees the value in a hybrid model. “Now that I've been remote for a while, the luster has worn off a bit, but I definitely wouldn’t want to go back to primarily working in an office. I don't mind the cubicle setting occasionally, but to do my job effectively it's not a necessity, especially given that most of what I do is virtual, and my team is geographically distributed.” z
April 2021
SD Times
home to work. “We believe the distrib< continued from page 15 agreed upon set of practices and invest- uted manner in which we are working ing in people being able to self-serve now has a lot of advantages. Diversity, their way to success.” improvement, engagement and flexibilRemote camaraderie, team build- ity for people to work in the ways they ing, and human connection really are most productive while still connectrequires good video and microphone ing to their team,” said Deasy. setups so people can put a face to a Rookout’s Haimovitch found that the name, look at each other, and see their pandemic really highlighted the imporemotions, according to Rookout’s tance of independent engineers. “It is Haimovitch. “Once you move to asyn- even more important that engineering chronous communications, you can managers provide team members with reduce costs and have more flexibility the right tools and guidance to allow on who and how you hire,” he said. them to work efficiently and independOne of the things Atlassian tried to ently,” he explained. “You need to make implement to keep the human connec- sure your engineers can do whatever tion was to try and organize a social they need to do alone by themselves event every week. However, the compa- because that is how they are going to ny found that this was challenging in the spend their day. You can’t rely or allow beginning because not everyone was silo knowledge or lack of privileges. available at the same time and it was Give them the tools to deal with their taxing for them to have to add another own day-to-day tasks independently.” meeting on their calendar. The compaIBM Garbage’s squads worked on a ny found that replacing a standup meet- co-located model in a physical space ing with a social event and consolidating before the pandemic because they found the number of meetings really helped it provided a faster turnaround for misalleviate some stress of sion-critical development their team members. projects. When the panThe pandemic Managers also really demic hit, the company need to listen and learn has highlighted had to move to more virtual from their team mem- the importance ways of working. According bers, according to Atlass- of independent to IBM’s Vo, video was key ian’s Deasy. Some ways to making sure there was to gain feedback is to run engineers. still that personal connecregular polls at the team tion. The challenge in the and individual level to find out what is beginning was dealing with people’s working, what is not working, and how Internet connection and bandwidth. the organization can help. It also adopted new collaboration Deasy also found that the company tools to maintain interaction and had to reallocate or increase develop- engagement from all participants. ment team budgets so they could get Vo doesn’t expect to change much of their remote offices properly set up. the work environment post-pandemic, Since the company found that every unless it’s a request from a customer. individual has different requests and “There will still be a combination of needs, the company gave managers the going onsite with customers and teams anatomy to overwrite budgets and work coming into the office working together, out what was needed for each individ- but it’s not a requirement anymore,” ual case. “It’s hard to come up with a said Vo. “It actually opened up the possingle global approach to this because sibility for us to form more efficient the individual situations are so different squads. Instead of restricting us to one and it changes over time,” he said. office location, now we can have people When offices start opening back up, from the West Coast, East Coast, midAtlassian plans to retain office spaces, dle of the country, and get the right type but use them more for collaborative of folks to be more efficient or more workspaces or as an option for those specialized to execute on the project. It who need a physical place other than gives us a little more flexibility.” z
17
018_SDT046.qxp_Layout 1 3/26/21 11:17 AM Page 18
18
SD Times
April 2021
www.sdtimes.com
DEVOPS WATCH
DevOps Institute creates capability assessment model BY DAVID RUBINSTEIN
The DevOps Institute has announced an Assessment of DevOps Capabilities (ADOC) driven by a vendor-neutral crowdsource effort designed to help people measure their organization’s state of DevOps against other organizations. According to the Institute’s announcement, the assessment model looks at five dimensions of DevOps: the human aspects, process and frameworks, functional composition, intelligent automation and technology ecosystems. “The Humans of DevOps are in urgent need of an empirical model that supports their goals to improve organizational performance through the practice of DevOps principles,” Jayne Groll, CEO of DevOps Institute, said in the announcement. “They
want to be able to measure their progress as their capabilities increase during their DevOps journey and know where to invest their time and energy. Our vendor-agnostic model empowers teams and enterprises to grasp what DevOps means to them
and accelerate progress. We are pleased to make this available to the market.” The new ADOC is available at team and enterprise levels, with the Institute saying it has already enrolled 10 enterprise partners for the expanded level. The DevOps Institute’s Chief of
In other DevOps News n DevSecOps company Bridgecrew announced it is shifting cloud security left with a new solution that notifies developers about infrastructure as code (IaC) misconfigurations and policy violations early in the life cycle and directly inside their IDEs. The extension combines policies from Bridgecrew’s opensource tool Checkov with in-line fixes enabled by Bridgecrew’s APIs. In addition, it supports all of the major IaC frameworks such as Terraform, CloudFormation, Kubernetes manifests, serverless framework, and Azure Resource Manager (ARM). n Copado has acquires DevSecOps company New Context, a provider of multi-cloud security services to large enterprise, infrastructure and government cloud customers. According to the company, the acquisition will expand the Copado platform to enable enterprises to make quality, compliance and security more prevalent in their DevOps practices. n European DevOps company Eficode is teaming up with Tasktop to help businesses solve software delivery problems. As part of the partnership, Eficode will join Tasktop’s Flow Part-
Research Eveline Oerhlich and Chief Ambassador Helen Beal contributed to the effort to create the model, which assesses an organization’s people, process and technology capabilities. It allows organizations to baseline their current state and offers guidance to achieve the next target state while allowing continuous assessment at both team and enterprise levels, and promotes DevOps principles and good implementation practices, according to the announcement. Already, the Institute is looking at enhancements to the ADOC, including in the areas of site reliability engineering and DevSecOps, and will offer a bolt-on to the ADOC for the government market. The Team ADOC is an online version for assessing a team’s current DevOps capabilities. ADOC for Enterprises, an expanded version to assess capabilities across teams, is available in a pay-per-use model starting at $15,000 for up to 500 participants, the Institute announced. z
ner Program to help boost DevOps transformation services and value stream management. n IBM recently announced two new capabilities to help developers deliver intelligent application analysis throughout the DevOps pipeline. IBM Application Discovery and Delivery Intelligence (ADDI) for IBM Z V was designed to help developers accelerate application development and provide insight into their business-critical application estate. IBM Wazi Developer V1.2 features a new analyzed capability designed to help developers discover and analyze relationships between components of their z/OS apps and understand the impact of potential changes. n xMatters has announced new capabilities designed to help teams respond faster to incidents. According to the company, its data-driven DevOps approach helps DevOps, SRE and operations teams collaborate through the xMatters Incident Console, Slack, Microsoft Teams and Zoom. Other updates include a new “Incidents by Severity” widget, and new capabilities in its messaging user interfaces so teams can communicate better and react quickly to time-sensitive issues. z
Make Ideas Real.
Low-code. All devices. No limits.
Visual, collaborative application development in the cloud.
With Reify’s Hybrid Development model, you can combine traditional IDE develepment with visual development, in any mix.
Product Managers Business Analysts Designers Developers
Developers [optional]
Use Reify low-code platform for every project, no jut the simple ones! • Modernize exting applications • Extend existing applications • Visually build new applications
Reify. This changes everything. GET STARTED @ Reify.com
www.sdtimes.com
April 2021
SD Times
Buyers Guide
Automated testing is a must in CI/CD pipelines BY JAKUB LEWKOWICZ
A
s the software development industry has seen unprecedented levels of digital transformation, the demand for automated testing in the CI/CD pipeline has taken on greater urgency, especially at the early stages. Also, new advancements in AI are helping developers with some of the biggest challenges in testing: test creation, maintenance and many of the manual tasks. Many companies have noticed, and are spending more on their automated testing initiatives. Strong testing practices have gotten to be so important that they’ve become “the main differentiator between companies that are successful and those that aren’t,” according to Guy Arieli, the QA CTO at Digital.ai. When companies are out looking for an automated testing solution, they’re primarily looking for one that will increase the quality of their releases,
increase the speed at which they can be done, and the one that’s most cost-efficient. A common approach among enterprise customers is to seek out a vendor that satisfies the majority of their needs while integrating into their CI/CD pipelines. “We see customers want a unified solution. You don’t want to be using disparate tools for end-to -end testing of different types of clients,” said Dan Belcher, the co-founder of mabl. “Increasingly they’re pushing us to add value to those end-to-end tests with insight around things like performance and visual correctness and other kinds of attributes of quality, because they’re trying to move from pure quality assurance like ‘did I break this core feature?’ to quality engineering: Is the feature better than it was before? Is it faster? Is it more accessible? Is it visually appealing?” Chris Haggan, the product management lead at HCL OneTest, said it’s more than just getting the solution with
the most features. It’s also about supporting users with the tools that they already have and seeing if it’s a right fit with the overall approach the development organization is taking. Another issue is whether the organization has enough resources to deal with adding testing solutions to the mix since they can add complexity.
Where to start? To start with their automated initiatives, organizations need to build quality into the application earlier, as quality has become a core functional necessity and testing early on is a key part of that. “We see people all the time that want to fully automate everything in weeks. Yes of course that is technically possible but it takes time to evaluate what’s important to test, how solutions fit into your CI/CD chain and who generates test data and so on,” according to Kevin Surace, the CTO and co-founder of Appvance. “While no one wants to hear continued on page 22 >
21
22
SD Times
April 2021
www.sdtimes.com
< continued from page 21
it, the best automation strategy is one that is laid out over a year,” Surace added. Building in quality comes down to both the culture of the organization and in executing deep code analysis, as well as deep reliability and security at the earliest stages. “If you put it at the end, you really can’t kind of accelerate your delivery. You’re always kind of running
into a bottleneck at the end of the process,” said Mark Lambert, the vice president of Strategic Initiatives at Parasoft. This has led to continuous quality and continuous compliance as aspects that need to be tested in the CI/CD pipeline. Mabl’s Belcher said that as more expansive automated testing becomes available, his one concern is that it will create a test sprawl in which it’s so easy to create end-to-end tests and get the coverage that you want in place that perhaps teams will become more complacent about testing. “Just because it’s easy doesn’t mean it’s right,” Belcher said. “They have to put more thought into, you know, are these tests accomplishing the objectives that I set out? Are we doing only what is necessary? Are we thinking about the data? Do we have the right environments? And there’s a lot, a lot more than just the capability to add lots of requests. We keep score by the quality of what’s in production.” Organizations also need to prioritize
those tests that need to be automated first to avoid getting overwhelmed. “What I want to achieve is not more and more tests. What I actually want is as few tests as I possibly can because that will minimize the maintenance effort, and still get the kind of risk coverage that I’m looking for,” said Gartner senior director Joachim Herschmann, who is on the App Design and Development team.
In the past, what used to happen is that the organization used to say “we’ll recruit the developer and we’ll do the R&D and then when we need to test it, we’ll send it to India and then it will be tested there,” but now organizations realize that this has to be at the core of your R&D organization, Digital.ai’s Arieli explained. Now as developers are starting to get more and more involved in quality, the notion of building quality as part of the application started to take hold. “So developers have to think about how do I engender unit testing and more and more of it and when you reach the total extreme of it and you’re totally mature, they start thinking of automation,” said Anand Sundaram, the SVP of Products, UI, Device Cloud and Performance Testing at SmartBear Software.
Security and performance testing After quality, there are other aspects of the application for which automated testing can be leveraged: security and performance testing of your APIs and
microservices at the developer level before everything comes in for integration testing or the entire application comes together. “We’ve accepted that test automation is valuable, deep code analysis is valuable and now we’re actually starting to say the same thing around security; how can we embed security in each stage so that we can build security into the pipeline,” Parasoft’s Lambert said. Now, testers are trying to apply the same methods that they used for testing quality to security. So that means deep code analysis to identify potential runtime exceptions that could go uncaught. As they’re moving up the stack, they’re looking to leverage unit testing for fuzzing of the underlying code and seeing how they can utilize API tests for API security testing. Developers can start to build quality and security by taking advantage of those earliest-stage validation techniques, Lambert explained. While automated testing has received widespread recognition as a must for today’s software development environments, there are many challenges that organizations face when trying to set up effective testing strategies in their CI/CD pipelines. At the bottom of the testing pyramid, the struggle with unit testing is that there isn’t a lot of visibility and it’s difficult to understand how much it actually covers. On top of that is the service component testing usually driven by an API. At the top of the pyramid is system and UI testing, which can be the most challenging. Implementing all of these levels of testing can be a challenge, especially for legacy systems, since these aspects of testing were not initially accounted for when the applications were created, Digital.ai’s Arieli added. Another challenge in implementing automated testing is finding the staff with the appropriate skill set. Testing complex enterprise applications requires business domain expertise. Also, maintaining test scripts makes it difficult to achieve continuous test automation — as automation requires continued on page 25 >
Digital.ai Continuous Testing for mobile & web applications Digital.ai Continuous Testing (formerly Experitest) enables enterprises to increase release velocity while providing their customers with satisfying, error-free experiences across all devices and browsers. Digital.ai Continuous Testing seamlessly integrates with best-in-class tools throughout the DevOps pipeline and allows organizations to scale testing coverage without compromising web or mobile app quality. Accelerate release cycles, reduce risk, and deliver world-class experiences to all users, with Digital.ai. Learn more at www.digital.ai Agile Planning
DevOps
Application Security
Continuous Testing
AI-Powered Analytics
Intelligent Test Automation for Agile Teams TODAY, software development teams across the globe are facing the challenge of delivering high-quality web applications while keeping pace with business and customer demands. The risk of releasing bugs into production, impeded product velocity, and a diminished customer experience is too great. Built for CI/CD, mabl integrates automated end-to-end testing into the entire development lifecycle. Creating, executing, and maintaining reliable tests has never been easier. With mabl, teams can: Easily create automated UI tests - and save on test maintenance with the help of AI
Create automated end-to-end tests through the UI, capturing a true end-user perspective
Increase test coverage across applications and browsers with a single platform
Gain actionable insights from mabl’s rich application data for quicker issue resolution
Integrate directly into your workflow, with platforms such as GitHub, Bitbucket, Jira, and Gitlab
90%
3x
40%
Increase in test coverage
Faster test creation
Fewer bugs in production
Modern software development needs a modern testing solution. Try mabl free to see how easy it is to start testing. START YOUR FREE TRIAL:
mabl.com/trial-registration
www.sdtimes.com
< continued from page 23
teams to ensure that testing doesn’t become a bottleneck. Therefore, tests must be designed in a way that minimizes disruption to the continuous testing process. The goal is for test automation teams to build robust and reusable test scripts that don’t require constant attention and maintenance, according to Clinton Sprauve, the director of Product Marketing at Tricentis. Organizations also need to find a way to manage and track test automation efforts across multiple tools through observability and analytics. “There is a challenge to testing in the sense that we need to do it more frequently, we need to do it for more complex applications, and we need to do it at a higher scale. This is not feasible without automation, so test automation is a must,” Gartner’s Herschmann said.
AI and observability in automated testing With value being a core tenet of DevOps, managers have to be able to see how each decision impacts the user experience, the revenue and entire business performance as a whole. This is why testing providers are looking to create more intelligent means of testing that can provide analytics. Intelligent testing can be a combination of data analytics, smart heuristics and algorithms, machine learning and anything that analyzes data in real time and makes decisions or recommendations that then help solve the problem. Developers can then use that need to have instant feedback of where exactly the problem occurred and move much more quickly. Observability is needed in the pipeline because it gives testers a clue as to where exactly the problem is, when the problem occurred and then alerts the tester. In addition to observability, automated testing solutions have also created ways to help developers with many of the pain points around testing and to speed up the process. “At the beginning of Agile, when you start talking about quarterly releases, you could still kind of fake it, right? You could still handle quality. You would have minimal amount of time to do all
of your regression testing and so forth, but you could build that into a schedule and make it work. When you move to CI/CD where change is continuous and disruptive you need to find new solutions,” mabl’s Belcher said. “And so for a few years, as an industry, we turned to, well, let’s just make us another thing that the developers have to worry about and have them write tests that do endto-end validation. But the problem with
that is that those tests relied on stability of the very thing that was changing constantly.” “Now we realize well, maybe actually you don’t need these scripts and you can use the power of cloud computing and data analysis and machine learning and AI to make it so that it’s really simple to create the tests and then rely on the system to adapt to the change automatically rather than people needing to go in and update scripts every time you make a small change,” Belcher added. The infusion of AI into these automated testing solutions has helped around aspects such as checking on quality, test maintenance and figuring out how to create the tests. When you go from version one to version two, AI can help by having a system update itself and carry on without involving the developers having to go in and fix a load of things. Also, machine learning becomes particularly important around performance testing and performance test result analysis to extract information from huge amounts of data and then help the users understand where there’s a performance problem and how to correlate that to some of the metrics that one gets from observability tools for example, HCL’s Haggan explained. And the infusion of AI won’t mean
April 2021
SD Times
that QA and dev teams get replaced, but rather their work will be augmented to work in tandem with more advanced tooling. AI can also relieve them of the majority of script writing and maintenance as a machine literally creates thousands of tests in minutes. “But the impact is profound. I’d say in virtually every case over years now, AI tests found critical bugs that the standard manual or automated tests would have never found,” Surace said. Another big trend in the automated testing space is around low code and codeless capabilities so that domain experts can build their desktop automation and know what goals they are trying to achieve with them. Automation solutions before were very developer-centric, but vendors now are seeking to democratize capabilities to others in an organization and also to companies that don’t have the personnel or resources to do the largescale shift left methodologies that were invented in organization like a Google, Facebook, Amazon where there are unlimited resources, according to Digital.ai’s Arieli.
Next: API and mobile testing Parasoft’s Lambert said there is increased interest in testing in the API layer for a few reasons. One is that API tests are quicker to run, and setting them up to be continuous tests at the API level rather than the UI level is easier and there’s less maintenance associated with it. API tests can be run more efficient because you don’t have to have all the browsers and you can execute in parallel. Another reason is that they’re easier to debug and diagnose because they’re closer to the code. Also, it’s easier for developers to reexecute those tests within their environment and it becomes a great communication mechanism between the test role and the developer role. This new adoption for end-to-end testing is in the API space both for companies that offer APIs as products or for companies that have integrated API-based services into their applicacontinued on page 31 >
25
021-35_SDT046.qxp_Layout 1 3/26/21 6:18 PM Page 26
26
SD Times
April 2021
www.sdtimes.com
How does your company help customers with their automated testing initiatives? Kevin Surace, CEO and co-founder of Appvance Appvance makes a platform called Appvance IQ, or AIQ for short. The platform is all-encompassing — web, API and native mobile, functional, compatibility, performance, load, security tests. It becomes a centerpiece of your quality initiative. We break test creation into two buckets. • Low code/no code ML-driven Test Designer • AI-based Autonomous Testing TEST DESIGNER – In Test Designer, you have a world class rapid script creator. It creates scripts at UX and API levels for every user flow. And, its compatible with every major UI library like React and Angular. We see people create base-level scripts their first day 20X faster than writing in Selenium. Test Designer alone garners 300% productivity improvement across the QA effort (dev or QA engineers). AUTONOMOUS TESTING – AI-based Autonomous Testing is 4 years old and augments specific use cases. You simply train an AI engine to act in certain ways with your web or mobile apps. Once it has learned what is important to you, it builds a baseline of your application and then on each new build it will look for bugs, differences, issues, failed validations. It is data driven, or it creates its own data, generating thousands of tests by itself in minutes. In addition, it’s able to simulate the flows of real user activities. Everyone who is using this says it’s a game changer for quality. Find up to 10X more bugs with 98% less effort.
Guy Arieli, QA CTO, Digital.ai Digital.ai Continuous Testing (formerly Experitest) enables organizations to increase release velocity while providing their customers with satisfying, error-free experiences across all devices and browsers. With Digital.ai Continuous Testing, users can test their mobile apps remotely from their browsers across 2,000+ real iOS and Android devices, emulators, and simulators hosted in Digital.ai’s global data centers. Manual testing features full device control, and large-scale automated testing is easily created and run using these cloud-based devices.
Automated and live cross-browser testing capabilities are offered for testing web applications remotely with secure manual interactions. Perform large-scale parallel test execution across real desktop browsers of any type and version. Digital.ai Continuous Testing also seamlessly integrates with best-in-class tools throughout the DevOps pipeline. The hassles around managing resources like Appium, Selenium, XCUI, Espresso, and Cyprus are removed, and your QA and testing teams can work comfortably and efficiently using the tools they are already most familiar with. Once your web or mobile app is fully developed, Digital.ai’s Performance Monitoring tool helps you analyze performance by simulating different servers, measuring transaction duration, and speed index. Digital.ai’s Accessibility Testing Cloud features real devices and browsers with full voice, talkback, and gesture support to help ensure that you deliver accessible web and application experiences for people with disabilities. Using the Appium integration, you can even automate your accessibility testing for faster compliance with all international web accessibility standards. Finally, Digital.ai Test Analytics comes with a complete, consolidated view of the test execution results using advanced testing analytics with AI. Cloud managers can then use the customized dashboards to improve the test automation quality and ensure that scripts are stable. Learn more about how Digital.ai helps make digital transformation deliver business value with automated testing and more at www.digital.ai
Chris Haggan, Product Management Lead, HCL OneTest HCL OneTest supports a DevOps testing approach with UI testing, API testing, performance testing, data fabrication, and service virtualization. The solution is designed to automate and run tests early and more frequently to discover errors faster. HCL OneTest helps with the connections and dependencies between services and components to help plan integration test continued on page 28 >
Zephyr
TestComplete
ReadyAPI
CrossBrowserTesting + more
021-35_SDT046.qxp_Layout 1 3/26/21 6:18 PM Page 28
28
SD Times
April 2021
www.sdtimes.com
< continued from page 26 strategies. With features like system modelling providing the overall visibility of the system under test architectures to help derive more comprehensive and cohesive tests. Covering the complete test landscape, from mainframe to mobile, HCL OneTest also includes HCL OneTest Embedded for testing microcontrollers and validating standards conformance, e.g., MISRA-C. Recent additions to the HCL OneTest platform include cloudnative technologies that offer users a solution, which is both secure and offers discoverability of tests to enable simple re-use and collaboration. As an open platform, HCL OneTest enables users to bring existing open-source tests e.g. Postman, JMeter, into a single execution environment, retaining the investment in open-source tests, whilst adding value with HCL OneTest’s robust reporting and integrated script management. As part of HCL Software DevOps, HCL OneTest supports a DevOps deployment life cycle through a wide range of integrations. With the increase in value stream management focus for many clients, being able to collaborate with all parts of the delivery life cycle through HCL Accelerate provides the complete transparency teams need.
Dan Belcher, Co-founder at mabl At mabl, we’re focused on solving an essential challenge: enabling software teams to innovate quickly while meeting high customer expectations for quality. In other words - to build useful things faster with fewer mistakes. Mabl is the simplest, most capable intelligent test automation solution on the market that’s designed to give software testers a centralized platform for endto-end testing Mabl’s low-code interface for test creation and maintenance requires up to 80% less effort than alternatives, improving collaboration and reducing the programming expertise required to write and maintain automated tests. Our auto-healing capabilities harness the power of AI and machine learning to automatically detect changes throughout the UI and update tests accordingly, significantly reducing the burden of test maintenance. The mabl desktop app also enables users to run browser, API, and local web tests in the cloud or locally through a single unified experience. Rather than worry about recreating a clean testing environment in a new browser every time they start a new test, the mabl app automatically opens a fresh browser, reducing the risk of faulty tests and allowing testers to move faster. Mabl offers integrations with Slack, Jira, and Postman that make it easy to integrate automated testing into existing workflows, including shift-left initiatives that bring developers into the testing strategy. Additional integrations with tools like Segment allow testers to align automated testing with actual user journeys, making it easier to connect testing success to business success. Quality professionals are quickly taking on a new — and critical — role in the enterprise as the keepers of product quality. To do so,
they need solutions that enable them to automate routine tasks, embrace a data-driven testing strategy, and focus their talents on high-level quality initiatives. Mabl is the only endto-end test automation solution designed to meet that challenge.
Mark Lambert, Vice President of Strategic Initiatives at Parasoft According to a recent Forrester survey, quality continues to be a priority and the primary metric for measuring the success of software deliveries. With the continued pressure to release software faster and with fewer defects, it’s not just about speed — it’s about delivering quality at speed. Managers must ask themselves if they are confident in the quality of the applications being delivered by their teams. Continuous quality is a must for every organization to efficiently reduce the risk of costly operational outages and to accelerate time-to-market. A critical element to reaching your quality targets is a scalable and maintainable automated testing strategy. When automated tests can be easily created and maintained, your team can focus on the overall quality of the application and verify the use cases, rather than the test scripts themselves. Parasoft solutions leverage artificial intelligence (AI) to enable rapid test creation, self-healing, smart test execution, and other capabilities that streamline your test automation workflows. A leader in the "Forrester Wave: Continuous Functional Test Automation Suites 2020" report, Parasoft provides a complete and integrated quality suite. From deep code analysis for security and reliability, through unit, API, and UI test automation, to performance testing and service virtualization, which enable verification of nonfunctional business requirements, Parasoft helps you build quality into your software development process. “Parasoft’s continuous testing shines in API testing, service virtualization and integration testing, and the combined automation context,” Forrester wrote in its Wave report. According to the report, if you are “looking for a genuine partner in testing, with strong and long-living roots in the testing space and complex technical systems to test, [you] should take a serious look at Parasoft.” Learn how Parasoft helps increase confidence and accelerate delivery of reliable, secure, and compliant software. www.parasoft.com
Anand Sundaram, SVP Products, UI, Device Cloud and Performance Testing at SmartBear Software SmartBear’s mission for over 10 years, making us leaders in this space, has been to meet organizations where they are and help them achieve quality. We help primarily in three journeys, serving everyone from manual testers to developers. First, we help those moving from manual testing to automacontinued on page 31 >
021-35_SDT046.qxp_Layout 1 3/26/21 6:19 PM Page 31
www.sdtimes.com
< continued from page 25
tions and they then need to test the functionality of those APIs. Now, teams are getting quality engineering involved in work around API testing and validation for the first time, whereas historically, that’s been strictly left to the developers, mabl’s Belcher said. There is also a lot of opportunity for API testing because, for example, server changes can be rapidly tested there, as well as microservices, Appvance’s Surace added. Highly data driven API tests will give teams tremendous information about a new server build in a few minutes. However, there are challenges that come up with API testing including the biggest challenge of them all: creating a test scenario that’s realistic. “So developers will deliver you a bunch of APIs and an OpenAPI doc. That’s great. I know what each of the APIs are, but I don’t know how they are
used and I have to now figure out how to chain them together. I need to figure out the payloads. I need to figure out what the data value is.” Lambert said. “With AI, we analyze how the tests are being operated, how the UI has changed, and then we can dynamically heal the tests at runtime, as well as optimize execution, and provide feedback to the development team quicker.” As organizations move more towards an API-centric development model and microservices balloon the complexity of the ecosystem, service virtualization can help to map out the test environment and help with plugging in internal or external dependencies, which are otherwise constraints within a test environment. Vendors have also recognized the increase in demand for mobile and that doesn’t just span phones but also smart TVs, tablets and also the growing embedded devices industry. “People get very focused on user
April 2021
SD Times
interfaces and performance testing and API testing, but actually there’s a whole other piece of this, which is IoT and how does that fit into the whole story as well and actually be able to test that code running on the device itself, which is what a lot of these customers have to have,” said Viktor Krantz, a senior product manager at HCL Software. Highly regulated industries that are increasingly using embedded devices such as the medical, avionics, rail and automotive industries have special requirements that emphasize the importance of testing compliance. The avionics industry for example requires that companies develop and test a device that will then last for 40 years. “If there’s any problem with that device 39 years later, it has to be done in the exact same version of the tool that you created 39 years ago, and test it with a tool from 39 years ago. And that’s literally a work lifetime,” Krantz said. “So it’s a crazy industry. z
Clinton Sprauve, Director of Product Marketing at Tricentis
< continued from page 28 tion. Next, our tools help organizations accelerate by scaling automation as they embrace Agile techniques with CI. Then, we help organizations as they shift left and shift right to release, manage, secure, and improve quickly in a DevOps/NoOps context. Our products cover the most critical aspects of quality across the product development life cycle. Our suite of Zephyr test management solutions enables teams to deliver quality software, resulting in tighter collaboration, endto-end visibility, and faster releases. We have tools that enable you to easily create, manage, and execute automated API and UI tests. The ReadyAPI platform accelerates functional, security, and load testing of web services right inside your CI/CD pipeline, ensuring end-to-end quality for all your web services. Manual testers to automation engineers can use code or codeless test creation with TestComplete to ensure quality across every desktop, web, and mobile application, including enterprise applications. CrossBrowserTesting and BitBar give testers instant access to thousands of browsers, devices, and configurations to achieve the quality consumers demand. A common thread that binds our products is the injection of AI/ML to advance test coverage, authoring, maintenance, execution, and collaboration. Our tools easily integrate with each other and with the ecosystem vendors you’re already using, so that we can be seamlessly embedded into your workflows.
Agile and DevOps have made Continuous Testing essential. Yet, software testing is still dominated by legacy tools and outdated processes— which don’t meet the needs of today’s digital transformation initiatives. Also, enterprises today are still performing over 80% of their testing manually — mostly at the UI layer. As a result, testing occurs late in the software development life cycle, leading to high costs, inefficiency, and delayed innovation. With Tricentis Tosca, customers can achieve over 90% test automation and “shift left” testing much earlier in the software development life cycle. One distinctive Tricentis innovation is Vision AI, a next-generation AI-driven test automation technology that allows teams to automate UI test cases independent of the underlying technology. Through machine learning, Vision AI sees and steers any UI just like a human user, making your automation future proof and as adaptable as the human brain. If you can see it, Vision AI can automate it. This includes anything from an app using now-deprecated technologies to an app using emerging technologies, to apps you access remotely. You can even start building test automation from mockups or whiteboard drawings. This brings a new meaning to test-driven development. Another key advantage of the Tricentis Continuous Testing platform is that it helps enterprise organizations break through the automation barrier. Companies take automation further by using our complete platform for continuous testing across their UIs, back end, and even their data. With an extensive set of integrated tools for designing, optimizing, and maintaining resilient automation, they achieve scalable, sustainable success. z
31
32
SD Times
April 2021
www.sdtimes.com
A guide to automated testing providers n
FEATURED PROVIDERS n
n Appvance: Appvance is the inventor of AI-driven autonomous testing, which is revolutionizing the $120B software QA industry. The company’s patented platform, Appvance IQ, can generate its own tests, surfacing critical bugs in minutes with limited human involvement in web and mobile applications. AIQ empowers enterprises to improve the quality, performance and security of their most critical applications, while transforming the efficiency and output of their testing teams and lowering QA costs. n Digital.ai: Digital.ai Continuous Testing (formerly Experitest) enables organizations to reduce risk and provide their customers satisfying, error-free experiences — across all devices and browsers. Digital.ai Continuous Testing provides expansive test coverage across 2000+ real mobile devices and web browsers, and seamlessly integrates with best-in-class tools throughout the DevOps/DevSecOps pipeline so developers can get test results faster and fix defects earlier in the process, allowing them to deliver secure, high-quality applications at-speed and at-scale. Learn more at www.digital.ai/continuous-testing n HCL Software: HCL Software is a division of HCL Technologies (HCL). HCL Software develops, markets, sells, and supports over 20 product families with particular focus on Customer Experience, Digital Solutions, Secure DevOps, and Security & Automation. Its mission is to drive ultimate customer success of their IT investments through relentless innovation of our software products. n Mabl: Mabl is the leading intelligent test automation platform built for CI/CD. It’s the only SaaS solution that tightly integrates automated end-to-end testing into the entire development life cycle. With mabl creating, executing, and maintaining reliable tests has never been easier, allowing software teams to increase test coverage, speed up development and improve application quality. To learn more about mabl, visit mabl.com. n Parasoft: Parasoft helps organizations continuously deliver quality software with its market-proven, integrated suite of automated software testing tools. Supporting the embedded, enterprise, and IoT markets, Parasoft’s technologies reduce the time, effort, and cost of delivering secure, reliable, and compliant software by integrating everything from deep code analysis and unit testing to web UI and API testing, plus service virtualization and complete code coverage, into the delivery pipeline. Bringing all this together, Parasoft’s award winning reporting and analytics dashboard delivers a centralized view of quality enabling organizations to deliver with confidence and succeed in today’s most strategic ecosystems and development initiatives — security, safety-critical, Agile, DevOps, and continuous testing. n SmartBear: At SmartBear, we focus on your one priority that never changes: quality. Our tools are built to streamline your process while seamlessly working with your existing products. Whether it’s TestComplete, Swagger, Cucumber, ReadyAPI, Zephyr, or one of our other tools, we span test automation, API life cycle, collaboration, performance testing, test management, and more. They’re easy to try, buy, and integrate, and are used by 15 million developers, testers, and operations engineers at 24,000+ organizations. n Tricentis: Tricentis Tosca, the #1 continuous test automation platform, accelerates testing with a script-less, AI-based, no-code approach for end-to-end test automation. With support for over 160+ technologies and enterprise applications, Tosca provides resilient test automation for any use case.
n Applitools is built to test all the elements that appear on a screen with just one line of code. Using Visual AI, you can automatically verify that your web or mobile app functions and appears correctly across all devices, all browsers and all screen sizes. Applitools automatically validates the look and feel and user experience of your apps and sites. It is designed to integrate with your existing tests rather than requiring you to create new tests or learn a new test automation language. Validate entire application pages at a time with a single line of code. We support all major test automation frameworks and programming languages covering web, mobile, and desktop apps. n Eggplant (acquired by Keysight Technologies) Eggplant Digital Automation Intelligence (DAI) is the first AI-driven test automation solution with unique capabilities that make the testing process faster and easier. With DAI you can automate up to 80% of activities including test-case design, test execution, and results analysis. This allows teams to rapidly accelerate testing and integrate with DevOps at speed. n HPE Software’s automated testing solutions simplify software testing within fast moving Agile teams and for Continuous Integration scenarios. Integrated with DevOps tools and ALM solutions, HPE automated testing solutions keep quality at the center of today’s modern applications and hybrid infrastructures. n IBM: Quality is essential and the combination of automated testing and service virtualization from IBM Rational Test Workbench allows teams to assess their software throughout their delivery life cycle. IBM has a market leading solution for the continuous testing of end-to-end scenarios covering mobile, cloud, cognitive, mainframe and more. n Micro Focus: Accelerate test automation with one intelligent functional testing tool for web, mobile, API and enterprise apps. AI-powered intelligent test automation reduces functional test creation time continued on page 35 >
Appvance-Final.qxp_Layout 1 3/26/21 10:43 AM Page 1
Can Autonomous Testing Change Your QA Trajectory? With AppvanceIQ, the answer is YES. AppvanceIQ can be trained one time and then learn changes from each build while
surfacing critical issues. AIQ augments
your existing tests by increasing application
coverage to near 100%. The world’s largest enterprises depend on AIQ daily to look for millions of potential issues, all with
little human intervention.
• Perfect for developers, QA engineers and even manual testers
• Identify API, UI and JS issues (end to end and API level)
• Web & native mobile – Salesforce, ServiceNow, SAP, Workday etc.
• Highlight performance issues within minutes of a new build
• Surface critical bugs in minutes with limited human intervention
• Flag changes between builds – missing features or defective server requests
• Augment your current tests to achieve near 100% application coverage
• Compare mobile user flow timings (with 33ms accuracy), even against competitive apps
Are you ready to change your trajectory? Request a technical consultation today at https://appvance.ai/get-demo
047_SDT032.qxp_Layout 1 1/17/20 5:23 PM Page 1
Reach software development managers the way they prefer to be reached A recent survey of SD Times print and digital subscribers revealed that their number one choice for receiving marketing information from software providers is from advertising in SD Times. Software, DevOps and application development managers at large companies need a wide-angle view of industry trends and what they mean to them. That’s why they read and rely on SD Times.
Isn’t it time you revisited SD Times as part of your marketing campaigns? For advertising opportunities, contact SD Times Publisher David Lyman +1-978-465-2351 • dlyman@d2emerge.com
021-35_SDT046.qxp_Layout 1 3/26/21 6:20 PM Page 35
www.sdtimes.com
< continued from page 32 and maintenance while boosting test coverage and resiliency. Users can test both the front-end functionality and back-end service parts of an application to increase test coverage across the UI and API. n Microsoft’s Visual Studio helps developers create, manage, and run unit tests by offering the Microsoft unit test framework or one of several third-party and opensource frameworks. The company provides a specialized tool set for testers that delivers an integrated experience starting from Agile planning to test and release management, on-premises or in the cloud. n Mobile Labs (acquired by Kobiton) Mobile Labs remains the leading supplier of in-house mobile device clouds that connect remote, shared devices to Global 2000 mobile web, gaming, and app engineering teams. Its patented GigaFox is offered on-premises or hosted, and solves mobile device sharing and management challenges during development, debugging, manual testing, and automated testing. A pre-installed and pre-configured Appium server provides “instant on” Appium test automation. n NowSecure is the mobile app security software company trusted by the world’s most demanding organizations. Through the industry’s most advanced static, dynamic, behavioral and interactive mobile app security testing on real Android and iOS devices, NowSecure identifies the broadest array of security threats, compliance gaps and privacy issues in custom-developed, commercial, and business-critical mobile apps. NowSecure customers can choose automated software on-premises or in the cloud, expert professional penetration testing and
April 2021
SD Times
managed services, or a combination of all as needed. NowSecure offers the fastest path to deeper mobile app security and privacy testing and certification.
more efficient in functional, performance and load testing, improving test coverage and reducing the number of bugs that slip into production.
n Orasi is a leading provider of software testing services, utilizing test management, test automation, enterprise testing, Continuous Delivery, monitoring, and mobile testing technology.
n Sauce Labs provides the world’s largest cloud-based platform for automated testing of web and mobile applications. Optimized for use in CI and CD environments, and built with an emphasis on security, reliability and scalability, users can run tests written in any language or framework using Selenium or Appium, both widely adopted open-source standards for automating browser and mobile application functionality.
n Perfecto: Users can pair their favorite frameworks with Perfecto to automate advanced testing capabilities, like GPS, device conditions, audio injection, and more. It also includes full integration into the CI/CD pipeline, continuous testing improves efficiencies across all of DevOps. With Perfecto’s cloud-based solution, you can boost test coverage for fewer escaped defects while accelerating testing. n ProdPerfect is an autonomous, end-toend (E2E) regression testing solution that continuously identifies, builds and evolves E2E test suites via data-driven, machine-led analysis of live user behavior data. It addresses critical test coverage gaps, eliminates long test suite runtimes and costly bugs in production, and removes the QA burden that consumes massive engineering resources. ProdPerfect was founded in January 2018 by startup veterans Dan Widing (CEO), Erik Fogg (CRO), and Wilson Funkhouser (Head of Data Science). n Progress: Telerik Test Studio is a test automation solution that helps teams be
n Synopsys: A powerful and highly configurable test automation flow provides seamless integration of all Synopsys TestMAX capabilities. Early validation of complex DFT logic is supported through full RTL integration while maintaining physical, timing and power awareness through direct links into the Synopsys Fusion Design Platform. n SOASTA’s Digital Performance Management (DPM) Platform enables measurement, testing and improvement of digital performance. It includes five technologies: TouchTest mobile functional test automation; mPulse real user monitoring (RUM); the CloudTest platform for continuous load testing; Digital Operation Center (DOC) for a unified view of contextual intelligence accessible from any device; and Data Science Workbench, simplifying analysis of current and historical web and mobile user performance data. n testRigor helps organizations dramatically reduce time spent on test maintenance, improve test stability, and dramatically improve the speed of test creation. This is achieved through its support of "plain English" language that allows users to describe how to find elements on the screen and what to do with those elements from the end-user's perspective. People creating tests on their system build 2,000+ tests per year per person. On top of it, testRigor helps teams deploy their analytics library in production that will make systems automatically produce tests reflecting the most frequently used end-to-end flows from production. z
35
036_SDT046.qxp_Layout 1 3/26/21 11:17 AM Page 36
36
SD Times
April 2021
www.sdtimes.com
Guest View BY THOMAS RICHTER
OLAP + OLTP = ...PostgreSQL? Thomas Richter is founder and CEO of Swarm64, a company specializing in high-performance PostgreSQL extensions
P
ostgreSQL is a popular open-source OLTP database for systems of record. It’s the fourth most-widely used database in the world, and its popularity has grown more than any other database for three of the last four years in a row, according to the recent DB-Engines database rankings. Despite the widespread use of PostgreSQL by application developers to manage transactional data, its use in analytic applications such as data warehousing has been quite limited. PostgreSQL is extremely versatile, but query performance often struggles as the quantity of data grows into the terabytes range. As a result, analytic data processing (OLAP) is still dominated by more mature SQL databases like Oracle, SQL Server, DB2, and relative newcomers like Amazon Redshift and Snowflake, or IBM Netezza, and Greenplum. Ironically, Redshift, Netezza, and Greenplum are all forks of PostgreSQL. But there is a perfect storm brewing that bodes well for PostgreSQL and people who want to save money with open source. There are three trends that are increasing the use of PostgreSQL for data warehousing and analytic applications.
The solution to these problems is hybrid transactional/analytical processing (HTAP).
Trend one: People like saving money By next year, Gartner Research has predicted that 70% of database applications will be running on open-source databases. Demand for new systems of insights continues to intensify, but data warehousing is an area that, historically, has not had many good, free, open-source SQL database options. That’s unfortunate because data warehousing platforms are very expensive. The annual maintenance alone on a legacy data warehouse database platform can cost many hundreds of thousands of dollars per year. The economic advantages of replacing legacy data warehouses with open source PostgreSQL are immense. As a result, we’re seeing many Fortune 1000 companies exploring open-source options like PostgreSQL.
Trend two: PostgreSQL has pumped up its performance PostgreSQL has always been a solid transactional performance engine, but historically it has lacked some of the query performance features that we see in commercial data warehouse databases.
Those features include parallel query processing, columnar storage (which greatly reduces the time required to scan and filter query data), and more mature query planning, optimization, and execution, which give you faster performance out of the box with less tuning. Within the last 18 months, these technical shortcomings have largely been addressed. The most recent release of PostgreSQL, version 13, included improvements in parallel processing and query planning. More importantly, PostgreSQL, due to its wonderful extensibility and enthusiastic community, now benefits from a number of extensions, which enhance it with greater parallel processing and columnar storage. So technically, we now see PostgreSQL performing on-par with most of the commercial data warehouse systems, but at 50% to 90% lower annual cost.
Trend three: Goodbye OLAP, hello HTAP The definition of a data warehouse is changing. Traditionally, an analytic system would copy data from one or more sources, via extract-transformload (ETL) programs, into a separate data warehouse database. The problems with this approach are a) cost of maintaining a separate database, b) reporting latency and c) massive effort maintaining ETL scripts. Sometimes minutes, hours, or even days pass before the data is ETL’d into the data warehouse, which means people are analyzing data that might be outdated already. The solution to these problems is hybrid transactional/analytical processing (HTAP). Instead of having separate databases for transactions and for analytics, you have a single database, which serves both needs. In the database management system market, we see two main paths to HTAP — you either add some “T” to an OLAP database, or more commonly we see OLTP databases being enhanced with some “A.” In PostgreSQL for instance, HTAP is primarily enabled by columnar indexes. Some of the extensions I mentioned earlier enhance PostgreSQL with columnar indexes on its base tables, which accelerate query performance by greatly reducing I/O. This is much easier for developers, and it gives businesses the ability to analyze data that is fresher, even in real-time. z
037_SDT046.qxp_Layout 1 3/26/21 11:17 AM Page 37
www.sdtimes.com
April 2021
SD Times
Analyst View BY JASON ENGLISH
Shift testing left, but bank right I
’ve spent most of my professional life convincing businesses to shift things left — shift-left testing for software, shift-left demand and supply forecasts for supply chains, shift-left analytics to understand future implications earlier than your competition. Hopefully that explains why it seems heretical for me to talk about shift-right testing at all. Will shift-right testing somehow cheapen shiftleft testing, making it old news? Or could it cause some teams to stop early preventative testing, just like internet memes can prevent some otherwise rational people from getting vaccinations?
Shift-right is happening anyway With intelligent CI/CD automation, DevOps practices and cloud-native delivery of software into microservices architectures, our software pipelines are moving at such breakneck speeds that much of the activity has moved into ensuring resiliency at change time and post-deployment phases. Shift-right everything — including testing — seems to be inevitable. Given how software development incentives are usually aligned with delivering more features to production, faster — rather than ensuring complete and early testing, I don’t expect many organizations will let shift-left testing activities gate or delay release cycles for very long. So what should we do now, allow end customers to become software testers? No matter how much we try testing earlier in the software lifecycle, with greater automation, there will always be too much change and complexity to prevent all defects from escaping into production — especially when the ever-changing software is likely executing on ephemeral cloud microservices and depending on calls to disparate APIs. There are several interesting vendors that offer pieces of the shift-right puzzle, and to their credit, none really touch the third rail of saying you can leave out QA teams, or call themselves ‘shift-right testing.’ That’s smart marketing. And it doesn’t really matter, they can call it progressive delivery. Canary releases, blue/green deployments, feature flagging, and even some observability, chaos engineering and fast issue resolution workflows. All things that advanced teams do to improve quality and performance nearer to
production, and even post-delivery.
Jason English (@bluefug) is a Principal Analyst and CMO of Intellyx.
Shift Left, but Bank Right Like a bike on a velodrome, or a NASCAR race track banking around the left turns — shift-right testing has less to do with validating what the software does, and more to do with accounting for everything the software might do under stress. Not to be dogmatic, but I don’t consider it testing to put trial releases in front of perhaps smaller groups of customers who aren’t being told they are beta testing. (I wouldn’t want to be holding a pager when a graduated release doesn’t blow up until it scales to half my user base…) It is, however, quite valid to call it validation. Or — maybe risk mitigation. Damage control. Blast radius reduction. Those are all great shift-right aspects of operational excellence to strive for. When you are shifting right you aren’t really shifting testing at all, you are banking the track. You are engineering more tolerance into the system. Bank-Right and build in more operational tolerance to your release track, so you can afford to Shift-Left testing and automated release, to go even faster. You still need early testing, but all the testing in the world will never reach the asymptote of 100% perfection in production. Bank-Right approaches offer slopes and guardrails to keep the race on the track, and put out fires faster, even if the racers behave abnormally.
Shift-Left and Bank-Right go hand-in-hand, just like design and engineering in the real world.
The Intellyx Take Shift-Left and Bank-Right go hand-in-hand, just like design and engineering in the real world. When you drive on a bridge, you hope that it was designed and tested using simulations to flex gracefully when confronted with a variety of natural forces and traffic contingencies. You would also want that bridge to be engineered and monitored post-production to provide early warnings and failsafes to mitigate risk and reduce harm if anything does go wrong. Ultimately, we’ll see both approaches as two different lenses for improving customer experience, no matter what they are called. z
37
038_SDT046.qxp_Layout 1 3/29/21 3:55 PM Page 38
38
SD Times
April 2021
www.sdtimes.com
Industry Watch BY DAVID RUBINSTEIN
Internet crime complaints rise David Rubinstein is editor-in-chief of SD Times.
T
he warranty on your car is about to expire. Press 1 to purchase an extension. Someone has stolen your bank information. Click here to change your social security number. Your grandchild has been kidnapped. Send money to help us return the child safely. We get phishing emails like these literally all day long, and now we also get text messages — literally all day long — trying to get us to part with money or private information that can harm us financially. National do-not-call registries are ineffective. Blocking the number of your cell phone does no good, as the robo-dialers these scammers use seem to have unlimited access to new numbers. For many of us, these texts and emails are an inconvenience. But for those of us who are unwitting, the damage these come-ons do is very real. According to a new report by the FBI’s Internet Crime Complaint Center, (IC3), there were 791,790 complaints registered with the center in 2020, reporting losses exceeding $4.1 billion — a 69% increase in total complaints from 2019. And, in the last five years, there were more than 2,211,396 complaints, with more than $13.3 billion in losses reported. The report noted that in 2020, business email compromise schemes were found to be the costliest — there were 19,369 complaints with an adjusted loss of about $1.8 billion. Moreover, the IC3 saw an increase in the number of complaints related to the use of identity theft and funds being converted to cryptocurrency. The center defines business email compromise as a scam targeting businesses working with foreign suppliers or those regularly performing wire transfer payments. Phishing complaints also took a big toll, as adjusted losses from 241,342 complaints totalled more than $54 million. In fact, since 2016, phishing complaints have risen from 19,465, a more than 12x increase over those five years. The coronavirus pandemic created further opportunities for fraudsters to exploit both businesses and individuals, the report stated. In 2020, the report said, more than 28,500 complaints were filed related to COVID-19.
In 2020, the report said, more than 28,500 complaints were filed related to COVID-19.
“Most of the IC3 complaints related to CARES Act fraud involved grant fraud, loan fraud and phishing for Personally Identifiable Information (PII),” the center said in its report. “Many victims of this identity theft scheme did not know they had been targeted until they attempted to file their own legitimate claim for unemployment insurance benefits. At that time, they received a notification from the state unemployment insurance agency [or] received an IRS form… showing the benefits collected from unemployment insurance, or were notified by their employer that a claim had been filed while the victim was still employed.” In its report, the IC3 noted that the elderly are prime targets of these types of fraud, as they usually have larger savings and are less savvy about the ways of the internet. In 2020, the center got 105,301 complaints from victims over the age of 60, and they reported total financial losses of more than $966 million. These people are usually the victims of what the IC3 calls confidence fraud or romance fraud, which plays on people’s heartstrings to trick them into believing a family member, friend or romantic partner needs their money. The elderly also succumb to tech support fraud, in which a criminal claims to provide customer, security or technical support, offering to fix compromised emails or bank accounts or computer viruses and directing the victims to make wire transfers or purchase large numbers of prepaid cards, the report found. Another area in which the IC3 received thousands of complaints is identified as ransomware, with adjusted losses of more than $29.1 million. The most common means of computer infection for ransomware, according to the report, are email phishing, in which a malicious link deploys malware when clicked by the recipient; remote desktop protocols, which allows individuals to control a computer’s data and resources over the internet by gaining access via brute-force methods or credentials purchases on the internet; and software vulnerabilities identified by OWASP and other sites. SD Times has a number of resources to help you and your organization secure your systems against hackers and scammers in the ‘resources’ section of sdtimes.com. z
Collaborative Modeling
Keeping People Connected ®
®
®
®
®
Application Lifecycle Management | Jazz | Jira | Confluence | Team Foundation Server | Wrike | ServiceNow ®
Autodesk | Bugzilla
sparxsystems.com
SDTimes-PCS-Nov-2020.indd 1
TM
®
®
®
| Salesforce | SharePoint | Polarion | Dropbox
TM
®
| *Other Enterprise Architect Models
Modeling and Design Tools for Changing Worlds
17/11/20 12:55 pm
Full Page Ads_SDT016.qxp_Layout 1 9/21/18 4:14 PM Page 28
SD T Times imes News on Mond day The latest news, news analysis and commentary delivvered to your inbox!
• Reports on the newest technologies affecting enterprise deve developers elopers • Insights into the e practices and innovations reshaping softw ware development • News from softtware providers, industry consortia, open n source projects and more m
Read SD Times Ne ews On Monday to o keep up with everything happening in the software devvelopment industrry. SUB BSCRIBE TODA AY! Y!