DevOps: CI/CD A GUIDE FOR BUYERS
Buyers Guide
Adding value to your CI/CD pipeline BY CHRISTINA CARDOZA
O
ne of the first principles of the Agile Manifesto says to satisfy customers by delivering working software frequently. The problem, however, is that it doesn’t say exactly how we can do that. “Working software is the only measure of progress, but how to you measure that? You need to integrate and test the software as often as possible and fix errors when and as soon as possible,” said William Holz, senior director analyst at the research firm Gartner. According to Holz, in order to move faster and be successful in the areas of
Agile and DevOps, you need to add Agile technical practices to your software development, such as test-driven development and refactoring. One of the best technical practices out there is to create a continuous integration and continuous delivery (CI/CD) pipeline. According to Gartner, CI and CD are the most widespread Agile practices organizations are currently using or plan to use. Continuous integration is the “automation of the software build and validation process driven in a continuous way by running a configured sequence of operations every time a
software change is checked into the source code management repository,” according to Gartner. Dan Packer, industry specialist for the software company Plutora, explained the benefits here are testing, issue resolution, phased changeover, improved team morale, increased velocity, improved quality and improved budget. “With CI, the handoff from one stage to the next is fully automated up to the completion of the testing stage. This is starkly different than that of the waterfall methodology, where the handoff between stages is typically fully manual
Bringing value stream management into the mix Once you are delivering working software faster, the next thing to ask yourself is are your processes becoming more efficient? Are you not only working faster, but delivering value? According to Aaron McCaughan, product owner at the software company Plutora, this is where value stream mapping comes in. Gartner’s William Holz explained a value stream is a collection or series of steps that deliver customer value. For instance, if you buy something on Amazon, the order button is a value stream. You click the button and it goes through a series of steps to get your package to your door. Value stream mapping provides insights into those steps, where the value is, where the value isn’t, and detects areas that can be improved. “With CI/CD, you might be able to react, you might be able to deliver faster, but you also have to factor in am I delivering value faster?” McCaughan said. “What am I putting into the pipeline that is satisfying my customer so that I am actually keeping everyone happy and increasing the value of the product?” Many of the metrics teams are looking at are deployment frequencies, number of check-ins per day, build failure rates, or mean time to recovery. While those are interesting indicators of work,
they don’t really measure customer value or help you understand what is flowing through the system, according to Jeffrey Keyes, director of product marketing for Plutora. Value stream management provides a broader view of the entire delivery life cycle long before the software becomes software, McCaughan explained. It goes from ideation to production so you can start tracking your value stream map as soon as you identified your strategic goals for the organization and start identifying bottlenecks. “For instance, if there are three months of planning on average for each change coming through, you can start addressing that,” said McCaughan. “It’s about capturing those dependencies, wait times and overburdens.” Specific metrics the value stream map brings to light are the average cycle times it takes to deliver a feature, process time, waste time, and actual time spent working on the feature, according to Keyes. “If you are spending half your time putting out fires, you are not adding value. You are just reacting to production defects or technical debt,” said McCaughan. “Value stream management is the combination of Agile plus DevOps plus the measured outcomes at each phase,” Keyes added. z
throughout the entire lifecycle and deals primarily with completed applications vs. small code segments,” Packer wrote in a blog post. CD is an evolutionary step of CI designed to take automation further, according to Packer. Continuous delivery is the act of releasing reliable software faster through technical delivery and deployment practices like working in small batches and automating repetitive tasks. Benefits include improved velocity, phased progression, always production ready and release control, Packer wrote. Together, they make up the CI/CD pipeline: A continuous flow of software designed to reduce manual and errorprone work, and result in higher quality software. “This is important because the ninth principle of Agile states attention to technical excellence and design increases your agility. In Agile, rework is waste. The time you spend fixing bugs is time that can be better spent delivering new features and functionality,” said Gartner’s Holz. “You need to be able to do CI and DevOps to achieve continuous delivery. It is no longer about your software. It is about delivering the entire solution.” According to Holz, this is how you accelerate your Agile and DevOps initiatives successfully, reduce risks and catch bugs. CI/CD enables the ability to build, package, integrate, test and release code with automation. “Complex operations like CI/CD cannot be accomplished without significant engineering effort. DevOps recognizes the importance of joining tooling and thought throughout the entire process of development and deployment rather than isolating build and production. Rather than having distinct steps when creating a service, DevOps encourages simultaneous development and testing. CI/CD pipelines fit into this perfectly by providing a way to automate the testing portion,” explained Abhinav Asthana, CEO and cofounder of the API development solution provider Postman. z
How do you add value to the CI/CD pipeline? Jeffrey Keyes, director of product marketing for the software company Plutora: At a high level, we address three areas. The first is that we integrate with and unify the entirety of the Agile and DevOps toolchain, including CI/CD tooling. The point of unification is eliminating the inefficiencies and loss of fidelity of handoffs. We also correlate the data and artifacts into the stages of delivery and relate all of the information together. This is critical as you need to see what features are actually being delivered, how that relates to code being built, where it sits in the pipeline and the relationship of test to all of that data. The second area is the management of key processes. We provide release orchestration enabling additional visibility and logic enhancements augmenting the CI/CD pipeline. We can orchestrate between the manual and automated tasks of any pipeline, decomposing delivery into phases and gates ensuring governance is maintained and you have appropriate levels of quality. We have a deployment planning and orchestration which augments application release automation managing the go-live activities. We also have a non-production environment management solution centralizing the requests, orchestrating the provisioning and manages the utilization of pre-production environments. The third area is the analytics and the visualization of the value stream itself. We provide out-of-box visualizations, including a value stream map, for the flow of work along the entire process. We provide rich “what-if” scenario analysis and comparison metrics including using teams and time as dimensions. We enable you to answer the most important question of digital transformation — are we improving?
Abhinav Asthana, CEO and co-founder of API development solution provider Postman: Postman offers a comprehensive API testing tool that makes it easy to set up automated tests. You can aggregate the tests and requests you’ve created into a single automated test sequence that you can reuse again and again. Integration testing is hard. Running through sad paths consisting of hundreds of failing dependencies is a nearly impossible task for all but the largest organizations. From simple happy path debugging to thorough sad path deployment, Postman keeps your tests tightly coupled with your services. The tool is approachable, allowing even less technical QA team members to contribute to a testing suite. At the same time, it is robust in allowing the simulation of complex workflows and business logic. Every test that can be run manually via the Postman GUI, can be automated in Postman’s command line tool, Newman, and can be included as a build step in your pipeline. With Postman you aren’t testing just your code, but the fabric of your entire service, and those it relies on. Hundreds of organizations have built Postman collections alongside their development and employed them as integration tests. Your developers are already debugging with Postman, why not put that work to good use? z
A guide to DevOps CI/CD tools
n Atlassian: Atlassian offers cloud and onpremises versions of continuous delivery tools. Bitbucket Pipelines is a modern cloud-based continuous delivery service that automates the code from test to production. Bamboo is Atlassian’s on-premises option with first-class support for the “delivery” aspect of Continuous Delivery, tying automated builds, tests and releases together in a single workflow. n API Fortress: API Fortress is a continuous testing platform for APIs. It is the final piece to complete your continuous integration vision. One platform to test functionality, performance, and load. Save time with automated test generation, benefit from true cross team collaboration, leverage your existing version control system, and seamlessly integrate with any CI/CD platform. Catch problems before they are pushed live — automatically.
n Automic: Automic from CA Technologies, a Broadcom Company, is leader in business automation software. Automic V12 is a unified suite of business automation products for driving agility across enterprise operations and empowering DevOps initiatives. n Chef: Chef Automate, the leader in Continuous Automation, provides a platform that enables you to build, deploy and manage your infrastructure and applications collaboratively. Chef Automate works with Chef’s three open-source projects; Chef for infrastructure automation, Habitat for application automation, and Inspec for compliance automation, as well as associated tools. n CloudBees: CloudBees is powering the continuous economy by building the world’s first end-to-end system for automating software delivery, the CloudBees Suite. The CloudBees Suite builds on emerging DevOps practices and continuous integration (CI) and continuous delivery (CD) automation adding a layer of governance, visibility and insights necessary to achieve optimum efficiency and control new risks.
n
FEATURED PROVIDERS n
n Plutora: Plutora provides the most complete value stream management solution for enterprise IT, improving the speed and quality of software creation by capturing, visualizing and analyzing critical indicators of every aspect of the delivery process. Plutora orchestrates release pipelines across a diverse ecosystem of development methodologies, manages hybrid test environments, correlates data from existing toolchains, and incorporates quality metrics gathered at every step. The Plutora Platform unifies existing application delivery toolchains ensuring assets, data and artifacts flow between systems. It ensures organizational alignment of software development with business strategy and provides visibility, analytics and a system of insights into the entire value stream, guiding continuous improvement through the measured outcomes of each effort. n Postman: As APIs are becoming increasingly more important to the development lifecycle, a proper API strategy is a crucial piece of your CI/CD pipeline. Postman provides tools that support every stage of the API lifecycle from design and testing to monitoring and debugging. Postman’s command line tool, Newman is designed to help developers run and test Postman Collections directly from the command line and integrate Postman tests within their CI/CD build process. Developers can run Postman tests every time their build process kicks off, and integrate it with their CI service such as Jenkins, Travis CI or any other code deployment pipeline tool. n Datical: Datical brings Agile and DevOps to the database to radically improve and simplify the application release process. Datical solutions deliver the database release automation capabilities IT teams need to bring applications to market faster while eliminating security vulnerabilities, costly errors and downtime. n Dynatrace: Dynatrace provides software intelligence to simplify enterprise cloud complexity and accelerate digital transformation. With AI and complete automation, our all-in-one platform provides answers, not just data, about the performance of applications, the underlying infrastructure and the experience of all users. n Electric Cloud: Electric Cloud helps software-driven companies like E*TRADE, GM, Hyundai, Intel and Samsung build and release applications and devices at any speed the business demands, with the acceleration, orchestration and insight needed to continuously improve their results. n GitLab: GitLab is the only single application for the entire DevOps lifecycle, allowing Product, Development, QA, Security, and
Operations teams to work concurrently on the same project. Designed to provide a seamless development process, GitLab’s built-in Continuous Integration and Continuous Deployment offerings enable developers to easily monitor the progress of tests and build pipelines, then deploy with the confidence that their code has been tested across multiple environments. n IBM: UrbanCode accelerates delivery of software change to any platform — from containers on cloud to mainframe in data center. Manage build configurations and build infrastructures at scale. Orchestrate, automate and deploy applications, middleware and database changes. Release interdependent applications with pipelines of pipelines, plan release events, orchestrate simultaneous deployments of multiple applications. Improve DevOps performance with value stream analytics. n JetBrains: TeamCity is a continuous integration and continuous delivery server that takes moments to set up, shows your build results on-the-fly, and works out of the box. It will make sure your software gets built, tested, and deployed, and you get notified about that appropriately, in any way you choose. TeamCity integrates with all major development frameworks, version control systems, issue trackers,
IDEs, and cloud services. n Microsoft: Microsoft’s Azure DevOps solution is a suite of DevOps tools designed to help teams collaborate to deliver high-quality solutions faster. Azure DevOps marks an evolution in the company’s Visual Studio Team Services. VSTS users will now be upgraded to Azure DevOps. The solution features Azure Pipelines for CI/CD initiatives, Azure Boards for planning and tracking, Azure Artifacts for creating, hosting and sharing packages, Azure Repos for collaboration and Azure Test Plans for testing and shipping. n Octopus Deploy: Octopus Deploy is an automated release management tool for modern developers and DevOps teams. Features include the ability to promote releases between environments, repeatable and reliable deployments, ability to simplify the most complicated application deployments, an intuitive and easy to use dashboard, and first-class platform support. n Redgate Software: Including SQL Server databases in Continuous Integration and Continuous Delivery, and stopping them being the bottleneck in the process, is the mission at Redgate. Whether version controlling database code, including it in continuous integration, or adding it to automated deployments, the SQL Toolbelt from Redgate includes every tool necessary. Many, like SQL Source Control, SQL Compare and SQL Change Automation, integrate with and plug into the same infrastructure already used for application development. n Rogue Wave Software: Rogue Wave helps thousands of global enterprise customers tackle the hardest and most complex issues in building, connecting, and securing applications. From API management, web and mobile, embeddable analytics, static and dynamic analysis to open-source support, we have the software essentials to innovate with confidence.
n Sauce Labs: Sauce Labs provides the world’s largest cloud-based platform for automated testing of web and mobile applications. Optimized for use in CI and CD environments, and built with an emphasis on security, reliability and scalability, users can run tests written in any language or framework using Selenium or Appium, both widely adopted open-source standards for automating browser and mobile application functionality. n Tasktop: Tasktop provides the backbone for the most impactful Agile and DevOps transformations by connecting all the best-of-breed tools used for planning, building and delivering software at scale. With its unique model-based integration, Tasktop automates the flow of information from tool to tool, removing the duplicate data entry and manual handovers that are slowing teams down. By normalizing and standardizing data as it flows, Tasktop provides a one-of-akind set of metrics that tells leadership exactly how well they are performing against the business objectives and where they can improve. n TechExcel: DevSuite helps organizations manage and standardize development and releases via agile development methods and complete traceability. We understand the importance of rapid deployment and are focused on helping companies make the transition over to DevOps. To do this, we have partnered with many automation tools for testing and Continuous Integration, such as Ranorex and Jenkins. Right out of the box, DevSuite will include these technologies. n XebiaLabs: XebiaLabs develops enterprise-scale Continuous Delivery and DevOps software, providing companies with the visibility, automation and control they need to deliver software faster and with less risk. Global market leaders rely on XebiaLabs to meet the increasing demand for accelerated and more reliable software releases. z
Why a good API strategy matters Application programming interfaces (APIs) are increasingly becoming more important to software development as organizations embrace connected services and microservices architectures. “Meaning that APIs are consumed by many different people and are integrated with diverse and complex services,”said Postman CEO and cofounder Abinhav Asthana. According to Gartner’s William Holz, in order to do effective CI/CD, APIs create a separation of concerns and enable the ability to test at the unit level, feature level, integration level and performance level. “APIs gives me an idea of where to look for a problem, help me solve that problem sooner, and minimize the amount of waste or rework I need to do,” he said. “I use the example that I can have 100 percent unit test coverage, but I can still break features. I can still break my application because unit tests don’t test the future.” However, according to Asthana, testing is difficult, and in order for an API strategy to be a powerful tool, dependencies need to be removed. “With connected services, dependencies are a huge concern. If a team updates an API, it could potentially break an API consumer’s service, but a CI/CD pipeline can solve this problem,” Asthana said. “CI/CD pipelines ensure that connected services are healthy by consistently checking for broken dependencies with full system tests at every build. This means that broken dependencies are most often caught in development rather than in production. The later a bug is caught, the more expensive and timeconsuming it becomes to fix it.” Once a CI/CD pipeline is set up, developers can run integration tests as part of their build. According to Asthana, those tests stay with the developer until they pass, which means the production environment remains safe from harm. “A good CI/CD pipeline will have reporting built in, so testers can review their automatically run test results and determine the source of not just errors in their code, but errors in interaction with dependencies,” Asthana said. Asthana adds that teams should also find a tool or framework that can write tests and maintain a test library for the pipeline. This will ensure a good set of tests that can be used and reused. Tools should also provide the ability to test against live environments, services and data, and have a system of reporting implemented so developers and tests can get access to insights quickly, according to Asthana. “Implementation of CI/CD greatly simplifies API management. The consistency and reliability of a CI/CD pipeline mean that developers aren’t bothered with manually managing dependencies between versions. Instead of putting out fires, developers can spend their time improving products. CI/CD pipelines catch errors early on, which saves time and money,” he said. z