SD Times API Management Buyers Guide

API

Management

A GUIDE FOR BUYERS

2

SD Times

October 2021

www.sdtimes.com

API management is a data integration problem

BY JENNA SARGENT

W

ith data being increasingly stuck behind different services, API management is becoming more and more of a data integration challenge.

Currently most companies view API management as an access problem, but Avadhoot Kulkarni, product manager at Progress, recommends they shift their mindset and view it as a data problem instead.

According to Kulkarni, APIs are nothing but “just ways to expose your data in user consumable ways.”

As such, managing APIs leads to a number of data management challenges, including how to maintain data quality, data profiling, and data ownership. API developers and maintainers are concerned about data integrity and data consistency across their APIs, but the emergence of microservices architectures have helped in breaking down monolith applications into smaller services, which creates data silos.

“Information, which is critical for organizations for their decision making, is locked behind different services. And it’s not easily accessible to the tooling that helps them integrate that data and get a business decision out of those,” said Kulkarni.

One way to address that challenge is to give access to back end data directly, but that comes with its own set of new challenges, according to Kulkarni. It can create issues with data ownership as the user role access constraints put in place as a security measure into the API logic might not be accessible.

This can be fixed for a small number of APIs by implementing custom integrations, but as the number of connections needed grows, it becomes less manageable.

www.sdtimes.com

In addition, being able to write those custom integrations for data warehouses, data lakes, or business intelligence tools requires a deep knowledge of the API itself. This is another reason why this solution isn’t scalable, according to Kulkarni.

“You sort of waste your engineering capacity on that instead of putting it on your business. You start spending on this side project, which is, most likely not the best avenue for spending your resources,” said Kulkarni.

Progress’ Kulkarni predicts that more and more of the industry will soon accept this idea of API management being a data management concern. AI and machine learning have permeated so much of what is done in the tech space, and data-driven or data-aware decision making is becoming more of the norm.

“API management will be treated more like a data management problem in the near future. So the question about data quality, data profiling, how data gets moved between the different components, who has access to this data was also what privileges that particular person has on that data like who can modify versus who can only read how that data integrates with different solutions, that would be not only considered, but it will be also baked into the API architecture going forward,” said Kulkarni.

Data mesh emerges According to Eric Madariaga, chief marketing officer at CData, data mesh is a technology that is emerging to help

companies with this challenge. A data mesh helps to decouple data entry points.

Data mesh was included in ThoughtWorks’ Technology Radar, first in November 2019 in the “Assess” category, and then moving into the “Trial” category in the October 2020 Radar.

ThoughtWorks defines data mesh as “an architectural and organizational paradigm that challenges the age-old assumption that we must centralize big analytical data to use it, have data all in one place or be managed by a centralized data team to deliver value.”

According to ThoughtWorks, the concept is built on four principles: 1. Decentralized data ownership and architecture 2. Domain-oriented data served as a product 3. Self-serve data infrastructure 4. Federated governance, enabling interoperability between systems “Different data assets within an organization become surfaced through a mesh-like architecture, so that they can be consumed and integrated from a variety of different resources,” said Madariaga.

The concept isn’t that far off from the original concept of APIs, Madariaga explained. The data mesh provides a common interface for communication between different data resources, much like how API infrastructures help applications communicate with each other.

“It becomes kind of an entire architectural paradigm,” said Madariaga.

“It’s something that large organiza-

Best practices for API creation According to Forrester’s Mooter, it’s best to develop APIs by looking “outside in” rather than “inside out.” What this means is that rather than looking inwards at how the IT systems are already implemented, API developers should look outwards towards who will actually be using the API and what their needs are.

He explained that further down this planning process it might be necessary to start considering your internal IT constraints due to factors like cost, but the process “should always begin and largely be driven by end user need, not what's already in your IT system.”

Another important consideration for API management is governance. Mooter explained that sometimes companies tend to either under-govern or over-govern, neither of which are ideal. Over-governing could result in things getting slowed down too much, while not having enough governance can result in targets not being met.

“Finding that sweet spot is rather challenging for organizations,” said Mooter. z

October 2021

SD Times

Buyers Guide tions are using when they have multiple data warehouses and things. Conceptually, you know, it’s the idea of having a common interface for communicating with these resources, and solving the distributed dispersed data asset issues that organizations are facing and dealing with today. In API infrastructure, people are having applications that are trying to communicate with each other. They’re trying to do that in a common and consistent way. Data mesh, similarly, is solving that problem.”

Event streams also gaining popularity According to David Mooter, senior analyst at research firm Forrester, eventdriven architecture is another technology that is coming into play in the API management equation, specifically event streams.

Mooter described a number of vendors already playing in this space of applying event streams to API management, such as IBM and Solace, and there is demand from clients. REST APIs have opened the doors for a lot of business innovation, but they do have their limitations, and event streams are helping to fill in some of those gaps.

“It’s growing in popularity, but I’ve seen a lot more growth in demand for event streams as not an alternative to REST, but as an additional pool set that complements REST,” said Mooter.

According to CData’s Madariaga, standardization of APIs is important, yet there are many different API frameworks that are in use today, such as REST and SOAP. “So there’s this huge landscape of how applications are talking to one another, and all kinds of different API interface standards,” said Madariaga.

Madariaga believes it’s important to have a common language for these APIs to communicate through.

Democratizing data management “It enables citizen immigration and citizen developers and citizen integrators continued on page 20 >

3

4

SD Times

October 2021

www.sdtimes.com

< continued from page 3

to use their tooling to work with APIs and data … If you want to increase adoption of your APIs of which you as a developer worked hard to build, providing tooling that gets all the way down to the end user is a very popular way, it’s a very important way to enable the broadest usage of your APIs,” said Madariaga.

The beauty of low-code is that it allows non-developers to build applications through a drag-and-drop UI interface, but according to Forrester’s Mooter, those UI portals aren’t very useful unless they’re able to talk to IT systems.

Therefore it’s important that citizen developers are able to connect via a robust suite of APIs. According to Madariaga, there can be a lot of complexity in the way citizen developers connect to APIs. If they want to integrate with an API, they must first define inputs and outputs, and may also have to configure the authentication settings.

This can be a barrier to entry for those without the technical knowledge needed. “By abstracting that into, say, a common database standard interface, you literally just drop in a driver and start working with back end APIs, like you would a standard traditional database, and every low-code and no-code application knows how to work with a traditional RDBMS database,” said Madariaga.

This abstraction not only benefits citizen developers, but saves traditional developers time as well.

“Because really, ultimately, what happens is you’re submitting queries and getting back tables of data, and those tables are self describing,” said Madariaga. “So they come back, and they provide the columns of data that are there that are exposing underlying APIs. You can do things like joins and aggregates, and you could do all that in in way less code than it would be to go connect to an API itself, get data, do the transformations, do the integration, or anything else on the back end, it is a lot more complex when you are not using one of these API standards.” z

How does your company help companies manage their APIs? Eric Madariaga, chief marketing officer at CData APIs are mere table stakes in the world of integration. They offer the promise of connectivity, but require IT and developer resources to integrate, discouraging widespread adoption.

At CData Software (www.cdata.com) we simplify API integration through Data APIs. Our universal data connectivity solutions deliver powerful database abstractions on APIs, connecting business users, analysts, and integrators with API data without code. Leveraging a traditional database metaphor, Data APIs provide tables, views, and stored procedures, that map to resources and operations exposed by each data source. With standard database interfaces like ODBC, JDBC, and ADO.NET, Data APIs act as a real-time universal translator between data consumers (Data Governance, Data Prep, Data Integration, Master Data Management, AI & ML, Data Catalog, Data Warehousing, Data Science, BI, Analytics, and even developer technologies like Low-code integration) and all the applications and data sources used across an organization.

For DevOps and DataOps teams, standardization on Data APIs establishes a common semantic layer that simplifies the ingestion, curation, and orchestration of siloed data and helps support data democratization initiatives. Instead of having to code integrations against one-off APIs, and continuously stay up to date on every API change, Data APIs provide a layer of abstraction that protects consumers from the constantly shifting elements. At CData, our Data API-based solutions offer a tactical approach to connectivity, augmenting existing integration, processing, and analytics tooling to support broad data access capabilities.

To learn more about Data APIs, standards-based drivers, and their impact on data integration, visit us online at www.cdata.com.

Avadhoot Kulkarni, product manager at Progress Organizations, like never before, are embracing business intelligence and analytics solutions to drive decisions within their business. What organizations are learning, however; is that these solutions can only reach their full potential when populated with all relevant data sources.

Empowering data access for your applications is what we do at Progress DataDirect. Our products offer secure data connectivity solutions for enterprises needing to integrate data across relational, big data, non-structured and cloud databases to make better, and more informed decisions. And with unrivaled speed, Progress DataDirect ensures the data is always relevant and timely.

With APIs quickly becoming the standard across organizations for sharing data internally and externally, the Progress DataDirect Autonomous REST Connector delivers seamless, real-time connectivity between REST data and your ODBC/JDBC tools and applications.

By opening your application’s data through API standards like REST, Progress DataDirect improves accessibility from widely used BI, analytics and development tools, as well as reducing the amount of reworking of established analytical and reporting task/jobs.

With Autonomous REST Connector organizations can expect: l A built-in user interface where organizations can quickly create connectors that you and your end users need l Out of the box recipes offer out of the box connectivity for business-critical systems that are ready to use as-is or easily customizable l Reduced time/effort to adopt APIs and services by your organization l Continued value from existing analytic and reporting tools when moving to APIs and services l Reduced risk of vendor lock-in and poor data quality l The ability to simplify and accelerate the adoption of your own APIs z

Full Page Ads_SDT052.qxp_Layout 1 9/23/21 5:07 PM Page 21

6

SD Times

www.sdtimes.com

October 2021

A guide to API management tools n

FEATURED PROVIDERS n

n CData: Connect, Integrate, and Automate your enterprise data. At CData (www.cdata.com), we simplify connectivity between all of the applications and data sources that power business operations, making it easier to unlock the strategic value of your data. By focusing on established standards for data access, our solutions plug into all of the business applications that you use today (like BI, Reporting, ETL, & Integration) and connect them with live data from just about anywhere. n Progress: The Progress DataDirect Autonomous REST Connector offers intelligent data connectivity to API sourced data from SQL based applications such as BI, Analytics, and ETL tools. With Autonomous REST Connector organizations can expect: l Reduced time/effort to adopt APIs and services l Continued value from existing analytic and reporting tools when moving to APIs and services l Reduced risk of vendor lock-in and poor data quality l The ability to simplify and accelerate the adoption of your own APIs.

n Apigee is an API management platform for modernizing IT infrastructure, building microservices and managing applications. The platform was acquired by Google in 2016 and added to the Google Cloud. It includes gateway, security, analytics, developer portal, and operations capabilities. n Akana by Perforce provides an end-toend API management solution for designing, implementing, securing, managing, monitoring, and publishing APIs. The Akana API Platform helps you create and publish secure, reliable APIs that are elegant, easy to consume, built the right way, and running as they should. n Boomi’s API management solution provides a unified and scalable, cloud-based platform to centrally manage and enrich API interactions through their entire life cycle. With Boomi, users can rapidly configure any endpoint as an API, publish APIs on-premises or in the cloud, manage APIs with traffic control and usage dashboards. n CA Technologies, a Broadcom company, helps customers create an agile business by modernizing application architectures with APIs and microservices. Layer7 API Management provides the most trusted and complete capabilities across the API life cycle for development, orchestration, security, management, monitoring, deployment, discovery and consumption.”

n Cloud Elements delivers an API integration platform on three pillars: “Elements” unify APIs with enhanced capabilities for authentication, discovery, search, error handling and API maintenance.

“Formulas” combine those Elements to automate business processes across applications. “Virtual Data Hubs” provide a normalized view of data objects. n IBM API Connect on IBM Cloud is an API life cycle management offering that allows any organization to secure, manage and share APIs across cloud environments — including multi-cloud and hybrid environments. n Kong delivers a next-generation API and service life cycle management platform designed for modern architectures, including microservices, containers, cloud and serverless. Kong is building the future of service control platforms to intelligently broker information across services.

existing services or secure APIs with an API management gateway; add or remove pre-built or custom policies; deliver access management; provision access; and set alerts so users can respond proactively. n Nevatech Sentinet is an enterpriseclass API management platform written in .NET that is available for on-premises, cloud and hybrid environments. Sentinet supports industry SOAP and REST standards as well as Microsoft-specific technologies and includes an API Repository for API Governance, API versioning, autodiscovery, description, publishing and Lifecycle Management. n Postman is the leading collaboration platform for API development, used by more than 7 million developers and 300,000+ companies worldwide. Postman allows users to design, mock, debug, test, document, monitor, and publish APIs — all from one place. n Red Hat 3scale API Management gives control, visibility and flexibility to organizations seeking to create and deploy an API program. It features comprehensive security, monetization, rate limiting, and community features that businesses seek backed by Red Hat’s solid scalability and performance. n SmartBear Software: With Swagger’s easy-to-use API development tools, SoapUI’s automated testing proficiency, AlertSite’s API-monitoring and ServiceV’s mocking and virtualization capabilities, users can build, test, share and manage the best performing APIs.

n Microsoft’s Azure API Management solution enables users to publish, manage, secure and analyze APIs in minutes.

It features the ability to create an API gateway and developer portal quickly, ability to manage all APIs in one place, provides insights into APIs, and connects to back-end services.

n SnapLogic Lifecycle API Management is an end-to-end solution designed for managing, scaling and controlling API consumption quickly, seamlessly and securely.

Features include request/response transformations, API traffic control and productization, OAuth2 authentication support, advanced API analytics, threat detection, and the developer portal.

n MuleSoft’s Anypoint API Manager is designed to help users manage, monitor, analyze and secure APIs in a few simple steps. The manager enables users to proxy

n TIBCO Cloud Mashery’s capabilities include API creation, productization, security, and analytics of an API program and community of developers. z

IBC_SDT049.qxp_Layout 1 6/25/21 3:29 PM Page 2

Time to go with the flow!

Organizations today are turning to value streams to gauge the effectiveness of their work, reduce wait times and eliminate bottlenecks in their processes.

Most importantly, they want to know: Is our software delivering value to our customers, and to us?

VSM Times is a new community resource portal from the editors of SD Times, providing guides, tutorials and more on the subject of Value Stream Management.

Sign up today to stay ahead of the VSM curve! www.vsmtimes.com