Radiantadvisors research cisco data virtualization adoption by Radiant Advisors

OVERCOMING BARRIERS TO DATA VIRTUALIZATION ADOPTION A Cross-Talk Research Paper Lindy Ryan, Research Director

Research

This document contains Confidential, Proprietary and Trade Secret Information (â&#x20AC;&#x153;Confidential Informationâ&#x20AC;?) of Radiant Advisors. No part of this publication may be reproduced, stored in a retrieval system, or transmitted in any form or any means electronic or mechanical, including photocopying and recording for any purpose other than the purchaserâ&#x20AC;&#x2122;s personal use without the written permission of Radiant Advisors.

While every attempt has been made to ensure that the information in this document is accurate and complete, some typographical errors or technical inaccuracies may exist. Radiant Advisors does not accept responsibility for any kind of loss resulting from the use of information contained in this document. The information contained in this document is subject to change without notice.

All brands and their products are trademarks or registered trademarks of their respective holders and should be noted as such.

This edition published September 2014.

Overcoming Barriers to Data Virtualization Adoption

Research

Table of Contents

Introduction

Cross-Talk Research Methodology

Barrier #1: Choosing Data Virtualization

Barrier #2: identifying the Lynchpin Use Case

Barrier #3: Assigning Roles and Responsibilities

Barrier #4: Selecting a Vendor

Conclusion

Overcoming Barriers to Data Virtualization Adoption

Research

Introduction

The challenges of data management are getting exponentially harder. With the ever-increasing quantities, sources, and structures of data – as well as the insurgence of new tools and techniques for storing, analyzing, and deriving deeper insights from this information – data-driven companies continue to evaluate and explore data management technologies that better integrate, consolidate, and unify data in a way that offers tangible business value. Data virtualization is so compelling because it addresses business demands for data unification and supports high iteration and fast response times, all while enabling self-service user access and data navigability. However, adopting data virtualization is not without its set of barriers. Primarily, these relate to building a business case that can articulate the value of data virtualization in terms of speed of integration alongside the ability to manage ever-growing amounts of data in a timely, cost-efficient way. Supported by Cisco Data Virtualization, Radiant Advisors had the opportunity to further explore and understand the barriers experienced by companies considering data virtualization adoption, and then to pose these questions to companies that have already adopted data virtualization to glean their insights, best practices, and lessons learned. Together, the two halves of this research facilitate a practicable, independent, and unscripted “cross-talk” to fill information gaps and assist companies in overcoming barriers to data virtualization adoption.

Overcoming Barriers to Data Virtualization Adoption

Research

Cross-Talk Research Methodology

The first portion of the research independently solicited the participation of many companies within the Radiant Advisory Network to properly harness a wide perspective across many industries and organizations to identify common concerns and questions regarding barriers to data virtualization from a broad, preadopter community. The companies included were very interested in adopting data virtualization and either currently not using data virtualization or evaluating the technology for potential adoption. Gathering this perspective ranged from structured discussions at industry events to formal interviews with BI team leads. As a result, we identified areas of concern and isolated critical questions in need of response from the post-adoption community. These top-of-mind concerns were then posited to companies that have already adopted and implemented data virtualization within their organization. To facilitate this exchange of information, current data virtualization-successful customers were solicited from Cisco’s existing customer network. Representatives from each company were invited to independently and anonymously share insights, best practices, and lessons learned in overcoming barriers to data virtualization adoption. Following the completion of all interviews, detailed research notes from both consorts of companies were synthesized with Radiant Advisors’ expertise and thought-leadership to distill key insights that filled the information gaps identified by pre-adopter companies. Ultimately, this research and the following report are intended to guide potential adopters to overcome perceived barriers to data virtualization adoption by facilitating an independent, unscripted “cross-talk” between potential adopters and those who have already moved forward.

Overcoming Barriers to Data Virtualization Adoption

Research

Barrier #1: Choosing Data Virtualization

With so many available technologies and approaches to data management, the first question that potential adopters consider is the rationale for the decision to adopt data virtualization opposed to other approaches (i.e. data federation, master data management (or, MDM), the data lake, extract-transform-load (or, ETL), etc.). Some companies have expressed that MDM provides a more manageable and intuitive approach, rather than consolidating multiple copies of data. Others wonder if data virtualization will eventually take a back burner in data management as the concept of the data lake grows in popularity and acceptance. Finally, there is lingering ambiguity about how much data virtualization is needed for batch analytics, real-time insight, and interactive queries – and if data virtualization can

“When you move really fast,

become oversaturated with too much data from the same source.

customers are really happy

Then, after deciding on data virtualization has been nominated as the preferred

– every bit of slowness directly impacts customer

approach, the next “biggest hurdle” has been expressed as how to effectively communicate what data virtualization is – both as a data management construct and as a value proposition – to C-level executives who are critical for buy-in and championship at the business level. It has been noted by IT teams that business

happiness.”

executives tend to “get lost in the sauce” and dwell on trying to navigate the differences between different data integration technologies, and that distinctions

- A commercial, web-based

aren’t meaningful at the C-level. Frankly, this makes the ability to gain executive

services company

buy-in and support difficult. Companies evaluating data virtualization would like to better understand how to take data virtualization “from a concept to a framework” in a way that is communicable and meaningful to executive stakeholders.

Overcoming the Barrier Current data virtualization users note that sometimes you can’t be traditional – you have to do something different. That said, the value proposition is simple: data virtualization avoids longer data integration development lifecycles that are riskier to meet fast paced business needs, is faster to market and deploy, and supports the changes that happen on-the-fly in the organization. Companies that use data virtualization have described it as a seamless and quick way to “insulate and adapt,” by using an abstraction layer to “snap together” data,

Overcoming Barriers to Data Virtualization Adoption

Research

Barrier #1: Choosing Data Virtualization

change views, present the same data from multiple places, and change the logic behind it. The reality is that (business) users want unfettered access to unaltered data, and they want it fast. Using data virtualization avoids depleting alreadyscarce IT time to build a model that ends up not meeting expectations – inevitably frustrating business – and instead offers a quick scanning tool to build in a way that responds immediately to business changes and needs. Data virtualization has, too, been seen as a way to expand IT’s potential – described as a “capacity increasing initiative,” if you will. As a layer of abstraction, data virtualization lives between data sources to establish unified semantic context and data access without actually impacting the data source structure. It provides a mechanism to bring IT closer to the business, too, moving away from a very techheavy and IT-driven function to one that can be translated into business language and moved closer to the business user for richer enterprise-wide collaboration. Finally, data virtualization has been implemented as a vehicle to polyglot persistence (persisting data in the data store best suited for its purposes). With the influx of new databases, the ability to store data in more performant ways than traditional databases becomes increasingly more imperative. A move to an abstraction layer has proven both empowering and agile to understand what data exists where and what the access patterns are around that data, as well as enables the ingestion of more on-premise and cloud data. Ultimately, from an executive management perspective, the value of data virtualization aligns with the ability to provide enterprise data back out for analytics and visualization, and moves away from the use of “black box” enterprise systems that can make it difficult to acquire data in real-time. One interesting note is that the reasons for choosing data virtualization as a core technology and the reasons for keeping it are not always one in the same. For example, one Colorado-based non-profit explained that they choose data virtualization to provide a layer of abstraction between data stores and users of those stores with the idea that IT would be easily able to switch out the back end with no changes required to the front end. However, while the use case remains valid, at this particular organization it turned out to be an internal fallacy as the

Overcoming Barriers to Data Virtualization Adoption

Research

Barrier #1: Choosing Data Virtualization

companyâ&#x20AC;&#x2122;s user groups all had power users who demanded access to unaltered source system data and resisted the approach. In the end, this company chose to keep data virtualization while other investments were canned (including MDM, ETL, and rapid DW development) because of its success in using data virtualization to enable both logical data warehousing and real-time data provisioning for services.

Overcoming Barriers to Data Virtualization Adoption

Research

Barrier #2: Identifying the Lynchpin Use Case

Potential data virtualization adopters are eager to understand (and learn from) the experiences of post-adopters regarding how to “get started” using a data virtualization layer – including what elements informed that decision, the resulting lessons learned, and how that initial experience has been applied to an overall data virtualization growth strategy.

“You can’t rebuild data every day, so you need the ability

With so many possible “lynchpin” use cases, many potential data virtualization adopters have expressed concern that data virtualization can seem “too

to get what you want on the

conceptual” of a concept to hone in one a single opportunity that simultaneously

fly. In a [traditional] data

justification opportunity. Specific questions from potential adopters asked for

warehouse you build all or

takes advantage of agility, is speedy to start, and illustrates a real business value further exploration into whether there have even been use cases that have been proven to be impossible to do without data virtualization – as opposed to more

you build nothing because

difficult – and if there were any specific implementation challenges (or “gotchas”)

the data is persisted there –

were addressed.

but with data virtualization

that were uncovered during a data virtualization implementation, and how these

Moving forward, once the initial hurdle of launching data virtualization has been

you can build and rebuild on

overcome, potential adopter thoughts turn to how data virtualization is expanded

the fly with data as needed.”

phased adoption and implementation. This path of thought also includes which

within the enterprise as part of a larger growth strategy and seek insights to guide implementation strategies to avoid with data virtualization, to ensure that it is

- A cable telecommunications

deployed efficiently, effectively, and properly throughout the organization.

company

Overcoming the Barrier Post-adoption companies validate the importance of the proper lynchpin use case to prove the value of data virtualization to the organization and earn executive buy-in. The following represent a sampling of data virtualization lynchpin use cases that successfully introduced data virtualization to the organization: • One media company had invested nearly $500k in the development of a physical customer data mart that was, in the end, unable to be built according to established

Overcoming Barriers to Data Virtualization Adoption

Research

Barrier #2: Identifying the Lynchpin Use Case

timeline parameters and money allocated. Because the business had already invested significant resources in the project, IT was unable to ask for additional funding and, instead, had to deliver on the requirements document without additional cost or time. With data virtualization, they were able to build virtually and deploy within three months with only supplemental professional services assistance. For this company, their lynchpin case was initiated as an opportunity for creative problem solving. • A commercial, web-based services company approached data virtualization adoption as a way to overcoming a broken data environment. Previously at this company, some databases weren’t well designed and were “bulled together” with brute force, without consideration for architecture and business rules. Because of this, additional BI tools couldn’t be added on top of the existing infrastructure and technologies already in place couldn’t be used as intended. The initial hurdle was trying to run poorly performing queries with handwritten SQL code, and initiatives in ETL helped to properly structure for analytics but still left other operational and real-time data unable to be seamlessly interlinked. Then, when the company’s contact management system began a redesign of ownership hierarchies that rendered hundreds of reports obsolete, IT was left with no time to rebuild a fragmented data warehouse. Instead, they turned to data virtualization to insulate and adapt the business as the data environment changed from underneath too fast to approach in traditional ways. • Another multinational agrochemical and biotechnology corporation tucked data virtualization adoption discreetly within a $1 million business transformation project aimed at enabling a “new start” in its business information platform. As the project rolled out, data virtualization was simultaneously deployed across the organization. The reasoning was straightforward: within any organization there is a relatively small set of data (say, 15% by this company’s estimate) that is used over and over again. This company wanted to take this highly-reusable data and make it self-serving, thereby partnering business and IT to collaborate directly to address the business’ desire to see their own data in their own language, vet definitions, and then translate that into myriad data virtualization-enabled reporting tools as a single source of extraction and value.

Overcoming Barriers to Data Virtualization Adoption

Research

Barrier #2: Identifying the Lynchpin Use Case

• Lastly, a children’s non-profit organization used data virtualization to spur a project that had been languishing for over a year to due the fact that modeling it in a traditional EDW process (staging, ETL, business model data warehousing, data view – “or worse yet, a cube” --, report) had proven far too cumbersome and interdependent. After hearing other industry success stores on data virtualization, this company was emboldened to dust off the old project, and set about to retiring its traditional data warehouse. This was replaced with a logical data warehouses that could meet organizational real-time data needs, however even this approach was still unfamiliar territory fraught with struggle. Management stepped in and helped foster organizational agreement on key terms and definitions. Ultimately, the inclusion of a data virtualization layer allowed this company to provide both operational and analytical reporting out of the same set of views, maximizing the company’s operations and business value to its charitable endeavors.

Overcoming Barriers to Data Virtualization Adoption

Research

Barrier #3: Assigning Roles and Responsibilities

As potential adopters begin to roadmap data virtualization as an integral part of data and organizational architectures, many see the identification and designation of resources and roles within the BI team as an obstacle due to their inherently competitive natures (i.e., who has the “rights” to work within the data virtualization tool and is this person a data modeler or an ETL developer? Moreover, how do they interact with the DBA?). There is the perception that creating a data virtualizationspecific team drives agility by consolidating work in a tight area with limited

“Even with those basic (and

resources, however there is a lack of clarity as to whether these roles should be isolated to IT, or whether they should be isolated within the business, as an IT

significant) flaws, we saw tremendous success and acceptance [from using

brokerage team, or as part of a cross-functional team. Separately from ownership, under the helm of roles and responsibilities, there is additional concern regarding the handling of security and governance within a data virtualization environment. With the need to drive access and visibility for

data virtualization] and soon started to suffer from

discovery, many companies wonder how to enable a governed data discovery environment that leverages data virtualization while still controlling individual access and database security.

the semi-mythological Overcoming the Barrier

‘perils of success.” Adopted companies consistently noted that they retain small, interdisciplinary, - A non-profit organization

and agile teams to fully deploy and manage data virtualization. Many companies noted a team of between five to seven people that is centralized and consists of both business and IT roles (two to three IT FTE with others part of the organizational matrix). One investment-banking firm noted that responsibility shared among multiple departments has enabled the capacity to virtually grow in a wellgoverned consistent fashion across the organization. Another non-profit noted that these roles should, to some extent, be overlapping with each member having some capacity to assist in other areas while maintaining specializations that align to the core functions of the team.

Overcoming Barriers to Data Virtualization Adoption

Research

Barrier #3: Assigning Roles and Responsibilities

Specific core roles and responsibility main responsibility is to ensure the server remains available and accessible with no downtime to correctly Application Administer meet incoming demands, honor business service-level agreements, and anticipate future demands

Governance Architect

Business Information Owner

Developer

takes ownership of decide what should or should not be built as virtual objects â&#x20AC;&#x201C; and then, how to build it â&#x20AC;&#x201C; within the abstracted environment to sustain a sharable, consistent model that provides proper business semantics across the organization work in tandems with the governance architect to avoid a purely technical approach and retain the business value of data virtualization acts as the single point of authority to vet and implement proposed changes within the data virtualization environment as well as builds, optimizes, and tunes the virtual objects

Most data virtualization teams also have a non-technical data virtualization Champion that actively communicates with executive management to champion data virtualizationâ&#x20AC;&#x2122;s value and inclusion as a core technology within other areas of the business. As a measuring point for data virtualization adoption within the organization, some companies have also launched an Integration Competency Center, too, to leverage the small data virtualization team for development and training across the larger organization. Communication, community, and awareness are essential, especially in areas of governing the abstraction layer itself, which remains a moving target as both the role of data virtualization and the business process of discovery continue to mature in the organization.

Overcoming Barriers to Data Virtualization Adoption

Research

Barrier #4: Selecting a Vendor

Finally, potential data virtualization adopters seek to cut through the marketing of vendors and understand key differentiators between data virtualization vendors in the space today. Additionally, potential adopters hope to navigate beyond the vendor marketplace and benefit from lessons learned and best practices of adopters that have already had the opportunity to evaluate competing vendors and have had the opportunity to learn from hands-on, already in-use experience with chosen tools. Further, they would like to insight on how the evaluation and selection of a data virtualization vendor applies relative to the total tool and vendor

“We went with the

landscape and technology architecture, how it has performed since its adoption, and how this affects the ongoing roadmap of the organization.

[data virtualization] As a vendor differentiator, beyond product capabilities has been a distinct interest

approach because the environment was changing out from underneath way

in how vendors support ongoing training and learning events for users, including the performance and responsiveness of vendor technical support. Additionally, because potential adopters recognize that there will be a learning curve inherent in adopting a new technology, they are interested in the level of training and support services available, what has proven to be the most useful and invaluable

too fast to approach in traditional ways.” - A commercial, web-based services company

for companies – and its technical and non-technical users – that are new to data virtualization.

Overcoming the Barrier Key differentiators when evaluating data virtualization vendors, according to those companies who have already undertaken this journey and made a buying decision, can be distilled into three areas: vendor/product maturity, industry impact, and roadmap; product technical capabilities; and level of vendor support and services. Vendor/Product Maturity and Impact Across Industries Time and time again, adopting customers remarked the importance of seeing a vendor client-base that resonates with their respective industries as a measuring stick by which to weigh the vendor’s strength and permeation amongst its competitors. Another indicator of vendor/product maturity is its time in the market

Overcoming Barriers to Data Virtualization Adoption

Research

Barrier #4: Selecting a Vendor

and longevity with proven customer use cases. Further, adopting customers looked for synergies in use cases that showcase product maturity and illustrate how vendors have handled challenges trending in the market. Ongoing transparency in where the vendor has been and where they are going proves just as important to adopting customers, noting that the vendor roadmap is critically important to the ongoing success of data virtualization. Adopting customers also desired a vendor that allows users a voice into the product development roadmap, and one that actively looks to incorporate emerging technologies (such as the Cloud) to expand and refine existing capabilities. Product Technical Capabilities Of course, how the technology works and what it is capable of is a huge differentiator during data virtualization vendor comparisons. In addition to the distilled list of key differentiators below, price and total cost of ownership is, of course, always an important factor in the buying decision. •

Ease of use (User interface; allows development using SQL)

•

Availability of source system connectors (Databases, Web APIs, RESTful APIs)

•

Ease of ingesting new types of data sources

•

Ease of accessing data output (ODBC, JDBC)

•

Support for complex caching needs

•

Availability of clustering for redundancy

•

Ease of deploying changes to production environment

•

Visibility into execution plans

•

Scalability (a big concern in large enterprises -- DV strategic architecture)

•

Extensibility (ease of calling into and out of the DVL)

•

Impact analysis, or ease of identifying lineage both backward and forward for any entity or attribute at any level (i.e. sources, source abstraction layer, middle/conforming layers, demand layer, and publications)

•

Performance monitoring

•

Object level security

•

Optimization engine -- ability to create high performance execution plans

•

Pass-down optimizations

•

Audit logs for reviewing user access for governance and compliance

Overcoming Barriers to Data Virtualization Adoption

Research

Barrier #4: Selecting a Vendor

Level of Vendor Support and Services Lastly, adopting customers chose a vendor that not only has proven successful and has the right capabilities and roadmap to help customers continue to be successful, but for one that takes the initiative to help their customers actively succeed. This is measured in terms of both responsive technical support in helping to investigate and resolve issues, as well as the level of vendor training provided to the customers that minimize the learning curve and helped to leverage existing internal knowledge. Finally, customers noted that professional services offered by the vendor would be helpful as they work to develop the right data virtualization environment, particularly at the onset of implementation. This is primarily because customers noted that they still struggle with what goes into a classic layer and what goes into a data virtualization layer, and how these complement each other. One customer called this balance between access and ownership a â&#x20AC;&#x153;political turf warâ&#x20AC;? and said that governed data virtualization requires a large cultural change, as it is difficult to get true ownership, especially pertaining to semantic definitions.

Overcoming Barriers to Data Virtualization Adoption

Research

Conclusion

Data virtualization is a mature technology that has evolved with the data management industry. However, while nearly every major research and consulting firm delivers architectural blueprints that include a logical business semantic layer, only some companies have implemented that component -- and even fewer have enterprise-wide â&#x20AC;&#x201C; leaving data virtualization the often-missed piece of an overall unification strategy that includes incumbent, traditional ETL and data replication. As a technology, data virtualization in itself isnâ&#x20AC;&#x2122;t a silver bullet, but when implemented with governance and a rich methodology, it solves the bigger problem of managing data volatility and addresses many pain points of business and IT. The reality today is that, without a robust consolidation strategy, business and IT simply cannot keep up with the volatile landscape of surging data. More and more, integration is becoming a lackluster blanket integration strategy: even if you manage to do it, it invariably wonâ&#x20AC;&#x2122;t last. Instead, a data unification strategy that adopts data abstraction through data virtualization as a key integration strategy solves data volatility and offers very immediate benefits to alleviate the pain points of the organization, and maximizes the value of integrations whether physical or abstracted. To achieve the benefits of data virtualization, companies need to take the leap. So, while barriers exist, they can be easily overcome -- many have done so already and are achieving these benefits today. This research is a guidebook, providing a living conversation from both sides of the adoption table, and intended to assist potential adopters in overcoming barriers by leveraging the insights and lessons learned of those who have already moved forward. After overcoming barriers to data virtualization adoption, the next step is to plan this technology as part of a long-term data integration strategy. Therefore, the same discipline and diligence that is put into physical data integration should be applied into the abstracted environment. Seek to quantify the reduction in both development time and time to deployment leveraging data virtualization, and, once these are quantified, double the benefit quotient by moving beyond the immediate, localized benefit and looking farther to drive larger, enterprise-wide strategic benefits. Overcoming Barriers to Data Virtualization Adoption

About the Author Lindy Ryan, Research Director, Data Discovery and Visualization, Radiant Advisors As Research Director for Radiant Advisors’ Data Discovery and Visualization practice, Lindy leads research and analyst activities in the confluence of data discovery, visualization, and data science from a business needs perspective.