Smart Management for Virtual Infrastructures™
Introduction Organizations have begun adopting virtualization technology at a rapid pace. With the ability to quickly and flexibly deploy virtual x86 machines onto standardized hardware platforms, managers plan to realize a number of goals. However, unless managers ensure they incorporate three critical capabilities into their virtual infrastructure management suite, they will likely be disappointed. The Move To Virtualization
For almost all organizations, today’s principal virtualization focus is cost savings. With virtualization, and its intrinsic ability to manipulate and move individual virtual machines, managers reduce the number of physical servers they must buy and achieve greater utilization of the servers they have already installed. However, a growing number of IT executives Virtualization is the highhave also discovered that virtualization serves as a solid est impact thing changfoundation for a wide range of advanced IT initiatives, ing infrastructure and including green IT, real-time infrastructure, and more. Unfortunately, after deployment, most companies find they don’t achieve the benefits they expected - either strategic oIntror tactical - because they haven’t addressed the three central requirements for successful management of virtual machines:continuous visibility and intelligence; real-time policy management; and automation.
operations through 2012 - changing how you manage, how and what you buy, how you deploy, how you plan, and how you charge. - Thomas Bittman, Gartner
These tasks are complicated by the fundamental changes that virtualization brings to IT management tasks. For example, in the past, systems could be reliably targeted and tracked using hard-coded identifiers like a serial number or MAC address. With virtualization, those values are
subject to change at the whim of an administrator. The many new states introduced by virtual machines add more complexity - physical servers run continuously, and can be reliably queried by management agents to retrieve status and configuration data. Virtual servers can be paused, placed offline, and even moved, placing vital data out of reach of management systems. Virtual machine snapshots add even more complexity to the task by requiring analysis of the many different versions of a system which can exist at any point in time. Reverting to an older snapshot can create a literal disconnect between the actual configuration of a VM and data previously captured by management agents. It’s already estimated that 60 percent of all operational problems result from configuration errors of one kind or another. Keep in mind this failure rate doesn’t reflect the challenges of virtualization which, as demonstrated, are almost guaranteed to make the situation worse. It’s clear then that if organizations are to fully realize the tactical and strategic benefits of virtualization, they must master configuration management within this new and different environment.
The initial benefits promised by virtualization are largely driven by cost savings. By consolidating multiple systems on a single physical machine, organizations save in many ways. The upfront capital expense required to purchase servers is reduced, as are ongoing space, power, and cooling costs. Important as those savings are, as IT staffers become more experienced with virtualization, a broad range of new initiatives - collectively dubbed “Virtualization 2.0” - has emerged, promising significant new strategic benefits. These include the ability to more quickly address new market opportunities, ensure reliable high availability and disaster recovery capabilities, the implementation of real-time infrastructure, and many more.
A Flood of VMs
Because of its potential for delivering both cost savings and increased organizational agility, virtualization has become wildly popular. But not all virtualization technologies are the same - and each technique brings a variety of configuration management challenges.
This paper describes these new and unique challenges, explains why traditional systems fall short in managing the virtual environment, and outlines the capabilities managers should evaluate when management tools for the virtual environment.
The most common form of virtualization today consists of the migration of existing physical servers, intact, to a virtual form. Since straight P2V - physical to virtual - conversions maintain unique identifiers associated with a machine, the configuration management impact may initially be slight.
The Promise of Virtualization
Problems almost immediately arise, however, when servers are created directly in virtual form, or when existing virtual machines are used as the basis for clones or copies. With no means to enforce configuration policies, arbitrary names and identifiers can be selected. Wholesale copies of existing systems can easily in result in duplication of critical attributes, confusing management systems.
Virtualization is the decoupling of physical computing resources (such as processors, storage, and peripherals) from logical resources (including the operating system and applications). This decoupling allows for multiple virtual servers to run within a single physical server. These virtual machines (VMs) appear and operate like physical servers to applications, to users and to the network - even though they exist only as a partition within a physical server.
The increasing movement toward downloadable virtual appliances poses a quandary for IT managers. Virtual appliances are preconfigured virtual machines which offer readymade solutions for implementing messaging and collaboration systems, CRM solutions, and many other applications. On the one hand, the hundreds of virtual appliances now available offer free or low cost solutions for vital IT applications. But managers have no way to examine the systems for compliance with IT standards and corporate dictates, since they are often based on sometimes obscure OS distributions that are unsupported by management solutions. Managers, then, must select between a cost-effective business solution and the risk associated with bringing an unmanageable system onto the production floor. Yet another form of virtualization, which optimizes operating system or run-time environment components, also serves to vastly complicate configuration management and those IT disciplines that depend on configuration data. These tools take two general approaches. In some cases, existing virtual machines are examined and non-essential code and sub-systems are stripped away. In other cases, the optimization takes place at compile time when a virtual machine containing only those components needed to support a specific application is created. Most existing configuration and change management systems are structured to expect a standard system configuration - a golden or standard image. With these new approaches, it’s possible for each and every
system to be unique. Trying to identify and target systems in such an environment is well beyond the scope of existing tools, hopelessly complicating patching, testing, and other normal application deployment functions. The portability of virtual machines - which often are compact enough to be stored on a USB drive - along with workstation implementations of virtual host software, bring other challenges. Combine a downloaded virtual appliance with desktop or laptop virtualization software, and you have recipe for havoc as individual users start up full-blown server instances at their desks. These virtual systems are easily installed without the knowledge of IT managers or administrators. But each offers the potential for conflict with existing network services, such as routing or addressing. Further, each of the security and compliance risks posed by VMs on servers – such as outdated, un-patched or unauthorized software – are also posed by virtual machines on the desktop.
The Real Challenges of Virtualization
It’s clear these challenges present real and significant cost; security and compliance; and performance and reliability issues to managers attempting to deploy virtualized systems. Given that, it’s not surprising so many organizations will fail to realize their tactical and strategic virtualization goals. For a variety of reasons, the full realization of the differences between managing physical and virtual systems has been slow in coming.
Unlike traditional server farms - which, for all their dynamism, are physically relatively static - virtual machines are marked by movement. It’s all a bit like hordes of brown boxes speeding along an indefinite number of conveyor belts in a shipping center. Given emerging trends in application delivery and deployment, the analogy is especially apt. As already discussed, organizations are turning more often to virtual appliances to satisfy application and system needs. And virtual machines, rather than code or installers, are also becoming the default container in which applications are moved between phases of internal development and test processes. Like the conveyor belt of boxes speeding by, from the outside it’s impossible to know what application or function a given virtual machine performs; what version of an application it represents; what other components (such as middleware or databases) it contains; who developed it; whether and how completely it has been tested; which other applications it was derived from or are based on it and what possible impacts the contents of the box will have on other processes within the organization. Identifying and tracking these containers is difficult on multiple levels. By its nature, virtualization eliminates much of the paper trail - invoices, work orders, change orders, and the like - normally associated with systems. In our shipping center, we take for granted the existence of unique identifiers, attached via bar code, that allow us to locate and identify a specific box anywhere in the system. As already noted, with virtual servers it’s possible to arbitrarily change attributes now routinely viewed as unique system identifiers, such as a serial number, MAC address, or other characteristic. Because virtual machines are not anchored to a single physical machine, and can be moved on demand, it’s also helpful to understand where a virtual machine has been in addition to where it is now. Like an airplane’s black box, if
important configuration or operational events are recorded, troubleshooters will have a far more productive experience when trying to establish “what went wrong” with a recalcitrant virtual machine. Virtualization completely overturns the conventional life cycle in which applications once moved as a single, coherent, body of code from development through build, test, production and maintenance. Now, multiple versions of applications – comprised of any of a variety of software components – can move into and out of production at the whim of their owners, yielding a myriad number of possible permutations of virtual machine contents. Just like in the physical world, configuration management serves as an essential foundation in the virtual environment for critical systems management disciplines including change management, problem determination, service level management, and many more. However, as we’ve seen, virtualization increases the complexity of the configuration management effort by at least an order of magnitude. Overcoming that challenge, and ensuring accurate and effective virtual configuration management, requires three essential capabilities - continuous visibility and intelligence about the virtual environment; real-time policy-based management, and automation.
Virtual Management Requirements
of versions of specific virtual machine.
Continuous Visibility and Intelligence
Scanning tools capable of examining virtual machines in all states, and in the broad range of formats (both virtual containers as well as guest operating systems) encountered today.
Achieving continuous visibility requires having an up-to-date and accurate understanding of the configuration of virtual machines in the environment - including new systems introduced as virtual appliances, or copied from portable media. As noted, this is a particularly complex challenge within the virtual environment - some machines may be offline or paused, and out of reach of traditional agent probes. The diversity of guest operating systems is another complicating factor, as are VM snapshots which expand the potential number
At a minimum, overcoming these challenges requires three capabilities:
Ability to compare and identify changes in configuration, both for a grouping of virtual machines, as well as with respect to a specific machine over time or in contrast with a desired baseline configuration. Because existing virtual machines so often serve as the basis for new systems, it’s also
critical that configuration management solutions understand and report on the genealogical relationships between specific system instances. While accurate configuration data is a critical requirement, simply overloading an administrator with raw data can leave them confused and overwhelmed, or even mask important issues that need to be addressed. To transform this raw data into actionable knowledge, a virtualization management product must provide information in an easyto-use and understand interface, and provide context, insight and prioritization of information. Required features include:
Real Time Policy-Based Management The fluid nature of the virtual environment dictates configuration management solutions provide more than just scanning and reporting of configuration data. Consider a case where an administrator attempts to start a virtual machine which lacks a critical security patch - starting the system creates significant risks to the integrity of the production environment, and may well endanger the organization’s compliance with organizational and legislative mandates. It’s essential the configuration management system have the ability to respond to - or to refer to other management systems - emerging issues and problems. A basic set of capabilities would include:
‣
‣
‣
‣
‣
The ability to assess and classify virtual machines based on their attributes (operating system, enabled accounts, applications installed, patch levels, and many more); organizational factors (service levels, department, application or system, etc.), and even individual factors (i.e., “my virtual machines”). Visualization of relationships and dependencies among virtual machines, including their genealogy, affinity with specific hosts, and other links.
‣
‣
Customizable and extensible reporting that highlights potential problems and emerging trends. A long-term database that maintains configuration data about the environment, hopefully supporting or federated with existing CMDB’s or asset management systems. Alerting and alarming capabilities supporting both traditional approaches - SNMP traps, system and log messages and alerts, and the like - as well as more modern techniques such as the publication of RSS feeds containing information of interest to an individual or other system.
‣
The ability to evaluate and implement policy checks at key points in the creation and operation of a virtual machine - when created, cloned, or copied; at start-up or shut-down time; and prior to sensitive operations such as relocation of the VM from one host to another. Flexible policy definition, with the ability to incorporate system attributes such as the operating system of a virtual machine; organizational attributes like service levels or departmental affiliations; and the current state of the virtual machine. Adaptive policy application, to ensure only relevant policies are applied. For all but the most simplistic environment, automatically selecting the proper policies to apply in a given situation is a complex undertaking. For example, different policies may come into play at different points in time, or in response to different events. Policies may need to be synthesized to address the requirements of different parts of the organization (operations, security, business units, etc.) and it’s likely that conflicts
between those policies will require mediation. An effective policy engine must be capable of resolving these questions in real-time. •
Support for a broad portfolio of responses to policy failures - should an action be prevented? Allowed, but with a warning to an operator? Ignored? The policy engine should also be capable of integrating with other management systems. For example, an organization may already have defined and implemented the desired workflow for applying and validating software patches. If an un-patched system is discovered by the configuration management system, it should have the ability to invoke the existing automation.
Integration with Existing Tools and Processes Most organizations which have developed IT infrastructures of any size or sophistication have also developed processes to manage that infrastructure. These processes are often built up over years around critical business functions, or regulatory and security requirements. They specify how, for example, new servers will be created and old servers will be disposed of; how user access rights will be created, modified or revoked, and how security patches or updates will be tested, applied and recorded. These processes, increasingly based on ITIL or other industry best practices, are often integrated into IT management software or workflow tools which the organization has invested in over the years. Managers would be foolish to reinvent the wheel by spending
the time and money required to recreate these processes simply to accommodate a virtualized environment. Similarly, many companies have already implemented complex and costly management systems that maintain and report on critical systems. For all its differences, it’s unlikely and unreasonable - to expect that administrators, managers, and executives will suddenly find it acceptable to begin accessing a specialized system just to understand the status of the organization’s virtual infrastructure. These realities dictate that investments in new tools for addressing the unique requirements of the virtual infrastructure be capable from the outset of integrating with and sharing functionality across the existing management infrastructure. One critical area for integration relates to the wealth of volatile data a virtual configuration management system gathers during operation. As already noted, access to this information should be supported on an information sharing or federation basis. More transactional interactions - such as providing information about a specific resource - must also be enabled. This integration is especially critical for organizations attempting to implement a Configuration Management Database (CMDB). The fast-moving virtual environment will likely result in high levels of latency with respect to information updates. The ability to federate data, with the virtual configuration management system providing a trusted source of configuration data about the virtual environment, mitigates these challenges.
However, besides reading and writing data to and from the CMDB, a virtualization management tool should be able to exploit all the CMDB’s capabilities, such as the ability to search for configuration items, explore the relationships among those configuration items, and to view additional information in the CMDB such as details about business processes and users’ identities. An additional layer of integration lies with the processes and workflows embedded within existing management systems within the enterprise. These might include, for example, asset management, change control, access control, service ticket tracking or diagnostic systems. As with data integration, this functional integration must be bi-directional. The virtual configuration management system should be able to respond to requests from other systems, providing insight into the virtual elements of the IT infrastructure. And, as suggested, the system should also be able to invoke previously defined workflows to respond to configuration events as they arise. Fortunately, a variety of standards and technologies exist today that support these critical integration requirements. APIs exposed via web services provide an easy avenue for coordinating data among multiple applications. For individuals, web-based mashups and applets can be used to enable reports, graphics, and data to flow to alternative reporting systems. Standards like RSS can easily be exploited to provide a flow of data to a broad range of applications and devices, ensuring ready access to critical data.
Enterprise Virtualization Manager™
Smart Management for Virtual Infrastructures™ ManageIQ's Enterprise Virtualization Management™ (EVM™) Suite provides the insight, control, and automation capabilities organizations need to effectively manage their virtual infrastructure. ManageIQ products are designed specifically to address the challenges inherent to dynamic virtual environments EVM Benefits ‣
Agent-free technology increases discovery efficiency and minimizes risk.
‣
Secure, fine-tuned virtual appliance solution enables rapid implementation and quick timeto-value.
‣
Scalability ensures ample capacity exists to meet growing virtualization management demands.
‣
Integration with existing enterprise infrastructure protects existing investment, supports IT processes.
‣
Cross-platform support allows flexibility in virtualization technology choices.
‣
Role-based administration and management enables enforcement of access and functional controls.
EVM Technology and Products The EVM suite of products provide comprehensive management of virtual assets. They provide deep visibility into virtual machines - even when they're offline or brand new, capture the interdependencies of virtual components, enforce controls over VM execution and operations, and easily integrate with existing management systems and processes.
The EVM Suite is comprised of: EVM Insight™ establishes a foundation for control and automation of virtualized environments by discovering and maintaining accurate configuration information, unique identification, genealogy, and relationship mapping for virtual assets. Comprehensive information and analysis is provided for hosts, virtual machines, virtual appliances, VMware® VirtualCenter instances, storage and network elements, operating systems, applications, patches, and accounts. EVM Control™ provides real-time, policy-based management, security, and compliance controls over virtual assets. These policy-based controls, applied against virtual assets at key operations and configuration life cycle points, ensure the enforcement of IT standards, improve reliability and availability, and reduce risk. The EVM suite leverages advanced technologies that deliver the scalability, security, ease of use, and integration required to support enterprise-level virtual machine deployments. Agent-free, non-invasive architecture - dramatically reduces risk and time-to value compared with agent- and driver-based systems, and reduces operational complexity. The architecture enables broad support for virtual machines and appliances due to the absence of an OS-specific agent. Comprehensive configuration management capabilities, using patent-pending SmartState™ technology enable the combination of real-time discovery, analysis, policy-based control of virtual machines across all states even offline or paused. Easy-to-use console - leverages powerful Web 2.0 technologies - including RSS, AJAX, tagging, and mashups - to support information sharing and collaboration. Provides a rich set of out-of-the box reporting, visualization, and analytical capabilities to simplify management. Role-based ac-
cess controls limit console access, reporting, and views, with all activities logged for auditing purposes. The EVM Virtual Management Database™ (VMDB™) maintains configuration and relationship information about the virtual infrastructure, supporting change and release management processes. The VMDB provides a definitive and trusted source of virtual configuration information, and can be federated with a Configuration Management Database (CMDB) or other asset management databases.
About ManageIQ
ManageIQ (www.manageiq.com) is the emerging leader in the management and automation of virtualized computing environments. ManageIQ solutions are designed and built for the unique challenges introduced by virtualization and enable IT organizations to realize the benefits of virtualization while achieving higher levels of insight, control, and automation. ManageIQ was founded by a team of technology visionaries formerly of Novadigm, where they spent over a decade providing configuration management solutions enabling hundreds of enterprises to securely, efficiently, and effectively manage millions of computing devices. The ManageIQ Team has a strong track record of partnering strategically with customers to ensure rapid return on investment and long-term value. ManageIQ is a VMware® Technical Alliance Partner, and a Citrix Global Alliance Technical Partner.
ManageIQ One International Boulevard, Mahwah, New Jersey, USA Phone 201.962.3388 Fax 201.962.3236 info@manageiq.com
Š 2008, ManageIQ, Inc. All Rights Reserved