A White Paper from the Experts in Business-Critical Continuity™.
Storage Strategies for AdvancedTCA Platforms A Four-tier Storage Approach
Brian Carr, Strategic Marketing Manager September 2008
Storage Strategies for AdvancedTCA Platforms
Table of Contents 3 Introduction 3 Storage Primer 4 ATCA Applications Requiring Storage 4 System Controller Storage Application 4 Database Storage Application 4 Content and Streaming Storage Application 4 ATCA Storage Topologies 5 On-blade Storage 5 Direct Attached Storage (In-shelf) 5 Direct Attached Storage (External) 5 Shared Storage (In-shelf) 6 Shared Storage (External) 6 Storage Application Solution Matching 6 The Future of Storage 6 Summary
2 Storage Strategies for AdvancedTCA Platforms EmersonNetworkPower.com/Embedded Computing
Introduction In order to reduce costs, a growing number of companies are trying to limit the number of different platforms on which they base applications. Each platform type requires a different tool chain for development, a different supply chain to manage and different processes to deploy and maintain in the field. Increasingly, telecom equipment manufacturers (TEMs) are implementing common platform strategies, where their goal is to minimize the number of platform types, yet still support the breadth of elements and applications required by the service providers. AdvancedTCA® technology has been established as one of the key components of this approach — an industry managed, open standard with the required performance, manageability and availability characteristics required by the telecom industry. This should not be surprising, as all the major TEMs were involved in the specification of ATCA®, and now, most if not all Tier 1 TEMs have some form of ATCA based common platform initiative underway. Adopting a common platform approach benefits the industry by increasing the number of applications that can be developed for that platform. For example, initial deployments of AdvancedTCA based elements were focused on signalling and control plane applications such as SIP-based softswitches and radio network controllers. Improvements to ATCA switch blade technology have increased the application scope to include data plane and packet processing elements. In addition, we are now seeing a trend to push the common platform into more server-based applications such as billing engines, authentication databases, service delivery platforms and IPTV content caching. These applications are much more diverse than other network element classes, and require a more flexible approach to data storage. Storage can be complicated. It seems that every application has a different mass storage requirement and each development organization has different ideas on what topologies are acceptable. This means that there is no “one size fits all” storage solution. The approach adopted by storage providers for the enterprise server market is to provide a tiered storage strategy where the customer can choose the right capacity, performance, topology and price point for any given application.
This white paper explains that although there are many different applications for systems based on AdvancedTCA server blades, it is possible to simplify them into a small number of storage application classes with similar needs, and propose a tiered storage strategy that can be employed. Finally, it gives recommendations for narrowing the choice for each storage class. First, though, it provides a quick primer for the various terms used in storage discussions and identifies their specific relevance.
Storage Primer The purpose of any storage product is to reliably store and to provide access to such data that is required by an application in a timely and cost effective manner. This storage is normally implemented using hard drives (rotating magnetic media), although solid state drives (SSD) using flash memory technology are becoming rapidly more popular as their capacity increases and cost decreases.
Scalability can be discussed in two balancing dimensions: capacity and performance. You can scale capacity by increasing the number of Drive Form Factors drives in the storage array. Increasing For AdvancedTCA in-shelf storage, the lower profile the speed of the drives and/or 2.5” drives are the most popular since they can be number of CPU blades that access mounted on ATCA blades or AdvancedMC™ (AMC) the storage array scales application modules with hot swap capability. More traditional, performance. The performance of 3.5’ drive form factors are also available for use with the storage array is usually governed ATCA products. by the type, interface and spindle Drive Technologies speed of the drives used, and the SAS (serial attach SCSI) drives are the benchmark — the amount of cache that the application most reliable and highest performance hard drives can utilize. available with MTBF figures of more than 1.2 million For example, for certain applications, hours and high spindle speeds of 10K and 15K RPM. One drawback of SAS drives is a limit on case tempera- it may be more beneficial to fit more ture that often leads to difficulties cooling the drives at drives of smaller capacity than fewer the 55ºC, NEBS environment. In most cases, SAS drives drives of a larger capacity. The application characteristics determine are fairly expensive when compared to other drive how capacity and performance are technologies. related. SATA (serial ATA) drives are a lower performing and Application characteristics also lower cost option to SAS drives but are not usually dictate how reliable the access and considered reliable enough for carrier grade equipretention of data must be. A typical ment. However, some vendors produce “extended” or telecom application requires no “extreme” versions of their SATA drives and in some single point of failure in the overall cases are acceptable for carrier grade equipment. architecture and requires data to be These drives can therefore be used for applications accessible at all times. Of course, it where reliability and cost is critical, but performance would be best if no drive failures ever can be sacrificed. took place, but unfortunately a hard SSD (solid state drives) are of limited capacity and exdisk drive is usually the most pensive, but are high performance, reliable, low power unreliable component in a system. and can work in high ambient temperatures. They are Therefore, steps must be taken to becoming popular when capacity needs are low, or protect against failures within a when performance is required and SAS drives cannot be storage array. used.
3 Storage Strategies for AdvancedTCA Platforms EmersonNetworkPower.com/Embedded Computing
Storage Strategies for AdvancedTCA Platforms
This is usually achieved by means of RAID (Redundant Array of Independent Disks) technology coupled with the ability to hot-swap failed drives to get the storage array back to full operation as soon as possible. RAID can be implemented in software or hardware using dedicated RAID controller chips. In general, more sophisticated RAID implementations are best done using hardware RAID controller.
Content and Streaming Storage Application The content storage application is used to provide access to media files and other related user storage. Examples include video mail, audio ring tones and local cache files for IPTV. A special challenge for this application would be to predictably deliver very large digital media files to customer endpoints.
Very large capacity needs are common, and read performance must be consistent to avoid stuttering and glitches of video and audio The challenge then is how to extend the common platform in a cost- files. This application could benefit from active data management effective way that still expands the applicability to these new areas. where the least-accessed data is treated differently from the mostaccessed data. In the case of a streaming One answer is to better classify the application, there will be a lot of parallel Standard RAID Levels applications, and here it is possible to RAID 0: Striping — data is stripped across multiple shared access to the storage array. break them down into fundamental drives to increase performance and capacity. No storage application classes (on-blade protection is present, so data is lost in the event of ATCA Storage Topologies storage, direct attached storage and a drive failure. shared storage). As previously stated, the enterprise storage industry approaches the problem by RAID 1: Mirroring — the same data is written to These storage application classes have a providing a tiered storage solution set, two (2) drives in parallel. In this scenario, if one narrower range of storage capacity, capable of scaling between very small and connection and performance criteria, and drive fails, data can still be accessed, but has the very large, and the same approach can be draw-back of reducing the capacity in half. therefore can be used to narrow the taken for AdvancedTCA based systems. selection of storage technologies for the RAID 1E: Mirroring — This is the same as RAID 1, AdvancedTCA common platform. A challenge for the AdvancedTCA platform, but can be implemented with more than two (2) not present in the enterprise, is the drives. System Controller Storage environment in which they operate. The Application requirements for central office equipment RAID 5: Parity — combines three or more disks in include the requirement to run for 96 hours such a way that if one fails, data can still be acThe purpose of system controller storage at 55ºC without failing. This can impose cessed via the operational drives, although any is to hold and provide boot images, host further failures would result in data loss. Capacity additional limitations on storage options. the root file system and provide storage is reduced by one drive. for status and log data. The system drive Another specific choice to make relates to also hosts configuration data that is keeping all components in-shelf versus RAID 6: Enhanced parity — protects against two subject to change from time to time. connecting to separate external (off-shelf) drives failing rather than just one, but capacity is Capacity needs are usually quite limited, reliable storage arrays such as those reduced by two drives. and performance is usually not critical. available from EMC. In-shelf storage keeps
ATCA Applications Requiring Storage
Nested RAID
everything within the same reliable, managed enclosure as the blades it serves, sharing power and cooling and minimizing cable connections. However, the bladed form factor of AdvancedTCA imposes RAID 10 (1+0): Stripe of mirrors — performance of scalability limitations if reliability is to be Database Storage Application maintained, and the small drives used lag striping with the reliability of mirrored drives. behind the larger enterprise SAS and Fiber Minimum of four drives and useable capacity is Typically used with a database application Channel drives in both performance and like Oracle, a database storage application half of the number of drives. capacity. However, both implementations is used to store customer and session data, RAID 50 (5+0) or 60 (6+0): Stripe of RAID five or six have their place in the tiered storage and can be used to create billing engines. parity protected drive sets. Useable capacity restrategy. Capacity needs are moderate (up to 4 duced by the parity drives for each striped set. Terabytes) but random access read/write There are several ways to set up the storage performance and particularly write latency RAID 51 (5+1) or 61 (6+1): Mirror of RAID five or topology for ATCA based platforms. are critical since the system must six parity protected drive sets. Highest fault tolerguarantee the security of any data written ance, but with lower capacity and expensive to to disk. implement. A special challenge for the system controller is logging data. These records are expected to be continuously up to date in order to provide good post mortem data after a system failure.
To gain performance and/or additional redundancy, standard RAID levels can be combined to create hybrid or “nested” RAID levels. The most common are:
4 Storage Strategies for AdvancedTCA Platforms EmersonNetworkPower.com/Embedded Computing
On-blade Storage
Direct Attached Storage (External)
On-blade storage is characterized by ATCA CPU blades with at least one drive. Blades targeting server applications with larger storage requirements can be configured with two drives and potentially an associated SAS controller. If required, additional storage can be added via AMC expansion. By using hot swappable drives, it makes it easier to perform field service and replace defective drives. It is also possible to add a storage drive directly to the blade by using a rear transition module (RTM), a companion blade fitted into the rear of the ATCA platform.
For applications that require large storage capacity, yet access from only a one or two CPU blades, a simple cost-effective external storage array can be utilized. This can be implemented using either external SAS or Fibre Channel network connections.
Shared Storage (In-shelf) By using in-shelf, shared storage, developers can provide access to common storage from multiple CPU blades using the ATCA fabric interface via iSCSI protocols. The overall storage capacity is the same
ATCA CPU blades can use SATA and SAS drives as well as SSDs. As previously stated, SAS drives may run into thermal challenges given the environmental constraints within which most telecom equipment is subjected.
Astute Caspian iSCSI Blade1
as using a JBOD blade — with up to four hot swappable drives plus the capability for adding two more via RTM. It is also possible to extent the storage capacity by daisy-chaining a JBOD blade off a shared storage blade.
Emerson Network Power ATCA-7150 Processor Blade
Direct Attached Storage (In-shelf)
One advantage of using a shared storage architecture is the ability for multiple host CPU blades to access storage data without complicated, external SAS cabling. In addition, shared storage blades usually provide sophisticated RAID controller and iSCSI accelerator chips that off-load the CPU blade processing element.
Directly attached storage makes it easy to significantly increase overall data availability. By using ATCA bladed drive carriers, developers can attach additional drives (usually 2.5” form factor), via a cable or by utilizing the ATCA update channel. It is possible to attach up to four individually hot swappable 2.5” drives to the front blade. By using an RTM, two additional hot swappable drives can be attached as well. By using a JBOD (Just a Bunch of Disks) configuration as shown at the right, these drives can be daisy chained to other blades as well. In the direct attached storage topology, the CPU blade would handle RAID management. It is also possible to configure the JBOD blade to enable dual CPU blade access, but no more than two blades can have access to the JBOD storage array. SanBlaze AMC Carrier-based JBOD Blade2
5 Storage Strategies for AdvancedTCA Platforms EmersonNetworkPower.com/Embedded Computing
Storage Strategies for AdvancedTCA Platforms
Shared Storage (External)
Summary
External shared storage provides shared access to common storage data via either Fibre Channel or Ethernet/iSCSI. The advantages include the ability to support additional CPUs by adding a Fibre Channel switch. It also enables developers to use larger 3.5� form factor drives. By using extremely sophisticated RAID controllers and management software, external storage arrays can divide up large storage capacity and share it among many CPU blades. Using this architecture, external storage capacity is almost infinite.
As ATCA platform developers expand beyond telecom and begin to target new markets such as enterprise and other server-based applications, data storage and data management becomes more and more important. ATCA platforms offer a wide range of options for incorporating storage — from moderate, on-blade solutions to large-scale, external storage and SAN networks.
EMC Clariion AX43
Storage Application Solution Matching If a system controller application is being developed, an on-blade storage solution using SAS, high temperature SATA or SSD is usually more than adequate. For large database storage applications, direct attached or shared storage solutions are recommended depending on the number of CPU blades that require access to the storage array. JBOD blades can be used for applications requiring single/dual CPU blade access, shared storage blades can be used for applications requiring multiple CPU blade access. For extremely large content supply applications, the best option is an external, shared storage solution that can use multiple Fibre Channel connections and/or large capacity, high RPM SAS drives.
The Future of Storage With the increased popularity of video and digital graphics, combined with a never-ending requirement for data storage and records, the demand for storage in the enterprise and telecom markets is exploding. Storage is one of the most exciting and dynamic spaces in the IT, telecom and communication industries. Every year, there are increases in overall capacity; hard drive speeds and the number of bits that can be stored per square millimeter. Reliability is improving — MTBF numbers are increasing. As a result, both hardware and maintenance costs are dropping. In addition, with the nonstop evolution in solid state drives (SSD), continually improving capacity and speed, while cutting costs, also offers new and exciting opportunities for integrating storage with the high performance and availability provided by ATCA based systems.
Photo provided by Astute Networks. Photo provided by SANBlaze Technology, Inc. 3 Photo provided by EMC. 1 2
6 Storage Strategies for AdvancedTCA Platforms EmersonNetworkPower.com/Embedded Computing
About Emerson Network Power Emerson Network Power, a business of Emerson (NYSE:EMR), is the global leader in enabling Business-Critical Continuity™. The company is the trusted source for adaptive and ultra-reliable solutions that enable and protect its customers’ business-critical technology infrastructures. Through its Embedded Computing business, Emerson Network Power enables original equipment manufacturers (OEMs) and systems integrators to develop better products quickly, cost-effectively and with less risk. Our business was strengthened by the acquisition of Motorola's Embedded Communications Computing group, which has driven open standards and pioneered technologies based on them for more than 25 years. This positions Emerson Network Power as the recognized leading provider of products and services based on open standards such as ATCA®, MicroTCA™, AdvancedMC™, CompactPCI®, Processor PMC, VMEbus and OpenSAF™. Our broad product portfolio, ranging from communications servers, application-ready platforms, blades and modules to enabling software and professional services, enables OEMs to focus on staying ahead of the competition. Manufacturers of equipment for telecommunications, defense, aerospace, medical and industrial automation markets can trust Emerson’s proven track record of business stability and technology innovation. Working with Emerson helps them shift more of their development efforts to the deployment of new, value-add features and services that create competitive advantage and build market share. Emerson’s commitment to open, standards-based solutions and our deep understanding of the embedded computing needs of OEMs provide the foundation for the market to look to us for leadership and innovation.
Regional Offices: Tempe, AZ U.S.A. 1 800 759 1107 or +1 602 438 5720 ▪ Madison, WI U.S.A. 1 800 356 9602 or +1 608 831 5500 Shanghai, China +86 10 8563 1122 ▪ Paris, France +33 1 60 92 31 20 ▪ Tokyo, Japan +81 3 5403 2730 Munich, Germany +49 89 9608 2333 ▪ Hong Kong, China +852 2176 3540 ▪ Tel Aviv, Israel +972 3 568 4387
Emerson Network Power. The global leader in enabling Business-Critical Continuity™. AC Power Connectivity DC Power
Embedded Computing Embedded Power Infrastructure Management & Monitoring
EmersonNetworkPower.com/EmbeddedComputing Outside Plant Power Switching & Controls Precision Cooling
Racks & Integrated Cabinets Services Surge Protection
Emerson, Business-Critical Continuity, Emerson Network Power and the Emerson Network Power logo are trademarks and service marks of Emerson Electric Co. AdvancedTCA, ATCA, AdvancedMC, CompactPCI and MicroTCA are trademarks of PICMG. Service Availability Forum is a proprietary trademark used under license. All other trademarks are the property of their respective owners. ©2008 Emerson Electric Co. STORAGE-WP0 September 2008