The Changing Perception & Adoption of Flash in the Public Sector RESEARCH BRIEF
The Changing Perception & Adoption of Flash in the Public Sector
1
Introduction Today, government agencies are tackling new and growing data challenges. Limited resources, tightening budgets and increased workloads require the public sector to become more efficient in how it acquires, stores, uses and analyzes data. At the same time, government data generation and archiving are on the rise because of the rapid growth of useful data sources like mobile devices and applications, smart sensors and Internet of Things devices, cloud computing solutions and publicfacing solutions. Coupled with rising public expectations of service and the need to reduce the number of data centers and optimize those remaining, government agencies are looking at a range of solutions, including new storage options, to address these demands. These new requirements, trends and expectations mean that more than ever, government needs scalable, flexible storage solutions that are reliable, efficient and easy to manage. And one potential solution is flash storage. Flash storage is a type of data repository or storage system that uses flash memory, and its use in the public sector has been growing. To understand more about the use, perception and adoption of flash storage in government, GovLoop partnered with NetApp, a leader in public-sector data storage, to survey 156 public-sector employees about their storage needs and challenges. We also sat down with Tom Rascon, CTO for Defense and Intelligence, and Mike Dye, CTO for Civilian Agencies at NetApp to learn more about the future and benefits of flash storage in the public sector.
Research Brief 2
Survey Results: Data Storage and Flash Storage in the Public Sector
FIGURE 1
FIGURE 2
FIGURE 3
Do you work with data or data storage in your day to day job?
Does your agency face challenges in maintaining and storing its data?
What are the priorities of your agency when it comes to data and data storage?
Yes (54%) No (14%)
Some (32%)
Yes (74%)
Cybersecurity (43%) Dealing with unstructured data (16%)
No (26%)
Data center consolidation (14%) Other (11%)
Performance (15%)
G
overnment agencies must keep pace with technology changes while at the same time operate in tighter budget environments. They strive to foster better communication and information-sharing among their workforce, utilize data analytics to make smarter decisions, meet the increasing connectivity demands of their constituents and transition to cloud offerings and shared services. While all of this is taking place, government data is growing exponentially, putting greater demands on agencies’ data storage capabilities at a time when cybersecurity, data loss prevention and data breaches have become national security issues. In our survey of 156 public-sector employees, we were interested in learning from those who deal with all of these demands, challenges and expectations in regard to storage needs, day in and day out in government. GovLoop received insights into these day-to-day issues, as 86 percent of respondents to the survey
work with data or data storage in their daily jobs (see Figure 1). One thing the survey revealed is that agencies attempting to meet these new demands find that their IT infrastructures aren’t up to the task. A primary stumbling block? The current state of storage solutions. This is a massive issue for agencies that GovLoop surveyed – 74 percent of respondents say their agency faces challenges in managing and storing its data (see Figure 2). “The challenge with data storage is often being able to identify the workload that agencies are trying to manage,” said Mike Dye, CTO for Civilian Agencies at NetApp. “They may not know how much data they need to store or what they need to do with it, so agencies end up with mismatched storage to the application that they’re trying to deploy.”
The Changing Perception & Adoption of Flash in the Public Sector
Tom Rascon, NetApp’s CTO for Defense and Intelligence, agreed – and added that the exponential growth curve of data is also a problem for agencies. “Just think about Department of Defense as an example,” he said. “In 2003, we were lucky to have one standard definition video stream per sensor, but now we have multiple high definition video streams coming from the same platform and this presents a huge storage problem for many of our organizations.” When it comes to data and data storage, the priorities of survey respondents varied, but most respondents (43 percent) said cybersecurity was their biggest area of concern (see Figure 3) – unsurprising, considering the massive amounts of data breaches the government has faced. Dealing with unstructured data (16 percent), performance (15 percent) and data center consolidation (14 percent) were also of note in concerns about data storage.
3
“Continuous innovation and continuous deployment is driving people to get applications deployed faster and faster, and government needs to be able to keep up.” Mike Dye CTO for Civilian Agencies at NetApp
Unsure of benefits of flash storage (38%) Current storage approach meets my needs (32%) Other (22%) No leadership buy-in (15%) Too expensive (10%) No skillset to deploy flash storage (10%) Flash is typically for service providers or high performance computing (6%) FIGURE 4
FIGURE 5
Do you use flash storage to serve your data needs?
If no, why do you not use flash storage to serve your data needs? (Select all that apply)
Yes (38%)
No (62%)
To solve these issues, government agencies at all levels now are looking for storage solutions that are priced for performance and offer a secure, dedicated, high-performance storage platform to support workloads. Allflash storage array solutions can help government leaders and IT managers address these demands successfully.
“There’s a perception in the public sector that flash is for high-performance workloads,” said Dye. “And sometimes agencies don’t believe that they have high-performance workloads, so they may not be turning to flash.” In fact, Dye added, audiences don’t realize the extent to which flash can help government with its capacity and security concerns.
Historically, however, the perception of flash storage, particularly in the public sector, has been that it’s not a reasonable solution for IT departments. And in fact, many agencies continue to rely on traditional disk technology for their more critical or frequently accessed data. Disk arrays are equipped to house large amounts of data, but might not be the best fit for frequently accessed data.
This proved true in GovLoop’s survey as well, where the majority of respondents who did not use flash said it was mainly because they were unsure of the benefits and capabilities of flash storage. The reasons that the respondents do not currently use flash storage to serve their data needs ranged into other issues. As noted, one of the biggest reasons was respondents were unsure of the benefits of flash storage (38 percent; see Figure 5). Other reasons included the fact that current storage solutions met their needs (32 percent); there was no leadership buyin for flash storage (15 percent); and their department lacked the skills to deploy flash storage (10 percent).
This perception continued to hold out in our survey results, as the majority of respondents are not using flash storage for their data needs. Only 38 percent of GovLoop survey respondents currently use flash storage to serve their data needs (see Figure 4).
Research Brief 4
“Agencies that don’t think they need flash storage for their current storage needs at this very moment actually need to be thinking to the future,” said Dye. “The fact is that the industry and the storage needs of government are not going to be the same in two years as they are today. Agencies will have different requirements and different necessities. Continuous innovation and continuous deployment is driving people to get applications deployed faster and faster, and government needs to be able to keep up. And those applications are going to have different input/output models, so you need to be able to address those needs as you grow.” In short, agencies need to think about the storage needs of the future in a much more aggressive way. So, if an agency’s current disk storage solution fits its needs today, it may not tomorrow or six months in the future.
Better availability (52%) More flexibility (48%) Better performance (36%) To increase capacity (31%) To reduce power usage (19%) Other (13%)
FIGURE 6
FIGURE 7
If yes, what are the primary reasons you use flash storage? (Select all that apply.)
Do you believe that flash storage is more expensive than disk-based storage solutions? Yes (37%)
And in fact for agencies that are tasked with leveraging data in innovative new ways, disk-based storage solutions are becoming less applicable. Not only are they unable to support the advanced functionality of modern government digital services, they also require an unsustainable amount of resources to function long-term.
Flash can help agencies stay within budget while providing a higher level of storage performance for future needs, with the high input/output operations per second (IOPS), consistent sub-millisecond latency and bandwidth that are essential for applications such as databases, analytics, backup and cloud services.
Some survey respondents who do use flash storage solutions are aware of this fact. Those who use flash storage cited better availability (52 percent) and better flexibility (49 percent) as the major reasons they prefer flash storage (see Figure 6). Other reasons included better performance (36 percent); an increased capacity (31 percent); and the ability to reduce power usage (19 percent). (See Page 10, Flash Storage and Power Usage Requirements, for more on new power outage reporting and the Data Center Optimization Initiative.)
Though flash can help agencies do more with less and stay in budget, the fact is that in the past, many thought flash storage was too expensive for government. According to the GovLoop survey, however, that perception seems to be changing. Only 37 percent said they believe flash storage to be more expensive than disk-based storage options; 63 percent did not believe it to be more expensive (see Figure 7).
The Changing Perception & Adoption of Flash in the Public Sector
No (63%)
“Not only is flash becoming lower and more stable in price, if not less expensive than spinning media,” said Dye, “it saves people money because you more efficiently utilize your CPU cores. With flash, you’re not waiting on I/O so your CPU can spend more of its resources on applications; it saves you on heating and cooling because there’s no moving parts; and it saves you on energy, because it doesn’t create nearly as much heat as a motor does that’s spinning a platter.” Flash’s efficiency in fact saves money by avoiding capacity overprovisioning to achieve performance gains. By adopting flash, data centers are able to capture resource efficiencies that allow them to host more IT services and store more data into the future.
5
The Benefits of Flash for Government As revealed earlier in the GovLoop survey, one of the largest reasons that government is not yet adopting flash storage at a faster rate is that folks are unsure of its benefits. Here are four of the key benefits the government could expect to see resulting in deployment of a flash storage solution.
Performance Unlike hard disk drives, flash has no moving parts, so there are no performance hits due to seek time or rotational latency in the hardware. Flash storage performs read and write operations in microseconds and provides tens of thousands to millions of IOPS.
Reliability As mentioned above, flash storage has no moving parts. This design factor alone greatly improves reliability and results in a mean time between failures (MTBF) of approximately 2 million hours, compared with approximately 1.2 million hours for HDDs.
Efficiency Flash is a flexible technology that is fabricated in a number of different form factors, such as PCI-e flash cards and SSDs. It can be deployed as a memory tier in the server, network, storage controller or storage array. With this flexibility, and higher performance, applications run faster and your users are more productive.
Budget Flash offers a dedicated high-performance capability without consuming too much rack-unit space, so agencies save on all the additional capital expenditures that go with that, including power usage and cooling requirements. Additionally, organizations can get 100 times or even 1,000 times the performance of traditional spinning disks for the same or less cost per gigabyte.
Research Research Brief Brief 66
NetApp: A Leader in Flash Storage
FIGURE 8
FIGURE 9
Do you know that NetApp is a provider of flash storage for the public sector?
Do you believe all flash storage implementations are the same?
Yes (17%)
No (83%)
Yes (21%)
A
strong majority (83 percent) of respondents were not aware that NetApp is a provider of flash storage for the public sector (see Figure 8). In fact, NetApp offers all-flash arrays that deliver flash performance suited to the specific workload requirement of the environment, thereby accelerating speed, improving customer experience and reducing total cost of ownership. And this matters. One GovLoop survey perception around flash storage that seems to be evolving is the one that all flash storage implementations are equal. Only 21 percent of GovLoop survey respondents said they think all flash storage implementations are the same; 79 percent did not believe that (see Figure 9). Indeed, it matters which flash storage implementation you go with. “Part of what NetApp offers as a data storage vendor for the public sector is our Data Fabric,” said Rascon. “We have an entire ecosystem of products and services, anything from hardware to the software management tools that enables
No (79%)
our customers to use flash, coexisting with all the other storage an agency may have already in their data center.” Dye pointed out that NetApp offers a fundamentally sound architecture and operating system that meets specific customers’ needs. “Our ecosystem offers products that work well together,” he said. “We can have a data lifecycle that goes from flash to disk to cloud, instead of just from disk to tape.” In particular, NetApp has a portfolio of offerings that meet a user’s unique needs, and distinguishes the company from other flash storage vendors.
NetApp All-Flash FAS: This is suited for users who want the performance of all-flash arrays and data management features. All-Flash FAS provides multiple protocol support, snapshot and provisioning, replication and disk backup with SnapVault cloning. Inline compression and deduplication are turned on. Plus, it integrates with the cloud.
The Changing Perception & Adoption of Flash in the Public Sector
“We can have a data lifecycle that goes from flash to disk to cloud, instead of just from disk to tape.” Mike Dye CTO for Civilian Agencies at NetApp
NetApp EF Series: Agencies that don’t need all the data management features can opt for the EF Series. For instance, they might be running Oracle RMAN for backup instead of using NetApp Snapshot. In a small 2Unit form factor, the EF Series can push through a lot of I/O.
NetApp SolidFire: This is targeted at cloud providers that need the scalability to address the needs of hundreds of thousands of multitenant users. NetApp guarantees 100,000 IOPS can be accessible to a particular application. The arrays are also self-healing, so a cloud provider can add or remove SolidFire nodes automatically to other parts of the cloud environment. Each node has its own controller and storage SSD built in, so cloud providers can handle unpredictable growth and capacity.
7
Case Study: How NetApp Helped the National Energy Research Scientific Computing Center
L
ocated at Lawrence Berkeley National Laboratory, National Energy Research Scientific Computing Center (NERSC) is a premier facility for scientific research sponsored by the U.S. Department of Energy’s Office of Science. More than 6,000 scientists from universities, laboratories and industry worldwide use NERSC to tackle problems that span from climate research to studying the universe and its evolution to understanding new materials. Since 1974, NERSC has used powerful supercomputers to perform research across a range of disciplines. In their important work, NERSC scientists across a wide range of disciplines are creating increasingly larger datasets at an ever-increasing rate. Keeping up with user demands for storage capacity and performance became an ongoing challenge. “We support a diverse research community, so user needs can be very different, and they are always increasing,” said Jason Hick, storage group leader at NERSC. “Many of our
researchers, such as those doing genomic sequencing, need low latency from storage. Others have high bandwidth requirements. If throughput or bandwidth is insufficient, compute jobs can be seriously impacted, which could stall research.” These challenges were increasing. With the huge amount of data, access needs and number of users needing to download the data, NERSC was struggling to provide sufficient bandwidth and meet scalability requirements for the growing archive. It decided to replace the archival disk cache to achieve new peak performance. “We evaluated a wide range of vendors, looking for flexible, highdensity storage that would meet our researchers’ requirements,” said Hick. “We chose NetApp because it offered the combination of features and functionality we needed.” With assistance from NetApp engineers, NERSC deployed NetApp E5560 storage systems in the NERSC Global File System. Research Brief 8
This deployment also allowed NERSC to expand the Global File System to the Parallel Distributed Systems Facility (PDSF), a separate computing system that is managed by NERSC. To increase metadata access speed, NERSC also deployed NetApp EF550 all-flash arrays. “The key problem we had was metadata performance,” said Hick. “It really wasn’t keeping pace with the scientists. We had been waiting for a production-quality flash storage product to come along that we felt would be worthy of deploying. The NetApp engagement came from looking at a cross-section of performance, value and reliability. After putting the NetApp EFSeries all-flash arrays in, we saw a fivefold performance gain in backups alone.” “Being able to do queries on an all-flash array is huge for us,” said Peter Nugent, Computational Research Division Deputy for Scientific Engagement, NERSC. “We can now spend more of our time focusing on how we can benefit science.”
Flash Storage & Power Usage Requirements Flash arrays can play a critical role in helping government IT managers consolidate and optimize data centers to meet future data storage demands. This is more necessary than ever, because in March, the Office of Management and Budget shifted the focus of federal chief information officers from the consolidation of data centers to optimization under a new policy called the Data Center Optimization Initiative (DCOI), which supersedes the government’s Federal Data Center Consolidation Initiative (FDCCI).
FIGURE 10
Are you aware of the OFICIO’s new data center power usage reporting requirements as outlined in the new Data Center Optimization Initiative? Yes (13%)
No (87%)
Virtualized compute and storage resources (56%) Consolidate to other data centers (56%)
FIGURE 11
If yes, how do you plan to address the power reporting requirements? (Select all that apply)
Implement flash storage (39%) Reduce capabilities (28%)
As datacenters.cio.gov notes: “OMB will monitor the energy efficiency of data centers through a Power Usage Effectiveness (PUE) metric. Energy metering tools shall enable the active tracking of PUE for the data center and shall be installed in all tiered Federal data centers by September 30, 2018. Agency CIOs are required to ensure that existing tiered data centers achieve and maintain a PUE of less than 1.5 by September 30, 2018. Effective immediately, all new data centers must implement advanced energy metering and be designed and operated to maintain a PUE no greater than 1.4, and are encouraged to be designed and operated to achieve a PUE no greater than 1.2.”
“A huge amount of the energy that’s deployed in the United States is for data centers, over 70 billion kilo-watt hours in 2014. That’s equivalent to 6.4 million average American homes.” Dye said. “It’s a hugely disproportionate number to what we should be focusing on in the world today. So we have to take a look at these issues and address them and help reduce that power usage.” Unfortunately, when asked if they were aware of the OFICIO’s new data center power usage reporting requirements as outlined in the new Data Center Optimization Initiative, only 13 percent of respondents said they were (see Figure 10). The majority of those who were aware of the reporting requirements said they planned to address them via virtualized compute and storage resources (56 percent) or consolidation to other data centers (56 percent); see Figure XX. If agencies were to deploy flash storage technology, they’d have great success in reporting on these new power usage effectiveness metrics, Rascon said. “Flash
The Changing Perception & Adoption of Flash in the Public Sector
can really reduce power requirements,” he said. “For example, NetApp’s All-Flash FAS storage systems are purpose-built for capacity-intensive environments that require optimal space utilization and reduced power and cooling requirements.” Using spinning disks, 1 petabyte of storage consumed 2 entire racks of gear. Using NetApp’s new All-Flash FAS systems, agencies can achieve 1 petabyte of storage in as little as 4 rack units (RU) of space. This is a dramatic reduction in the physical footprint required for their storage, but it also significantly reduces the heat and power required to run the same capacity system. All-Flash FAS systems can change your data center economics dramatically by reducing power consumption and rack space to a fraction of what a traditional HDD-based data center needs. Ideally, going forward in meeting these new DCOI requirements, the public sector will deploy more flash storage solutions to reduce and report their power usage metrics. 9
Conclusion Increasingly, the applications that control key IT operations demand improved responsiveness to handle transactional workloads more quickly, with extremely low latency, over high-end IOPS. Flash storage array has proven to offer the consistent performance needed to increase efficiencies while maintaining a lower total cost of ownership. As the GovLoop survey shows, even with these cited benefits, the growth and adoption of flash storage has a long way to go in the public sector. Key to continuing the use and adoption of flash storage will be better understanding its benefits, continued education about its affordability and more awareness about how flash storage can help with data center optimization and power usage reporting requirements. As government continues to embrace flash to accelerate more applications across data centers, flexibility will be key. Flash storage can help the public sector create great solutions to power its public, private or hybrid clouds with uncompromising adaptability from a single architecture – serving the needs of agency IT departments and end users.
About NetApp
About GovLoop
Government agencies of all levels count on NetApp for software, systems, and services to manage and store their most important asset, their data. With solutions ranging from data protection and recovery to cloud computing, data analytics, and flash solutions, NetApp has become government customers’ top choice for key technologies that drive data center transformation. Top counties, cities, and states count on NetApp and value our teamwork, expertise, and passion for helping them succeed now and into the future.
GovLoop’s mission is to “connect government to improve government.” We aim to inspire public-sector professionals by serving as the knowledge network for government. GovLoop connects more than 250,000 members, fostering cross-government collaboration, solving common problems and advancing government careers. GovLoop is headquartered in Washington, D.C., with a team of dedicated professionals who share a commitment to connect and improve government.
To learn more, visit: netapp.com/us/products/flash-storage
For more information about this report, please reach out to info@govloop.com.
Research Brief 10
The Changing Perception & Adoption of Flash in the Public Sector
11
1152 15th St. NW, Suite 800 Washington, DC 20005 (202) 407-7421 F: (202) 407-7501 www.govloop.com @govloop
Research Brief 12