Big Data in GOVERNMENT
industry perspective
Big Data Overview Leveraging big data to optimize and streamline business functions has been one of the leading technology trends in government throughout 2012. In this Industry Perspective, GovLoop and Oracle have collaborated to explore the current state of big data and share best practices with the government community. Throughout this report, Peter Doolan, Group Vice President and Chief Technologist, Oracle Public Sector and Izzy Sobkowski, CIO, NYC’s Health and Human Services, share their perspectives and insights on how big data is impacting government. Izzy also shares his experiences about how the Health and Human Services Department of New York City has managed vast amounts of data to improve efficiency and productivity within the City.
PETER DOOLAN Group Vice President Chief Technologist Oracle Public Sector
IZZY SOBKOWSKI
CIO NYC Health and Human Services
Big Data For centuries, government organizations have been collecting data from a variety of sources, and using the collected data to inform their decision-making. With the boom in non-traditional forms of data such as social media, video, digital photographs and email, government agencies are now challenged how to best collect, manage, and drive decisions from data. Today, government leaders are increasingly exploring innovative ways to use big data to glean insights to improve efficiency within their agency. Further, as more kinds of data on the web become transactional, government agencies are pressed to develop a multidimensional view of their customers and citi-
zens. For instance, a government agency that provides permits online can track dozens of statistics about visitors. The agency can see how customers arrive, page views, collect comments, exit paths and nearly any type of interaction on the website. Armed with this kind of knowledge and a long history of user behavior, organizations can have a full view of the customer. This information can then be used to improve services for the end user, driving improved productivity and efficiency for the agency. At the federal level, the White House has encouraged agencies to explore big data programs and use technology to help enable improved services. The Center for Disease Control and Prevention
i n p racti ce has an interesting program called BioSense 2.0. A press release from the White House describes the program as, “BioSense 2.0 is the first system to take into account the feasibility of regional and national coordination for public health situation awareness through an interoperable network of systems, built on existing state and local capabilities.� The CDC is leading the charge in sharing health information between state, local and federal government to help distribute information and resources to protect citizens’ health and improve their standard of living. This is just one of many exciting big data initiatives occurring at the federal level.
Although there are dozens of exciting big data initiatives occurring in government, it is important to remember that government has been collecting data for centuries and big data is part of an evolution of how government uses data. Izzy Sobkowski states, “I think big data as a concept has been around for a long time. We used to capture things in files, and then we went to databases and different types of databases. This is just an evolutionary phase, recognizing the fact that we have a vast amount of data and are learning new methods of capturing, storing and retrieving data.” As more big data initiatives are
starting to develop throughout government, Peter Doolan reminds us that the way data is being used is different than from how data has been traditionally used by government. Peter reminds us, “There is clearly something happening in the data business that is different from what has happened before.” With emerging technology, more kinds of data are being collected, stored and managed. Peter asserts, “This [big data] is new, and there’s something driving big data programs. And the something that is driving it of course are things such as machine data, coming from
“
We now have a larger ability, which mean disciplinary is important. These are intend It’s not just big data but it’s big
machine sensors, and also from consumers through social media, social networks.” As Peter and Izzy have both identified, big data is an evolution of technology, and with advancements to technology, agencies can begin to leverage and find value in the vast amounts of data that they traditionally collect.The hope is that in the end, through improved technology, government agencies can improve the kinds of services they are providing to their core constituents and stakeholders. Big data has been described in a variety of different ways, and there still remains some ambiguity about how big data is defined. Peter stated that big data can be described using the four v’s, “The industry is coalescing around the four V’s of Volume, Variety, Velocity, and Value,” Peter stated. The first “V,” Volume, describes the mass amounts of data collected by agencies, sometimes terabytes or petabytes of data. The second “V,” Variety, identifies the multiple sources of data created, this includes nontraditional data sources like images, video and audio. The next “V,” Velocity, shows the shift in the rate at which data is created.
Often, agencies cannot perform an analysis of the data at the rate data is generated. The final “V,” Value, describes that with so much data available, different techniques and strategies need to be used to elicit the right information for agencies. Izzy builds on the “four V’s” while providing his definition and insights of big data. Izzy states, “The ability to capture data in some meaningful way, to be able to retrieve it and to act upon it.” The ability to act and derive value from data is one important function of big data. Izzy also mentioned that big data encompasses all kinds of different data forms, but the important element is how the different data
interacts and then how government can use data to drive informed decision-making.
“
ns that the ability or the need to go interded to be large projects tackling big ideas. g ideas, big concepts, big need.
One common challenge with big data initiatives is learning what skills are needed within the agency. According to Peter and Izzy the three core skills needed are communication skills, a background in statistics and an understanding of IT and how IT systems can interact. Izzy stated, “We now have a larger ability, which means that the ability or the need to go interdisciplinary is important. These are intended to be large projects tackling big ideas. It’s not just big data but it’s big ideas, big concepts, big need.” As big data programs typically work to tackle big ideas, it will be important that big data staff can work across government and focus on being knowledgeable in multiple disciplines. In order to truly leverage big data, agencies need to be able to recruit employees who can collaborate across the agency. A complementary skill for those collaborating across the agency is to be a talented communicator. Peter states, “You need an IT person who can speak the language of the business.” Peter is clear to identify that there are dozens of talented people with an IT background, but a differentiator
is often the ability to speak the business language. Peter continues to express the importance of strong communication skills for
an employee, “Regardless of the person or those individual skills, they have to be able to speak the business.” Peter continues, “I have seen so many projects fail, not because of technology, the project failed because of the inability for IT professionals to explain and communicate their capabilities to the business side, and for the business to communicate back to the IT side, stating their needs and requirements to have a tangible impact on the outcome.” Like with any project, identifying a shared problem, and working towards a common vision is critical for big data programs. Izzy provided some great insights about his experiences operat-
ing big data initiatives in New York City. One of the first lessons learned from NYC is having a clear scope for the big data initiative. In New York, Izzy and his team had a concrete problem they were attempting to solve. Izzy states, “The problem that we are trying to solve is to understand the citizen from a 360-degree holistic view.” In order to have a 360-degree view of the citizen, Izzy identified the importance of working across departments in NYC. Izzy stated, “Being able to go across our silos, across our agencies and understand the citizen as best as we possibly can, to see what services can be applied and have the best possible outcome, that was the goal.” One of the benefits Izzy has seen from big data programs in New York City is a decrease in costs; Izzy attributes the cost savings to collaboration across silos. “Our costs are lower. The citizen gets better services and we have better outcomes on both sides. And the way that you do that is by crossing agencies,” states Izzy. Working in health and human services, Izzy operates in a tightly regulated environment. Izzy mentioned that on staff he has a full time attorney to ensure that any kind of data exchanged between agencies is legal. He also mentioned that the data is “rule based.” “The data is rule-based. Even though I may be able to see a piece of information under a certain circumstance, I am prohibited from seeing that same information under a different circumstance,” stated Izzy. This pro-
cess has led to Izzy being able to share data across agencies, which has been critical to using health data in New York City. “In a very highly regulated environment, we’ve found ways to operate in that environment and operate appropriately for the best outcome of the citizen,” commented Izzy. The sharing of data has helped solve the initial problem that Izzy and his team set out to solve, viewing a citizen of New York
from a 360 degree perspective. Izzy also added the importance of the legal framework the City of New York has developed, “Innovation comes in a lot of different ways and the technical innovation could never have occurred without the legal framework to ensure privacy and security.” As big data initiatives continue to be implemented at all levels of government, it is always important to take a step back and identify best practices, share resources and
collaborate with peers. Although big data has a unique set of challenges for government, big data is part of the solution to solving some of governments most complex issues. As more data is collected, and technology is developed to improve how data is managed, agencies will be able to unlock the secrets of big data within the agency and use data to optimize and streamline their most critical business functions.
Best Practices for Big Data One of the hardest parts about a big data initiative is figuring out where to start. Peter Doolan provides some insights on how to jump-start a big data project “To get a project off the ground, find a business owner, find a problem that’s highly sensitive, ensure that you have the ability to source the data, and to fix the problem. In many cases you’ll find there are resources available to achieve the mission, they’re just not getting done through IT today. They’re getting done through some other way.”
Quick Wins
Have a Clear Scope
Communicate
Izzy mentioned the importance of having quick wins while running your big data program. Izzy states, “Like everything in the government, just really showing the value of what you’re doing. This is the quick win. Show the impact that it’s having on the agency.” By pointing to quick wins and being able to quickly identify ways the big data initiative is enabling the agency to become more productive and efficient, the team will be energized and will be able to gain support across the organization.
Izzy advises to start with a clear scope of the project, “Don’t start with an openended project. Try and solve one problem and answer one question. You can expect some kind of iterative approach as you go forward. Big data is a data discovery exercise and it almost becomes like a habit.” Having a clear scope and knowing what problem is trying to be solved is critical for big data. With so much data and information that agencies collect, having a clear scope and project goals are essential to success.
Izzy identified numerous best practices related to clear communications. Izzy suggests to set reasonable goals, meet deadlines, control cost and value of the project, deliver on budget, and to build the project with confidence. Izzy states, “Setting reasonable goals, not overpromising in terms of time, cost and value, getting consensus that these are items which are valuable, and then delivering them on time and on budget. This process builds a lot of confidence and allows the program to thrive.”
About Oracle Oracle offers an optimized and fully integrated stack of business hardware and software systems that helps organizations overcome complexity and unleash innovation. Check out their Optimize with Oracle group on GovLoop as well as the Technology Sub-Community of which they are a council member.
About GovLoop GovLoop’s mission is to “Connect government to improve government.” We aim to inspire public sector professionals by serving as the knowledge network for government. GovLoop connects more than 55,000 members, fostering cross-government collaboration, solving common problems and advancing government careers. GovLoop is headquartered in Washington D.C with a team of dedicated professionals who share a commitment to connect and improve government. This Industry Perspective was authored by Pat Fiorenza, GovLoop Research Analyst and designed by Jeff Ribeira, GovLoop Community and Content Coordinator. GovLoop 734 15th St NW, Suite 500 Washington, DC 20005 Phone: (202) 407-7421 Fax: (202) 407-7501