ANALYTICS IN ACTION
HOW GOVERNMENT TACKLES CRITICAL ISSUES WITH DATA
Sometimes we hold our hope that if we just get this analytics thing, it’s going to be a magic bullet, it’s going to be that silver bullet to all of our problems. I think that like any good solution, it takes refinement, it takes effort and being patient, [plus focus] on that goal of solving whatever that data problem is. You just have to keep hammering at it, and you have to stick with it. JOAH IANNOTTA,
Director of the Treasury Department’s Do Not Pay Business Center
A GovLoop Guide 2
CONTENTS CASE STUDY 1: IMPROVING VETERAN CARE
06
CASE STUDY 2: COMBATING OPIOID ABUSE
10
CASE STUDY 3: DETECTING FRAUD
14
CASE STUDY 4: MANAGING PUBLIC HOUSING
18
Executive Summary 04 Get Smart Quick on Analytics 05 Adding More Value to Data with In-Document Analytics 09 The Power of Operationalizing Data Analytics 13 Using Data Analytics to Maximize Mission Impact 17 How Actionable Data Improves Digital Communications 21 10 Tips for Briefing Your Boss With Data 24 Tips for Success 26
CASE STUDY 5: REDUCING IMPROPER PAYMENTS
Conclusion, About & Acknowledgments 27
22
Analytics in Action: How Gov Tackles Critical Issues With Data 3
EXECUTIVE SUMMARY In July 2017, the federal government announced the largest health care fraud takedown in the history of the Justice Department. The government charged 412 defendants with fraud over a scheme involving $1.3 billion in false billings. Among the defendants, 56 were doctors and more than 120 have been charged with opioid-related crimes, making this the largest opioid-related fraud takedown in American history. Other offenses included billing government health services for fraudulent claims and swapping prescriptions for money. “It was the leveraging of many resources that made the takedown such a huge success,” William Parente, Health Care Fraud Unit Chief of the FBI’s Criminal Investigative Division, said during an ACT-IAC forum in Washington, D.C. Detecting and thwarting fraud is a challenge across government. At the same time, agencies are working to address other pressing issues, including opioid abuse, affordable housing, integrity of government programs and improving veteran care. The common thread tying together the government’s handling of all these issues? It’s data analytics. In this GovLoop guide, we share firsthand accounts from government employees at all levels who are using analytics to not only identify critical issues but to also find solutions. Their comments were edited for length and clarity and focused on key issues, including: ›› How analytics is being used to drive decision-making ›› Tips for getting the data they need and using the right tools ›› What makes their analytics projects successful ›› Future plans for using analytics Before we get to those interviews, let’s review some quick facts about how analytics is used today.
A GovLoop Guide 4
We asked the GovLoop community to name the biggest driver of data analytics adoption at their organizations. We received 145 responses. Here’s what they said:
13% 30% 27% 30%
GOVERNMENT MANDATES & POLICIES IMPROVE CUSTOMER SERVICE MISSION REQUIREMENTS PRESSURE FROM SENIOR LEADERS
GET SMART QUICK ON ANALYTICS FACTS YOU SHOULD KNOW 0.5 PERCENT:
The amount of personal, digital data created globally that’s actually analyzed. (MIT Technology Review)
1.7 MEGABYTES:
The amount of new information to be created every second for every human being on the planet in 2020. (Forbes)
$3.1 TRILLION:
The amount that poor-quality data costs businesses and government agencies per year. (Harvard Business Review)
40 PERCENT:
The amount of data science tasks that will be automated by 2020. Data and analytics software platform vendors are making simplification a top goal by automating tasks. (Gartner)
4 MILLION:
The amount of government documents that the public can access, thanks to the Notice and Comment project. The project uses advanced analytics and natural language processing to ingest government documents and track changes in policies, laws or regulations. Users can comment or vote on pending federal regulations or local public notices more easily. (IBM Center for the Business of Government)
ANALYTICS USE CASES PREDICTIVE POLICING:
By applying advanced analytics to various datasets, in conjunction with intervention models, law enforcement can move from reacting to crimes to predicting what and where something is likely to happen. (National Institute of Justice)
HONEY BEE HEALTH:
Data visualizations are being used to save honey bees. Engineers are finding ways to monitor beehive conditions and present the data in graphs and charts. They are also eyeing the use of advanced data analytics to better interpret the data. (University of Massachusetts at Amherst, College of Information and Computer Sciences)
FRUIT FINDER:
The website FallingFruit.org combines public information from the Agriculture Department, municipal tree inventories, foraging maps and street tree databases to provide an interactive map of edible, natural organisms growing in neighborhoods worldwide. (Falling Fruit)
HUMAN GENOME SEQUENCING:
In 2003, the process took several years, but thanks to data analytics and advancements in technology, it now takes a few days. (Science Daily, PBS.org)
FASTER SAFETY TRACKING OF MARKETED PRODUCTS:
The Food and Drug Administration uses complex data analytics to speed identification of potential safety issues associated with marketed products, including vaccines, artificial hearts and food. (U.S. Food and Drug Administration)
Analytics in Action: How Gov Tackles Critical Issues With Data 5
CASE STUDY #1
IMPROVING VETERAN CARE
The Veterans Affairs Department
is the largest integrated health care system in the United States. Through its Veterans Health Administration, the VA provides care at 1,245 health care facilities, including 170 VA Medical Centers and 1,065 outpatient sites. VHA serves more than 9 million enrolled veterans per year.
Dr. Peter Almenoff, Chief Improvement and Analytics Officer What issues is your organization trying to tackle? VA care providers are trying to ensure that we have safe, effective, efficient and sustainable high-level care. [Our] areas of specific focus align to the VA Secretary priorities, along with what we look at in the private sector. For the Secretary, his priorities include eliminating suicide among veterans, enabling more choice for veterans regarding where they seek their care, and then regaining the trust [of] veterans and the nation by publishing measures of quality, access, patient experience, including comparisons with the private sector when those are available.
How are you using data and analytics to address those issues? We use data for the leadership and for staff at all levels to identify gaps in opportunities. We [also] do education and in-service [training] for our stakeholders on our tools — interpreting the findings [and] connecting the dots to make sustainable and systematic change. We try to give them internal and external benchmarks for quality and efficiency that integrate into analytics to allow comparisons of performance within the VA and community partners. We use Statistical Process Control methods to establish trigger systems for health outcomes, like productivity [and] access. We do that in both acute care and outpatient care. The idea behind that is to really detect deviations in care processes to see whether we have scheduling practices that we need to address, [and] staffing and productivity issues. We have some real-time process control systems that look at things like outcomes, patient safety and hospital complications. On a daily basis, VA can look across the country with a trigger system [and] be able to identify those issues and address them. We can’t predict the weather, so these systems are designed to detect signals of abnormality and then the facility’s first job is to see whether it’s explainable or not. If not, then they have to look at their processes to see what the issues are. We also use a whole series of visual analytics. We have something called the goal-setting calculator, which is [a] tool that allows medical centers to set their goals for improvement for the year. Each medical center can set their goals. It’s designed [as] a prediction model, so it predicts where they’ll be in six months and where the system is going. It shows them the areas of opportunity and allows them to set their goals locally so they can continue to move in a good direction.
A GovLoop Guide 6
Can you share a time when your organization used analytics to drive better decision-making? We build early warning systems and risk stratification systems to identify areas of risk in performance. We [also] have an improvement side that will target those areas and work with them to improve. We can identify what we consider high-risk areas for facilities to work on.
What did you learn and what were the outcomes? In 2016, we looked at what’s called an effect size. This is a way to see whether medical centers are better than they were a year earlier. Last year, 82 percent of our VA [medical centers] performed better than their prior year. What contributed to this was leadership engagement at all levels of the organization.
Can you share tips on how you got the data you needed and decided what tools/ techniques to use? The secret is understanding what’s important and what’s valid. Most of our staff has at least 10 or [more] years working with the VA dataset. So [the] first piece is you have to have somebody with credibility and understanding of [the] systems.
How are you creating a culture for analytics at your agency? The first thing we do is develop the analytic projects that are necessary [by doing] an assessment of the needs. We’ll look both internally and externally. We are very responsive to VA [central offices’] senior leadership and program offices [and] to facilities and regions because they’re really the stakeholders. We’re always looking for field input. When they feel like they own it —or are part of it — they buy in a lot quicker. You get a lot of buy-in when you’re responsive to your clients [and] partners. We provide a lot of educational sessions — both for the leadership and for field [workers]. You can’t make this purely a black box; you have to try to make it as understandable to people as possible. [Also] visual analytics is key. If you don’t have it in a format that they understand, it just doesn’t work.
What do you hope to do with data and analytics in the near term and long term that you cannot do today? Our hope is to get to more real-time. We do something called the G-chart, which is the geometric charts. So every day across the country, you can look at things like outcomes, patient safety issues and hospital complications. That’s our first test, and it’s been very successful. It’s a balance; you can’t just give everything to everybody every day. That’s one of our hopes for the future: to get them more real-time. In addition, try to become more predictive. The idea is to build more predictive tools in more areas that make them seamless within our system. [It’s] not just about building them, but [finding] solutions. We need to try to find the early signs of facilities that are not performing where they need to.
Analytics in Action: How Gov Tackles Critical Issues With Data 7
Business Intelligence
Better Data. Better Analytics. Learn why Information Builders gets the highest percentage of recommendations and the widest adoption rates of any analytics platform at informationbuilders.com/products/intelligence.
Data Integrity
Data Integration
informationbuilders.com WebFOCUS
iWay Software
Connect With Us
Omni
INDUSTRY SPOTLIGHT
ADDING MORE VALUE TO DATA WITH IN-DOCUMENT ANALYTICS
An interview with Rado Kotorov, Vice President of Marketing Strategy and Chief Innovation Officer, Information Builders Government agencies have begun collecting and analyzing more and more data, with a goal of improving decision-making and ultimately bettering the lives of citizens. Although many agencies are making great strides in data analytics, others are finding themselves with mountains of data, unsure how to find meaning in it all. To gain a better understanding of how government organizations can improve how they analyze data, GovLoop sat down with Rado Kotorov, Vice President of Marketing Strategy and Chief Innovation Officer at Information Builders, a leader in enterprise business intelligence, integration and data integrity software. Kotorov said that one of the bigger data problems government faces is that leaders are unsure how to make the massive and ever-increasing amount of data useful. Kotorov explained that the life cycle of data starts from the raw data, which turns into insight, which drives action. That’s not what many agencies are doing, though. “What you mainly see in government is that a lot of the data sources are being put out just as a raw material,” he explained. Compared to the private sector, few government agencies are working on turning raw data into insights. This is problematic as employees then must spend a good amount of time mining the data and making it usable. Additionally, end users are often analyzing data that is not connected to a server or cloud platform. This creates a problem with analytics because the end user will be able to analyze only the data that they have on hand or on premise. The server disconnect often leads to incomplete analysis because of the partial availability of the data. Mobile solutions are one way to overcome these challenges. “Mobile is facilitating the switch from gut-based decisionmaking to fact-based decision making, so making analysis easy and consumable will drive the adoption of data and analytics for decision-making,” Kotorov said. Moving to a mobile solution where frontline employees can get accurate information quickly is critical for making good decisions in the field. Operational employees need an analytics solution that allows them to engage iteratively with the data systems
to get the answers they need. They also need a solution that allows them to properly analyze the information they are receiving. Performing in-document analytics is one way end users can quickly interact with data systems. PDF documents are the largest vehicle for distributing information and reports among agencies, Kotorov said. Unfortunately, these documents are not interactive in a way that fosters quick analysis. To overcome this challenge, Information Builders has created technology called analytic document format (ADF). ADF essentially embeds analytics directly in a PDF document so the end user can sort, filter and make calculations directly into the document. In-document analytics also solves the issues associated with data disconnected from the server. The analytic component of the software is bound to the document so analytics can be done within the report and without having to be connected to the server or cloud platform. As a result, anyone can analyze data on any device, in any location. Although having the ability to access and analyze data anywhere any time provides unparalleled value to government agencies, it also illuminates some inherent security concerns. The government is a large workforce with very different levels of security depending on what information is involved. As technology scales across departments, it becomes open to more users and access points. “As a result, it is very important to take a platform approach so you can make sure that all access points and their combinations are equally protected,” Kotorov said. Additionally, the document respects the security of the application with the ability to encrypt the documents. Technology has evolved to a point where information is widely accessible, Kotorov said, and now it becomes a question of approach and methodology to make information distribution more pervasive. By implementing a more interactive approach to data analytics, government employees will be able to quickly make data-driven decisions to better citizens’ lives.
Analytics in Action: How Gov Tackles Critical Issues With Data 9
CASE STUDY #2
COMBATING OPIOID ABUSE
Northern Kentucky Independent District Health Department uses its $16.3 million budget to fund programs that prevent disease, promote wellness and protect against health threats.
Debbie Young, Public Health Informatics Manager
Emily Gresham Wherle, Public Information Administrator
What issues is your organization trying to tackle? Debbie: In Northern Kentucky right now, the largest health threats are obesity, heart disease and substance abuse. One of the things that we wanted to do in response to the substance abuse issue, which in Northern Kentucky is opioid abuse, is put together a central repository of information where we at the health department could be the chief health strategist on this issue, where people come for information on the opioid epidemic.
How are you using data and analytics to address those issues? Emily: We’ve been working on educating people about the need for a needle exchange, and data and a story map can be very helpful when you’re advocating for policy change. The story map includes things like treatment capacity in the region, so [when] advocating for policy, advocating for funding, advocating for change, data can help drive a lot of that. Ned: Geographic Information System in general is a very handy tool for combining multiple sources of data that share common geography. Being able to combine things like police reports, emergency medical services, as well as hospitalizations or ER utilization rates on a ZIP code level really shines a light on areas that may have higher activity and associated strain on your public resources.
A GovLoop Guide 10
Ned Kalapasev, Database/GIS Manager
Can you share a time when your organization used analytics to drive better decision-making? Debbie: It’s data that we collected for the purpose of a hepatitis C epidemiological profile. We took that data and mapped it over time to show hot spots in our area of high hepatitis C [rates]. Ned: By monitoring the hepatitis C rates in a community by ZIP code, you can see which areas are more active in IV drug use. Debbie: The same areas which have the high rate of Hepatitis also have the high rate of police reports and overdoses. So all of those different pieces of data that we collected — overdoses, hospitalization, police runs, EMS [Emergency Medical Services] runs — they all correlate to the same areas on our map.
How are you creating a culture for analytics at your agency? Debbie: It’s definitely top-down. Dr. [Lynne] Sadler [District Director of Health] came to us. We have this initiative, which is the Heroin Impact Response team. But the missing piece in that organized effort is the data collection, analytics and presentation. We want to provide that piece.
What do you hope to do with data and analytics in the near term and long term that you cannot do today?
What did you learn and what were the outcomes? Emily: We’ve been advocating for syringe exchange. Kentucky has requirements to operate syringe exchanges where you have to get governmental approval from the city and county in which you operate, and that has been a difficult task to accomplish in Northern Kentucky. As we continue to look at how we can come at the problem from a different angle, we have some partners who wanted to call us out into various communities, and we had to come back to the data and say, “No, these are the communities that we need to focus our energy and our advocacy on because this is where the problem is the greatest.”
Ned: I would like to see more real-time, mobile data collection [and] assessment on the fly. Being able to pull that data when you need it would be fantastic for our emergency response efforts. Any time there’s any kind of public health disaster or emergency, having that at your fingertips would be [the] way to go.
What advice do you have for others in government who want to use data analytics? Ned: Share your data. A lot of organizations just hold onto their data because it’s their data, and they’re going to do something about it, or they’re going to publish some kind of plan for it that may or may not come to fruition. Debbie: Don’t be afraid to ask questions, and find the person who knows how to answer the question for you.
Can you share tips on how you got the data you needed and decided what tools/techniques to use? Debbie: We listened to the partners and sought advice from them [about] what [they] want to see. Then we reached out to the organizations that might have that data, such as state police [and] EMS. Then we had to establish relationships with the people who supplied us with the data. We requested their data, put it into workable formats and then shared [it] with them so that they could see what their data was doing — bringing them on board to supply us with additional information in the future. Our plans are to keep this updated so that the latest and greatest data is available for our community. Emily: Some organizations may not have capacity to do [that] kind of analysis, so that may be a service that we can offer.
What were the resource requirements? Ned: The man hours [for] obtaining data and actually [putting it] in usable formats, those were the highest cost for us.
What was key to your success with analytics? Debbie: Working closely with the organizations that supply the data, support[ing] partnerships for future initiatives, sharing results so that they can see their data in use [and] making the data available to the public. I think behind it all is really the support from upper management to devote the time and resources so that [data] can be used in decision-making.
Analytics in Action: How Gov Tackles Critical Issues With Data 11
Turn big data into big insight with MicroStrategy. Connect to any data, big or small, without limits.
Tap into any data source with optimized connectivity
Get the full-range of data analytics functionality, from self-service data preparation and visualization to advanced predictive analytics
Modernize legacy technologies and upgrade point solutions to a standard, scalable, secure, and governed platform
Every day government agencies around the world rely on MicroStrategy to support all of their data analytics needs. Shouldn’t yours?
TO LEARN MORE, VISIT: microstrategy.com/government
A GovLoop Guide 12
Analytics and Mobility
INDUSTRY SPOTLIGHT
THE POWER OF OPERATIONALIZING DATA ANALYTICS
An interview with Danielle Ruppel, Senior Director of Federal Sales and Chris Sotudeh, Client Executive at MicroStrategy Data analytics is no longer a nice-to-have for agencies that rely on timely information to make decisions. They aren’t looking for lengthy analyses but rather the ability to get actionable data to the right people at the right time. What they need is a way to operationalize data analytics. “It’s the ability to use the analytics of a business holistically, to be able to pull together data from different places, different sources — including legacy systems — and get an overall view of whatever you’re looking to analyze,” said Danielle Ruppel, Senior Director of Federal Sales at business intelligence firm MicroStrategy. The Transportation Security Administration (TSA) is one example. During the spring of 2016, TSA was bombarded with long wait times at security checkpoints. Mounting pressure to reduce wait times led to the agency creating an operations command center. They used MicroStrategy’s analytics solution to view and analyze various metrics, such as wait times and workforce assignments at airport checkpoints. In the initial years of the deployment of this performance management application, TSA was able to recognize over $100M in cost avoidance through efficiency gains and performance optimizations. “The MicroStrategy approach is that analytics covers a broad spectrum of things, and it’s not just a dashboard,” Ruppel said. “For some people it’s a grid report, for others it’s the ability to do predictive analytics. The key is having the ability to alert the proper decision-makers about a particular anomaly.” Ideally, they shouldn’t have to dig through reports if they need information. For example, the best solution may be a pop-up alert on their phone or an email that says a particular threshold has been met and action is required — both of which are capabilities that MicroStrategy provides. “In other words, this is not analytics purely for the sake of doing analysis,” said Chris Sotudeh, Client Executive at MicroStrategy. The goal is being able to act on timely data by operationalizing data analytics as part of their business processes in solutions ranging from command and control to threat monitoring and fraud detection. Another way that analytics is being used to increase productivity is in data-driven mobile solutions, Sotudeh said.
Federal agencies are exploring new and innovative ways of enhancing field applications such as inspections and audits. These were jobs routinely completed using pen and paper, but now employees can easily access analytics tools, incorporate geospatial data and telemetry, and view past inspections all on their mobile devices. In addition, the data capture capabilities in MicroStrategy extend the traditional data analytics paradigm. This allows agencies to easily deploy innovative applications on MicroStrategy’s “build once, deploy everywhere” platform. Part of the challenge for agencies is that they are trying to collect as much data as they can about their operations, but not all that data is easily accessible for analytics. Data resides in legacy systems, third party data sources and big data repositories. “We offer native connectivity to many legacy systems, flat files, screen scraping, and all the different distributions of Hadoop” Ruppel said. To take full advantage of this large, diverse and growing data, agencies must connect to disparate systems and build a unified view of the data. That’s where MicroStrategy has been a gamechanger for numerous agencies by providing a platform that supports a metadata model architecture with re-usable objects. Benefits include reduced complexity and lower total cost of ownership with data governance, security, and scalability to support analytics needs ranging from self-service data discovery to predictive algorithms. Data analytics can also play a strategic role in the path to IT modernization. “All agencies want to do more with data analytics, yet many face challenges due to the high costs and risks of the IT modernization journey,” Sotudeh said. Some federal agencies have launched high-impact initiatives to demonstrate the value of modern analytics by using MicroStrategy to get the most out of existing investments. “MicroStrategy has provided these agencies with the ability to take an incremental approach to modernization by connecting to and blending data from legacy assets.” “It’s not a rip and replace where you’re starting over again with MicroStrategy,” Sotudeh said. “We have ways of connecting existing analytics tools and ingesting some of the data from those tools to get you up and running quickly so that there’s really no downtime.”
Analytics in Action: How Gov Tackles Critical Issues With Data 13
CASE STUDY #3
DETECTING FRAUD
The Government Accountability Office’s Forensic Audits and Investigative Service (FAIS) provides Congress with forensic audits and
investigations of fraud, waste and abuse; special investigations; and security and vulnerability assessments. The roughly 60-member team often reviews programs with multibillion-dollar budgets.
Seto Bagdoyan, Director of Audit Services What issues is your organization trying to tackle?
How are you using data and analytics to address those issues? We are dependent on having access to federal databases of various programs occasionally. We purchase access to proprietary commercial databases like medical licensure information, as well as the Postal Service’s comprehensive database of all the residential and commercial addresses in the country. We use that information in various ways to do data matching.
Our charge is to look at program integrity. Our work combination of performance audits, as well as investigative techniques, very frequently involves a data analytics component that helps inform our findings.
We also do data mining, which is essentially looking into the databases to see patterns. Our investigative techniques usually test controls to see whether the findings match up with our data analytics.
We look at pretty much anything and everything that involves the expenditure of federal dollars. We do focus a lot on means-tested benefits programs, and the FCC [Federal Communications Commission] Lifeline [program] is certainly one of those. It’s the most recently completed work. [Lifeline helps make communications services more affordable for low-income consumers.]
Whatever we find in terms of the results of our data analytics, we usually make referrals to the agency that runs the program for them to take more in-depth looks at what we have and see if their review corroborates our initial suspicion that something untoward is happening.
Can you share a time when your organization used analytics to drive audit findings?
Most recently we looked at the FCC Lifeline program. We got the list of current beneficiaries — roughly 12 million — and we matched them against selected databases of other benefits programs that they claimed in their applications as being their eligibility determinant, like Medicaid, or supplemental security income or food stamps.
A GovLoop Guide 14
What did you learn and what were the outcomes? We found that at least 1.2 million subscribers did not appear on the programs that they claimed. So that to us is a significant flag that something might not be matching up appropriately, that these people may either [have] made an honest mistake, or they are not eligible for the benefit. We’re in the process of making referrals to the FCC, and its inspector general, for them to take whatever action they deem appropriate with these individuals.
Can you share tips on how you got the data you needed and decided what tools/techniques to use? By statute, GAO has access to any relevant data deemed essential to conducting an audit. We work with the agencies, and in a matter of weeks — sometimes months — we get the data we need, and we start first and foremost doing data reliability assessments to see whether the data are even usable for our purposes. If they happen not to be usable because they’re deemed unreliable through our testing, then that itself becomes a story. If the data are of such poor quality, the problem for the agency is: How would they be able to oversee this program when they don’t have reliable data? We use things like SAS and Python and R, which are data analytics software programs, and we customize them to the specific needs of an audit. That can take time.
What was key to your success with analytics? The willingness to do it certainly is one. It has to be part of a sound audit plan that has objectives, that also manages the scope and has the right people assigned, whether at the leadership level or at the subordinate level. At a minimum, I think it is imperative that if it is going to be a data analytics-heavy engagement, the analyst in charge be proficient in data analytics. Many of the audits that are starting up have significant data analytics components to them, so the team design is essentially the key aspect of success.
How are you creating a culture for analytics at your agency? A couple of years ago, we had our road show for FAIS, where we went to each mission team and staff office at GAO, and we had an hourlong presentation of who we are, what we do, how we do it and a lot of our results. For most people, it seemed to be a well-kept secret. Government auditors are very oriented to a routine approach, and they don’t shy too far away from tried-and-true audit approaches. So data analytics is a big disruptor. It may show aspects of a particular program that nobody really wants to know, so there are a lot of potentially ugly truths.
What advice do you have for others in government who want to use data analytics? Don’t be afraid of it. It’s a powerful tool to be used in combination with other things. Bring people along. Continue to show value [and] do the small wins.
What do you hope to do with data and analytics in the near term and long term that you cannot do today? A data analyst’s dream is to be able to do more predictive work. Learning from the past and trying to project, for example, the prospect of fraudulent activity in program X over the next five years. Even retrospectively, we hope to continue to do deeper dives into the data that we have.
Analytics in Action: How Gov Tackles Critical Issues With Data 15
ANALYTICS FOR EVERY AGENCY With Tableau, Federal, State and local organizations can quickly and easily connect to all their data improving mission critical outcomes and performance. tableau.com/analyticsinaction A GovLoop Guide 16
INDUSTRY SPOTLIGHT
USING DATA ANALYTICS TO MAXIMIZE MISSION IMPACT
An interview with Robert Dolan Jr., Market Segment Director for Public Sector at Tableau As government’s mission grows in complexity and scope, so has the amount of data that agencies are expected to manage, analyze and use for decision-making. For data stewards operating in an increasingly digital environment, this is no small feat. Consider the sheer volume of data that agencies use to make decisions about the citizens they serve, their workforce and the programs they manage. They’re relying on data from the public, internal sources and from other agencies and organizations. With so much information from multiple sources, agencies need a way to connect the dots, and that’s where data analytics comes in. “It helps us to get a 360-degree view of the citizen and the program or service that they’re delivering,” said Robert Dolan Jr., Market Segment Director for Public Sector at Tableau, a data visualization software company. Data analytics is a key enabler for agencies — regardless of their mission. But before agencies can reap the benefits of what analytics has to offer, they first have to create a culture that embraces the right tools, skills and mindset. “Analytics has to be pervasive,” Dolan said. “Everybody in the organization needs to have access to the information to consume it. If you look at the promise of analytics, it’s in large part about the consumption of information, keeping people informed and driving transparency and accountability.” When you embrace a culture of data analytics it eliminates the silos of decision-making that can hinder government’s ability to deliver optimal services. Another benefit is that employees can consume data and information in a way that best suits their role. For example, a senior executive may require a high-level view of what’s happening at the agency, while an analyst may need information about a specific program. To reach that point, agencies have to understand what questions people are trying to answer, how information can be presented to best answer their questions and how the agency can enable employees to drill down and answer detailed questions. These are the types of capabilities that Tableau users depend on, Dolan said. And he’s not just talking about super users who are highly technical, but also non-IT employees who rely on
the Tableau analytics platform to combine data from different sources and find new trends. “It’s not just about creating a dashboard, although that’s very important, but about self-service analytics,” Dolan said. “We want to make data analytics easy for people to use” without creating an added burden for the IT department. By adopting a powerful analytics platform that offers robust self-service capabilities that all employees can use, agencies enable their workforce to improve collaboration and reduce government silos, increase operational efficiency, and improve transparency and security. Dolan highlighted a state department of juvenile justice that has reaped those benefits. In terms of boosting collaboration, analytics allowed this agency to create a more holistic view of atrisk youth in their system and how best to target those with the most pressing needs. This assessment was made using internal data and information from the department of education, children and family services and other agencies. Analytics helped the agency to align data from these sources around each youth and create a collaborative plan to best serve them. To improve efficiencies, the department used analytics to determine which children to assign to each social worker. They used data to determine which pairings would maximize the time social workers spent with each child, by grouping youth in the same vicinity with the same social worker. This cut down on travel time. Another benefit of data analytics is transparency. Government’s at all levels are investing in dashboards that provide performance metrics to the public on a number of programs. In Ohio, for example, residents can view the state’s budget and assess whether agencies are being good stewards of taxpayer dollars. Some federal agencies are also pushing more performance data out to the public in response to the Digital Accountability and Transparency Act. “All of this alignment of data helps to shed a bright light on those mission goals and really allows government organizations to report back with confidence whether they are hitting their goals and objectives, or if they need to make some course corrections,” Dolan said.
Analytics in Action: How Gov Tackles Critical Issues With Data 17
CASE STUDY #4
MANAGING PUBLIC HOUSING
New York City Housing Authority (NYCHA)
is the oldest and largest public housing authority in the United States. Nearly 400,000 residents live in NYCHA developments, or about one in 14 New Yorkers. The agency maintains 2,500 buildings and generates 2.6 million work orders annually.
Zodet Negron, Deputy Press Secretary Sybille Louis, Director for Performance Tracking and Analytics Anne-Marie Flatley, Vice President for Performance Management and Analytics What issues is your organization trying to tackle? Zodet: We’re trying to do more with less, and that’s where I think analytics and technology come in. Our current NYCHA chair wants to make NYCHA a more efficient landlord [and] a more efficient organization, so we launched a long-term strategic plan called NextGeneration NYCHA.
How are you using data and analytics to address those issues? Sybille: We work very closely with our IT department. We developed an Oraclebased performance-tracking dashboard, which allows us to track key performance indicators for the agency — anything from how many vacant apartments we have, rent collection, work order repair and how long it takes to address emergencies. We also use our mapping software to determine geographical patterns and trends.
Can you share a time when your organization used analytics to drive better decision-making? Zodet: We used our analytics and our reporting to look at where were we scoring low in the [Housing and Urban Development Department] assessment. Every housing authority, including NYCHA, has to go out every two years to inspect that [housing] unit to ensure that it’s up to housing quality standards. What we were finding is that we were losing points on our HUD assessment because we weren’t getting enough of those inspections done on time. We looked at the analytics to look at where were we seeing delays in the inspection. Were there particular inspectors that maybe were not completing their routes on time? Maybe the routes had to be adjusted? So we developed these reports to track what were the inspections, what were the number of inspections due that month, how far we were behind and how many did we have to go to catch up. It really helped focus efforts on the inspections that were due. And that was one of the things that helped us become a high performer, was to use these reports to improve the performance of inspections.
A GovLoop Guide 18
What were the resource requirements?
What did you learn and what were the outcomes? Zodet: We started out with a pilot program when we launched NextGeneration NYCHA with about 15 [housing] developments. We’ve continued to add more developments, and we’re finding that these developments are performing at a higher level than our other developments. Sybille: When we first began this program in 2015, we developed the dashboard for those developments. For example, one of the key [indicators] was: How fast are we addressing our simple [maintenance] repairs? We’ve seen a dramatic improvement. When the program began, they were at about 21 days, and now they’re under four days. We’ve always had the data, but in this specific scorecard, we could really see the direction with colors [red, yellow and green]. The visual graph was really instrumental in putting the data out there [so] people can see that at a glance.
Can you share tips on how you got the data you needed and decided what tools/techniques to use? Sybille: The process has to be collaborative. We work very closely with our IT department, [and], we do have data in separate instances. But we have been able to pull the data from the system and create a data warehouse where we can pull data from. The front end is the dashboard itself and creating the actual report.
Anne-Marie: We don’t have a huge team – 18 staff members in the Performance Management and Analytics Department. We’re not an IT department, but what we’ve done through the years is to send some staff through Oracle training. We definitely help out IT that way because they don’t necessarily have the staff that can spend the time figuring out the colors and the query, but they’ll set up the table, then we’ll work with IT as well as the business owner to validate the data.
What was key to your success with analytics? Ann-Marie: Very close collaboration with IT and the business owners [and] really listening to them as to what information they need to manage what they have to do. I would also say investing in your staff because it’s not enough to create the report, but what does the report mean, what does it mean for the future, and what are the past trends?
How are you creating a culture for analytics at your agency? Anne-Marie: It’s been something that has been developing over the years, and now it’s refining and also just having that information be more accessible so that it can trickle down faster to all NYCHA staff, including property managers and maintenance supervisors.
What do you hope to do with data and analytics in the near-term and long-term that you cannot do today? Sybille: I just completed an applied data analytics course. We were introduced to new techniques that we have not done before, such as machine learning, where we can use the power of predictive analytics to help us think about the state of our buildings in the future. We can predict which buildings will have more repair needs. Anne-Marie: We’d also love to do more with GIS.
Analytics in Action: How Gov Tackles Critical Issues With Data 19
Technology for people who are changing our world. 3,000 public sector agencies use Granicus to connect with citizens. Granicus helps turn government missions into quantifiable realities. Granicus products connect more than 150 million people, creating a powerful network to enhance government transparency and citizen engagement. By optimizing decision-making processes, Granicus strives to help government realize better outcomes and have a greater impact for the citizens they serve.
Learn more at granicus.com.
INDUSTRY SPOTLIGHT
HOW ACTIONABLE DATA IMPROVES DIGITAL COMMUNICATIONS
An interview with Matt Macy, Federal Account Executive at Granicus The ability to access vast amounts of data can be helpful, especially when projecting future trends or trying to understand historical events. But the important question is how to make sense of that data and put it to good use. For many government agencies, the issue isn’t a lack of data but rather the inability to quickly analyze it and turn those insights into actions. One area where agencies are working to change that paradigm is in their digital communications with the public. “Actionable data will help you to understand where you should be investing your money, what’s going to be worthwhile and what’s going to make it as easy as possible for citizens to receive your information,” said Matt Macy, Federal Account Executive at Granicus, a leading company in the public-sector digital software space. When agencies can easily analyze a variety of metrics, including social media stats and email open rates, it empowers them to be more strategic with their investments. The data helps them better understand how citizens respond to their digital content. Not only that but they can quickly determine which channels are most effective for communicating with the public and make adjustments accordingly.
understand the effectiveness of their digital communications programs and areas for improvement. “In a lot of ways, analyzing data is like stepping on the scale,” he said. “You have to face the music, whether that’s good, bad, or otherwise because there’s no way to get better without understanding where you’re at now.” When seeing negative results, agencies must learn how to digest the information, understand what isn’t working and respond. But it’s common when agencies begin an initiative that the results are less than ideal. Macy warned against ending an effort before allowing it to fully mature or responding too hastily to data reports. “Data is not always going to give you the perfect answer, but it’s going to give you an educated guess,” he said. He offered several tips for analyzing data. First, it is important to have a third-party perspective that can guide agencies and their teams. The reason? It can be hard to view your data objectively.
“This gives agencies and their communications teams more power in decision-making when it comes to figuring out where to allocate time, money and resources,” Macy said.
Second, agencies should look at their communications data — at most — on a weekly basis. Looking at data daily can be counterproductive and cause agencies to chase after the latest findings. Macy also noted the benefits of using automation for reporting. Even if agencies have experienced communicators, there’s still a lot to ingest. Automation helps target the points that are important and delivers them to agencies quickly and effectively.
They can also compare their efforts and results to those of other agencies using Granicus benchmarking data. “Not only do we give you data, we give you action with the data [by] providing a game plan or insight into what other agencies have done,” Macy said, noting that customers include federal, state and local governments in North America and the United Kingdom.
More than 3,000 government agencies use Granicus’ communications suite to grow their digital audience, build communities around data and better connect with citizens. The GovDelivery Communications Cloud in particular enables agencies to better manage their digital communications across multiple channels, including email and social media.
Several agencies — including the Veterans Affairs and Defense departments — are reaping the benefits of actionable data. Take the International Monetary Fund (IMF), for example. The agency recently had issues informing citizens in the United States and Western Europe about IMF efforts in developing nations. With Granicus’ help, IMF used analytics and performance data from other organizations, including the European Parliament, to learn how they communicate with citizens through blogs and other digital platforms.
Finally, it is important to build action plans around your data. “Data without action is pointless,” Macy said. Having a plan tied in with data helps agencies look at how they did in the past and plan for the future. They can better see the results of engagement reports and develop an action plan surrounding those numbers that will ultimately help them successfully reach their customers.
Macy noted the importance of using analytics to help agencies
Analytics in Action: How Gov Tackles Critical Issues With Data 21
CASE STUDY #5
REDUCING IMPROPER PAYMENTS
The Do Not Pay Business Center
within the Treasury Department’s Bureau of the Fiscal Service provides analytics services to agencies across government. Its goal is to improve the integrity of federal payments and detect, prevent and stop improper payments in addition to payments affected by fraud, waste and abuse of taxpayer dollars.
What issues is your organization trying to tackle? Do Not Pay was created as a way to help agencies address the governmentwide challenge of improper payments. We do that in two key ways. The first is that we have an online portal that automates data matching between different data sources than can help agencies screen against eligibility issues. The second thing that we do is provide analytics services. Legislation has given Do Not Pay six different data sources related to death, federal debt and lists of individuals and parties who’ve been excluded from doing business with the government. Our analytics services are about trying to help agencies at a more programmatic level tackle an improper payment challenge that’s specific to that program.
How are you using data and analytics to address those issues? Our analytics service has developed a few different areas of focus. One is payment integrity controls. These are internal controls [and] data-centric tests that agencies can use to identify risk in a payment stream. For example, if you look at payments to vendors, there may be certain types of vulnerabilities around payment errors or payments that may have been affected by fraud, waste or abuse. And so developing analytics to detect how those kinds of risks show up in the data [helps] an agency flag a payment before they disburse it. Fiscal Service disburses close to 90 percent of the federal government’s payments. We’ve developed a suite of tools that we can offer to agencies where we can go in and do pattern and trend analysis and anomaly detection to try to identify risk in the data that could indicate an improper payment, whether it’s due to administrative error or something intentional.
A GovLoop Guide 22
Joah Iannotta, Director Can you share a time when your organization used analytics to drive better decision-making? The whole process of identifying payments that are at risk is a way of facilitating agency decision-making. We’re providing information on individual payment records that allows an agency to make a determination about whether or not that payment was truly improper.
How are you creating a culture for analytics at your agency? Within Fiscal Service, people crave information. It hasn’t been much of a lift here to get people to be excited and want to leverage analytics. What has been more challenging has been our work with agencies. Getting folks to see how analytics can benefit them has been really important. For us, the solution was being able to talk to agencies about how the information can help them to make good resource decisions and how it can help them to identify improper payments in a way that is efficient and effective. It’s really making that business case for the value of analytics.
Can you share tips on how you got the data you needed and decided what tools/techniques to use? The legislative landscape can be challenging. Make sure that you have excellent privacy and legal staff with you to help navigate that legislative landscape. Our strategy has been to facilitate data sharing to help agencies leverage and harness the power of data. We have the staff who can help agencies walk through that legislative landscape. At the end of the day, if you have amazing data scientists and analysts, that is an incredible resource.
What were the resource requirements? We have about nine data scientists and data analysts on the analytics team, who turn out a lot of work. I think people get hung up on the tools and forget just how important it is to have somebody who can code the data and also translate the results into something that’s actionable and business-oriented.
What do you hope to do with data and analytics in the near term and long term that you cannot do today? We would like to be able to help agencies address a broader range of improper payments. So broadening out the data sources that can touch different kinds of eligibility issues is key. We’ve got the right analytics tools, and now we need to focus on expanding our data resources. A second thing we’d really like to do is research on crossgovernment overlap. That’s when a number of different programs have been asked to tackle a particular [issue], [such as] disaster recovery. And so we’re starting to work on strategies that might help agencies detect instances in which overlap is occurring, in a way that would be in violation of program requirements.
What advice do you have for others in government who want to use data analytics? Patience. Analytics [is] an iterative process, recognizing that like any good solution, it takes refinement and effort.
The other resource that people don’t necessarily think about is [all] the data [that] is available. Finding that right balance between what kind of data you need on hand in order to really put your analysts in a position to do really interesting things and solve problems is another resource balance that you have to think about.
What was key to your success with analytics? Something that’s been extremely important for Do Not Pay is applicability. [It’s] about trying to refine the solution for that particular program so you get true matches. You get high-risk payments and you don’t get false positives. What’s been key to being successful in that space has been a strong partnership with the agency that we’re working with and being able to use an iterative process with them to refine those results.
Analytics in Action: How Gov Tackles Critical Issues With Data 23
10 TIPS FOR BRIEFING YOUR BOSS WITH DATA Using data to brief your boss can be fulfilling and frustrating. Data is powerful and can visualize problems and solutions — sometimes better than words. But what happens when managers don’t know what kind of data they want – they just want to see something? Which metrics are best to use, and how do you show you are doing a good job? How do you frame data and information in a way that is most effective? These are valid concerns that are top of mind for many employees, especially those in government. Speaking at GovLoop’s Next Generation of Government Training Summit, Dave Uejio, Acting Chief Strategy Officer at the Consumer Financial Protection Bureau, shared practical tips on using data to brief your boss. “One of the mistakes people make is that the data steals the show, and it is left to tell its own story,” Uejio said. That shouldn’t be the case. If you’re giving a data-driven briefing, put yourself in your boss’ place, he added. Generally speaking,agency leaders have limited time to focus on many things. Data is leadership’s best approximation of what is happening at the ground level, and they put a lot of faith in it. Make the assumption that more often than not, you have spent more time with the data than the leader. “You are taking a person who has to make an important decision — a decision you are not empowered to make — with the data that they have not read through, with data that they may not be capable of analyzing, and then you’re setting them loose,” Uejio said. “That means that we need to stack the deck toward communication,” he said. “It is so important for us as analysts or people that work in subject-matter areas that we take not just what is interesting about the analysis, but what is imperative about the information and frame that up in a narrative way — in a story.”
A GovLoop Guide 24
Uejio shared these 10 things to remember when leading data-driven briefings: 1. Storyboarding sets the stage for your talk. Don’t think about how wed you are to any one slide in your presentation. And don’t set out to simply design nice slides. Be a ruthless editor and make each one count. 2. Tables provide a baseline comparison. There are many ways to visualize data beyond bar and column graphs. If your boss doesn’t know what she’s looking for, show her the numbers or give her the impression you are showing her the numbers. In that case, tables can provide that insight. 3. Pie charts have clear limits in conveying insights. They are terrible for showing proportionality or comparing different datasets. The only thing worse than one pie chart is two pie charts. 4. Column and bar graphs convey greater depth of analysis. You should have clear axis labels on your graph, and they should be relevant to what you are trying to communicate. Include a title, don’t have hanging decimals, and provide a source and legend to add clarity. 5. Waterfalls can demonstrate proportionality and magnitude. These are a combination of bar or chart graphs. They are useful for financial analysis, and they provide a clean way to tell consistent stories. Plus, they don’t take up a ton of real estate.
7. False proportionality creates abstract perspectives. You have to balance ruthless storytelling with a baseline level of rigor and integrity. 8. Y even have an axis? Bottom line: Don’t screw around with axes and tell misleading stories with data. Proportionality matters, and it’s on you to conduct analyses in a responsible way. 9. Draw upon open source design communities to develop a unique palette. Color.adobe.com is an open source community of people who design palettes. Using color palettes gives your presentation a unified look and feel. Also, fonts set the tone for your assertions, so do what you can to differentiate your work. Whatever you do, don’t let work go out of your shop that is not pristine. You don’t want people to question the veracity of your report. 10. Pictures directly invoke emotions. Check out Flickr and thenounproject.com for neat visuals.
It’s imperative that you learn the preference and style of the person you are briefing, even if you have to get that information secondhand, Uejio said. “You are most likely to be successful if you hone in on your audience and structure it for their learning and decision-making.”
6. Infographics can be framed to tell a story. It isn’t fair yet to assume everyone in an analyst role is a graphic designer. Bosses don’t expect that. But there are plugins online that you can use to create infographics.
Analytics in Action: How Gov Tackles Critical Issues With Data 25
TIPS FOR SUCCESS We compiled this easy-to-read one-pager with helpful tips from our experts on using data analytics. 1. 1 Roll up your sleeves and dig in. 22. Expertise comes with experience and trial and error. When you don’t know the answer, don’t hesitate to ask questions. 33. Don’t be afraid of data analytics. It’s a powerful tool to be used in combination with other resources. Use it as an enabler. 44. Bring people along on your analytics journey. Continue to show value by producing small wins. 55. Share your data. 66. Work with your business owners to understand what information they need to be more efficient. 77. Build relationships and develop trust among the groups providing data to you and receiving data from you. They have to trust that you will listen to them and develop an understanding of what the data means. 88. You can’t use a one-and-done approach and declare victory when developing analytics capabilities. Analytics requires an iterative approach and patience.
A GovLoop Guide 26
CONCLUSION Data analytics is transforming the way government agencies tackle our nation’s most pressing issues, whether that’s combating opioid abuse or ensuring veterans receive quality care. Thanks to ongoing advancements in data collection, storage and analytics tools, agencies can better track trends and even plan for future events to get ahead of pending problems. For agencies, the ultimate goal is to make data-driven decisions in real time. They want to use predictive analytics to improve how they allocate limited resources and effectively execute their missions. They’re even marrying GIS with analytics capabilities to get a more accurate picture of conditions at the local and micro levels. The challenge is getting access to data from internal and external stakeholders and ensuring it is accurate, timely and relevant. But don’t let the hurdles deter you. Ask questions, share ideas and invite others to join you on your analytics journey.
ABOUT GOVLOOP
THANK YOU
GovLoop’s mission is to “connect government to improve government.”
Thank you to Granicus, Information Builders, MicroStrategy and Tableau for their support of this valuable resource for public sector professionals.
We aim to inspire public-sector professionals by serving as the knowledge network for government. GovLoop connects more than 250,000 members, fostering cross-government collaboration, solving common problems and advancing government careers. GovLoop is headquartered in Washington, D.C., with a team of dedicated professionals who share a commitment to connect and improve government. For more information about this report, please reach out to info@govloop.com.
AUTHORS Nicole Blake Johnson, Senior Editor, Technology Lucy Sears, Editorial Fellow
www.govloop.com | @GovLoop
DESIGNER Kaitlyn Baker, Lead Graphic Designer
Analytics in Action: How Gov Tackles Critical Issues With Data 27
1152 15th St. NW Suite 800 Washington, DC 20005 P: (202) 407-7421 | F: (202) 407-7501 www.govloop.com @GovLoop
A GovLoop Guide 28