Feasibility Analysis Final Report

feasibility analysis final report

Data Management Partnership You can’t manage what you don’t measure

contents Executive Summary...............................................................................................................................I Introduction and Background..............................................................................................................1

Document Structure and Feasibility Questions.........................................................................2 Methodology.............................................................................................................................3 Feasibility Question 1..........................................................................................................................5 What benefits would your agency receive by sharing data? What is the value to the collective community of sharing data among agencies?.......................................................................................5

Is there a need for a Calgary Child and Family Community Data Repository?.........................9

Advisory Committee Findings and Recommendations...........................................................10 Feasibility Question 2........................................................................................................................11 Is there a core group of agencies willing to move forward to create a Calgary Child and Family Community Data Repository? Is there leadership capacity and commitment to do so?.....................11

Knowledge Management Organizational Prerequisites for Capacity to

Data Share...............................................................................................................................11

Understanding Organizational Culture...................................................................................13

Collaborating for Data Sharing...............................................................................................15

Advisory Committee Findings and Recommendations..........................................................16 Feasibility Question 3........................................................................................................................17 What resources – financial, capital, and human – need to be available to ensure success of a Calgary Child and Family Community Data Repository?..............................................................17

Advisory Committee Findings and Recommendations...........................................................19 Feasibility Question 4........................................................................................................................20 What governance structures need to be developed within agencies to ensure success of a Calgary Child and Family Community Data Repository?..................................................................20

Privacy and Security of Data...................................................................................................25

Funder Considerations...........................................................................................................27

Advisory Committee Findings and Recommendations..........................................................28 Conclusion..........................................................................................................................................29 Advisory Committee Action Plan – Phase Two...............................................................................30 References..........................................................................................................................................31 Appendix A – Community Engagement Process............................................................................36 Appendix B – Bi-weekly Communique Example............................................................................38 Appendix C – Summary of Data Sharing Models..........................................................................39

executive summary Data Management Partnership Feasibility Analysis

Kerry Coupland, Praxis & Theoria Inc. Dawne Clark, Centre for Child Well-Being, Mount Royal University Elaine Danelesko, Integrative Health Institute, Mount Royal University

8/15/2013

executive summary

I

“A little knowledge that acts is worth infinitely more than much knowledge that is idle.”

– KAHIL GIBRAN (AS CITED IN (CALABRESE & ORLANDO, 2006, P. 253)

executive summary

II

introduction & background With financial support from The Calgary Foundation’s Community Grant Program, the Centre for Child Well-Being and Integrative Health Institute at Mount Royal University (MRU) partnered to facilitate a year-long investigation into the feasibility of developing a Calgary-wide shared data system or data sharing initiative for child, youth, family, and human service providers. In pursuit of determining feasibility, a review of both academic and grey literature was completed and community conversations with stakeholders were held. Key informant interviews with persons involved in the development of shared data systems in other provinces and countries were conducted to supplement information obtained in the literature review. The present summary provides the feasibility analysis findings and recommendations resulting from the community engagement process. It is organized by the project’s four questions that were used to determine feasibility.

executive summary

III

feasibility questions 1

Is there a need for a Calgary Child and Family Community Data Repository?

Findings •

•

•

There is a need for, and it is feasible to create, a Calgary Child and Family Community Data Repository. Currently, there is not a mechanism to communicate a broader picture of service provider accomplishments to funders and the public at large in Calgary, including how we are doing overall regarding child well-being; or, how we are supporting families to thrive; or; how we support families to be successful and resilient. The proposed data repository holds the potential to respond to this need.

The repository needs to gather quality aggregate community-level child and family service data that can be used for multiple reporting purposes, including: trends, gaps, fund development, planning, assessment, outcome evaluation, program improvement, education, and impact.

The Committee debated the differences between a shared database which collects data according to commonly defined variables, outcomes and indicators, and a data repository which collects quality data to answer specific questions but without identical variable definitions, outcomes and indicators. It was felt that a data repository was the best first step as it is not currently feasible to develop a shared database.

Recommendations •

The repository focuses on children’s wellness from a cross-sectoral perspective.

•

The repository is hosted on a public open access website.

•

The repository targets service providers, The Calgary Foundation donors, agency funders, and Calgary citizens.

•

The creation of the repository continues to be a community-driven process.

•

To safeguard the value and utility of the data repository, only quality data are included.

•

Data are organized and analyzed around a central question that captures the imagination and interest of many Calgary agencies. Examples include: Are Calgary families thriving? How are we supporting families to be successful and resilient?

A snapshot of child well-being in Calgary.

•

The central focus question is determined by the community and the data repository leadership.

•

Data are categorized using a framework of indicators.

executive summary

IV

2

Is there a core group of agencies willing to move forward to create a Calgary Child and Family Community Data Repository?

Is there leadership capacity and commitment to do so?

Findings •

3

There is a core group of motivated agencies willing to champion a Calgary Child and Family Community Data Repository. Partners to date include: Homeless Foundation, Food Bank, City of Calgary Family and Community Support Services, SouthWest Community Resource Centres, an independent researcher representing the Domestic Violence Sector, Elizabeth Fry, Closer to Home, Calgary Reads/First 2000 Days Network, Families Matter Society, Rocky View Schools, UpStart (United Way of Calgary and Area), ECMap, independent consultant with expertise in collective Impact, MRU Institute for Non-Profit Studies, MRU Centre for Child Well-Being, and MRU Integrative Health Institute.

•

The leadership capacity and commitment is present to create a Calgary Child and Family Community Data Repository.

•

It is clearly understood that not all agencies in Calgary currently have the capacity to collect and use data.

Agencies need support to learn how to collect quality data, what data to collect, and how to analyze and use collected data. However, comprehensively addressing this need is beyond the scope and capacity of the data repository project.

•

Agencies with capacity to collect and use data are already often over-burdened. Agencies often have to report to multiple funding organizations using multiple report formats and data.

Recommendations •

The data repository begins with a small number of committed Calgary and area partners who share a focused agenda and purpose. When success is built, then leadership may explore data repository expansion.

•

Ensure cross-sectoral participation in the data repository.

•

Resources are needed to support and ensure participation in the repository, available to any interested organization. A specific capacity building component should be included in the repository project. However, this does not include the provision of evaluation consultants in a traditional form (i.e., how the Canadian Outcomes Research Institute (CORI/HOMES) database operated).

What resources – financial, capital, and human – need to be available to ensure success of a Calgary Child and Family Community Data Repository?

Findings •

Within Calgary, there are agency specific assets and expertise that could benefit the data repository (e.g., GIS mapping, social economics, experience with large databases, and so on).

executive summary

V

Recommendations

4

•

Leverage committed partners’ existing social capital and technology assets in the form of human resource talents and technological infrastructure.

•

Specific staff roles required to support the repository project:

2. Data Scientists or Wizards – answer specific questions using the data, determine quality of data, provide data analysis, generate reports and narratives from repository data, map the data, and more generally turn the data into a more usable form of information and knowledge and

1. Data Connectors or Masters – resource persons who provide general support to agencies, lead capacity building efforts, ensure project does not duplicate others’ efforts, and possibly provide advocacy on behalf of the cross-sector partnership

3. Creative Communicator – presents the analysis in appealing and creative ways that can be readily understood by less experienced employees, funders, and the public (e.g. narratives, video presentations etc.)

What governance structures need to be developed within agencies to ensure success of a Calgary Child and Family Community Data Repository?

Findings •

The repository should not be linked to any single funding organization or source of support.

•

Commitment from senior leadership within an agency is required for successful participation in a data sharing initiative.

Requirements

1. Leadership team: • requires partner representatives who

can commit • agency resources and capacity • develops the focus questions for the

data repository • identifies the indicator framework and

organizing structure of the data

repository, and • engages community stakeholders and

new partners 2. Human Resources: • proposed positions (Data Connectors,

Data Wizards, and Creative

Communicators) may currently exist

and be available to be shared across

partner agencies 3. Infrastructure includes: • technological infrastructure: data

repository and storage • website • data sharing agreements and • consultative staff

executive summary

VI

conclusion Whether explicit or implicit, the idea of transparency, collaboration, and open exchange of data are a foundation for all data sharing initiatives.

Harnessing the power of data, information and knowledge for the promotion of public good will help build the capacity to meet the urgent health and human service needs. As one agency alone cannot solve the complex web of interwoven determinants of social ill, it will require the effort of coalitions of independent organizations, many agencies in collaboration, empowered by shared knowledge to generate sustained impact (Walker et al., 2012). There is a readiness in the Calgary human service sector to begin embracing data for social change.

executive summary

VII

advisory committee action plan – phase two 1

Create a five-year multi-stage plan for the development of the data repository, which includes a distinctly defined ‘do-able’ project with a small number of agencies in Year One.

2

Submit a grant application to The Calgary Foundation for $75,000-$100,000 for Phase Two of the DMP project.

3

Explore funding opportunities beyond The Calgary Foundation, including the possibility of corporate support or sponsorship.

Also consider in-kind donation of expertise.

4

Explore partnerships with other organizations such as the Alberta Centre for Child Family and Community Research (ACCFCR) and Rocky View Schools which have expressed interest and have existing data storage capacity, data analysis skills, and data sharing agreements.

5

Identify opportunities for integration with existing initiatives.

Approach UpStart (United Way of Calgary and Area), Calgary Homeless Foundation, Calgary Community Data Consortium (City of Calgary) to explore how initiatives can be integrated rather than replicated.

6

Collect aggregate data only to circumvent privacy and confidentiality issues.

7

Educate agencies on the Freedom of Information and Protection of Privacy Act of Alberta (FOIP) and the Personal Information Protection Act (PIPA), and clarify misperceptions and confusion around data sharing and compliance with these legislative acts.

8

Explore the use of the Collective Impact Framework for implementation of the data repository.

feasibility analysis final report The Feasibility of a Data Management Partnership

Kerry Coupland, Praxis & Theoria Inc. Dawne Clark, Centre for Child Well-Being, Mount Royal University Elaine Danelesko, Integrative Health Institute, Mount Royal University

8/15/2013

introduction

1

introduction & background Using data to support the creation of actionable intelligence in the service of society is increasingly finding momentum within the non-profit sector. While collecting, tracking and using data to improve products and services has long been a key strategy employed by the for-profit sector to increase revenue and ensure sustained market share, understanding the utility of data has been relatively more slowly embraced by the non-profit sector (Kanter, 2011; Renshaw & Krishnaswamy, 2009). This has been especially true for smaller organizations with limited resources and capacity. However, as demand from donors and funders for evidence of the impact of their investment grows, and the cost of technology to collect, store and analyze data lessens, the interest across the non-profit sector in better understanding data and what benefits can be derived by systematically utilizing it, increases. There is also growing sentiment that the current system of non-profit operation is challenged by inefficiencies, ineffectiveness, redundancy, and overly idealistic visions (Tapia, Maldonado, Tchouakeu, & Maitland, 2012). And concurrently, there is an expanding appreciation for the idea of Collective Impact, a framework for mobilizing multiple organizations and sectors around one social issue in the pursuit of social change (Kramer, Parkhurst, & Vaidyanathan, 2009; UN Women, 2012). The confluence of these disparate yet overlapping trends provides the impetus for the present project and the creation of the Data Management Partnership (DMP).

With financial support from The Calgary Foundation’s Community Grant Program, the Centre for Child Well-Being and the Integrative Health Institute of Mount Royal University (MRU) partnered to facilitate a year-long investigation into the feasibility of developing a Calgary-wide shared data system or data sharing initiative for child, youth, family, and human service providers.

Confidence in the principles of the community development approach ensured the community’s leadership and engagement in the project. Not only is this imperative for developing a detailed and accurate understanding of what the community’s capacity presently is, and where it would like to evolve, but importantly, this approach builds a strong foundation for later success.

The DMP is a collaboration of data-interested Calgary community-based organizations. It is governed by a Steering Committee and Advisory Committee who provided oversight of the feasibility analysis, and are directing the long-term vision of the initiative.

introduction

2

Document Structure & Feasibility Questions The document is structured around four feasibility questions created by the Steering Committee. The four feasibility questions are:

1

What benefits would your agency receive by sharing data? What is the value to the collective community of sharing data among agencies?

Is there a need for a Calgary Child and Family Community Data Repository?

2

Is there a core group of agencies willing to move forward to create a Calgary Child and Family Community Data Repository? Is there leadership capacity and commitment to do so?

3

What resources – financial, capital, and human – need to be available to ensure success of a Calgary Child and Family Community Data Repository?

4

What governance structures need to be developed within agencies to ensure success of a Calgary Child and Family Community Data Repository?

Each feasibility question section integrates the theoretical, empirical, and practice-based findings related to determining feasibility and readying the non-profit community for engagement in a shared data, information, or knowledge management initiative as identified in the literature review. It then presents relevant community dialogue results, the outcomes of interviews with operating data sharing initiatives, and the sections conclude with the final findings and recommendations from the Advisory Committee. The document is structured to enable the reader to quickly review the results; as such they are presented in combination of paragraph and point form.

introduction

3

Methodology The feasibility analysis included two distinct processes: facilitating community engagement and the completion of an academic literature review. Both of these processes were embedded within a Collective Impact framework.

Community Engagement The process of engaging the Calgary community to determine the degree of interest and support for data sharing began with the formation of the DMP Steering Committee, a group of stakeholders who conceived and developed the feasibility study. The Steering Committee comprised nine members, and met nine times from August 2012 to May 2013, usually by telephone conference (see Appendix A for a diagram of the community engagement process and list of committee members). With the deliberate intent to involve diverse stakeholders who represent a broad range of community-based services, including agencies and organizations not routinely involved in community conversations and dialogue of this nature, the Steering Committee directed the use of multiple strategies in the identification and engagement of participants.

To begin, a data set of non-profit organizations in Calgary was created using listings and networks provided by the members of the Steering Committee and Calgary’s 211 registry. In total 225 community stakeholders were identified and invited via email to attend a community consultation on data sharing. Invitees represented a considerable range of organizations with regards to size and organizational priority areas. These organizational areas included: poverty, child maltreatment, immigrant services, aboriginal services, sport and recreation, and child care. Thirty-five agencies attended the October 18, 2012 community consultation. The meeting included an introduction to the project, a presentation on the Collective Impact framework driving the project process, and an invitation for further participation in three possible ways: at a subsequent focus group, as an Advisory Committee member, or as an active stakeholder.

Two follow-up focus groups were facilitated by Mike Bowerman, a social impact and evaluation consultant. The focus groups were designed to generate unconstrained dialogue and to gain a better understanding of the 1

current knowledge, attitudes and practices regarding data management in the human service sector in Calgary.

The first focus group, held November 21, 2012, engaged ‘heavy’ data users with established data management systems and data collection processes, both agencies and funders. The second, held November 23, 2012, concentrated on agencies less experienced in data management but who were interested users of data.

Selected examples and themes gathered in the focus groups have been included throughout the document as part of the ‘community dialogue’ findings.

A DMP Advisory Committee was established with the intent to assist the DMP team determine the feasibility of a data sharing project, and if deemed feasible, to assist in developing an implementation plan, and conceptualizing and preparing follow-up grant applications. The nine members met five times from February 2013 to May 2013. The last meeting on May 30, included the Steering and Advisory Committees as well as potential partners and funders, directed the final feasibility findings and recommendations articulated in this report, as well as the action plan for Phase Two. Members of both the Advisory and the Steering Committees were given the option to continue to be involved in Phase Two and will be providing grant submission support over the summer of 2013.

Upon the recommendation of the Advisory Committee, supplementary telephone interviews were completed with key agencies in Calgary who had not otherwise participated in the DMP community engagement. The intent was to gain further insight into the viability of a data sharing initiative in Calgary. The four feasibility questions guided the telephone interviews, which were completed June 7 to 20, 2013. The themes and content from the interviews are included throughout the document as part of the ‘community dialogue’ findings.

A total of 211 of the original 225 stakeholders remained ‘active stakeholders’. They received DMP bi-weekly communiques as part of a modified Delphi-process to encourage the continued engagement of the community in data sharing conversations1. The communiques included an educational portion, where relevant data sharing information was briefly explored, and an opportunity to build the collective knowledge of current data practices and data management needs of Calgary stakeholders

See Appendix B for a sample or visit http://www.mtroyal.ca/ProgramsCourses/FacultiesSchoolsCentres/

CentreforChildWell-Being/dataManagement.htm for the complete series of bi-weekly communiqués.

introduction

4

via participation in short DMP survey. Through social networking promotion and word of mouth, the distribution list for the bi-weekly communique increased to over 1,400 stakeholders during the five months of community engagement.

Literature Review A systematic approach to constructing the literature search strategy was taken to ensure the process was as transparent and reproducible as possible. The intent of the literature review, however, was not to be an exhaustive review of evidence but to be a more precise examination to inform the feasibility analysis. Working with members of the Steering Committee, the broad research question chosen was: is it feasible to develop and implement a Calgary-wide data management system for child, youth, family, and human service providers? At the time the literature review was conducted, the four feasibility questions had not been developed. Therefore, this research question guided the review.

Of primary importance to the Steering Committee was an understanding of current shared data or knowledge management examples operating nationally and internationally. Given multiple cultural and legislative differences, an emphasis on interviewing Canadian models was chosen. After the strategy was approved by the Steering Committee, it was then reviewed by an MRU academic librarian and modifications were made to narrow scope. The search incorporated multiple databases and multiple methods of manual searching (e.g. SocoINDEX, Web of Science, LISTA, SCOPUS, Computer Science Database). Bibliographies and reference lists were scanned and hand searches of relevant journals completed. The database search strategy restricted literature to that published between 2007 and present unless it was considered a seminal document, and also restricted sources to those written in English.

The same research question guided the exploration of grey literature using the internet search engines Google and GoogleScholar. No date restrictions were applied and the search string was considered complete when two consecutive search result ‘pages’ did not yield new or relevant material. Given the infancy of data and knowledge management as applied to non-profit and communitybased organizations, it was expected that the grey literature search would result in a large number of relevant sources. Indeed, this was the case. The need for less exact

search terms resulted in a greater number of feasibility relevant documents, directed much of the hand searching of specific journals, and the manual search of specific authors and organizations. Not all of the articles included in the review are specific to the non-profit sector or data management. All are, however, related to information and knowledge management and may be applied to work in the non-profit sector. The literature search commenced January 2013 and finished at the end of February 2013.

Follow-up with key informants and the addition of relevant information as directed by the Steering and Advisory Committees resulted in new information being continuously acquired until the conclusion of the feasibility portion of the project in June 2013.

In determining the feasibility of developing a data sharing initiative in Calgary it is important to understand current data sharing models being employed in other jurisdictions.

Ten models relevant to this project were identified using the internet search engine Google, and by a snow ball technique whereby interviews regarding one data sharing model resulted in locating other models. Of the ten identified six were Canadian, three were American, and one was Australian. Two Canadian models were applications of another model, and therefore were not investigated. Of the remaining eight models, six responded to requests for an in-depth interview and five interviews were successfully completed.

2

Models Chosen : 1.

2.

3.

4.

5.

6.

7.

8.

2

CommuntyView Collaboration (CVC) Community Indicators Victoria (CIV) Connecticut Nonprofit Strategy Platform (CNSP) Community Accounts a. Newfoundland Community Accounts (NCA) b. Nova Scotia Community Counts (NSCC) c. PEI Community Accounts (PCA) Greater New Orleans Community Data Center (GNOCDC) PEG (PEG) Policy and Analysis Center (PAC) Wellbeing Toronto (WT)

See Appendix C for brief descriptions of each of the Models reviewed.

feasibility Q: one

5

feasibility question one 1

What benefits would your agency receive by sharing data? What is the value to the collective community of sharing data among agencies?

Benefit 1: Advocacy, Policy Analysis and Development Advantages As identified in the literature, data sharing may: •

• • • • •

• •

Facilitate a comprehensive understanding of the ‘chain of consequences’ of a series of services and programs.

Without this understanding it is difficult for decisionmakers both within an organization and across a sector, or at a policy or system level, to understand and identify the impact of current decisions and policies, or what the best policy solutions might be (Kumar, n.d.).

Create a stronger advocacy position for the non-profit sector (Kumar, n.d.).

Ensure greater accountability to clients, funders, and the general public (Gil-Garcia, Chengalur-Smith, & Duchessi, 2007).

Cultivate positive public-image, which strengthens the sector (Gil-Garcia et al., 2007).

Enable the optimal allocation of resources (Bissell & Miller, 2007; Eglene & Dawes, 1998; Kumar, n.d.).

Improve quality of programs and services and facilitate better prediction of program success (Boruch, n.d.; Gil-Garcia et al., 2007; Kumar, n.d.; U.S Department of Justice, 2006).

Facilitate the longitudinal analysis of the cost and benefit of interventions (Carlson et al., 2011).

Assist funders and policy makers who want to support effective programs (Carrilio, 2008).

feasibility Q: one

6

As identified in the community dialogue, data sharing may: •

• • • • • • • •

Facilitate the collecting of data in service of the greater good, which may provide the community with a more holistic understanding of the social issue(s) and another perspective on the ‘real story’.

Assist the sector in communicating the differences its work is making in the community.

Permit the better quantification of impact.

Produce tangible evidence of service sector successes.

Enhance the understanding of long-term impact of programming.

Identify services and programs having the greatest impact.

Help community-based organizations and non-profit sector with service and program planning.

Demonstrate accountability to clients, funders, and larger community.

Create a more powerful policy position and stronger opportunities to change malfunctioning systems.

As identified in interviews with existing data sharing initiatives, data sharing:

“... Permits initiatives to uniquely link ‘hard’ data with ‘real-life’ stories (GNOCDC; PEG).

The premise is that, Stories allow users to gain another level of information and knowledge. Stories... are personal accounts of people with lived experience and individuals helping to make a difference, tied to a particular indicator or theme.The stories are chosen to: i) inspire and empower others to

take action; ii) help improve general understanding of

an issue; iii) illustrate the impacts of an action, to

serve as a tool for sharing best

practices and celebrating success; and iv) help establish why, how and who is

affecting change in a given indicator.”

– (About Peg: Why does Peg use stories? 2013).

Benefit 2: Assisting in Need Identification As identified in the literature, data sharing may: •

•

At a population level, permit viewing a more complete set of data. This helps in the identification of overserved and underserved populations, gaps in services and programming, and successful intervention strategies (Kumar, n.d.).

At individual or client level, allows linking unique individual data, which better supports a client’s individual need and also permits better coordination of services (Carlson et al., 2011).

As identified in the community dialogue, data sharing may: • •

Assist in decreasing gaps and overlap in programming and services.

Facilitate the more easily mapping of community assets.

As identified in interviews with existing data sharing initiatives, data sharing may: •

Bring together data from a variety of organizations and sectors to gain a more complete understanding of the community (CVC, PEG, WT).

Benefit 3: Technical Infrastructure Advantages As identified in the literature, data sharing may: • •

•

Promote the development of formal data standards and consistent practices across the sector and community (Eglene & Dawes, 1998).

Decrease duplication of data collection, processing, storage, and increases effectiveness and efficiency (Bissell & Miller, 2007; Gil-Garcia et al., 2007; Motorola, 2009; U.S Department of Justice, 2006).

Facilitate ability to employ technology that would exceed smaller organizations’ resources. For example, creating quality spatial data requires expensive GIS programming that would be unobtainable for many non-profits (Bissell & Miller, 2007; Gil-Garcia et al., 2007).

feasibility Q: one

7

Benefit 4: Having Timely Data – the Right Information at the Right Time (Markets for Good, n.d.)

As identified in the literature, data sharing may: • • •

Facilitate the ability to locate the specific ‘touch points’ across a community, which assists in ensuring fewer people ‘fall through the cracks’ (Carlson et al., 2011).

Allow decision-makers to electronically access and exchange crucial information in real-time (U.S Department of Justice, 2006).

Provide the availability of real time data, which allows for immediate course correction and the almost instantaneous handling of community problems and crises (American Public Human Services Association., 2010).

As identified in the community dialogue, data sharing may: •

As identified in the community dialogue, data sharing may: •

• • •

Facilitate a shorter intake/assessment process that respects clients’ time, privacy and comfort and ensures clients do not have to give the same data and information to each agency where they access support.

Promote the formation of a common language for data collection and communication within the sector.

Allow a two-way data information system between agencies and funders.

Encourage donors and funders to prioritize building non-profit data collection capacities.

As identified in interviews with existing data sharing initiatives, data sharing may: •

Support smaller organizations with limited budgets to access and use technology that would be beyond their individual organizational resource capacity (e.g.

GIS mapping functions) (CVC, WT).

Encourage the development of the infrastructure required to obtain timely and relevant data. This may include: • a designated data management role within

their organization, • ongoing education and training around data,

especially important given high staff turnover, and • buy-in and active management of data collection

from program managers.

As identified in interviews with existing data sharing initiatives, data sharing may: •

Facilitate the delivery of quality data to local key stakeholders at critical times, and thereby enable better and more evidence-based decision making.

To this end many initiatives produce their own reports and publications with the data housed on their websites. In particular, PAC articulates the desire to create actionable intelligence for social policy, and the GNOCDC develops action-oriented research.

For these organizations, the analyses of data and translation into usable data products is cited as a critical piece of their work. As an illustration, the GNOCDC described their documents as foundational to work in housing and elder support being completed in New Orleans.

feasibility Q: one

8

Benefit 5: Monitoring and Evaluation Advantages

As identified in the community dialogue, data sharing may: •

As identified in the literature, data sharing may:

•

•

•

• • • • • •

Improve sector data quality (Eglene & Dawes, 1998; Motorola, 2009; U.S Department of Justice, 2006).

Improve the ability to evaluate impact at the level of service-provider (Carlson et al., 2011).

Permit better program and service refinement through enhanced monitoring and evaluation (Carlson et al., 2011).

Deepen the understanding of evaluation for both the funder and the sector (Carlson et al., 2011).

Enhance the quality and detail of reports to funders (Drezelo & Lepore, 2008; Eglene & Dawes, 1998).

Improve the ability to predict and change outcomes (Bissell & Miller, 2007).

Improve the capacity for sector-wide evidence-based decision-making (American Public Human Services Association., 2010; Bissell & Miller, 2007; Carrilio, 2008; Drezelo & Lepore, 2008; Jones et al., 2012; Motorola, 2009). This systematic method requires the ability to record service and program delivery and track outcomes, which strengthens organizational decision-making abilities and improves the availability of actionable administrative information (Carlson et al., 2011; Gil-Garcia et al., 2007).

• •

Encourage better understanding of the long-term impact of services.

Enhance and build organizational data management capacities.

Assist in program and service evaluation and refinement.

Create a data system that would permit more easy entry and retrieval of data and information.

Increase an organization’s capacity and capability to perform data analyses.

As identified in interviews with existing data sharing initiatives, data sharing may: •

Meet a local need for neighbourhood level data and to meet a need, sometimes combined with other objectives, to measure community well-being.

Community indicators of well-being were generally holistically understood across the models and include such themes as sustainability of built and natural environments and resiliency of local economies (e.g. economic activity, employment, income and wealth) in addition to health, basic needs, social vitality and so forth. For data models interviewed who use community indicators, development of their indicators was completed using community engagement and a community development processes.

feasibility Q: one

9

Benefit 6: Stronger Collaborations and Partnerships

• •

As identified in the literature, data sharing may: •

•

Build trust, accountability, and strengthen business practices of the sector (Howard, 2012; Lecarpentier, 2012; Purcal, Muir, Patulny, Thompson, & Flaxman, 2011).

Strengthen the professional networks and advance service coordination by minimizing overlap and redundancy in programming and services (Boruch, n.d.; Kumar, n.d.; U.S Department of Justice, 2006).

As identified in the community dialogue, data sharing may: • • • •

Increase collaboration and networking.

Enhance knowledge of available agency programs and services for better referral partnerships.

Identify how organizations and the sector can work better together, especially around data management.

Collapse silos and thereby increase communication and collaboration.

As identified in interviews with existing data sharing initiatives, data sharing is most successful when: • •

•

•

•

Initiatives use a collaborative structure and emphasize community engagement.

Initiatives use a group of committed, cooperative, and excited individuals who see the idea of data sharing as valuable and worthwhile on community wide basis, and are willing to continue to work for it (CVC, WT).

Initiatives devote a large portion of resources in money and time to engaging the community. The actual web-based platform is but a small reflection of the magnitude of effort that is spent in developing it.

Engagement creates buy-in and ensures sustainability of the system from perspective continued participation and funding (PEG).

Initiatives are built for the local context - this requires learning what the community needs and wants via community engagement. Successful initiatives gather extensive feedback to ensure the model uses measures that matter to community and meet their unique needs (GNOCDC, PEG, WT).

Engagement is broad to ensure community buy-in, not just government buy-in. Citizens need to feel involved and understand its benefits (PEG; GNOCDC).

•

Good relationships with data providers from a variety of domains are developed (WT, GNOCDC) Initiatives use collaboration to develop indicators, which permits people to contribute expertise and not feel as though they are being used for their data.

This helps stakeholders feel more involved and they are more likely to be continuous contributors to the initiative because they are shaping its creation.

Collaboration leads to participants’ investment in its success (PEG, WT).

Initiatives prioritize the ongoing maintenance of collaborations and partnerships. Partnerships and relationships require constant work to be sustained, otherwise they quickly atrophy. Changes in leadership and administration can alter the level of an organization’s commitment. Therefore successful data sharing initiatives continuously cultivate relationships to ensure sustained involvement (PAC, PEG, WT).

Is there a need for a Calgary Child and Family Community Data Repository?

Yes, the conclusion of the Advisory Committee is that there is undoubtedly a need for a Calgary Child and Family Community Data Repository. This finding was supported by the results of the community engagement processes.

Most of those engaged in the DMP community dialogue agreed that there was a need for a repository, or if they did not identify it as need believed that it could be greatly beneficial. However, there was a small minority who expressed reservation for fully embracing the idea until they better understood how it would operate, its goals and objectives, and how it would benefit their organization specifically. Furthermore, there was a sense of ‘cynicism’ and articulated reluctance on the part of a few to embrace any new initiative given a history of projects making promises that were not kept, or collaborations departing from their original intent when compromises are made to make it operational.

feasibility Q: one

10

Advisory Committee Findings & Recommendations Findings: •

There is a need for, and it is feasible to create, a Calgary Child and Family Community Data Repository. Currently, there is not a mechanism to communicate a broader picture of service provider accomplishments to funders and the public at large in Calgary, including how we are doing overall regarding child well-being; or, how we are supporting families to thrive; or; how we support families to be successful and resilient. The proposed data repository holds the potential to respond to this need.

•

The repository needs to gather quality aggregate community-level child and family service data that can be used for multiple reporting purposes, including: trends, gaps, fund development, planning, assessment, outcome evaluation, program improvement, education, and impact.

•

The Committee debated the differences between a shared database which collects data according to commonly defined variables, outcomes and indicators, and a data repository which collects quality data to answer specific questions but without identical variable definitions, outcomes and indicators. It was felt that a data repository was the best first step as it is not currently feasible to develop a shared database.

Recommendations: •

The repository focuses on children’s wellness from a cross-sectoral perspective.

•

The repository is hosted on a public open access website.

•

The repository targets service providers, The Calgary Foundation donors, agency funders, and Calgary citizens.

•

The creation of the repository continues to be a community-driven process.

•

To safeguard the value and utility of the data repository, only quality data are included.

•

Data are organized and analyzed around a central question that captures the imagination and interest of many Calgary agencies. Examples include: Are Calgary families thriving? How are we supporting families to be successful and resilient? A snapshot of child well-being in Calgary.

•

The central focus question is determined by the community and the data repository leadership.

•

Data are categorized using a framework of indicators.

feasibility Q: two

11

feasibility question two 2

Is there a core group of agencies willing to move forward to create a Calgary Child and Family Community Data Repository? Is there leadership capacity and commitment to do so?

This section reviews the literature on what is required to share data, information and knowledge. It is followed with a short summary of community dialogue and the findings of interviews with operating data sharing initiatives.

Knowledge Management Organizational Prerequisites for Capacity to Data Share “Non-profit organizations lack the critical processes and knowledge needed to help them develop, evaluate, document and share successful programs.� – (Hurley, Green, & Antonio, 2005, p. 5)

As identified in the literature: Establishing good data and knowledge management practices at the individual non-profit level is the foundation for good data and knowledge sharing across organizations in the future. Certain prerequisite organizational attributes are needed before a crossorganization culture of sharing can be fostered.

Prerequisite 1: Organization understands the key processes in data, information and knowledge management (American Public Human Services Association., 2010) The key processes involved in data and knowledge management include: 1. performing data collection and storage 2. conducting data analyses 3. establishing data access policies 4. facilitating data sharing within the organization 5. ability to disseminate and communicate information effectively

feasibility Q: two

12

6. establishing training and continuous quality improvement protocols 7. setting governance standards for data such as • forming an information management

advisory committee • determining data access and level of authority 8. determining accountability for managing agency performance.

Prerequisite 2: Organization has a multi-year data, information, and knowledge management plan (blueprint): (American Public Human Services Association., 2010; Eckerson, 2013) The data, information, and knowledge management plan (blueprint) should identify and establish: 1. Leadership’s vision for managing data, information, and knowledge. This should reflect their: • support for a coordinated and collaborative

knowledge management approach within agency • understanding that knowledge management is

cultivated by highest level of management • understanding that knowledge management is

part of an overall management strategy • understanding that a plan requires iterative

refinement, as well as an understanding of the

program and service delivery part of the

organization, and • encouragement of data sharing within

the organization.

5. Policy development protocols, which: • describe standards and protocols that will connect

technology and management processes • clarify process for policy and program changes to

system, and • outline system training for users.

6. Continuous Quality Improvement (CQI) processes, which: • establish benchmarks to measure progress towards

goals, and • describe CQI and change management plan.

7. Monitoring and evaluation plan, which: • defines strategy for ongoing assessment of

information capacity and a strategy to address

gaps, and • creates target goals, describes routine monitoring

of data collected, and plans for the assessment of

staff data collection.

Prerequisite 3: Organization has a data-informed, knowledge-driven organizational culture Organizational culture is inevitable. It is strong. It enables or disables data sharing (Siakas et al., 2010). Without an organizational culture that strongly values knowledge management and data-driven decision-making, future data-sharing across organizations within the non-profit sector will be categorically unsuccessful (Hurley et al., 2005). While changing non-profit culture is anything but simple, fundamental changes to organizational beliefs and work styles are possible (Winship, 2012).

2. Organizational commitment to knowledge management practice. The commitment is documented and resource allocation emphasizes leadership’s commitment.

Culture begins with senior leadership who:

3. Purpose and objectives of data, information and knowledge management. These align with the organization’s strategic plan.

•

4. An environmental scan, which: • identifies both internal and external data,

information and knowledge needs, and • identifies programs that are working and not

working (goals being reached and not reached

•

• • • • •

Cultivate role models within and outside the organization to champion data and knowledge management.

Reinforce behaviours by recognizing and rewarding based on data-driven decision-making.

Start with quick wins to demonstrate the value of data.

Plan long-term but start short-term with baby-steps.

Communicate successes with staff (Hurley et al., 2005; Lindberg, 2012a; Lindberg, 2012b).

Are knowledge or data-driven (Eglene & Dawes, 1998; Renshaw & Krishnaswamy, 2009).

Demonstrate the five practices of exemplary leadership, which include: modeling the way, inspiring a shared vision, challenging the process, enabling others to act, and encouraging the heart (Kouzes & Posner, n.d.).

feasibility Q: two

13

Understanding Organizational Culture Organizational culture forms when employees share stories and experiences. Individual members are ‘socialized’ to believe certain ideas about partners, systems, and even the organization itself. This creates shared assumptions and practices that become ingrained into the organization and operate as if they are ‘fact’ (Sandfort, 1999). Any future event is then interpreted through this organizational culture lens, which justifies actions and protects the status quo. “[P]eople exist within social contexts – societies, communities, organizations – that do not possess inherent structure. [Therefore] the structures that guide people’s actions are from a group’s daily experience within a particular context, as they collectively try to make sense of the circumstances in which they find themselves” (Sandfort, 1999, p. 332). One cannot discount the collective experiences and informal channels of staff communication.

Changing culture requires leaders to use systems thinking and find natural points of intervention (Winship, 2012).

With regards to organizational culture and data, information, and knowledge management, organizations can fall anywhere along the continuum of disbelieving entirely in the merit of investing resources in their management, to organizations who invest too many resources into the production of meaningless data.

According to Beth’s Blog: How Networked Non-profits Leverage Networks and Data for Social Change, every non-profit can adopt a data-informed culture, but it takes time and leadership (Kanter, 2011). The author suggests beginning with an assessment of where the organization is currently positioned, and then adapting one’s change strategies accordingly. See Text Box for more detail on Kanter’s (2011) stages of adopting a data-driven culture.

As previously discussed, individual attitudes toward data and technology, and perceptions of lack of relevant content (Siakas et al., 2010) results in resistance to data and knowledge management, and the belief that data has no meaning or significance to practice (Carrilio, 2008).

Reframing to alter personal and organizational opinions is needed.

4 Stages to Adopting a DataDriven Organizational Culture (Kanter, 2011) The first is the Dormant stage where no systems are in place and the organization has no simple data collection methodologies or technologies. This does not necessarily mean an organization is not collecting data and information but it is not applied as part of making meaningful decisions. This would be case for a non-profit that may have a data management system but it is exclusively used for reporting and maintaining its complex accountability requirements to funders, not for informing decision-making (Schwartz, 2009). At this stage no consideration is given to interpreting the data or using it to improve programs and services.

Stage two is Testing and Coordination. Here an organization collects data to make decisions but there is no system linking program data to other organizational data, or to the larger organization mission. Data is disconnected. Data is used narrowly for specific programs and services.

In the Scaling and Institutionalizing stage there is a system in place for collecting measurement data across the organization. This data management system can be used by all organization staff, from frontline to senior management, to measure impact.

Organization employees are trained to use the system and to utilize the data in program improvement and in enhancing organizational capacity.

Last, in the Empowering stage the organization is completely data-driven, knowledge management is embedded into organizational practices. It devotes resources to employing a staff person to be the steward of data and information. It is the responsibility of this role to perform frequent assessments and audits to safeguard the quality of data, and to ensure the organization is using its data and knowledge to make decisions. This protects against complacency or backsliding, and helps guarantee the continued capacity of the organization to make data-driven decisions.

feasibility Q: two

14

As identified in the community dialogue: There remains a considerable deficit in the capacity of many local community organizations to collect, store, manage and utilize data, information and knowledge.

Large-scale capacity building efforts are required to develop the requisite knowledge and competence in data, information and knowledge management within the sector, especially for smaller agencies. True data-driven decision making is employed by a very small minority of agencies and organizations. Therefore, a need exists to foster a culture of quality data, information and knowledge management within the human service sector, and to foster the understanding of the utility and importance of data to the provision of effective services and programming.

However, there does appear to be an energy and interest from agencies across the sector in understanding how to work better with data, and to share data for collective impact.

The community dialogue suggests that even those with established data management practices remain unsure of how to collectively gather, house, and work with shared data in the most efficient and effective manner. The interest in learning about data management and data sharing potentially suggests a readiness for change.

“Shared values and interests provide a strong foundation on which to build productive partnerships� – (Wolf, 2012, p. 106).

feasibility Q: two

15

Collaborating for Data Sharing As identified in the literature:

As identified in the community dialogue:

Data sharing is not a simple endeavor but approaching the development of a data system or initiative collectively and systematically helps build its success. While much has been written regarding the importance of collaboration, partnership and the coordination of services in non-profit sector, research on the actual process of collaboration is largely absent (Sandfort, 1999). Fortunately, the processes used and the lessons learned by innovative non-profits working collaboratively have been documented in grey literature (Kramer et al., 2009; Lindberg, 2012b; Purcal et al., 2011; Tapia et al., 2012; U.S. Department of Health and Human Services., 2013).

The current enthusiasm in Calgary for the Collective Impact framework has ignited an interest in data sharing to generate bigger and better community and system impact.

An articulated understanding that any one organization alone cannot elicit the change necessary to alter the social issues around which their programming and services seek to address, has fostered a curiosity for exploring data sharing collaborations. While mixed opinions are expressed, the community dialogue overall indicated a probable presence of the requisite commitment and leadership necessary to build the repository. Overall, the community believed that at a minimum there are pockets of leadership within the sector who would be well positioned to champion a data sharing initiative.

Key findings from literature on collaborating in data sharing initiatives: 1. Initiatives involving collaboration are challenging to develop, implement and sustain (Carlson et al., 2011).

2. Integration of data systems requires patience, tolerance for complexity, and a methodical plan to build trust (Carlson et al., 2011).

3. The nature of the inter-organizational network is key to the success or failure of an initiative (Carlson et al., 2011).

4. Collective beliefs (culture) influence and define the boundaries of the collaboration, serving to mobilize or immobilize action – mandatory collaboration creates resistance and resentment (Carlson et al., 2011).

5. Ultimately the ability of human service organizations to collaborate hinges on frontline staff, who recall past experiences and who create a collective assessment about partners and the collaboration itself (Carlson et al., 2011).

6. Strong collaborations have a shared vision and common norms of cooperation and trust (Berdou, 2011; Kumar, n.d.).

7. Ideological resistance to data collection will influence collaborative success.

8. There needs to be incentives for organizations to collaborate and participate in data sharing (Eglene & Dawes, 1998).

There are mixed opinions on whether the timing is right to establish a data sharing initiative of this nature. One perspective is that indeed it is politically relevant and timely given the provincial government’s identified focus on early childhood education and resource development.

Therefore, these community members consider it an ideal time to champion the initiative.

However, there is also a segment of the community who are experiencing substantial cuts to their funding and insecurity about future funding. Therefore, these community members suggest that it might be difficult to gain support of agencies and organizations within the sector at this time.

Several community members suggested that DMP begin small and grow incrementally as success and capacity is built. They also encouraged quality to be prioritized over quantity. They cautioned that participation in the Calgary data repository not result in punitive action, or be used to mediate competition between organizations or services. Thus, community-wide indicators should be used only for collaboration, as opposed to competition.

feasibility Q: two

16

Advisory Committee Findings & Recommendations Findings: •

There is a core group of motivated agencies willing to champion a Calgary Child and Family Community Data Repository. Partners to date include: Homeless Foundation, Food Bank, City of Calgary, Family and Community Support Services, SouthWest Community Resource Centres, an independent researcher representing the Domestic Violence Sector, Elizabeth Fry, Closer to Home, Calgary Reads/First 2000 Days Network, Families Matter Society, Rocky View Schools, UpStart (United Way of Calgary and Area), ECMap, independent consultant with expertise in collective Impact, MRU Institute for Non-Profit Studies, MRU Centre for Child Well-Being, and MRU Integrative Health Institute.

•

The leadership capacity and commitment is present to create a Calgary Child and Family Community Data Repository.

•

It is clearly understood that not all agencies in Calgary currently have the capacity to collect and use data. Agencies need support to learn how to collect quality data, what data to collect, and how to analyze and use collected data. However, comprehensively addressing this need is beyond the scope and capacity of the data repository project.

•

Agencies with capacity to collect and use data are already often overburdened. Agencies often have to report to multiple funding organizations using multiple report formats and data.

Recommendations: •

The data repository begins with a small number of committed Calgary and area partners who share a focused agenda and purpose. When success is built, then leadership may explore data repository expansion.

•

Ensure cross-sectoral participation in the data repository.

•

Resources are needed to support and ensure participation in the repository, available to any interested organization. A specific capacity building component should be included in the repository project. However, this does not include the provision of evaluation consultants in a traditional form (e.g., how the Canadian Outcomes Research Institute (CORI/HOMES) database operated).

feasibility Q: three

17

feasibility question three 3

What resources – financial, capital, and human – need to be available to ensure success of a Calgary Child and Family Community Data Repository?

Data Sharing Initiatives are Resource Intensive As identified in the literature: • • •

Measurement and data sharing requires strong financial and time commitment from participants (Kanter, 2011; Kramer et al., 2009; Kumar, n.d.).

Data management must be incorporated into job descriptions and specific organizational roles, otherwise it will not be prioritized (Kanter, 2011).

Organizations need a data management team or data steward that is NOT the organization’s IT staff person (American Public Human Services Association., 2010; Eckerson, 2013).

feasibility Q: three

18

As identified in community dialogue:

As identified in interviews with data sharing initiatives:

•

•

• •

• •

•

•

Community members identified an organizational need to have sufficient capital infrastructure and technology to participate.

There was an overwhelming request that sufficient supports be in place to assist the participating organizations in using and working with the repository.

There was a sense that striking the right balance of people on the project will be challenging and require specialized skill sets and dedicated staff outside of the participating agencies. This would include someone to liaise with organizations on regular basis, to answer questions about the operation of the technology, using the data for analysis and so forth.

Participating organizations will need to assign someone to collect and provide data as part of his or her job description.

There also was a repeated need to have someone formally in charge of actually using the data submitted to the repository to move forward policy and systemic change objectives. Some community members suggested having a specific agency spearhead the project.

Community members wanted the money to be secured ahead of starting the project in order to ensure that it is built properly. Furthermore: • there was cautionary advice about the difficulty

of obtaining funds and the multiple years required

to acquire sufficient funding for a project of this

size and complexity • one funder alone will rarely supply the necessary

funds needed, which can result in compromising

the integrity of the original idea in the pursuit of

funding. One agency suggested that many funders

currently do not have an appetite for supporting

these types of projects, and • finding funding to support the ongoing

operation of the initiative after the pilot

or launch was also a worry.

Concern was identified for the extra administrative load and increased administrative resources required to share data, as well as the cost of keeping up with software customizations.

• •

•

•

•

•

Multiple sources of funding are generally required; none of the initiatives are exclusively resourced by one funding body. Of interest is that the local United Way organizations are either major funding agents or partners in the development of three of the initiatives interviewed: Peg, Connecticut Nonprofit Strategy Platform, and Wellbeing Toronto. Other large foundations contribute to Policy and Analysis Center and Greater New Orleans Community Data Center, while government funding supports CommunityView Collaboration, Wellbeing Toronto, Connecticut Nonprofit Strategy Platform, Community Indicators Victoria, and Newfoundland Community Accounts.

The cost associated with the development ranged from several hundred thousand to over a million dollars.

The time between the inception of a data sharing initiative and implementation for the majority of projects was many years (anywhere from three to eleven). However, once key stakeholders were committed to the project’s success, it generally took only two to three years to launch. The speed of completion is also dependent on existing infrastructure. For example, Wellbeing Toronto leveraged existing City of Toronto technological infrastructure, which reduced overall cost and expedited the process.

Designated staff is required. All of the data sharing initiatives had staff resources devoted to development and implementation of the initiative, often with fulltime staff, and at times even multiple staff, averaging approximately two staff with additional support around web development. Human resources were often donated in-kind by local municipalities, health regions, and non-profit organizations. Now in operation, initiatives still have designated staff, but fewer and often at a 0.6 to 1.0 FTE. Initiatives must have the appropriate infrastructure.

Web-based platforms require good geospatial infrastructure, which includes web servers that can handle the expected load (CVC, WT).

The web-based platform must be designed so that the application is aesthetically pleasing. To be used, it cannot simply be a series of JAVA drop down menus (CVC, GNOCDC, PEG, WT).

It is also essential that the development team includes knowledge and expertise in data collection, GIS, web programming, statistics, tiled web-services, and so on (CVC, WT).

feasibility Q: three

19

Advisory Committee Findings & Recommendations Findings: •

Within Calgary, there are agency specific assets and expertise that could benefit the data repository (e.g., GIS mapping, social economics, experience with large databases, and so on).

Recommendations: •

Leverage committed partners’ existing social capital and technology assets in the form of human resource talents and technological infrastructure.

•

Specific staff roles required to support the repository project: 1.

Data Connectors or Masters – resource persons who provide general support to agencies, lead capacity building efforts, ensure project does not duplicate others’ efforts, and possibly provide advocacy on behalf of the cross-sector partnership

2.

Data Scientists or Data Wizards – answer specific questions using the data, determine quality of data, provide data analysis, generate reports and narratives from repository data, map the data, and more generally turn the data into a more usable form of information and knowledge and

3. Creative Communicator – presents the analysis in appealing and creative

ways that can be readily understood by less experienced employees,

funders, and the public (e.g. narratives, video presentations etc.)

feasibility Q: four

20

feasibility question four 4

What governance structures need to be developed within agencies to ensure success of a Calgary Child and Family Community Data Repository?

There are a number of issues and concerns as discussed in the literature, by community members and by operating data sharing initiatives that should be considered in the governance of data, information, and knowledge sharing. These include:

Data Differences among Organizations As identified in the literature: • • • • •

Agencies define clients, services and programs differently, and collect different information (Kumar, n.d.).

Organizations may have different policies on data ownership, maintenance and liability (Eglene & Dawes, 1998; Kumar, n.d.).

Quality of data and records vary (Boruch, n.d.; Kumar, n.d.; U.S Department of Justice, 2006).

Systems are often not designed for usability or analysis but for simple frequency tabulations.

Organization systems are heterogeneous which means systems may present data differently and exist in non-matching data formats (Tapia et al., 2012).

As identified in community dialogue: • •

Current organizational data systems and processes are primarily for the purposes of reporting to funders only.

Organizations collect the same data in different ways and use multiple outcomes, indicators, and other variables.

As identified by interviews with existing data sharing initiatives: •

All data sharing initiatives have data eligibility standards. As an example, for data to be included in Wellbeing Toronto, it must be accessible (i.e., obtained at zero or minimal cost), comparable and consistent across the city, statistically valid and not just proxies for other indicators, credible according to domain experts, and relevant to measuring progress towards their goal.

feasibility Q: four

21

Complexity of Data Sharing As identified in the literature: •

•

• • • •

Actually integrating data requires mathematical models and statistical theories; it is a complicated process to identify match thresholds, probabilistic weight assignments, link cascades, and decide on a series of other analytic choices and calculations (Boruch, n.d.) that are well beyond the scope of this document.

Legislation and regulations around privacy and confidentiality of information (i.e., FOIP, PIPA) can be challenging especially given their unclear and ambiguous nature (Eglene & Dawes, 1998; U.S Department of Justice, 2006).

Clear policies around data access and dissemination are required (Eglene & Dawes, 1998).

Bureaucracy and resistance can interfere (Eckerson, 2013)(M. Krepicz, personal communication, February 11, 2013).

Reaching consensus in data sharing initiatives is difficult (U.S Department of Justice, 2006).

Relatively few models and best practices exist to guide development (Gil-Garcia et al., 2007).

•

As identified in interviews with data sharing initiatives: •

•

As identified in community dialogue: • • •

•

Identify processes for protecting client data privacy.

FOIP may interfere with service delivery and data sharing.

FOIP may restrict information that could be used to provide more efficient and effective services (i.e., difficulty in sharing data and information about children and youth).

The initiative having a relationship with an academic institution was both a suggestion and a concern articulated by community participants. It was suggested that housing the operation out of an academic institution would offer critical credibility and also provide an invaluable link to research and expertise. For example, data could be validated or connected to other research being conducted.

However, one participant who was both a sessional instructor and has an adjunct position at a local university expressed concern for the complexity of overlapping issues when an initiative is associated with academia, especially for someone working in both academia and at an agency. The participant suggested that there would need to be a clarification

around ethical approval policies and procedures. For example, when a research grant is administered through a university, ethical approval must be gained before any work can begin. Would this also apply to agencies and organizations with data housed at the academic institution? This answers would need to be made clear to participants of the repository.

Ownership of data needs to be defined with agreements and explicit understanding.

• • •

•

Start small and safeguard quality - it is not an IT project and it is enormously complicated. Get small data sets with usable fields and then grow technology as the demand and funding continues to expand (PAC, PEG).

Do a demonstration project with high visibility in the community to demonstrate the value to community and stakeholders (PAC).

The key to success is providing quality and being credible (GNOCDC).

Have patience. It is difficult at times, and time consuming (PEG).

Advice from Wellbeing Toronto was to avoid formal data sharing agreements as in their experience “lawyers get in the way” (M. Krepicz, personal communication, February 11, 2013).

Wait for the right time, when the right players are engaged and ready to commit (CVC, PEG, WT).

feasibility Q: four

22

Information Overload for Organizations Involved in Data Sharing As identified in the literature:

As identified in community dialogue:

•

•

• • •

• •

The ability to efficiently and effectively process and utilize relevant information is obstructed by the sheer quantity of information being presented or ‘pushed’.

Information overload may be the consequence of information and data being perceived as a hindrance rather that a support tool.

Information will be ignored if it is determined to be valueless or inaccessible.

While technology has been beneficial and important, it has been largely responsible for overload. It can lead to what those in information management refer to as ‘satisficing’, identifying just enough information to meet the need, as in ‘just good enough’.

The problem, however, may not be information overload per se but instead may be a symptom of work overload.

When contemplating data sharing, specialized training and education, workload management, and pragmatic solutions are required to navigate information overload (Bawden & Robinson, 2008).

•

There is a desire to track impact, but a lack of expertise and knowledge on how to do so. This may lead to staff burden and burn out.

Many agencies and organizations are operating with fewer resources, an unsure funding future. If agencies are to join, their participation cannot add significantly to staff workloads, otherwise agency participation would be unfeasible.

As identified in interviews with data sharing initiatives: • • •

To ensure the greatest usability and prevent information overload, systems are tailored to the understanding of the community (PEG).

Community engagement is employed across every piece of system development (marketing, communication, framework development (PEG).

Wellbeing Toronto and the Peg understood that one size does not fit all; the tool needs to be flexible to suit needs of users.

feasibility Q: four

23

Technological Impediments to Organizational Impediments to Data Sharing Data Sharing As identified in the literature:

As identified in the literature:

•

• • • • • • •

•

•

•

Researchers have found a positive correlation between the size of an organization and their use of innovation.

Having the technical expertise is a primary barrier to incorporation of technology, especially for smaller agencies. This is often also associated with financial barriers. Given the choice to serve more clients in the short term versus implementing a new technology to help clients in the future, small nonprofits often choose the former (Lee & Bhattacherjee, 2011).

An agency needs organizational technological infrastructure for data and knowledge management before it can share data (Shank, 2009).

Complexity of hardware and software, incompatibility of system components, conflicting database designs, and mismatched data structures may be challenging (Gil-Garcia et al., 2007).

Systems that are not user friendly will not be meaningful (Carrilio, 2008).

As identified in community dialogue: •

•

Organizations currently use multiple data systems to fulfill funder reporting requirements. The addition of ‘yet’ another database is a concern.

The need to having sufficient resources to maintain technology at the organizational level made some community members hesitant about being involved in a community data sharing project.

As identified in interviews with data sharing initiatives: • •

The technology required to share data can be complex and difficult to manage.

Building tools and websites piece by piece takes time and tremendous patience (PEG, WT).

Resistance to change.

Differing individual agendas and goals.

Misinterpretation of shared information.

Competing organizational priorities.

Overly ambitious goals.

Lack of respect and understanding between organizations.

Diversity of cultures and structures (Carrilio, 2008; Pleace, 2007).

As identified in the community dialogue: • • • • • • • •

Resistance of staff to organizational or operational change.

Resistance of staff to the addition of more ‘data requirements.’

Sensitivity of organizational data that could be misconstrued or used inappropriately.

Vision and goals of the initiative not reflecting organizational strategic priority areas.

Privacy and confidentiality not being sufficiently addressed.

Lack of clear benefit to the individual organization.

Vague or uncertain purpose of the data sharing initiative.

Lack of resources needed to participate or maintain participation.

As identified in interviews with data sharing initiatives: •

Organizations may need to be convinced that no raw data will be available, data will be processed to an aggregate or neighborhood level, intensive security protocols are in place, data will be read only so that one cannot write-over or alter any data, and that strict protection of privacy procedures are established and overseen before an organization can get their own leadership or board of directors to agree to participation (WT).

feasibility Q: four

24

Governance in Data Sharing As identified in the literature: •

• •

Formal governance structure(s) define roles and responsibilities; institutionalize commitment; establish ownership for data; and processes for collecting, reporting, and using data; as well as accountability protocols to safeguard data quality (Carson et al., 2010).

Steering or advisory committees provide guidance and working group(s) accomplish tasks (Drezelo & Lepore, 2008; Eglene & Dawes, 1998).

Data sharing agreements are frequently used to determine data custodians and accountabilities (Eglene & Dawes, 1998).

As identified in community dialogue: •

• •

•

Proposed governance frameworks or models to ensure the success of such an initiative was by far the most difficult question for community stakeholders to answer.

It was unanimously suggested governance will be one, if not the, most important structure for any data sharing initiative.

Many community stakeholders recognized the often problematic nature of governance in communitybased projects: waning commitment, change or dilution of intent over time, funder interference, lack of community relevance, impractical or unreasonable expectations of agency’s involvement, and unsustainable funding model.

Governance frameworks suggested were: a. a constellation governance structure where fiduciary responsibility is separate from constellations of other community stakeholder groups who direct the operation of the project(e.g.

The Genesis Project in Calgary) b. a multi-layered governance structure where change makers at the top are educated in the area but are far enough removed from the agencies involved and actually doing the work, to objectively direct project (separation between the ‘governors’ or ‘guiding force’ and the ‘implementers’)

c. a steering committee with membership of key

stakeholders from each organization (top level

decision makers), with or without operational/

working groups

i. a steering committee that is not

too big nor too small

ii. a steering committee with a mixture of

public and private sector members can be

helpful, but comes with inherent

challenges. Sectors have different priority

issues and mandates, and different ways

of tracking outcomes; statistics and

numbers carry different meaning in terms

of funding and opportunity and how that

information can be interpreted, and d. an overarching governance body that directs the

development, above the advisory committee,

ending up with some levels of bureaucracy

representative of the Calgary community.

• A reoccurring theme was the need for a mechanism within the governance structure to represent the community/agency interests and ensure the project does not alter under pressure from funders.

• To prevent overlap with other initiatives doing similar work, knowledgeable initiative representatives should be involved somewhere in the governance structure (e.g., City of Calgary, other community initiatives like ECMAP, UpStart).

As identified in interviews with data sharing initiatives: •

•

•

The primary governance models are steering committees with or without additional sub-groups, and the traditional non-profit model of executive oversight with accountability to board of directors.

Sub-groups commonly used were research advisory boards, project advisory teams, academic expert panel groups in addition to working groups, and engagement groups.

Stakeholders involved in governance included representatives from municipal government, provincial government, local chamber of commerce, and the non-profit sector.

feasibility Q: four

25

Privacy and Security of Data As identified in the literature: While information and communication technology (ICT) has provided unparalleled opportunities for the sharing of data and the advanced ability for interorganizational collaboration, it also gives rise to critical privacy, security and confidentiality concerns (Petrila, n.d.; Tapia et al., 2012).

Data sharing initiatives must systematically determine how best to protect the privacy and confidentiality of clients, while not negating the benefits and purpose of sharing and still adhering to the applicable laws and regulations (Carson et al., 2010; Petrila, n.d.). The salience of this issue is magnified when sharing individually identifiable information, or when the linking of discrete datasets could disclose an identifiable amount of detail for a single individual (Petrila, n.d.). Ethically protecting privacy and confidentially of client information is, therefore, of paramount concern (Shank, 2009).

Stiles and Boothroyd (n.d.) describe four important principles for consideration by any data owners regardless of whether they intend to share data with other organizations or not. The first is data security, which is the protection of data from unacceptable disclosure in accordance with laws of the jurisdiction in which the data is collected, stored and shared. Appropriate training, organizational policies and technology tools should be employed to ensure data security. The second principle is the confidentiality of information. Confidentiality is central to the maintenance of trust between client and human service providers and must be safeguarded.

Third is disclosure permission, at both the individual and organization level. The appropriate use of data requires processes for acquiring permissions. And the last ethical principle for consideration by data custodians is the principle of appropriate use. This necessitates data users and custodians be skilled in the management and application of data (organizing, manipulating and analyzing), and educated to do this ethically.

Data custodians and users must also consider administrative, physical and technical safeguards for the mitigation of risk (Petrila, n.d.). Administrative safeguards refer to a number of standards, most important being standards around conducting risk analysis of security issues, the delegation and designation of person(s) to complete these risk analyses, and the creation of a

management plan in the case of an unforeseen breach of confidentiality or privacy. An outside third party hired to complete periodic risk analyses is frequently recommended.

Physical safeguards are the physical precautions taken by organizations in the protection of privacy of data stored electronically (Petrila, n.d.). These largely encompass building and equipment rules to prevent unauthorized access. These safeguards include protections related to workstation use, building access, workstation security, and the regulated ability to access data (US Department of Health and Human Services). Technical safeguards relate directly to the technological measures that can

feasibility Q: four

26

be employed to defend privacy and security of client information (Petrila, n.d.). The Treasury Board of Canada Secretariat outlines these safeguards as including: the identification and authentication of authorized users; access controls to limit access to authorized users, including the types of functions that authorized users are permitted to exercise; audit logs and records created, protected and retained to verify that all access is authorized; segregation of sensitive personal information through logical or physical data separation; inspections conducted to provide confidence that appropriate controls are in place and are properly implemented and operate as intended; and privacy breach detection, response and recovery practices (Government of Canada, 2013). For example, not all data users require the same type, level or detail of data. One application of ‘authorized users’ includes not allowing universal access to all data, by all users (Carson et al., 2010). The encryption of data with a decryption key provided only to authorized users is another example of a technical safeguard (Eckerson, 2013; Pervez, Khattak, Lee, & Lee, 2011).

Although directed specifically at researchers, the authors of the study Accessing Health Care Utilization Databases for Health Research interviewed data stewards and privacy commissioners across Canada (Raina et al., 2009). These privacy specialists underscored the need for studied consideration of privacy when sharing data. Privacy Impact Assessments (PIA) were recommended to identify potential privacy concerns, to help develop an informed consent process, and as an expression of researchers’ ‘commitment to transparency and accountability’. Consideration should also be given to data validity, data security, and data retention.

There was a sense from the reviewed literature that, overall, organizations have a very poor understanding of what can and cannot be shared (Pleace, 2007; Quigg et al., 2012). This is not surprising given the confusion even within regulatory bodies about privacy and confidentiality related to human and social services sharing data (Raina et al., 2009). Recommendations from the literature were to: • • •

Gain clarification from government bodies about data privacy regulations.

Strike inter- and intra-agency coordinating bodies to help determine privacy, confidentiality, ethical standards and data access.

Replicate other sharing initiatives who largely avoided privacy and confidentiality issues by starting with aggregated data (Carson et al., 2010; DeSantis & Hiatt, 2012). As previously mentioned, with one exception, data sharing initiatives that were reviewed or interviewed included only aggregated data, with no ability to identify unique individuals.

This is an incredibly complicated and complex issue that will require careful follow-up if a data sharing initiative is deemed feasible3.

As identified in the community dialogue:

“Dictatorship is cheap and consensus is expensive”

• • • •

Participants suggested it would be beneficial to have an articulated strategy to address FOIP privacy issues to gather stakeholder support for a data sharing initiative in Calgary.

There remains great concern and misunderstanding about FIOP and PIPA and its impact on an agency’s ability to share data.

Capacity and knowledge around the legislative acts are required.

Concern was expressed by a few about adhering to funder agreements around data sharing.

Sharing aggregate level data only may alleviate concerns about FOIP and PIPA.

– (Carlson et al., 2011, p. 22)

3

The Treasury Board of Canada Secretariat’s websites offers detailed information and resources on

national information and privacy policy http://www.tbs-sct.gc.ca/ip-pi/index-eng.asp

feasibility Q: four

27

As identified in interviews with data sharing initiatives: •

All data sharing models (with one exception, PAC), exclusively use aggregate data at the level of community or neighborhood. As would be expected with a system tracking unique and identifiable individual data, the PAC database is the only model not available online. The remaining models employed aggregate data that can be narrowed to the level of the community or neighborhood.

Given that no personally identifiable data is used, privacy and confidentiality were not large obstacles to any of the community based initiatives. “Essentially [we] are sharing aggregate data… [it’s] rolled-up to a level where there is never a privacy and security issue” (B. Holden, personal communication, February 4, 2013). Some systems such as CommunityView Collaboration have the ability to combine different layers and levels of data and can be configured to allow sharing more sensitive and personally identifying information by restricting access to that section of the website.

Funder Considerations As identified in the literature: A sizable number of sources referred to a collective need for change in funder perspectives and practices with regards to how the non-profit sector operates, which then influences data, information, knowledge and ultimately social impact. The two major themes that emerged from the literature are highlighted below.

One major theme was shifting focus from funding single organizations, to strengthening the collective response to a single issue. The current focus of funding bodies on single organizations is believed to weaken the sector’s ability to meet complex social challenges (Kramer et al., 2009). As complex problems, these challenges require complex solutions.

Complex solutions require the integration of interdependent systems across all three sectors (for profit, non-profit and government) (Kramer et al., 2009). Therefore, funders should concentrate on strengthening the sector and cross-sectoral collaborations. Funders are uniquely positioned to act as bridge-builders championing the creation of networks, coalitions, partnerships and collaborations across the non-profit sector and between the non-profit sectors and the other sectors (Carson et al., 2010; Hurley et al., 2005). This requires moving from an isolationist perspective that views individual organizations as central, to the adoption of a collectivist perspective that identifies coalitions, cross-sectoral participation, and inter-agency collaborations as fundamental to the whole sector’s success in solving complex social challenges. While many funders now mandate program partnerships and interagency collaborations, true change will require revamping the current competitive and limiting tendering system (Renshaw & Krishnaswamy, 2009).

The other theme prioritized building the capacity of the sector with better measurement infrastructure. Why has the non-profit sector not more widely adopted for profit data, information, knowledge management strategies? A large determinant reported in the literature is the general lack of financial support from funders to develop required infrastructure (Menon, 2012). Interestingly, this is in spite of the challenges of duplication, fragmentation and the unsystematic organization of programs and services across agencies repeatedly being a source of concern of funders (Kramer et al., 2009). Yet their funding strategies fail to support the infrastructural change needed to address this sectorlevel challenge (Carlson et al., 2011). Change requires alignment of measurement infrastructure demand with funding strategies. Funders must be willing to financially support the creation of the infrastructure necessary for measurement and impact assessment (Schwartz & Austin, 2009; Shank, 2009). And funders must work to streamline measurement and reporting to avoid wasting limited organizational resources on reporting to multiple funders who require different measurement of outcomes, using different electronic reporting systems (Bernholz, Skloot, & Varela, 2010; Philanthropy News Digest., 2012).

feasibility Q: four

28

Advisory Committee Findings & Recommendations Findings: •

It is feasible to create a Calgary wide child and family community data repository.

•

The repository should not be linked to any single funding organization or source of support.

•

Commitment from senior leadership within an agency is required for successful participation in a data sharing initiative.

Requirements: 1. Leadership team: • requires partner representatives who can commit agency resources

and capacity • develops the focus questions for the data repository • identifies the indicator framework and organizing

structure of the data repository, and • engages community stakeholders and new partners.

2. Human Resources: • proposed positions (Data Connectors, Data Wizards, and Creative

Communicators) may currently exist and be available to be shared across

partner agencies.

3. I nfrastructure includes: • technological infrastructure: data repository and storage • website • data sharing agreements, and • consultative staff.

Recommendations for this section have been included in the final Advisory Committee Action Plan for Phase Two (see page 30).

conclusion

29

conclusion Whether explicit or implicit, the ideas of transparency, collaboration, and open exchange of data are the foundation for all of the data sharing initiatives.

While opportunities to source knowledge via partnerships and collaborations help reduce redundancy and facilitate efficiency, these require the agencies’ ability to develop, evaluate, document and share successes. To convert data into knowledge requires management, technology, and social infrastructure (Bolisani & Damiani, 2010; DeSantis & Hiatt, 2012; Kanter, 2012; Renshaw & Krishnaswamy, 2009; Shank, 2009). Few data management systems currently capture or disseminate knowledge. Few data repositories of expert knowledge exist both within and outside organizations. Harnessing the power of data, information and knowledge for the promotion of public good will help build the capacity to meet the urgent health and human service needs. One agency alone cannot address the complex web of interwoven determinants of social well-being; coalitions of many agencies in collaboration, empowered by shared knowledge to generate sustained impact, will be required (Walker et al., 2012).

This feasibility study identified that there is a readiness in Calgary human service sector to begin embracing data for social change.

action plan phase two

30

advisory committee action plan – phase two 1

Create a five-year multi-stage plan for the development of the data repository, which includes a distinctly defined ‘do-able’ project with a small number of agencies in Year One.

2

Submit a grant application to The Calgary Foundation for $75,000-$100,000 for Phase Two of the DMP project.

3

Explore funding opportunities beyond The Calgary Foundation, including the possibility of corporate support or sponsorship.

Also consider in-kind donation of expertise.

4

Explore partnerships with other organizations such as the Alberta Centre for Child Family and Community Research (ACCFCR) and Rocky View Schools which have expressed interest and have existing data storage capacity, data analysis skills, and data sharing agreements.

5

Identify opportunities for integration with existing initiatives.

Approach UpStart (United Way of Calgary and Area), Calgary Homeless Foundation, Calgary Community Data Consortium (City of Calgary) to explore how initiatives can be integrated rather than replicated.

6

Collect aggregate data only to circumvent privacy and confidentiality issues.

7

Educate agencies on the Freedom of Information and Protection of Privacy Act of Alberta (FOIP) and the Personal Information Protection Act (PIPA), and clarify misperceptions and confusion around data sharing and compliance with these legislative acts.

8

Explore the use of the Collective Impact Framework for implementation of the data repository.

references

31

references About peg: Why does peg use stories? (2013). Retrieved from http://www.mypeg.ca/about American Public Human Services Association. (2010). Positioning public child welfare guidance: Strengthening families in the 21st century - information management guidance. Retrieved from http://www.ppcwg.org/information-management-overview.html Bawden, D., & Robinson, L. (2008). The dark side of information: Overload, anxiety and other paradoxes and pathologies. Journal of Information Science, 35(2), p.180-191. doi: 10.1177/0165551508095781 Berdou, E. (2011). Reimagining participation: Opportunities and challenges of the open source model of collaboration for development thinking and practice. IDS Bulletin, 42(5), 104-108.

Bernholz, L., Skloot, E., & Varela, b. (Eds.). (2010). Disrupting philanthropy: Technology and the future of the social sector. Duke University.

Bishop, S. W. (2009). Building programmatic capacity at the grassroots level: The reactions of local nonprofit organizations to public participation geographic information systems. Nonprofit and Voluntary Sector Quarterly 2010, 39(991), pp. 991-1013. doi: DOI: 10.1177/0899764009334984 Bissell, M., & Miller, J. (2007). Child welfare and technology. Retrieved from http://www.cwla.org/ voice/0707technology.htm#top Bolisani, E., & Damiani, F. (2010). Knowledge management in complex environments: The UN peacekeeping. Measuring Business Excellence, 14(4), pp.76 - 84. doi: 10.1108/13683041011093776 Boruch, R. (n.d.). Administrative record quality and integrated data systems. (). University of Pennsylvania: Center for Research in Education and Social Policy.

Calabrese, F. A., & Orlando, C. Y. (2006). Deriving a 12-step process to create and implement a comprehensive knowledge management system. VINE., 36(3), p. 238-254. doi: 10.1108/03055720610703533 Carlson, N. E., Hernรกndez, E., Edmond-Verley, C., Rotondaro, G., Feliz-Santana, E., & Heynig, S. (2011). Developing a master data sharing agreement: Seeking student-level evidence to support a collaborative community effort in education. The Foundation Review, 3(4), p. 14-33. doi: 10.4087/FOUNDATIONREVIEW-D-11-00024 Carrilio, T. E. (2008). Accountability, evidence, and the use of information systems in social service programs. Journal of Social Work, 8(2), p. 135 -148. doi: 10.1177/1468017307088495 Carson, R., Laird, E., Gaines, E. & Ferber, T. (2010). Linking data across agencies: States that are making it work. Retrieved from http://dataqualitycampaign.org/files/DQCbrief_Mar19_2_.pdf

references

32

Christensen, P. H. (2007). Knowledge sharing: Moving away from the obsession with best practices.11(1), p. 36-47. doi: 10.1108/13673270710728222 de Vasconcelos, J., Seixas, P., Lemos, P. & Kimble, C. (2005). Knowledge management in nongovernment organizations: A partnership for the future. Retrieved from http://papers.ssrn.com/ sol3/papers.cfm?abstract_id=734223 DeSantis, C., & Hiatt, S. F. (2012). Data sharing in public benefit programs: An action agenda for removing barriers. Retrieved from http://www.singlestopusa.org/wp-content/uploads/2012/11/ Data_Sharing_in_Public_Benefit_Programs_11_2_12.pdf Dormont, J. (2012). Business intelligence for non-profits. Retrieved from http://www.printfriendly. com/print?url=http%3A%2F%2Fcollaborationforgood.org%2F2012%2F01%2F11%2Fbusinessintelligence-for-non-profits%2F&partner=Shareaholic-Publishers Drezelo, P., & Lepore, A. (2008). Building bridges: The case for sharing data between the court and child welfare systems. Retrieved from http://www.courts.state.ny.us/ip/cwcip/Publications/ BuildingBridges-TheCaseForDataShare.pdf Dyer, C. (2009). Opposition MPs object to data sharing clause in coroners bill. BMJ, 338(2), p.412412. doi: 10.1136/bmj.b412 Eckerson, W. (2013). Creating an enterprise data strategy: Managing data as a corporate asset.

Retrieved from http://docs.media.bitpipe.com/io_10x/io_100166/item_417254/Creating%20 an%20Enterprise%20Data%20Strategy_final.pdf Eglene, O., & Dawes, S. (1998). New models of collaboration: GIS coordination in New York State. (CTG Evaluation Report No. CTG.GIS-06). Albany, New York: Center for Technology in Government.

Gil-Garcia, J. R., Chengalur-Smith, I., & Duchessi, P. (2007). Collaborative e-government: Impediments and benefits of information-sharing projects in the<br />public sector. European Journal of Information Systems, 16, p. 121-133.

Goedert, J. (2012). Obama, make federal data available, friendly and useful-and do it soon. Health Data Management, 20(7), p. 12.

Government of Canada. (2013). Information management - guidelines. Retrieved from http://www. tbs-sct.gc.ca/pol/doc-eng.aspx?section=text&id=13832 Gurstein, M. (2011). Open data: Empowering the empowered or effective data use for everyone?

First Monday, 16(2), p. 1-7.

Harrison, L. (2012). Using agency-wide dashboards for data monitoring and<br />Data mining: The solano county health and social services department. Evidence-Based Social Work, 9(1-2), p.

160-173. doi: 10.1080/15433714.2012.636322 Harrison, C. (2012). GlaxoSmithKline opens the door on clinical data sharing. Nature Reviews Drug Discovery, 11(12), p. 891- 892. doi: 10.1038/nrd3907 Howard, A. (2012). Data for the public good. (1st ed.). Sebastopol, CA.: Oâ&#x20AC;&#x2122;Reilly Media.

references

33

Hurley, T., Green, C., & Antonio, S. (2005). Knowledge management and the nonprofit industry: A within and between approaches. Journal of Knowledge Management Practice, January Jones, R. B., Reeves, D., & Martinex, C. S. (2012). Overview of electronic data sharing: Why, how and impact. Springer Science + Business Media, 14(6), p. 486-493. doi: doi: 10.1007/s11912-0120271-7.

Kanter, B. (2011, ). How can nonprofits switch to a data-informed culture? Message posted to http:// www.bethkanter.org/switch-data-driven/ Kanter, B. (2012). Doing the math ourselves. Retrieved from http://www.marketsforgood.org/doingthe-math-ourselves/ Kouzes, J., & Posner, B. (n.d.). The five practices of exemplary leadership® model. Retrieved from http://www.leadershipchallenge.com/About-section-Our-Approach.aspx Kramer, M., Parkhurst, M. & Vaidyanathan, L. (2009). Breakthroughs in shared measurements and social impact. Retrieved from http://www.fsg.org/portals/0/uploads/documents/pdf/ breakthroughs_in_shared_measurement_complete.pdf Kumar, P. (n.d.). An overview of architectures and techniques for integrated data systems (IDS) implementation. (Best Practice). Philadelphia: University of Pennsylvania.

Lecarpentier, D. (2012). Towards a collaborative data infrastructure for science. Retrieved from http:// www.isgtw.org/feature/towards-collaborative-data-infrastructure-science Lee, R., & Bhattacherjee, A. (2011). A theoretical framework for strategic use of the web among nonprofit organizations. Atlanta, GA,.

Lindberg, A. (2012a). The use of key indicators as a foundation for knowledge management: The experiences of Monterey County’s social and employment services department. Journal of Evidence-Based Social Work, 9, 149. doi: 10.1080/15433714.2012.636320 Lindberg, A. (2012b). Linking departmental priorities to knowledge management: The experiences of Santa Cruz County’s human services department. Journal of Evidence-Based Social Work, 9(1-2), 57-67. doi: 10.1080/15433714.2012.636311 Mandayam, G. (n.d.). Geographic information systems: A planning tool for human service agencies.

Retrieved from http://www.gisdevelopment.net/application/health/overview/healtho0007.htm Markets for Good. (n.d.). What is markets for good? Retrieved from http://www.marketsforgood.org/ learn/ Mccombs, J. S., Orr, N., Bodilly, S. J., Naftel, S., Constant, L., Scherer, E. & Gershwin, D. (2010).

Hours of opportunity: The power of data to improve after-school programs citywide. mccombs, Retrieved from http://www.rand.org/content/dam/rand/pubs/monographs/2010/RAND_ MG1037.1.sum.pdf Menon, S. (2012). From raw data to informed decisions: What we can learn from the financial sector.

Retrieved from http://www.marketsforgood.org/from-raw-data-to-informed-decisions-what-wecan-learn-from-the-financial-sector/

references

34

Motorola. (2009). White paper, the value of real-time data in human service delivery. Retrieved from http://www.motorola.com/web/Business/Solutions/Industry%20Solutions/Human%20Services/ Images/Static%20Files/Motorola_White_Paper_-_The_Value_of_Real-Time_Data_in_Human_ Service_Delivery.pdf National Geospatial Advisory Committee. (2011). National geospatial advisory committee interagency data sharing- A primer. Retrieved from http://www.fgdc.gov/ngac/ngacinteragency-data-sharing-primer-june-2011.pdf Oâ&#x20AC;&#x2122;Mahony, S. (2007). The governance of open source initiatives: What does it mean to be a community managed? Journal of Management & Governance, 11: 139-150 doi: 10.1007/ s10997-007-9024-7 Pervez, Z., Khattak, A. M., Lee, S., & Lee, Y. (2011). SAPDS: Self-healing attribute-based privacy aware data sharing in cloud. Springer Science and Business Media, 62, p. 431-460. doi: 10.1007/s11227-011-0727-9 Petrila, J. (n.d.). Legal issues in the use of electronic data systems for social science research. ().

University of South Florida: College of Behavioral and Community Sciences.

Philanthropy News Digest. (2012). Foundation center announces open grantmaking data initiative.

Retrieved from http://foundationcenter.org/pnd/news/story.jhtml?id=396000002 Pleace, N. (2007). Workless people and surveillant mashups: Social policy and data sharing in the UK. Information, Communication and Society, 10(6), p.943-960. doi: 10.1080/13691180701751171 Purcal, C., Muir, K., Patulny, R., Thompson, C., & Flaxman, S. (2011). Does partnership funding improve coordination and collaboration among early childhood services? â&#x20AC;&#x201C; experiences from the communities for children programme - purcal - 2011 - - wiley online library. Child & Family Social Work, 16(4), p. 474- 478. doi: 10.1111/j.1365-2206.2011.00766.x Purcal, C., Muir, K., Patulny, R., Thomson, C., & Flaxman, S. (2011). Does partnership funding improve coordination and collaboration among early childhood services? experiences from the communities for children program. Child and Family Social Work, 16(4), 474 - 484.

Quigg, Z., Hughes, K., & Bellis, M. A. (2012). Data sharing for prevention: A case study in the development of a comprehensive emergency department injury surveillance system and its use in preventing violence and alcohol-related harms. International Society for Child and Adolescent Injury Prevention, 18(5), p. 315- 320. doi: 10.1136/injuryprev-2011-040159 Raina, P. S., Kirkland, S. A., Wolfson, C., Szala-Meneok, K., Griffith, L. E., Keshavarz, H., . . .

Pelletier, A. (2009). Accessing health care utilization databases for health research: A Canadian longitudinal study on aging. Canadian Journal on Aging / La Revue Canadienne Du Vieillissement, 28(3:287) doi: 10.1017/S0714980809990079 Renshaw, S., & Krishnaswamy, G. (2009). Critiquing the knowledge management strategies of nonprofit organizations in Australia. World Academy of Science, Engineering and Technology, 49, p. 456-464. doi: 10.1.1.192.8007

references

35

Sandfort, J. (1999). The structural impediments to human service collaboration: Examining welfare reform at the front lines. Social Service Review, , p. 314-339.

Schwartz, S., & Austin, M. (2009). Implementing performance management systems in nonprofit human service organizations: An exploratory study. Retrieved from http://system6.net/AnExploratory-Study-download-w916.pdf Shank, N. (2009). Understanding human services utilization: Opportunities for data sharing between federally funded programs. Nancy Shank Publications.

Shank, N., Sokol, B., Hayes, M., & Vetrano, C. (2008). Human services data standards: Current progress and future vision in crisis response Nancy shank Publications.

Siakas, K. V., Georgiadou, E., & Balstrup, B. (2010). Cultural impacts on knowledge sharing: Empirical data from EU project collaboration. Vine., 40(3/4), p.376-389. doi: 10.1108/03055721011071476 Stiles, P., & Boothroyd, R. (n.d.). Ethical use of administrative data for research purposes. (). Louis de la Parte Florida Mental Health Institute College of Behavioral and Community Sciences University of South Florida: Department of Mental Health Law and Policy.

Tapia, A., Maldonado, E., Tchouakeu, L. N., & Maitland, C. F. (2012). Coordinating humanitarian information: The problem of organizational and technical trajectories. Information Technology and People, 25(3), p. 240 - 258.

Tarsiero, R. (n.d.). Knowledge management and organizational capacity: The case of communities of practice in volunteer management<br />. Retrieved from http://www.knowledgeboard.com/ download/3256/TarsieroARNOVA.pdf Tatham, P., & Spens, K. (2011). Towards a humanitarian logistics knowledge management system.

Disaster Prevention and Management, 20(1), p.6- 26. doi: 10.1108/09653561111111054 U.S Department of Justice. (2006). Guidelines for juvenile information sharing. Retrieved from https://www.ncjrs.gov/pdffiles1/ojjdp/215786.pdf U.S. Department of Health and Human Services. (2013). Community health data initiative. Retrieved from http://www.hhs.gov/open/plan/opengovernmentplan/initiatives/initiative.html UN Women. (2012). Reflecting on gender equality and human rights in evaluation. Retrieved from http://www.unwomen.org/wp-content/uploads/2011/06/Reflecting-on-Gender-Equality-andHuman-Rights-in-Evaluation.pdf Walker, K. E., Farley, C. & Polin, M. (2012). Using data in multi-agency collaborations: Guiding performance to ensure accountability and improved program. Retrieved from http://www. childtrends.org/Files/Child_Trends-2012_02_23_FR_UsingData.pdf Winship, K. (2012). Data mining in children and family services: The contra costa county experience.

Journal of Evidence-Based Social Work, 9(1-2), p. 19-26. doi: 10.1080/15433714.2012.636306.

Wolf, R. (2012). The overlap between public health and law enforcement: Sharing tools and data to foster healthier communities. International Review of Law, Computers & Technology, 26(1), p.

97 -107. doi: 10.1080/13600869.2012.646802

appendix A

36

appendix A Community Engagement Process Funding secured from the calgary foundation

Steering committee created Dawne Clark, Director Centre for Child Well-Being, Mount Royal University

two focus groups held November 21, 2012 November 23, 2012

Community consultation October 18, 2012

Elaine Danelesko, Director Integrative Health Institute, Mount Royal University Paula Woolley, CEO Families Matter Society Mike Bowerman Social Impact and Evaluation Consultant Launa Clark, Community Coordinator â&#x20AC;&#x201C; Calgary & Area Early Child Development Mapping Project (ECMap) Lawrence Gervais Gervais Consulting Peter Elson, Acting Director Institute for Non-Profit Studies, Mount Royal University Darlene Pevach, Undergraduate Research Assistant Mount Royal University Kerry Coupland, Contracted Research Support Mount Royal University

Active stakeholder engagement Bi-weekly communiques to 1,400 stakeholders

appendix A

37

recommendations for follow-up interviews

Advisory committee created

determination of feasibility

Chantal Hansen, Program Manager – HMIS Calgary Homeless Foundation Oksana Grynishak, Data Analyst – HMIS Calgary Homeless Foundation

final recommendations

Randy Thornhill, Manager of Quality Assurance and Information Systems Closer to Home Community Services Paula Salter, Social Planner Family and Community Social Services, City of Calgary Melody Wharton, Executive Director SouthWest Communities Resource Centre James McAra, CEO Calgary Food Bank

Invited guests Kiran Manhas, Post Doctorate Fellow Child Data Centre Denise Clovechok, Project Manager Rocky View School Division

Steacy Collyer, Executive Director Calgary Reads

John Burger, Director of Schools – Research and Data Analytics Rocky View School Divisions

Michelle Bickley, Evaluation Specialist Elizabeth Fry Society

Phil Carlton, Director UpStart at United Way of Calgary and Area

Irene Hoffart, Director Synergy Research Group

Pamella Simpson Senior Development Officer Mount Royal University Foundation Sherry Ferronato, Committee Volunteer The Calgary Foundation Carol Adair, Consultant Child Data Centre Development Project at ACCFCR

appendix B

38

DATA MANAGEMENT PARTNERSHIP:

DECEMBER 11, 2012

You can’t manage what you don’t measure

appendix B Bi-weekly communique example

PARTICIPATE IN OUR SURVEY [click link above] Did you know?

A Calgary city-wide data system would be a LEADING INNOVATION in Canada.

You have been chosen by our Data Management Steering Committee, and Inform Alberta, as a human service professional in Calgary and area who may be interested in exploring the feasibility of developing a shared data management system. We are reaching out to the Calgary community to gather input and feedback, and to hopefully spark community-wide interest in determining the feasibility of building a cohesive and useable data management system for human service agencies.

WHAT’S IN IT FOR YOU? Collect better data

Why a data management partnership? One word...

IMPACT!

A data management partnership would help partners to: Make a bigger difference with each dollar invested in the human service sector

Better analyze trends and changes, ultimately improve agencies and community programming, and position agencies to better meet needs of clients

Support strategic decision making

Share information and build common understanding of community needs

Foster social change and improve the lives of Calgarians

PRELIMINARY RESULTS FROM THE NOV. 19 SURVEY

88% 25%

of respondents require routine evaluation

Thank you to those who participated in the Nov. 19 survey. Your feedback is greatly appreciated. If you have not had a chance to participate, the survey is still open! Click on the fluid survey link above.

The survey will take only 3–5 minutes of your time.

of respondents currently

cannot demonstrate their IMPACT

Overall, you are interested in a data management partnership, but are concerned about: • aggregating data • building consistent outcomes • managing a data collection implementation process

Look for another email in two weeks time where we will summarize the results from our focus groups with stakeholders.

We thank you for your patience as we develop our engagement with our stakeholders. If you have comments or suggestions for the improvement of our emails, please drop us a line or give us a call at 403-440-6941. We want to be read and relevant!

DATA MANAGEMENT PARTNERSHIP Funded by The Calgary Foundation and directed by Mount Royal University

CONTACT Dawne Clark, PhD 403-440-6941 dmp@mtroyal.ca

appendix C

39

appendix C Summary of data sharing models 1.

CommunityView Collaboration (Interview â&#x20AC;&#x201C; Yes)

City of Saskatoon and Saskatoon Health Region Description of Initiative: The Community View Collaboration (CVC) is a web-based information system housing a variety of local data and document resources. It is also a partnership, or collaborative effort, between local human service agencies to develop common community reporting measures, and share data and knowledge with each other and the community. Vision is to bring together data, resources, projects, and research from human service and community-based organizations that are contributing to the well-being of Saskatoon.

Goal: Provide relevant, reliable, local information and evidence to inform planning, decision making and policy for Saskatoon and surrounding area.

Governance: Steering committee.

Target audience: All. Data type: Aggregate.

Community indicators: Yes. Fee for service/membership: Free.

URL: www.communityview.ca

2.

Community Indicators Victoria (Interview â&#x20AC;&#x201C; No) Community Indicators Victoria

Description of Initiative: Community Indicators Victoria aims to support the development and use of local community wellbeing indicators in Victoria, Australia, with the purpose of improving citizen engagement, community planning and policy making. Community Indicators Victoria presents data and reports on the wellbeing of Victorians using an integrated set of community wellbeing indicators. These indicators include a broad range of measures designed to identify and communicate economic, social, environmental, democratic and cultural trends and outcomes.

Goal: Provide a sustainable mechanism for the collation, analysis and distribution of local community wellbeing indicator trend data across Victoria.

Target audience: All. Data type: Aggregate.

Community indicators: Yes. Fee for service/membership: Free.

URL: http://www.communityindicators.net.au/

appendix C

40

3.

Connecticut Nonprofit Strategy Platform (Interview – No) University of Connecticut Nonprofit Leadership Program & the United Way of Connecticut

Description of Initiative: One State, shared strategy for action. The Connecticut Nonprofit Strategy Platform is a shared web-based research and communication tool for Connecticut’s nonprofits, state policy makers and funders to use for public policy planning and action. It is a program of the Connecticut Data Collaborative, a collaborative public-private effort to improve the quality of and access to policy-related data in the state. It seeks to build advocacy for impact, create common data standards, and improve access and build capacity.

Goal: Make data accessible and useable for decision-making regarding the resources of Connecticut’s nonprofit sector. Map Connecticut nonprofit resources to human needs. Strengthen the impact of nonprofit collaborations through web-based communication, planning and document storage feature of the Community Impact Circle.

Target audience: All. Data type: Aggregate and can view and map own data.

Community indicators: Yes. Fee for service/membership: Free.

URL: http://nccsweb.urban.org/

4.

Greater New Orleans Community Data Center Greater New Orleans Community Data Center

Description of Initiative: The Greater New Orleans Community Data Center (GNOCDC) is a non-profit organization that works to create actionable information for use by everyone, from the general public to key government decision-makers and community leaders. It gathers, analyzes and publishes neighborhood level data. The web-based platform is designed to facilitate its easy use and includes the ability to map and to download data spreadsheets.

Goal: Help non-profit organizations, government and communities better understand the people and the neighborhoods where they live, work and play.

Governance: Executive Director and Board of Directors.

Data type: Aggregate.

Community indicators: Yes. Fee for service/membership: Free.

URL: www.gnocdc.org

appendix C

41

5.

Newfoundland Community Accounts (Interview – No) Government of Newfoundland & Labrador

Description of Initiative: Sharing data, providing information, developing knowledge. Community Accounts was developed as part of the Strategic Social Plan’s objective to measure social progress.

It is an information system providing users at all levels with a reliable source of community, regional, and provincial data. A public-wide, online data retrieval system for locating, sharing and exchanging information related to the province and its people, the Community Accounts provides users with a single comprehensive source of community, regional, and provincial data that would normally not be readily available, too costly to obtain, or too time consuming to retrieve and compile.

Target audience: All. Data type: Aggregate.

Community indicators: Yes. Fee for service/membership: Free.

URL: http://nl.communityaccounts.ca/

6.

Peg (Interview – Yes)

International Institute of Sustainable Development & United Way of Winnipeg Description of Initiative: Peg is a web-based community indicator system designed to track Winnipeg’s wellbeing across eight areas (basic needs, health, education and learning, social vitality, governance, built environment, economy, and natural environment) and for one overarching concern, poverty. Its vision is to build the knowledge and capacity of Winnipeggers to work together to achieve and sustain the well-being of current and future generations.

Goal: Tracking progress. Taking action.

Governance: Steering Committee, Engagement Group, Indicators Working Group.

Target audience: All. Data type: Aggregate.

Community indicators: Yes. Fee for service/membership: Free.

URL: www.mypeg.ca

appendix C

42

7.

Wellbeing Toronto (Interview – Yes) City of Toronto

Description of Initiative: Wellbeing Toronto is a web-based knowledge mapping tool that permits users to assess wellbeing across 140 Toronto neighborhoods by choosing, combining and weighting different data. Indicators of wellbeing fall under ten general domains (civics, culture, economics, education, environment, health, housing, recreation, safety, transportation).

Goal: Be a research based indicator tool that provides a common fact base across neighborhoods, and over time.

Governance: Steering Committee, Staff Reference Group of City Divisions, Community Reference Group, Academic Experts Panel.

Target audience: All. Data type: Aggregate.

Community indicators: Yes. Fee for service/membership: Free.

URL: http://map.toronto.ca/wellbeing

8.

Philadelphia’s Policy Analysis Center (Interview – Yes)

University of Pennsylvania, City of Philadelphia, and School District of Philadelphia Description of Initiative: University of Pennsylvania, the City of Philadelphia, and the School District of Philadelphia have collaborated to form the Policy and Analysis Center (PAC) (Intelligence for Social Policy, N.D). The Center builds on and merges the previous work of the Kids Integrated Data System (KIDS) and City of Philadelphia’s data management system (CARES) but is no longer exclusively focused on children. This data sharing system combines data from administrative databases of local agencies “womb to tomb”. It is strongly research focused.

Goal: To improve the health, education and social services for residents of Philadelphia.

Governance: Executive Leadership, Research & Data Advisory Boards, Project Advisory Teams.

Target audience: Primarily Researchers & Municipal Government.

Data type: Individual.

Community indicators: Available.

Fee for service/membership: N/A. URL: N/A – intranet

Kerry Coupland, Praxis & Theoria Inc. Dawne Clark, Centre for Child Well-Being, Mount Royal University Elaine Danelesko, Integrative Health Institute, Mount Royal University