The Outcomes Toolbox: What You Need to Know to Use Outcomes Robert M. Penna, Ph.D. Author: The Nonprofit Outcomes Toolbox
MNA Conference 2012 September 19, 2012 Helena, Montana
Today’s topics:
An introduction to Outcomes: What is an “outcome”? How is an “outcomes approach” different? What is the language of outcomes? What are the characteristics of a good outcome? Crafting an outcomes statement An introduction to “outcomes tools”;; the view from 30,000 feet Tools for planning Tools for tracking and management Tools for reporting Tools for learning and assessment Using some tools Building your “Outcomes Tree” Understanding and Assessing Capacity Reporting 2
Our targeted outcome: ď Ž
That within one week you will do something different at work because of something you learned today
3
A quick bit of history……
4
So, think of it this way…. Outcomes: the 3rd Stage of Management • Stage 1—Agriculture and Early Industrial Economy: Management of Workers • Stage 2—1920–1970: Management of Work • Stage 3—The Age of Outcomes: Management of the Product of Work. . . The Results 5
An introduction to “Outcomes”
What is an “outcome”?
6
What makes an “outcomes approach� different?
7
The Problem Approach
8
The Activity Approach
9
The Process Approach‌
10
The Vision Approach‌
11
The Outcomes Approach
12
Compare this……
13
To this…. What’s different??
14
Even better‌..
15
Three Changes in Thinking 1. 2.
3.
From Funder to Investor From Activity to Effectiveness From Services to Change ď Ž
The BACKS Measures
16
The “Language” of Outcomes Väljund
Processus Ingresso
השפעת Επιπτώσεις
ﻧﺗﺎﺋﺞ
17
The Characteristics of a Good Outcome
Are some outcomes “better” than others? What does a “good” outcome look like? Positive Improvement Meaningful Sustainable Bound in Time Bound in Number Narrowly Focused and “Doable” (with a stretch!) Measurable
Clearly Definable Discernable…can be “evinced” Quantifiable…according to an agreed-upon scale!
Verifiable
18
The “Outcomes Statement” What
is it? Why Use it?
19
Practice
Your turn……..
20
Take a break!
21
An Introduction to “Outcomes Tools”: a view from 30,000 feet…..
What are “outcome tools”? 4 sets:
Tools for Planning Tools for Tracking & Managing Tools for Reporting Tools for Learning & Assessment
22
Tools for Planning
You Get What You Measure A Common Outcomes Framework
23
Tools for Tracking and Managing
The Logic Model RBA TOP GTO TRI Outcomes Management Results Mapping The Balanced Scorecard
24
Snapshot: The Logic Model
3225
A Detailed Logic Model
3326
But it gets tricky when‌.
27
Snapshot: Results-Based Accountability
4228
Snapshot: TOP Targeting Outcomes of Programs (TOP) Model Program Development (Goals/Objectives)
Program Performance (Evaluation Measures) SEEC
SEEC
Practices KASA Reactions
Practices KASA Reactions Participation SEEC: Activities S = Social Resources E = Economic E = Environmental C = Conditions
Participation Activities KASA: Resources K = Knowledge A = Attitude S = Skill A = Aspiration
Source: Claude Bennett & Kay Rockwell, 1994
29
TM The Getting To Outcomes Snapshot: GTO Process 4
3
2
Find Findexisting existing programs programsand and best bestpractices practices worth worthcopying. copying.
Identify Identifygoals, goals, target targetpopulation, population, and desired and desired outcomes. outcomes.
Modify Modifythe theprogram program or orbest bestpractices practices to fit your needs. to fit your needs.
5 Assess Assesscapacity capacity (staff, (staff,financing, financing, etc.) to implement etc.) to implement the theprogram. program.
Steps 1-6 PLANNING
6
1 Choose Choosewhich which problem(s) problem(s)to to focus focuson. on.
10
Make Makeaaplan plan for forgetting gettingstarted: started: who, what, when, who, what, when, where, where,and andhow. how.
DELIVERING PROGRAMS
Steps 7-10 EVALUATING AND IMPROVING
7 Evaluate Evaluate planning planningand and implementation. implementation. How Howdid didititgo? go?
Consider Considerhow howto to keep keepthe theprogram program going goingififititisis successful. successful.
9 Make Makeaaplan planfor for Continuous Continuous Quality Quality Improvement. Improvement.
8 Evaluate Evaluate program’s program’ssuccess success ininachieving achieving desired desiredresults. results.
30
Snapshot: TRI/Outcomes Management
31
Snapshot: Results Mapping MLS7 MLS6 ACT5
>
LEV5
MLS4 ACT3
>
LEV3
ACT2
>
LEV2
ACT1
>
LEV1 3832
Snapshot: A Balanced Scorecard
33
Tools for Reporting
Scales & Ladders The STORY format
34
Snapshot: Scales & Ladders
35
Telling your story First
off….begin by thinking of it as a story….
36
A Quick Test: These
37
As opposed to these…… 100 East West North
50 0
1st 3rd Qtr Qtr
A
82
45
54
62
B
37
97
17
12
C
10
83
7
7
HQ
Boston
Seattle
Dover
38
Tools for Learning and Assessment 1. 2. 3. 4. 5. 6. 7.
8.
Lessons Learned Six Sigma SERVQUAL Value Engineering FMEA TRIZ CAT GTO 39
LUNCH!!!
40
Building Your Outcomes Tree 1. 2. 3.
Outcome Performance Targets Milestones
41
You try it….
42
Take a break!
43
Understanding and Assessing Capacity
What is “capacity”? Three levels
Structural Functional Implementation
Two Tools
CAT GTO
44
CAT
VPP/SVP Capacity Tool
Essentially a self-scoring grid/matrix format…can be created in Excel
ASPIRATIONS
Mission
1 Clear need for increased capacity
2. Basic level of capacity in place
3. Moderate level of capacity in place
4. high level of capacity in place
No written mission, or only a limited expression of the organization’s reason for existence; lacks clarity or specificity; either held by very few in organization or rarely referred to
Some expression of organization’s reason for existence that reflects its values and purpose, but may lack clarity; held by only a few; lacks broad agreement or rarely referred to.
Clear expression of organization’s reason for existence which reflects its values and purpose; held by many within organization and often referred to.
Clear expression of organization’s reason for existence which describes an enduring reality that reflects its values and purpose; broadly held within organization and frequently referred to.
45
TM The Getting To Outcomes Snapshot: GTO Process 4
3
2
Find Findexisting existing programs programsand and best bestpractices practices worth worthcopying. copying.
Identify Identifygoals, goals, target targetpopulation, population, and desired and desired outcomes. outcomes.
Modify Modifythe theprogram program or orbest bestpractices practices to fit your needs. to fit your needs.
5 Assess Assesscapacity capacity (staff, (staff,financing, financing, etc.) to implement etc.) to implement the theprogram. program.
Steps 1-6 PLANNING
6
1 Choose Choosewhich which problem(s) problem(s)to to focus focuson. on.
10
Make Makeaaplan plan for forgetting gettingstarted: started: who, what, when, who, what, when, where, where,and andhow. how.
DELIVERING PROGRAMS
Steps 7-10 EVALUATING AND IMPROVING
7 Evaluate Evaluate planning planningand and implementation. implementation. How did it How did itgo? go?
Consider Considerhow howto to keep keepthe theprogram program going if it is going if it is successful. successful.
9 Make Makeaaplan planfor for Continuous Continuous Quality Quality Improvement. Improvement.
8 Evaluate Evaluate program’s program’ssuccess success ininachieving achieving desired results. desired results.
46
GTO capacity question CAPACITY TOOL: Program specific staff capacities (full version) Capacity assessment item
Requirements
Is current capacity sufficient? (Y/N)
Plan to enhance the capacity
Staff Training needed
Staffing level required
Staff qualifications (e.g. minimum degree, certification or training needed; years of experience)
47
Capacity grid for Recruiting potential trainees Challenge
Unready
Marginally Ready
Fairly Ready
Fully Ready
Identifying and contacting 125 unemployed adults in the community who will be potential candidates for job training and placement program
Not sure how we'll do it
Some idea of how we'll do it, but few details worked out
Pretty good idea of how we'll do it; only a couple of details still need to be resolved
Firm idea of how we'll do it; details all worked out
1
Communicating with potential candidates; fielding and answering their questions
Not sure how we'll do it
Some idea of how we'll do it, but few details worked out
Pretty good idea of how we'll do it; only a couple of details still need to be resolved
Firm idea of how we'll do it; details all worked out
2
Holding orientation session for 100 potential candidates
Not sure where or how we'll do it
Some idea of where and how we'll do it, but few details worked out
Pretty good idea of how we'll do it; only a couple of details still need to be resolved
Firm idea of where and how we'll do it; details all worked out
3
Processing enrollment of 75 potential candidates
Not sure where or how we'll do it
Some idea of where and how we'll do it, but few details worked out
Pretty good idea of where & how we'll do it; only a couple of details still need to be resolved
Firm idea of where and how we'll do it; details all worked out
7
Identifying prospective local employers who might be willing to hire program graduates
Not sure how we'll do it
Some idea of how we'll do it, but few details worked out
Pretty good idea of how we'll do it; only a couple of details still need to be resolved
Firm idea of how we'll do it; details all worked out
Milestone
48
Reporting: Using Scales & Ladders
49
Next Steps: Action Planning Come up with a specific action step to be started next week
63
Wrap-up, feedback, and Questions
51
Thank you for your time and participation‌.. Contact: Dr. Robert Penna PO Box 38081 Stuyvesant Station Albany, NY 12203-8081 518-438-7502 drbob@outcomestoolbox.com Website: www.outcomestoolbox.com
52