Measuring Change Outcomes Toolkit Helping you to measure the difference you make
The Measuring Change Toolkit is about involving the people and communities you work with in measuring what difference you make. It is not finished . . . and it never will be! The Measuring Change Outcomes Toolkit is meant to adapt and change as you use it. There is one condition – if you find something that does not work, or something new that does – you change it and join our community. If you agree with this condition – read on (and keep in touch)!
1
CONTENTS
2
Welcome to the Measuring Change Toolkit The Measuring Change Toolkit helps you to involve people in measuring the impact (the outcomes!) of what you do. It can be used for:
Planning what you do
Involving the people you work with
Measuring the difference you make
Telling others about what you achieve
It can be used by different people in your community, group or organisation (depending on what you want to do). You can use all the information by reading through the different sections or go directly to the place you need. It is designed to follow the Big Lottery’s approach to developing your project and setting outcomes. We also introduce you to Results Based Accountability or RBA, which provides a way for you to measure the impact of your project.
Measuring Change is structured in different parts, depending on what you want to us it for. Part One introduces you Outcomes - what are they and how you do you use them? Part Two helps with Understanding Need for your project which helps you to focus on what needs to be done. Part Three has advice on Setting Aims, Objectives and Outcomes for you to be clear on what you want to achieve and how you will achieve it. Part Four helps you to Track Your Progress when working toward achieving your outcomes and how you can measure the difference you will make. Part Five provides examples of the different Tools for Measuring Outcomes you can use when working with groups. Part Six has suggestions on how you can Collect Evidence and make use of the information you have collected. Part Seven gives ideas on how to Use the Information and how you can learn from and communicate the evidence you have gathered. Part Eight introduces different ways to measure Soft Outcomes and Distance Travelled including Results based Accountability, Social Return on Investment, The Outcomes Star and the Richter Scale Part Nine is all about Results Based Accountability as a way to move quickly from strategy to action. Part Ten take a brief look at Social Return on Investment at a way of assessing the value of your outcomes Part Eleven points to sources of Help and Information.
3
W H A T I S A J A R G O N B O X? Understanding the Jargon: Jargon are special words or expressions that are used by a particular group that can be difficult for others to understand. Their use and meaning can change over time. This can be a little confusing, so throughout this toolkit we have inserted these jargon boxes to help explain their meaning. It is important that you explain what you mean and you ask if the people you work with understand what you mean.
4
PART ONE: INTRODUCTION TO OUTCOMES What is an outcome? An Aim or an Outcome? Outcomes, Outputs and Targets Soft or Hard Outcomes? Project or Strategic Outcomes? What is an Outcome? You are probably reading this because you have a project or an idea for a project . This part of the Toolkit will help you to look at developing and planning your project and be clear about what outcomes you want to achieve. This Toolkit sets out some of the main steps that you will have to take to plan and manage a project using an outcomes approach. This Toolkit is aimed at people developing or taking part in a project. Sometime project outcomes need to fit in with more strategic outcomes, and we describe how your project might fit in with ‘population outcomes’ used in Results Based Accountability. We cover:
understanding and explaining the need you want to meet
being clear about your overall aim and how your project will meet the need
measuring your success
learning from what you have achieved to improve your project
This will be useful for a wide range of people but it is aimed at those who are new to the outcomes approach and want to find out more. For this reason we provide further information, this guide is available on-line and gives lots of examples and sources of support. An Outcome is ‘the way a thing turns out; a consequence’
So an outcome is the end result. The term outcome can be used to describe the end result of:
a strategic programme
your whole project
a piece of work within a project
You will need to be clear when using the word outcome what you mean. You will need to make sure when others use the term outcome, you are clear what they mean by the term.
5
JARGON BOX People will use different words for the same things, and the same words for different things! We will talk about using a method called ‘results based accountability’ or ‘RBA’. In RBA an outcome is defined as ‘A condition of well-being for children, adults, families or communities’. For example, ‘older people in my community will be active and healthy’
An Aim or an Outcome? When we are planning to do something, we usually start by looking at what we want to achieve or the outcome. This usually means a positive outcome such as ‘I want children in my community to be healthier’. A common approach to planning your project is to use Aims, Objectives and Actions (these can also be called Activities or Tasks).
An Aim ‘a purpose or intention; a desired outcome’
So an ‘aim’ is what your projects wants to achieve. It can be described as the outcome of your project. In this Toolkit we use project outcomes to describe the end result of a piece of work within your project. We use aims to describe the overall purpose of your project.
JARGON BOX Be careful, RBA refers to outcomes only when talking about whole populations. It uses the term outputs when describing what happens as the result of a project or service.
An Objective ‘breakS down aims into clearly defined steps with a beginning and end that can be measured.’ Your objectives can be broken down into more specific actions that can be achieved within an agreed period.
JARGON BOX Aims and objectives are often used to write an ‘action plan’. An action plan is usually in the form of What you will do, why you are doing it, who will do it, where and when (or What, Why, Who, Where and When!). SMART Objectives: SMART is also used to help you to develop objectives. It stands for Specific, Measureable, Achievable, Realistic and Time-bound. Do you find this confusing? We do, so we just used SAM instead - Specific, Achievable and Measureable).
Outcomes, Outputs and Targets An Output is ‘the amount of something produced.’ To measure an output is to measure what you have done or the effort you have made. To measure an outcome is to measure the end result, not the work involved in getting there.
6
COMMUNITY LEARNING EXAMPLE A community project in Talygarn was paid for putting on courses in their community. They had to put on 10 courses, that was their target. They put on all the courses and met their targets. The output was 10 courses delivered. However, no-one turned up! There was an output – training was delivered. But there was no outcome. No-one turned up and no-one learned anything.
What should have they been asked to do? A starting point would have been to count the number of people turning up to take part on the course! It would also have been useful to know how many people finished the course. But if we want to know what the outcome, we want to find out what people gained from the course. This could be confidence, meeting new people, learning a new skill or gaining a qualification.
A Target is ‘an objective or result towards which efforts are directed.’
Targets are often very specific, set by you or others, that you aim to achieve within a set time. Targets can be useful in setting a clear measure of what you aim to do by when. However, they can cause problems when you concentrate only on the target. For example, when there are too many targets, it is difficult to decide which is the most important. Sometimes people concentrate only on the target, and sometimes forget about why the target was set in the first place. Targets are best used as a tool to achieve your outcome, not an end in themselves.
Performance Indicators are ‘key bits of information that are used to define and measure performance.’
People have tried to improve the use of targets by looking for better measures that tell you and other more about what you are providing. That is, they are good indicators of how well you are doing. People use the terms performance indicators (or PI’s) or key performance indicators (or kpi’s).
Soft or Hard Outcomes? A hard outcome is easy to fix, define and measure. It may be number of qualifications gained or jobs taken up. As a result people have often steered towards using hard outcomes to show they are making a difference because they are thought to be easier to demonstrate. A soft outcome measure changes in behaviour or confidence. Soft outcomes are often based on asking the opinions of individuals you work with. Some people may feel these outcomes are less important or less robust. We would argue the opposite, as they report on what difference you have made by the people who benefit from your project or service.
7
This Toolkit will provide you with ways of capturing and measuring soft outcomes. Often when working with people and communities it is easy for you to see the change that has occurred as a result of your Work. However it can be much more difficult to provide hard evidence. This toolkit will help you bridge that gap between individual detailed case studies and hard outcomes. Sometimes soft outcomes are referred to ‘user reported outcomes’. COMMUNITY LEARNING EXAMPLE
A soft outcomes in the Talygarn Community Project could be 15 people felt they had learnt a new skill.
A hard outcome could have been 15 people attended and 5 gained a qualification.
Project or Strategic Outcomes? What is an outcome? Outcomes that are the changes or difference that your project can make over time are called project outcomes. An easy way to think of them is that they are the result of what you do, rather than the activities or services you provide. For people, this might be things like improved health, new skills, more confidence or self-esteem, or getting a job. Some projects focus more on communities, the environment or organisations. The outcomes might relate directly to them (such as tidier communities or more biodiversity) or changes in people that help benefit them (such as more awareness of environmental issues, reduced fear of crime, or more community participation). Your project might apply for funding under a larger programme or strategy which already have included within them wider ‘strategic’ or ‘programme’ outcomes. Your project will need to show not only how it meets your own project outcomes, but also these strategic or programme outcomes.
Population Outcomes using Results based Accountability Results Based Accountability (RBA) using the term ‘population outcomes’. This Toolkit uses RBA as the method of developing a thorough OUTCOMES PROCESS. This is because:
It starts with looking at outcomes.
It looks at outcomes in relation to people and communities
It is about wellbeing
It is a clear process that works
It is easy to follow JARGON BOX
In RBA an outcome is used differently to describe the outcome of a whole population and is used to describe ‘A condition of well-being for children, adults, families or communities’. For example, ‘older people in my community will be active and healthy’. RBA refers to outcomes only when talking about whole populations. It uses the term outputs when describing what happens as the result of a project or service instead of the term project outcome.
8
PART TWO: UNDERSTANDING THE NEED FOR YOUR PROJECT You probably have a good idea about what you want to do and why you want to do it. Most projects start off trying to solve a problem. One of the first steps in taking an outcomes approach is to write down the problem (or issue). This is the first step to identify why your project is needed. Try to be clear. There may be many reasons why your project is needed, but try to stick to the main ones that you think are most important and you want to work on. Working through why your project is needed will others will help you focus on what is most important. If you write this down and agree a few short sentences to describe why your project is needed, you will be able to make sure everyone involved understands the problem you are trying to solve. Examples of overall need
there is lots of litter and dog mess around the estate
young people are hanging around the shops and intimidating people
families with children with a disability are isolated
Evidence of Need How do you know the need exists? Before moving forward, you will have to provide some evidence. The amount of evidence you need often depends on the size of your project. Evidence can come from different places, such as:
talking to local people
discussing the issue with those most likely to be affected
local research such as a survey of users of a service or a community
information about the people or community most likely to be affected. For example, the number of people that are unemployed or the number of children with a disability.
You will almost certainly have to spend some time on research. In general, the time you spend will be related to the size of your project. In this Toolkit we focus on smaller community projects. For larger organisations and projects, you will need to know more about local, regional or national strategies and how your project fits with them and look at what evidence they include.
People will not share the same views all of the time. Some groups may have very different opinions, for example, young people usually want different things to older people. Therefore you need to think carefully 9
about what you need to find out from whom. You might need to make an extra effort to get the views of people who spend more time at home, are at work or might not mix as much as others in the community. Example: Evidence of needs experienced by families with children with disabilities There are 80 families with children with disabilities recorded in the area your project covers. You have asked 20 of the parents and asked them about their experiences. Of the 20, 16 (or 80 per cent) said that they felt isolated. This suggests that of the 80, about 64 families in the area will also feel isolated. The parents also said that the main reasons were lack of contact with other families with children with a disability and lack of places to go with their children.
Understanding the reasons for the need There may be a number of possible causes for the need you intend to address. Again, it is important to state which needs you will address and which you will not. It is also worth having clear reasons why. The table on the right gives some examples of problems, reasons and possible solutions.
Overall
Understanding of the
Possible solution (project)
problem
reason for the problem
Too much litter and
Too many young people
Talk to young people in schools and youth
dog mess
people dropping litter
clubs
Young people
They are not doing
Talk to young people about what they
Are hanging around
anything wrong, there is
would like to do and why they hang around
the shops and
just a lot of them and
the shops. Try to find somewhere else for
intimidating people
they play music
them to go where people would be less
on the estate
intimidated Families with
There is no place for
disabled children are families to get together
Arrange an event for families and children to get together in the local area
isolated
Get together with other people involved with your project. Write down the need that you have identified, and get everyone to come up with possible answers to the following questions: 
What is the need?

What is the evidence for it?

What is causing it? 10
Who else should we ask?
Then decide as a group what the main reasons are. This is likely to be the need that your project plans to address. Once you have decided, you need to check with the other people you identified and ask their advice. This is about making links and is an important stepping stone in developing your project. This may help you to find out:
Why this project has not been tried before.
Who can help with advice, practical help or funding.
If there are similar projects that you can work together with.
If volunteers are willing to help.
Things to look into further like safety issues.
If someone is already doing what you want to do.
If people are aware of the problem.
Thinking about and talking to other projects may help you to find evidence about what is needed and what is likely to work. Similar projects to yours that are already working elsewhere can often give you a lot of good advice. If you can answer the following questions, you know you are on the right track.
What need is your project trying to meet?
Have you all the evidence you want to show that your project is needed?
What people and how many of them will benefit from your project?
Will your project help those people most in need?
How will they benefit?
Is your project the best way to meet the need you identified?
Once you understand the need you want to address, you can start work on being clear about the difference that you want to make, and then how you are going to do it.
11
PART THREE: SETTING AIMS, OBJECTIVES AND OUTCOMES Aims are what your project aims to achieve. They are sometimes described as outcomes, but in this Toolkit we refer to:
Project Outcomes are what you expect to happen as a result of your work; or
Strategic Outcomes that relate to a strategy your project will help achieve.
Population Outcomes are strategic outcomes used to describe an improvement in wellbeing for a whole population when using in RBA.
JARGON BOX Be careful, RBA refers to population outcomes only when talking about whole populations. It uses the term outputs when describing what happens as the result of a project or service.
How to agree your Aims? If you do not know where you are going, it is difficult to plan how to get there! People like to jump in and get on with planning activities. But when you are clear about the need or problem that you want to solve, the first step is to be clear about and agree your Aims. Top Tip – try to keep to one aim. The more you have, the harder it is will be to achieve them.
Working with others Working with others to agree your Aims is really useful. You get to discuss different issues and different ways of doing this. The more people you talk to, the better. This is true of a lot of things in this Toolkit – the more people you talk to, the better, including understanding the need for your project.
There are lots of benefits from talking to different people and organisations when agreeing your aims, these include:
They are much more likely to want to get involved,
They will have evidence and information.
They may have useful knowledge and skills.
They are more likely to want to be involved in making it happen.
More people will know what you are trying to do – your first step in promoting your project.
You need to work closely with others to agree what your aims are, You will need to be clear on what is realistic for your project to achieve and how it fits with what others are doing.
12
JARGON BOX Stakeholders is a term used to describe anyone who might be affect by or involved in a project. It can be people who use the project or service (service users); people involved in carrying out the project (such as staff, volunteers and partners), and partners who can help make it happen (or stop happening!) such as people who might provide funding for the project. Your aim is the main drive of your planning and everything you do should contribute towards achieving it. Your project outcomes are the signs that will tell if you are achieving your aims. Here are some examples of aims:
To reduce unemployment in Ponty Bryn.

To improve the confidence of vulnerable young people in Abertaf.
The Planning Triangle shows how you can directly relate your outcomes to your aims. This is very useful to keep you on track. Looking at the triangle below, imagine the outcomes are missing. Look at your aims and outputs on their own. It is now difficult to link your aim with what you provided. Project outcomes help you make that link.
To improve the confidence of young people
Aims
Project Outcomes
50% of young people report improved confidence
Outputs
To provide 10 drop in sessions for young people. 75 young people attend
JARGON BOX O Aims, Outputs and Project Outcomes:
O
These 3 words can cause confusion. Here is a breakdown:
13
Aims are what you are trying to achieve. For example: To reduce unemployment in Ponty Bryn. Project Outcomes are the positive changes that happen as a result of your work. For example, we helped 13 people get a job. Outputs are what you do, the activities you deliver in your project For example, we provided a job club once a week for unemployed people.
Project Planning
Once you have decided on your aims and project outcomes, it is time to look at your objectives. An Objective ‘breaks down aims into clearly defined steps with a beginning and end that can be measured.’ Depending on the size of your project, your objectives may be broken down into more detailed actions. When setting your objectives, ask yourself if they are Specific, Achievable and Measureable (SAM). These make up three of the five element of ‘SMART’ Objectives).
14
PART FOUR: TRACKING YOUR PROGRESS 1. Introduction To show change or track progress you need to first find your starting point. That is where your start measuring from before you begin your project. You then need to gather information at set times during the project (for example beginning, middle and end) which shows you what progress you are making. Some organisations working call this ‘distance travelled’. This method can useful to track the progress of soft outcomes. JARGON BOX Monitoring: You might check the performance measures regularly over the course of your project and this is known as monitoring. Evaluating: At the end of your project you will review your performance measures to see if you have indeed made a difference and achieved your outcomes.
The things that you pick to measure the progress you are making can be called different things, including performance measures, outputs, targets and outcomes. Here we will call the things you measure as performance measures. Achieving your performance measures should tell you are achieving your project outcomes. The job of finding the right performance measures usually involves finding specific things you can measure in your project. Don’t be afraid of coming up with an unexpected way to see if something has worked or not. According to R.B.A. The performance measures relating to effect are the most important. The example on the following pages should help you get a feel of the kind of questions you should be asking and help you work out the best questions for your project. 2. Establish a baseline Establishing your Baselines: This is the term used by R.B.A. which refers to checking the things you are going to measure before you begin your project, activity or service. Your baseline is your starting point. It’s the information you collect that tells you where you are at the beginning of the project. Baselines are also a useful tool to evidence need. The best way to check your project is making a difference is to ask people;
15
before you do your project
and then ask again after you've done your project
You need to be quick off the mark. You want to show your project has made a difference so you need to check where you are starting from. Choose your activity and then plan to do your start or ‘before’ measure at the soonest possible opportunity. Then you'll need to do it again at the end. You need to agree when the end is and when you will collect this information. Think about when you'll see the people to ask them these questions. For longer and ongoing projects you will need to check progress during the project. This could be in the every three months, in the middle. Try and decide how often you need this information and how realistic is it to collect it. By measuring how you are doing against your start point or baseline, you will be able to quickly identify whether it’s working. If not, what you need to change to make it work before it’s too late! Make sure that you leave enough time to show a difference has happened. What if the project has already started? Some of the measures don't need to happen before and after. You can ask people what they have gained, such as more confidence or skills. It can be a simple question at the end of your project. 3. Gather information You have agreed what changes you want to make (outcomes) and how to measure them to check these changes have happened (performance measures). You now need to decide how you will gather the right information and how often. We have made this easy for you by giving ways to involve your participants and groups. Of course, you may have your own ideas about how to collect this information. Try to think of different and fun ways to involve people, they usually work best. Some people choose traditional methods of collecting information such as:
written surveys - paper, email, online;
interviews - face to face or telephone interviews;
observing people
Written information, such as progress reports.
You may use this for a community project, but we ALWAYS we recommend that you involve people to collect this information. You may find that written surveys do not really tell you what you want to know.
16
Some people do not like filling out written surveys. We think it’s important to ask people what they prefer. This can help them feel involved rather than ‘having something being done to them’. This approach is especially meaningful when measuring soft skills as it allows people to be part of and see their own progress. This can be very powerful and inspiring. You will need to decide when you will gather this information. It may be easier to collect it during day to day activities. Sometimes you might find it easier to hold a separate event before, during and after the project just for this. 4. Asking the Right Questions It is important to make sure that you ask the right questions in the right way. Gathering information takes time and effort. It can be frustrating if you don’t get the information you need. Think carefully about what you want to find out. The answers need to tell you if you are achieving your outcome. Make sure you are not putting anyone on the spot or asking them to reveal something they may not want to share, especially if you are working in a group. Here is an example. You want to find out if your café for people with experiencing poor mental health has impact on how they feel. If you ask ‘How do you feel?’ they will answer based on how they are feeling at that specific time. This will be influenced by lots of things that have happened to them. Instead make your question more specific. You could ask,
‘How confident do you feel as a result of coming to the café?’; or ‘Have you met new people as a result of coming to the café?’ Asking people to give a score or rating for this kind of question at the beginning, middle and end of your project can be very useful. This can help you quantify the impact of your project on the people involved. There are tools in the activities section to help you do this. Things to think about when asking questions:
Make your questions are specific and relate to your outcome.
Explain to your participants what you are doing and why.
Ask a mixture of closed and open questions. With closed questions you can collect things like numbers and percentages (e.g. 15 people or 20% felt that they felt more confident after visting the café’). With open questions, you can often get more in depth responses (e.g. I felt more confident because Jenny made me feel really welcome). This is called qualitative information.
Explain what you are going to do with the information and talk about what is confidential.
17
Try a pilot with a small group to test that your questions are working and you are getting the information you need to measure change.
Make sure
Try a range of different ways of asking the questions so you can adapt to your participants and how they like to work.
5. Involve groups in information gathering Here are a few golden rules of working with groups;
Make sure they know why they are there
It is vital that you make sure your aims are clear to people and allow them to decide if they wish to take part. Explain fully what you are hoping to get out of the process and what you can and cannot do . It is important that all those involved have realistic expectations.
Agree the boundaries
Try to set boundaries early on. We favour an approach called ‘Making it Work’, the first activity of the toolkit because it should come early in any group work setting. It allows everyone involved to have a say in how the process will work and express any specific needs they may have.
Find out about any specific needs or disabilities
If you are responsible for the activity, it is your job to make sure that everyone can be involved. Try to: -
Gather information about the group before the sessions
-
Be prepared to make changes if something doesn’t work for someone in your group does not get involved
-
Be aware that lots of people are not comfortable with reading and writing
-
Make pans for people who are unable to move around easily.
-
Give people the opportunity and support to get involved, but let them have the space and choice to decide not to get involved
Take care of basic needs
Make sure you provide a pleasant environment to work in, it should be warm and welcoming. People will respond better when they are comfortable. At the very least make sure you have drinks available, brains don't work very well without them. Breaks are essential, and best will also be valued and valuing.
Don't put people on the spot
The activities in the toolkit are designed to allow people to contribute without feeling pressured or 'on the spot'. Some people will volunteer themselves, this is fine but allow the group to contribute as much or as
18
little as they want. For example avoid directing people to speak, allow them to let you know when they have a contribution to make. Avoid activities which move relentlessly round the group like a searchlight.
Move around
Brains fall asleep or at least, lose concentration very quickly if you leave them in one place for too long. Give you group the opportunity to move around. Divide people into groups or use games and activities in your session. Giving plenty of breaks can do the job. There are plenty of good books and on line resources that will help you with games in training.
Have some support and look after yourself
Most of the activities you will find in this toolkit benefit from one person leading with someone to help. Being fully prepared with all the materials ready will help. You may want to get some training in group work to help you plan and deal with challenging situations
Feedback
Make sure your group know what you are going to do with their views and how they are going to help with your planning, monitoring, evaluation and funding processes. Let them know how they can access the findings.
Ending
Make sure you prepare an activity to end your session. This should always include a clear 'thank you and good bye' and an explanation of the next steps for them or the information they have provided. We have provided some ‘end’ activities in the Toolkit.
Remember
Before you run any of the activities in this toolkit, you should decide how you are going to record and collect the information. Remember to record:
-
The date of the session
-
Where it was held
-
How many people participated
-
Age and whether male or female
Good Practice
We recommend that you check out: -
The National Principles for Public Engagement in Wales; and
-
The National Children and Young People’s Participation Standards for Wales.
19
PART FIVE: TOOLS FOR INVOLVING PEOPLE IN MEASURING OUTCOMES The tools are described as group activities but many can easily be adapted. We have regularly used these methods at community events in which we ask participants to just drop in and spend a few minutes giving us feedback individually or in small groups. Monitoring: If you intend to monitor and use these tools frequently, you want to choose the tools that are quick to use. Avoid tools that are time consuming if they have to be repeated quite often. Before and after: If you need to establish your baseline, you do this by doing these information gathering activities before the project begins and then repeating this at a later stage to show the progress. What if its tool late for establishing baselines? Let’s be realistic, many people will pick up this toolkit when they are half way through their project or approaching the end. DON'T PANIC. There are ways to ask questions that ask your participants to give their opinions about how things have changed or what difference has been achieved. When we are trying to look back we call this ‘retrospective evidence’.
Good for Good for Can be Can be Tools
Can be used to
Good for
repetition to
Good for
whole
small
used in
used
groups
groups
drop in
1 to 1
Dot voting
Equaliser
Value lines
Grid
Worlds best
Stars
Bricks in the wall Gingerbread man Close to the goal
Hot spots & grot spots
establish a baseline
regular
Good for
monitoring
show before & after
retrospective evidencing
20
Making it Work Introduction activity – an activity that allows the whole group to agree what will make the session successful Time: 10 – 20 minutes You will need Flip chart and a marker pen, really good listening skills.
Why do we like it? It gives the group ownership, allows you to tailor the session to their needs and preferences, it makes it clear that you are listening to them and that they are involved in a participative process. How does it work?
Sit the group in a circle.
Explain that you want the group to decide what they want and need to make the group work for them.
Give some examples of the kind of things you mean: Do they want breaks? How long do they want to meet? Do they want games and activities or serious discussions? Ask what they want you to do as facilitator, what they want to opportunity to do themselves and what they want the rest of the group to do to make the session successful. Remind them that the session has to achieve the aims already set out - you should have these write up and displayed.
Tell them they have 3 minutes to find out from the people on either side of them what they want from the group to make it work.
Whilst they are talking go around to the pairs and start to collect some of their ideas. As this is the first time you have asked them to speak it is really important that you go to each pair and ask them for their contributions rather than ask them to call them out in front of the whole group.
After 3 minutes feedback the ideas you have collected.
Prompt them to suggest any other ideas.
Make sure you've included ideas about how they will treat each other, and how they will listen and respect each others ideas. These normally come from the group, but they may need some prompting.
Now check that everyone in the group agrees to this as a way of working. Display it for the session and refer back to it if you feel that the session is not going well.
21
Archipelagos Introduction activity – an activity that helps the group get to know each other and start to share their views Time: 10 – 20 minutes You will need room for everyone to move around
Why do we like it? It gives everyone a chance to speak to some new people. It can gently break up cliques. It gives everyone a very safe experience of speaking in the group which helps for later contributions. It works equally well for groups who know each other already and those who don't or groups who are a mixture. How does it work?
Ask the group to arrange themselves in to alphabetical order according to their first name.
Once they have found their position ask them to speak to the person next to them, introduce themselves if they have not met before and then share some information about their name, this could be the meaning of their name, if they like it, why they were given it, if they have any nick names. Give them the options and allow them to choose. After a few minutes ask if anyone want to share the story they heard or repeat their own story to the whole group.
Repeat the process two or three times with a different order and different subject to discuss these could be; order of the month you were born in and a present you were given or gave someone else, distance for where you are right now to where you were born or the first home you remember. For the final order you can ask the group to discuss something relevant to the session such as when they first heard of your organisation, how long they have lived in the area or attended the service.
Avoid orders or questions that might make people feel uncomfortable such as age or height.
22
Bricks in the Wall Information Gathering Activity – an activity that explores barriers and solutions to those barriers Example of how to use it: Health If an intended outcome of your project is to increase the uptake of services for health, well being and mental health in the area then you might use the following performance measure. Performance measure – Community members are able to identify a wider range of barriers to accessing services and suggest more practical ways of breaking them down. Time: 20 to 40 minutes You will need small rectangles of paper and pens.
Why do we like it? It is a great way of pooling concerns and bringing sometimes hidden barriers out into the open. It empowers groups to take responsibility for finding their own solutions and can help promote empathy.
How does it work?
Write 'Accessing health, mental health and well being services' on a large sheet of paper.
Explain that the group is going to be exploring barriers first and then looking at solutions.
Give the group five or six of the small rectangles of paper each and ask them individually or in small groups to come up with barriers that may prevent a person from accessing health, mental health and well being services. Writing one barrier on each rectangle of paper. These should be placed on the large sheet of paper so that they build up a wall.
Once they have written all the ideas they can think of compile any duplicates and review the 'bricks' as a group.
Now ask the group to suggest solutions to individual barriers, if they can identify a solution they can turn the 'brick' over and note it on the reverse side.
Measuring Measuring from this activity requires some analysis. You will need to decide whether the barriers are realistic, varied and thorough and how easy the group found it to identify solutions. Some quantitative data can be gathered from the number of barriers and solutions expressed but you will need to be clear that those counted are robust.
How to adapt it? This activity can be used to explore barriers and solutions to any issue. You can develop the thinking process by asking the group to prioritise the barriers and then discuss them in more depth. 23
Why it works The 'wall' image is useful for a dialogue about barriers. Allowing barriers to be expressed freely before solutions are explored helps the discussion to flow. The barriers that the groups struggle to resolve can be useful pointers for future work. No-one is asked to talk about themselves but they are likely to include anonymised personal experiences.
Gingerbread Man Information Gathering Activity – an activity that gives and individual picture of hopes and plans for the future Example of how to use it: Learning If an intended outcome for your service is to help people access further learning opportunities then you might use the following performance measures. Performance Measure – More community members see learning and further education as a viable step to achieving their goals Performance Measure – More community members recognise that more learning and further education opportunities are available to them. Time: 15 minutes You will need outlines of gingerbread man shapes printed on paper. You can find an example in the resources section.
Why do we like it? It encourages individual reflection it gives a thorough picture of how people feel about the opportunities available to them. How does it work?
Give each person a gingerbread man outline and ask them to picture themselves in five years time. It can help to make a note of the age you and significant people around you will be at that time.
Ask the group to fill the outline with the things that the hope to be doing in five years time, include work situation, home life, free time, the people around you.
On the rest of the piece of paper – outside the outline, ask them to note any significant steps they will need to take to get to their image of themselves in five years time. You can share 24
and discuss the gingerbreads. Measuring Look at the gingerbread people and note how many times learning opportunities are mentioned and how concrete they are. Some people might be very specific, for example 'I need to complete the Access to Health and Social care course at Coleg Morganwg' others may be more vague for example 'Do some book keeping training' You can score the level of detail and certainty and the number of times learning opportunities are mentioned.
How to adapt it? For some groups starting with a gingerbread person that depicts their life now can be useful. You can use the same method for a variety of measures for example you could note how many people see themselves as still living in the area for perception of area.
Why it works This is a very personal activity and demonstrates how relevant people feel opportunities are for them. Some people might find imagining the future very difficult or might see very little change happening. It is important to offer support and reassurance and allow people to comfortable step out of the activity if it is not appropriate for them. It is also useful to mention that sustaining situations and keeping things the same are valid goals.
Close to the Goal Information Gathering Activity – an activity that gives a snap shot of opinions in a easy visual format Example of how to use it: Prosperity If an intended outcome of your service is to ensure the community view the area as prosperous then you might use this performance measure. Performance Measure – More community members believe that people in the area have enough money, jobs available to them, support when they need it, could set up their own business Time: 10 – 20 minutes You will need an image of a football pitch with goals. This could be anything from an a4 sized picture to a table sized model; you can find an example in the resources section. A football or picture of a football.
25
Why do we like it? The analogy of football goals is easily and immediately understood, it is fun and it allows for a range of opinions to be gathered. How does it work? Explain to the group that one goal is the opposition goal where we score, if a subject scores a goal then we are doing really well in this area , alternatively if it is an 'own goal' then we are not achieving on this measure. Anywhere between the two goals can represent a degree of success in this area. I.e. just outside the opposition goal is doing well but not quite there, two thirds of the way along is progressing well. Write the subject to be measured on the football for example 'Most people living in the area have enough money' and ask the group to place it on the football pitch according to what they believe. You can either ask them to reach a consensus or you can give each person their own football and compile the results later. You can record by taking a photograph. Repeat with any other questions
Measuring When compiling the results you can translate the position on the football pitch to a numerical score, so 'own goal' is zero and the opposition goal is ten. Alternatively you could translate it to strongly agree, agree, not sure, disagree, strongly disagree. If it is useful you can also ask the group why they chose that point on the pitch and record their responses.
How to adapt it? Use any question you want or use a different image netball, a race would work just as well. You can also use a meal so the lowest rank is just ordering our food, followed by starters to represent we are just beginning to make progress all the way to desert where much has been achieved.
Why it works It gives a clear picture of peoples perceptions, it allows for a wide range of opinions, it can be easily translated to numerical data. People are generally comfortable with the football pitch image.
Dot Voting Information Gathering Activity – an activity that quickly gathers evidence of knowledge of and use of a service Example of how to use it: Environment
26
If the aim of your project is to reduce the carbon foot print of you community then you might use this performance measure. Performance Measure - Community members are aware of more ways to reduce their environmental impact and use more of them. Time: 10 minutes You will need sticky dots and a big grid drawn up with a list of services, behaviours and actions that can help reduce carbon footprints.
Why do we like it? It is speedy and straight forward, it anonymises responses but you can track gender, age or other characteristics if you want to. How does it work? Create you grid. Compile a list of services, behaviours and actions that can help reduce carbon footprints that are available in your area, this might include energy efficient appliances, home insulation, solar panels, using public transport, local food initiatives. Write this list on one side, create two columns one headed 'I know about this' the other 'I do this / have this'. Give your group sticky dots, if you want to analyse based on gender, age, or any other characteristic you can give different people different colour dots for example under 18 yellow dots, 18 to 30 blue dots, 30 to 50 green dots and 50+ red dots. Ask them to put their dots in the appropriate square to indicate which services etc they know about and which they use. Measuring Count the dots, this works really well if you want to measure a change it is a very simple activity to do before and after an initiative.
How to adapt it? You can ask any questions you want and list any factors you want. You could add the option 'I don't know about this but would like to find out more' Why it works Participants can chose the level of their participation, there is no stigma attached to options as you have to look pretty closely to spot where someone puts there dot.
27
Equalizer Information Gathering Activity – a brilliant visual way of getting your group to think about before and after Example of how to use it: Crime / Community Safety / Perception of the area If an intended outcome of your service is to ensure that crime and perception of crime are low and that people do not move out of the area then you might use these performance measures. Performance measure - Residents report higher satisfaction levels with the area Time:10 minutes You will need copies of the equalizer hand out, which you will find in the resources section or a large version drawn up.
Why do we like it? Easy to use in a drop-in setting and also to adapt to workshop setting, it looks good.
How does it work?
Write up a copy of the equalizer volume chart on a large piece of wallpaper with the questions you wish to ask.
Give your participant some post its. Get them to place one post it on the volume scale to show where they feel the answer is to this question at the moment.
Now ask the participant to agree where they would like post it to be.
Repeat this for each question.
Now get them to write their name on a card and take a photo of the answers for this person with the name in the picture.
Questions you might use; 1. There is a problem with anti social behaviour in my area 2. I am happy living here 3. I am proud to be from here 4. There is a good level of community spirit in the area
Measuring This activity is very straight forward to measure and gives you a clear distance travelled response.
How to adapt it? This activity can be done on A4 sheets using sticky dots instead of post its. It's still nice to use the wall paper version to share between each other what the general consensus is, but the A4 sheets can be kept to record the individuals answers. 28
At the end of project if the measure is not at the level the group hoped t would be you can ask why and incorporate responses into future planning. Why it works It is a very engaging image; it is easy to record and straight forward to measure.
Hot Spots & Grot Spots Information Gathering Activity – an activity that collects view about geographical areas Example of how to use it: Crime / Community Safety / Perception of the area If an intended outcome of your service is to ensure that crime and perception of crime are low and that people do not move out of the area then you might use these performance measures. Performance measure - Residents identify more hot spots than grot spot in their area Time: 20 minutes You will need a clear map of the area you want to assess the impact of your work on, dots, flags or post it notes.
Why do we like it? People engage well with maps and enjoy finding places they know, it doesn’t rely on strong literacy skills. It can inform future planning as well as assessing impact. This can be built up over time you don't need everyone you want to consult to contribute to the map at the same time How does it work?
Ask the group to mark on the map, using the dots, flags or post it notes (these can be colour coded to collect several pieces of information at a time) areas that they like, don't like, consider safe or unsafe.
You can ask very specific questions such as 'where dog fouling is a problem', 'pleasant outdoor spaces for families' or general topics as above according to what you want to measure.
Measuring Count the number and ratio of safe / unsafe places, note where perceptions have changed.
How to adapt it? If you are using this as an end only evaluation then you can ask the group to flag areas that have 29
improved. If you are able to use the information ask why and what the group would do to make things better.
Why it works It roots your questions clearly in the physical area, it is very practical and hands on, it allows a range of opinions, different people might consider the same are to be safe and unsafe.
Values Line Information Gathering Activity – an activity that get people moving around to express their opinions Example of how to use it: Engagement If the intended outcome of your service is to ensure that a range of community members participate in a range of activities then you might use these performance measures. Performance measure - Community member report that they feel more involved in their community Time: 10 minutes You will need a line of tape along the floor, we like masking tape.
Why do we like it? People get to move around, it allows a range of views, it should be non-confrontational How does it work?
Explain that the line across the floor represents a continuum of views. One end represents a particular view and the other the opposing views, any point along the line can be used to show a degree of opinion.
Explain that one end of the line represents the view point 'I am very involved in my community' and that the other end represents 'I am not involved in my community at all'
Ask the group to stand on the line to show their view. You can record this on paper or by taking a photo.
Measuring Complete this before and after a project and measure how much each person has moved along the line.
How to adapt it? You can use string rather than a line on the floor, this can be more appropriate for people with physical or sensory impairments. You can draw the line on paper and ask people to mark their 30
position. If you are only able to measure at the end of the process then you can ask people how much more involved in their community they now feel. Why it works Getting people moving keeps their brains awake, people can chose and justify their own point on the line rather than fitting with your pre- set values.
Grid Information Gathering Activity – an activity that lets you measure two values at once. Example of how to use it: Empowerment If the intended outcome of your service is to ensure that community members make decisions at every level in and about their community then you might use this performance measure. Performance measure – Community members are able to identify a wider range of ways they can impact on change and report that they find it easier to do so.
Time:20 minutes You will need small squares of paper, two crossing lines making a cross on large paper
Why do we like it? It allows you to measure two factors; the group have a lot of control of the values attributed. How does it work?
Ask the group to think of as many decisions they would like to be able to make about their community and area they live in. These can be things they already have a say n and things they would like to have a say in. Record these, one per small square of paper.
Introduce the grid, the grid works like a graph with two axis.
The vertical line measures whether people know how to have a say about an issues and the horizontal line measures how easy they think it is to have a say.
Take each subject and find a consensus within the group as to where it should be placed on the graph. I know how to have a say about this
31 I don't know how to have a say about this
It is difficult to have a say about this
Who is our councillor
Making people clear up dog mess
It is easy to have a say about this
In the example above 'who ss our councillor' is easy to have a say in the group know how to have a say in it, whereas 'making people clear up dog mess' is both difficult to have a say in and the group are not sure how to have a say about it.
Measuring Note how many subjects are suggested by the group, the number of areas they want to have a say in and feel they do have a say in is a good measure of their level of empowerment. An empowered group will have a high number of subjects in the top right corner of the graph. Note how many move there at the end of your project if you are able to do before and after measurements.
How to adapt it? Mark the grid out on the floor and do a giant version, use a4 paper for personal versions. Use any factors you want to measure.
Why it works The grid sorts information and gives clear picture of where your group is at, the debates about where to place issues can be useful to assess their level of empowerment and knowledge.
World's Best Information Gathering Activity – an activity that allows the group to imagine an ideal scenario Example of how to use it: Perception of Change If the intended outcome of your project is to improve the communities perception of the area Performance measure – When measured against an ideal the positive perception of the area increases Time: 15 minutes
32
You will need some paper
Why do we like it? It allows imaginative input but then measure real change. How does it work?
Ask the group to imagine the world's best version of the area or service. Encourage them to draw or write a description of it.
Look at the different version and together draw out the common themes, even if there are unrealistic ideas in there, there should be some core ideas that are relevant and shared, For example the group may describe luxurious facilities and excellent weather, the core idea might be pleasant, usable outside spaces.
Take these core ideas and put them on an Equalizer sheet (Page ??) ask the group to rate how they fell they are now and how the want them to be.
At the end of the project, measure again.
Measuring Drawing out the common core themes is part of you measurements plus the measurements described in the Equalizer activity.
How to adapt it? You can ask the group to create a 'world's worst' version first to filter negative opinions and perceptions.
Why it works It can be hard to measure an area or service if you have never lived anywhere else or experienced similar services, this activity allows the group to create some ideals and pin down what they would like to see. By exploring the extremes you can find a realistic level.
Stars Information Gathering Activity – an activity that gives individuals a chance to reflect on their soft skills Example of how to use it: Soft Skills If the intended outcome of your service is to improve community members soft skills then you might use these performance measures. Performance measure – Community members rate their soft skills higher than they did at the beginning of the process
33
Time: 20 minutes You will need stars sheets
Why do we like it? The groups are able to reflect, record and measure any increase in their soft skills. How does it work?
Ask the group to suggest skills and attributes that might be used in the project or work you are doing with them. This might include communication skills, confidence, assertiveness, decision making, planning skills, negotiation.
Then ask them to write them on the starts sheet, you can find an example in the resources section. They should then rate themselves by colouring the number of stars, one for 'I am just starting to develop this', all five for 'I do this really well'.
Measuring Compare a before and after recording to assess distance travelled. A note of caution – sometimes at the beginning of projects bravado or lack of knowledge can result in high levels of self assessment and then a build up of trust and better understanding of the skills can make the second recording lower. If you think this might b the case with your group you will need to find another way of measuring. You could ask them to rate how their understanding of the categories has improved during the process and ask them to rate the trust within the group.
How to adapt it? Use the skills and attributes you plan to develop rather than asking the group. Ask the group to work in pairs and rate each other, it can be hard to evaluate yourself in this way.
Why it works The group have ownership over the process; simple questions can reveal a lot about how you project has impacted on the group.
Jellyfish Ending Activity – An activity that makes people laugh and gives a speedy end to the session Time: 5minutes You will need room for everyone to move around
34
Why do we like it? It's fun, a bit silly and very physical, the group end up back in a circle and you can thank them and say good bye. How does it work?
Get the group to stand in a circle. Ask them to orientate themselves by checking the floor beneath them and ceiling above them and noting reference points.
Tell them to pick a person in the room without giving any indication of who they have chosen and put their hands on their own heads once they have done so.
Explain that when you shout go they have to get to the person they have selected circle them three times as quickly as possible and then return to their own original position.
Shout go.
There will be a few minutes of chaos and people will gradually end up back where they started.
Thank them and say goodbye.
Dot Target and Graffiti Wall Ending activity – an activities that gives people a final chance to have a say and evaluate the session. Time: 10 minutes You will need pen paper and sticky dots
Why do we like it? It gathers final views and gives you a clear idea of how successful the session was for the group. How does it work?
On a large sheet of paper draw concentric circles split by a cross, like a simple dart board or target. The cross should spilt the target in to four sections, label these section with the questions you want to ask. For example 'I enjoyed the session'. The session was useful', 'I could say what I thought'
Then explain to the group the centre of the target represents 'yes' and the outside represents 'no' , they can use the space in between to signify different degrees. Give each participant four dots and ask them to place them on the target to indicate their views.
At the same time split another sheet of paper into three sections labelled 'I enjoyed...' I 35
would have changed....' and 'Anything else...''. Ask the group to add their comments under those headings.
Closing Circle Ending activity Closing circle – an activity gives anyone who wants to a chance to comment on how the session has gone Time: 10 -15minutes You will need a bean bag, ball or small soft toy.
Why do we like it? It ends the session with quiet reflection, giving everyone a chance to address the whole group if they want to. How does it work?
Explain that this is a chance for anyone to say any final thoughts about the session and the subjects you have been talking about, to speak you have to be holding the ball or toy, explain that anyone can speak but no-one has to.
Begin with a positive comment reflecting on the session and praising the group for their effort and then offer the ball or toy out.
Avoid handing it to the person next to you as this can build the expectation that it has to be passed sequentially around the circle.
Once everyone who wants to speak has done so take the ball or toy back, thank the group and end the session.
36
PART SIX: COLLECT EVIDENCE Once you start gathering information together, you can find you have so much, you do not know here to start. It is a good idea to get organised early to make sure information does not get lost and you do not get confused. A few top tips...
Choose one person to collect the information.
Be clear about what information gets stored and where. If you keep personal information, make sure you out a data protection policy in place.
Record details of each activity. Most things are likely to be entered onto a PC in either a word document, Excel or database.
Name it and file it clearly. It is a good idea to think about how someone would find it if you were not available to ask.
It is helpful to think about how you are going to use the information and the best way to present it.
If you are collecting information from lots of groups, try to use a simple template into which you can type the information for each group. For example use common headings. Write up your notes as soon as possible after each session.
Collect evidence in different ways. Consider taking digital photos of notes and flip charts to record what people said. If suitable, use IT, such as SurveyMonkey.
Who was involved? It is easy to forget the basics when starting out. It is a good idea to keep a record of all the important information from the start for all your activities. This will mean you will be able to give a general picture of all the work you have done. This may include sorting by event, by place, by individual, by group, by the time and date. Sort your information in ways useful for you . To do this you will need to make sure you collect information such as:
How many people were asked? How many were male and female? How many were in each what age group? Where they were from (e.g. their post code, street, village or town). Where they a member of a certain group? For example, where disadvantaged or minority 37
groups represented? Times and dates
You can choose one of the following methods to record info for each of the tools Collect in each
Record on your
based tools
groups piece of
own master
recorder
items
work
sheet
Dot voting
Equaliser
Value lines
Grid
Worlds best
Stars
Tools
Have a
Take
scribe record
digital
voice
on flipchart
photos
Bricks in the wall Gingerbread man Close to the goal
Hot spots and grot spots
Use a digital Collect in paper
What do you with your results? When you have finished gathering all the information you will need to gather it and present it so it is meaningful. Managing numbers: If you gathered information from lots of groups you will need to add your number together to get a total. It is also a good idea to start to represent your results in percentages so you can easily compare your results before and after.
38
JARGON BOX Qualitative & quantitative data A simple way to understand quantitative data is to know the information it gives you is numerical (numbers), and qualitative is when the information is in words.
You can use different tools to change some qualitative information into quantitative data. How? You might use the outcome star for people to record if they feel they are better at managing money. If 50 people used the outcome star, and 40 reported they were better at managing money, you can say, 40 or 80% of people you worked with reported they were better at managing money. That sounds like a great outcome! Managing words: If the information collected is in words then group the ideas into themes to make it easier to get a clear picture of it. Some people suggest you summarise the results of the information on one page, so you can easily see it. You can also enter it into a web-based tool to pick up some common themes. For example, Wordle is great for generating ‘word clouds’ from text that you provide. The clouds give greater prominence to words that appear more frequently in the source. Digital media: If you use digital photos you still need to work out how these results can be collected and represented to tell a story. If you use digital recorders you need to scribe it. If you want to find out more about visit: Evaluating Community Projects from the Community Tool Box .
The Community Tool Box
39
PART SEVEN: Use the Information 1.
Reviewing what you've found out and suggest improvements
Once you’ve collected information, you should bring everyone together to review the findings. Some critical questions you might want to ask include. Outcomes:
Are the outcomes what you expected them to be?
If they are different, what caused the difference?
Have there been any unintended outcomes?
Participants:
Did everybody get to participate who wanted to? What helped or prevented them?
Are some stakeholders under-represented?
Information collection:
Did you get information on all the performance measures you wanted to?
Are there any gaps in data?
What worked well what did not work so well?
What do you need to do to improve your impact in the future? If the process reveals you are failing to make the impact you wanted to make. Think about why you didn't succeed in some areas and what you could have done differently.
If it is the end of your project then you now need to decide what new project you need to do in future that will achieve the impact you had hoped to achieve.
If your project has come to an end you can also think about how you can pass on these lessons to other similar projects.
If your project is ongoing, you can make recommendations of actions for your project to make this impact happen.
You may find it useful to think of using this information in a cycle of ‘Plan - Do - Review’. So you first plan your project, which includes planning all the information you need to collect and how to do it. This stage is then followed by delivering your project, and this will include collecting and sorting all your monitoring information. The final stage in the cycle is to Review, when you look at all the information you have. This may include attendance numbers, soft outcomes as well as thinking about what went well and what didn’t. You can then use all this information to work with everyone involved to plan the next phase of the project and start the cycle again.
40
PART EIGHT: SOFT OUTCOMES AND DISTANCE TRAVELLED The Outcomes Star, Richter Scale and The Warwick-Edinburgh Mental Wellbeing Scale (WEMWBS) are excellent ways to measure soft outcomes and distance travelled. 1. Outcomes Star This is a visual, graph or map in the shape of a many pointed star. Each point of the star represents a different scale and these are used to measure the change when working with people. There are a number of different versions of the star dpending on the people you are working with and what your project aims to achieve. There are Stars to measure outcomes in:
Community development
Worklessness
Youth work
Mental health
Wellbeing Visit: http://www.staronline.org.uk/
41
2. Richter Scale This is a user friendly tool based around a set of sliding scales which are designed to measure soft indicators and distance travelled. The Rickter Scale速 is a simple motivational assessment designed to measure soft indicators and distance travelled. The tool itself is an A4 size hand-held board with ten headings down the left hand side and a magnetic slider for each heading. This slider can be moved along a scale of 0 to 10, enabling the user to scale how they feel about each topic. The process is empowering for the individual, encouraging them to see the 'bigger picture' and realise how different aspects of their life impact on each other. With the interviewer, the individual can explore possibilities, make informed choices and take responsibility for their own goals. A larger group might want to follow the Richter Scale and purchase the resources, or a smaller group or project might want to sue the general idea to develop your own tool for the people you work with. Visit: http://www.rickterscale.com/
42
3. The Warwick-Edinburgh Mental Wellbeing Scale (WEMWBS) Newly developed scale for assessing positive mental health (mental well-being). A 14 positively worded item scale with five response categories. It covers most aspects of positive mental health (positive thoughts and feelings). You use the charge as a baseline and then at regular intervals to assess distance travelled.
STATEMENTS I’ve been feeling optimistic about the future I’ve been feeling useful I’ve been feeling relaxed I’ve been feeling interested in other people I’ve had energy to spare I’ve been dealing with problems well I’ve been thinking clearly I’ve been feeling good about myself I’ve been feeling close to other people I’ve been feeling confident I’ve been able to make up my own mind about things I’ve been feeling loved I’ve been interested in new things I’ve been feeling cheerful
None of the time
Rarely
Some of the time
Often
All of the time
1
2
3
4
5
1
2
3
4
5
1
2
3
4
5
1
2
3
4
5
1
2
3
4
5
1
2
3
4
5
1
2
3
4
5
1
2
3
4
5
1
2
3
4
5
1
2
3
4
5
1
2
3
4
5
1
2
3
4
5
1
2
3
4
5
1
2
3
4
5
43
PART NINE: RESULTS BASED ACCOUNTABILITY OR RBA RBA is an outcomes framework for tackling strategic or population outcomes to address the wellbeing of a whole population. It is best used where the strategic outcomes are agreed, for example, across a county borough. This is followed by a process where different services or projects are then sought to contribute to meeting the outcome. The project would work with a customer population, and the results described as outputs (not outcomes). It has a very simple and effective way of looking at what a project may deliver, looking at the effort (how much you do), and the effect (the quality of what you do). R.B.A. comes from an American book called “Trying Hard is Not Good Enough” by Mark Friedman. Below are some of the key terms in R.B.A. Result is the outcome you want to achieve. Accountability is about who is responsible and what they did to make something happen. Indicators are a way to measure the result is being achieved. R.B.A. suggests having 3 to 5 indicators for each result. Baselines are where you are at the start of your work for each of your chosen indicators. A baseline chart shows how you are doing versus your indicators. Turning the Curve refers to the point where things start to change for the better. Turning the Curve is the point on a graph where you stop going in one direction and start going in another. Strategies/ Action Plans are the methods, plans and actions that will achieve your results. Performance Measure’s show how you measure the impact of your project or activities. It looks at what you did, how you did it and whether anyone else was better off. R.B.A. suggests having 3 to 5 performance measures per outcome. The basic principle of R.B.A.: Start with the end result and work backwards step by step to the means. Establish what the problem is before you decide what to do. Remember it’s all about results, not how you do it Think about the needs before the solutions. Other principles of R.B.A.: R.B.A. is about how to plan in a way that takes people from talk to action quickly. It aims to use simple language, common sense and little paper. Every aspect of what you do should be useful. Think about ways of working that cost little or nothing, they can be the most effective aspects of work with communities. This is a free resources, visit: http://www.raguide.org/
44
PART TEN: SOCIAL RETURN ON INVESTMENT Social Return on Investment is a way of assessing the financial value of a project. It calculates the financial value of the inputs against the outcomes. Social outcomes can often be ignored as it is difficult to assess or compare their impact. The Social Return on Investment is a way to put a financial value on social outcomes in a consistent way. It is often used by social enterprises. The key stages of Social Return on Investment (S.R.O.I.) are: 1. Identify the key stakeholders 2. Map outcomes. This involves producing an impact map which shows the relationship between inputs, outputs and outcomes. 3. Evidence outcomes and giving them a value. This stage involves finding data to show whether outcomes have happened and then valuing them. 4. Establishing impact. This looks at those aspects of change that would have happened anyway or are a result of other factors are eliminated from consideration. 5. Calculating the S.R.O.I. This stage involves adding up all the benefits, subtracting any negatives and comparing the result to the investment. 6. Report, use and embed. This model is underpinned by 7 key principles, which include:
Involve stakeholders Understand what changes Value the things that matter Only include what is material Do not over-claim Be transparent Verify the result
The model is available for use and can be downloaded from www.thesroinetwork.org.
45
PART ELEVEN: HELP AND INFORMATION
Interlink RCT - Contact: 01443 846200 or visit www.interlinkrct.org.uk
Dynamix - Contact: 01792 466231 or 0845 4786395 or visit www.dynamix.ltd.uk
The Community Toolbox - Contact: 01792 466231 or 0845 4786395 or visithttp://ctb.ku.edu/en/default.aspx
The Big Lottery Fund http://www.biglotteryfund.org.uk/sitecore/content/Home/Funding/Funding%20guidance/Ap plying%20for%20funding
Commission Accomplished - www.commissionaccomplished.co.uk This is the website developed for third and public sector organisations in Merthyr Tydfil and Rhondda Cynon Taff. It is a useful source of information. It provides extensive advice about developing a project.
46
Glossary Accountability: Who is responsible and what they did to make something happen Activities: The tasks, actions or services that take place in your project to achieve its outcomes Aims: The overall change you are trying to achieve. Attribution: Working out if anyone else contributed to the achievement of the outcomes Baselines: Where you are at the start of your work
Big Lottery Outcomes Approach: The way the Big Lottery expects the organisations it funds to plan, monitor and evaluate their work (useful even if you are not being funded by the Big Lottery) Commissioning: The process of specifying, securing and monitoring services to meet needs Confidentiality: Keeping things private and not sharing information about people without making it anonymous or getting their permission first, unless someone is at risk of harm Data: The information you have gathered Data analysis: Understanding the information you have gathered Deadweight: Working out if changes would have happened without your intervention Distance travelled: The difference between where you started and where you end up Inclusive: an inclusive process ensures that everyone can be involved in a way that is safe, comfortable and meets their needs Indicator: How you measure that the result is being achieved Jargon: Words that are only used in certain circumstances and lots of people won’t know what they mean Measuring: Gathering information on how your project is going Meaningful participation: When the people who are affected by something are able to have a significant say in how it is organised and what happens Monitoring: Checking how you are doing regularly over the course of your project Need: The reason your project exists Outcomes: The specific changes that result from your project Outcomes Process: The process by which you decide what difference you expect your project to make and how you will be able to tell if it does so Outcomes Star: A tool that you can use to measure changes
47
Outputs: What you do, the activities you deliver in your project Participative: A process that support people to join in, have a voice and really get involved in what is going on Participant: A person who is taking part in a process, project or event Performance Accountability: Planning how you measure the impact of your project, service or activity Performance Measure: How you measure the impact of your project or activities Pilot: A practice run
Population Accountability: Planning what you will do and how you can check if it has worked Quality: How good your services are, how well you have reached your target and what percentage of those you were trying to reach did you reach Qualitative data: Information gathered in words. Quantity: how much you've done, how much effect you've had Quantitative data: Information gathered in numbers Result: What you want to achieve Results Based Accountability: A model that can help you measure outcomes Richter Scale: A tool that you can use to measure changes Tracking Progress: Measuring the extent of change that is happening Social Return on Investment: A model that help you measure outcomes
Stakeholders: All the different people who have an interest, a say or might be affected by the work you do Strategies/ Action Plans: Methods, plans and actions that will achieve your results. Triangulation: Getting more than one person's perspective on a subject Turning the Curve: The point where things start to change for the better
48
Resources Gingerbread outline
49
Goal image
50
Equaliser
51
Stars Sheet
52