Impact Measuring
ToolKit
Impact Measuring
ToolKit
© The National Coaching Foundation, 2012 This resource is copyright under the Berne Convention. All rights are reserved. Apart from any fair dealing for the purposes of private study, criticism or review, as permitted under the Copyright, Designs and Patents Act 1988, no part of this publication can be reproduced, stored in a retrieval system, or transmitted in any form or by any means, electronic, electrical, chemical, mechanical, optical, photocopying, recording or otherwise, without the prior written permission of the copyright owner. Enquiries should be addressed to Coachwise Ltd. sports coach UK is the brand name of The National Coaching Foundation and has been such since April 2001. Author Jo Schultz Technical Editors John McIlroy and Vicki Sutton Coachwise editorial and design team Craig Smith and Carl Heath Cover photos © Alan Edwards and Mark Bullimore. Inner photos © Alan Edwards unless otherwise stated.
Published on behalf of sports coach UK by
sports coach UK Chelsea Close Off Amberley Road Armley Leeds LS12 4HP
Coachwise Ltd Chelsea Close Off Amberley Road Armley Leeds LS12 4HP
Tel: 0113-274 4802 Fax: 0113-231 9606 Email: coaching@sportscoachuk.org Website: www.sportscoachuk.org
Tel: 0113-231 1310 Fax: 0113-231 9606 Email: enquiries@coachwise.ltd.uk Website: www.coachwise.ltd.uk
Throughout this resource, the pronouns he, she, him, her and so on are interchangeable and intended to be inclusive of both males and females. The term parent includes carers, guardians and other next of kin categories. To enquire about accessing alternative formats of this resource, contact sports coach UK on 0113-274 4802 or email coaching@sportscoachuk.org sports coach UK will ensure that it has professional and ethical values and that all its practices are inclusive and equitable. 90789:16
Impact Measuring Tool Kit
Contents Introduction
1
Where to Start – The Things to Consider
3
-
7
The project plan
Tools, Guidance and Examples
10
Working with Case Studies
11
-
12 12 13 14 15 16 17 17
Some things to remember with case studies How should you use this section? Promotional case study Best practice case study Process case study Impact case study A portfolio approach to impact case studies Output mechanisms
Tracking Top Tips
18
Survey Design Top Tips
20
-
20 20 20 21 23 23 23 24 26
Clearly define your aims Decide on your method Questionnaire content Question types Question wording Question order Additional tips Example workshop evaluation survey Example survey questions
Discussion/Interview Guide Top Tips
29
Producing a Video Case Study (VCS) Top Tips
31
Data Analysis
32
-
32 32 33
Qualitative analysis Quantitative analysis Using the qual to support the quant
Good Practice Case Study – Recruit into Coaching
35
Recruit into Coaching Tracking Study Discussion Guide
38
-
38
First interview
Recruit into Coaching Good Practice Evaluation Case Study Interview
41
-
41 41 41 41
Start The programme The impacts Final questions
Impact Measuring Tool Kit
1
Section 1
Introduction As part of the £1.96million investment from Sport England into the delivery of coaching, the 49 county sports partnerships (CSPs) have been tasked with providing information by March 2013 that starts to show the impact of coaching on the governing body participation and talent agendas. This is an important year for the local Coaching System Support Network (CSSN) across England and there is real opportunity to show funders and government that coaching is key in driving up participation and supporting talent development. To this end there is a need to demonstrate the value of the local CSSN through a range of impact measurements. As part of the guidance provided by Sport England it has been recommended that each CSP provides some/all of the following to demonstrate end-of-year impact against the delivery of the eight coaching objectives*: •
Impact studies (two or three)
•
Coach tracking studies
•
Good practice case studies
•
End-of-year impact report.
* Each CSP will have detailed what they will be providing at year end within their 2012–2013 Coaching Delivery Plan. The skill set and expertise needed to understand and implement a series of impact measurements is relatively specialised. This tool kit has been developed in recognition of this point and is to be used to support the CSPs to feel comfortable in planning for impact measurement in the short, medium and long term.
2
Impact Measuring Tool Kit
Impact Measuring Tool Kit
3
Section 2
Where to Start – The Things to Consider This section is designed to take you step by step through all the key things you will need to consider when planning your impact study. In taking time to consider the 10 points outlined over the next few pages, you will be on your way to developing good quality reports to demonstrate the positive impact the work you carry out has on coaches and their participants. At the end of this section you will find a template Project Plan that has been designed to help you think through all 10 points. Please feel free to copy this document to help you structure your planning. 1
Choose the projects you want to evaluate
Sport England has requested you produce two or three impact studies, so think carefully about which projects/programmes you have that you will be able to collect information on, and from which people. 2
Identify the aims and objectives of the projects
•
What are they trying to achieve?
•
What do you want to see as a result of the project?
Once you’ve chosen your projects, for each one you should begin by defining the ‘problem statement’. In other words, why has the project been created and what is it trying to achieve? Being clear on this from the outset will help you to identify what success will look like and what it is that you need to measure to demonstrate this success. It is often helpful to return to these when
you are writing your questions for a survey or interview – they will keep you on track and prevent you from going off on a tangent and asking questions for the sake of it that won’t actually help answer the question. 3
What do the projects look like?
•
What do they involve?
Setting out clearly what the project consists of will help you to think about all the different activities and elements you could evaluate and collect information/data on. 4
Who does the project involve?
•
Who are the coaches and the participants?
This may seem obvious, but identifying who is involved (ie the coaches and the people/groups they coach) will ensure you include all the relevant people in your study. Remember, the more people you can include in a survey the better!
4
2
Impact Measuring Tool Kit
5
What data will you need to demonstrate success?
•
Quantitative: numbers; percentages; statistics (eg attendance figures; satisfaction ratings; retention numbers; pass-rate percentages)
•
Qualitative: thoughts; opinions and attitudes.
In order to produce a good quality impact study it is recommended you collect both quantitative (figures and numbers) and qualitative (interviews, quotes etc) data to add more detail and provide further understanding to your results.
Large/whole sample
Smaller subsection of sample
In general, quantitative research involves as large a sample as possible. This is to make sure you get data that is representative and produces results that are valid. In contrast, the qualitative research is used to get more detail from a small subsection of the larger sample. The qualitative data collection also tends to be more time consuming and so, unless you are starting with a very small sample of people, you wouldn’t realistically be able to consult all individuals via interview or focus groups.
Quantitative (survey)
Qualitative (interviews, quotes etc)
Impact Measuring Tool Kit
When thinking about the quantitative data you will be collecting, think about what you might need to act as your baseline. In order to demonstrate that things have changed (eg coaches feel more confident to deliver sessions as a result of your project, or there has been an increase in the number of coaches going on continuous professional development [CPD] courses), you will need at least two sets of data to compare. Having already identified your project aims and expected outcomes you will know what changes you want to see and so this should make it easier for you to plan the data you will need to compare at the start and end of the project.
2
In terms of qualitative data, case studies are a good way to present information you may have collected via interviews, focus groups etc. However, less formal methods such as videos and photographs can help to deliver important messages and need not be costly. 6
How are you going to collect the data?
•
Surveys, registers, online forums, databases, interviews, email feedback, focus group discussions, anecdotal comments
•
How will you contact/get hold of contacts?
Once you know what data you will need to demonstrate success, take some time to think through how you will go about getting the data. Do you have registers or databases you can use and/or adapt? Can you set up an online survey – Survey Monkey is free to use. If you plan to do online surveys consider whether you have the coaches’ email addresses and if not, how you will go about getting them. You could also set up online forums to initiate discussions and get feedback. Don’t be put off by the concept of a focus group. They are not as scary as they might sound. It’s simply about getting a group of your coaches, or indeed their participants, together in a room and having a discussion led and directed by a discussion guide (information on how to design a discussion guide is included later on in this document). Equally, interviews with individuals do not need to be particularly long or formal. Having an informal chat over a coffee is sometimes the best way to get people to relax and feel confident to chat to you about their experiences. This form of data collection is just as valid as structured interviews; however, don’t forget to explain to the person why you are having the chat, and get their permission to use their quotes and feedback in your impact study and to record the conversation if you plan to (see Discussion/Interview Guide Top Tips on page 29).
5
7
Which other organisations may have data you can use?
•
Governing bodies of sport, local authorities, local health authorities, schools, leisure centres, private clubs.
Depending on your projects there may be other organisations involved whose data you can access; for example, registers and databases. Using other sources of data is no less valid than using data you have collected yourself and can add value while also saving you time and resources. 8
Timings
•
At what points will you collect the data?
You will be expected to report back to Sport England on your impact studies at the end of March 2013 and so you will need to plan your data collection process as early as possible. This will ensure you get all the information you need and still have sufficient time to look at what it is telling you, and put together a good quality report. There is nothing more frustrating than trying to pull together evidence retrospectively and wishing you had collected certain information months ago, so think it through now and plan it in.
6
Impact Measuring Tool Kit
2
9
Consider any costs and resource implications
•
Will you require any additional resources to collect this data?
•
Do you have any links to universities/higher education institutions (HEIs) who may have capacity to help?
While you are writing your plan you will need to consider who will carry out the tasks of data collection and report writing, and how much time this is likely to take. You will need to make sure your plan is realistic and within your means. If you feel you would benefit from additional support then think about any relationship you may have with universities or educational institutions who may have
an interest in getting involved. Or indeed this might be the perfect opportunity to form new relationships. 10 Reporting •
How will you report back all your data?
When planning how you will report back to Sport England, consider all the available options to supplement your written document. For example, graphs and charts for quantitative data, quotes for qualitative feedback. Photographs and video clips are just a few examples of how you could present your information. Shown below is an example of what one of your charts may look like:
Completed CPD course Satisfied with course content Instigated further learning Felt they had learnt something new 0%
“
Going on the development course and attending the three workshops has given me so much new knowledge and skills, that I now feel more confident in going out there and delivering sessions to my groups. As a result, I am planning on running more sessions over the coming months.
“
Comment from a coach
100%
Impact Measuring Tool Kit
7
The project plan Project Title
2
Aims and Objectives What does the project aim to achieve?
Project Outline What is the project and what does it involve?
Project Sample Who is involved (coaches and participants)?
8
Impact Measuring Tool Kit
The project plan (continued) Data Sources and Collection
2
What data are you going to collect and how?
Other Stakeholders/Data Holders Are there any other agencies/organisations that may have data you can make use of?
Timings and Deadlines This will help you to see what needs to be done and when. Activity
eg Collate attendance figures
Date
August 2012
Impact Measuring Tool Kit
9
The project plan (continued) Costs and Resource Implications To include any costs (financial and time) that might be incurred as a result of your impact measurement activities.
2
Reporting
Š sports coach UK
How will you present your data? (eg quotes, graphs, photos, videos, audio clips)
Impact Measuring Tool Kit
Section 3
Tools, Guidance and Examples The following sections consist of a number of tools, templates and guidance documents to help you with your information gathering. Working with Case Studies presents a number of different options. For example, providing definitions of each one and detailing what they consist of, how they can be used, their value and any resource implications.
Discussion/Interview Guide Top Tips provide guidance for conducting interviews or focus groups, including how to structure them and tips for questioning.
Tracking Top Tips includes some general advice on how to conduct an effective tracking survey by encouraging people to take part and importantly stay involved.
Data Analysis looks briefly at how to go about analysing your qualitative and quantitative data and how to bring both sets together in your reports.
Survey Design Top Tips provide guidance on putting together a survey questionnaire, taking you through the different methods, what a good survey should include, examples of different question types, and guidance on question wording and order. The Example Workshop Evaluation Survey is taken from sports coach UK’s online evaluation of the Breakfast Clubs.They provide an example of the sorts of questions you may wish to ask when evaluating a workshop and are keen to understand how delegates benefited from attending. Example Survey Questions include a selection of example questions sports coach UK has used in previous surveys.You can use any of these questions in your own surveys if you feel they would be useful and help you to collect the information you need. However, it is important to note that they are not compulsory questions to ask and indeed some may not be relevant to your projects and data collection.
Producing a Video Case Study (VCS) Top Tips present five steps to develop a video case study that captures a story worth telling. Good Practice Case Study – Recruit into Coaching (RiC) is an example of a best-practice case study sports coach UK produced in 2010 based on the RiC programme. Although it is not an example of an Impact Case Study, which may be more relevant to your reporting, it is a good example of how qualitative data can be presented alongside photographs and quotes. This is followed by the two discussion guides which were developed for the data collection process; the Recruit into Coaching Tracking Study Discussion Guide which was used when interviewing a sample of the programme recruits on a number of occasions throughout the programme, and the Recruit into Coaching Good Practice Evaluation Case Study Interview which was used when one programme recruit was chosen to feature in the case study.
© Mark Bullimore
10
Impact Measuring Tool Kit
11
Section 4
Working with Case Studies As you will see from this section, there is a wide variety of case study options to work with. The choice of case study is likely to depend on four key considerations: • What are you trying to show? • What level of resources do you have (staff, money, expertise etc)? • How much time do you have? • What level of complexity do your audience need/are prepared to accept? The Impact Case Study presented last is the one most relevant to CSPs for the purpose of reporting back to Sport England in 2013. However, you may find it useful to see the evolution of case study production and you may find them helpful for other reporting purposes.
With so many different options it is hard to generalise on what a case study should cover, but as a general rule you should try and include: •
the definition of the problem
•
how the problem was solved
•
what issues arose and how they were dealt with
•
what the results were
•
what you have learnt.
12
Impact Measuring Tool Kit
4
Some things to remember with case studies •
Make a short list of what you are trying to demonstrate with the case study. Remember to refer back to this as you develop your questions and write your case study.
•
Think about all the different sources of information you can collect from. For example: project documents, interviews, focus groups and observations.
•
Make sure you explain clearly what happened with the programme.
•
Initially you want to prove that a project has been a success. Provide the evidence for success (eg numbers and statistics or acknowledgements from participants).
•
Don’t forget to explain how and why the project was a success. If you’re interviewing someone be sure to ask lots of ‘how’ and ‘why’ questions.
•
Be prepared to add extra questions if you need further clarification on a subject.
How should you use this section? •
Have a look at the description of each case study provided in bold at the top of each page. This is the big picture. Based on your knowledge of the area, what would be the most useful or informative angle – long-term change; how to run a project; what’s happening on the ground; promoting a project?
•
Have a look at the table with each case study. In the ‘how do we get the information?’ section, do you have the resources, contacts and/or expertise to collect this information? If not, you may want to think of another case study option.
•
Look at the output options. Which would be most suitable? Make sure you also take into account what impact these might have on data collection (eg if your output will be a video make sure you video your interviews!).
Impact Measuring Tool Kit
13
Promotional case study This places an emphasis on reporting that a project exists, what it is trying to achieve or what the results are. It is descriptive rather than trying to explain how or why something worked. As such, it is much easier to put together at short notice.
What is it?
Reporting what a project is trying to do or what results it has achieved
How do we get the information?
Reading promotional material Interviews with the people involved or quotes Looking at published results
Traditional/ suggested output
Numerous options, including one-page document, magazine article, video/audio/blog
Value
Highlighting good work that is happening
Resources/implications
One-day interview and write-up.
With this approach we could write a magazine article about the initiative. We could use material already produced and possibly some quotes from those involved in the project. We might also add some data tables to show how it met the participation targets.
This would be aimed at others sports to show how an organisation went about addressing their participation targets. To discover more about the initiative, other sports would be signposted to talk to relevant people.
Š sports coach UK
Example: A Participation Initiative
4
14
Impact Measuring Tool Kit
Best practice case study With these case studies we are interested in how projects are implemented on the ground, and in particular what the most successful people are doing. We want to understand what they do and how this can be shared and copied by others.
4
What is it?
Demonstrating new approaches that have led to improvement
How do we get the information?
Interviews with people who implemented the approach and those it has been used with (eg a coach and their athletes)
Traditional/ suggested output
A 2–3-page written case study with top tips that others can use Alternatively this could be produced as a video case study with interviews or observations of what the coach is doing A third option would be to write an article for a publication, such as Coaching Edge
Value
Tips and ideas an organisation or individual can try themselves
Resources/implications
1–2 days of interviews and two days to write up For a video it depends on existing footage; if it doesn’t exist then 1–2 days filming plus editing If writing for publication in a magazine you might want to think about hiring a journalist.
Example: A Participation Initiative Identify a coach who significantly increased the number of participants attending their sessions. We attend a session and interview the coach and the participants to discover what it is they do that
works, or what it is about them that has encouraged participants. Alternatively we could video a session and interview the coach and participants, editing the video to create a short film.
Impact Measuring Tool Kit
15
Process case study These types of case studies are most useful to those developing programmes and developing coaches. They focus on how the project was planned and seek to identify good practice or problem areas that need to be addressed before implementing another project. This will not prove the impact of the project but rather what lessons have been learnt during the process.
4 What is it?
Understanding what lessons have been learnt from running a project/programme
How do we get the information?
This will tend to be based around logic models Analysis of programme documents to identify objectives and targets Interviews with people who planned the programme Interviews with those who implemented it and those who benefited from it (eg coach development manager, coach, participants)
Traditional/ suggested output
A 3–5-page written case study with recommendations on how the programme could be improved in the future This could be accompanied by some audio/visual materials such as listening to parts of an interview with a coach
Value
An organisation can benefit from specific project delivery ideas and the wider coaching network can receive generic recommendations for how to implement projects
Resources/implications
Two days of interviews and three days to write up.
Example: A Participation Initiative This approach is mainly interested in speaking with staff. How did they come up with the idea for the initiative? How easy was it to implement? We would then talk to some regional staff to see if their experience was the same as that described at national level? Was there an issue at a local level that needed local intervention? The results will most likely be a report showing what works in planning and implementing projects nationally, or highlighting the lessons learnt. Alternatively, it could show the importance of local knowledge to implement the initiative. Areas that might be covered include planning, communication, reporting, funding etc.
16
Impact Measuring Tool Kit
Impact case study This answers the big question. Has the programme led to a change in behaviour or made a lasting impact? Such impacts are long term and therefore this approach requires data collection over a longer period of time.
What is it?
The most comprehensive evaluation focusing on the long-term effects of a programme
How do we get the information?
Analysis of programme documents to identify objectives and targets Interviews with people who planned the programme and those who implemented it Longitudinal analysis of participants through surveys or interviews
Traditional/ suggested output
A report demonstrating the long-term impact of a project
Value
This will provide a detailed understanding of cause and effect in programme delivery
Resources/implications
Long-term resource intensive (see next page for ideas on shorter-term opportunities).
This could be accompanied by some audio/visual materials such as listening to parts of an interview with a coach
Example: A Participation Initiative With this approach we would want to know if the participation initiative has increased the number of participants and are they continuing to take part several years down the line? In the first instance we would speak to staff about how they came up with the idea. How easy was it to plan and implement?
We would also follow participants and coaches over an extended period of time. Has there been a long-term change in their behaviour that demonstrates the success of the project? The results would most likely be a report. For the governing body of sport this would be useful information on their ability to implement projects and how they could do this better. It would also allow them to demonstrate to the public/ funders/Board members the long-term impact of the programme.
Š sports coach UK
4
Impact Measuring Tool Kit
A portfolio approach to impact case studies Although your Impact Case Study is a longer-term project there is still plenty you can do in the short term.
Video Probably best used when the information is more tailored to a visual demonstration or to create an instant impact using images and sound. It also gives a personal feel if people are being interviewed.
•
Why not write a promotional case study at the start of the project to show what you are doing?
•
Even after the first year is there any good practice you would want to share more widely?
Example: ‘Become a Coach – Getting Involved’ on the sports coach UK website – www.sportscoachuk.org
•
Did you have any issues setting up or implementing the project? How did you overcome these and would it be valuable to share the lessons learnt with others?
Audio
Do you have any positive feedback from participants (such as videos or blogs)?
Similar to video and useful to record interviews if video footage is not available.
•
Using this approach will help keep the project in the minds of the public and funders. It will also help you at the end of the project as you have a record of what happened. When you have to demonstrate your final impact you will now have a portfolio of case studies that demonstrates good practice throughout the life cycle of the project.
Output mechanisms There are several potential outputs which these case studies could appear in. One case study could be delivered using a combination of these, as appropriate. For the purposes of your report to Sport England we would recommend you produce a written document, but fully encourage you to incorporate other formats to supplement it, in particular video and audio.
Example: Sound bite.
Article These are best used to target a specific audience depending on where the article appears. The messages are then tailored to that specific audience. Therefore a case study could have several articles about it all written from different angles to engage the different audiences. Example: Magazine or journal article.
Blog Reports Probably best used when the information is detailed, needed for reporting, or if it is likely to be given out at events/forums. Reports can be in many forms and will vary in length dependent on the complexity and audiences of the case study. Examples: Factsheets, impact documents, guides of best practice, promotional posters/flyers used at events.
17
Blogs are a more informal method of communication to get a message across, almost like telling a story but with a serious purpose. Case studies written this way are designed to engage the reader in a topic which interests them and provides key messages within the blog. Examples: Visit the blog section on the Resource Bank of the sports coach UK website – www.sportscoachuk.org
Tweet Short statement signposting audience to further information – more than likely to be a website where a full article/case study will be displayed. Examples: sports coach UK Twitter page.
4
18
Impact Measuring Tool Kit
Section 5
Tracking Top Tips When it comes to tracking a cohort of individuals, whoever they might be, there are a few things to consider and remember. 1
Different coaches may need different strategies – bear in mind that not everyone is computer literate and may not check their emails regularly. Therefore if you are planning on carrying out a survey, depending on who your coaches are, you may want to consider creating a paper version as well as an online version.
2
Incentives help maintain involvement – we all know how precious our time is and being asked to complete a survey can seem like a task we could do without. With that in mind, offering an incentive to those who take part can significantly increase your response rate.
You don’t necessarily have to offer every respondent an incentive however; doing so would inevitably become quite costly. Alternatively, offering respondents entry into a prize draw is a commonly used incentive. It goes without saying that what you can offer as an incentive will depend upon your budget and industry, but for some ideas to start with, see below: •
discounts on a CPD course
•
prize draw for an iPad/iPod/anything you have access to
•
prize draw for tickets to a sporting event of interest.
3
Sell the benefits of taking part – as we have already acknowledged, completing a survey isn’t always the top of our priority list of things to do. However, one way to increase the likelihood that an individual will complete your survey is to fully explain the purpose of the survey/project. Knowing why their involvement is important and how getting involved may benefit them in the long run (eg continuation of funding to support them and their development) is likely to significantly increase the likelihood a coach will take the time to answer your questions.
4
Updates keep people involved throughout the study – whenever you are asking people to repeatedly take part in some research, it is a good idea to keep them updated on a regular basis. This is especially important if your study is taking place over an extended period of time. It is easy for people to forget that they have taken part in your project and they may forget the previously mentioned benefits to them of remaining involved. Remember though that updates don’t necessarily need to be extensive or overly detailed. Simply letting respondents know where the project is up to, what has happened so far and the next steps would be enough. It might also be worth letting people know when you next expect to be contacting them to ask for their involvement again.
Impact Measuring Tool Kit
19
© sports coach UK
5
5
6
Make it as easy as possible – another way to encourage people to get and stay involved is to put as little demand on them as possible. Keeping your surveys and questions short and to the point is key. Don’t fall into the trap of asking question after question simply because you think it might be interesting to know the answer. Keep referring back to your project aims and objectives to stay on track and avoid asking questions that won’t provide you with the evidence you need and that tire/bore your respondents. Expect people to drop out – it is highly likely that people will drop out when you are carrying
out a piece of research over an extended period of time. Life changes and time pressures affect us all and inevitably result in some respondents dropping out of your project at some point or another. 7
When you are following a group of people over a period of time and you plan to send them a number of surveys, it can be tempting to simply send out the same survey each time and see if there has been any change in the responses given. However, it is important to revisit your original survey and check whether any questions need adapting or removing and whether you need to add any new ones.
20
Impact Measuring Tool Kit
Section 6
Survey Design Top Tips The next few pages provide some guidance on designing surveys. Although fairly straightforward, there are a number of guidelines that should be followed to ensure you are collecting information that is both useful to you and reliable. Clearly define you aims
Questionnaire content
•
What are you trying to find out?
•
Why?
•
What are you going to do with the information afterwards?
A questionnaire should have a good introduction. It is important the introduction explains the nature of the research, the topic(s) under investigation and the time needed to complete the questionnaire.
This will help you to determine the questions you will need to ask, who you will need to ask and what breakdowns/cross tabs you will be interested in analysing (eg to know whether more men than woman engage in a particular sport).
Decide on your method
Think about aims and create more detail for each one. Break the subject down into topic areas or types of information (eg facts and knowledge, opinions and attitudes, motives, past behaviours and future behaviours). Try and group questions that naturally flow together.
•
Face-to-face
•
Self-completion (postal, online etc)
•
Telephone.
Impact Measuring Tool Kit
21
Question types Lists What qualifications do you hold or are you currently working towards? (Please mark all that apply.)
6 Qualification type
Hold
Working towards
Leadership qualification Level 1 Level 2 Level 3 Level 4
Categories
Attitude
Age groups – please mark all that apply to the people you work with:
Five-point scales with a midpoint are thought to be the easiest to understand and allow respondents to express their points of view.
4–6 years
Overall, how satisfied are you with the Breakfast Club event that you attended?
7–11 years 12–16 years
Very satisfied
17–21 years
Quite satisfied
Over 21 years
Neither satisfied nor dissatisfied Quite dissatisfied
Ranking Please rank the following in terms of how important you feel they are to your role as a coach, where 1 is most important and 5 is least important. Professional development courses Mentoring Communities of practice Self-reflection Governing body of sport workshop
Very dissatisfied Don’t know
22
Impact Measuring Tool Kit
Grids (to record answers to more than one question at a time) How well supported do you feel you’ve been in relation to the following issues over the last 12 months from agencies such as your governing body of sport and sports coach UK? (Please tick one box in each row.)
6
Very well supported
Supported
Identifying your development needs Knowing the next steps for your coaching Identifying development opportunities Help with the cost of development opportunities Help with finding appropriate opportunities to coach Help with career opportunities in coaching Help with moving up the ‘coaching ladder’ Overall support
Open/verbal What suggestions do you have for improving the support available to coaches?
Supported a little
Not supported at all
Impact Measuring Tool Kit
23
6
Question wording
Question order
•
Use simple language.
•
Use audience appropriate language and communicate the meaning of questions precisely.
Having established you are using the right words and phrases and the right type of question, the next step is to establish an order in which to ask the questions and an overall layout for the questionnaire.
•
Don’t ask two questions in one (eg ‘Do you enjoy watching and playing sport?’).
•
Start with the easier questions to answer – gives respondent confidence!
•
Be specific and not ambiguous (eg ‘Do you often play sport?’ is ambiguous as people’s definition of ‘often’ is likely to be different).
•
Start with the more interesting questions – grabs interest.
•
Avoid leading questions.
•
Start with the more generic questions, getting more specific as the questionnaire progresses.
•
Do not rely upon respondents’ memories – tie a question down to a specific event/time. If a question taxes respondents’ memories, provide a list of options.
•
Personal, emotive or complicated questions should come later in the questionnaire.
•
Questions should flow in a logical order.
•
Be aware of sensitive questions (eg provide categories for age ranges).
•
Use subheadings and titles to help respondents to move from one topic to another.
•
Avoid leading questions (eg ‘Are you intending to vote for the Liberal Democrats in the next general election?’ ). This wording suggests the answer.
•
Avoid repetition and use a variety of question types.
•
Avoid hypothetical questions (eg ‘What would you do if your house burnt down?’). What people think they would do and how they would actually behave may be entirely different. Collecting info on people’s past actions is a better predictor of future behaviour.
Additional tips •
Provide a ‘don’t know’ option – Market Research Society guidelines specify this (these can be accounted for/removed during any analysis).
24
Impact Measuring Tool Kit
Example workshop evaluation survey Thank you for attending the Talent Coaches Breakfast Club.
6
So that we can continue to provide you with informative and useful sessions, we would be grateful if you could complete this short survey to provide us with your feedback on the content of the workshop and your overall thoughts and opinions. Please be advised that all responses will be kept anonymous, with names kept confidential. Therefore please be honest in your responses.
About You Name: Email: Main sport coached: Highest level of sporting qualification (eg Level 1, Level 2):
Would you be happy for us to contact you in the future in relation to Talent Development communications and your attendance at the sessions? Which Breakfast Club did you attend? Adherence – The Power of Athlete ‘Buy In’ Elite Communication Skills What is a Master Talent Coach Anyway? Assessing and Developing Physical Competence Using the ‘C System’ to Develop Excellence in Coaching Developing Talent – The Role of the Coach from the Frontline ‘What About me’ – Athlete/Player Perspectives on Talent Coaching Talent ID to Junior National Squad – The Journey Measuring and Developing Resilience and Mental Toughness Other. If other, please tell us which Breakfast Club it was and who the speaker was:
Date and venue of Breakfast Club:
Yes
No
Impact Measuring Tool Kit
25
Content Was the content and information presented current to you and your role?
Yes
No
Don’t know
Did the content meet your expectations?
Yes
No
Don’t know
Were you able to express your own views and experiences?
Yes
No
Don’t know
As a result of attending the Breakfast Club do you feel you have learnt something that you will use to inform the way you coach?
Yes
No
Don’t know
If you answered ‘Yes’ above, please tell us what this is and how you intend to use it to inform the way you coach (if you indicated that you do not mind being contacted again in the future, we may contact you to find out how you have used what you learnt):
Overall Ratings Would you recommend this deliverer for future Breakfast Clubs?
Yes
No
Don’t know
Overall, how satisfied are you with the Breakfast Club event you attended? Very satisfied Satisfied Neither satisfied nor dissatisfied Dissatisfied Very dissatisfied Don’t know. Do you have any comments you’d like to make about the Breakfast Club?
Thank you for taking the time to complete this survey.
6
26
Impact Measuring Tool Kit
Example survey questions The following are some examples of questions that sports coach UK have used in previous surveys. They are included to provide you with some examples of how to ask certain questions. It may be that some of
them have relevance to the surveys you may conduct, in which case you should feel free to copy them into your questionnaires. However, it is important to note that these are not compulsory and indeed may not be relevant to your survey.
6 Demographics and Contact Information First name: Surname: Address:
Postcode: Tel Numbers:
Home: Mobile:
Email address: Would you be happy for us to contact you in the future in relation to...and your attendance at...?
Yes
No
Coaching Careers Over the last 12 months have you typically coached in an unpaid/voluntary, paid part-time and/or paid full-time capacity?
Thinking about your paid coaching, are you self-employed, employed or both?
(Please tick all that apply.)
(Please tick all that apply.)
Unpaid (voluntary)
None of these (volunteer) Self-employed part-time
Paid part-time or paid sessional (less than 30 hours paid work per week) Paid full-time (over 30 hours paid work per week)
Self-employed full-time Employed part-time Employed full-time
Note: Coaches should not code themselves both as employed full-time and self-employed full-time.
Impact Measuring Tool Kit
27
On average, how often have you coached in the last 12 months or, if applicable, in the last season? (Please tick one.)
Almost every working day
6
At least once a week At least once a month At least once every six months At least once in the last year
On average, how many sessions have you delivered per week over the last 12 months or, if applicable, over the last season? (Enter number per week.)
On average, for how many sessions do you receive payment for per week? (Enter number.)
Do you expect to do more, about the same, or less coaching over the next 12 months? (Please tick one.)
More
About the same
Less
Do you intend to stop coaching in the next 12 months? (Please tick one.)
Yes
No
If yes, provide details why:
28
Impact Measuring Tool Kit
Coach Qualifications Do you have an up-to-date governing body of sport recognised coaching qualification? (Please note: Preliminary awards, leadership awards, CPD workshops, short courses and PE teaching qualifications, are not included as governing body of sport recognised coaching qualifications in this question.)
6
Yes
No
Please provide information on all your governing body of sport coaching qualifications. (Please enter qualifications in relation to the order you achieved them.) Awarding governing body of sport
eg Football Association
Qualification name
Approximate level Levels 1–5
eg Level 2 Certificate in Coaching Football
Workshop Feedback As a result of attending the workshop do you feel you have learnt something that you will use to inform the way you coach?
Yes
No
If ‘yes’ please tell us what this is and how you intend to use it to inform the way you coach. (If you indicated that you do not mind being contacted again in the future, we may contact you to find out how you have used what you learnt.)
Was the content and information presented current to you and your role?
Yes
No
Did the content meet your expectations?
Yes
No
Overall, how satisfied are you with the workshop you attended? Very satisfied
Quite dissatisfied
Quite satisfied
Very dissatisfied
Neither satisfied nor dissatisfied
Don’t know
Impact Measuring Tool Kit
29
Section 7
Discussion/Interview Guide Top Tips This section provides guidance on how to write you discussion or interview guide. As with survey designing, it is fairly straightforward, but there are a number of guidelines to follow that will make the process easier (for yourselves and the interviewees) and will also ensure you are collecting information that is valid and accurate. In particular the way you structure your guide is important in helping you get good quality information. You need three things: a clear introduction, clear headings, and a clearly signalled ending.
•
tell them you might make some notes during the interview
•
if recording, make sure you have their permission to do so
•
tell them how the information will be used
1
•
let them know their involvement is voluntary and they can stop the discussion at any time and do not have to answer everything
•
mention that there are no right or wrong answers, that it is not a test, and it is their experiences, feelings and opinions you are interested in.
2
Clear headings or statements about the topics that the questions will look at are useful in signposting things for you.
A clear introduction will help to give you confidence and help you to settle into the interview and begin building a rapport with the individual(s).
In the introduction you should: •
say something about yourself (name, organisation)
•
tell individuals about the reasons for the discussion or interview
•
tell individuals how long it is going to take
•
tell them about your role and that you are there to listen and collect information
For example, you can insert statements or sentences along the following lines:
•
tell them how and why they were chosen
•
The next set of questions is about...
•
give assurances about confidentiality and/or anonymity unless they have agreed to be named
•
I’d now like to ask you about...
Impact Measuring Tool Kit
3
7
A clearly signalled ending or wind down is very important in ensuring the interview does not come to an abrupt end. You should say something like: ‘We’re almost finished now. Just a few more questions’. Remember to thank individuals for giving up their time and for sharing their views and experiences with you.
The style of questions used in qualitative interviewing varies from those used in quantitative interviewing. Questions on a quantitative survey will be mostly closed, whereas questions on an interview or discussion guide are mostly open-ended, non-directive questions. These sorts of questions allow respondents to relate to the topic in their own way, to use their own language with little or no direction on how to answer. Examples of these sorts of questions include: •
‘Tell me about a typical day.’
•
‘Describe your first impressions.’
•
‘How did you find out about this?’
You might also want to use questions like those to probe, follow up or clarify – ‘Who told you?; How did they tell you?; Was someone available to talk to you afterwards?’ Probing is a way of encouraging individuals to give you more detail. Prompting is another way to encourage them to answer in more detail: techniques include repeating or rephrasing the question or using non-verbal cues – encouraging looks, nods of the head, and pauses and silence. It is very important of course to keep a balance between encouraging the individual to answer and leading him, or putting words in his mouth; for example by saying, ‘I suppose you were sorry you even asked’.
It is a good idea to record your interviews/ discussions if possible so you don’t have to write down everything they say, and you can go back and listen to it again if you want to check or clarify what was said.
© sports coach UK
30
Impact Measuring Tool Kit
31
Section 8
Producing Video Case Study (VCS) Top Tips Below are five steps to producing a video case study that captures a story worth sharing.
Content and Format
Script and Set-up
Capturing on Camera
Review and Edit
Presentation and Promotion
•
Tell the story to a colleague or friend who is unfamiliar with it and decide on the key messages/events that need to be communicated.
•
Decide on a filming format (eg interview, narrative, hosted1).
•
Identify individuals and situations that need to be captured and make the appropriate arrangements.
•
Construct a script to be followed throughout filming, have colleagues read over it and consider doing a run through with those involved if time allows.
•
Ensure filming venues are quiet and have suitable lighting. Have a contingency plan if unforeseeable factors prevent filming (ie another venue or another date).
•
An effective case study will last between 6–10 minutes.
•
Most static action can be captured easily enough on a good quality video camera, though if you want to ensure a professional presentation then it is worth hiring a professional filming and editing company.
•
Play back each section after filming to determine if you need to do another take. Don’t be nervous about shooting until you get it right – it’s a lot harder to correct film later.
•
There are numerous free programmes that allow you to edit video footage, but again if you want it to appear professional it is worth contacting a specialist.
•
Make sure you give those individuals who feature in the VCS an opportunity to see it before it is signed off.
•
Ensure that the style and effects within the video do not distract viewers from the true message.
•
Once signed off, use the VCS as a tool for promoting your organisation/ programme. How about posting it on YouTube, airing it at your local cinema, emailing the link to local press, or using it as part of a Facebook/Twitter campaign?
1
Suggested formats: Interview: Questions are asked or displayed on screen followed by an individual answering them clearly. Narrative: Story is told over images/clips of related activity or text. Hosted: An individual plays host and provides the links to interviews/clips that occur within the VCS. There are some completed video case studies on the sports coach UK YouTube channel (eg http://goo.gl/esLZl).
32
Impact Measuring Tool Kit
Section 9
Data Analysis Qualitative analysis The aim when analysing qualitative data is to identify the key themes and messages emerging that relate to the project objectives. •
The first thing you need to do is to refamiliarise yourself with all of your data – this will ensure your analysis is not biased or overly influenced by the last interview or group discussion you ran.
•
As you go through your data (notes, audio recordings, video clips etc), it is important to keep your project objectives close to hand. These will remind you what the project is aiming to achieve and will help to keep you focused on the relevant information.
•
There is no right or wrong way to go about your analysis – some people like to make notes as they read through their data, others will highlight sections to revisit, and some will do both. Whatever your preferred method, by going through this process you will produce a number of key headings under which you can assign quotes.
For example, by reading through your notes and listening to your audio recordings, you may identify a common finding that coaches feel more confident to coach their sport as a result of attending your course. You would then group all comments relating to increased confidence and pick a selection to include in your impact report. Bear in mind it is not necessary to include all the quotes that relate to a specific theme. If you only have a few then it may be relevant to include all of them, but if you have many that are more or less saying the same thing, then you only need to select a handful that best represent the point.
Quantitative analysis In social and market research, quantitative data is often analysed using statistical tests. However, for the purposes of the reports you will be producing for Sport England, you will not need to do any statistical tests. The quantitative analysis relevant to your reports will mostly likely include: •
frequency counts
•
averages
•
percentages
•
comparative figures (where two or more sets of data have been collected).
Impact Measuring Tool Kit
Where you have used scales (eg five-point satisfaction scale), you may wish to combine some of the groupings in the analysis. For example:
Strongly agree
Agree 70%
4%
Neither/nor 18%
When asked how far they agreed or disagreed that the information they had received had a positive impact upon the work they had done, three-quarters of respondents agreed to some extent.
Disagree 6%
Strongly disagree 0%
Don’t know 2%
74% agree to some extent.
present your qualitative and quantitative data separately. The purpose of qualitative data often is to add detail and provide support for the quantitative information. Therefore, where relevant, the two should appear together. For example:
When deciding how to present your data, think about how it can be done in the clearest way. Graphs can sometimes be a good alternative to tables to visually present your data. This can be especially useful when you have two sets of data to compare.
Using the qual to support the quant When you come to write your impact studies it is important to remember that you do not need to
Strongly agree
Agree 61%
0%
Neither/nor 29%
Of the 31 respondents who completed this question, two-thirds agreed to some extent that the information they had received had a positive impact on the work they do.
Disagree 76%
Strongly disagree 0%
Don’t know 3%
61% agree.
“
“
Additional comments included:
“ ““
33
It is very much like a supermarket with shelves and we are at a position where we need specialist shops now. Being new into the coaching role I found all the resources of value.
“
For general resources they are very expensive and become a barrier to people attending CPD courses.
9
34
Impact Measuring Tool Kit
Section 10
Good Practice Case Study A Coach’s Journey How do you solve a problem like disruptive behaviour in afterschool clubs? With a boxercise teaching lollipop lady! This is just one of the results of the RiC Programme in Wolverhampton. This case study looks at how Westcroft Sports and Vocational College in partnership with Westcroft School Sports Partnership (SSP) successfully used RiC funding in afterschool clubs and follows the story of one recruit, Carla. Transforming from lollipop lady to coach, Carla’s story demonstrates the positive impact RiC can have on individuals, participants and the community.
Local coaches Background Westcroft SSP covers one third of the Wolverhampton district, an urban Unitary Authority ranked the 28th most deprived local authority in England. Within the area covered by Westcroft SSP there are local areas ranked within the worst 1% for overall deprivation and for child well-being (source: Wolverhampton City Council, Window on Wolverhampton, 2009). When the SSP initially got involved with RiC they enlisted the help of a Coach Manager – who had considerable experience of training and employing coaches – to put together robust planning and delivery mechanisms for the programme for the area.
Recruiting local people into the RiC programme was seen as a priority. The SSP wanted coaches who connected with the area and were known to local people. This, it was argued, would help the RiC coach to engage with the community as both parents and children feel safe with people they know and would therefore be more willing to get involved. Carla was one such local. Her job as a lollipop lady was an obvious connection with the community and the Westcroft Partnership Development Manager noticed ‘everyone tended to be around her chatting’. Carla was initially involved in a scheme called Women Get Set Go, involving leadership and multiskills training, and had shown a definite talent that could be channelled towards coaching. For Carla, Women Get Set Go was an enjoyable experience that she felt had been very interesting and helped boost her confidence. She was therefore grateful for the chance to take this to the next level with RiC. Continued overleaf
Good Practice Case Study
Continued from previous page
Since coming on the RiC programme Carla has become a focal point in the community. Her position leads to regular contact with parents and she has used this to emphasise the importance of healthy eating and physical activity. This has had a positive influence on a number of parents, especially with weight loss and exercise. Carla has since been approached by these parents looking for more information on courses. Her progression from lollipop lady to coach is also an inspiration to others and she says to them: ‘If I can do it, so can you.’ For the children too, having a coach who they see every day in the community creates the feeling of belonging and trust that they do not get with a coach they might only see once a week.
The more sports the better... As part of their induction, the recruits were asked to provide information about themselves such as: their favourite sports; what physical activities they do; any leadership courses they had taken; and what sports/activities they wanted to get involved with. The SSP wanted recruits to take courses in sports they had an interest in and as a result there was a diverse range of Level 1 courses covering traditional sports such as football and netball, but also new activities including cheerleading and boxercise. By choosing their favourite sports the SSP contacts believed recruits would have a passion for it when they were out delivering. An additional consequence of the diversity of sports was that for many children it was these newer sports, such as cheerleading, that attracted the most interest.
Building confidence in the group Twenty recruits from the local area joined the Westcroft SSP programme in the first year. They represented a diverse range of people and interests including secretaries, dinner ladies, sixth formers and firemen. For many in the group, confidence was an issue and this was something that needed to be addressed from the outset. ‘Some of the people on the course would not have gone off and done a course on their own. We needed to build up their confidence and build a team,’ said the Coach Manager. Every other Tuesday, the recruits would meet at a local primary school and it was there they completed their First Aid and Safeguarding Children courses. For many, this was a reintroduction to a learning environment and, by doing these courses together, they were comfortable with the environment, which helped build their confidence. As a result, when the time came to do their Level 1 course, they were quite happy to proceed. In this environment the Coach Manager noticed the recruits wanted to do more because they felt part of a team, and they were keen to support each other. The range of ages and experiences also allowed people to learn new things. ‘There’s a big age gap, but I’m not alienated...we click and bond,’ said Carla.
Opportunities for recruits Recruits were involved with delivering sessions within the multiskill academies organised by the SSP. This provided practical opportunities for coaches to use their new skills linked to the sports they were qualified in.
Together with the Coach Manager, she approached schools with the offer of taking an afterschool session in boxercise. The response from girls and boys alike was amazing.
The SSP also plan to use the new coaches during National School Sport Week. This, it was argued, will increase the profile of the coaches and the RiC programme among other schools.
Although most of the 15-year-old lads were expecting an ex-boxer rather than someone like Carla, the fact they continued to attend week after week speaks volumes for the success of the sessions.
A particular success for Carla was working with both the RiC programme and Sport Unlimited in an afterschool boxercise club.
For Carla, bonding with the children and earning their respect made this an invaluable experience. Continued overleaf
Continued from previous page
In another afterschool club there was an issue with boys in the club constantly fighting. When asked why they did it the children said ‘fighting is what we do, we want to fight.’ The Coach Manager was approached by the school to see if he could come up with a programme to help which included Carla’s boxercise sessions for Years 3, 4 and 5. Again, the results were good. Everyone involved, including the boys, were so focussed on the classes that behaviour improved. Recently, requests have been made for a community based club to be set up in the area, so friends who are not at the school can get involved.
What next for recruits? Now that the recruits have finished the programme the SSP was keen to establish a pathway to further coach development opportunities. The recruits were also keen and were in touch with the SSP looking for more opportunities. As a first step, the recruits were set to act as ‘buddies’ to help develop next year’s trainees. The plan is to involve them in the induction by talking about their experiences with RiC and they might possibly run some coaching sessions with the new recruits. This will be a real life inspiration for the new recruits who can say ‘that will be me next year’. Ultimately the SSP would like to see a RiC Stage 2 to offer Sportsability training and
continue the development with local clubs by offering them a role within the programme. As the Coach Manager said of Carla: ‘The pathway is important...we need to make sure she doesn’t just stop now...we can set up programmes and continue with the training.’ The SSP have been keen to use RiC recruits to link with other programmes in the community. The Change for Life campaign had a high profile in the area and recruits have been active in promoting it. With Cycling for Life and Dance for Life soon to be promoted in the area the SSP wants to make sure they have cycling and dancing recruits coming through to seize these highprofile opportunities.
Top tips and recommendations •
Finding recruits who are well known in the local area can help promote the programme and build trust among children and parents.
•
If you’re working with a group of people who are low on confidence, building a supportive team environment and starting with courses just for that group are good ways to address this.
•
Recruits with a diverse range of sporting interests provide opportunities to capture children’s attention with something new. Not everyone is interested in the traditional school sports and activities such as boxercise and cheerleading have proved particularly attractive for those traditionally disinterested in sports.
•
Look for ways to promote the RiC programme through high-profile campaigns such as the National School Sport Week and Change for Life. Getting your coaches involved in these areas can help reach a wider audience.
•
Bringing Year 1 recruits back to act as ‘buddies’ for new trainees may have a positive benefit for both. The coaches get experience and continue to feel involved while providing an inspiration to the new trainees.
•
An experienced coach manager can be a real asset to any SSP in planning and delivering coaching programmes.
A final word... The final word about RiC in the area goes to Carla: ‘I never expected it to go this far. Just thought I was going to be left there having a certificate and little bit of experience. But to be put on RiC and go through courses and actually delivering round schools has met more than my expectations.’
38
Impact Measuring Tool Kit
Section 11
Recruit into Coaching Tracking Study Discussion Guide First interview Introduction The research has been commissioned by Sport England and Youth Sport Trust to examine the impact of the RiC programme (eg what has been working, not working, and why). Note: You may not be aware that you are an RiC candidate or coach! RiC provides opportunities for new coaches to receive training and to go on coaching placements. The fact I am talking to you means you are an RiC candidate or coach! When I ask you questions about the RiC programme, or just ‘the Programme’, please talk about your recent induction, development and placement experiences. sports coach UK’s Research Team are undertaking the evaluation. The interview will probably last between 30 minutes and an hour, but it really depends on how much you have to say. This work is confidential – no-one will be able to identify you, and we will never attribute your name to any comments we publish. We would like to record the interview/discussion so we don’t miss anything – is this OK with you?
To begin, can you explain a little bit about you and your background in sport? •
Tell me a little bit about your sporting experiences...have you played much sport? – Where? When? What level? How long? Did you enjoy it? What sport?
•
Have you any previous coaching experience? – Where? When? What sport(s)? How long? Did you enjoy it? – Who did you coach (eg children, participants, high performance athletes)?
•
Where do you live? Are you local?
•
What is it like living in this area? – What are the opportunities (sporting/coaching opportunities?) like for local people?
•
What were you doing before you joined the RiC programme? – Employment? – Education? – Life in general?
Impact Measuring Tool Kit
Can we now move on to talk about the programme in more detail? •
How did you become aware of the RiC programme? – Who told you? How did you find out? Did you see advertising? Did you find out about it through another course or programme?
•
•
– Have you had any support? •
– Coaching qualifications? – Mentoring? – Chance to observe other coaches?
– To do something different?
•
– Other people/agencies? •
– Books/DVDs?
How did you get involved in the programme?
Has anyone else been involved at the application/set-up stage?
What are you looking to achieve from the programme?
– Active Passport? For each development opportunity received, ask: •
What happened?
•
Who was involved?
•
Was it useful?
•
How did it help you?
•
Were there any problems or issues accessing these development opportunities?
Can we now talk about your involvement in the RiC programme? •
– What were the issues (eg not available when you wanted it)?
What involvement have you had with the programme to date?
– What would you improve?
– What have you done? – Who has been involved? – What has been good/bad? – What has helped/not helped? Interviewer: Many of the following prompts are included if the following issues have not been discussed. •
Did you go to an induction session? – Where was it held? – What was it like? – What information were you given? – What attracted you to the programme? – What would you improve if anything?
What development opportunities have you been offered? – Leadership awards?
Why did you decide to get involved?
– What did you have to do?
What paperwork have you had to fill in? – Was it easy/difficult?
– To become a coach?
•
39
– What would you like more/less of? •
Have you had a coaching placement as part of the RiC programme (recruits have to undertake 20 hours of volunteering to complete the programme)? – If no, why not? – If yes, where? When? How long? – What happened? – Who was involved? – Was it useful? – How did it help you?
11
Impact Measuring Tool Kit
•
11
Have you had any problems accessing coaching placements (these are usually in clubs, schools, youth centres, community projects)?
– Your club, the athletes/participants? – Your local community? – Your family and friends? •
Are there any other benefits you have experienced from being on the programme that we haven’t talked about?
•
Are there any other problems or issues you have not mentioned?
– Could they have done anything else?
•
What would you improve?
Have you had to do anything else (eg complete an Active Passport)/had any other support as part of the RiC programme?
Finally, I would like to ask what your plans are for the next 12 months?
– What happened/what are the details?
•
Plans for coaching?
•
Plans for your life?
•
What are you excited about (in relation to coaching or life)?
What support have you had from others in relation to the RiC programme (eg a mentor)? – Who has been most involved? – In what way have they helped/not helped?
•
Have the improvements you have experienced had an impact on others?
– What would you improve? – What would you like more/less of? •
•
– What were the issues?
Can I now ask you about the impact (or effect) of the RiC programme since you started? •
In what ways has the programme helped you?
•
What have been the main problems with what you have been offered?
•
What are you nervous about (in relation to coaching or life)?
•
Has the programme had an impact on your coaching?
•
Before finishing, is there anything else you would like to add about what we have talked about?
– In what ways? •
Thank you for your time!
Has the programme had an impact on your life and life plans? – In what ways?
© sports coach UK
40
The next steps are: •
We will write up the notes from the interview.
•
We will then share them with you to see if they are accurate.
•
These notes will be used (anonymously and collectively with all the other interviews) to inform Sport England and Youth Sport Trust about the success and problems with the programme.
Impact Measuring Tool Kit
41
Section 12
Recruit into Coaching Good Practice Evaluation Case Study Interview The interviewer should start by explaining why the interview is being carried out, ask for permission to record the interview and agree to let the interviewee see the report before it is published.
Start Can you give a quick overview of Carla’s involvement in the RiC programme and then we can go into more detail? Initial planning and set-up: •
How did the SSP first become involved in the programme?
•
What courses did you do as part of the programme?
•
What was the placement/deployment?
•
Were there any other development opportunities offered?
•
Was anyone else involved in training and deployment?
•
What are you doing now?
•
What sort of planning was required?
•
Was there anyone else involved in the initial set-up?
The impacts
•
Are there any issues with planning in a schools setting?
What have been the main impacts for the:
•
How did you go about finding recruits?
•
How did you come up with the idea to use Carla?
•
What were you expecting to be the result?
•
Recruit: Can you tell me a bit about yourself
•
Recruit: How did you first hear about the programme?
•
recruit?
•
schoolchildren?
•
community?
Final questions •
What have been the key learning points from this experience?
•
If there was anything you could do differently what would it be?
•
Is there anything else that we have not talked about that you would like to add?
The programme •
Can you tell me a bit about what you did as part of the RiC programme?
•
Was there an induction? What happened?
42
Notes
Impact Measuring Tool Kit
Impact Measuring Tool Kit
Notes
43
Our Vision What we would like to see UK coaching excellence enabling all children, players and athletes to follow their dreams, have fun and fulfil their potential.
Our Mission Why we exist To support our UK partners to recruit, develop and retain coaches to achieve their participation and performance goals (in the context of The UK Coaching Framework).
Our Strategic Objectives What we will do To achieve our mission, we will: 1 champion and drive policy and investment in coaching 2 support and challenge our partners to improve their coaching systems and grow their contribution to a cohesive UK coaching system 3 provide products and services that add value to our partners’ coaching systems and their coaches 4 provide research and share good practice that will benefit coaching 5 develop quality leadership, good governance and a skilled team to ensure an effective UK coaching agency.
Impact Measuring
ToolKit This resource has been developed to help county sports partnerships effectively measure the impact of the work they do. It is designed to be a practical tool and takes the reader through the key components for planning and collecting data.
sports coach UK Chelsea Close Off Amberley Road Armley Leeds LS12 4HP Tel: 0113-274 4802 Fax: 0113-231 9606 Email: coaching@sportscoachuk.org