RVC Lates 2013 Evaluation Report
The ‘wordle ‘ image above is an illustration of paper survey responses to RVC lates, where words are shown in size proportional to their frequency of occurrence. Response n=61. 1 Draft report compiled by Grace Kimble 21.10.2013
Contents
1. Aims of Evaluation 2. Event Context 2.1 Aims 2.2 Objectives 2.3 RVC staff 2.4 Advertising 2.5 Web page 3. Event 3.1 Live events 3.2 Stand descriptions 4. Impact 4.1 Public 4.1.1 Who was there? 4.1.2 Where had they come from? 4.1.3 Why were they there? 4.1.4 What did they do? 4.1.5 What did visitors think of RVC Lates? 4.1.6 How did RVC Lates change visitor attitudes? 4.2 Researchers 4.3 Student ambassadors 5. Operational improvements 6. Recommendations 7. Conclusions 8. Strategy: future directions 2
1. Aims of evaluation This evaluation report aims to summarise the impact of the public engagement event ‘RVC Lates‐ The Heart of RVC’ which took place on October 17th 2013. Evaluation aimed to: • Provide a comparison with the inaugural RVC Lates event held in 2012 • Provide evidence to meet requirements of the Physiology Society, who funded the event • Collate evidence about the impact of the event for visitors, students and RVC staff • Provide insight to inform future RVC Access strategy
Evaluation Process
Event evaluation
• Visitors completed paper feedback form on exiting RVC • Visitors were interviewed by Student Ambassadors
RVC staff feedback
• Student Ambassadors were asked to complete an online survey • Researchers and presenters were asked to complete an online survey
Debrief meeting
• This evaluation draft circulated for comment • Final report produced following debrief meeting to incorporate viewpoints from as many staff involved as possible
3
2. Event Context 2.1 Aims ‘RVC Lates: The Heart of RVC’ took place on Thursday October 17th 2013, from 1845‐ 2200, on the Camden Campus at the Royal Veterinary College, London. It is the second ever RVC Lates event; the first was initiated in October 2012 by former Widening Participation Officer James Cannon. James worked with Lisa Pritchard, PhD student at RVC, to submit a successful bid to the Physiological Society to fund two subsequent public engagement events. The aims of the event for public audiences were: • Educate people about physiology through anatomy and dissection – something many of them will never have nor ever will have the chance to do otherwise. • Allow people to perform laboratory techniques so that they understand exactly what research entails. • Demonstrate clearly the importance of research to human health. • Wipe out the stereotypes of scientists as reclusive eccentrics or socially inept “nerds”. • Raise awareness of the various career streams available in this area in teachers. A great many of the visitors in October 2012 were teachers – the more they know about physiology and its relevant careers, the more they can tell their pupils. (source: Funding bid application submitted Feb’ 2013)
2.2 Objectives • • • • •
To run between 5 and 10 walk‐up activity stands for each event. To run an animal dissection demonstration each event. The dissections will focus on each physiological system in turn and how they differ between species. To run 2 – 3 mini‐lectures of around 20 minutes in length each event To encourage creativity alongside scientific attention to detail with drawing workshops in our Anatomy Museum. Recruit a minimum of 200 people for each event using advertising in papers to create an extensive mailing list.
4
2.3 RVC Staff involvement RVC staff were recruited to run research stands by Lisa Pritchard during summer 2013. Thumbnails of their profiles are shown here. The event was supported by student ambassadors from a range of years and courses.
5
2.4 Advertising The event was advertised in the Camden New Journal, Time Out, and via online marketing sites Meetup , Biology week promotion, and TES Connect. Twitter and Facebook social media advertising linked to an online booking form. Education contacts were emailed in advance by Chris Hobson.
6
2.5 RVC web page The following description appeared on the website, leading to an online booking form which gathered information about visitors address, and how they found out about the event.
7
3. Event 3.1 Live events The images below show the front and reverse of a floor plan leaflet which was used to guide visitors during the event. You can see a description of live events and summarie4s of the activities taking place.
8
3.2 Research stands The stand labels below described the range of research stands on offer. We also had stands from the Francis Crick Institute and the Physiology Society.
9
4. Impact 4.1 Public Public impact will be considered under the following questions, in order to characterise the audience and provide information on which to decide future public engagement event planning. • Who was there? • Where had they come from? • Why were they there? • What did they do? • What did they think of it? • How did it change their attitudes?
Methods Visitor responses to the event were gathered using three methods, with a range of closed and open question types: a)
A paper survey on exiting the event. This survey was designed to gather demographic information and to allow comparison with the 2012 event by repeating questions asked last year. Answers provided formative information about how to improve the event, and summative information about the activities which took place. 61 forms were collected.
b)
Interviews with student ambassadors. These interviews focussed on event aims (see context section). 19 interviews were collected.
c)
A quiz about research stand information. 17 quizzes were correctly completed.
Qualitative and quantitative data will be presented to compare the event with the 2012 event, and to demonstrate evidence that the event aims have been successfully met. 10
4.1.1 Who was there? The following data are results from questions which were asked using the paper exit survey, n=61.
What are the age ranges of people you have visited with? 32
The majority of visitors were 18‐ 25, with frequency decreasing with age. Even if data show a bias towards younger people completing the form, then there is still evidence that the public who visited RVC Lates were largely under 35.
Number of visitors
25
10
8 3
18‐25
26‐35
36‐45
46‐55
56‐65
1 66‐75
What is your occupation? Student Teacher/education Finance/ law Art/crafts Administration Vet nurse Scientist Lecturer Writer Sports coach Actor
The largest sector of RVC Lates audiences were in the education sector; either as students or as teachers. This may be linked to advertising to RVC schools contacts, and adverts online using TES connect.
0
2
4
6
8
10
12
14
Are these the audiences that RVC public engagement events should aim to engage?
Number of visitors
How many people in your group? The majority of groups surveyed contained two visitors. Compared to similar events, the number of individual visitors and groups of four people was high. 11
4.1.2 Where had they come from?
Event Reach The data below are anonymous records from booking data, and show the reach of advertising. It is clear that Word of Mouth (Friends and Colleagues), Facebook and Emails were the top three ways visitors found out about the event.
Time Number 1815 1 1845 2 1855 2 1900 6 1915 2 1920 1 1930 3 1945 1 2000 3 2045 1
Arrival The data above are corroborated by interview data asking visitors how long they took to arrive at the event. The journey times ranged from 10 minutes (Zoo staff) to two hours. The most common journey time was one hour. Surprisingly, visitors arrived later at the event than expected: as shown in the table left (data from interview sample) .
12
4.1.3 Why were they there?
Interview question 3. asked: What attracted you to the event? From the 19 people surveyed, three groups emerge: 1. RVC connections Alumni or former employees, or people whose family work or study there wanted to see the site, see if there were any changes, remember being there, and to find out about what their family do all day! 2. Science supporters Some people who work in related industries such as biomedical research wanted an insight into latest research at RVC. They might be prospective students, or teachers looking for subject enrichment or career information for their pupils. They might be studying a related subject; for example a group of 3 sports science students and their lecturer from St. Mary’s College.
3. Curious public Public who did not have connections with RVC were attracted by the unique opportunity to see dissections, and the fact that it is a free event and a new experience.
Visitors taking part in dissection room workshop activities
13
4.1.4 What did they do?
Where did visitors go? The graph below was calculated by subtracting ‘NA’ data from the total responses per activity type (survey question 2) , to show which aspects of the event visitors did not attend. The overall pattern is similar, with slightly greater percentages of visitors taking part in the quiz and visiting the Haxby bar.
What did visitors participate in? (%) 100 90 80 70 60 50 40 30 20 10 0
2012 2013
Did you try any practical skills? Data from paper survey, n=61. Proteins Make model 7% hearts 7% Dissection lecture 21%
Atherosclerosis 7% Microscope 7%
Cow Heart dissection 20% Haptic Cow 17%
Exercise and cardiology 14%
14
4.1.4 What did they do (continued) ?
Did you see anything which links to human health? Data from interview, n=19)
Heart rate
Cardiovascular pathology stand Posters
Atherosclerosis stand
Proteins Human skeleton Blood pressure monitor
Exercise bike
15
4.1.5 What did visitors think of RVC Lates? Activity ratings The graphs below show the proportions of ratings per activity. Comments given are shown in speech bubbles (2013 survey data only, question 2b, n=61).
Overall, how would you rate RVC Lates? 100% of the 61 survey respondents rated RVC Lates good or excellent. This is similar to 2012, where 100% of visitors rated the event good or excellent.
2012 activity ratings (%) 100 80 60 40 20 0
2013 Activity ratings (%) 100 Below Average 90 80 Average 70 Good 60 Excellent 50 40 30 20 10 0
‘came some way to satisfying my terminal curiosity’
More time ‘Wish it was longer’ ‘We didn’t have time to try everything’ ‘didn’t see everything but what I saw was great‐ will be back next time’
Poor Below Average Average Good Excellent
Approachability ‘Very friendly students’ and ‘very friendly staff and interesting lecture’
16
4.1.5 What did visitors think of RVC Lates? (continued) Which part of RVC lates did you enjoy the most?
Number of visitors
Survey data Q3, n=61.
3
30
4
6
6 2
3
1
Which part of RVC Lates did you enjoy the least? Survey data Q4, n=61. There were very few comments about the least popular part of the evening. A number of people said nothing, none, or words to the effect that they enjoyed everything e.g. ‘All of it was wonderful!’ However, the only area mentioned as an issue was the smell of the dissection!
Would you recommend RVC Lates?
100% YES
100% of 61 survey respondents answered Yes to this question (Q5). In addition, there were some supporting comments: • ‘Because it’s awesome‐ really educational and suitable for public and students’ • ‘Good for students, especially others working with me in veterinary practice’ • ‘very useful study for drawing, as an art graduate I would recommend’
How does this event compare with your expectations? Survey question 8, N=61.
70% exceeded, 30 % met.
‘Better than science museum lates as more personal’
17
4.1.6 How did RVC Lates change visitor attitudes? Has this event changed your perception of physiology?
82% YES 33 respondents answered this question in the exit survey. Comments included: YES • • • •
• • • • • • •
My enthusiasm has continued to grow! made it very interesting learnt a lot Learnt much more about changes in physiology, especially during foetal stages widened it, increased my curiosity, research for myself (in a general way) always learning new things came last year, still find it fascinating Made me realise how quickly research is advancing for medicine made me more interested cuttlefish have three hearts! Not much previous knowledge!
NO • already fab • I was interested in it before I came! • I already work in a medical museum and have an interest in the area • Mother is a vet, very interesting though
18
4.1.6 How did RVC Lates change visitor attitudes? (continued) What surprised you about the event? Survey question 10, n=61 • Amount of displays and hands on activities • Angiogenesis research‐ would definitely see it used in the veterinary field in future • attention to detail • Didn't want to leave • Dissection • Enthusiasm • Excellent facilities • excellent museum‐ should've been expecting it but wasn’t! • Exercise is good for the immune system! Sheep are fed magnets! • First time I've ever seen a dissection • Free! • Front legs are not attached to the skeleton! • happiness of students and teachers • haptic cow • How close you get to the body, great lecturer! • How friendly and knowledgeable everyone is • how hands on it is • how much I would learn tonight • How well it was presented • I came last year; and same as intriguing, what's not to find worthwhile. Thank you for the evening • I didn’t realise that sheep's stomachs were so big… • I felt that the talks were pitched at the same level as my 18 year old students could do… • Making hearts • sheep dissection! • The detail that everything was explained • the haptic cow • The huge variety of stands on offer • The knowledge and enthusiasm of students • The range of things to do and see • The size of a sheep stomach • The smell in dissection • The smell of sheep innards. Also size, if I'm honest. • the smell of the dissection • wide range of displays the RVC has
19
4.1.6 How did RVC Lates change visitor attitudes? (continued) Were the researchers like you expected them to be? Interview question 6, n‐19 • • • • • • • • •
friendly and open and not boffins friendly and approachable friendly and open Andrew Crook excellent. Otherwise all very friendly geeky but nice approachable. Especially pathology stand knowledgeable and helpful yes, informative, easy to chat to no, they don’t look like researchers!
Have you found out about careers in this area? Interview question 7, n‐19 • • • • • • • • • • • •
veterinary medicine and science already PhD students didn’t know about pathologist academic not for me yes, saw people doing things, makes me want to research roles came for interest more, and to see about possible research pathways yes, about how people get into it yes, research. Enjoy the students being here being a vet don’t know yet‐ want to work with wild animals researcher wanted to talk to vet med students about applying
20
4.2 Research Staff Data based on 8 responses to online survey (so far, 21/10/2013)
Did you find out anything new through taking part in RVC Lates? • • • • • • •
I didn't have a huge chance to visit the other stalls but when discussing the prep for making them I learnt a great deal about animals hearts which I'd never studied. I heard some interesting heart murmurs in people (that they already knew about... ;‐)) Some of the stands provided useful information. How quickly a chick embryo develops a beating heart Yes. Many things. No. I talked to some colleagues about their research and found out new things.
What did you enjoy about being part of the event? • • • • • • • •
The positive atmosphere and meeting the public I enjoyed engaging with the public and encouraging them to be as enthusiastic about science as I am. The enthusiastic reactions from the public! Engaging with the public Talking to the public and seeing how interested they were. Good variety of questions, the other stands, the help of the ambassadors Uniting parts of the College and meeting new guests onsite. It was nice to see the RVC so lively in the evening.
21
4.3 Student ambassadors Data based on 9 responses to online survey (so far, 21.10.2013)
Did you learn anything new through taking part in the event? • • • • •
• •
•
Not really because I was at the entrance most of the time. Yes ‐ all about the cuttlefish heart and how our slides are created! Yes from a few of the stands I got a chance to go around. They were very interactive which made them great fun! Yes, a little about how they produce 3‐d images of protein structures and about the research they have done into the link between atherosclerosis and immunology. Yes, the event was amazing! I improved my understanding massively on the Haptic cow ‐ Tierney was really lovely to work with and explained everything I needed to know so I could teach the guests when they came to our stand to have a go on the Haptic cow. It made me revise some of the topics we did last year and was useful to know roughly how things feel. It was very interactive which I think was good and looking round the other stands I picked up some information on other areas of the cardiovascular system. Yes: about angiogenesis research and atherosclerosis research and proteins involved. Yes. Lectures provided great way to learn, listening to the research, E.i. diet affecting endothelium and therefore circulatory function, and also valvular degeneration and its symptoms (Boswood). Yes in Adrian Boswood's lecture
Did you try out any practical skills? • • • • • • • •
No because I was at the entrance. Making hearts out of plasticine! Used the microscopes. Yes, at the beginning of the evening at the stalls I looked at No (x2) Yes ‐ the Haptic cow was evidently very practical and allowed you to practice how it would feel to palpate certain structures within the abdomen and on the pelvic floor. Used the microscope to show different bits of the slide to people. Had a go at the haptic cow Staff view: My student ambassador was absolutely great. She hardly got a break at all and remained as engaged at 10pm as she had started at 5.30pm. Really nice idea to partner us up, and for me it wouldn’t have been possible without her.
22
4.3 Student ambassadors (continued) What did you enjoy about being part of the event? •
• • • • •
• • •
I enjoyed hearing how much so many people enjoyed themselves. It was great to talk to people as they came in and hearing their expectations and then most were satisfied upon leaving. Learning more about what RVC Lates actually is and seeing some of the research carried out here Love talking to people about Veterinary Medicine and the college. Enjoyed working with Elaine and meeting members of staff. Hearing such positive feedback whilst people were leaving‐ they all seemed to have had a great night! It was nice to talk with a variety of different people and see some of the research that goes on fairly hidden away in the RVC. I enjoyed meeting a range of different people who work at the university who I hadn't ever met or spoken to before, for example Tierney, so it was great for networking. Also, I enjoyed how practical and hands on all the stands were ‐ it is much more interesting than just reading things on stands. I thought the guests were all friendly and approachable too, which really helped, and as the Haptic cow was very popular, I was constantly busy, so the night went very fast! Explaining things to people coming round to view stands, learning what goes on in the research departments at RVC. Was nice to able to go to the different stations and see what they were providing and teaching. Getting to speak to the attendees about visiting our university and what they got from coming along to the event.
23
5. Operational improvements In order to structure recommendations, they will be considered under the same headings as shown in Sections 2 ‘Context’ and 3 ‘Event’
Staff profiles and stands There were no criticisms of the activity titles or staff profiles; thank you to the staff involved for providing this information. Activities were frequently praised for being hands on, and there were no reported problems or issues. There were few comments from either RVC staff or student ambassadors about visitor responses to research stands. One person stated that the most common questions was: ‘What animal did those cells/tissue come from?’ (and disappointment if the answer was "human") Activities were highly rated and interactive and should continue with this engaging approach So far, staff and students have felt well‐prepared for the event: • ‘I think there was sufficient time at the start for the researcher to explain to you what to do and what the stand was about, and sending out the information prior to the event like you did about the researchers was helpful’ (SA) • I felt totally prepared and was able to answer any queries’ (SA). Other organisations offer public engagement training for researchers, hence research staff were asked: is there any additional training you feel would be beneficial. So far, one person has suggested creative thinking and public engagement training. Many already feel prepared. Public engagement training opportunities should be considered but are not a high priority Logistics were praised in terms of extension cables, poster boards being present and in the right place; with thank s to staff involved. However, there was concern that safety was compromised in the pack up. Additional care should be taken to pack equipment away safely at the end of the night Student ambassadors were offered the opportunity to swap at 9pm, to work with a wider range of stands. However, not all people did this. Student ambassadors to decide whether swapping is beneficial or not. GK facilitate majority opinion. Food was provided for researchers and student ambassadors. However, the following quote explains an issue: I believe the event offered a good level of snack and food to keep us all going during the event. but I was just disappointed that the ambassadors ended taking advantage of this and by 7pm most of the snacks had been eaten and then by 8:15pm a lot of the wine had gone, with most researchers not having access to much of these. I appreciate it was great to get plenty of pizza though :). Breaks need to be better structured. 24
5. Operational improvements (continued) Advertising and attendance • •
•
Suggestions have been made for Society of Biology, Vet Record, and better intranet advertising , for example in Hawkshead. Posters on both sites are recommended. There were no complaints about the web page and the online booking form provided good information for the dissection event and analysis. Jack Sisterson quickly updated the webpage when the event automatically stopped receiving bookings on the day, which can be learnt from for next time. The web page needs to accept bookings on the day. Although 198 people booked online and seven additional people were welcomed on the evening, the was a high occurrence of non‐attendance. £5 cancellation fee recommended.
Event •
•
• • • •
• • • • • • • •
The dissection event was well attended, although booked visitors did not attend and places were filled by other visitors. Consequently, concurrent lectures were not well attended. Lectures need not to be scheduled concurrently with dissections. More people attended the Haxby bar in 2012 compared to 2013, due to better signage. However, this area was the only one scoring ‘poor’ in the activity ratings. Advertising should state that the Haxby Bar is a student bar, to manage visitor expectations. Researchers need general RVC handout information for visitors. The British Museum and Royal Society are good at providing these, for example. The background music needs to be quieter, particularly for stands with stethoscopes. The research stand area needs to be enlivened during the dissection event. Scheduling is a key area to address e.g. I think there were a few guests who felt there wasn't enough time to see everything e.g.. some still wanted a try on the Haptic cow when we were already packing up, so maybe the time could even be extended slightly, as a lot of guests wanted to see the dissection, go to a lecture and still have time to see all the stands which I don't think they could squeeze in! Better signage about live events taking place Pub quiz event earlier; use another event to denote the end of the evening? Straight table cloths Use the Plasma screens to advertise the RVC Offer tours of campus Use the pod Offer guests Wi‐Fi passport to try and make a social media impact ACT need to be there to sell merchandise
25
6. Recommendations summary
Staff profiles and stands • • • • •
Activities were highly rated and interactive and should continue with this engaging approach Public engagement training opportunities should be considered but are not a high priority Additional care should be taken to pack equipment away safely at the end of the night Student ambassadors to decide whether swapping between stands is beneficial or not. GK facilitate majority opinion. Breaks need to be better structured.
Advertising and attendance • • •
Posters on both sites are recommended. web page needs to accept bookings on the day. £5 cancellation fee recommended.
Event • • • • • • • • • • • • •
Lectures need not to be scheduled concurrently with dissections. Advertising should state that the Haxby Bar is a student bar, to manage visitor expectations. Researchers need general RVC handout information for visitors. The background music needs to be quieter, particularly for stands with stethoscopes. The research stand area needs to be enlivened during the dissection event. Scheduling is a key area to address Better signage about live events taking place Pub quiz event earlier; use another event to denote the end of the evening? Straight table cloths Use the Plasma screens to advertise the RVC Offer tours of campus Use the pod Offer guests Wi‐Fi passport to try and make a social media impact ACT need to be there to sell merchandise
26
7. Conclusions RVC Lates 2013 met objectives, however there are some issues which need addressed before the next event on Spring 2014: • To run between 5 and 10 walk‐up activity stands for each event. • To run an animal dissection demonstration each event. The dissections will focus on each physiological system in turn and how they differ between species. • To run 2 – 3 mini‐lectures of around 20 minutes in length each event • Issue: low lecture attendance owing to scheduling issues • To encourage creativity alongside scientific attention to detail with drawing workshops in our Anatomy Museum. • Recruit a minimum of 200 people for each event using advertising in papers to create an extensive mailing list. • Issue: low attendance on the evening; cancellation fee recommended together with higher capacity for event overall To return to the initial funding bid for the Society of Physiology, the aims of the event for public audiences were (evidence summarised in purple text ): • Educate people about physiology through anatomy and dissection – something many of them will never have nor ever will have the chance to do otherwise.] • The public took part in unique anatomy and dissection events, and 83% of responses indicated that audience perception of physiology had changed. • •
Allow people to perform laboratory techniques so that they understand exactly what research entails. Research stands allowed visitors to take part in practical research activities.
• •
Demonstrate clearly the importance of research to human health. Interview visitors were able to state links between RVC research stands and human health.
• •
Wipe out the stereotypes of scientists Scientists were seen to be friendly, positive, enthusiastic and approachable, and this was unexpected for some visitors.
• •
Raise awareness of the various career streams available in this area in teachers. Interviewed visitors were able to comment on career information, although this could be improved. 27
8. Strategy: Future directions
28
Post script: General comments
Research staff • •
• • •
•
I liked the quiz! I was surprised at how much people seemed to enjoy my stand, I had thought that matching up hearts and heart rates with species would be a simple fun activity but nothing special. Lots of people commented on how much they enjoyed it and what a good idea it was. I'm sure you could do a similar thing with lots of different organs/body systems. None ‐ thanks for organising it all. All in all I thought it ran very well and was happy to be a part of it. It was really nice of you to provide snacks and drinks in a designated room for us. For those of us who don’t know Camden very well, it was good to have a space to escape to for a few minutes! And the minibus was great!! The announcements for the lectures, and ambassadors taking the public to the rooms worked very well.
Student ambassadors • •
•
Maybe try two shorter dissections so more people get to do that part seeing as it is such a big hit for the evening. Brilliant event, well organised! All the staff were really friendly and approachable. Definitely my favourite event so far in terms of student ambassador events ‐ thanks for the pizza and drinks too, that was really great! Was a great event that brought everyone together. Wondered about the target audience, prospective students, community, current students. Seemed like could be a real draw for prospective students and current wishing to revise!
29
30