No
External Evaluation Report: UCM ‘Big Draw on Tour’
P a g e | 1
Dr Eric Jensen1 University of Warwick 1.
Assistant Professor Department of Sociology University of Warwick e.jensen@warwick.ac.uk http://warwick.academia.edu/EricJensen
January 2013
No
External Evaluation Report: UCM ‘Big Draw on Tour’
P a g e | 2
Table of Contents
Executive Summary .................................................................................................................... 3 Executive Summary of Visitor Profile Results ................................................................................. 3 Executive Summary of Feedback and Impact Results .................................................................... 4 Introduction: Intended Outcomes and Context ........................................................................... 5 Methods ..................................................................................................................................... 6 Data Collection ............................................................................................................................... 6 Data Analysis .................................................................................................................................. 7 Quantitative Content Analysis. ................................................................................................... 7 Qualitative Analysis. ................................................................................................................... 8 ‘Big Draw on Tour’ Evaluation Results ......................................................................................... 9 Visitor Profile: Demographics, Geographic and Attendance ........................................................ 9 Group Size ...................................................................................................................................... 9 Age Distribution .............................................................................................................................. 9 Gender Distribution ........................................................................................................................ 9 Ethnicity ........................................................................................................................................ 10 Level of Education ........................................................................................................................ 10 Employment Status ...................................................................................................................... 11 Geographic Distribution ............................................................................................................... 12 Identification of Event Organiser .................................................................................................. 13 Recognition and Cross Visiting between University of Cambridge Museums .............................. 14 ‘Big Draw on Tour’ Marketing ...................................................................................................... 16 Feedback Results ....................................................................................................................... 17 Workshop Participant & Staff Interaction .................................................................................... 17 Novel Experiences: Art appreciation and cross-‐museum visiting ................................................. 17 Enjoyment and Desired Future Participation ............................................................................... 18 ‘Difficulty’ Level of Big Draw on Tour Activities ............................................................................ 19 ‘Best Aspects’ of Big Draw on Tour Programme from Participant Perspective ............................ 19 ‘Least Liked Aspects’ of Big Draw on Tour Programme from Participant Perspective ................. 23 Self-‐Reported Gains and Losses from Attending the Event .......................................................... 24 Impact Evaluation Results: Thought-‐Listing Data ....................................................................... 27 Participants’ Thinking about ‘Drawing’ ........................................................................................ 27 Participants’ Thinking about ‘Museums’ ...................................................................................... 29 Participants’ Thinking about the ‘University of Cambridge’ ......................................................... 31 Case Studies: New Audiences ....................................................................................................... 33 Conclusion ................................................................................................................................. 35 Future Evaluation Recommendations .......................................................................................... 36 Recommendations for Future Programmes ................................................................................. 38 References ................................................................................................................................. 41
No
External Evaluation Report: UCM ‘Big Draw on Tour’
P a g e | 3
Executive Summary
This report describes key findings from a pilot evaluation study conducted at an early stage in the development of new University of Cambridge Museums (UCM) partnership projects (supported by Arts Council funding). The research gathered mixed methods survey data from visitors participating in the ‘Big Draw on Tour’ (BDoT) programme hosted by the University of Cambridge Museums. Evaluation data were gathered from respondents before and after BDoT events in order to gain insights into visitors’ experiences and programme impacts. The results show that visitors were overwhelmingly positive about the engagement experiences, which were facilitated by artist volunteers and UCM staff. There is direct evidence of impact, along with self-‐reports of a range of benefits from attending the BDoT events. The events: • Engaged new audiences with the UCM collections, promoting greater cross visiting between museums • Enhanced visitors’ enthusiasm and interest in both drawing and University of Cambridge museums’ collections • Changed perceptions of University of Cambridge museums or the University of Cambridge
Executive Summary of Visitor Profile Results •
Majority attending Big Draw on Tour events were visiting in family groups (93%, n=58). Relatively few visitors came alone (2%) or with non-‐family members (5%).
•
Gender breakdown of survey participants was 65% female and 35% male.
•
The data suggest that the predominant demographic pattern for ‘Big Draw on Tour’ at the University of Cambridge Museums was ‘family visits’, including parents (in their 30s and 40s) and younger children (most under age 8).
•
Most respondents were educated, at least to secondary level (83%); 47% had a first degree; 35% had a post-‐graduate degree.
•
Respondents were majority ‘white’ (71%), but other ethnicities were also represented.
•
Results indicate that visitors to the ‘Big Draw on Tour’ hosted by University of Cambridge Museums are predominately local, with 49% of visitors travelling just 3 miles or less.
•
31% of visitors reported awareness of one of the University of Cambridge’s embedded museums (other than the one they were visiting). However, just 18% had previously visited one of the ten museums. This leaves a great deal of room for expansion (and highlights the value of partnership projects with the goal of increasing cross-‐museum visiting)
No
External Evaluation Report: UCM ‘Big Draw on Tour’
P a g e | 4
Executive Summary of Feedback and Impact Results •
•
There was evidence of a shift in visitors’ thinking from pre-‐ to post-‐visit, with ‘festival’, ‘architecture and ‘public’ more closely associated with the ‘University of Cambridge’ post-‐ visit (and ‘hard’ and ‘academic’ becoming less prominent associations). Furthermore, visitors’ increasingly viewed the university as a ‘public’ institution, rather than a place for ‘research’ and ‘education’. This data implies BDoT visitors developed a more favourable orientation towards the University of Cambridge. The BDoT programme has been influential in changing common conceptions of museums as ‘old’, ‘boring’ facilities that are often silent and only contain ‘things’ from ‘history’. There was an aggregate change in visitors’ perceptions to a more favourable attitude, viewing museums as ‘interesting’ places, where ‘learning’ takes place through interaction with ‘artefacts’.
•
There is evidence of positive impact on visitors’ thinking about ‘drawing’, which is more connected to ‘children’ and ‘ideas’ in visitors’ minds post-‐visit.
•
A majority (88%) of visitors stated that they had come across new and interesting objects or ideas during their visit. Self-‐reported impacts include:
o Unexpected new appreciation for art o Visited new UCM museums or exhibits
•
•
• •
Participants listed ‘best aspects’ of ‘Big Draw on Tour’ programme as those aspects that satisfied ‘children’, created ‘drawing’ experiences, ‘engaged’ adults or their children and involved interactions with ‘staff or volunteers’.
•
For most visitors (76%), an effective level of guidance and support was provided, while, in a minority of cases (22%), more was still desired.
•
Only 19% (n=6) of respondents reported finding the BDoT activities ‘difficult’
Very high levels (100%) of enjoyment were reported. 97% said they would like to participate in a similar event in future. As a result of participating, many of the respondents (54%) commented that they had gained increased skill and/or confidence in their drawing skills. The positive impacts of the BDoT events were no doubt linked to the unusually high level of interaction with volunteers and staff: 94% of visitors reported such interaction.
No
External Evaluation Report: UCM ‘Big Draw on Tour’
P a g e | 5
Introduction: Intended Outcomes and Context Photo: Participants across the UCM ‘Big Draw on Tour’
This external evaluation research report focuses on a series of artist-‐led drawing workshops held at 10 different venues around Cambridge in October and November 2012 called ‘The Big Draw on Tour’ (BDoT), which was well-‐staffed with volunteers and engagement practitioners at the museums. The project made connections between the different UCM collections. BDoT worked with 744 participants of many different ages including pre-‐schoolers, children, students, adults and grandparents. Evaluation data were collected from a representative sample of this visitor population in order to assess the effectiveness of the project based on feedback and impact evidence from audiences. A separate internal evaluation report by Dr Kate Noble presents UCM practitioners’ perspectives on the BDoT project and its effectiveness (Internal evaluation report entitled: ‘Big Draw on Tour: Connecting Collections Working Together: Project Partners’ Reflections a UCM Joint Project’). The BDoT project builds on previous outreach efforts led by University of Cambridge museums under the auspices of Renaissance East of England funding (e.g. Jensen 2013; University of Cambridge Museums 2008). BDoT included public workshops held at Romsey Mill Community Centre, Cambridge Central Library, Museum of Classical Archaeology, Kettle’s Yard, Museum of Zoology, Scott Polar Research Institute, The Fitzwilliam Museum, The Sedgwick Museum, Museum of Archaeology and The Whipple Museum. The project was aimed at supporting the University of Cambridge Museums’ ‘Connecting Collections’ goals: •
An outstanding cultural offer: […including an] integrated Cambridge-‐wide cultural experience.
•
Wider audiences: Increased participation in our diverse cultural offer, in particular by individuals and groups who do not currently engage with culture and/or who have limited cultural opportunities
No •
External Evaluation Report: UCM ‘Big Draw on Tour’
P a g e | 6
Deeper engagement: […] unlocking access to the wider research activities of the university. 1
The BDoT was intended to develop enhanced links between the different UCM collections and encourage new and more extensive numbers of people to visit and be inspired by museums across Cambridge. This external evaluation addresses the following research questions: Has the Big Draw on Tour… •
…engaged people with drawing? Other kinds of impacts on people's thinking?
•
…engaged people with the UCM collections? Changed perceptions of University of Cambridge museums or the University of Cambridge?
•
…developed new audiences across the University of Cambridge Museums?
Methods The research methods for this study were designed to provide insight into the impacts stemming from the Big Draw on Tour programme run by the University of Cambridge Museums (UCM). Survey questions were reviewed and agreed by UCM staff in advance to ensure that they would reflect the aims and intentions of the pilot programme.
This study focused on visitors’ thinking about drawing, museums and the University of Cambridge. An electronic questionnaire was used for the pre-‐visit and post-‐visit. Where possible, participants were initially surveyed prior to partaking in a ‘Big Draw on Tour’ activity at a University of Cambridge museum. In this pre-‐visit survey, visitors were asked open-‐ended (qualitative) impact evaluation questions and close-‐ended (quantitative) questions about their demographic characteristics and other variables. The post-‐visit questionnaire included feedback items, as well as the second iteration of open-‐ended impact evaluation questions and further demographic variables. In some cases, it was only possible to collect pre-‐visit data, while in other cases only post-‐visit data was available. However, for the cases when both pre-‐ and post-‐visit responses were available, these were compared to directly measure the impacts of the Big Draw on Tour activities.
Data Collection The data collection plan was to distribute questionnaires before (pre-‐visit) and after (post-‐visit) visitors’ participation in the Big Draw on Tour Programme to gather data on the demographic profile of visitors, gather feedback on visitors’ experience and to directly measure the impacts of the programme on participants’ thinking about drawing, museums and the University of Cambridge. The ‘On Site’ (Pre-‐Visit) Questionnaire The questionnaire was distributed using iPad hardware and software to visitors during the ‘Big Draw on Tour’ programme across University of Cambridge museums. Respondents were asked open-‐ended questions about what came to mind when they think of ‘drawing’, ‘museums’ and the ‘University of Cambridge’. The questionnaire also included closed-‐ended questions asking what University of Cambridge museums the respondent had ‘heard of’ and ‘visited’. A question asked respondents to indicate their level of agreement with the statement, ‘I found it hard to find my 1
Cited from Connecting Collections blog http://camunivmuseums.wordpress.com/connecting-‐collections-‐our-‐role-‐as-‐ a-‐major-‐partner-‐museum 2 The data reported above excludes individuals (n=8) who only participated in the post-‐visit component of the
No External Evaluation Report: UCM ‘Big Draw on Tour’ P a g e | 7 way here for this event’. Visitor respondents were then asked with whom they were visiting and what interested them in attending. To understand about the distance visitors travelled to participate in the ‘Big Draw on Tour’, they were asked to provide their postcode. Finally, they were asked whether they had any other museum in the previous year. They were also asked if they would be willing to participate in an online evaluation of their visit experience. The Post-‐Visit Questionnaire The post-‐visit questionnaire for visitors, sent after the event via email, investigates impacts and feedback on UCM BDoT activities. The questionnaire first asks the same open-‐ended question as above about what comes to mind when they think of ‘drawing’, ‘museums’ and the ‘University of Cambridge?’ The responses to these survey items are compared to those taken in the pre-‐visit questionnaire given on the day of their visit to the museum hosting the ‘Big Draw on Tour’ programme. In a further step to determine visitors’ self-‐reported impacts and feedback, respondents are asked what they felt the most and least successful components of the BDoT programme, as well as what they thought they had gained by participating in the activity. One open-‐ended question was posed to gauge visitors’ thinking about the building in which the activity was held. Visitors are then asked to rate their agreement with 5 statements regarding their participation in the ‘Big Draw on Tour’ programme on a five-‐point Likert scale from ‘strongly disagree’ to ‘strongly agree’.
Data Analysis The approach for analysing the evaluation questionnaire data was to elucidate the ‘Big Draw on Tour’ visitor profile, identify patterns in visitors’ reported experiences and evidence of impacts. Quantitative Content Analysis. Survey data were imported into an Excel spreadsheet from the online survey system (wufoo.com) in preparation for analysis. Some qualitative data was coded to transform the responses into simple quantitative data to provide an overview of visitors’ sentiments. Comments concerning the visitors’ response to ‘What words come to mind when you think of ‘drawing’, ‘museums’, or ‘University of Cambridge?’’ were coded as follows: ◦ A positive comment includes positive describing words (e.g. ‘interesting’, ‘important’) and/or words that positively described certain aspects of the experience (e.g. "pencils", ‘creating’) ◦ A negative response includes negative describing words (e.g. ‘boring’) and/or words with negative connotations to describe aspects of the experience (e .g. ‘complicated’) ◦ A positive response was given if the positive words out numbered the negative words. Conversely, a negative response was given if the negative words outnumbered the positive words. ◦ If a response was ambivalent (i.e. had equally bad/good comments) it was assigned a neutral standpoint. Additionally, phrases that were not easily categorised as positive/negative from the context of the description (e.g. ‘things’, ‘university’) were coded as neutral. Respondents that had missing data (either their pre-‐ or post-‐ response) were coded as incomplete.
No External Evaluation Report: UCM ‘Big Draw on Tour’ P a g e | 8 Qualitative Analysis. The qualitative data collected were also analysed to identify patterns and themes (for full discussion of data analysis methods used, see Jensen & Holliman 2009. The results of this thematic analysis are presented in the Results section below).
No
External Evaluation Report: UCM ‘Big Draw on Tour’
P a g e | 9
‘Big Draw on Tour’ Evaluation Results Visitor Profile: Demographics, Geographic and Attendance
In order to assess the profile of BDoT visitors, this section presents the results of analyses of the quantitative demographic and attendance questions.
Group Size A majority of respondents reported attending in family groups (93%, n=58). Whilst only 2% (n=1) visited by themselves and 5% (n=3) visited with other unrelated persons. Fourteen individuals chose not to respond to the question.
Age Distribution A small proportion of respondents included adolescents in the age category 8-‐19 (3%). There was very little representation of individuals in the 20-‐29 age category. This category consisted of only five individuals (7%). There was a significant increase in participation in the 30-‐39 year old age category (28%). The majority of the respondents in this age category were age 37-‐39 (84%). Over half of participants (52%) came from within the 40-‐49 year old age category. Respondents’ ages were skewed towards the 40-‐45 year old range (78%). There was a sharp decline in participation of individuals in the subsequent age categories. Only 9% of the individuals stated that they were 50-‐59 years old, whilst 3% of respondents indicated they were 60-‐65. None of the participants stated that they were older than 65.
Gender Distribution The gender breakdown of the survey participants was 65% (n=34) female and 35% (n=18) of the respondents indicated they were male. In the 8-‐19 age category, 67% of respondents were female and 33% were male. 75% of the individuals in the 30-‐39 year-‐old age category were female, as were 60% of the 40-‐49 year-‐old age category and 50% of participants in the age categories from 50-‐89. Thus, the overall gender distribution for this sample of survey participants was skewed towards woman and girls. However, the distribution of genders is nearly uniform between males and females over age categories, with the exception of the 37-‐45 year-‐old age range (suggesting that there was greater participation from mothers).
No
External Evaluation Report: UCM ‘Big Draw on Tour’
P a g e | 10
Distribution of Participants by Age and Gender
Ethnicity Survey participants were asked to indicate their ethnicity. The analysis of ethnicity indicates that the ‘Big Draw on Tour’ participants are predominantly ‘White’ (71%, n=22). Few participants indicated another race, instead citing that they would prefer not to state their ethnicity (19.4%, n=6). Other ethnicities mentioned include: ‘Asian’, ‘Mixed’, and ‘Other’, each of these ethnicities was mentioned by one individual respectively.
Level of Education A majority of respondents indicated that they had at least a secondary education (83%, n=47). 47% (n=27) of the participants indicated that they had obtained their first degree, while 35% (n=20) of the participants had obtained a post-‐graduate degree. Only 4% (n=2) of the individuals had only obtained GCSE qualifications. Seven (12%) individuals had A-‐level qualifications, or equivalency. One of the participants preferred not to indicate an education qualification.
No
External Evaluation Report: UCM ‘Big Draw on Tour’
P a g e | 11
Participant Level of Education
Employment Status The participants were asked to indicate their current employment status. The most frequent response was working part-‐time, full-‐time or as a volunteer (53%, n=16). Homemaker was cited as the second most frequent status (33%, n=10). Two participants indicated that they were students, as well as two participants indicating that they preferred not to respond to this question. Participant Level of Education
No
External Evaluation Report: UCM ‘Big Draw on Tour’
P a g e | 12
Geographic Distribution As can be seen, in the maps below, the greatest proportion of visitors (49%, n=36) travelled less than three miles to partake in the ‘Big Draw on Tour’ (Map 1, three mile radius from the city centre represented by the two yellow lines). Of those participants, 41% travelled from within a one-‐mile radius of Cambridge’s city centre (within the red circle, Map 1). Map 1: Participants Residing in Cambridge
In the map above, yellow lines represent a 3-‐mile radius from the city centre, whilst the red circle encompasses an area 1-‐mile from the city centre.
All of the visitors indicated that they reside somewhere within the United Kingdom. However, visitors were from diverse locations, including: London (5%), Guernsey, near Preston, near Hull and Chelmsford.
No
External Evaluation Report: UCM ‘Big Draw on Tour’
P a g e | 13
Map 2: Participant Residence within the UK
Identification of Event Organiser Visitors were asked in the pre-‐visit survey to identify ‘who is running this event’. The question was asked as an open-‐ended question, so as to not constrain or prejudice visitors’ answers. The responses indicate that the majority of visitors believed the event was run by either a University of Cambridge museum (UCM, Fitzwilliam, Kettle’s Yard, and Sedgwick), the University of Cambridge itself or the Big Draw on Tour.
No
External Evaluation Report: UCM ‘Big Draw on Tour’
P a g e | 14
Visitor Responses to question: ‘who is running this event?’
The diversity of responses indicated above shows that there was not consistent and coherent branding of BDoT from visitors’ perspectives.
Recognition and Cross Visiting between University of Cambridge Museums While most visitors (pre-‐visit) had heard of the Fitzwilliam Museum (89%, n=57), there was a great deal of variability in the awareness of the other museums in the University of Cambridge museum group (as seen in the table below). This indicates that there is some cross-‐museum awareness within the UCM museums’ visitor populations. However, when asked which of the museums the participants had visited in the past, the majority of visitors had visited the Fitzwilliam Museum (98%, n=60), whereas only 18% of participants (n=11) had visited the Whipple Museum of the History of Science. Table Comparison of Visitors: ‘Heard of’ a Museum vs. ‘Visited’ a Museum Fitzwilliam Kettle's Yard MAA Zoology Classical Archaeology Whipple Sedgwick Polar Museum
Heard of Museum
Visited Museum
89% 66% 67% 72% 31% 38% 67% 53%
98% 57% 56% 54% 26% 18% 53% 43%
No
External Evaluation Report: UCM ‘Big Draw on Tour’ Folk Museum**
56%
39%
Botanic Garden Technology Other
69% 39% 5%
62% 23% 3%
P a g e | 15
** The Cambridge and County Folk Museum is not a part of the UCM group, but is supported by Cambridge City Council, National Lottery, Heritage Lottery Fund, Cambridge 800 Committee and the Friends of the Folk Museum. The UCM work closely with the Folk Museum, the Museum of Technology and other museums in the area on a number of programmes and promotional initiatives.
Graph Comparison of Visitors: ‘Heard of’ a Museum vs. ‘Visited’ a Museum 100% 90% 80% 70% 60% 50% 40% 30% 20% 10% 0%
Heard of Museum Visited Museum
The gaps between awareness and actual visiting of museums were relatively small for most of the University of Cambridge Museums, with the largest gap evident for the Whipple Museum. This gap could suggest that there is a problem in Whipple’s marketing and conversion of awareness into visits. However, it is equally possible that the profile of visitors for whom ‘Big Draw on Tour’ was an appealing activity where less interested in the history of science than in the content available from other museums. To make this determination, we would need robust data on the normal visitor profiles of the embedded museums in order to make a comparison with the profile of BDoT visitors. There is also a strange apparent discrepancy in these data wherein the number of visitors to the ‘Big Draw on Tour’ who had heard of the Fitzwilliam museum prior to the event is lower than the number of visitors who reported having visited the Fitzwilliam—a discrepancy that can be most easily explained by first time visitors to the Fitzwilliam to participate in BDoT. These visitors may
No External Evaluation Report: UCM ‘Big Draw on Tour’ P a g e | 16 have mistaken the ‘heard of’ question to instead mean ‘had they had heard of the Fitzwilliam prior to the day of their participation’. Conversely, the visitor may have felt that they had already visited the Fitzwilliam because the BDoT activity took place within the museum.2
‘Big Draw on Tour’ Marketing When visitors were asked if they had heard of the ‘Big Draw on Tour’ before the day’s events, the result was an even split amongst respondents: 50% indicated that they had heard of ‘Big Draw on Tour’ before, whilst 50% said that they had not. The participants were also asked how they found out about the ‘Big Draw on Tour’ programme. The largest category of respondents had found out about the ‘Big Draw on Tour’ from the website (34%, n=26), while 32% found out about the event via a marketing leaflet. When responses are analysed in terms of their previous response (had they previously heard of the ‘Big Draw’, yes or no), it becomes clear that certain forms of marketing were more effective at reaching new audiences. Specifically, ‘website’ was disproportionately cited by new BDoT audiences, while those who had already heard of the Big Draw were more likely to find out about these events through ‘word of mouth’ and ‘posters’. This can be seen in the graph below. Comparison of Marketing Approaches to Visitors: ‘Heard of Big Draw Before’ (Yes) vs. ‘Not Heard of Big Draw’ (No)
This result suggests that the web is a particularly important tool for the University of Cambridge Museums to reach out to new audiences.
2
The data reported above excludes individuals (n=8) who only participated in the post-‐visit component of the evaluation, as information on visited museums was collected pre-‐visit.
No
External Evaluation Report: UCM ‘Big Draw on Tour’
P a g e | 17
Feedback Results
This section presents the result of analyses of visitor comments regarding their experiences with the ‘Big Draw on Tour’ workshops, which were mostly held at University of Cambridge Museums. This post-‐visit feedback sheds light on visitors’ experiences at these workshops.
Workshop Participant & Staff Interaction A very high percentage (94%) of visitors reported interacting with staff or volunteers at the BDoT activities. These 94% were asked to specify ‘what happened’ and ‘what [they] talked about’ with the staff or volunteers. One theme in their responses was that adult visitors talked to staff and volunteers about how to encourage their children’s interests and engagement in the arts: ‘Big Draw. My daughter’s art and how to encourage her to maintain an interest (she's 3 1/2). The projects of the day.’ (Female, 41) ‘My daughter's drawing and making contacts for future workshops with outreach staff for home-‐education group.’ (Female, 42) Additional respondents conversed with staff on developing specific skills in drawing. These visitors were quite eager to learn more about drawing. Many of these individuals also mentioned aspects of drawing technique in their most liked aspect of the ‘Big Draw on Tour’ programme: ‘We talked to a museum person about the exhibits and to two or three of the Big Draw people about art -‐ two of them were very helpful in giving hints and ideas for improving our drawings.’ (Female, 39) ‘They shared their experience and were kind enough to give me tips on drawing and shading. We discussed children's illustrations as well.’ (Female, 29) A number of other visitors gave broad descriptions of their discussion, without providing further detail: ‘Interaction regarding the activities my young niece was taking part in.’ (Female, 43) ‘I gave feedback’ (unspecified gender, 40) Nevertheless, the feedback provided on this question indicates very positive and extensive levels of personal interaction between visitors and staff.
Novel Experiences: Art appreciation and cross-‐museum visiting Most respondents (88%) stated that they had come across something new and interesting during their BDoT visit. Visitors stated that one of the most novel experiences was a new appreciation for
No External Evaluation Report: UCM ‘Big Draw on Tour’ P a g e | 18 the art they were seeing. Several of the visitors stated that the drawing sessions helped them ‘look’ at art using unique techniques: ‘The fact that you are drawing makes you look better at the objects’ (Male, 44) ‘I really liked the way the drawing made me look at things differently.’ (Male, 49) Whilst other visitors specifically mentioned that their participation in BDoT enticed them into visiting new UCM museums or exhibits: ‘Whole of the Museum of Archaeology and Anthropology. New to us!’ (Female, 41) ‘Exhibits in the museum which I otherwise wouldn't have seen’ (Unspecified Gender, 43) ‘Had not visited Museum of Zoology before; and had no idea it would be so absorbing’ (Female, 43) Such responses provide direct evidence of the BDoT project’s success at bringing in new audiences across the UCM collections.
Enjoyment and Desired Future Participation Participants in the Big Draw on Tour activities were asked to indicate their level of agreement with the statement ‘I enjoyed the activity’ on a scale of 1 (Strongly Disagree) to 5 (Strongly Agree). Overall, visitors were very positive about their UCM ‘Big Draw on Tour’ experiences. 72% of the respondents ‘strongly agreed’ that they enjoyed the activity, whilst 28% of the respondents ‘agreed’ that they enjoyed the activity. Only one respondent chose not to respond to this question. The overall results show a very high level of enjoyment. Participant Enjoyment: ‘I enjoyed the activity’
No External Evaluation Report: UCM ‘Big Draw on Tour’ P a g e | 19 Interested in Future Participation Respondents were then asked if they ‘would like to do something like this again.’ 63% of the visitors ‘strongly agreed’ and 34% ‘agreed’ that they would like to participate in a similar activity in the future. Indeed, only 3% were ‘neutral’ about participating in similar event in the future.
‘Difficulty’ Level of Big Draw on Tour Activities When respondents were asked if they ‘found the activity difficult’, 28% of the respondents ‘strongly disagreed’ that the activity was difficult and 34% of visitors ‘disagreed’. Only 19% of the visitors agreed that the activity was difficult. 19% were neutral regarding the difficulty of the activity. One survey question assessed the level of guidance and support offered during BDoT activities, asking if more was needed. 13% of the visitors ‘strongly agreed’ and 9% ‘agreed’ that they could have used more guidance. Overall, visitors ‘disagreed’ (38%) and ‘strongly disagreed’ (38%) they needed more guidance in the activity. These results suggest that for most visitors (76%) an effective level of guidance and support was provided, while in a minority of cases (22%) more was still desired.
‘Best Aspects’ of Big Draw on Tour Programme from Participant Perspective Participants were asked to list ‘the best aspects’ of ‘Big Draw on Tour’. A number of categories emerged from the analysis, a majority of these comments can be categorised as those that emphasise satisfying ‘children’, ‘drawing’, ‘engagement’, concerning the ‘museum’ and interactions with ‘staff or volunteers’. A majority of the comments (43%, n=27) pertained to how the visitors engaged with the programme or how they reflected on their (or their groups’) ‘engagement’ with the ‘Big Draw on Tour’ programme. For instance, select visitors made general comments on the whole of the activity programme: ‘Range of activities on the site. [There was a] good selection for all ages, [such as] colouring knights to knock over with siege engine.’ (Female, 37) ‘Easy to drop in & participate’ (Male, 47) Other visitors (16%, n=10) focused on the experiences of their children or grandchildren. They reported that their (grand)children were engaged by ‘Big Draw on Tour’ and found it enjoyable: ‘My daughters loved playing with the shadows.’ (Female, No Age) ‘Suited all ages and my 5 and 7 year olds were really engaged in the activities-‐ more so than in other drawing / activities’ (Female, 41) ‘You were able to keep children interested all the time’ (Female, 19)
No
External Evaluation Report: UCM ‘Big Draw on Tour’
P a g e | 20
Photo: Participants at Kettle’s Yard interacting with Shadow Activity
Others indicated their enjoyment with being able to see their children enjoying the activity or enjoyed participating with their children in the activity: ‘Opportunity to share ideas with my daughter; and seeing her interest in both drawing and the museum ‘ (Female, 41) ‘Doing the same thing with my daughter’ (Male, 49) ‘Ben drew a quick pastel drawing of my baby, which was the best thing for me for the day :)’ (Female, 29)
No
External Evaluation Report: UCM ‘Big Draw on Tour’
P a g e | 21
Photo: Father and Daughter Working on a Drawing Activity at MoAA3
17% (n=11) of respondents mentioned an aspect of ‘drawing’ as what they liked most about the ‘Big Draw on Tour’. Responses often mentioned the ability to be creative and experience new aspects of drawing: ‘The feeling when I was drawing -‐ I was able to express myself!’ (Female, 10) ‘Adapting artefact image into own creature’ (Female, 41) ‘I love drawing and learning new things and it was great to be able to do both’ (Female, 29) Visitors mentioned their appreciation for learning aspects of ‘drawing’: ‘I really enjoyed being taught proper techniques and having the chance to try them out with guidance and support -‐ and enthusiasm!’ (Female, 39) ‘Drawing items from museums to add to the large picture and [I am] inspired to hang the drawing material from the ceiling.’ (Female, 37) ‘Having a different choice of drawing materials.’ (Female, 43)
3
Museum of Archaeology and Anthropology
No
External Evaluation Report: UCM ‘Big Draw on Tour’
P a g e | 22
Photo: Drawing from a Participant at MoCA 4
Aspects of the ‘museum’ were also mentioned as the best component by 24% of the participants (n=15). These responses focused on the design and arrangement of the museum. For example: ‘Being able to sit on a cushion! And to be able to take your time.’ (Female, 51) ‘Seeing exhibits with a fresh eye and feeling the whole atmosphere of the museum becoming relaxed and lively.’ (Female, 43) ‘Opportunity for the children to see more of the museum.’ (Male, 43) A smaller group (10%) of the participants indicated their appreciation for the ‘staff and volunteers’ of ‘Big Draw on Tour’. ‘A young lady called Sally and her colleague spent some time talking to my 4 year old daughter Fion and I thought this was lovely and very special for my daughter so thank you.’ (Female, 43) ‘The motivation of the volunteers’ (Female, 45) Overall, participants were very positive about their involvement. The majority enjoyed the unique experience of the ‘Big Draw on Tour’ programme, especially the opportunity to express themselves in innovative ways.
4
Museum of Classical Archaeology
No
External Evaluation Report: UCM ‘Big Draw on Tour’
P a g e | 23
‘Least Liked Aspects’ of Big Draw on Tour Programme from Participant Perspective This section covers what visitor respondents listed as the ‘least liked aspects’ of BDoT. The largest category of responses (28%) did not find anything wrong with the programme or their experience. Whereas, the other responses were dispersed over six different categories: ‘directions’, ‘space’, ‘participation’, ‘duration’ and ‘staff’. From the table below, it can be seen that some of the comments regarding the ‘least liked’ aspects are not really negative (e.g. individuals would have liked to continue on with the activity for a longer period of time) or are somewhat out of the control of organisers (e.g. other parents not participating in art work). However, other comments are clearly negative and within the reasonable control of organisers, including the comment about directions, crowding and feeling ‘bossed around’. Category Directions
‘Least Liked’ Aspects of ‘Big Draw on Tour’ Percent and Number Examples of Responses ‘Wasn't clear that aim was to create character for the wall’ (Male, 47) 16%, n=4
‘Hard to Find’ (Female, 40) Space
12%, n=3
‘The crowdedness of the room.’ (Female, 20)
28%, n=6
‘There weren’t any other parents having a go” (Female, 26) ‘That it worked brilliantly for my 7 year old; not so well for my 4 year old; but that is no reflection on the activity; more about age and concentration levels’ (Female, 43) ‘If I'm honest, I would really liked to have done it myself (as an adult!). It's great there are so many things for kids but sometimes feel adults miss out on creative opportunities.’ (Female, 37)
Duration
16%, n=4
‘It was maybe too short!?’ (Female, 42) ‘Not enough time; could have spent a whole day’ (Female, 42)
Staff
4%, n=1
‘Being bossed around by the volunteers’ (Female, 45)
Participation
No
External Evaluation Report: UCM ‘Big Draw on Tour’
P a g e | 24
Self-‐Reported Gains and Losses from Attending the Event Visitor respondents were asked in an open-‐ended question what, if anything, had they gained from attending the ‘Big Draw on Tour’ programme. As a result of participating, many of the respondents (54%, n=20) self-‐reported improvements in their drawing (or confidence drawing):
‘Got some good tips about shading with pastels specially when drawing faces’ (Female, 29) ‘Encouragement & advice to improve drawing skills’ (Male, 47) ‘New ideas and renewed interest in drawing & ideas for visits to museums.’ (Female, 42) Within this category, a few of the participants perceived changes in children’s confidence levels. The first example below is actually self-‐reported by a child respondent to the survey: [A benefit of participating in the workshop is] ‘Confidence to show my art’ (Female, 10)
Another response in this category was from a mother who reported on her child’s increased self-‐confidence resulting from the BDoT workshop they attended. [The event was a] booster for my daughter's confidence: she was very proud that her drawings became part of the art project.’ (Female, 45)
No
External Evaluation Report: UCM ‘Big Draw on Tour’
P a g e | 25
Photo: Two adolescent girls work on developing ‘Dream Machine’ using ideas from artefacts in Whipple5 collection
Other parents reported their children learned new artistic skills: ‘Children learned about new style of drawing.’ (Female, 38) 5
Whipple Museum of the History of Science
No
External Evaluation Report: UCM ‘Big Draw on Tour’
P a g e | 26
‘I think my children learnt about how colours blend and my younger daughter learnt about new textures.’ (Female, No Age)
Additionally, 27% (n=10) of the participants commented that they had increased appreciation and enthusiasm for the art and drawing: ‘Confidence and enthusiasm -‐ my son was so enthused that we had to go and buy art supplies so he could keep going.’ (Female, 39) ‘An appreciation of drawing and ideas on how to look at art in a different way’ (Female, 41) These results show that workshop participants feel they gained something valuable from the BDoT project. Furthermore, parents felt their children gained important benefits in confidence, learning and art appreciation.6 Photo: Children at Sedgwick Museum Gathered to Work on a Drawing Activity
6
However, these are self-‐reported claims, or claims by parents for children, of increased knowledge and appreciation. Therefore, the validity of such feedback-‐only claims cannot be guaranteed based on this aspect of evaluation data.
No
External Evaluation Report: UCM ‘Big Draw on Tour’
P a g e | 27
Impact Evaluation Results: Thought-‐Listing Data
This section of the report presents the results of analysis of pre-‐ and post-‐participation questionnaire thought-‐listing (‘list what comes to mind’) data relating to key intended outcomes.
Participants’ Thinking about ‘Drawing’ Photo: Adult Visitors to BDoT at MoCA Working with Sculptures
This item asked participants to list the words that came to mind when they thought of ‘drawing’. 50% of responding visitors demonstrated positive impacts to their attitudes and knowledge regarding ‘drawing’, whereas 39% demonstrated no change and 12% demonstrated negative change. For instance, positive change individuals included changing thinking about art or improving concept knowledge: Positive Impact: ‘Drawing’ Pre-‐Visit Survey Response
Post-‐Visit Survey Response
Capturing of an object
Capturing an object/person on paper
Pencils
Pencils; colour; pictures
Shapes; colour; designs
Freedom to express non-‐written ideas. Shapes; colour; form
No
External Evaluation Report: UCM ‘Big Draw on Tour’ Communication
Colours; different types of pencils; rubbers; objects; nature
P a g e | 28
A picture is worth a 1000 words; Communicating ideas; Abstract Drawing as an expression of your inner freedom; pencils; erasers; colours; nature; children
I' m rubbish but my daughter loves it
Fun; artistic; My children love doing it!
Black and white
Black and white; perspective; still live; charcoal; pencils
Photo: Drawings of Sculptures by Participants at MoCA
Whereas, 12% of visitors showed possible negative impacts, the only unambiguously negative case is the first one listed below, in which ‘difficult’ appears for the first time in the post-‐visit survey: Negative Impact: ‘Drawing’ Pre-‐Visit Survey Response
Post-‐Visit Survey Response
Pictures; museums; painting
Pencils; crayons; difficult
Children; Sketching; Colouring
Children
No
External Evaluation Report: UCM ‘Big Draw on Tour’
P a g e | 29
N/A
GCSE
(*respondent wrote this response)
The table below presents concepts that were prevalent in either the pre-‐visit or post-‐visit survey. Positive numbers in the ‘change’ column represent concepts that increased in prevalence in participants’ thinking about drawing from pre-‐ to post-‐workshop visit. Frequency of Thoughts (Drawing) Concept Pre-‐Visit Post-‐Visit Change Fun 7 8 +1 Children 4 7 +3 Colour 5 6 +1 Expression 1 2 +1 Ideas 0 3 +3 The changes in the thought-‐listing data focused on ‘drawing’ suggest positive development from pre-‐ to post-‐BDoT visit. For post-‐visit, ‘drawing’ is more connected to ‘children’ and ‘ideas’ in visitors’ minds.
Participants’ Thinking about ‘Museums’ The results showed clear patterns in the kinds of thoughts participants associated with museums pre-‐visit. The data below indicates a positive starting position in participant’s knowledge pre-‐visit, whilst the post-‐visit data illustrates the majority of visitors adjusted their responses to include education aspects and a place for relaxation. 50% of the respondents demonstrated positive change, whilst 15% of the respondents experienced negative change. The table below depicts individual participant’s change in attitude and perception: Positive Impact: ‘Museum’ Pre-‐Visit Survey Response
Post-‐Visit Survey Response
Artefacts
Artefacts; exhibitions; archaeology; old things; collections of things
Models; excited; new things; Exciting place great activity
Lovely paintings and display; good place to relax; learning new things A wonderful experience. A great place to learn interesting
Ideas; people; history; nature; beauty
Beauty; history; big rooms; darkness; learning; [I] never get tired of visiting a museum
Contemplating sharing learning beautiful celebration
Peaceful; learning; collective memory; responsibility; family; heritage; important; civilising
Quiet
Quiet; interesting; children
No External Evaluation Report: UCM ‘Big Draw on Tour’ P a g e | 30 Visitors who displayed possible negative impacts on their thinking of museums revealed this regression of thinking most frequently by the exclusion of previously listed concepts, rather than the addition of negative concepts. This may suggest that visitors are fatigued or chose not to respond for unclear reasons. Negative Impact: ‘Museum’ Pre-‐Visit Survey Response
Post-‐Visit Survey Response
Excited; big; learning; calm; tranquillity; interesting things
Place of quiet; interest
Art; archaeology; fossils
Art; architecture
The concepts listed in the table below show visitors’ aggregate thinking about ‘museums’ before and after their BDoT experience. The prevalence of concepts in the pre-‐visit or post-‐visit survey represents how concepts changed in visitors’ thinking about drawing. Negative concepts (such as ‘boring’) decreased in frequency, whilst positive concepts (such as learning) increased in frequency. Frequency of Thoughts (‘Museums’) Concept Pre-‐Visit Post-‐Visit Change History 10 5 -‐5 Interesting 6 9 +3 Things 8 4 -‐4 Artefacts 4 6 +2 Old 4 2 -‐2 Learning 5 11 +6 Boring 2 0 -‐2 These responses are of particular interest as they suggest that the BDoT programme has been influential in changing the common conceptions of museums as ‘old’, ‘boring’ facilities that are often silent and only contain ‘things’ from ‘history’. These data illustrate a change in visitor perception to a more favourable attitude, viewing museums as ‘interesting’ places as well as where ‘learning’ takes place through interaction with ‘artefacts’.
No
External Evaluation Report: UCM ‘Big Draw on Tour’
P a g e | 31
Photo: Romsey Mill Visitors Drawing Artefacts from Museum Collection
Participants’ Thinking about the ‘University of Cambridge’ There was evidence of transformations in visitors’ thinking from pre-‐visit to post-‐visit regarding the University of Cambridge. The majority of visitors (44%, n=11) showed a pattern of positive change in which they focused more on the beauty of the university buildings and the learning aspects of the university. Positive Impact: ‘University of Cambridge’ Pre-‐Visit Survey Response
Post-‐Visit Survey Response
Beautiful; illustrious; inspiring; desirable
Illustrious; excellent; beautiful; history; desirable; privilege; inspiring; generous (with sharing resources with public for wider learning-‐ e.g. museum collections)
Being a student
Memories; amazing building; brain food
Brains; education
Learning; brains; brilliance
Excellence
Centre of Excellence; beautiful buildings; clever people; great ideas
Colleges
Self-‐satisfaction; Insularity; Great cultural assets
No
External Evaluation Report: UCM ‘Big Draw on Tour’ Resources
P a g e | 32
Highly regarded; Close Resource; rich
Key patterns are in evidence in the data table above. Firstly, there is clearly a value-‐laden ambivalence in respondents’ thinking about the University of Cambridge. For example, the first response above presents a positive post-‐visit set of thoughts, yet this set includes the negatively coded term ‘privilege’ as well (also ‘rich’ mentioned in the final row above). This same individual example evinces a positive change in thinking about the University of Cambridge, which could be reasonably tied to the BDoT programme. This can be seen by the addition of this new thought about the University of Cambridge: “generous (with sharing resources with public for wider learning-‐ e.g. museum collections)”. This result suggests the goal of improving the relationship between the University of Cambridge and broader public was met (at least in part). Although negative words are interspersed throughout the comments (such as: rich, archaic, toffs and boring), only a few individuals (n=4) evinced possible negative impacts on this dimension. Only 2 of these four were unambiguous cases of negative change. Negative Impact: ‘University of Cambridge Pre-‐Visit Survey Response
Post-‐Visit Survey Response
Home
Establishment for the elite
Learning
Aaaaahhhhh…worries me
Using the data from this question, it is possible to get a general idea of what visitors think of the University of Cambridge by aggregating the listed concepts. The concepts, listed in the table below, depict what visitors’ thought of ‘University of Cambridge’ pre-‐visit and post-‐visit. Prevalent concepts in visitors’ thought listings from pre-‐visit to post-‐visit are also shown in the next table. The change in prevalence is indicative of the impacts of visitors’ experiences and any attitude changes. Frequency of Thoughts (‘University of Cambridge’) Concept Pre-‐Visit Post-‐Visit Change Learning 6 4 -‐2 Work 4 3 -‐1 Education 6 3 -‐3 Knowledge 4 1 -‐3 Public 0 2 +2 Beautiful 5 4 -‐1 Research 5 2 -‐3 Academic 2 0 -‐2 Hard 2 0 -‐2 Science 1 3 +2 Festival 0 3 +3 Architecture 0 3 +3
No External Evaluation Report: UCM ‘Big Draw on Tour’ P a g e | 33 These aggregate data show shifts in visitors’ thinking, with ‘Festival’, ‘Architecture and ‘public’ more closely associated with the University of Cambridge post-‐visit (and ‘hard’ and ‘academic’ becoming less prominent associations). This data implies visitors partaking in the BDoT are developing a more favourable relationship with University of Cambridge. Following the event, more visitors imply that they believe the university is a ‘public’ institution, not just a place for ‘research’ and ‘education’.
Case Studies: New Audiences The ‘Big Draw on Tour’ programme offered UCM collections an opportunity to reach new audiences. In this case, visitors were sometimes completely new to University of Cambridge Museums or just new visitors to a particular museum within the set of UCM collections. These visitors are of particular interest as they may shed light on how to reach and retain these visitors. Overall, 13.5% (n=10) of the participants were either new to University of Cambridge museums or were visiting the hosting museum for the first time. Two of these participants’ stories have been select as ‘case studies’, which are described below. Each of the participants presented different experiences. In Sara’s case (Female, 41, A-‐Levels), she visited with her 3-‐½ year old daughter, whom she cares for full-‐time, with the intention of having a nice, entertaining day out. Sara heard about the event through the website and, although she was previously aware of a number of University of Cambridge Museums, had not visited any of the University of Cambridge Museums. Sara also spoke with members of staff, however she spoke primarily about her daughter’s interest in art. Sara asked the staff how she might be able to ‘encourage her [daughter] to maintain an interest [in art]’. When asked to indicate what comes to mind regarding the three thought-‐listing topics, Sara was already positive regarding the topics: Pre-‐Visit: DRAWING
Pre-‐Visit: MUSEUM
Pre-‐Visit: CAMBRIDGE UNI.
Recreating what you see in your own unique way
Fossils! Exhibits. Interaction. Learning.
Knowledge. Excellence.
Whereas, in her answers below, when compared with her pre-‐visit data, demonstrate little to no change post-‐visit.
Post-‐Visit DRAWING
Post-‐Visit MUSEUM
Post-‐Visit: CAMBRIDGE UNI.
Creative expression on paper
Fossils; interaction; exhibits; learning; interest
Quality; excellence
No External Evaluation Report: UCM ‘Big Draw on Tour’ P a g e | 34 However, Sara does state that best part of the activity was the ‘opportunity to share ideas with my daughter and seeing her interest in both drawing and the museum’. Sara indicated that she thought the building was amazing and interesting and ‘strongly agreed’ that she would like to do something like this again. Sara ‘strongly disagreed’ that the activities were difficult and ‘strongly agreeing’ that she was able to actively participate in the activity. As a new visitor, Sara seems very likely to be positively predisposed to visiting the museum or programmes held by the museum again in the future. Our second case study, Jenny, is a frequent museum visitor. She is a 43-‐year-‐old female who has a post-‐graduate degree and currently cares for her home and family. Although this was Jenny’s first time to the Cambridge Zoology Museum, she has visited several other museums within Cambridge: Fitzwilliam Museum, Kettle's Yard, Polar Museum, County Folk Museum, Botanic Gardens and the Cambridge Museum of Technology. She is also a frequent visitor of museums outside of Cambridge, such as the Bethnal Green Museum of Childhood, York Castle Museum and Norwich Castle Museum. Jenny was visiting the Zoology Museum with her family as a way to spend time together and have an entertaining day out. Jenny chose to ask for just a bit of guidance ‘about the drawing materials’ from the staff, but disagreed that she required further guidance nor did she find the activity difficult. Photo: Young Child Sketching a Skeleton in the Zoology Museum.
Jenny’s responses to the pre-‐visit question on what she thought of drawing, museums and the University of Cambridge are provided below:
No
External Evaluation Report: UCM ‘Big Draw on Tour’ Pre-‐Visit DRAWING
Pre-‐Visit MUSEUM
Pre-‐Visit: CAMBRIDGE UNI.
Concentration
Excited
Colleges
P a g e | 35
In her post-‐visit survey, Jenny stated she ‘had not visited the Museum of Zoology before and had no idea it would be so absorbing.” She agreed that she enjoyed the activity and strongly agreed that she would like to do something like this again. Therefore, we could expect that Jenny would show an overall positive impact in her thinking, which is indeed seen below: Post-‐Visit DRAWING
Post-‐Visit MUSEUM
Post-‐Visit: CAMBRIDGE UNI.
Children getting lost in their own world
Stimulation; the chance to reflect and slow down
Self-‐satisfaction; Insularity; Great cultural assets
Despite the fact that Jenny is a repeated museum visitor, her visit to the Zoology Museum was a first and made a strong impression. When Jenny was asked what she liked most about the activity, she stated that she enjoyed ‘seeing exhibits with a fresh eye and feeling the whole atmosphere of the museum become relaxed and lively”. Jenny found the building ‘more accessible than [she] had previously assumed and gained and appreciation for the difficulty of ‘drawing bones’. Overall, the greatest impact on Jenny appears to be an appreciation for both the arts and for University of Cambridge Museums, as she states in her final comments, ‘my daughter is still raving about [‘Big Draw on Tour’], I will definitely look out for the Big Draw again.’ In the case studies above, both Jenny and Sara were familiar with the museums and were previously familiar with the ‘Big Draw on Tour’. The project clearly had an impact on Sara’s thinking. However, she did seem to gain more from her discussions with staff and is dedicated to providing access to arts for her young daughter. Jenny showed some changes in her thinking and increased support for University of Cambridge Museums. If ‘Big Draw on Tour’ were to consider the aspects of the programme that could better suit new or inexperienced visitors, ‘direction and assistance’ should be a top priority. Both Sara and Jenny sought assistance during their activity and cited it as an important part of their experience.
Conclusion The empirical evaluation evidence adduced in this report indicates that the Big Draw on Tour programme was successful at: •
Reaching new audiences
No •
External Evaluation Report: UCM ‘Big Draw on Tour’
P a g e | 36
Fostering new cross-‐collection visiting
•
Engaging visitors with new ideas about drawing
•
Fostering more positive perceptions of museums and the University of Cambridge
Nevertheless, while the Big Draw on Tour succeeded in these aspects, there is still room for improvement. This external report highlights the general success of this project from an audience perspective, with important development points also identified for future implementation to enhance visitor impacts.
Future Evaluation Recommendations The organisation and availability of personnel for data collection needs improvement for future evaluations, given that the sample size for this pilot study was substantially below target. However the survey questions did capture key information, including demographics, feedback and impacts. The survey questions can be adapted in small ways for new initiatives or programmes (e.g. the target concepts addressed by the thought-‐listing items can be changed to reflect the intended impacts of the new initiative). Procedures This exploratory project tested the feasibility of using iPads synced with online survey forms to increase the efficiency of evaluation data collection by UCM. The use of this digitally enhanced approach faced challenges in this pilot project, most notably when planned support in the form of individuals available to administer the survey data collection unfortunately did not materialise. This resulted in lower sample sizes than planned, also highlighting the need for more effective data collection organisation in future evaluation activities. Recommendation: To ensure effective data collection in future evaluation, dedicated data collectors should be used to administer the surveys. These individuals could be employed on a temporary basis or recruited as volunteers with training delivered either by the external evaluator or by UCM staff using evaluator-‐provided resources. Alternatively, the data collection activities could be organised by asking the external evaluator to lead the co-‐ordination of this aspect of on-‐going evaluation activities. Random sampling can be employed to determine which site(s) will receive the most attention in terms of data collection. However, in order to allow for collection of impact data, a minimum of two data collectors is recommended at each selected site. Ideally, a third person would be available, either on call or on site in case of unexpected cancellations by data collectors. In addition, in order to allow for variability in visitors’ willingness to participate in evaluation data collection on site and their level of comfort with new technology, it is important to have back-‐up options available to facilitate the participation of those unable or unwilling to provide data immediately using iPads: Recommendation:
No
External Evaluation Report: UCM ‘Big Draw on Tour’
P a g e | 37
To maximise data collection within time-‐sensitive event contexts with heterogeneous audiences, it is recommended that: 1. Hard copies of the evaluation forms should be available for individuals preferring not to give feedback using the iPads or verbal responses (or in case of the need to collect data from an entire group, but only having one device). 2. Be prepared to collect email addresses from anyone saying they do not have time to complete the questionnaire at that time. These individuals can then be sent the ‘Post-‐visit Only’ Feedback Form later that day or early the next day. Survey Instrument Carrying forward the survey instrument developed for this project into future evaluation activities at UCM sites, it is recommended that UCM staff use the current set of questions and the general approach of seeking to collect both pre-‐ and post-‐visit data where possible. Whilst the use of both pre-‐visit and post-‐visit surveys may require additional data collection resources, it is essential for capturing the impacts of UCM programmes. Specifically, thought-‐listing questions, such as ‘What comes to mind when you think of drawing?’, are critical indicators of impact. The target concepts for such thought-‐listing measures (‘drawing’, ‘museums’ and ‘University of Cambridge’) may be adjusted in future evaluations to better reflect a particular programme’s objectives, but it would be useful to identify one or two concepts of shared interest across the UCM collections that could be employed in thought-‐listing data collection across several programmes to provide data to compare impact trends. For example, the use of the question, ‘What comes to mind when you think of the University of Cambridge?’ could elicit distinctions between Fitzwilliam (art) and the Zoology museum (science) visitor impact trends. Indeed, it is difficult to develop a completely clear picture of the contribution of new interventions, such as BDoT, without robust baseline data on the current visitor demographic profiles and impacts under normal conditions across the UCM collections. Recommendation: Gather representative visitor profile and impact data from across each of the University of Cambridge collections in order to be able to: 1. Fully ascertain the effectiveness of new interventions 2. Evaluate current strengths and weaknesses (in terms of the main Arts Council objectives) across the UCM system to target the interventions where the need is greatest. The Evaluation Approach Finally, it is worth highlighting the strengths and weaknesses of the research approach that was employed for this evaluation. A survey-‐based system was selected in order to allow University of Cambridge Museums to lower costs and manage data collection (as well as limited data analysis activities) ‘in house’. The survey methodology allows direct measurement of impact and the efficient collection of feedback data from representative samples of participants both during normal circumstances at UCM collections and during special events like BDoT. With surveys, expertise can be used on a one-‐time basis to guide the survey design, while the individuals conducting the data collection only need very basic training in survey research. However, this methodology is limited in two major respects:
No •
•
External Evaluation Report: UCM ‘Big Draw on Tour’
P a g e | 38
The breadth and representativeness of this method sacrifices depth when compared to alternative methods, such as ethnographic research. In some ways, the individual voices and journeys of participants are muted by such survey methodology when compared to more depth-‐oriented approaches. Albeit, the collection of pre-‐ and post-‐visit qualitative data analysed ideographically (at the individual level) does mitigate this limitation to the greatest extent possible. The researcher does not have access to the full context and ‘feel’ of the event, due to not being physically present during data collection. Therefore, the researcher only has access to the experience of the events through the lens of visitors’ survey responses. For example, they are not able to contrast the aspects of the experience highlighted by visitors with the objective characteristics of the engagement opportunities.
A previous pilot research project conducted at the Fitzwilliam Museum focusing on a set of socially excluded young mothers engaging with the museum for the first time as adults highlights the difference between what is achievable with the survey methods employed in the present case and what is possible with a smaller scale depth study. The ethnographic study allowed for long-‐term tracking of the development of the young mothers, giving them voice in a way that is not feasible using larger scale survey data collection.7 Of course, large-‐scale representative survey data is very important for establishing the overall patterns of visitor characteristics, feedback and impact. At the same time, depth-‐oriented ethnographic evaluation offers unique insights that can inform practice and tell the stories of key visitor populations in a robust and colourful manner. As such, a combined approach is likely to be the most effective from the perspective of developing a long-‐term research and evaluation strategy for UCM collections. Recommendation: To establish both the breadth and depth of impacts achieved by UCM collections, a combination of survey and ethnographic approaches is recommended within the context of a long-‐term research and evaluation strategy and framework. Such a combination would meet the full range of visitor impact evaluation and audience research needs relevant to UCM engagement activities.
Recommendations for Future Programmes Marketing, Brand Recognition and Community Outreach The data showing high levels of variability in awareness and visiting of different museums indicate challenges brand awareness across the UCM partnership. Awareness of some of the embedded museums is particularly limited at this stage. The level of cross visiting between UC museums is even more limited. The BDoT programme reached new visitors and promoted cross visiting, suggesting that future interventions should build on the Big Draw on Tour’s strengths as they seek to extend awareness and participation in UC museums beyond current audiences. Such audience development can be challenging and resource intensive, requiring new approaches and fresh 7
The earlier ethnographic report is available at this weblink: http://www.fitzmuseum.cam.ac.uk/dept/education/researchandpublications/documents/FamilyOutreachPilotEvaluat ionMay2010.pdf
No External Evaluation Report: UCM ‘Big Draw on Tour’ P a g e | 39 ideas. However, this and the previous research at the Fitzwilliam Museum referenced above (Jensen 2013) have demonstrated that successfully drawing in new and diverse visitors to UC museums is possible. Recommendation: Address clear limitations of current marketing/outreach approaches, building on evaluation evidence from BDoT, Romsey Mill and other programmes’ successes to develop new and diverse audiences. As evidenced by the ‘who is running this event?’ question responses, the BDoT programme did not present an integrated brand to visitors. In order to make the UCM brand more prominent in future events, new tactics will be needed. Perceptions of the University of Cambridge This research suggests that many in the Cambridge community hold at least some salient negative views about the University of Cambridge. Such negative attitudes towards the most important cultural institution in this region are concerning from a broader social cohesion perspective, but they also portend particular problems for UC museums as they attempt to reach out to new and more diverse audiences. While well-‐designed UCM outreach programmes are able to combat negative views about the university, a key challenge facing such outreach programmes will be the potential negative ‘baggage’ that the University of Cambridge brand carries with it for a substantial minority of local community members. The kinds of negative views expressed in the pre-‐visit data for this study could prevent some families in the Cambridge community from visiting UCM sites. This challenge could be exacerbated if the ‘University of Cambridge’ branding gains greater prominence through the UCM partnership programmes in coming years.
Recommendation: Given substantial minority of negative attitudes towards the University of Cambridge, careful consideration of branding of new UCM partnership programmes aimed at reaching new and diverse audiences is warranted.
While UCM programmes may benefit public engagement with the University, prominent University of Cambridge branding at the outset may hamper such outreach. As such, it is recommended that: Recommendation: UCM should consider using branding distinct from the University of Cambridge for preliminary marketing to new audiences. Once audiences have been successfully engaged, the University of Cambridge branding could be more prominent. This two-‐stage approach to reaching new audiences could take the following form:
No
External Evaluation Report: UCM ‘Big Draw on Tour’
P a g e | 40
Stage 1: Bringing new audiences into at least
one UC museum. The outreach marke{ng should employ broadly accessible thema{c branding (ideally selected based on qualita{ve front-‐end evalua{on research with target audiences).
Stage 2: Once these audiences have
been successfully engaged by at least one UC museum, this can be leveraged into cross promo{on of other UC museums. At this point, the University of Cambridge branding could become more prominent.
This approach takes advantage of the finding of 100% enjoyment of the Big Draw on Tour activities. 97% show interest in taking part in further similar activities. That is, the present case demonstrates the capacity of well-‐planned UCM interventions to create interest in expanded participation that further extends engagement with UCM collections.
No
External Evaluation Report: UCM ‘Big Draw on Tour’
P a g e | 41
References
Jensen, E. (2011). Upstream public engagement at the zoo. Durrell Wildlife Conservation Trust and the University of Warwick. Jensen, E. (2013). Re-‐considering ‘The Love of Art’: Evaluating the potential of art museum outreach. Forthcoming in Visitor Studies. Jensen, E. & Holliman, R. (2009). ‘Investigating science communication to inform science outreach and public engagement’. In R. Holliman, et al. (Eds.), Investigating science communication in the information age: Implications for public engagement and popular media. (Oxford University Press) University of Cambridge Museums. (2008). ‘Making a difference: Community learning and outreach projects in the embedded museums of the University of Cambridge funded by Renaissance 2006-‐2008’. University of Cambridge: Cambridge. Available at: http://www.mla.gov.uk/what/programmes/renaissance/regions/east_of_england/news/~/me dia/East_of_England/Files/Making_a_difference.ashx