Evaluation of Mali Health’s outreach efforts leading up to October 2013
Based on a supporter survey conducted 1-8 October 2013
Mali Health Organizing Project 1
Report prepared by Stephen Muse, Mali Health Communications Manager October 2013
Introduction & Key Results In October 2013, Mali Health staff designed a brief electronic survey with the purpose of gathering feedback from supporters of all types on the content and frequency of outreach they prefer to see. This data, once gathered, would inform staff decisions regarding the content, frequency, and method of outreach to supporters going forward with the goal of improving supporter engagement with and interest in the organization and, ultimately, their likelihood of making a financial contribution. Data gathered through the survey paint a broadly positive picture of Mali Health’s communications and outreach strategies leading up to October 2013. In general, supporters feel
the methods of outreach used are welldesigned, frequent enough without being too often, and provide engaging and important content. Seventy-two individuals participated in the survey, the large majority of whom have known about Mali Health (and presumably have followed its outreach efforts) for over a year. In general, participants enjoy content focused on the organization’s projects and the people who benefit from those projects, and they like a more personal presentation, including pictures and anecdotal stories. Mali Health has room to improve in terms of showing its transparency to supporters, but donors feel that their contributions do have a direct positive impact on the ground in Mali.
2
Methodology The survey was open to the public for one week, from 8:00am on Tuesday, 1 October, to 8:00pm on Tuesday, 8 October. Invitations to participate in the survey were first announced to the public via an email to all recipients of the organization's quarterly newsletter, as well as public posts on Mali Health's Facebook and Twitter pages at 8:00am on Tuesday, 1 October. The duration of the survey's availability, it's purpose, and its openness to all supporters were made clear both in the email and in the introductory text at the head of the survey page. Reminders about the survey's closing date were posted on Facebook and Twitter throughout the week, as well as a second email to all recipients on Monday, 7 October. In order to encourage the most supporters to participate, the survey was hosted using a publicly-accessible Google Form, the link to which was provided in all emails and social media posts. The survey was designed to require approximately five minutes to complete, no question was mandatory, and participants were given the option (but not required) to include their name and contact information, all in order to increase the likelihood of participants completing as much of the survey as possible without feeling any
aggravation or frustration at the survey's length, restrictiveness, or invasiveness. All of these features of the survey were also included in all emails and social media posts. The format of questions used varied throughout the form and included multiple choice, rating on a 3- or 5-point scale, and open-text fields. The format of each question was chosen in order to elicit the most revealing information pertaining to that question. Space was also included at the end of each of the two pages of the survey in order to allow participants to include comments that may clarify or supplement their answers throughout the form. The survey was split into two pages. The first page consisted of questions regarding Mali Health's outreach efforts over the recent months, including via the website, email, physical mailings, and social media. The second page participants saw depended upon whether they had made a financial contribution to Mali Health at any point in the past. A "Yes" answer led to a page that included questions about their donor history and their motivation behind donating to Mali Health. A "No" answer led to a page that asked for their motivation for supporting the organization.
3
Results Of the five methods of outreach - email, website, physical mail, Facebook, and Twitter - email received the highest approval ratings from all respondents 88% ranked it in the top two slots (Very Interesting and Usually Interesting). The website also received high marks; though 40% of respondents said they do not check the organization's website, of those who do, all rated it within the top two slots. Even fewer participants make regular use of Facebook or receive the physical mailings (less than 50% each), but again, those who do use these methods mostly approve of the frequency and content in each. One respondent commented that “I would enjoy seeing more content on Facebook.” Threequarters of respondents do not follow Mali Health on Twitter. Of those few who do, there is a fairly even split between those who feel it is frequent enough and those who want to see more, and there is an even spread on the ratings of its content (from "very" to "not at all" interesting). What can be determined from this data is that Mali Health has found a fairly successful formula for the frequency and content of its outreach, and that direct emails are still the most popular way that supporters interact with the organization. Another participant offered this comment, which is representative of the majority of qualitative feedback: “I think you send out the perfect amount of outreach through email newsletters -- just often enough that I want to keep
helping, but not so often that the content is repetitive and I delete it.” Data from question five is more revealing, as all respondents were able to provide feedback regardless of whether or not they utilize a particular service. Nearly two thirds of all participants identified that they like to see content about Mali Health's programs, as well as stories about the beneficiaries of those programs, in organizational communications. One respondent noted that she would “like to have more ongoing updates on specific projects” in order to track how a project grows and changes over time. Content about Mali in general and about Mali Health staff received lukewarm reception, and participants did not indicate much interest in content related to global health outside of the Mali Health context. Therefore, the organization should make every effort to maximize content relating to its programs and beneficiaries, with occasional supplements focusing on the Mali Health staff and the country in general. When supporters engage with our content, they most enjoy a humanized presentation - over 50% of respondents said they "love" to see pictures/videos and personal stories in our outreach, and another 30%+ "like" to see these things. One participant specifically noted that a short video would be a great way to engage supporters, and another said, “gorgeous photos always catch my eye.” 4
Supporters also like to see hard data concerning the organization's impact (36% said they "love it" and 50% said they "like it"). Several respondents included comments in their survey encouraging a balance between personal stories and numerical data. Participants were decidedly against the idea of a more interactive form of content presentation (such as games or contests), with over half giving it the lowest interest rating. It is clear from this information that the organization's outreach should rely heavily on appealing graphics and personal anecdotes, and these should be paired from time to time with more concrete data to show the breadth of Mali Health's impact beyond the individual stories. One supporter suggested that email newsletters could be sent out that include more humanized presentations, and these could be supplemented with quarterly reports that are more datadriven. In Question 7, participants were asked to offer their interest in a number of potential new types of content - a webinar with Mali Health staff, notifications about Malian cultural events in the Boston area, release of the organization's quarterly report, a Twitter chat with a staff member, and birth announcements from the new maternity ward Mali Health recently inaugurated. None of these ideas received overwhelming enthusiasm. Cultural events and the quarterly report garnered the highest
interest, with one-quarter of respondents saying they would "definitely" be interested in these types of outreach. As expected with the low rate of Twitter use among respondents, the proposed Twitter chat received the lowest rank, with 55% saying they were not at all interested. Though it may be worth exploring some of these or other interactive opportunities in the future, it is clear that our supporters are, for the most part, satisfied with regular updates about our work on the ground and are not necessarily in need of these new attempts at engagement. A participant did offer this advice: “I’d like to hear from more voices than [Executive Director] Kris [Ansin].” Therefore, it is recommended that Mali Health continue to promote outreach written by other Mali Health staff members, as it has done with recent blog posts from Programs Director Mariam Fofana Diallo, Medical Advisor Dr. Diakaridia Traore, and Community Health Worker Coordinator Djibril Traore. All respondents were asked why they support Mali Health as an organization; numerous answers were offered, as well as space for a write-in response, and participants were able to select as many as applied. Seventy-eight percent of participants selected “I support its mission,” the most popular response to this question. The second most popular selection was a “personal connection” to Mali Health (69%), not surprising given 5
that 70% of respondents said they first heard about the organization through a personal connection. “It is making a difference in Mali” rounds out the top tier of motivations with 67% of supporters. Mali Health’s work with women, work with children, work in health care, and status as a small organization make up the middle range of answers, garnering 40-54% of respondents. Professional connection and a focus on the African region were the least popular motivations for our supporters. Interestingly, there was a disparity between donors and non-donors on several responses: 83% of donors selected “Support of its mission” and 70% chose “It is making a difference in Mali,” while non-donors included those responses at a rate of 61% and 56%, respectively. At the same time, over 60% of non-donors identified our work supporting women as one of the key factors behind their support, which only 46% of donors did. Non-donors were also much more likely to cite a “professional connection” as a motivation (28%, over donors’ 17%), while donors vastly outweighed nondonors when citing a personal connection (80%-39%, respectively). It is admittedly unsurprising that a person is more likely to make a financial contribution to an organization when she feels she has a personal connection to that organization. However, it is interesting to note that where donors have a broader perspective of the organization (“I
support its mission”, “It is making a difference in Mali”), non-donors are drawn to a more specific aspect of our work (such as our support for women). It is possible to surmise that as one gains a broader perspective of the organization, its work, and its impact, one is more willing to make a financial contribution. Two of the final questions on the survey asked supporters to rate the veracity of two statements on a five-point scale (a 5 being the highest). The first: Mali Health is a transparent organization. The second: Donations made to Mali Health will help improve lives in Mali. Both received very high marks among all respondents. A combined 74% rated our transparency in the top two slots, and 84% rated donation impact in the top two slots. Non-donors were more likely to leave these questions blank than donors, and one commented specifically that she left the question regarding transparency blank because she doesn’t “know enough about [the organization] so far.” On the first question, only two respondents (one donor and one non-donor) gave a score below 3. On the second question, no donors, and only one non-donor, gave a score less than 3. It is clear from this data that Mali Health can do a better job of proving itself to be transparent to average supporters, and that, although there is room to improve, the organization has done a good job of showing the impact of donations on the ground in Mali. One supporter wrote, 6
“I’d like to hear more about how our donations can support specific projects,” while another praised our campaign around the construction of the maternity ward for showing supporters how donations were specifically being used to improve the health of expecting mothers.
“Love the success stories and blurbs about [Community Health Workers], staff, etc. All of the first-hand reports are great!”
Breaking down the data Next, we break down how different variables affect participant responses. First, we look at those supporters who have known about Mali Health for the shortest amount of time: less than six months (six participants are in this category). In general, they find our emails engaging and informative, but they are split on whether the emails are too often or not often enough. Many don’t use Facebook, but of those who do, they like the content presented, and they’d like to see it more often. Like the population in general, they like to see content related to our programs and the beneficiaries of those programs, and they favor pictures and videos over all other forms of presentation. They are not interested in interactive opportunities; support for women
and support for health were the most popular motivations behind their interest in Mali Health; and 83% ranked our transparency in the top two slots. Among those who have known the organization a little bit longer (6 months-1 year, 14 people), the emails receive very positive support in terms of frequency and content. The website is used by fewer people, but is generally seen as interesting and up-to-date. The large majority don’t follow Mali Health on social media or receive physical mailings. Interest in content and presentation is similar to all respondents. Only 57% ranked our transparency in the top two slots, but 86% feel that donations do have a positive impact on the people of Mali. 7
The highest satisfaction with Mali Health communications leading up to October 2013 was among those who have known the organization for between one and two years. They are also the group with the most professed interest in interactive opportunities. When comparing donors with nondonors, those who have made a financial contribution to the organization have a (not-unexpected) higher rate of satisfaction with current outreach efforts. Meanwhile, those who have not made a donation are more interested than their counterparts in content that provides numerical data regarding our impact; they are also more interested than donors in receiving quarterly reports and in interacting with Mali Health staff via a webinar. Unsurprisingly, those individuals who contribute to Mali Health more than once a year are happier and more engaged with the organization's work in the recent past when compared to those individuals who donate once a year or less. The former group shows much greater interest in potential interactive opportunities, they are more interested in all forms of content and on all platforms, and they universally rank our transparency and donation impact in the top two slots on the five-point scale.
Conclusion Given that this data is based on responses from only 72 participants (out of the thousands who receive the quarterly newsletter and follow the organization on social media), Mali Health cannot accept any of these responses as universal fact. What this data is is a set of guidelines, a suggested way to proceed with communications going forward. Generally, our supporters are happy with what they've seen so far. One individual commented that our outreach makes her feel like she is part of “something meaningful.” As Mali Health anticipates the opening of a new year, and the organization plans to take on new challenges in new communities, it will be important to keep our supporters informed of our plans, how and why we are moving in this direction, and how their ongoing support will make the difference in the lives of real, individual Malian people. 8
Appendix A: Mali Health Supporter Survey Number of Respondents in Blue Percent of Respondents in Orange 1 1.4% 2 2.8% 17 23.6% 51 70.8% Brown University Alum
6 14 23 29
8.3% 19.4% 31.9% 40.3%
5
6.9%
57
79.2%
6
8.3%
3
4.2%
2
2.8%
26
36.1%
5
6.9%
29
40.3%
2
2.8%
20
27.8%
2
2.8%
38
52.8%
0
14
19.4%
9
12.5%
35
48.6%
0
3
4.2%
2
2.8%
54
75%
9
0
4
0
0
0
1
0 1
5.6%
29
40.3% 35
48.6%
10
13.9% 18
25%
1.4%
15
20.8% 10
1
1.4%
12
16.7% 10 13.9%
36
50%
1.4%
2
2.8%
1
1.4%
50
69.4%
7
9.7%
36
50%
26
36.1%
4
5.6%
27
37.5%
39
54.2%
1
1.4%
23
31.9%
45
62.5%
14
19.4%
35
48.6%
19
26.4%
17
23.6%
30
41.7%
19
26.4%
7
9.7%
36
50%
26
36.1%
4
5.6%
27
37.5%
39
54.2%
6
8.3%
23
31.9%
38
52.8%
41
56.9%
16
22.2%
11
15.3%
1
3
4.2%
30
41.2%
13.9% 35
48.6%
1.4%
10
54
75%
18
25%
36
50%
22
30.6%
11
15.3%
18
25%
33
45.8%
18
25%
20
27.8%
31
43.1%
17
23.6%
40
55.6%
22
30.1%
5
6.9%
23
31.9%
27
37.5%
14
19.4%
11
14 13
25.9% 24.1%
9 2
16.7% 3.7%
11 2
20.4% 3.7%
29 10
53.7% 18.5%
11 20.4% 13 24.1% 9 12 1 5
16.7% 22.2% 1.9% 9.3%
56 48 29 36
77.8% 66.7% 40.3% 50%
36 13 39 14 50
50% 18.1% 54.2% 19.4% 69.4%
2
2.4%
12
(1) Not true at all (2) (3) (4) (5) Completely True
1 1 10 30 24
1.4% 1.4% 13.9% 41.2% 33.3%
(1) Not true at all (2) (3) (4) (5) Completely True
0 1 6 14 47
0 1.4% 8.3% 19.4% 65.8%
(1) Will not donate (2) (3) (4) (5) Will donate
1 39 12
0 2 3 15 33
0 3.7% 5.6% 27.8% 61.1%
1.9% 72.2% 22.2%
13
Alternate final questions for non-donors
14