Evaluation Field Manual and Tools for the Knight International Journalism Fellowships by Knight International Journalism Fellowships

Evaluation Field Manual and Tools for the Knight International Journalism Fellowships, a Program of the International Center for Journalists

Updated January 2011

Purpose of this Manual

The International Center for Journalists, the John S. and James L. Knight Foundation, and the Bill & Melinda Gates Foundation value evaluation to track the success of their projects. The Knight International Journalism Fellowships program, which involves a substantial investment of time, funds and other resources, is an especially important undertaking for each of these organizations. This manual is intended to help Fellows work cooperatively with ICFJ and with an independent evaluation firm—Philliber Research Associates (PRA)—to: • • • • •

Clearly conceptualize the interim and long-term outcomes you seek; Delineate the strategies you will use to reach these outcomes; Record the populations reached by your strategies; Record their activities during your Fellowship year, and Measure your success in achieving desired outcomes.

The Manual is, of necessity, somewhat generic. We cannot anticipate all of the opportunities that Fellows will encounter or all of the outcomes they will want to capture. The Manual provides basic tools and conceptual frameworks for thinking about evaluation. Modifications or additions to these tools can be made at any time during the year by working with PRA staff.

Evaluation Basics

There are several different kinds of evaluation: •

Needs and assets assessment—Answers the questions “What exists now and what do we need to do?”

•

Process or activity evaluation—Answers the question “What did we deliver to whom by when?” What did YOU do?

•

Outcome evaluation—Answers the question “What shorter-term and longer-term changes were made as a result of what you did?”

•

Cost/benefit or cost/effectiveness evaluation—Answers the question “At what cost did all of this occur?”

In this evaluation, you will be using at least the first three kinds of evaluation. Cost evaluation may be done at some later time. Still, if you have opportunities to report achievements by their costs or cost savings, please add this information to your narrative reports. It is most important to distinguish between process and outcome—what YOU do, versus what results from what you do. Holding five training sessions is NOT an outcome—it is a strategy to produce an outcome.

Most evaluation systems include the following: • • •

Baseline measures—of what exists before you begin work. Process measures—of what you do to change what exists. Outcome measures—of what exists after you employ your strategies.

While the strongest evaluation designs are experiments (with control groups), your evaluation work will likely be confined to pre/post test designs—or comparison of what existed when you began work and what exists as you complete your work. This means that we can use the needs assessment as a time to collect valuable baseline data and thus get a two-pronged benefit from your early work in the country.

The Needs and Assets Assessment

Before it is possible to make a workable plan to improve any existing condition, it is important to understand what now existsâ&#x20AC;&#x201D;both strengths and needs. Here are some guidelines for doing your needs and assets assessments when you arrive in your country in the form of questions to ask. These are only examples and you should tailor the questions to the longer-term outcomes you wish to produce during the life of your Fellowship.

What are the most important social and economic features of the country that will affect your work such as a) poverty levels b) literacy levels c) job opportunities d) key social norms e) family structures and pressures f) political conditions (upcoming elections, competing interest groups, government strategies related to project, etc.) g) social conditions (health, infrastructure, tribal relations, etc.) h) mediaâ&#x20AC;&#x2122;s function (the role media plays in society â&#x20AC;&#x201C; whether there are limitations caused by political, social, economic or cultural pressures)

How is your media partner operating now, particularly relative to the outcome you wish to produce? a) What is the quality of their work? b) What is the amount of output of their work? c) What organizational features will challenge or facilitate the change you wish to make? d) Who is in charge or influential and thus, may influence your success? e) What is the level of openness to change? f) Are there any organizational, regional or national policies that need to be changed to facilitate your work? g) Are there organizational or national cultural factors that must be considered?

In what ways and with what strategies is it best to approach your task? a) What are the schedules of those you wish to reach that you need to consider? b) In what formats will they be most comfortable learning (e.g., workshops, individual technical assistance, mentoring)? c) Are there organizations, networks, databases, materials, programs, Web technology, or other things that need to be created to achieve the longer-term outcomes you desire? d) What is a reasonable work plan and timetable for your work?

In choosing measures to answer each of these questions and others that may be part of your needs and assets assessment, you may select either quantitative or qualitative measures. The important thing is to choose a measure that can be repeated so that you can compare your initial findings with what happens at the end of your work. For example, in measuring the amount of reporting done by your media partner on a given topic, you may want to actually select random days and count broadcast minutes or the number of articles devoted to this topic.

To measure the quality of work, you may want to look at several dimensions such as sourcing, fact checking, or style.

Logic models specify the strategies you will use and the shorter- and longer-term outcomes you expect as a result of those strategies. Your needs assessment should be based on a logic model or theory of change that will guide the remainder of your work as a Fellow. This logic model will also be the template against which you report during the year. Please create this model on the form on the following page. Begin working on your logic model from the righthand column first. In this column you should write what outcomes you expect to occur by the end of your Fellowship year. In the middle column, write the interim outcomes you hope to produce that will in turn, lead to these longer-term outcomes. For example, if your longer-term outcome is to produce a greater number of stories about health issues in your assigned country, you should list the other outcomes that need to be produced before this is possible, such as increased knowledge of health issues, improved reporting skills, improved writing skills, and so on. Finally, in the left-hand column, list the strategies you will use to achieve these outcomes. This might include individual coaching with media partner staff members, workshops, production of instruction manuals in the appropriate language, and so on. Do NOT try to make the model symmetrical—you may need to use three strategies (e.g., coaching, workshops, database development) to produce two interim outcomes (e.g, improved writing skills, greater knowledge of how to find and use government health data for stories), which in turn, would produce two longer-term outcomes (e.g., more stories on health issues being published, clearer writing on health topics, more balanced writing on health topics). You might need six strategies for one interim outcome and three longer-term outcomes. As you begin your Fellowship year, it is not possible to anticipate all the strategies you will use or all the opportunities you will have. You may modify your logic model—particularly on the lefthand column—as you find new opportunities to achieve your goals. You should be more cautious about changing your ultimate goals. Everything in your model should be clear and measurable. Do NOT write goals like: Staff members will achieve their fullest potential. Instead, write something much clearer such as: Staff members will: Produce XX% more health stories about health-care issues Use more sources in their health-care reporting Learn to use the Internet as a source of health information Remember that everything you write requires a measurement strategy that PRA will help you create.

Knight International Journalism Fellows: Logic Model I will use these strategies to achieve these outcomes (include what groups or individuals will be reached by what intervention by when)

During the Fellowship period, I expect these interim outcomes:

By the end of the Fellowship period, I expect these longer-term outcomes:

The Needs and Assets Assessment Report should give the reader a good idea of what the situation is in your country relative to your tasks there. It should include at least the following: 1. Your methodology for doing the assessment. How did you gather this information? 2. Any data you collected during the assessment, such as content analysis of the stories or broadcasts of your media partners, data from interviews with key informants, data on your specific topic (such as health or business data on the country). Please include any information you found to answer the questions posed on page 3. 3. The challenges and obstacles you faced in doing the needs and assets assessment and that you anticipate as you complete your project. 4. The logic model you created, using the format on page 5. This logic model will become the basis for future reporting and should in effect, be your strategic plan for your work. You will be asked to look back at it in your monthly reports to the Knight International team.

Recording Your Activities

The activity log is a way for you to keep a tally of all activities you have undertaken toward achieving your interim and longer-term outcomes. Depending on your own recall skills, you might want to make notes on the activity log each day, but certainly at no longer than weekly intervals. Please indicate the type of activity, with whom or for whom it occurred, and its purpose (e.g., workshop for 6 staffers at the newspaper to teach basic writing skills). Make as many extra forms of this kind as you need in a given month and submit these with your monthly report. The PRA evaluation team will code and computer-enter this information on an ongoing basis and will summarize it over the course of your Fellowship period for you to include in your final report.

Activity Log for ______/_____ Month

Activity Type                                        

Total # of hours this month

With or for whom?

Year

Purpose

coaching/mentoring large group training workshop materials development data base development group meeting policy work other: What? coaching/mentoring large group training workshop materials development data base development group meeting policy work other: What? coaching/mentoring large group training workshop materials development data base development group meeting policy work other: What? coaching/mentoring large group training workshop materials development data base development group meeting policy work other: What? coaching/mentoring large group training workshop materials development data base development group meeting policy work other: What?

The Evaluation of Group Training

Your year as a Fellow is likely to involve a great deal of training. Here we discuss how to evaluate training that occurs in workshops or with large groups, rather than one-on-one mentoring, coaching or modeling, which is discussed below. First, all training sessions should have a sign-in sheet that includes the names, titles, organizational affiliations, and contact information for those who attend. A draft form for this purpose appears at the end of this section of the manual. In addition, however, we want to evaluate the outcomes of these trainings. There are at least four dimensions by which training can be evaluated:1 1. Reaction This is a measure of how participants feel about the various aspects of a training program, including the topic, speakers, schedule, venue, logistics, and so on. It is like measuring customer satisfaction. We measure reactions of participants on the assumption that they will get more out of training if they liked it. 2. Learning This is a measure of the knowledge acquired, skills improved or attitudes changed as a result of the training. It measures what participants now know that they didn’t know before, what they believe that they didn’t believe before, and what skills they have that are new. There are two primary ways to collect such data. The first is to ask participants questions such as: Name two things that you learned in this training. This is an open-ended question that is easy to write but takes time to analyze across large groups of training participants. You also could ask something like: How much would you say you learned about crediting sources in writing stories?  a lot  some  a little  not much at all Both of these questions ask training participants to self-assess what they learned. These questions seek testimony rather than actually measuring learning. A second way to measure learning is to test it. Using this strategy, questions might look like this: A policymaker you interviewed wants to remain anonymous in your story. How would you handle this request? If you cited this source, how would you do it? Name two important considerations in citing sources for any story you write.

Donald Kirkpatrick, “Great Ideas Revisited,” originally published by the Journal for the American Society of Training Directors, 1959; subsequently reviewed and updated, 2006.

Adult learners tend to dislike tests of this kind—they are easily embarrassed. Still, if your pretests and post-tests of their knowledge are anonymous or you have succeeded in gaining their trust, this second strategy for measuring learning can work. 3. Behavior This is a measure of the changes in practice that are made by those you trained. Finding success according to this measure means that those you trained transferred their new knowledge into improved practice. To measure such behavior changes you can ask about them or you may have opportunities to directly observe them. For example, you might compare writing styles before and after your training session. Or, if the desired behavior is not readily observable, you might ask about it: Have your interviewing practices changed since the training?  no  yes: Please describe how they have changed. Obviously, these kinds of questions cannot be answered immediately after your training, but must be asked at some follow-up time, after participants have had a chance to put their new skills or knowledge into practice. 4. Results This is a measure of the longer-term results that occur due to training, learning, and behavior change. Such results might include higher quality journalism products, better management in the newsroom or any number of other changes in the industry. They might also include policy or awareness changes in the society as a result of more complete, clear, and accurate coverage of some issue. Your ambition should be your guide here. Results of this kind generally can be measured by interviews with key informants, by tracking policy change, or by tracking some population trend—such as greater access to health care or greater investment in the economy. It depends, of course, on the topic. What to Measure Most event feedback forms include measures of at least the following: Evaluations of each part of the event (e.g., each plenary or workshop session, speakers, presentations and the like) • Evaluations of the logistical aspects of the event (e.g., the transportation, hotel, meals, location) • Measures of the desired outcomes of the event (e.g., how much people learned, whether they will do anything differently as a result of their participation) •

In addition, you may want to include some information on the characteristics or demographics of those who complete feedback forms. Such data may later be useful in determining what “kinds” of people liked or did not like the event, learned or did not learn from it and so on.

How to Ask There are two important general guidelines for creating event feedback forms: 1. Unless there is some compelling reason not to do so, create forms that are anonymous.  The theory here is that participants will be more candid with you if their identity is protected. Guaranteeing anonymity also means collecting the forms in a box or in some other way that does not force a participant to put his/her form right into the hands of an event organizer. 2. Keep it short.  Event feedback forms are generally collected at the end of an event, when people are ready to leave. Ask only about what you really need to know. In addition to these general principles, here are some specific guidelines for gathering useful data: 1. Use structured, quantitative measures when you can rather than a lot of open-ended questions. Why? • People have to write less. • Your measures will be comparable, so you can see what participants liked most and least. • You can compare before and after data. • You can summarize data across all your participants since they all had the same responses to choose from. 2. Use scales, such as “excellent, good, fair, poor.” Here’s how they can be applied: • •

Use an even rather than an odd number of choices. People love to waffle, so this prevents them from picking the middle option. Use the same adjective in each response choice—

NOT: Pretty interesting Somewhat useful Not too satisfied I hated it •

BUT: Very useful Somewhat useful Not very useful Not useful at all,

OR: Very satisfied Somewhat satisfied Somewhat dissatisfied Very dissatisfied

Ask only one question at a time. (NOT: “How often do you buy and read the newspaper?”)

3. Think about your desired outcomes as you plan the training and certainly before you make your feedback form. •

• •

What do you expect to happen as a result of this event? Should people know something they didn’t know before? Will they believe something they didn’t believe before? Will they do something they didn’t do before? Some of your measures should focus on these desired outcomes. Remember that intent to act is not the same as actual behavior. If your event intends to change people’s behavior and you gather information as the event is ending, you may capture what participants intend to do but not changes in actual behavior. Measuring actual behavior change usually requires contacting participants after they have had time to return home and have a chance to make the desired change in behavior.

•

Remember that asking people whether they think they learned is not the same as an actual test of their knowledge.

4. Include one or at most two, open-ended questions asking for “other” comments or suggestions. It’s always good to allow participants to say whatever they want to say about training. This question generally should be last and may uncover outcomes of an event that you forgot to measure or that were not anticipated. 5. Think about how to maximize the response. Of course, the information you collect on forms like this is only as good as your sample. Create a plan to ensure a good response. Here are some ideas: • • • •

Make the last session “wrap-up and evaluation” so that the audience perceives this as part of the day and not something that happens after the training is over. Ask participants to turn in their evaluation forms as their “meal ticket” to the last luncheon. Give participants a tape of a session, a gift, or some other material as they leave, exchanging it for their completed evaluation forms. Create a plan for getting evaluation forms from those who leave the event early.

The next two pages contain a sample form that illustrates these guidelines. Below that form, there is a form to use to follow up on training participants at a later date.

Our Event: Evaluating Its Effectiveness Please do NOT put your name on this form. We need your candid feedback on this meeting so that we can make plans for future events. Thank you! 1. How would you rate each of the following? What about 2001? Smith, Wilson and Harris Press Freedom: Current Problems Vasquez Newsroom Changes Allen, Pace Credibility in the News Industry Smith, Allejandro Using the Computer in the Newsroom Arthur Dinner Speaker Bridgefield Communication Strategies with Your Advertising Department Palmer, Brown Making Evaluation Forms Philliber

Excellent

Good

Fair

Poor



2. We know there are many competitors for your time. Overall, how would you rate this training as a use of your time?  Excellent use of my time  Good use of my time  Not very good use of my time  A waste of time 3.    

How would you describe what you learned at this training? I learned a lot I learned some things I learned a few things I didn’t learn anything new

4. What about the logistics for the meeting? How would you rate the following? Excellent

Good

Fair

Poor

Service from the travel agent



Having the meeting in Bolivia



The particular facility chosen for the meeting



The food provided

 

Time to network with others

5. As a result of this event, how likely is it that you will do each of the following when you return home? Very likely Somewhat likely  

Make greater use of the computer

Somewhat unlikely 

Very unlikely 

Begin a conversation about changes in advertising



Write an article on press freedom issues in my country



6. Are you….  Male  Female

7.      

Are you from… South Africa Zambia Mozambique Montenegro Malaysia India

     

Bangladesh Pakistan Brazil El Salvador Philippines Syria

8. What topic that was NOT discussed at this training would you like included at a future event? _________________________________________________________________________ 9. Any other comments or suggestions? ____________________________________________________________________________ ____________________________________________________________________________ ____________________________________________________________________________ ____________________________________________________________________________ ____________________________________________________________________________ ____________________________________________________________________________ THANK YOU!

Follow-up to Training: Two months ago, you were kind enough to attend a training session at …. on writing skills for health reporters. We are interested in understanding your current feelings about that training and about any changes that you may have made as a result of it. 1.    

Looking back on that training now, how useful would you say it was? Very useful Pretty useful Somewhat useful Not very useful at all

2. Have you made any changes in your writing style or practices as a result of that training?  no: Why not? We are interested in what barriers you may have experienced. _______________________________________________________________________ _______________________________________________________________________  yes: Please describe those changes for us. OR  yes: Please check which of the following changes you have made: Then provide a check list of changes you recommended or tried to teach in training for them to check off.

Training Sign-in Sheet Name

Title

Organization

e-mail

Telephone

Evaluating Coaching, Mentoring or Other One-on-One Training Besides large group trainings and workshops, you also are likely to do a great deal of individual coaching, mentoring, modeling, or technical assistance. These activities should be noted on your activity log. As in larger-group trainings, individual training or teaching also can be evaluated by measuring a) reaction; b) learning; c) behavior change, and d) results. Because pre- and post-tests would be somewhat awkward for this kind of training, the data collection strategies for this kind of evaluation are somewhat different. Your success at this kind of training will depend on creating trusting and effective relationships. Their reactions are most easily measured by whether they continue to work with you and even seek you out for help. But you need to observe and note their learning, behavior change, and the ultimate results. These observations can be more formal and quantitative (e.g., doing an analysis of their writing or broadcasting before and after you work with them); or they can be more qualitative (e.g., notes on changes in behavior you have observed). A log for recording your observations of learning, behavior change and results follows. Since you are likely to be doing so much of this work and because it may take some time before you see important changes, we suggest that you reserve use of this log for the more â&#x20AC;&#x153;breakthroughâ&#x20AC;? or dramatic changes that occur. Sometimes, you may notice changes in learning, behavior, or results that are unexpected. Your work may have caused these changes but you may not have been deliberately seeking them. Be sure to record these changes as well, even if you cannot be sure exactly what created them.

Individual Training Outcomes for ______/______ Month

Who have you On what skills or been working topics? with (name and title)?

Any noted changes in learning?

Any noted changes in behavior?

Year

Any longer-term or broader results?

Notes on Meetings or Similar Gatherings

Sometimes accomplishments occur around a meeting table. A policy is changed, learning becomes apparent, or critical decisions are made. The log on the following page enables you to make notes on key meetings that you attend or host. Again, all of your meetings should be recorded on the activity log. But outcomes need to be recorded only for those meetings where you believe you have made steps toward achieving your interim and longer-term outcomes.

Meeting Notes Date: ____/____/____ month

day

year

1. Including yourself, how many people were present at this meeting? ______ 2. What were the main issues/topics discussed during this meeting? ______________________________________________________________________ ______________________________________________________________________ 3. Describe the important outcomes of this meeting. ______________________________________________________________________ ______________________________________________________________________ ______________________________________________________________________ ______________________________________________________________________ ______________________________________________________________________ ______________________________________________________________________ ______________________________________________________________________ ______________________________________________________________________ ______________________________________________________________________ ______________________________________________________________________ ______________________________________________________________________ ______________________________________________________________________ ______________________________________________________________________ ______________________________________________________________________ ______________________________________________________________________ ______________________________________________________________________

Tracking Materials or Other Tangible Products

To accomplish your longer-term and interim goals, you may need to produce a variety of materials or products including manuals, databases, newsletters, articles, reports, or videos. The log on the following page will allow you to record for ICFJ and for the evaluation team any such tangible products produced as part of your work. In any month in which such materials are created and/or distributed, update this log and send it in with your monthly report. Please also send a copy of these materials.

Tracking Materials/Products Type of material * and title

Date Number created distribute (month/ d

Target audience

How will this product/material help you achieve your project outcomes?

year)

*Examples of materials: databases, publications, videos, handbooks, newsletters, educational booklets, research reports, educational articles, etc.

Creating New Networks, Programs, or Institutions

Some Knight International Journalism Fellows have been able to create lasting programs, networks, institutions, or centers as part of their work. Certainly these are accomplishments that we want to know about as part of your outcome reporting. The log that follows can be used for notes, but feel free to also further discuss this work in your narrative reports.

Creating Programs / Networks / Centers / Institutions Date: ____/____/____ month

day

year

1. What type of institution have you helped to create? Please describe it for us. ______________________________________________________________________ ______________________________________________________________________ ______________________________________________________________________ ______________________________________________________________________ ______________________________________________________________________ ______________________________________________________________________ 2. What is its main purpose? ______________________________________________________________________ ______________________________________________________________________ 3. How many people will it affect? _________ 4. What population does it serve? ______________________________________________________________________ ______________________________________________________________________ ______________________________________________________________________ ______________________________________________________________________ 5. How will the creation of this enhance your project outcomes? ______________________________________________________________________ ______________________________________________________________________ 6. Additional notes or comments: ______________________________________________________________________ ______________________________________________________________________

Reporting Requirements

Here we summarize the reporting requirements for Fellows. On the following pages are report templates that all Fellows MUST use throughout their fellowship.

 

The Baseline Report Due:

Within four weeks of beginning your work.

Content:

See pages 26, 27 and 28.

The Monthly Report Due:

At the end of each calendar month.

Content:

See pages 29, 30 and 31.

Attachments: Your monthly activity log, any training forms, sign in sheets, notes on meetings, reports on materials produced, or any other forms you completed during the month. You may include clippings, copies of materials produced or anything else that will tell the story of your work.



The Six-Month Report Due:

At the end of each six months you spend in your assigned country.

Content:

See pages 32, 33 and 34.

Attachments: As in the monthly reports—submit all supporting forms collected during the month.



The Annual Report Due:

At the end of each year you spend in your assigned country.

Content:

See pages 35, 36 and 37.

Attachments: As in the six-month report—submit all supporting forms not previously submitted.



The Final Report Due:

At the completion of your time in country.

Content:

See pages 38, 39 and 40.

Attachments: Any final forms you have not yet submitted and other kinds of documentation including clippings. Please also attach your bio.

Baseline Report Template [Please save as: Report Baseline (fellow name) yyyy-mm]

Date of Report:

____/_____/_____ Month

Day

Year

Country:

_______________________________________

Fellow’s Name:

_______________________________________

Your baseline report should be based on research you have conducted during the first few weeks of your fellowship. This report should serve as the starting point from which all future activities will be measured, and will help you determine at the end of your fellowship what is different from now on as a result of your fellowship. Maximum length: 15 pages. Additional content can be included in appendices. 1. Assessment: a. Please provide a concise assessment of social/political/economic conditions and how they could impact your fellowship, including: •

Political conditions (upcoming elections, political tensions, competing interest groups, government strategies related to the project, etc.)

•

Social conditions (health, key social norms, family structures, tribal relations, etc.)

•

Poverty levels and economic opportunity (including job opportunities related to journalism)

•

Literacy levels

•

The role media play in society. Do political, social or economic pressures limit the role of media? If so, in what ways?

•

Access to information. How much access do people have to the types of news and information your fellowship will facilitate? What are the information sources? Who lacks access and why?

•

Access to media technology. What opportunities and obstacles are presented by available media technologies, such as mobile phones, Internet, computers, etc.? What media technologies are available, including the technologies you will use in your fellowship, and which people do not have access to them?

•

Anything else that should be considered?

***You can provide a more detailed account, with additional references, in an appendix. Here we just want a brief, focused assessment*** b. Please provide a detailed assessment of the targeted participants (individual journalists and media).

•

Provide a brief overview of the targeted media organizations: size of staff, medium (or media), audience, current circulation or website traffic, focus of coverage, competition. Who will participate in the project at all levels of the organization?

•

Based on your project goals, please assess the quality of their work produced at the start of the project to establish a baseline that will be used to assess your progress and measure your impact.

•

What is produced now by your targeted participants (and other media) in the area you plan to focus on during your fellowship? If nothing is produced, please note that for future reference.

•

What organizational features (i.e. management practices, groups within the organization, their infrastructure and resources, etc.) will challenge or facilitate the change you wish to make?

•

Who is in charge or influential and thus, may influence your success?

•

What is the level of openness to change, and who seems open to it?

•

Are there any organizational, regional or national policies that need to be changed to facilitate your work?

•

Are there organizational or national cultural factors that must be considered when working with these particular participants?

•

Anything else that should be considered?

2. Overview: What are your project goals? Please provide a general explanation of what you plan to do and how you plan to achieve your objectives. What will be different “from now on” as a result of your fellowship? 3. Activities: What activities do you plan to use to achieve your goals? Please explain which of the following activities you will utilize, who you will target within each activity, how the activity will benefit them, and how you plan to monitor their progress and evaluate their success. (For any activities that will not apply, write “N/A”) a. Training (workshops, seminars, classes, etc.) b. Mentoring c. Establishing networks, associations, etc. d. Creating products and resources (training manuals, guides, websites, Moodles, etc.) e. Creating educational and training institutions f.

Anything else?

4. Outcomes: Please list any individual-level changes that you would like to achieve as a result of your fellowship such as changes in the way individual journalists or groups of journalists work, or changes in staff output. These changes should be both qualitative and quantitative, and should answer the question, “From now on, what will be different on an individual level?”

5. Outcomes: Please describe any organizational-level changes that you would like to achieve as a result of your fellowship. For example, changes in how your media partner organizes work, newsroom restructuring, addition of or reassignment of reporters, editors, photographers, etc., production of new media products etc. These changes should be both qualitative and quantitative, and should answer the question, “From now on, what will be different on an organizational level?” 6. Outcomes: Please list any societal changes that you hope will occur as a result of your fellowship. For example, governmental policy changes, changes in laws or regulations, the redirection of government spending, new networks created, new educational institutions created, or any other action that will be taken. These changes should be both qualitative and quantitative, and should answer the question, “From now on, what will be different on a societal level?” 7. What challenges and obstacles have you experienced during your brief stay or what challenges can you now anticipate that may have an impact on the planned outcomes you mentioned above? 8. Next steps: What do you plan to do next? This can be a general plan or specific ideas and partners you have in mind. 9. Please provide a work plan that outlines planned activities, short-term objectives and long-term objectives using the logic model template attached.

Monthly Report Template [Please save as: Report Monthly (fellow name) yyyy-mm]

Date of Report:

____/_____/_____ Month

Day

Year

Country:

_______________________________________

Fellow’s Name:

_______________________________________

In filling in the summary of activities and answering the questions, please be concise and provide specific details. Additional content can be included in appendices and attachments. Early on in your fellowship, some of the areas listed throughout this outline may not apply. In those cases, please write “too soon.” Summary of monthly activities – Please complete the table below: Did you train? Who?

How many? About what?

What is the impact?

Did you mentor Who? or coach?

How many? About what?

What is the impact?

Did you facilitate networking?

How many? For what purpose?

What is the impact?

Did you create a For whom? new product?

How many? For what purpose?

Who uses this product and what is the impact?

Other activities? Who? If so, what?

How many? For what purpose?

What is the impact?

Among whom?

1. Briefly describe each of your primary activities during this reporting period. For each activity, please explain clearly: a. Situation – The situation that existed for your targeted participants before this activity and the areas your work targeted, in terms of individuals, partner organizations and society. b. Activities – The activities you performed (for example, mentoring/coaching, networking/ partnership building, training seminar/workshop, etc.), the approximate length of time it covered (two-day workshop, one-hour weekly meeting, etc.), whom you partnered with, where this took place, and how it addressed the needs you identified. c. Results – How many people were reached by these activities, who they were (for example, editors, citizen journalists, reporters, media managers, etc.; their type of medium; their media

organization), and the impact it had on individuals, organizations and society. Please answer the question, “As a result of the activity, from now on, what is different?” for each of the following changes: i.

Individual changes – concrete changes in the way individual journalists or groups of journalists work and produce stories. For example, using more sources, checking facts to ensure greater accuracy, including a wider variety of opinions in stories or having increased access to resources. Please be specific and quantify these changes.

ii. Organizational changes – concrete changes in how your media partner organizes work; newsroom restructuring; the addition or reassignment of reporters, editors, photographers; production of new media products; changes in audience or Web traffic numbers, etc. Please be specific and quantify these changes. iii. Societal changes – concrete changes that may have occurred as a result of stories produced by journalists you have trained, such as governmental policy changes, changes in laws or regulations, the redirection of government spending, new networks created, new educational institutions created, increased access to information or any action taken. Please be specific and quantify these changes. d. Monitoring & Evaluation – How you plan to monitor and evaluate the impact of this activity and follow up with participants in the future. ***When answering Question 1, please focus on the activities and results. If you faced challenges or obstacles, please comment on them in Question 4*** 2. During this reporting period, has your work resulted in any new completed products (for example, manuals, training materials, election guides, websites or any other product)? Please describe the product, how it is different from what was available before, your plan for disseminating it, and how you will monitor and evaluate its impact. Include hyperlinks for anything produced that is available on a website. 3. Of the changes you may have mentioned in this report (individual, organizational and societal), are there any that will last after your fellowship ends? If so, what has been established (or will be established) that makes those changes sustainable? 4. What challenges have you experienced during this time period that may have affected your outcomes? 5. Have you observed any new individual, organizational or societal changes from your activities in previous months? Please quantify the changes and explain what is different now. 6. Briefly list the activities you plan for next month/period of your fellowship: What you will do, who you will target, what you hope to accomplish. 7. We would appreciate any additional reflections you may have on your work this month. Tell us about your activities, observations, progress and challenges. If there is any way in which ICFJ can better support your work, please include that information as well. Finally, please attach your monthly training participants sheet using the Excel file ICFJ provided, your financial report and any products you created and news stories that resulted from your work. Thank you.

Six-Month Report Template [Please save as: Report Six Month (fellow name) yyyy-mm]

Date of Report:

____/_____/_____ Month

Day

Year

Country:

_______________________________________

Fellowâ&#x20AC;&#x2122;s Name:

_______________________________________

Your six-month report should focus on the impact of your work in your country now as compared to when you first started your fellowship (baseline). Have any conditions changed over the past six months? Do you see anything differently now than when you first started your fellowship six months ago? Please answer the questions below as they pertain to the entire six months. In filling in the summary of activities and answering the questions, please be concise and provide specific details. Additional content can be included in appendices and attachments. Summary of six-month activities â&#x20AC;&#x201C; Please complete the table below: Did you train? Who?

How many? About what?

What is the impact?

Did you mentor Who? or coach?

How many? About what?

What is the impact?

Did you facilitate networking?

How many? For what purpose?

What is the impact?

Did you create a For whom? new product?

How many? For what purpose?

Who uses this product and what is the impact?

Other activities? Who? If so, what?

How many? For what purpose?

What is the impact?

Among whom?

1. Overview: a. Please tell us which objectives/goals you have achieved to date. b. Which objectives/goals have you made progress on, but have not fully achieved yet? c. Which objectives/goals have you made no progress on? Why? d. How have your objectives/goals changed since baseline, if at all?

e. If you have new objectives/goals since you submitted the baseline report, please tell us what they are and how you intend to achieve them. f.

Are you targeting the same participants (individuals and media) as when your fellowship began? If your targets have changed, please tell us who your new targets are.

g. Have societal/political conditions changed since you submitted your baseline report? If they have changed, how are they different? 2. Impact: Briefly describe each significant impact your work had during this reporting period. For each impact, please explain clearly: a. Situation – The situation that existed for before this impact and the areas your work targeted, in terms of individuals, partner organizations and society. b. Activities – The activities you performed to achieve this impact (for example, mentoring/ coaching, networking/partnership building, training seminar/workshop, etc.), whom you partnered with, and how these activities addressed the needs you identified. c. Results – How many people were reached by this impact, who they were (for example, editors, citizen journalists, reporters, media managers, etc.; their type of medium; their media organization), and the impact it had on individuals, organizations and society. Please answer the question, “From now on, what is different?” for each of the following changes: i.

there any that will last after your fellowship ends? If so, what has been established (or will be established) that makes those changes sustainable? 5. What challenges and obstacles have you experienced during the first six months of your fellowship that have had an impact on your outcomes? 6. Briefly list the primary activities you plan for next six months: What you will do, who you will target, what you hope to accomplish. 7. We would appreciate additional reflections on your work so far. Tell us about your activities, observations, progress and challenges. If there is any way in which ICFJ can better support your work, please include that information as well. Finally, please attach your updated training participants sheet using the Excel file ICFJ provided, your financial report and any products you created and news stories that resulted from your work. Thank you.

Annual Report Template [Please save as: Report Annual (fellow name) yyyy-mm]

Date of Report:

____/_____/_____ Month

Day

Year

Country:

_______________________________________

Fellowâ&#x20AC;&#x2122;s Name:

_______________________________________

Your annual report should focus on the conditions in your country now as compared to when you first started your fellowship (baseline). Have any country conditions changed during your fellowship? Do you see anything differently now than when you first started your fellowship? Please answer the questions below as they pertain to the entire year. In filling in the summary of activities and answering the questions, please be concise and provide specific details. Additional content can be included in appendices and attachments. Summary of fellowship activities for the year â&#x20AC;&#x201C; Please complete the table below: Did you train? Who?

How many? About what?

What is the impact?

Did you mentor Who? or coach?

How many? About what?

What is the impact?

Did you facilitate networking?

How many? For what purpose?

What is the impact?

Did you create a For whom? new product?

How many? For what purpose?

Who uses this product and what is the impact?

Other activities? Who? If so, what?

How many? For what purpose?

What is the impact?

Among whom?

1. Overview: a. Please tell us which objectives/goals you have achieved to date. b. Which objectives/goals have you made progress on, but have not fully achieved? c. Which objectives/goals have you made no progress on? Why? d. How have your objectives/goals changed since baseline, if at all?

e. If you have new objectives/goals since you submitted the baseline report, please tell us what they are and how you intend to achieve them. f.

Are your targeted participants (individuals and media) the same as when your fellowship began? If your targets have changed, please tell us who your new targets are.

g. Have societal/political conditions changed since you submitted your baseline report? If they have changed, how are they different? 2. Impact: Briefly describe each significant impact your work had during this reporting period. For each impact, please explain clearly: a. Situation – The situation that existed before this impact and the areas your work targeted, in terms of individuals, partner organizations and society. b. Activities – The activities you performed to achieve this impact (for example, mentoring/ coaching, networking/partnership building, training seminar/workshop, etc.), whom you partnered with, and how these activities addressed the needs you identified. c. Results – How many people were reached by this impact, who they were (for example, editors, citizen journalists, reporters, media managers, etc.; their type of medium; their media organization), and the impact it had on individuals, organizations and society. Please answer the question, “From now on, what is different?” for each of the following changes: i.

6. During this reporting period, has your work resulted in any new completed products (for example, manuals, training materials, election guides, websites or any other product)? Please describe the product, how it is different from what was available before your fellowship, how you have disseminated it, and how you plan monitor and evaluate its impact in the coming months. Please include hyperlinks for anything produced that is available on a website. 7. Which of these products will be used/sustained after you are gone? 8. What challenges and obstacles have you experienced during your fellowship that have had an impact on your outcomes? 9. We would appreciate your reflections on your work. Tell us about your activities, observations, progress and challenges. If there is any way in which ICFJ can better support this work, please include that information as well. 10. Finally, what steps should ICFJ/KIJF take to ensure the sustainability of your work? Finally, please attach your updated training participants sheet using the Excel file ICFJ provided, your financial report and any products you created and news stories that resulted from your work. Thank you.

Final Report Template [Please save as: Report Final (fellow name) yyyy-mm]

Date of Report:

____/_____/_____ Month

Day

Year

Country:

_______________________________________

Fellowâ&#x20AC;&#x2122;s Name:

_______________________________________

Your final report should focus on the conditions in your country now as compared to when you first started your fellowship (baseline). Have any country conditions changed during your fellowship? Do you see anything differently now than when you first started your fellowship? Please answer the questions below as they pertain to your entire fellowship period. In filling in the summary of activities and answering the questions, please be concise and provide specific details. Additional content can be included in appendices and attachments. Summary of overall fellowship activities â&#x20AC;&#x201C; Please complete the table below: Did you train? Who?

How many? About what?

What is the impact?

Did you mentor Who? or coach?

How many? About what?

What is the impact?

Did you facilitate networking?

How many? For what purpose?

What is the impact?

Did you create a For whom? new product?

How many? For what purpose?

Who uses this product and what is the impact?

Other activities? Who? If so, what?

How many? For what purpose?

What is the impact?

Among whom?

1. Overview: a. Please tell us which objectives/goals you achieved during your fellowship. b. Which objectives/goals did you make progress on, but did not fully achieve? c. Which objectives/goals did you make no progress on? Why? d. How did your objectives/goals change since baseline, if at all?

e. Did you target the same participants (individuals and media) as when your fellowship began? If your targets changed, please tell us who your new targets were. f.

Have societal/political conditions changed since you submitted your baseline report? If they have changed, how are they different?

2. Impact: Briefly describe each significant impact your work had during this reporting period. For each impact, please explain clearly: a. Situation – The situation that existed before this impact and the areas your work targeted, in terms of individuals, partner organizations and society. b. Activities – The activities you performed to achieve this impact (for example, mentoring/ coaching, networking/partnership building, training seminar/workshop, etc.), whom you partnered with, and how these activities addressed the needs you identified. c. Results – How many people were reached by this impact, who they were (for example, editors, citizen journalists, reporters, media managers, etc.; their type of medium; their media organization), and the impact it had on individuals, organizations and society. Please answer the question, “From now on, what is different?” for each of the following changes: i.

7. Which of these products will be used/sustained after you are gone? 8. What challenges and obstacles have you experienced during your fellowship that have had an impact on your outcomes? 9. We would appreciate your reflections on your work. Tell us about your activities, observations, progress and challenges. If there is any way in which ICFJ can better support this work, please include that information as well. 10. Finally, what steps should ICFJ/KIJF take to ensure the sustainability of your work? Please attach your updated training participants sheet using the Excel file ICFJ provided, your financial report and any products you created and news stories that resulted from your work. Thank you.

Data and reports should be sent to Knight International Journalism Fellowships staff and Philliber Research Associates staff at the following addresses: Knight International Journalism Fellowships 1616 H St. NW, 3rd Fl. Washington, DC 20006 Knightstaff@icfj.org Philliber Research Associates 16 Main St. Accord, NY 12404 PH: 845-626-2126 FX: 845-626-3206 Email: sherrling@philliberresearch.com (Scott Herrling) Email: sphilliber@philliberresearch.com (Susan Philliber)