Section 2: Program Design

PHOTO COURTESY OF THE VALLEY-WIDE RECREATION AND PARK DISTRICT

Purpose: Early steps in the program design phase, such as engaging stakeholders and conducting a community needs assessment, are also important steps for framing an evaluation approach. Before you can develop your evaluation plan, you first need to develop and describe your program. Having a clear and defined program will set you up for a successful evaluation.

Learning Objectives

Designing an evaluation to measure the health impact of your program starts during the program design phase and includes setting up processes that will provide short-term feedback to track interim progress toward longer-term goals.

This section will focus on the tools that you can use in program design and management to prepare for a successful evaluation. These include:

• Stakeholder Engagement • Community Needs Assessments

• Mission Statement and Program Goals • SMART Objectives • Community Asset Mapping • Logic Models The information in this section is geared toward the development of evaluation plans for new programs (prospective evaluations). However, many of the same principles can be applied to evaluating programs that already have ended (retrospective evaluations).

Retrospective evaluations primarily rely on secondary data, or data that already have been collected, that are reanalyzed to determine possible outcomes and impacts of a program that are no longer operating. There are limitations to conducting retrospective evaluations but with the appropriate data, they can be a useful approach to determining program effectiveness.

Definitions

Prospective evaluations: Prospective studies ask a question and look forward. The studies are designed before any information is collected.

Retrospective evaluations: Retrospective studies look backwards and examine factors in relation to an outcome that is established at the start of the study. These studies use information that is usually collected for reasons other than research, such as administrative data and medical records.6

Evaluation Advisory Team: Also known as an evaluation advisory group or evaluation consulting group, this team is typically made up of a variety of individuals both internal and external to your project, including intended participants, evaluation experts and intended beneficiaries. Their role is to provide advice on an individual evaluation to improve the quality and utility of the work.7