Fall Semester 2009 Course Title:
Research, Design, & Evaluation of Non-Profit Programs (EDLD 500)
Course Credit: 4 Meeting Times: Monday evenings 5:45-9:15pm and Saturday, December 18 (8:00-4:30pm) Instructors:
Fernando Sanudo, MPH; 760-631-5000 x7164 Fsanudo@aol.com Laura Deitrick, MA; 619-770-7932 lauradeitrick@yahoo.com
Office Hours:
By appointment only.
Summary Description: Research, program design, and evaluation are critical functions in the sector. Leaders and managers of organizations are continually challenged to monitor community needs, select and develop services and programs that respond to those needs, and to evaluate and modify the services they provide. This recurrent process is the foundation of quality programs. This course prepares students to undertake the research, program design and evaluation process employed in successful organizations. Students will learn the fundamentals of: 1) research terminology and concepts, 2)assessing community needs, 3) using theory to guide program design, 4) identifying and evaluating research literature that supports program development, 5) data-based planning, 6) cultural competence in planning and evaluation of programs, 7) designing evaluation studies, 8) social marketing, 9) program monitoring, 10) program budgets and 11) modifying services based on evaluation results. The course will use both didactic and interactive instruction methods, and students will conduct a project that will provide applied experience in research, program design, and evaluation of programs. Course Objectives: 1. 2. 3. 4.
5. 6.
To teach students how to locate and evaluate research related to program development. To teach students a basic understanding of research methodology & terminology. To teach students the scientific concepts related to research, program design, and evaluation. To provide students with analytic skills necessary for students to develop, monitor, evaluate, and modify nonprofit services and programs that are responsive to community needs, culturally competent, scientifically grounded, and effective. To illustrate to students the importance of applying scientific methods in the development, design, implementation, and evaluation on nonprofit programs. Students will use their leadership skills to creatively solve problems as they work toward a final product.
Required Reading: All students are required to purchase 1. Designing and Managing Programs: An Effectiveness-Based Approach by Kettner, Moroney and Martin-Third Edition. Sage Publications ( 2008) 2. Understanding Research Methods: A Guide for the Public and Nonprofit Manager by Donijo Robbins (CRC Press, 2009). Available online from AMAZON and Barnes and Noble. Please note that additional reading handouts will be provided in class. Program Length: This course consists of 14 interactive 3.5-hour sessions and one 8-hour session to be held on Saturday, December 18, 2009.
Course Requirements: Final Project (worth 40% of grade): Students will work in teams of 4 to research, develop, design, and evaluate a science-based non-profit program. The program plan will include multiple steps (see below) that covers the entire program design and evaluation process. Students will present portions of the plan throughout the semester and receive feedback from the instructors and their peers. Students will have the opportunity to make modifications based on this feedback. Students will produce a scientifically researched program proposal that includes the following sections on 8.5x11 size paper, 12 point font (Times New Roman), and 1”margins: • Introduction & Needs Assessment (5-pages double spaced) (20 points) ¾ Introduction ¾ Summary of literature review findings ¾ Problem statement ¾ Summary of focus group and interview findings •
Program Design & Methodology (10-pages double spaced) (30 points) ¾ Literature Review (10 journal references) ¾ Logic Model ¾ Goals ¾ Objectives ¾ Operational Definitions ¾ Implementation Activities ¾ Methodology
•
Social Marketing Plan (3-pages double spaced) (10 points).
•
Cultural Competency (3-pages double spaced) (10 points)
•
Evaluation & Data Analysis Plan (5-pages double spaced) (20 points) ¾ Process & outcome measures ¾ Data analysis plan & software
• •
Program Budget & Budget Justification (2-4 pages double spaced) (10 points) References
FINAL PAPER IS DUE SATURDAY, DECEMBER 18TH. FINAL PAPER IS WORTH A TOTAL OF 100 POINTS OR 40% OF YOUR GRADE. Homework Assignments (worth 20% of grade): Students are required to complete all homework assignments and submit them by the next class meeting. NO late homework assignment will be accepted, unless approved by instructors. Course Quizzes (worth 20% of grade): Two scheduled quizzes and 1-2 surprise quizzes MAY be administered throughout the semester. The quizzes will be based on the course reading assignments and/or activities done in class. Class Presentations (worth 10% of grade): One to two work teams will be asked to present different sections of their final project throughout the semester. All presentations should be completed on power point. Class Participation & Attendance (worth 10% of grade):
Class participation and regular attendance is required for this course.
Grading: Please note that all grades are FINAL and will not be changed. 95-100 90-94 87-89 84-86 80-83 77-79 74-76 70-73
A AB+ B BC+ C C-
Course Outline: September 14, 2009 SESSION #1
• • •
• • • • • • Homework Assignment:
September 21, 2009 SESSION#2
COURSE OVERVIEW Course Syllabus Course Objectives Course Requirements ¾ Final Project ¾ Homework Assignments ¾ Class Presentations ¾ Class Participation & Attendance ¾ Course Quizzes Grading Reading Materials In & Out of class required activities Writing Style Expectations Manuscript Production (APA format) Course Overview -Research Methods (RM) Chapter 1 & 2 -Brainstorm topics for final project -Begin to identify potential project team members
• • • • •
RESEARCH PRINCIPLES Intro to Research Research Design for Clinical Research Koch’s Postulates Types of Research Designs Research Terminology
Homework Assignment: -Study for Quiz #1 -Review and understand research terms; and one-page paper on any subject of interest related to the field of non-profit management. Technical writing and a minimum of 3 journal references required. Paper should include the following components: introduction, research summary, and conclusion. Paper should be double spaced on 8.5 x 11 inch paper with 1” margins. Make sure the references are cited in APA format. -Robbins Chapters 6 & 7
September 28, 2009
SESSION#3
Homework Assignment:
October 5, 2009 SESSION#4
• • • • •
-Study for Quiz #2 - Designing & Managing Programs (D&MP) Chapters 1, 3 & 4 - RM Chapter 6
• • •
Homework Assignment:
October 12, 2009 SESSION#5
Homework Assignment:
October 26, 2009 SESSION#7
Homework Assignment:
PREPARING FOR COMMUNITY RESEARCH Problem Statement Needs Assessment QUIZ #2 -Meet with your team and: (1) begin review of existing data; (2) develop problem statement; and (3) compile data sources for needs assessment. -Complete on-line lessons 1, 2, & 3 at: www.etr.org/recapp/logicmodelcourse/ - Designing & Managing Programs (D&MP) Chapter 5 --Robbins Chapters 8 & 9
• • •
Homework Assignment:
October 19, 2009 SESSION#6
RESEARCH PRINCIPLES (PART II) Sampling Reliability & Validity Operational Definitions Research Terminology QUIZ #1
PREPARING FOR COMMUNITY RESEARCH (PART II) Focus Groups Surveys Interviews -Meet with your group and: (1) research data collection instruments, and (2) design/adapt data collection instruments -Submit a one page executive summary of the proposed project data collection measures, and a justification based on the review of the literature. -Robbins Chapters 10 & 11
•
CONDUCTING COMMUNITY RESEARCH Examples of Community Needs Assessments -Develop focus group questions, key informant interview questions and/or survey questions for your needs assessment. -Robbins Chapter 12
CONDUCTING COMMUNITY RESEARCH (PART II) NO CLASS MEETING -Conduct 2 focus groups and/or interview a minimum of 8-10 subjects and summarize results. Complete a 5-page double spaced needs assessment that includes the following: (1) introduction (2) problem
statement, (3) summary of literature review findings, (4) summary of focus group and/or interview findings. (Due 11/09/09) - Designing & Managing Programs (D&MP) Chapter 6
November 02, 2009 SESSION#8
•
Homework Assignment:
November 09, 2009 SESSION#9
PROGRAM DESIGN LOGIC MODEL -Conduct literature review on proposed intervention -Complete literature review matrix (10) journal references required)-due11/3/08; -Complete on-line, lessons 4,5,& 6 at: www.etr.org/recapp/logicmodelcourse/ - Designing & Managing Programs (D&MP) Chapter 7
• •
Homework Assignment:
PROGRAM DESIGN (PART II) GOALS, OBJECTIVES, ACTIVITIES Class Presentation on: Needs Assessment -Practice working on goals, objectives & activities and complete team workplan; -Complete first draft of 10-page double spaced program design narrative which includes: (1) literature review; (2) logic model; (3) goals; (4) objectives; (5) operational definitions; (6) implementation activities; (7) methodology; (8) workplan (sample of workplan will be given in class). (Due: 11/23/09)
November 16, 2009 SESSION#10
SOCIAL MARKETING
Homework Assignment:
RM Chapter 13 and 14 Complete and submit a three-page double spaced plan on incorporating social marketing into the proposed project. (Due
11/30/08) November 23, 2009 SESSION#11
CULTURAL COMPETENCY Class Presentation on: Program Design
Homework Assignment:
-Complete and submit a three-page double spaced plan on incorporating cultural competency in the proposed project. (Due
12/07/08). - Designing & Managing Programs (D&MP) Chapters 8 & 9 November 30, 2009 SESSION#12
• • • •
EVALUATION DESIGN Basic Evaluation Terminology Process and Outcome Measures Data Analysis Types of Data Analysis Software: Advantages& Disadvantages
• Homework Assignment:
December 07, 2009 SESSION#13
- Develop draft process and outcome measures. -Conduct literature review and identify evaluation instruments for the proposed project. - Designing & Managing Programs (D&MP) Chapters 12 & 13
• • • •
Homework Assignment:
December 14, 2009 SESSION#14
Homework Assignment:
Class Presentation on: Social Marketing
EVALUATION DESIGN (PART II) Identify and/or develop evaluation measures Evaluation Databases Examples of Community Evaluations Class Presentation on: Cultural Competency -Finalize evaluation measures and complete a 5-page narrative on the (1) evaluation plan (i.e., process & outcome measures); (2) description of the data analysis plan; and (3) software to be used for the proposed project. (Due 12/18/09) - Designing & Managing Programs (D&MP) Chapters 10 & 11
• • • •
PROGRAM MONITORING & BUDGETS Budget Terminology Federal, State, County, Private Budgets Budget Monitoring & Tracking Budget Justifications -Complete a 2-4 page project budget and budget justification. (Due
12/18/09). -Complete a 3-4 page narrative/reflection piece (done separately by each participating student) describing the leadership dimensions of the process you undertook to create this product. What were the dynamics you encountered as part of your working group and how did those dynamics you encountered as part of your working group detract from or contribute to the process and product (e.g., did your team cohere easily, and if so, how; did much negotiation take place; was there a need for conflict resolution, etc.)? What portion of the product were you specifically involved in creating or, what was your major role in developing that product? What organizational dynamics within your client organization impeded or contributed to the effectiveness of the work (for example, how did individual clients or the culture of your client organization shape your work?). What theories/models did you learn in the course that you applied in the process of creating this document (again, please be sure to write a sentence or two defining those for the benefit of the reader)? Please think about how you may have addressed or solved a problem creatively through this process and product. (Due 12/18/09) December 18, 2009 SESSION #15
-All Class Presentations -Final Projects Due -Individual Reflection Papers Due
(revised 8-25-09)