Oregon Quality Framework for Afterschool and Summer Programs Administration Guide by OregonASK

Oregon's Quality Framework for Afterschool and Summer Programs Administration Guide

2012

Oregon Quality Framework for Afterschool and Summer Programs Administration Guide Welcome to the Quality Framework for Afterschool and Summer Programs! This Administration Guide will provide you with an overview of the continuous quality improvement process as well as research, resources and practical tips for implementing a quality improvement process. We know how daunting quality improvement can be and we hope this guide will engage you, your staff, and program stakeholders in a process that will assist you in increasing the quality of your program as well as positive outcomes for children and youth.

Why a Quality Improvement Framework? There are many factors that impact the quality of an afterschool or summer program. Staff competency, program structure, activities implemented, and family engagement practices are just a few. A quality framework encourages you to view and measure the quality of your program from an all-inclusive approach with an understanding that each of these components influences the others. Through the framework you are not just measuring the quality of the program itself, you are thoughtfully considering the outcomes you want for participants as well as the competencies staff need to have in order to get there.

Why should I start a Quality Improvement Process? A quality improvement process takes time and resources to implement; two things most programs can be in short supply of. Yet engaging in a quality improvement process has long-term, positive effects on not only the quality of your program but the sustainability of it as well. Programs who engage in this process can: • • •

Reduce staff turnover Increase family engagement Increase child and youth engagement

• •

Better align resources to create program sustainability Engage school and community stakeholders

Where do I begin in the Quality Improvement Process? There are many ways you can begin the quality improvement process. We recommend that you start the process in one of the entry points that makes sense for you. Some programs begin by measuring the quality of their program practices, others start at identifying what outcomes they want for their program participants, or you may consider the competencies of your staff. What is important to this process is to be thoughtful and intentional about where you begin and where you intend to be. Hopefully, through this method, you will begin to change the process and relationships that create your program; providing the foundation for continuous program improvement. What is important is to consider all three of the core quality components, standards, competencies, and outcomes, when engaging in this progression.

How to use the Quality Framework and this Guide The quality framework is divided into seven domains. If you are not sure where to begin, we recommend that you start with one domain. Consider a challenge or issue you may be having in your program. Perhaps you don’t feel that you have the level of family engagement you would like to have. Take a look at the Family, Community, and Schools Domain. Think about the outcomes that you would like to have for the participants in your program. Consider what program structures you need to have in place by conducting the program self-assessment. Review the core competencies to identify areas for further staff development. Next you can begin to formulate specific action steps for improving your program quality in one domain. Once you feel comfortable with the process you can look at other domains. The important thing is to fully engage in the process and be intentional in your strategies.

Letâ&#x20AC;&#x2122;s Start at Program Mission... The mission of your program should be the guiding force for all program decisions, resource allocations, and quality improvement efforts. Your mission, philosophy, and goals create a powerful roadmap for all components of this process. Take a few moments to write down your program mission, philosophy, and goals.

Program Mission

(A holistic vision of the values and philosophy of the program. The question that the mission statement should answer is: What is the overall unique purpose of this program? What services do we provide, how do we provide them, and to whom?)

Program Philosophy

(This dives further in to the "how" of the services you provide. Are you recreation focused? Youth development? Academic enrichment? What specific values and beliefs to you utilize when implementing your program?)

Program Goals

(What are the specific things you want to accomplish with your program? What do you hope children, youth, families, or other stakeholder will be able to do or accomplish as a result of your program?)

Next stepâ&#x20AC;Ś Identify an Entry Point in to the Quality Framework Consider whether you want to begin at Outcomes, Standards, or Core Competencies. Do you want to start with the complete framework or just one domain? What resources do you have to commit to this process? Will you need assistance from an expert? Advice from your colleagues? Participation from your funder or other stakeholder?

Program Example Identifies Outcomes

Improve Program Outcomes

Increase Quality

Accessess Quality/Competency

Accesses PD/TA Resources

Improve/Change Practices

Gather an Advisory Team Don’t go it alone on the journey for Quality Improvement! Gather a team that can help you solve problems, evaluate your progress and celebrate your successes. Depending on your program and where you start the process you could have an advisory team of two or a larger group of ten. What is important is to engage those stakeholders who can assist you not only in measuring the quality of your program but also implementing strategies for improvements. Some stakeholders to consider: • • • • • •

A child or youth participant A parent or family member Board member Staff Principal or community partner Consultant or coach

Who will you invite to be on your team?

Determine a Timeline What is a reasonable timeframe for engaging in this part of the process? Are there external factors (such as a monitoring visit or funder request) that is influencing your timeline? It can be easy to start a quality improvement process and then get stuck. Be realistic about what you can accomplish within a specific time frame. Remember, this is a continuous process so you can take the time you need to be reflective and intentional.

Things to consider for your timeline:

Anticipated timeline:

Ready to Go? Letâ&#x20AC;&#x2122;s Get Started! The next section of the guide provides you with tools for creating and implementing your continuous quality improvement process. Feel free to use some or all of the tools to get your team started.

Using S.M.A.R.T. Goals to Create an Action Plan

S pecific M easurable A ttainable R ealistic T imely

Specific: A specific goal has a much greater chance of being accomplished than a general *Who: *What: *Where: *When: *Which: *Why:

goal. To set a specific goal you must answer the six “W” questions: Who is involved? What do I want to accomplish? Identify a location. Establish a time frame. Identify requirements and constraints. Specific reasons, purpose or benefits of accomplishing the goal.

EXAMPLE: A general goal would be, “Get in shape.” But a specific goal would say, “Join a health club and workout 3 days a week.”

Measurable: Establish concrete criteria for measuring progress toward the attainment of each goal

you set. When you measure your progress, you stay on track, reach your target dates, and experience the exhilaration of achievement that spurs you on to continued effort required to reach your goal. To determine if your goal is measurable, ask questions such as……How much? How many? How will I know when it is accomplished?

ttainable: When you identify goals that are most important to you, you begin to figure out ways you can make them come true. You develop the attitudes, abilities, skills, and financial capacity to reach them. You begin seeing previously overlooked opportunities to bring yourself closer to the achievement of your goals. You can attain most any goal you set when you plan your steps wisely and establish a time frame that allows you to carry out those steps. Goals that may have seemed far away and out of reach eventually move closer and become attainable, not because your goals shrink, but because you grow and expand to match them. When you list your goals you build your self-image. You see yourself as worthy of these goals, and develop the traits and personality that allow you to possess them.

ealistic: To be realistic, a goal must represent an objective toward which you are both willing and able to work. A goal can be both high and realistic; you are the only one who can decide just how high your goal should be. But be sure that every goal represents substantial progress. A high goal is frequently easier to reach than a low one because a low goal exerts low motivational force. Some of the hardest jobs you ever accomplished actually seem easy simply because they were a labor of love. Your goal is probably realistic if you truly believe that it can be accomplished. Additional ways to know if your goal is realistic is to determine if you have accomplished anything similar in the past or ask yourself what conditions would have to exist to accomplish this goal.

imely: A goal should be grounded within a time frame. With no time frame tied to it there’s no sense of urgency. If you want to lose 10 lbs, when do you want to lose it by? “Someday” won’t work. But if you anchor it within a timeframe, “by May 1st”, then you’ve set your unconscious mind into motion to begin working on the goal.

S.M.A.R.T. Goals Worksheet Name: Date: State your goal (what do you want to achieve?) .............................................................................................................................................. .............................................................................................................................................. .............................................................................................................................................. .............................................................................................................................................. .............................................................................................................................................. ..............................................................................................................................................

Specific details: Who? ...................................................................................................................................... What?....................................................................................................................................... Where?..................................................................................................................................... When?...................................................................................................................................... Which?...................................................................................................................................... Why?........................................................................................................................................

Measureable: How much?............................................................................................................................... How often?................................................................................................................................ How many?...............................................................................................................................

Attainable: Is this goal achievable? Describe why or why not:........................................................................... ............................................................................................................................................... What obstacles do you think you will have to overcome?.................................................................. ...............................................................................................................................................

Relevant: Is it important to what you ultimately want to achieve? Describe why or why not:............................... ............................................................................................................................................... ...............................................................................................................................................

Time-based: When will this goal be accomplished?............................................................................................ ............................................................................................................................................... What are the key steps or progress points?.................................................................................... ............................................................................................................................................... ............................................................................................................................................... ...............................................................................................................................................

Bringing it All Together: Now combine all of the above elements into one statement that defines your goal. Be sure to include steps you will take to meet the goal, the timeline for completion, why it is important, and how you will know when you have met your goal. Goal 1:

..............................................................................................................

...................................................................................................................... ...................................................................................................................... ...................................................................................................................... ...................................................................................................................... ............................................................................................................. ...................................................................................................................... ...................................................................................................................... ...................................................................................................................... ...................................................................................................................... Goal 5:

Self-Evaluation Timeline

Responsible

July Aug Sept Oct Nov Dec Jan Feb Mar Apr May

Use the table below to plan your evaluation timeline.

Goal/Action Steps

Program Progress Chart

Quality Standard

Core Competency

Outcomes

Use the table below to track your progress as you work through each Domain in the Standards, Competencies and Outcomes.

Domain

Activities, Curriculum & Environment Diversity & Inclusion Families, Communities & Schools Health, Safety & Nutrition Highly Skilled Personnel Program Management Youth Development & Engagement

Staff Progress Chart

Activities, Diversity Curriculum & & Inclusion Environment

Families, Communities & Schools

Health, Safety & Nutrition

Highly Skilled Personnel

Program Management

Youth Development & Engagement

Use the table below to track your staffâ&#x20AC;&#x2122;s progress as they work through each Domain in the Core Competencies.

Staff Name

Staff Progress Chart

Activities, Diversity Curriculum & & Inclusion Environment

Families, Communities & Schools

Health, Safety & Nutrition

Highly Skilled Personnel

Use the table below to track each siteâ&#x20AC;&#x2122;s progress as they work through each of the Domains.

Site Name

Program Management

Youth Development & Engagement

Worksheet A Domain: Standards Self-assessment results:

Action steps for improvement:

Timeline:

Who is Responsible:

Worksheet B Domain: Core Competency Self-assessment results:

Action steps for improvement:

Timeline:

Who is Responsible:

Worksheet C Domain: Outcomes selected:

Standards Self-assessment results

Actions steps for improving outcomes

Timeline:

Who is Responsible:

Worksheet D Domain: Outcomes selected:

Core Competency Self-assessment results

Actions steps for improving outcomes

Timeline:

Who is Responsible:

Worksheet E Domain: Outcomes selected:

Tools selected to measure our progress on each outcome:

What will we do with this information?

Who do we share this information with?

Who is Responsible:

Worksheet F Domain: Outcomes selected:

Tools selected to measure our progress on each outcome:

What will we do with this information?

Who do we share this information with?

Who is Responsible:

Research and Resources by Domain Activities, Curriculum & Environment Anderson-Butcher, D., Lawson, H., Bean, J., Boone, B., Kwiatkoski, A., et al. (2004). Implementation guide: The Ohio Community Collaboration Model for school improvement. Columbus, OH: The Ohio Department of Education. Babo,

G., & Ramaswami, S. (2011). Principal evaluation and the application of the ISLLC 2008 standards functions by school superintendents: A national study. International Studies in Edu cational Administration (Commonwealth Council for Educational Administration & Management), 39(3), 77-90.

Cosden, M., Morrison, G., Albanese, A. L., & Macias, S. (2001) When homework is not home work: After school programs for homework assistance. Educational Psychologist, 36, 211-221. Cross, A. B., Gottfredson, D. C., Wilson, D. M., Rorie, M., & Connell, N. (2009). The impact of afterschool programs on the routine activities of middle-school students: Results from a random ized, controlled trial. Criminology & Public Policy, 8(2), 391-412. Fredricks, J. A., & Eccles, J. S. (2006). Extracurricular involvement and adolescent adjustment: Impact of duration, number of activities, and breadth of participation. Applied Developmental Science, 10(3), 132-146. Huang, D., Cho, J., Mostafavi, S., & Nam, H.H. (2010). What works? Common practices in high functioning afterschool programs across the nation in math, reading, science, arts, technology, and homework â&#x20AC;&#x201C; a study by the national partnership. University of California: National Center for Re search on Evaluation, Standards, and Student Testing. Institute of Education Sciences (IES). (2009, July). Structuring Out-of-School Time to Improve Academic Achievement. What Works Clearinghouse. Mahoney, J.L., Cairns, B.D., & Farmer, T.W., (2003). Promoting interpersonal competence and educational success through extracurricular activity participation. Journal of Educational Psychology, 95, 409-418. National Center for Education Evaluation and Regional Assistance, IES, U.S. Department of Education. (2008, June). The Evaluation of Enhanced Academic Instruction in After-School Programs: Findings After the First Year of Implementation. Washington, D.C. National AfterSchool Association. (2011, September). Core knowledge and competencies for afterschool and youth development professionals. McLean, VA. Nelson, I. (2009, Fall). The differential role of youth development program participation for latina/o adolescents. Afterschool Matters, 8, 20-31. Surr, W. (2012, Spring). A new approach to accountability: Creating effective learning environments for programs. Afterschool Matters, 15, 38-45. Schwarz, E., & Stolow, D. (2006). Twenty-first century learning in afterschool. New Directions For Youth Development, 110, 81-99.

Diversity & Inclusion Bathgate, K., & Silva, E. (2010). Joining forces: the benefits of integrating schools and community providers. New Directions For Youth Development, 2010(127), 63-73. Beecher, M., & Sweeny, S.M. (2008). Closing the achievement gap with curriculum enrichment and dif ferentiation: One schoolâ&#x20AC;&#x2122;s story. Journal of Advanced Academics, 19, 502-530. doi: 10.4219/ jaa-2008-815 Galloway, F., & McAllister Shea, M. (2009) Does your organization welcome participants with disabili ties? A new assessment tool. Afterschool Matters 9, 13-19. Lee, S., Borden, L.M., Serido, J., & Perkins, D.F. (2009). Ethnic minority youth in youth programs: Feelings of safety, relationships with adult staff, and perceptions of learning social skills. Youth Society, 41, 234-255. doi: 10.1177/0044118X09334805 Keller, T. E., Bost, N. S., Lock, E. D., & Marcenko, M. O. (2005). Factors associated with participation of children with mental health problems in structured youth development programs. Journal of Emotional & Behavioral Disorders, 13(3), 141-151. Murray, C., & Pianta, R. C. (2007). The importance of teacher-student relationships for adolescents with high incidence disabilities. Theory into Practice, 46(2), 105-112. Yager, R.E., & Yager, S.O. (2012). Enhancing success with Iowa chautauqia when considering distributed school leadership: How it helps and hinders student learning. National Forum of Educational Administration & Supervision Journal, 29(2), 4-22.

Families, Communities & Schools Anderson-Butcher, D., Stetler, E. G., & Midle, T. (2006). A case for expanded school-community partnerships in support of positive youth development. Children & Schools, 28(3), 155-163. Beets, M. W., Flay, B. R., Vuchinich, S., Acock, A. C., Li, K. K., & Allred, C. (2008). School climate and teachersâ&#x20AC;&#x2122; beliefs and attitudes associated with implementation of the positive action program: A diffusion of innovations model. Prevention Science, 9(4), 264-275. Cornelli Sanderson, R., & Richards, M. (2010). The after-school needs and resources of a low-income urban community: Surveying youth and parents for community change. American Journal of Community Psychology, 45(3), 430-440. Durlak, J., Taylor, R., Kawashima, K., Pachan, M., DuPre, E., Celio, C., & Weissberg, R. (2007). Effects of positive youth development programs on school, family, and community systems. American Journal of Community Psychology, 39(3-4), 269-286. Hall, K. W., Williams, L.M., & Daniel, L. G. (2010). An afterschool program for economically disadvan taged youth: Perceptions of parents, staff, and students. Research In The Schools, 17(1), 12- 28. Mahoney, J. L., Levine, M. D., & Hinga, B. (2010). The development of after-school program educa tors through university-community partnerships. Applied Developmental Science, 14(2), 89- 105. doi:10.1080/10888691003704717 Public Agenda. (2004). All Work and No Play. Listening to What Kids and Parents Really Want from Out-of-School Time. New York, NY. Smith, C. & Van Egeren, L. (2008, Fall). Bringing in the community: partnerships and quality assurance in 21st century community learning centers. Afterschool Matters, 15-33.

Health, Safety & Nutrition Beets, M. W., Wallner, M., & Beighle, A. (2010). Defining standards and policies for promoting physical activity in afterschool programs. Journal Of School Health, 80(8), 411-417. Center for Collaborative Solutions Healthy Behaviors Initiative. (2010, March). Changing Lives, Saving Lives. A Step-by-Step Guide to Developing Exemplary Practices in Health Eating, Physical Ac tivity and Food Security in Aftershool Programs. Sacramento, CA. Collaborative for Academic, Social, and Emotional Learning (CASEL). (2007). The Impact of After- School Programs That Promote Personal and Social Skills. Chicago, IL: Desktop Edit Shop, Inc. National Institute on Out-of-School Time. (2006, April). Healthy Choices Afterschool. New York, NY. Ohio Afterschool Network. (2011). Ohio Kids on the Move: Physical Activity Guidelines for Afterschool Programs. Columbus, OH. Pierce, K., Bolt, D., & Vandell, D. (2010). Specific features of after-school program quality: associa tions with children’s functioning in middle childhood. American Journal Of Community Psychol ogy, 45(3-4), 381-393.

Highly Skilled Personnel Cole, P. (2011, Spring). Building an afterschool workforce: Regulations and beyond. Afterschool Mat ters, 13, 12-21. Hartje, J., Evans, W., Killian, E., & Brown, R. (2008). Youth worker characteristics and self-reported competency as predictors of intent to continue working with youth. Child & Youth Care Forum, 37(1), 27-41. doi:10.1007/s10566-007-9048-9 Hartmann, T., Good, D., & Edmunds, K. (2011, Fall). Exito: Keeping high-risk youth on track to gradu ation through out-of-school time supports. Afterschool Matters, 14, 20-29. Koyama, J. (2011). Principals, power, and policy: Enacting supplemental educational services. Anthro pology & Education Quarterly, 42(1), 20-36. Peck, C., & Reitzug, U.C. (2012). How existing business management concepts become school leader ship fashions. Education Administration Quarterly, 48(2), 347-381. doi:10.1177/0013161X11432924 Reardon, R. (2011). Elementary school principals’ learning-centered leadership and educational out comes: Implications for principals’ professional development. Leadership & Policy In Schools, 10(1), 63-83. doi:1080/15700760903511798 Renihan, P., & Noonan, B. (2012). Principals as assessment leaders in rural schools. Rural Educator, 33(3), 1-8. Robinson, V.M.J., Lloyd, C. A., & Rowe, K.J. (2008). The impact of leadership on student outcomes: An analysis of the differential effects of leadership types. Educational Administration Quarterly, 44, 635-674. doi:10.1177/0013161X08321509 Summers, J. & Price, L. (2008). Administrative management capacity in out-of-school time organiza tions: An exploratory study. Retrieved July 20, 2012 from http://www.wallacefoundation.org/ knowledge-center/after-school/financial-management-for-nonprofits/Documents/Administra tive-Management-Capacity-in-Out-of-School-Time-Organizations.pdf Samuelson, L. (2007, Spring). After the last bell: The multiple roles of principals in school-based after school programs. Afterschool Matters, 6, 6-14. Timperley, H. (2011). Knowledge and the leadership of learning. Leadership & Policy In Schools, 10(2), 145-170. doi:10.1080/15700763.2011.557519

Program Management Birmingham, J., Pechman, E. M., Russell, C. A., Mielke, M. (2005). Shared features of high-performing after-school programs: A follow-up to the TASC evaluation. Washington DC: Policy Studies As sociates, Inc. Finance Project (2007a). Building professional development systems for the afterschool field. (The Afterschool Investments Brief ). Washington, DC: Sandel, K. & Dobbins-Harper, D. Finance Project (2007b). Estimated federal investment in Out-of -School Time. The Finance Project. Washington, DC: Author. Granger, R. (2010). Understanding and improving the effectiveness of after-school practice. American Journal of Community Psychology, 45(3), 441-446. Harvard Family Research Project. (2011, December). Afterschool Evaluation 101: How to Evaluate an Expanded Learning Program. Cambridge, MA. Harvard Family Research Project. Measurement Tools for Evaluating Out-of-School Time Programs 1.1 Academic Achievement. Herman, J.L. (2010). Coherence: Key to Next Generation Assessment Success (AACC Report). Los Angeles, CA: University of California. New York State AfterSchool Network. Program Quality Self-Assessment Tool. NY Oregon Department of Education. Leading Indicators for Program Quality. Smith, C., Peck, S., Denault, A., Blazevski, J., & Akiva, T. (2010). Quality at the point of service: Profiles of practice in after-school settings. American Journal of Community Psychology, 45(3), 358-369. The Forum for Youth Investment. (2009, January). Measuring Youth Program Quality. A Guide to Assessment Tools, Second Edition Executive Summary. Washington, D.C: Published by The Forum for Youth investment. The Forum for Youth Investment. (2009, January). Measuring Youth Program Quality. A Guide to Assessment Tools, Second Edition. Washington, D.C: Published by The Forum for Youth Investment. Vandell, D. L., Reisner, E. R., Pierce, K. M. (2007). Outcomes linked to high-quality afterschool programs: Longitudinal findings from the study of promising afterschool programs. Washington, DC: Policy Studies Associates, Inc.

Youth Development & Engagement Achenbach, T. M. (1991). Manual for the Child Behavior Checklist/4-18 and 1991 Profile. Burlington, VT: University of Vermont Department of Psychiatry. Benson, P.L. (2003). Developmental assets and asset-building community: Conceptual and empirical foundations. In R.M. Lerner & P.L. Benson (Eds.)., Developmental assets and asset building communities: Implications for research, policy and practice (pp. 19-43). Norwell, MA:Kluwer. Bradshaw, C. P., Koth, C. W., Thornton, L. A., & Leaf, P. J. (2009). Altering school climate through school-wide positive behavioral interventions and supports: Findings from a group-randomized effectiveness trial. Prevention Science, 10(2), 100-115. Deschenes, S. N., Arbreton, A., Little, P. M., Herrera, C., Grossman, J. B., Weiss, H. B., et al. (2010). Engaging older youth: Program and city-level strategies to support sustained participation in out-of-school time. Cambridge, MA: Harvard Family Research Project. Deschenes, S., McDonald, M. & McLaughlin, M., (2004). Youth organizations: From principles to prac tice. In S.F. Hamilton & M.A. Hamilton (Eds.), The youth development handbook: Coming of age in American communities (pp. 25- 50). Thousand Oaks, CA: Sage. Hartmann, T., Good, D., & Edmunds, K. (2011, Fall). Exito: Keeping high-risk youth on track to gradu ation through out-of-school time supports. Afterschool Matters, 14, 20-29. High/Scope Educational Research Foundation. (2005). Youth PQA (Program Quality Assessment) Form B – Organization Items. Ypsilanti, MI: HIGH/SCOPE PRESS. High/Scope Educational Research Foundation. (2005). Youth PQA (Program Quality Assessment) Form A – Program Offering Items, Older Youth – Grades 4 – 12. Ypsilanti, MI: HIGH/SCOPE PRESS. High/Scope Educational Research Foundation. (2005). Youth PQA (Program Quality Assessment) Ad ministration Manual. Ypsilanti, MI: HIGH/SCOPE PRESS. Institute of Education Sciences (IES). (2009, July). Structuring Out-of-School Time to Improve Aca demic Achievement. What Works Clearinghouse. Nelson, L. P., McMahan, S. K., & Torres, T. (2012). The impact of junior high school community in tervention project: Moving beyond the testing juggernaut and into a community of creative learners. School Community Journal, 22(1), 125-144. Peck, S. C., Roeser, R. W., Zarrett, N., & Eccles, J. S. (2008). Exploring the roles of extracurricular activity quantity and quality in the educational resilience of vulnerable adolescents: Variable- and pattern-centered approaches. Journal of Social Issues, 64(1), 135-156. Stonehill, R., Lauver, S., Donahue, T., Naftzger, N., McElvain, C., & Stephanidis, J. (2011). From after-school to expanded learning: a decade of progress. New Directions For Youth Develop ment, 2011(131), 29-41. doi:10.1002/yd.406 Sunset Neighborhood Beacon Center. (2008). The Best of Both Worlds. Aligning Afterschool Programs with Youth Development Principles and Academic Standards. San Francisco, CA. Vance, F. (2010). A Comparative Analysis of Competency Frameworks for Youth Workers in the Out-of- School Time Field. Child & Youth Care Forum, 39(6), 421-441.