Multiple Measures

The mission of the Orleans/Niagara Board of Cooperative Educational Services is to be the premier provider of innovative and effective solutions for the evolving needs of our component districts and other learning collaboratives.

Clark J. Godshall, Ed.D. District Superintendent

Mickey Edwards Coordinator of Instruction & Curriculum

Multiple Measures Overview Webinar and Documents To address components 1, 2, and 5 of the 9 requirements for 302.9b (Evaluator Training Criteria)

Please initial your Evaluator Training Criteria Sheet with the date and title when you‘ve completed the viewing of the webinar and read through the associated documents.

District Superintendent & Orleans Career and Technical Education Center  4232 Shelby Basin Road, Medina, New York 14103 Niagara Career and Technical Education Center & Niagara Academy  3181 Saunders Settlement Road, Sanborn, New York 14132 Niagara Conference and Technology Center  4124 Saunders Settlement Road, Sanborn, New York 14132 1-800-836-7510 ALBION  BARKER  LEWISTON-PORTER  LOCKPORT  LYNDONVILLE  MEDINA  NEWFANE  NIAGARA FALLS NIAGARA-WHEATFIELD  NORTH TONAWANDA  ROYALTON-HARTLAND  STARPOINT  WILSON

1|Page

Table of Contents Overview…………………………………….………..3 Observations Review………………………………..4 Conferences…………………………………………..5 Samples and Resources………….…………….6 Reflections…………………………………………….13 Samples and Resources……….………………14 Artifacts………………………………………..………19 Samples and Resources………………………..20 Portfolios……………………………………………….24 Samples and Resources………………………..25 Surveys……………………………………….………..50

2|Page

Establishing Multiple Measures of Teacher Effectiveness SETTING THE STAGE: Regular and consistent assessment of teacher skill and knowledge at various points along the educator‘s career continuum will result in sound hiring and performance management decisions, needs-based professional development, and data to support opportunities for career growth. But determining teacher effectiveness is no easy task. It first requires a thorough understanding of what an effective teacher does and the competencies needed to carry out these tasks. The process should also take into account the specialized roles and context of teachers. Defining effectiveness also requires using valid and reliable tools for assessing whether an individual teacher possesses these competencies.

Action Principles For District    

 

Determine what skills and knowledge an effective teacher must possess.

Determine what outcomes an effective teacher must produce.

Determine if the context and role in which teachers work require that additional competencies be identified.

Develop a comprehensive system, including multiple valid and reliable tools, and regular assessments for determining whether a teacher possesses the necessary skills, knowledge, and competencies.

Establish clear outcomes for identified levels of effectiveness, from remediation, to professional development, to positive incentives.

Involve teachers in the process of defining teacher effectiveness.

3|Page

MULTIPLE MEASURE #1 OBSERVATIONS Measures observable classroom processes including specific teacher practices, aspects of instruction, and interactions between teachers and students.

NYS ALIGNMENT: III, IV, V STRENGTHS:    

Provides rich information Considered a fair and direct measure by stakeholders Can be used in various subjects, grades, and contexts, depending on the observation tool Can provide useful information for formative and summative purposes

WEAKNESSES:   

Dependent on the quality of the rubric and observation tool Requires evaluator training to ensure validity and reliability Multiple observations can be intensive, specifically relating to evaluator scheduling

EXAMPLES:    

Walkthroughs Formal Observations Peer Reviews Videotaped Observations

Notes from previous workshop:    

Observation is just one tool to evaluate teachers.

60% of the Teacher Evaluation Score can be any configuration of multiple measures as decided locally and collectively bargained.

Observation alone will not address each of the teaching standards regardless of the rubric that is being used.

The rubric that your district chooses is a way to measure the effectiveness of the NY State Teaching Standards.

4|Page

MULTIPLE MEASURE #2 CONFERENCES Measures non-observable classroom processes including knowledge of students and student learning, content and instructional planning, assessments for learning, and professional roles and responsibilities.

NYS ALIGNMENT: I, II, V, VI, VII STRENGTHS:    

Provides information about non-observable practices essential to teaching and learning Considered a fair and direct measure by stakeholders Can be used in various subjects, grades, and contexts Can provide useful information for formative and summative purposes

WEAKNESSES:  

Dependent on the quality of the rubric and observation tool Requires evaluator training to ensure validity and reliability

EXAMPLES:    

Pre-observation conference Post-observation conference End of year review Professional goal setting conference

5|Page

Samples and Resources‌

Pre-Observation Lesson Plan Form

6|Page

Teacher Date of Visit

Grade/Content Area Time

Content Standards: Identify one or two primary local, state, or national curricular standards this lesson is designed to help students attain. How will the learning tasks lead students to attain the identified standards?

Learner Background: Describe the students‘ prior knowledge or skill related to the learning objective(s) and the content of this lesson, using data from pre-assessment as appropriate. How did the students‘ previous performance in this content area or skill impact your planning for this lesson?

Student Learning Objective(s): Identify specific and measurable learning objectives for this lesson.

Assessment: How will you ask students to demonstrate mastery of the student learning objective(s)? Attach a copy of any assessment materials you will use, along with assessment criteria.

Materials/Resources: List the materials you will use in each learning activity including any technological resources.

Learning Activities: Identify the instructional grouping (whole class, small groups, pairs, individuals) you will use in each lesson segment and approximate time frames for each.

Initiation: Briefly describe how you will initiate the lesson. (Set expectations for learning; articulate to learners what they will be doing and learning in this lesson, how they will demonstrate learning, and why this is important)

Lesson Development: Describe how you will develop the lesson, what you will do to model or guide practice, and the learning activities students will be engaged in order to gain the key knowledge and skills identified in the student learning objective(s).

Closure: Briefly describe how you will close the lesson and help students understand the purpose of the lesson. (Interact with learners to elicit evidence of student understanding of purpose(s) for learning and mastery of objectives.)

7|Page

Individuals Needing Differentiated Instruction: Describe 1 to 3 students with learning differences. These students may be special or general education students and need not be the same students for each lesson. Students may represent a range of ability and/or achievement levels, including students with IEP‘s, gifted and talented students, struggling learners, and English language learners.

Note: Differentiated instruction may not be necessary in every lesson. However, it is expected that each teacher will demonstrate the ability to differentiate instruction in order to meet the needs of students with learning differences.

Which students do you anticipate may struggle with the content/learning objectives of this lesson?

Student Evidence that the student needs How will you differentiate instruction in this lesson differentiated to support Name instruction student learning?

Which students will need opportunities for enrichment/higher level of challenge?

Student Evidence that the student needs How will you differentiate instruction in this lesson differentiated to support Name instruction student learning?

Notes from the pre-conference

8|Page

OBSERVATION SUMMARY Attachment C-1 Teacher __________________ Grade/Content area Date of Visit Time CCI INDICATOR & DEFINING ATTRIBUTES IA. The teacher promotes a positive learning environment -Rapport -Communication of expectations for IB. Theachievement teacher maintains appropriate standards of behavior -Physical environment and standards behavior are in maintained.

IC. -Rules The teacher engagesofthe students the activities of the lesson -Student engagement ID. The teacher effectively maintains routines and -Re-engagement transactions -Effectiveness Instruction IIA. The teacher presents appropriate lesson content -Choice of content -Level of difficulty -Accuracy IIB. The teacher creates a structure for learning -Initiations -Closures IIC. The teacher develops the lesson to promote achievement of the lesson objectives -Lesson development -Use of instructional arrangements and materials IID. The teacher uses appropriate questioning techniques -Cognitive level -Responding to students IIE. The teacher communicates clearly using -Opportunities for students involvement precise language and acceptable oral expressions -Precision of communication -Clarity of speech Assessment -Oral expressions IIIA. The teacher monitors student understanding of the lesson and adjusts when necessary Additional Comments/Recommendations: (Use back if needed) Teacher’s signature/date Administrator’s signature/date 9|Page

OBSERVATION SUMMARY Attachment C-2 (Administrator) Teacher Grade/Content Area Date Time I.

MANAGEMENT

II.

INSTRUCTION (Plan/Teach)

III.

ASSESSMENT

Teacher’s signature

Administrator’s signature

Date

Supervisor’s signature (as appropriate)

10 | P a g e

Teacher Performance Rubric Summary Description The Teacher Evaluation Committee created the new yearly Teacher Performance Rubric Summary to replace the End of Year Teacher Performance Narrative. The new Rubric Summary provides the teacher and the supervisor/evaluator with clear indicators of accomplishments and for continuous teacher improvement. The descriptive statements in the rubric are linked to the foundational skills and competencies as well as the discipline-based professional teaching standards of Connecticut’s Common Core of Teaching (CCT). The rubric is organized around the six CCT instructional standards (i.e. planning, instructing, assessing, adjusting professional and ethical practice, reflection and continuous learning, leadership and collaboration). The rubric summary is used as an analytic tool to provide feedback for teachers to help them plan for further professional growth. As part of the dialog at the End of Year Performance Conference, the teacher and supervisor/evaluator will use the rubric to assess the yearly performance and mutually agree on areas of focus for the next school year.

The rubric being used is based on the Connecticut Common Core of Teaching (CSDE, 1999) and Enhancing Professional Practice: A Framework for Teaching by Charlotte Danielson (ASCD, 1996). These are ―best practices‖ for all teachers. As the rubric is modified by the State Department of Education, Trumbull Public Schools will use the new version(s).

The four performance continuum columns for each discipline rubric, reading from left to right, were given a rating of 1 through 4 by the Teacher Evaluation Committee, as outlined below: 1.

2.

3.

4.

Needs support to meet district expectations (column 1) Demonstrates growth toward district expectations (column 2) Meets district expectations (column 3) Exceeds district expectations** (column 4 -- In order for a teacher to receive a rating of 4, the competency must be consistently practiced by the teacher.)

The ratings from the rubric will be recorded on Attachment E (Teacher Performance Rubric Summary). During the dialog between the teacher and supervisor/evaluator during conferences, the teacher will share artifacts (student data) from the classroom, and through formal and informal observation(s) by the supervisor/evaluator. Attachment E will be completed later by the supervisor/evaluator and given to the teacher for signature and further discussion if necessary. If any issues/concerns arise, the teacher may request an additional conference with the supervisor/evaluator.

New teachers (years 1, 2, 3, 4), and teachers receiving increased supervisory support on the Teacher Assistance Track will use Attachments E and E-1 at mid-year and end of year conferences.

11 | P a g e

Example from Delaware Department of Education: Teacher

Evaluator

School

Date of Conference

Grade(s)

Subject Area(s)

The teacher completes this form and provides a copy to the evaluator before the Preobservation Conference. The detailed lesson plan, assessments, and any other relevant documents must be attached.

1. Outline your goals for this lesson. How does this lesson fit into overall goals of the unit?

2. How will you determine student progress toward lesson and/or unit goals?

3. Describe any challenges you are having with this class and describe how you are addressing these challenges.

4. Describe any physical characteristics of the classroom that negatively impact learning.

(Examples: excessive heat, excessive external noise, classroom areas obstructed from view)

5. Provide any additional comments or information you would like to share with your evaluator (optional).

12 | P a g e

MULTIPLE MEASURE #3 REFLECTIONS Teachers report on what they are doing in and outside of the classroom. May focus on broad and overarching aspects of teaching in all contexts or on specific subject matter, content areas, grade levels.

NYS ALIGNMENT: I, II, V, VI, VII STRENGTHS:    

Used to report otherwise unobservable factors that may affect teaching, such as knowledge, intentions, expectations, and beliefs.

Provides the unique perspective of the teacher.

Is easily implemented and cost-efficient.

Can assemble a large amount of information at once (when gathered for the whole faculty).

WEAKNESSES:  

The reliability and validity of self-report measures are not fully established and depend on the instrument used.

Self-report measures should not be used as the sole or primary measure in teacher evaluation.

EXAMPLES:     

Surveys Instructional Logs Interviews Reflections Conference Forms

13 | P a g e

Self-Reflective Assessment

If reflective practice is part of your district‘s evaluation or appraisal process, this component can be set up easily with any number of applicable self-assessment forms. Optionally, this part of the process can even be initiated by the end user, while the administrator stays abreast of the process, and gets involved when it‘s time for review, sign-off, and any other action required.

A Template for Self-Reflection on Teaching       

Ask yourself the following questions immediately after session: What were my goals for this class period?

What instructional strategies did I use to achieve those goals?

How did I try to assess student understanding of this content?

How satisfied am I with their progress?

Were they attentive during class? Did they interact, take notes, etc.?

Were there other observers in the session (e.g. other instructors, the teaching  assistants)? If so, what did they observe?  What areas should I target for improvement?  What can I do to improve the teaching of this material?

14 | P a g e

Self Reflection Template

Purpose: The self-reflection will serve three purposes: (1) to create a time and place for you to reflect on your practice and assess your performance; (2) to help inform and facilitate a reflective dialogue on your performance with your appraiser; and (3) to help you identify areas for improvement and areas for professional growth.

Process: It is suggested that you complete a self-reflection in private at least three times a year, prior to the Goal Setting, Progress, and End of Year Conferences held with your appraiser.

Although it is recommended you engage in the self-reflection process below, doing so is not a requirement. Also note that your appraiser will not collect your self-reflection.

Part 1: Suggested Prompts for Reflection Goal-Setting Conference Prompts 1. In what areas did my students grow last year and where did they struggle, based upon state-, district- or teacher-created assessments?

2. Based on my students‘ growth from last year, what are areas of strength in my instructional practice?

3. Based on my students‘ growth from last year, what are the areas would I like to focus on this year for my professional development? o What types of activities do I think would be helpful to continuously grow and improve my instructional practice and impact on student learning? o What specific supports will I need to improve my instructional effectiveness this year? o What can I do to make sure my support team knows my needs and can help guide me through tough decisions?

4. What strategies can I employ to help improve student learning outcomes this school year?

5. What are ways that I can collect, track, and use student data to drive my instructional choices to ensure that I meet the educational needs of all the students in my classroom?

6. In what ways could I improve and monitor the extent to which I set and implement effective discipline management procedures?

7. How can I build a positive and respectful classroom environment to help boost student learning?

8. How can I plan so that the reflective cycle of inquiry (plan-teach-assess-reflect) is routinely part of my daily practice?

15 | P a g e

Progress Conference Prompts: 1. As I review my students‘ data thus far, are they making significant progress? In what area(s) are the students excelling/struggling?

2. Am I differentiating instruction sufficiently to meet the instructional needs of all my students? o Am I providing rigor to the assignments? o Are my instructional choices meaningful and relevant to my students? o What changes do I need to make now to ensure success for all my students?

3. What specific skills, protocols, and tools do I need to improve my instructional effectiveness throughout the rest of the school year?

4. What is the progress that I have made toward my professional learning/development target to date? o What are some examples to show that I am making progress toward or have accomplished my professional learning target (e.g., student data, student work, and change in instructional practice)? o If I have not made as much progress as I would have hoped, what are specific supports (e.g., tools, protocols, PD opportunities) that I would need to achieve my professional learning/development target?

5. Are there any new focus areas that I would like to add or amend in my development plan?

Or additional activities I would like to consider?

End of Year Conference Prompts 1. What is the progress that I made toward my professional learning/development targets this year?

2. Did my students make significant learning gains? What were some of instructional practices that allowed my students to grow and learn this school year? What practices did not promote growth?

3. How can I prepare/change my instructional practices for next year to drive improved student learning?

4. How can I use data more effectively to drive instruction and ensure that all of my students are successful next year?

5. What has been the area of my greatest professional learning and development this school year? To what can I attribute to that growth?

6. In what ways should I focus my professional growth and learning for next year?

16 | P a g e

Teacher Self-Reflection Template Please Note: This template is intended to provide guidance, and reflects one suggested way to self-reflect. Completion of this template is not a requirement, and appraisers will not collect this template.

Instructional Practice Strengths/Development: Using the Instructional Practice rubric, for each domain identify at least one criterion as an area of strength, at least one criterion as an area for development, and provide a rating for each criterion you identify.

Instructional Practice Strength/Successes

Instructional Practice Area for Development/Growth

Individual Criterion Ratings: Rank your performance on each criterion on a scale from 14, based upon the Instructional Practice rubric.

Develops student learning goals: _________ Collects, tracks, and uses student data to drive instruction: __________ Designs effective lesson plans, units, and assessments: __________ Facilitates organized, student-centered, objective-driven lessons: _________ Checks for student understanding and responds to student misunderstanding: ________ Differentiates instruction for student needs by employing a variety of instructional strategies: ________ Engages students in work that develops higher-level thinking skills: _______ Maximizes instructional time: __________ Communicates content and concepts to students: _________ Promotes high academic expectations for students: _________ Students actively participating in lesson activities: _________ Sets and implements discipline management procedures: __________ Builds a positive and respectful classroom environment: __________

17 | P a g e

Professional Expectations Strengths/Development: Using the Professional Expectations rubric, identify at least one criterion as an area of strength, at least one criterion as an area for development, and provide a rating for each criterion you identify.

Professional Expectations Strength/Successes

Professional Expectations Area for Development/Growth

Individual Criteria Ratings: Rank your performance on each criterion on a scale from 1-4, based upon the Professional Expectations rubric.

Complies with policies and procedures at school: ________ Treats colleagues with respect throughout all aspects of work: ________ Complies with teacher attendance policies: ________ Dresses professionally according to school policy: _________ Collaborates with colleagues: _________ Implements school rules: __________ Communicates with parents throughout the year: __________ Seeks feedback in order to improve performance: __________ Participates in professional development and applies learning:_________

18 | P a g e

MULTIPLE MEASURE #4 ARTIFACTS Used to analyze classroom artifacts to determine the quality of instruction in a classroom. May include lesson plans, teacher assignments, scoring rubrics, and student work.

NYS ALIGNMENT: I, II, III, V STRENGTHS: 



Can be a useful measure of instructional quality when a validated measure is used, raters are well trained for reliability, and assignments show sufficient variation in quality.

Practical and feasible because artifacts have already been created for the classroom.

WEAKNESSES:   

More validity and reliability research is needed.

Training knowledgeable scorers can be costly but is necessary to ensure validity.

In terms of feasibility and validity, this method may be a promising middle ground between full observation and less direct measures such as selfreporting.

EXAMPLES:      

Lesson Plans Unit Plans Curriculum Maps I.E.P.s Student Work Assessments

19 | P a g e

Lesson Plan Self-reflection and Evaluation Self-evaluation is a powerful tool that will help you become a better teacher. Reflecting on and evaluating your teaching after a lesson is over will give you insights that may save you lots of trouble later. Even a few brief evaluative notes on a lesson plan will help you immensely the next time you teach that lesson.

In the rush of teaching, you may be tempted to skip self-evaluation. There are always plenty of other pressing things that need to be done. But if you don't evaluate yourself, you will be the loser. You will be surprised how much you forget if you don't write your ideas down, and you may end up making the same mistakes over and over. Also, you will be surprised at how just a few minutes of reflective writing can help you discover things you would have otherwise not noticed.

Self-evaluation is a very important part of every lesson even though it typically takes place after the lesson is over. It requires you to think back on the lesson and consider the answers to general questions like these: 1.

2.

3.

4.

5.

What went well in this lesson? Why? What problems did I experience? Why? Was it “student centered”? Should it have been?

What could I have done differently?

What did I learn from this experience that will help me in the future?

It is also helpful to break the lesson plan into its different components: 1. Preparation and research - Was I well prepared? - What could I have done differently?

2. Written plan – Was I organized? Did the written format work? Is there a better form?

3. Presentation – Were the students involved? Was I clear in my presentation? How was the pacing?

4. Assessment – Does my method(s) of assessment measure what I want? How did the class do? What should I change for next time?

Take the time (while you still have it!) to seriously reflect on your lesson. It is important to train yourself to be self-reflective/critical and ―process‖ what your experience was. It is the primary way to learn from past experience. After a while it will become second nature.

20 | P a g e

Sample Narrative reflections: Sample Self-evaluation (written in teacher's log after lesson) #1 This group of students is great! They remembered a lot from the last lesson. All of them demonstrated eagerness to learn.

Overall, the lesson went really well. I called on students by name and used information about them (from the information sheets they filled out during the first class period) in the examples I gave.

Unfortunately, I spent so much time on the presentation stage of the lesson that we had to rush through the other activities. I think I had too much vocabulary to cover. Next time, I will start with the specific information on the back of the application.

I felt like I talked too much. The more timid students didn't get a chance to practice speaking. I wish I had included more communicative activities involving the students. That way they would get more of the practice they need.

Sample Self-Evaluation #2 During the first part of the lesson I felt insecure. I temporarily lost my focus because of the new circumstances and because many of the students arrived late. As a result, the lesson did not start well. It was choppy and disjointed.

Once I got into the lesson, I felt more comfortable. When we began the exercises and the students started participating, things got better. The laughter and interactions at the end humanized what had started out to be a structured, dry lesson. I could actually sense that learning was taking place and that I was directing that learning.

From this experience I learned that I need to create a more open teaching style. I must also remember to spend less time on explanations and get right into the practice activities. They were more effective than reading the story and underlining the prepositions. I also realized more than ever the importance of being flexible. Changes may be necessary in the best of lesson plans.

21 | P a g e

Self – assessment form (use the back if necessary) 1. What went well in this lesson? Why?

2. What problems did I experience? Why?

3. Was it ―student centered‖? Should it have been?

4. What could I have done differently?

5. What did I learn from this experience that will help me in the future?

6. Preparation and research - Was I well prepared? - What could I have done differently?

7. Written plan – Was I organized? Did the written format work? Is there a better form?

8. Presentation – Were the students involved? Was I clear in my presentation?

How was the pacing?

9. Assessment – Does my method of assessment measure what I want? How did the class do? What should I change for next time?

22 | P a g e

Another template: Teacher

Evaluator

Date & Time of Observation Grade(s)

Subject Area(s)

The teacher may choose to complete this form, in whole or in part, and bring it to the Postobservation Conference. The teacher has full discretion as to whether this form is completed and/or shared with the evaluator.

Reflect and comment on the different aspects of your instructional delivery for this particular lesson. To what extent was the lesson effective? What would you do differently to improve the lesson?

Were the following aspects of your instructional delivery effective? Why or why not?

What would you do differently to improve the lesson?

Instructional Strategies Student Grouping(s) Student Activities Materials, Resources, and Technology Assessment Methods

Classroom Management/Student Behavior Student Engagement/Interest

23 | P a g e

MULTIPLE MEASURE #5 PORTFOLIO Used to determine a large range of teaching behaviors and responsibilities.

NYS ALIGNMENT: I, II, III, IV, V, VI, VII STRENGTHS:    

Is comprehensive and can measure aspects of teaching that are not readily observable in the classroom.

Can be used with teachers of all fields.

Provides a high level of credibility among stakeholders.

Good tool for teacher reflection and improvement.

WEAKNESSES:     

Time-consuming for both teachers and scorers.

Scorers should have content knowledge.

The stability of scores may not be high enough to use for high-stakes assessment.

Difficult to standardize.

Portfolios represent teachers‘ exemplary work but may not reflect everyday classroom activities.

EXAMPLES:      

Instructional Documents Student Work Reflections Professional Goals Conference Forms Surveys

24 | P a g e

Annual Performance Appraisal/Goal Setting Form This annual practice is designed to have the staff member and their supervisor complete the Performance Appraisal/Goal Setting form together. The completed form should contain mutual agreed on goals and comments. Completed forms are to be turned in by April 15th to HR for the employee‘s personnel file unless a different turn in date has been communicated to HR. Name: _____________________________________ Date _______________________ Position: __________________________ Supervisor: ____________________________

1. Review the status of each of your professional or job related goals and objectives identified last year, including any changes/adjustments made throughout the year.

2. What specific developmental areas would you like to concentrate on in the coming year? (types of training, development or skill building)

3. Are there issues you would like to address to help you succeed in your position? How would you envision changing or overcoming these issues?

4. List the impact this position has on the team within your department and to the College.

5. How could we collectively make a better team environment and working relationships?

6. What suggestions do you have for process improvement that you would like to see your department consider? (please consider ideas gathered through conferences, social networks, peers, etc.)

7. What are your thoughts or ideas to reduce costs in your area, considering short term and long term impact?

8. List professional or job related goals and objectives for the coming year.

This Performance Appraisal/Goal Setting form has been prepared by me and my supervisor and future goals have been agreed upon.

_____________________________ _____________________________ Staff Signature Supervisor Signature

____________ Date 25 | P a g e

From: Linking Teacher Evaluation and Student Learning

by Pamela D. Tucker and James H. Stronge Chapter 5. Assessing Teacher Quality Through Goal-Setting: The Alexandria, Virginia, School District

by Melissa McBride and Mason Miller

Goals determine what you are going to be.

—Julius Irving

In 2000, the Alexandria City Public School system implemented the Performance Evaluation Program (PEP), a comprehensive teacher evaluation system with four components: formal observations, informal observations, teacher portfolios, and academic goal-setting. The decision to design a new evaluation system drawn from multiple data sources was driven by the call for accountability within the Commonwealth of Virginia and by the desire to paint an ―authentic portrait‖ of the complex nature of teaching. PEP seeks to link teacher evaluation to student achievement via the academic goal-setting component, which requires teachers to set annual quantifiable goals related to their students' progress. Throughout the school year, goals are reviewed by PEP specialists and administrators. Similar to the Thompson School District evaluation model, the Alexandria City Public School district endeavors to answer the call for accountability via the connection of teacher evaluation and professional development with the goal of increased student learning, which is described in the next chapter. Since its inception in 2000, only 9 of the 18 schools have fully implemented all components of the program. Although full, district-wide implementation of the PEP will not occur until the start of the 2004–05 school year, all schools have been involved in the academic goal-setting component of PEP since the autumn of 2003.

26 | P a g e

A Brief Description of the Alexandria City School District Alexandria, Virginia, is a seaport city located within the greater metropolitan area of Washington, D.C. Regarded as a smaller school district within the Commonwealth of Virginia, the school system is comprised of 18 K–12 schools and serves approximately 11,000 students from diverse ethnic and socioeconomic backgrounds. Student demographics as of September 30, 2003 were as follows:

• Black

43.00 percent

• Hispanic

27.00 percent

• White

23.02 percent

• Asian/Pacific Islander

6.70 percent

• American Indian/Alaskan Native

0.30 percent

Eighty-eight countries of origin are represented and 69 different languages are spoken in Alexandria's classrooms.1 Roughly 25 percent of the total student population (approximately 2,625 students) has been identified as Limited English Proficient (ESL students). 2 Fifty-one percent (5,493 students) of Alexandria public school students are eligible for free and reducedprice lunch meals.3 Over 50 percent of the student body is considered ―at-risk‖ and require additional services. In addition, approximately 15 percent (1,641 students) are eligible for special services.4 In light of these major challenges to student learning, the Alexandria City Public School system dedicates 85 percent of its budget (approx. $123,094,863) to instruction and instructional support.5 Alexandria is a technology-rich school district with a student-to-computer ratio of 3:1,6 far exceeding the Commonwealth of Virginia's student-to-computer ratio of 6:1. The average teacher salary is greater than the state mean, $54,224 versus $41,731 respectively. 7 Average classroom size ranges between 20 and 23 students.8 Above all, Alexandria City Public School educators are passionate about their students' success.

27 | P a g e

What Are the Purposes of the Assessment System and How Was It Developed?

Influenced by the intensifying ―call for tangible evidence of student learning‖ 9 within the Commonwealth of Virginia and nationally, the Alexandria school board initially was interested in developing a merit pay system that integrated some measure of student achievement. However, the use of student achievement data in teacher appraisal systems remains controversial. 10 Many within the district feared that implementing a merit pay system would polarize the educational community. ―We did not want this to be an evaluation system that was an ‗I gotcha!‘ We wanted it to be a system that really promoted professional growth.‖11 Although teachers and administrators were somewhat uncomfortable with the idea of using measures of student learning in the evaluation process, they perceived this as a challenge they needed to embrace.

Designing the new evaluation program was a collaborative effort among several internal stakeholders within the Alexandria school district: teachers and principals ranging from the elementary to the secondary levels of education, and central office administrators who worked with James H. Stronge as a consultant for the development process. The new performance assessment process is based on the Goals and Roles Evaluation Model,12 a six-step approach to performance assessment. The development team reviewed, and in some instances adapted, evaluation materials from 11 public school divisions within the Commonwealth of Virginia and one school in Michigan. A complete listing of these school divisions is provided in Appendix E. Adhering to the advice of researchers in the field that student data be used ―as only one component of a teacher evaluation system that is based on multiple data sources,‖ 13 the architects of the Alexandria system strove to build one that was comprehensive and recognized the complexities of teaching. Five main data sources were chosen: formal observations, informal observations, portfolios, goal-setting, and student achievement. Definitions of each data source are presented in Figure 5.1. The designers felt that an ―authentic portrait of the teacher's work‖14 would be painted by these multiple data sources. It is the first two data sources, goal-setting and student achievement, on which this chapter is focused. The purpose of academic goalsetting is to 

Establish a positive correlation between the quality of teaching and learning,



Make instructional decisions based upon student data,



Create a mechanism for school improvement, and



Increase effectiveness of instruction via continuous professional growth.

Conversely, the purpose of academic goal-setting is not to 

Replace classroom observations or other means of documenting performance or



Be utilized as the sole measure of teacher effectiveness.

Additionally, it is important to understand that Alexandria's academic goal-setting process is not the creation of a teacher's personal or professional goals (e.g., ―I plan to improve instruction through . . . ,‖ or ―I plan to complete a master's degree.‖). Rather, the academic goal-setting process is explicitly focused on student academic progress: 1. Where are students in terms of academic progress at the beginning of the school year?

2. What am I planning to do to help this group of students succeed this year?

3. Where are the students at mid-year?

4. Where are students, in terms of academic progress, at the end of the school year?

28 | P a g e

5. How much progress did the students make?

Thus, in a very direct sense, the Alexandria City Public School Performance Evaluation Program incorporates a value-added approach to student learning that can be applied to teachers at various grade levels and in different subjects.

Figure 5.1. Definitions of Main Data Sources Data Source

Definition

Goal-Setting

Teachers have a definite impact on student learning and academic performance. Depending on grade level, content area, and ability level, appropriate measures of student performance are identified to provide information on the learning gains of students. Performance measures include standardized test results as well as other pertinent data.

Student Achievement

Teachers set goals for improving student achievement based on appropriate performance measures. The goals and the goal fulfillment are important data sources for evaluation.

Formal Observations

Observations are an important source of performance information.

Formal observations focus directly on 17 teacher performance responsibilities (see page 59). Classroom observations may also include review of teacher products or artifacts.

Informal Observations

Informal observations are intended to provide more frequent information on a wider variety of contributions made by the teacher.

Evaluators are encouraged to conduct informal observations by visiting classrooms, observing instruction, and observing work in nonclassroom settings at various times.

Portfolios

The portfolio includes artifacts that provide documentation for the 17 performance responsibilities.

Source: Reprinted with permission from Alexandria City Public Schools.

How Does the Assessment System Work?

As noted earlier, academic goal-setting is one of five components in the PEP. The other components are (1) student achievement, (2) formal observations, (3) informal observations, and (4) teacher portfolios. In recent years, substantial research has indicated that teacher effectiveness is the strongest school-based predictor of student achievement.15 To better understand the goal-setting component of PEP and how it relates to evaluating teacher effectiveness, it is imperative to outline the guiding principles of the Alexandria teacher evaluation system.

29 | P a g e

Adapting Stronge's Goals and Roles Evaluation Model,16 the Alexandria PEP examines teacher performance via a three-tiered approach (Figure 5.2). Five general domains, or categories, provide a conceptual framework: instruction, assessment, learning environment, communications and community relations, and professionalism. A table defining each of the teacher performance domains is provided in Appendix E. The following example is the definition of the Assessment domain: This domain includes the processes of gathering, reporting, and using a variety of data in a consistent manner to measure achievement, plan instruction, and improve student performance.17 A total of 17 performance responsibilities exist for teachers; a listing of these within their respective domains is provided in Appendix E. The following is an example of a performance responsibility within the Assessment domain: Performance Responsibility A-3: The teacher provides ongoing and timely feedback to encourage student progress.18 Performance indicators have been developed for each performance responsibility and are used to identify observable behaviors of the major job expectations. The lists of sample behaviors are not exhaustive, but they illustrate the typical actions that indicate satisfactory implementation of a performance responsibility. Examples of performance indicators for Performance Responsibility A-3 follow, in which the teacher  Gives performance feedback to students before, during, and after instruction, 

Collects sufficient assessment data to support accurate reports of student progress, and



Provides opportunities for students to assess their own progress and performance.

Figure 5.2. Three-Tiered Teacher Evaluation Approach

Source: Reprinted with permission from Alexandria City Public Schools.

30 | P a g e

Data are collected through observation, portfolio review, goal-setting, and student performance measures to provide the most comprehensive and accurate feedback on teacher performance.

Evaluators use two tools to complete teachers' summative evaluations: the performance indicators and the performance rubric. The performance rubric is based upon a behavioral summary scale. It guides evaluators in an effort to increase inter-rater reliability (the consistency of ratings by different supervisors). The rubric is a four-level continuum that ranges from ―exceeds expectations‖ to ―unsatisfactory.‖ What Are the Student Assessment Strategies?

Student performance measures are vital to the goal-setting process. Teachers can use gathered student information as evidence of fulfilling a specific responsibility. Teachers have a variety of measures for gauging student progress. To accommodate the wide variety of learners, all three of the following criteria are considered when selecting appropriate measures of learning: grade level, content area, and ability level of students. The focus is to select student assessment measures that are closely aligned with the curriculum. The following is a list of assessment strategies and examples of data sources to be used for the documentation of student learning: 

Norm-referenced tests (e.g., Stanford Diagnostic Reading Test [SDRT4])



Criterion-referenced tests (e.g., Phonemic Awareness Literacy Screening [PALS])



Authentic assessments (e.g., portfolios, projects, writing assessments)



In-house tests (e.g., district-wide quarterly tests, teacher-made tests)



Standards-based assessments (e.g., Virginia Standards of Learning [SOL])

How Is the Assessment System Related to Teacher Evaluation?

Virginia state law requires that the performance evaluation of instructional personnel include measures of student academic progress: School boards shall develop a procedure for use by division superintendents and principals in evaluating instructional personnel that is appropriate to the tasks performed and addresses, among other things, student academic progress and the skills and knowledge of instructional personnel, including, but not limited to, instructional methodology, classroom management, and subject matter knowledge. (§22.1–295)19 Although academic goal-setting is not mandatory, it is one reasonable method of satisfying the Commonwealth's requirement. In the goal-setting process, teachers must link their goals to one or more of the 17 teacher responsibilities. At the beginning of each school year, tenured and nontenured teachers collaborate with administrators and PEP specialists to develop at least one goal for improving student learning. In order to define annual goals that are SMART (specific, measurable, attainable, realistic, and timebound), teachers first do the following:  Collect and review student and teacher evaluation data. 

Analyze the data selected to determine student and professional needs.



Interpret the data looking for patterns or areas of weakness.



Determine the areas of need based upon these concrete data sources.



Select a focus for the goals.

Data that are collected and reviewed include student test results, previous teacher evaluations, and teacher portfolios. Teachers and PEP specialists work together to identify areas of student 31 | P a g e

performance and instruction that require improvement. Once patterns are identified, teachers select areas that they would like to improve for both themselves and their students. Again, this determination is based upon concrete data sources. The overarching purpose of these steps is to identify and define a baseline of performance for teachers and their students. The actual development of goals involves the following steps: 1. Define a clear objective. 

Use a specific assessment strategy or type of performance.



Set a measurable target (e.g., percent, number correct).

Select assessment strategies that are aligned with the goal. 

Collect data before and after instruction (if possible).



Use multiple measures of student learning to analyze and verify results.

End-of-the-Year Review 

Make adjustments where appropriate (e.g., instruction, groupings).

Annual goals are customized for each teacher and include specific information in order to accommodate the context in which teaching and learning occur, thereby enabling the evaluator to make a more appropriate assessment of the teacher's performance. Goals include the following information: 

Demographic information about the teacher (e.g., content area, grade level, school).



Baseline information about the students (e.g., pre-test scores, attendance records, standardized test scores, gifted, at-risk).



Goal statement describing desired results.



Strategies that have been selected to accomplish the goal.



Progress report at mid-year or at other appropriate intervals.



Summary of end-of-year accomplishments.20

The following examples are provided to assist the reader in visualizing the design of an annual goal. Figure 5.3 is one form teachers may use to document a goal. Teachers complete these forms in collaboration with the PEP specialist. Figure 5.4 is an example of an actual annual goal.

Figure 5.3. Sample Goal-Setting Form Alexandria City Public Schools Teacher Annual Goals for Improving Student Achievement Teacher______________________________

Evaluator_____________________

Grade/Subject ________________________

School Year_______________________

School_______________________________ 32 | P a g e

Setting[Describe the population and special learning circumstances.]

Content Area[The area/topic I will address (e.g., reading instruction, long division, problem solving).]

Baseline Data[Where I am now (e.g., status at beginning of year).]

Goal Statement[What I want to accomplish this year (i.e., my desired results).]

Strategies for Improvement[Activities I will use to accomplish my goal.]

________________________________

________________________________

Evaluator's Signature/Date

Teacher's Signature/Date

End-of-Year Data and Results[Accomplishments by year-end.]

Source: Reprinted with permission from Alexandria City Public Schools.

33 | P a g e

Figure 5.4. Example of Completed Goal-Setting Form Teacher Annual Goals for Improving Student Achievement TeacherBlaise Pascal

EvaluatorMrs. Humane

Grade/Subject9 Algebra I

School Year2002–2003

SchoolJames Madison H.S. Setting[Describe the population and special learning circumstances.]

James Madison High School is located in an urban setting and has an enrollment of 1,920 students in grades 9–12 with an average daily attendance of 91 percent and a Free/Reduced Lunch rate of 40 percent. In 2001–02, 37 percent of the students passed the end-of-course SOL Algebra I test (compared to 27 percent in 2000–01).

Content Area[The area/topic I will address (e.g., reading instruction, long division, problem solving).] Instruction — Algebra I Baseline Data[Where I am now (e.g., status at beginning of year).]

Test results in 2002–03 indicate that the total math average gain for my five classes is 10.54 compared to the division norm of 15.6, the problem-solving gain is 9.6 compared to the division norm of 17.4, and the procedures gain is 11.96 compared to 13.8.

Overall, my classes are near the division norm for procedures but are low in problem solving, which reduces the total math results.

Goal Statement[What I want to accomplish this year (i.e., my desired results).]

I will meet or exceed division norms for the total average math gain in my five classes using the Tests for Accountability. I will show an improvement of 4 points average gain or more in the problem-solving subscale scores on the same test.

Strategies for Improvement[Activities I will use to accomplish my goal.]

I will work with the mentor teacher and math department chair to infuse more problemsolving activities in my lesson plans, along with supportive instructional strategies such as cooperative work groups, use of manipulatives, and student explanations of the problems. I will ask the mentor teacher to work closely with me and offer demonstration lessons, team-teaching opportunities, and opportunities to visit other Algebra I classrooms. __________________________

__________________________

Evaluator's Signature/Date

Teacher's Signature/Date

End-of-Year Data and Results[Accomplishments by year-end.]

Source: Reprinted with permission from Alexandria City Public Schools.

34 | P a g e

Teachers are encouraged to organize and display their students' academic progress by using the following sources: 

Tables of raw student data by class and their assessment scores.



Tables of compiled data (e.g., percent of students at a certain benchmark, such as proficiency level).



Graphs of compiled data (e.g., pie charts, stacked graphs).



Simultaneous graphing of multiple measures (e.g., a mix of various standardized measures).

Staff Development The design of the Performance Evaluation Program emphasizes both formative and summative aspects of evaluation. In particular, the goal-setting component relies on continuous feedback and staff development that complements the teachers' annual goals. PEP specialists play an important role in this program, working with teachers to design goals based upon student data and assisting in the selection of appropriate instructional strategies to achieve these goals. PEP specialists also provide continuous support and lead staff training on various aspects of the goalsetting process throughout the year, which is necessary to ensure that student data are appropriately used and interpreted.

Safeguards The goal-setting process is just one component of the PEP; no adverse personnel decisions are based solely on the failure of teachers to achieve their annual goals. Teacher evaluation is no longer something that is ―done to them,‖ such as a 10-minute observation. Instead, the architects designed it to be a collaborative effort among teachers, evaluators, and PEP specialists.

Regarding the goal-setting process specifically, teachers are empowered to determine the selection of their own goals and student assessment measures. The intention is to provide professional development and support to improve the effectiveness of instruction. ―There has been personnel interaction[based on goal-setting], but not action.‖21 What Are the Advantages and Disadvantages of the Assessment System?

In determining the advantages and disadvantages of the student achievement academic goalsetting component of Alexandria's PEP program, interviews were conducted with central office administrators, instructional specialists, principals, and teachers, and the relevant information was incorporated.

Advantages The advantages cited focused primarily on the reflective and collaborative aspects of the goalsetting process: 

Encourages teacher reflection and data-driven decision making. ―It makes you reflect on your practice and how to come up with better ways to do things.‖ 22 ―Importance is placed on how the strategies come to life in the classroom context.‖ 23 ―Instead of just looking at scores, we look (now) at test questions. So, now my goals are related to more specific content areas that I want to improve.‖24



Fosters teacher collaboration and collegiality. ―We are discussing it a lot more amongst ourselves. Our PEP specialist had us go around and talk about our individual goals. It was really helpful to hear what somebody else was doing, and we could offer up 35 | P a g e

suggestions.‖25 ―I think the biggest thing that has changed my style of teaching is the people I am working with. I have learned so much from this one teacher in particular. We work together, brainstorm together, to come up with ways to accomplish our goals.‖26 

PEP specialists assist evaluators and serve as instructional leaders. ―Our PEP specialist has been really great in trying to explain the whole process, and making it more of our goal, looking at our data.‖27 ―She [the PEP specialist] did an excellent staff development on goal-setting this year and has conferenced often with lots of teachers to improve instruction.‖28



Process enables teachers to be active participants in their evaluation. ―Teachers can take ownership of what they write. They don't have to write a goal based on something that someone is mandating.‖29



Emphasizes formative as well as summative evaluation. ―We definitely get an opportunity to suggest through the entire PEP process that there are certain workshops that certain people should attend. We really get a huge opportunity to suggest appropriate staff development.‖30

Disadvantages Disadvantages focused on the time demands of academic goal-setting and implementation issues: 

Can be time-consuming. ―The biggest disadvantage is finding the time to have conversations about the actual work of making the goal happen and meeting with teachers to discuss strategies.‖31 ―The obstacles are time for administrators, time for teachers; there's never enough time in a day.‖32 ―If teachers are to engage in the tough work of instructional improvement, the school must organize for it.‖ 33 Goal-setting encourages reflective practice, but time needs to be allocated for it to take place.



Student data may be misused or misinterpreted. ―They have us using different children's data to set goals. At the beginning of the year we looked at last year's group's SOL scores. It's helpful in that we can see what we taught well and what we need to improve on, but I think that another piece is that we have to look at data from the group of kids coming to us because they might be weak in other areas.‖34 It is critical that schools develop data management systems for making assessment results readily available for teachers to use, both for analyzing student learning patterns from the previous year and identifying the learning needs of incoming students. Even more optimal would be the development of benchmarking tests that some school systems are now beginning to use.35



Evaluating teachers based on student academic progress can be threatening and increase stress. ―I think part of the fear is that goal-setting and teacher evaluations are going to be linked just to standardized tests. If they did that, then I would not agree with it.

I wouldn't want to see standardized test scores be the only measure of my competence as a teacher.‖36 Teachers need to be supported in the process of developing goals and have a sense of trust in the constructive purposes of goal-setting.



Effectiveness is contingent upon well-trained, accessible PEP specialists. ―You have to have PEP specialists who are qualified, get involved, and know what they are doing in order to do an effective job.‖37 ―You have to have a PEP specialist who is capable of making sure that teachers understand the importance of the process to their overall teaching assignment.‖38 36 | P a g e

What Are the Results of Implementation?

Because academic goal-setting is a work-in-progress for the Alexandria City Public Schools, only preliminary results for this initiative are available at present. At this point, we do know there is a substantial research base for this approach, with its heavy emphasis on identifying the instructional needs of students and focusing teacher effort on these areas.

What Research Supports a Process Such as Academic Goal-Setting?

Academic goal-setting is closely linked to mastery-learning practices (feedback-corrective teaching), which entails 

Giving students formative tests for the purposes of feedback,



Providing corrective instructional procedures, and



Administering additional formative tests to determine the extent to which students have mastered the subject content.

In fact, solid evidence indicates that formative assessment is an essential component to classroom work that can raise student achievement.39 Researchers such as Benjamin Bloom have found that students taught under mastery learning achieve, on average, approximately a 1.0 standard deviation above the average of students in conventionally taught classrooms (e.g., 84th percentile vs. 50th percentile). 40 Academic goal-setting also is linked to enhancing the students' initial cognitive entry prerequisites, which entails 

Developing an initial skills assessment of prerequisites for a course,



Administering the assessment to students at the beginning of a course, and



Teaching students specific prerequisites they lack.

Research indicates that, on average, students that are taught the entry prerequisite skills achieve approximately a .7 standard deviation above the average of students in conventionally-taught classrooms (e.g., 76th percentile vs. 50th percentile).41 Marzano, Pickering, and Pollock, in their research into research-based strategies for increasing student achievement, reported studies showing percentile gains in student achievement ranging from 18 to 41.42 Additionally, they drew the following three conclusions from the research on goal-setting: 1. Instructional goals narrow what students focus on. Therefore, while students generally score higher on the instruction related to the specific academic goal, they likely would score lower (about 8 percentage points) on information that is incidental to the goal, but still covered in the class.

2. Instructional goals should not be too specific. In other words, instructional goals stated in behavioral objective format do not produce student learning gains as high as instructional goals stated in more general formats.

3. Students should be encouraged to personalize the teacher's goals. Once classroom academic goals are set, students should be encouraged to customize them to fit their personal needs.43

37 | P a g e

Early Perceptions About Academic Goal-Setting Although it is not possible to present tangible results of the goal-setting process, interviews with internal stakeholders within the Alexandria public school community provided feedback on the perceptions among teachers and administrators (including central office administrators, principals, and program specialists) of academic goal-setting as a component of the teacher evaluation process.

One observation multiple people supported was the pivotal role of the PEP specialists. Simply put, PEP specialists were considered the keystone of the goal-setting process. They were the decisive element in determining whether or not a teacher or administrator felt this program added value to professional development and student learning. Extensive training is necessary to enhance the effectiveness of the specialists because they are responsible for a wide variety of tasks, including staff development regarding instructional strategies, training of teachers in how to appropriately use and interpret student data, and providing continuous support to teachers.

The specialists also need to be readily available to assist teachers. Ideally, a PEP specialist should be housed at each school to enhance the effectiveness of the goal-setting process.

Administrator Perceptions Administrators note that the goal-setting process is helpful and enables them to identify where teachers require instructional assistance, but acknowledge that the process can increase stress and workloads for teachers. They perceive goal-setting to be an important complement to the other components of the evaluation system that include observation and teacher portfolios. They also view the process as a fair one that places responsibility for success upon the teachers' shoulders. Overall, administrators believe that the goal-setting component of the evaluation process has a significant impact upon teacher instruction and student academic progress.

Teacher Perceptions Like the administrators, teachers report that the goal-setting process does help them focus on their students' instructional needs more clearly and adjust instruction accordingly. They also note that the process can increase their stress levels and workloads. Teachers view the goal-setting process as fair as long as the focus remains on professional development and student academic growth. And again like the administrators, teachers believe goal-setting is an important component of the evaluation process, but should be balanced by other elements in the system.

Conclusion Although still in its infancy, it is apparent that the Alexandria City Public School system's goalsetting process has the potential to transform how teachers plan and deliver instruction. The assistant superintendent reports that the school system is seeing ―a paradigm shift in how teachers and evaluators think about evaluation.‖44 We believe the Alexandria public school goalsetting process provides a reasonable way to connect student academic performance and the teacher. Academic goal-setting is linked to mastery learning practices and initial cognitive entry prerequisites, which have been shown to increase student achievement. While goal-setting is only one facet of the comprehensive teacher Performance Evaluation Program, the focus of the overall system is improving the quality of instruction. As reported by Alexandria public school teachers and administrators, the goal-setting process fosters teacher reflection and collegiality, and encourages a collaborative approach to teacher evaluation. Finally, the process encourages teachers to focus on their students' learning needs and make data-driven decisions based upon student data. As one administrator eloquently stated, ―The goal really gives us something to 38 | P a g e

shoot for. If we don't get there . . . well, it's kind of like shooting for the stars and landing on the moon. We are moving in a much more positive direction.‖45 Endnotes 1 Alexandria City Public Schools. (2003a). Fast facts: Alexandria City Public Schools at a glance.

Retrieved February 15, 2004, from http://www.acps.k12.va.us/fastfact.php 2 Alexandria City Public Schools. (2003b). Proposed operating budget FY 2005: Special needs enrollment. Retrieved February 15, 2004. http://www.acps.k12.va.us/budgets/op2005_b.pdf 3 Alexandria City Public Schools. (2003c). ACPS food and nutritional services. Retrieved February 15, 2004, from http://www.acps.k12.va.us/fns/stats.pdf 4 Alexandria City Public Schools, 2003a.

5 Alexandria City Public Schools. (2003d). About ACPS. Retrieved February 15, 2004, fromhttp://www.acps.k12.va.us/promo.php 6 Alexandria City Public Schools, 2003a.

7 Alexandria City Public Schools. (2003a); Virginia Department of Education. (2003). Summary FY 2003: Increases in classroom teacher salaries. Retrieved February 25, 2004, fromwww.pen.k12.va.us/VDOE/Finance/Budget/2002-2003SalarySurveyFinalRptforweb.pdf 8 Alexandria City Public Schools, 2003a.

9 Wilkerson, D., Manatt, R., Rogers, M., & Maughan, R. (2000). Validation of student, principal, and self-ratings in 360-degree feedback for teacher evaluation. Journal of Personnel Evaluation in Education, 14(2), 179–192.

10 Wright, S., Horn, S., & Sanders, W. (1997). Teacher and classroom context effects on student achievement: Implications for teacher evaluation. Journal of Personnel Evaluation in Education, 1(11), 57–67.

11 Administrator A, personal communication, October 30, 2003.

12 Stronge, J. H. (1997). Improving schools through teacher evaluation. In J. H. Stronge (Ed.),Evaluating teaching: A guide to current thinking and best practice (pp. 1–23). Thousand Oaks, CA: Corwin Press.

13 Stronge, J., & Tucker, P. (2000). Teacher evaluation and student achievement. Washington, DC: National Education Association, p.53.

14 Alexandria City Public Schools. (2000a). Teacher evaluation technical manual. Alexandria, VA: Author, p. 8.

15 Wright et al., 1997.

16 Stronge, 1997.

17 Alexandria City Public Schools, 2000a, p. 28.

18 Alexandria City Public Schools, 2000a, p. 31.

19 Virginia State Department of Education. (2000). Virginia school laws. Charlottesville, VA: The Michie Company.

20 Alexandria City Public Schools. (2000b). Academic goal-setting. Alexandria, VA: Author, p. 53.

21 Administrator E, personal communication, December 3, 2003.

22 Teacher B, personal communication, November 25, 2003.

23 Administrator B, personal communication, October 30, 2003.

39 | P a g e

24

Teacher A, personal communication, November 25, 2003.

Teacher E, personal communication, December 3, 2003.

26 Teacher B, personal communication, November 25, 2003.

27 Teacher F, personal communication, December 3, 2003.

28 Teacher E, personal communication, December 3, 2003.

29 Administrator C, personal communication, October 30, 2003.

30 Administrator E, personal communication, December 3, 2003.

31 Administrator C, personal communication, October 30, 2003.

32 Teacher F, personal communication, December 3, 2003.

33 Little, J., Gearhart, M., Curry, M., & Kafka, J. (2003). Looking at student work for teacher learning, teacher community, and school reform. Phi Delta Kappan, 85(3), 185–192.

34 Teacher B, personal communication, November 25, 2003.

35 Carey, K. (2004). The real value of teachers: Using new information about teacher effectiveness to close the achievement gap. Thinking K–16, 8(1), p. 6.

36 Teacher B, personal communication, November 25, 2003.

37 Teacher E, personal communication, December 3, 2003.

38 Administrator E, Personal communication, December 3, 2003.

39 Black, P. & Wiliam, D. (1998). Inside the black box: Raising standards through classroom assessment. Phi Delta Kappan, 80, 139–148.

40 Bloom, B. S. (1984). The search for methods of group instruction as effective as one-to-one tutoring. Educational Leadership, 41(8), 4–17.

41 Walberg, H. J. (1984). Improving the productivity of America's schools. Educational Leadership, 41(8), 19–27.

42 Marzano, R. J., Pickering, D. J., & Pollock, J. E. (2001). Classroom instruction that works: Research-based strategies for increasing student achievement. Alexandria, VA: Association for Supervision and Curriculum Development.

43 Marzano, Pickering, & Pollock, 2001, pp. 94–95.

44 Lois Berlin, personal communication, March 10, 2004.

45 Administrator E, personal communication, December 3, 2003.

25

40 | P a g e

Evidence Binder Guide:

Guide to Evidence Binders Purpose An evidence binder is required for the full evaluation process. The purpose of including an evidence binder in the evaluation process is to provide the evaluator with a more complete and accurate picture of your teaching standards.

An evaluator, no matter how insightful and discerning, cannot make a valid judgment of teaching from a few observations. It is possible that the observed lessons do not touch on many of the chosen subcategories or standards. The main purpose of the evidence binder is to demonstrate the standards not evident during classroom observations. The key factor is using the evidence binder as a chance to reflect upon strengths and weaknesses and spur constructive dialogue in conferences.

Form, content, and organization The evidence binder should display the teacher‘s best work that demonstrates the CSTPs. Examples should portray classroom activities and student learning.

The evidence binders should be well organized. There is no one standard format. Note also that BTSA teachers may use their BTSA portfolio for this purpose. The JC makes the following recommendations and clarifications for the development of the evidence binder: A. Begin the evidence binder with a cover page with name, year, school, evaluator, and standards that are being evaluated.

B. Each subcategory should be separated in the evidence binder. Evidence for the subcategories should be included in the appropriate subcategory section.

C. Each piece of evidence should be labeled to indicate subcategory. For example, an assignment or student work that follows standard 1.1 (builds on student prior knowledge, life experience, and interests to achieve learning goals), should have 1.1 in the top right corner of the page. Some examples will cover more than one subcategory, in which case record each subcategory to which it applies. In some instances, explanatory annotations for a piece of evidence will help the evaluator skim the page and still understanding the reason for its inclusion. This is especially important if the item involves role playing, parody, satire, or what may be open to misinterpretation.

41 | P a g e

D. ―Evidence‖ is defined as a physical document or recording that demonstrates a performance level of 1, 2, or 3 on the rubric scale. This may include but is not limited to photocopies of student work, a case study of a particular student‘s progress, copies of e-mail, log of phone calls, lesson plans, teacher-developed curriculum materials, or student surveys.

E. If an administrator and teacher agree that a subcategory has been met during observations, additional evidence of that subcategory is not necessary in the evidence binder. However, if the teacher is not satisfied with the rubric score, he or she may include additional evidence in the evidence binder.

F. If meeting the goals of a subcategory is demonstrated by evidence binder evidence, then at least one example of evidence and no more than three is needed for the evidence binder. Student work included as evidence should be from the evaluation year but may also include work from the previous spring semester. Professional work included as evidence may be from the last two calendar years.

G. At the interim conference, to be completed by November 30, the progress of the evidence binder should be reviewed. If the evidence binder is completed at the final conference, teachers will share their completed evidence binder with the evaluator. If the evidence binder is not completed by the final conference, a brief written summary (1-3 sentences) explaining how the evidence binder demonstrates meeting the rubric scale should be included for each subcategory. Teachers and administrator have both expressed a more satisfactory experience when the evidence binder is completed for the final conference. Non-permanent teachers must turn in their evidence binders no later than March 15, and permanent teachers must turn in their evidence binders no later than April 1.

Teachers may ask evaluators for the names of colleagues who had model evidence binders the previous year. Note that BTSA teaches may use their BTSA portfolio for this purpose.

Evidence Binder Development Guidelines To guide evidence binder development, the following clarifications are offered.

There are many things teachers can include in evidence binders. Below is a possible list of suggestions: � student assignments � student assessments � lesson plans � student or parent comments, communication, or evaluations � journal entries and reflective paragraphs � comments and observations by colleagues � evidence of collaboration 42 | P a g e

� transcripts or resumes � honors or awards � letters of recommendation � workshops or conferences attended or given � personal statement of teaching philosophy � class syllabus � unit calendars � evidence of field trips, guest speakers Examples of student work are particularly effective. Possible criteria include examples that: � reflect alignment with state content standards and ESLRs � demonstrate higher order thinking � accommodate different learning styles and backgrounds � show evidence of cooperative learning � show student improvement over time � reflect creative or innovative projects � correspond to lessons, assignments, or assessments you have included � photos and/or videos can be used to convey teaching successes.

Possible options: � bulletin boards reflecting the classroom environment � oversized work that won‘t fit in the evidence binder � presentations or projects � field trips/speakers � lesson excerpts

43 | P a g e

Individual Professional Growth Goals Teacher Goal Setting Form Teacher’s Name Grade/Subject Taught:

EIN: Date:

Supervising Administrator: Goal-setting process • Use the Framework for Teaching Descriptors of Practice/Self Assessment form to complete a self-assessment of your teaching practice.

• Review the data and feedback provided from previous observations.

• Identify areas of practice for professional growth within the eight components you and your administrator have agreed to as areas of focus. It is recommended that teachers select two goals to address in a school year.

• Complete this form to indicate the domain(s) and component(s) selected for growth, and provide information describing how you will improve in the selected areas, and how you will measure / document growth.

• Attach this form, when completed, to the Framework for Teaching Descriptors of Practice/Self Assessment form.

• Use this form to refine goals and to develop professional growth plans.

44 | P a g e

Goal: During the initial year of participation, all teachers will learn The Framework for Teaching components and become familiar with the Evaluation system as their professional growth goal.

Domain: Component(s) Element(s) Rationale for selecting this goal:

Steps to achieve the goal:

Method(s)/procedure(s) for gathering

evidence: How will you measure progress?

Signature of Teacher:

Date:

Signature of Administrator:

Date:

45 | P a g e

Goal Setting Guidelines Objective The objective of the formative process is to improve professional practice. To achieve this objective, learners take ownership of the learning goals, established by thoughtful self-assessment, personal reflection on teaching practices, and specific feedback based upon standards of teaching practice.

Overview Teachers and administrators will identify and agree upon professional growth goals which align with the Framework for Teaching (FFT) Domains of Professional Practice. Teachers will self assess their practice utilizing the FFT Domains and review feedback and data received from previous observations. Goals will then be developed related to areas of growth they have identified from the self-assessment and the observation feedback.

Domain 1: Planning and Preparation Domain 3: Instruction Domain 2: The Classroom Environment Domain 4: Professional Responsibilities Procedures 1. Self assessment and goal setting: Teachers will self assess their practice using the FFT and evidence from the first cycle of evaluations from the prior year.

From this information, teachers will identify professional growth goals and develop a professional growth plan to achieve those goals in the current year.

2. Agreement of goals and professional growth activities: The teacher and supervising administrator will discuss options for professional growth related to the selected Components.

3. Developing a professional growth plan: Once approved, teachers will develop an individual Growth Plan (IGP) by completing the form Individual Professional Growth Goals: Teacher Goal Setting Form. This form is available electronically at: https://sites.google.com/site/pgcpsteachereffectiveness/home 4. Completing and reflecting upon goals: Teachers will collect and present artifacts which document growth in the selected areas. Observation evidence gathered by the administrator can also be used to determine the degree to which the teacher attained the goals.

5. Reviewing progress toward meeting the goals: The professional growth plan and progress made to attain the goals will be reviewed during the end of the year conference.

46 | P a g e

Professional Responsibilities Reflection Teacher

Evaluator

Date & Time of Summative Conference Grade(s)

Subject Area(s)

The teacher may choose to complete this form, in whole or in part, and use it to complete the Professional Responsibilities form. The teacher may also bring this form to any evaluation conference. The teacher has full discretion as to whether this form is completed and/or shared with the evaluator.

Reflect and comment on the different aspects of your professional practice. To what extent are you effective? What are your professional practice goals? How will your goals support student learning? What opportunities or supports do you need to achieve your professional practice goals?

Are the following aspects of your professional practice effective? Why or why not?

What are your professional practice goals? How will your goals support student learning?

What opportunities or supports do you need to achieve your professional practice goals?

Family Communicatio ns/Engagemen t Record Keeping Professional Growth and Development Reflection on Professional Practice Other Reflections

47 | P a g e

Professional Responsibilities Form:

Teacher School

Grade(s)

Evaluator Date of Conference Subject Area(s)

Teachers must complete this form for the current school year and submit it to the evaluator no later than their first Pre-Observation Conference. Teachers on a twoyear evaluation cycle must submit the form during the first year of the cycle. This form may be updated at any time during the evaluation cycle.

Part I –Communicating with Family: How do you share instructional program information with families? How do you engage families in the instructional program?

In what ways do you ensure that families and appropriate staff are informed of student progress or needs?

Part II – Recording Data: How do you ensure student data is recorded accurately and kept up to date?

48 | P a g e

Part III – Growing and Developing Professionally: Describe any professional learning activities in which you regularly participate. How do these activities enhance your professional practice?

Part IV – Additional Reflections: Describe any extra-curricular activities in which you participate.. How do these activities enhance your professional practice and/or engagement in the school community?

Part V – Professional Growth Goals: Describe any professional growth goals you may have related to Parts I, II, and III in this form. Include information about any additional professional learning activities you would like to participate in this year. Also include any resources you may need to achieve your goal(s).

List any other information you would like to share with your evaluator.

49 | P a g e

MULTIPLE MEASURE #6 SURVEYS Used to gather stakeholder opinions or judgments about teaching practice as part of teacher evaluation and to provide information about teaching as it is perceived by others.

NYS ALIGNMENT: I, II, VI, VII STRENGTHS:  

Provides the perspective of others, who have the greatest amount of experience with teachers.

Can provide formative information to help teachers improve practice in a way that will connect with students.

WEAKNESSES:  

Ratings have not been validated for use in summative assessment and should not be used as the sole or primary measure of teacher evaluation.

Cannot provide information on aspects of teaching such as a teacher‘s content knowledge, curriculum fulfillment, or professional activities.

EXAMPLES:    

Student Surveys Parent Surveys Peer Review Self-Assessment

50 | P a g e