2011 Interpretive guide of Alabama Accountability System

2011 INTERPRETIVE GUIDE Alabama Accountability System

Alabama Department of Education Joseph B. Morton State Superintendent of Education

July 2011

Table of Contents Page Introduction ............................................................................................................................. 3 Adequate Yearly Progress (AYP) .......................................................................................... 4 Assessments Included in Accountability ............................................................................... 8 Academic Achievement Levels ............................................................................................... 9 Annual Measurable Objectives (AMO) ............................................................................... 10 Student Groups for Disaggregation ..................................................................................... 11 Group Size .............................................................................................................................. 12 Proficiency Index ................................................................................................................... 13 Confidence Interval ............................................................................................................... 16 Uniform Averaging ................................................................................................................ 18 Safe Harbor ............................................................................................................................ 21 Relationship between AYP and School Improvement ...................................................... 22 Glossary .................................................................................................................................. 24 Sample Status Reports .......................................................................................................... 26

2

2011 INTERPRETIVE GUIDE Alabama Accountability System Introduction In August 2004, Alabama reported the results of its new accountability system. Many factors, including both state and federal laws, were influential in the development of this new accountability system. As Alabama worked to comply with these laws, parents, teachers, school officials, and policymakers also worked to ensure that the new accountability system was consistent with the state’s educational needs. In order to understand how this all came together, one must look at accountability in Alabama over the past several years.

A state law was passed in 1995 requiring the administration of a nationally normed achievement test in Grades 3-11, and requiring the results be used for identifying schools in need of improvement. As a result, beginning in 1996 schools and school systems were identified as Alert, Caution, and Clear based on the results of scores from the Stanford Achievement Test. After much effort by educators and others, another state law was passed in 2000 that gave the State Board of Education the authority to determine the assessment and accountability programs for Alabama. Based on this new authority, the State Superintendent of Education appointed the Test Advisory Committee to make recommendations for assessment and accountability programs. This committee took their charge seriously and began work developing recommendations for assessment and accountability programs.

As the Test Advisory Committee worked, the No Child Left Behind Act of 2001 (NCLB) was being drafted. Many of the provisions that the Test Advisory Committee considered were consistent with those being considered by drafters of the new federal law. NCLB required criterion-referenced achievement tests to be administered in Grades 3-8 and at least once at the high school level and that these assessments be used for determining adequate yearly progress (AYP) for schools.

In consideration of these requirements, the State Board of Education passed a resolution on July 9, 2002, outlining a long-range assessment plan and principles to be used in the development of an assessment program and an accountability system. In June 2003, the State Board of Education adopted the recommendations of the Accountability Advisory Committee for implementation beginning 2003-2004.

The accountability program that resulted was intended to be a single state accountability system in compliance with NCLB and the Code of Alabama (1975), Chapter 16-6B-1-3. This required Alabama to put into place an interim accountability plan for school years 2001-2002 and 2002-2003 while the new assessments and the accountability system were being developed for implementation in 2003-2004.

The changes were significant and complex. To help educators, concerned citizens, public officials, and the media better understand these complex changes, this Interpretive Guide has been developed. It is designed to introduce the key components of the resulting accountability system. As the accountability program has been implemented, issues have emerged that required decisions and actions that were not anticipated. It is intended that this Interpretive Guide will explain the accountability program based on assessments administered in 2010-2011 and provide guidance through 2013-2014.

3

Adequate Yearly Progress (AYP) NCLB uses the term AYP to describe whether a school or school system has met its annual accountability goals. AYP is “what the school or school system did this year.” Three factors are considered in determining AYP. These factors are participation rate, annual measurable objectives in reading and mathematics, and additional academic indicators.

Participation Rate To meet AYP, a school, school system, and each group (students in the aggregate and each subgroup) must have at least a 95% participation rate on assessments. This participation rate is calculated separately for reading and mathematics.

Participation rates are calculated as the number of students who participate in a state assessment (either the Alabama Reading and Mathematics Test [ARMT], the Alabama High School Graduation Exam [AHSGE], or the Alabama Alternate Assessment [AAA]) divided by the number of students enrolled on the first day of the state testing window. Students that previously passed the AHSGE are counted as participating regardless of when the student passed. For limited-English proficient students for who 2009-2010 was their first year in an U.S. school, participation in the ACCESS for English Language Learners (ACCESS) will be used for reading participation.

If the school, system, or a group does not have at least a 95% participation rate, the following method is applied: Uniform Averaging – If the participation rate for the current year does not meet the 95% goal, the participation rate for the most recent three years will be averaged using a weighted average. If this average meets the goal, the school, system, or group will be considered to have met the participation rate. (See pages 18–19 for additional information.)

Please note that a student must have a valid reading test score to be included for participation in reading and a valid mathematics test score to be included for participation in mathematics.

Annual Measurable Objectives for Reading and Mathematics (AMO) It is the goal of NCLB that all public school students perform at the “proficient” level in reading and mathematics by the end of the 2013-2014 school year. In Alabama “proficient” is defined as Level III – Meets Academic Content Standards. The assessment results from spring 2004 test administrations were used to determine the starting points or baselines for Grades 4, 6, 8, and 11 reading and for Grades 4, 6, and 11 mathematics. The assessment results from spring 2005 test administrations were used to determine the starting points or baselines for Grades 3, 5, and 7 reading and Grades 3, 5, 7, and 8 mathematics. With the baselines established, yearly target goals or annual measurable objectives were established, according to NCLB requirements, which will be used to determine whether schools and school systems are making adequate yearly progress toward meeting the 2013-2014 requirement of 100% proficient.

The scores of all student groups (students in the aggregate and each group of students) must meet or exceed the state’s annual measurable objectives for reading and mathematics. (See page 11 for explanation of student groups used for this disaggregation.) Annual measurable objectives are established separately for reading and mathematics for each grade included in accountability calculations.

4

Annual measurable objectives are expressed as the percent of students scoring Level III and above. (See Annual Measurable Objectives on page 10.) For determining AYP, scores are combined across grades using a proficiency index. (See Proficiency Index, pages 13–15, for a more detailed explanation.)

Although all students in the school must be tested and their assessment results reported, schools are only held accountable for scores of those students enrolled for a “full academic year.” All students enrolled in schools or school systems for a “full academic year” are included in the calculations for determining if the proficiency goal has been met. A “full academic year” is defined as students enrolled as of September 1 and continue to be enrolled as of the first day of the state testing window without a break in enrollment. (See Student Groups for Disaggregation, page 11, for exception for limited English proficient students.)

If the school, system, or a group does not make proficiency, the following methods are applied: Confidence Interval – The confidence interval provides a test for whether or not a proficiency index is statistically different from the goal of zero at the .01 level. If a school, school system, or group’s proficiency index falls below zero, but is within the confidence interval, the school, school system, or group is considered to have met its proficiency goal. (See pages 16–17 for additional information.)

Uniform Averaging – The proficiency indexes for the most recent three years will be averaged. If this average is zero or higher, the school, system, or group will be considered to have met its proficiency goal. (See pages 18 and 20 for additional information.)

Safe Harbor – A school, school system, or group is considered to have met its proficiency goal via the safe harbor provision if it meets the following criteria:   

The group decreases those who are not proficient by at least 10% from the preceding year The group meets the 95% participation rate, and The group meets the goal or makes the required improvement on the additional academic indicator. (See page 21 for additional information.)

Additional Academic Indicators In addition to the annual measurable objectives in reading and mathematics, NCLB requires at least one additional academic indicator (AAI) be used to determine if schools and school systems have made AYP. Students in the aggregate, the All Students group, must meet the state’s requirements for the additional academic indicators. However, if safe harbor is invoked for a disaggregated group, the group also must meet the additional academic indicator. Alabama uses two different additional academic indicators.

Alabama uses attendance rate as the additional academic indicator in elementary schools, middle schools, and any schools without a Grade 12. A school or school system is considered to have made its attendance goal if it meets the goal of 95% attendance rate or if it improves the attendance rate from the previous year.

For high schools or schools with a Grade 12, the additional academic indicator is stipulated by NCLB to be the graduation rate. Currently, the graduation rate is calculated using the NCES Leaver Rate formula and is reported one (1) year in arrears. This will be the last year Alabama uses this formula for calculating the AYP graduation rate. Beginning next year, Alabama will be using the 4-year adjusted cohort rate formula for calculating the graduation rate and will be reported one (1) year in arrears.

Schools and school systems may meet the graduation rate goal in one of the following ways: 5

 Current year graduation rate is greater than or equal to the goal of 90% (same as previously used for AYP).  Average graduation rate for the most recent three years is greater than or equal to the goal of 90%.  Current year graduation rate is greater than or equal to the previous year graduation rate plus 10% of the difference between the goal of 90% and the previous year’s graduation rate.  Average graduation rate for the most recent three years is greater than or equal to the average graduation rate for the previous three years plus 10% of the difference between the goal of 90% and the average of the previous three years.

When reporting AYP for a school, the matrix below is completed for all grades combined.

Matrix for AYP Determinations 2011-2012 School Year (Based on School Year 2010-2011 Data) Component 

Group 

Reading

Met Participation Goal

Met Proficiency Goal

Mathematics

Met Participation Goal

Met Proficiency Goal

Additional Academic Indicator* Met Graduation Goal/ Attendance Goal

All Students Special Education American Indian/ Alaskan Native Asian/Pacific Islander Black Hispanic White Limited English Proficient Free/Reduced Meals * Schools with students in Grade 12 are required to meet the graduation rate as the additional academic indicator.

Additionally, schools and school systems are given an AYP status for the reading component, the mathematics component, and the additional academic indicator component. To make AYP in reading, a school/school system must have at least 95% participation in all applicable groups (the All Students group and all applicable disaggregated student groups) and meet the annual measurable objectives in reading in all applicable groups. To make AYP in mathematics, a school/school system must have at least 95% participation in all applicable groups (the All Students group and all applicable disaggregated student groups) and meet the annual measurable objectives in mathematics in all applicable groups. To make AYP in the additional academic indicator(s), a school/school system must meet the goal(s) or make the required 6

progress toward the goal(s) from the previous year. If a school meets all applicable goals for a component, it will make AYP for that component. A school or school system may make AYP in one component and not make AYP in another component. A school or school system must make AYP in all three components (reading, mathematics, and additional academic indicators) in order to make overall AYP. A school is identified as not making AYP in a component when it does not meet all applicable goals for the component (reading, mathematics, or additional academic indicators).

For a school system, the status report below is completed separately for each grade span (3-5 Grade Span, 6-8 Grade Span, and High School Span). If the span fails to meet the goal in any applicable cell, they will not have made AYP for that span.

A school system that meets all applicable goals for a component (reading, mathematics, and AAI) for at least one grade span will make AYP for that component. (e.g., If a school system makes AYP in reading at the 3-5 Grade Span, the school system makes AYP for the reading component. It does not matter what happens in the 6-8 Grade Span or the High School Span.) A school system is identified as not making AYP in a component when it does not make AYP in all three grade spans (3-5 Grade Span, 6-8 Grade Span, and High School Span).

Two colors are used to display the status of a school or system. Green indicates that the school or system made the goals for participation and proficiency in reading or mathematics, or the AAI; red indicates that the school or system did not. Both should be considered when evaluating need for improvement. A closer look at a green cell may indicate that while a group may have made AYP in that component, it may require intervention in order to maintain AYP in that component.

Example of System Status Summary Report

Reading AYP Mathematics AYP AAI AYP Overall AYP

3-5 Grade Span No No Yes

6-8 Grade Span Yes No Yes

High School Span No No No

System AYP Yes No Yes No

It should be noted that the term “failing school” is not used in the state accountability plan and is not used in federal law or regulations. If a school or school system does not make AYP, a description should be attached to the phrase “did not make AYP” to explain the basis on which it was determined that the school/ school system did not make AYP. For example, “the school did not make AYP in reading proficiency among special education students (one cell) and in reading participation rate among limited English proficient students (one cell)” provides much more information than saying that the school is a failing school.

All schools are given an academic status. Schools without students in Grades 3-8 and/or 11 will receive the academic status of the school within the school system to which the majority or largest number of students will attend when they leave. Schools that are newly reconfigured or newly formed will, for their first year of existence, receive the academic status of the school which contributed the majority or largest number of students.

7

Assessments Included in Accountability The assessments used to determine AYP for NCLB for the 2010-2011 school year were the ARMT, the reading and mathematics subject-area tests of the AHSGE, the reading and mathematics subject-area tests of the AAA, and ACCESS. All assessments used for accountability are administered in English.

The ARMT consists of selected test items from the Stanford Achievement Test, Tenth Edition (Stanford 10) that match the Alabama courses of study in reading and mathematics plus additional test items that, when added to the selected Stanford 10 items, give complete coverage of the content contained in those courses of study. Students in Grades 3-8 take the ARMT during the spring of the year.

Students in Grade 11 take the AHSGE, the exam required for students to pass in order to get a high school diploma. The AHSGE also is based on the Alabama courses of study in reading and mathematics.

The AAA is the assessment administered to special education students whose Individualized Education Program (IEP) teams determine they cannot participate in general state assessments, with or without accommodations. The AAA is based on a student’s mastery of the Alabama Extended Standards in reading and mathematics.

ACCESS is the assessment administered to limited-English proficient (LEP) students. For LEP students who are in their first twelve months of enrollment in a U.S. school, ACCESS will count as their participation for reading. These students’ test scores will not be included in the reading and mathematics proficiency calculations.

8

Academic Achievement Levels Academic achievement levels define how well students are mastering the state’s academic content standards at each grade level. The State Board of Education adopted academic achievement standards in a resolution dated July 9, 2002. The results of the ARMT, AHSGE, and AAA are reported in four academic achievement levels: Level IV – Level III – Level II – Level I –

Exceeds Academic Content Standards Meets Academic Content Standards Partially Meets Academic Content Standards Does Not Meet Academic Content Standards.

Level III is considered proficient. Therefore, a student scoring a Level III or Level IV meets the proficiency standard. A student scoring a Level I or Level II does not meet the proficiency standard.

The same reporting system is applied to all criterion-referenced assessments in Alabama.

9

Annual Measurable Objectives (AMO) In accordance with NCLB, the results from the ARMT and the AHSGE (cumulative passing rates for Grade 11 students) were used to determine baselines or starting points. These starting points were established using criteria set forth in NCLB. These baselines are applied to schools, school systems, and groups and are calculated separately by grade and subject.

Once the baselines were established, intermediate goals were established with equal increases in the required percent proficient. AMOs, the annual requirements for percentage of students at proficiency, will be the same as the most recent intermediate goals. The annual measurable objectives are listed below. Note that the annual measurable objectives for two consecutive years may be the same. The ultimate goal for NCLB is that all students reach Level III (proficient) or higher by 2013-2014.

2004 Grade 3 Grade 4 Grade 5 Grade 6 Grade 7 Grade 8 Grade 11

68 74 43 81

2004 Grade 3 Grade 4 Grade 5 Grade 6 Grade 7 Grade 8 Grade 11

61 39

68

2005 73 68 73 74 63 43 81

2005 63 61 59 39 40 48 68

Reading Annual Measurable Objectives Percentage of Proficient Students 2006 2007 2008 2009 2010 2011

2012

2013

2014

88 86 88 89 84 76 92

92 91 92 93 89 84 95

96 95 96 96 95 92 97

100 100 100 100 100 100 100

Mathematics Annual Measurable Objectives Percentage of Proficient Students 2006 2007 2008 2009 2010 2011

2012

2013

2014

89 89 88 83 83 85 91

95 94 94 91 91 93 95

100 100 100 100 100 100 100

73 73 73 78 63 51 84

63 67 59 48 40 48 73

77 73 77 78 68 51 84

68 67 65 48 49 55 73

77 77 77 81 68 59 86

68 72 65 56 49 55 77

81 77 81 81 74 59 86

74 72 71 56 57 63 77

85 82 85 85 79 67 89

79 78 77 65 66 70 82

84 83 82 74 74 78 86

10

Student Groups for Disaggregation NCLB requires the inclusion of the following groups, in addition to the All Students group, in determining AYP: special education students, major racial/ethnic groups, students with limited-English proficiency, and economically disadvantaged (free/reduced meals). Groups of sufficient size (see Group Size, page 12) must meet the same annual measurable objectives as the All Students group. Groups also must meet the 95% participation rate and, if safe harbor is invoked, meet the additional academic indicator requirement for that group. The major racial/ethnic groups that have been identified are as follows: American Indian/Alaskan Native, Asian/Pacific Islander, Black, Hispanic, and White.

Special Education Students One of the most difficult issues facing schools is the inclusion of special education students in the assessment and accountability systems. Federal regulations (December, 2003) clarified that a state is permitted to use alternate achievement standards to evaluate the performance of students with the most significant cognitive disabilities and give equal weight to proficient and advanced performance (Levels III and IV) based on the alternate standards when calculating school, school system, and state AYP. Although the regulations do not put a limit on how many special education students may participate in these alternate achievement standards, the regulations do stipulate that the number of proficient and advanced scores based on the alternate achievement standards cannot exceed 1.0 percent of all students enrolled in the grades tested at the state or school system level for accountability. Therefore, any number of students over the 1.0 percent cap scoring in Levels III and IV on the AAA will count at the lowest proficiency level (Level I – Does Not Meet Standards) regardless of the students’ actual scores. If a school system exceeds the 1.0 percent, the State Department of Education will randomly select which students are counted as proficient and which students are counted as not proficient for accountability purposes regardless of their actual scores. The scores of these students will be counted the same at the school, school system, and state levels.

In Alabama, students with the most significant cognitive disabilities are defined as those students with cognitive functioning at least three standard deviations below the mean (I.Q. of 55 and below) and whose cognitive impairments may prevent them from attaining grade-level achievements, even with the very best instruction. (Note: A school system may apply to the State Department of Education for an exception to the 1.0 percent cap.)

In past years, if a school or school system did not make AYP based solely on the proficiency index of the special education group, the U. S. Department of Education had approved the use of the Interim 2% Flexibility Option. However, effective with the 2009-2010 school year, this option was no longer available.

Limited English Proficient Students All limited-English proficient students (LEP) must participate in the statewide assessments, with or without accommodations, regardless of their level of English language proficiency or length of time in school. An exception is allowed for limited-English proficient students in their first twelve months of enrollment in U. S. schools. LEP students are not required to participate in the statewide reading assessment during their first twelve months of enrollment in U. S. schools, but must participate in ACCESS (the English language acquisition test) and in the statewide mathematics assessment. These tests will not be included in the reading and mathematics results for the proficiency index for accountability, but they will be counted in the participation rate.

11

Group Size Schools and school systems are only responsible for groups of students that are large enough to yield statistically reliable data for AYP determinations.

Forty has been established as the minimum number of students required. If a group contains fewer than 40 students, the proficiency index for determining AYP will not be applied to that group. Schools and school systems that do not meet the required minimum of 40 students in the aggregate (all students) will be assigned a proficiency index based on the number of students they do have and a notation will indicate that the school/school system did not meet the minimum requirement of 40.

Forty also has been established as the minimum number required for the participation rate for determining AYP. If a group has 40 or more students, a 95% participation rate is required. If a group contains fewer than 40, accountability will not be applied to the group. Schools and school systems that do not meet the minimum requirement of 40 in the aggregate (all students) will be required to test at least two fewer students (N-2 rule for small schools) than their enrollment in order to meet the participation rate requirement.

It should be noted, however, that although a school may not have 40 students in a group (e.g., special education students), those special education students will be included in the school system’s number of special education students. As a result, the school system may have at least 40 special education students and the special education group will be counted at the school system level.

12

Proficiency Index Because each grade has its own goal (annual measurable objective), it was necessary to develop a means of combining the percent proficient across grades. The proficiency index is the reporting metric that was developed which allows test scores to be combined across grades in determining AYP status. The proficiency index includes the following: (1) separate starting points and growth trajectories for each grade and subject, (2) a comparison of percent proficient for each grade/subject to the annual measurable objective and calculation of a difference score for each grade/subject, (3) a procedure to weight the scores based on the number of students in each grade (i.e., a weighted constant), and (4) a determination of a proficiency index in each subject by summing across grades the products of the difference scores and the weighted constants. A proficiency index score of zero or higher indicates that a group made its goal.

The proficiency index is an indication of where the group is in relation to meeting its goals across grades. A proficiency index of zero means that, on average, students are meeting the goals across grades. A positive index indicates that the group is exceeding the goals. For example, an index of 3.0 means that, on average, the group has 3 percent more of its students scoring proficient than is required. Likewise, an index of negative 3.0 means that, on average, the group has 3 percent fewer students scoring proficient than is required.

13

Calculations for Determining the Proficiency Index Students scoring in Level II will count 0.5 and students scoring in Levels III and IV will count as 1.0 when determining the number of proficient students. The use of Level II in the proficiency index calculations applies to the ARMT and the AHSGE, but does not apply to the AAA. The following is a hypothetical example of the reading proficiency index for the Hispanic student group for a hypothetical school with Grades 4 and 5 to illustrate how the proficiency index is computed.  Grade 4 reading annual measurable objective for 2011 = 86% proficient Actual percentage of Grade 4 Hispanic students (20 students) proficient = 87.500000% Difference = 1.500000% Grade 4 Calculations for Percent Proficient Grade 4 has 20 students tested who were full academic year.

6 students at Level IV = 6 x1 = 6.0 10 students at Level III = 10 x1 = 10.0 3 students at Level II = 3 x 0.5 = 1.5 1 student at Level I = 1 x 0 = 0.0 17.5 proficient Formula for determining percent proficient: # proficient / # of students tested who were full academic year 17.5/20 = 87.500000% proficient

 Grade 5 reading annual measurable objective for 2011 = 88% proficient Actual percentage of Grade 5 Hispanic students (30 students) proficient = 78.333333% Difference = -9.666667% Grade 5 Calculations for Percent Proficient Grade 5 has 30 students tested who were full academic year 7 students at Level IV = 7 x 1 = 7.0 14 students at Level III = 14 x 1 = 14.0 5 students at Level II = 5 x0.5 = 2.5 4 students at Level I = 4 x 0 = 0.0 23.5 proficient Formula for determining percent proficient: # proficient / # of students tested who were full academic year 23.5 / 30 = 78.333333% proficient

14



Weighted constants (Number Tested in Grade/Total Tested) Grade 4 = (20/50) = .4 Grade 5 = (30/50) = .6



Hispanic reading proficiency index = .4(1.500000) + .6(-9.666667) = (.600000) + (-5.800000) = -5.20%

The proficiency index in this example shows that the Hispanic group is below the goal by 5.20% percentage points. The 99% confidence interval will then be applied to determine if the group meets AYP. If a difference of -5.20% is not significant, the Hispanic group will be considered to have made its goal. (See Confidence Interval [pages 16 - 17], Uniform Averaging [pages 18 and 20], and Safe Harbor [page 21] for additional ways to meet AYP.)

15

Confidence Interval The reporting of confidence intervals is a commonly used reference. In fact, polling companies often report the results of surveys using confidence intervals around reported percentages. For example, “The results of the poll indicate that 69% of the people would vote for Candidate X – the margin of error (confidence interval) for the survey is plus or minus 4 points.”

As indicated earlier, schools, school systems, or groups with proficiency index scores of zero or higher are considered to have made their goals. However, in order to ensure statistically sound decisions, confidence intervals (CI) were placed around the proficiency index scores for schools, school systems, and groups. The confidence interval provides a test for whether or not a proficiency index is statistically different from the goal of zero at the .01 level. If a school, school system, or group’s proficiency index falls below the goal, yet the goal is within the confidence interval, then the school, school system, or group is considered to have met its goal.

The confidence interval constructed around the proficiency index is based on the formula for the standard error of a proportion, giving an indication of how much sampling variability can be expected with the size of the sample.

16

Calculations for Determining Confidence Interval The formula used to calculate the confidence interval is as follows: CI = (t-critical) x where

((100 - p) x p)/N

t-critical is the critical value for a one-tailed t-test at the .01 level (99% CI), p is the absolute value of the proficiency index, and N is the number of students in the group.

The confidence interval is calculated and added to the proficiency index value. If the sum of the proficiency index and the confidence interval is equal to or greater than zero, the group is considered to have met its goal.

The following is an example of the confidence interval calculation for the Hispanic student group with a reading proficiency index of -5.20% previously shown.

Confidence Interval Calculations for Hispanic Group 50 Hispanic students are in the group. The group has a proficiency index of -5.20%.

Formula: CI = (t-critical) x

((100 - p) x p)/N

2.4082 x ((100  5.2) x5.2) / 50 = 2.4082 x (94.8) x5.2 / 50 = 2.4082 x

492.96 / 50 =

2.4082 x = 9.8592 2.4082 x 3.139936 = 7.561594 = 7.56

When 7.56 is added to the proficiency index of -5.20, the result of 2.36 is greater than zero, so the group would be considered to have met the annual measurable objective in reading.

As indicated by the formula, the size of the confidence interval depends on the size of the group. Therefore, a confidence interval must be calculated for each group. The smaller the group is, the larger the confidence interval.

17

Uniform Averaging In order to increase reliability of decisions, Alabama incorporates a uniform averaging procedure. If the participation rate and/or proficiency index for the current year does not meet the goal, the participation rate and/or proficiency index for the most recent three years’ data, including the current year’s data, will be averaged. If this average meets the goal, the school, school system, or group will be considered to have met its goal.

Exception: If there is only one previous year’s data to average, the average will be based on two years’ data until three years’ data are available for averaging.

18

Calculations for Determining Uniform Averaging for Participation The following is an example of uniform averaging calculations for a student group with a reading participation rate of 94%.

A weighted average participation rate will be calculated using data for the most recent three years (including the current year) according to the following formula: Ti + T(i-1) + T(i-2) AP = -------------------Ei + E(i-1) + E(i-2) AP is the weighted average participation between the given year i and year (i - 2) Ti is the number of students tested for year i T(i-1) is the number of students tested for year (i-1) T(i-2) is the number of students tested for year (i-2) Ei is the number of students enrolled on the first day of testing for year i E(i-1) is the number of students enrolled on the first day of testing for year (i-1) E(i-2) is the number of students enrolled on the first day of testing for year (i-2)

Uniform Averaging Calculations for Participation Rate 2011: Tested 45 students and had 48 students enrolled.

2010: Tested 49 students and had 50 students enrolled.

2009: Tested 48 students and had 51 students enrolled.

94% Participation Rate 98% Participation Rate 94% Participation Rate

Formula: AP =

Ti + T(i-1) + T(i-2) -------------------Ei + E(i-1) + E(i-2)

45 + 49 + 48 AP = -------------------48 + 50 + 51 142 AP = ------149 AP = 95.30

When 95.30 is rounded, the result is 95. This group would be considered to have met the reading participation rate requirement of 95% for AYP.

19

Calculations for Determining Uniform Averaging for Proficiency Index The following is an example of uniform averaging calculations for a student group with a reading proficiency index of -2.35.

A weighted average proficiency index will be calculated using data for the most recent three years (including the current year) according to the following formula: (Ti)(PIi) + (T(i-1))(PI(i-1)) + (T(i-2))(PI(i-2)) API = ---------------------------------------------Ti + T(i-1) + T(i-2) API is the weighted average proficiency index between the given year i and year (i-2) Ti is the number of students enrolled Full Academic Year (FAY) and tested in year i T(i-1) is the number of students enrolled FAY and tested in year i-1 T(i-2) is the number of students enrolled FAY and tested in year i-2 PIi is the proficiency index for year i PI(i-1) is the proficiency index for year i-1 PI(i-2) is the proficiency index for year i-2

Uniform Averaging Calculations for Proficiency Index 2011: Tested 42 students enrolled for full academic year with a proficiency index of -2.35 2010: Tested 45 students enrolled for full academic year with a proficiency index of .98 2009: Tested 48 students enrolled for full academic year with a proficiency index of 3.04 Formula: (Ti)(PIi) + (T(i-1))(PI(i-1)) + (T(i-2))(PI(i-2)) API = ---------------------------------------------Ti + T(i-1) + T(i-2) (42)(-2.35) + (45)(.98) + (48)(3.04) API = --------------------------------------42+ 45 + 48 -98.70+ 44.10 + 145.92 API = ------------------------42 + 45 + 48 91.32 API = ------------135 API = .68 With a uniform averaging proficiency index of .68, this group would be considered to have met the reading proficiency index requirement for AYP.

20

Safe Harbor If students in a school/school system in the aggregate and each group do not have a proficiency index of zero or higher, the school/school system will be considered not to have made AYP. However, a safe harbor provision may allow the group (and as a result, the school) to make its goal. The safe harbor provision requires that:   

The group decreases those who are not proficient by at least 10% from the preceding year The group meets the 95% participation rate, and The group meets the goal or makes the required improvement on the additional academic indicator.

Attendance rates and graduation rates are disaggregated, as necessary, for applying safe harbor to groups within schools and school systems.

Calculations for Determining Safe Harbor The following is an example of the safe harbor calculation for a student group with a reading proficiency index of -2.64.

Safe Harbor Calculations 2011: -2.64 Proficiency Index 2010: -4.02 Proficiency Index

97% Participation Rate 96% Participation Rate

95% Attendance Rate 97% Attendance Rate

Did the group decrease by 10% those who are not proficient from the previous year?

YES Did the group have at least a 95% participation rate?

YES Did the group meet or make the required improvement on the additional academic indicator?

YES By virtue of having met all the criteria of safe harbor, this group would be considered to have made the reading proficiency index for AYP.

21

Relationship between AYP and School Improvement As stated previously, a school or school system must meet all of the following in order to make AYP: participation rate and annual measurable objectives in reading, participation rate and annual measurable objectives in mathematics, and the additional academic indicator(s). Whereas NCLB uses the term AYP to describe whether or not a school or school system has met its goals for a specific year, the term School Improvement is used to describe whether a school or school system has met its accountability goals over time. School Improvement is based on “what the school or school system did this year and prior years.”

To be identified for School Improvement, a school must miss AYP in the same component (reading, mathematics, or additional academic indicator) for two consecutive years. A school system will be identified as in School Improvement when it does not make AYP in the same component (reading, mathematics, or additional academic indicators) across all three grade spans (3-5 Grade Span, 6-8 Grade Span, and High School Span) for two consecutive years. The overall School Improvement status for a school or school system is the most “advanced” status of the three components. For example, if a school is in School Improvement Year 2 for reading, Not in School Improvement for mathematics, and School Improvement Year 1 for additional academic indicator, the school’s overall status is School Improvement Year 2. The overall School Improvement status is the status on which interventions are based.

A school or school system that makes AYP for two consecutive years in a component will be identified as Not in School Improvement for that component. A school or school system that does not make AYP for two consecutive years in the same component will be identified as in School Improvement Year 1 for that component. If the school or school system does not make AYP the next year in the same component, it will move to School Improvement Year 2, etc.

Once in School Improvement, if a school or school system makes one year of AYP, it does not advance to the next level of improvement, but retains its current improvement status in the component. (This is also known as the delay provision.) It must, however, continue to implement applicable interventions. The next year’s results determine whether the school or school system “advances” a level or returns to Not in School Improvement.

Once in School Improvement Progression of School Improvement (applied separately to reading, mathematics, and AAI) Made AYP Previous Year and in School Improvement Yes

Made AYP Current Year Yes

School Improvement Current Year Not in School Improvement

Yes

No

Progress to next level

No

Yes

Delay period

No

No

Progress to next level

22

Applying this progression to a school that was in School Improvement Year 2 the previous year, the following are possible outcomes for School Improvement for the current year.

Previous Year

Current Year

Scenarios Made AYP

1

Yes

2

Yes

3

No

4

No

Status

School Improvement Year 2 (Delay) School Improvement Year 2 (Delay) School Improvement Year 2 School Improvement Year 2

Made AYP

Yes

Status

Not in School Improvement

No

School Improvement Year 3

Yes

School Improvement Year 2 (Delay)

No

School Improvement Year 3

If a LEA requests and is approved for one of its schools to become a Turnaround School under a School Improvement Grant (SIG), the school will receive an adequate yearly progress report, retain their school’s history for the purposes of calculating uniform averaging and safe harbor, and have the sanctions of previous school improvement designations removed.

23

GLOSSARY AAA – Alabama Alternate Assessment; administered to a special education student whose IEP team determines the student is unable to participate in general state assessments, with or without accommodations Academic Achievement Levels – define how well students are mastering the state’s academic content standards at grade level Aggregate – total of all students; also called the All Students group AHSGE – Alabama High School Graduation Exam; based on the Alabama courses of study Annual Measurable Objective – state’s established annual requirement for the percentage of students scoring proficient or higher in a grade and subject ARMT – Alabama Reading and Mathematics Test; based on the Alabama courses of study for reading and mathematics AYP – Adequate Yearly Progress; used to describe if a school or school system has met its annual accountability goals Baseline – state’s established beginning point for percentage of students that must be proficient Confidence Interval – a method of meeting AYP by testing whether or not a proficiency index is statistically different from the goal Disaggregate – breakdown by group Grade Spans – for a school system, an accountability status is reported separately for three grade spans: 3-5 Grade Span, 6-8 Grade Span, and High School Span Group – distinct group within a larger group; Alabama identifies the following groups: All Students, special education, major racial/ethnic groups, limited English proficient, economically disadvantaged (free/reduced meals) Group Size – the minimum number of students (40) required for the group to be included in accountability N-2 Rule for Small Schools – for schools and school systems that do not meet the minimum requirement of 40 in the aggregate; required to test at least two fewer students than their enrollment in order to meet the participation requirement Participation Rate – percentage of students participating in state assessments Partially Proficient – partially meets academic content standards (Level II) Proficient – meets academic content standards (Level III or higher) Proficiency Index – reporting metric that allows test scores to be combined across grades in determining AYP 24

Safe Harbor – a method of meeting AYP if a group decreases by at least 10% from the preceding year those who are not proficient, meets the 95% participation rate, and meets goal or makes the required improvement on the additional academic indicator School Improvement – used to describe whether a school or school system has met its accountability goals over time Turnaround School – used to describe a school meeting specific criteria as outlined by the School Improvement Grant process.

Uniform Averaging – a method of meeting AYP by averaging the proficiency index or participation rate of the most recent three years, including the current year

25

26

System Status Summary Report

Welcome to

Alabama AYP Accountability Reporting & Appeals System System Status Summary Report Alabama Department of Education Adequate Yearly Progress Status for 2011-2012 Based on School Year 2010-2011 Data

019 Hennepin County Made AYP

2011-2012 AYP Status

Not in School Improvement

System Status Summary Report 3 - 5 Grade Span

6 - 8 Grade Span High School Span

System AYP

Reading AYP

Yes

No

No

Yes

Mathematics AYP

Yes

Yes

Yes

Yes

Yes

Yes

Yes

Yes

Additional Academic Indicator AYP

27

28

29

30