Collaboration a model for all learners by UCLA Luskin School of Public Affairs

COLLABORATION: A MODEL FOR ALL LEARNERS APPLIED POLICY PROJECT MASTERS OF PUBLIC POLICY PROGRAM UCLA MARCH 20, 2003

BY CHANDRA KELLER ERIN LILLIBRIDGE WALKER DEARTH

Table of Contents

TABLE OF CONTENTS Section

Page

Executive Summary

1. Introduction

2. Methodology

2.1 Literature Review

2.2 Qualitative Analysis

2.3 Quantitative Analysis

3. Background and History

4. Literature Review

4.1 Early Intervention Works

4.2 The Discrepancy Model is Flawed and Delays Service to Students

4.3 Research on Alternative Strategies

4.4 Criticisms of the Collaborative Model

4.5 Conclusion

27 28

5. Elk Groveâ&#x20AC;&#x2122;s Collaborative Model 5.1 The Population

5.2 Implementation

5.3 Key Components of CAST

5.4 Reported Results

5.5 Lessons Learned

42 47

6. Hesperiaâ&#x20AC;&#x2122;s Collaborative Model 6.1 The Population

6.2 Implementation

6.3 Key Components of ExCEL

6.4 Reported Results

6.5 Lessons Learned

Table of Contents

Section

Page 66

7. Model Program Comparison 7.1 Impetus for Change and Implementation

7.2 District Commitment

7.3 Program Elements

7.4 Principal Key Factor

7.5 Measuring Outcomes

7.6 Conclusion

69 71

8. Quantitative Analysis 8.1 District Descriptions

8.2 Statistical Analysis

8.3 Grade Level Descriptive Statistics

8.4 Cohort Descriptive Statistics

8.5 Regression Model on Score Gains

8.6 Limitations

8.7 Conclusion

8.8 Discussion

9. Implementation Analysis

9.1 Existing Resources

9.2 Potential Costs

9.3 Potential Savings

94 96

10. Implementation Recommendations 10.1 Create Teams, Define the Problem and Set Goals

10.2 Assemble Evidence and Devise a Program

10.3 Get Buy-in and Commitment

10.4 Start Small â&#x20AC;&#x201C; Pilot Schools Begin Planning

10.5 Monitor and Evaluate Effectiveness

99 102

11. Evaluation Plan 11.1 Design a Controlled Experiment

103

11.2 Identify Goals and Desired Outcomes

104

iii

Table of Contents

Section

Page

11.3 Use Data to Inform Decisions

105

11.4 Define Implementation

106

11.5 Conclusion

106

12. Conclusion

108

Endnotes

110

Bibliography

Bib-1

Appendices A â&#x20AC;&#x201C; I

A-1

EXECUTIVE SUMMARY In America’s current educational system, at-risk students are failed by the general and special education models due to a lack of collaboration and a neglect of early intervention. The current system for identifying students with specific learning disabilities is a “wait to fail” model. Students who are struggling are often left to fail for one to two years before receiving needed services through special education. These students at risk for school failure are falling through the cracks, while special education services come too late and at a greater cost to address their needs. Evidence of this failure is the fact that, of students labeled with a specific learning disability, it is estimated that 80 percent qualified because of reading difficulties as a result of poor or nonexistent instruction, not necessarily because of true learning disabilities. This is unacceptable. Public school systems must take the opportunity to reorganize their resources in order to provide services more efficiently to students in need. Two Districts in California, Elk Grove Unified and Hesperia Unified, have created innovative approaches to address this crisis. Our client, the Director of Special Education in Pasadena Unified School District, has asked us to investigate these programs and report on their success as well as the feasibility of implementation in Pasadena. We have collected an array of both qualitative and quantitative information on the program components, impact on academic achievement, and success factors in implementation from each of the model Districts. We report the following main findings: Program Findings The collaborative service delivery model implemented in both Districts includes extensive collaboration between general and special educators, early literacy intervention, and serving students based on need, rather than based on eligibility for special education. Quantitative Findings We find that the collaborative service delivery models are particularly effective in raising student achievement, as measured by standardized test scores, in the second grade. In addition, the collaborative service delivery models contributed to a significant reduction in special education caseloads. Implementation Findings In order to increase a District’s chances of successful implementation of a collaborative service delivery model, we recommend the following: • Top-level District administrators need to lead the change effort. • Create a broad-based team of District stakeholders to design a collaborative model. • Build upon existing resources and programs within the District. • Start small and implement in a few well chosen pilot schools first. • Design an evaluation plan prior to implementation.

Introduction

1. INTRODUCTION

Once upon a time, there was a town that had in it a playground located at the edge of a cliff. Every so often a child would fall off the cliff and would be seriously injured. At last the town council decided that something should be done. After much discussion, however, the council was deadlocked. Some council members wanted to put a fence at the top of the cliff, but others wanted to put an ambulance at the bottom.i The above parable is taken from a paper written by Robert E. Slavin, co-chair of the Center for Research on the Education of Students Placed At Risk at Johns Hopkins University. It illustrates a fundamental problem with the education model in Pasadena Unified School District and the majority of districts nationwide. The current identification and service delivery model of special education for students with specific learning disabilities is akin to the ambulance solution. This “wait to fail” model addresses students’ needs for extra support and services after the fact, often too late, and at a greater cost to the district and to the student. Early interventions, even very expensive ones, can be justified on cost-effectiveness if they reduce retentions and the need for later remedial and special education services.ii Our client, the Director of Special Education in the Pasadena Unified School District (PUSD), Judith Barhydt, has commissioned us to examine two collaborative educational service delivery models that appear to effectively address the needs of students at risk of school 1

Introduction

failure without necessarily entering them into the special education system. The main concepts of a collaborative service delivery model include close teamwork between general and special educators, early reading and literacy intervention, and serving students based on need, rather than based on eligibility for special education. There are some indicators that suggest these two model programs in Elk Grove and Hesperia Unified School Districts have been successful. A brief overview of Pasadena suggests there may be significant unrealized gains to be made. Current typical school organization does not foster meaningful and substantive collaboration among general education, special education, curriculum resource, Title 11, literacy, and language development staff.iii Students are often seen to be a member of a category and served only by the teacher(s) and support staff funded by that category. Thus, Pasadena is not maximizing the potential of its talented professionals or providing services to all students as efficiently as possible. The purpose of this investigation is three-fold. First, we will conduct a thorough qualitative and quantitative evaluation of the extent to which the two model programs have achieved success. Second, we will analyze the applicability of a similar program to the Pasadena Unified School District, highlighting key success factors as well as constraints and considerations particular to applying the model in Pasadena. Third, we will identify recommendations to plan, adopt, implement, and evaluate a collaborative service delivery model in Pasadena. In an effort to evaluate the models to the best of our ability, we used a comprehensive approach in our investigation.

Title 1 students are those scoring at or below the 36th percentile. A school with a majority of students scoring this low is considered a Title 1 school and receives federal funding through the Elementary and Secondary Education Act (now No Child Left Behind).

Methodology

2. METHODOLOGY

Educational practice involves a complex integration of variables that together impact a studentâ&#x20AC;&#x2122;s development and progress. Relying on any single measure of an educational intervention model can lead to an incomplete picture of its impact. In order to obtain a more nuanced view of the model programs, we took a comprehensive approach in investigating success and implementation. First, we conducted a literature review of the relevant research. Second, we completed an extensive qualitative analysis including interviews, site visits, and a review of District documents. Third, we performed statistical analyses using school-level test score and demographic data to assess the impact of the program as measured by student achievement. 2.1 Literature Review Our literature review included combing through the most prominent research on learning disabilities, at-risk students, and evaluations of existing intervention programs to address student achievement. We also consulted the results of the Presidentâ&#x20AC;&#x2122;s Commission on Excellence in Special Education, which issued its report in June of 2002. Leading experts in the area of learning disabilities and reading acquisition such as G. Reid Lyon, Chief of the Child 3

Methodology

Development and Behavior Branch within the National Institute of Child Health and Human Development at the National Institutes of Health, served on the Commission. The Superintendent of Elk Grove Unified School District, David Gordon, also served on the Commission. The Commissionâ&#x20AC;&#x2122;s report summarized findings based on testimony by experts and made specific recommendations to Congress for the purpose of reauthorizing the Individuals with Disabilities Education Act of 1997 (IDEA 97). The Commission report provides a framework for current special education reform and an overview of the relevant research that drives the concepts underpinning the models we are investigating. From the literature review we hoped to gain a sense of what current research tells us about the effectiveness of the traditional special education service delivery system, existing intervention programs such as preschool or class size reduction, and an understanding of the historical trends that drive the way the educational system has attempted to address the needs of students at risk of failure in school. 2.2 Qualitative Analysis We conducted phone and face-to-face interviews, District and school site visits, and communication via email to gather qualitative information about Elk Grove, Hesperia, and Pasadena Unified School Districts. We also heard presentations from each of the model Districts and were given copies of PowerPoint handouts. Both Hesperia and Elk Grove provided us with other related documents. Overall, we interviewed five District-level special education administrators, four school site administrators, six special education teachers, one special education assistant, and one School Board member in gathering information from both the model districts and from Pasadena.2 The interviews were conducted at three District offices, six elementary schools, one middle school, over the phone, and via email.

Methodology

School sites and interviewees in Elk Grove and Hesperia were selected by the Directors of Special Education. In Elk Grove, we visited Prairie Elementary and Rutter Middle Schools. Prairie Elementary was one of the five pilot schools to implement the model in 1992/93. In Hesperia we visited two school sites, Cottonwood and Maple Elementary Schools. Cottonwood and Maple Schools are considered showcase schools for this program. Both were among the first to fully implement the model in 1997/98. School sites in Pasadena were selected based on personal recommendations from one of the team members. The Resource Specialist Teachers at two of the sites, Noyes and Field Elementary, have already been serving small numbers of general education students on an ad hoc basis. The site administrator at the third school, Longfellow Elementary, is interested in improving the special education referral process and possibly tailoring instruction for general education students more to their individual needs. While the schools we visited had demographic differences, certain variables were comparable.3 Ethnically, the Hesperia schools were majority white and Latino while the Pasadena schools were mostly Latino and African American with a minority of white students. About eighty to ninety percent of the students in the schools we visited in Elk Grove were minority (African American, Latino, Asian, and other). We used CalWORKS4 and students receiving free or reduced price meals5 as poverty

See Appendix A for a complete list of interviewees. See Appendix B for a table of demographic variables for each school visited. 4 Formerly Aid to Families with Dependent Children (AFDC). CalWORKS stands for California Work Opportunity and Responsibility to Kids. This data represents all economically disadvantaged children ages 5 through 17 in the boundaries of the Local Education Agency (District). We used CalWORKS as descriptive data only to add information about the school neighborhood. This variable is not used in our regression analysis since it includes all CalWORKS students and not just those enrolled at a particular school site. 5 The Free and Reduced Price Meal Program is a federal program administered by the US Department of Agriculture. Program participation is based on the income of the childâ&#x20AC;&#x2122;s parent or guardian. The data reflect the number of children enrolled in the program, not all who are eligible. This is the poverty indicator we use in our regresion analysis. 3

Methodology

indicators. The schools we visited ranged from having 48% of their students receiving free or reduced price meals (Noyes in Pasadena) to 78% (Prairie in Elk Grove). All but one school (Noyes) had over half of their school population identified as receiving free or reduced meals. The Hesperia schools we visited had fewer students identified as English Language Learners than the schools in both Elk Grove and Pasadena. Overall, higher percentages of the teachers in Hesperia and Elk Grove had their full teaching credential than teachers at the Pasadena sites we visited. However, the average years of teaching experience tended to be the same or higher in the Pasadena schools. The focus of the qualitative analysis was to get a first hand look at the program in action as well as to speak face to face with key participants who initiated the process and who are actively implementing the model. We chose to engage in a dialogue with these key participants, rather than attempt to administer a survey of all teachers and administrators in each District, for two reasons. First, administering a survey to the professional staffs at two school districts was beyond the scope and time frame of our project. Second, we felt that gaining the insight of fewer people via more in depth, personal discussions, would provide us with the information necessary to understand the implementation process clearly. Through these discussions with teachers, principals, and administrators we hoped to identify problems and key success factors. The qualitative analysis allowed us to learn about the softer indices of success such as teacher and administrator perceptions of the implementation process. In addition to information about the challenges and successes of implementation, we gained invaluable information about measures of success other than student test scores, such as cost savings and reduction of special education caseloads. Seeing the model in action and having discussions with staff enabled us to present a much richer analysis of the model districts than if

Methodology

we had relied on student test scores alone to determine success. One limitation of our qualitative data is that the interviewees and site visits were determined by the districts we visited rather than a representative sampling plan. In addition, we were unable to conduct any parent interviews and consequently, the family perspective is not considered in our study. However, it can be inferred that there has been no movement in either district from the parents to change or remove the system since the program in Elk Grove has continued to evolve for ten years and in Hesperia for about five years. 2.3 Quantitative Analysis In order to assess the model programsâ&#x20AC;&#x2122; impact on student achievement, we examined schoollevel Stanford 9 (SAT9) test score data provided by the California Department of Education website. The SAT9 was administered to all students in grades 2-11 in the state of California for five years from the spring of 1998 to the spring of 2002. We used school-level demographic data on ethnicity, free or reduced meal program (poverty indicator), English language learners, and teachers characteristics for the same years. The quantitative analysis portion of the evaluation suffers from several weaknesses due to the limitations of the data. First, we have no pre and post treatment data to judge the true impact of the programs. The SAT9 was administered after Elk Grove implemented its model program and concurrently with the initial implementation in Hesperia. No state administered test was given in the two years prior to the SAT9. The CAT5 was the state standardized test given in the early 90s, but it is not directly comparable with the SAT9. Furthermore, the California Department of Education (CDE) does not provide historical data on the CAT5. Therefore, we do not have consistent test score data for before and after program implementation. We cannot directly assess the impact of the models on student achievement without comparable baseline

Methodology

data from before implementation. Second, as the qualitative analysis will reveal, the collaborative service delivery models were not implemented uniformly in schools throughout the model districts. Individual schools within Elk Grove and Hesperia did not all begin full implementation the first year. Also, schools did not all implement in exactly the same way. School administrators and staffs were given flexibility in designing the model to fit the needs of their site and in determining the time frame for implementation. For example, some individual schools implemented the program within only one grade level for the first few years. This variability in implementation clouds any changes seen in student achievement. It is more difficult to attribute school or district-wide success because each child was not exposed to the same exact treatment beginning at the same time. Third, the California Department of Education collects and reports only school-level test score data by grade level. No student level data is available. Each observation in our quantitative analysis is a school, rather than an individual student. Achievement is reported by percentage of students in a particular grade level at a specific school scoring at or above the 50th percentile on the SAT9 math or reading subtest. The result of this limitation is a low number of observations (87 schools total in all treatment and control districts), which negatively impacts the statistical power of our analysis. Fourth, our quantitative analysis focuses on measuring one outcome, standardized test scores. Districts report that increased student test scores is not the only outcome of interest with respect to the collaborative service delivery model. Other measures such as attendance rates, disciplinary referrals, special education caseloads, special education referrals, and stakeholder satisfaction are all relevant in assessing the impact of the program. For the purpose of our study,

Methodology

we did not have access to all of this data, hence, it is not included in the statistical analysis. Therefore, this study could be underestimating the impact of the program simply because we were unable to analyze all relevant variables. Finally, we were unable to find control districts for the analysis that match either of the model districts on all demographic and baseline test score data (1998 scores). The best available information led us to choose two control districts, Ventura and Redlands Unified, that are demographically similar to Hesperia on enrollment, percent minority, poverty, and English language learner variables. We did not include a direct control for Elk Grove because the California Department of Education reported no other district in California with similar enrollment, ethnic, poverty, and language status variables. Though they are demographically similar, Ventura and Redlands Unified Districts pose many problems as controls for Hesperia. The school test scores in Ventura are highly volatile from year to year and consistently report a significantly higher standard deviation in any given year in our analyses. Well into our project, we also discovered that a group of teachers in the Ventura Unified School District learned about Elk Groveâ&#x20AC;&#x2122;s service delivery model and have been serving non-identified students with special education resources within some of their schools for the past two years. We made attempts to gather information to correct for this, but it is clearly an imperfect control since it is confounded with the treatment. Redlands Unified reports consistently higher test scores starting in the first year of the SAT9, while Hesperiaâ&#x20AC;&#x2122;s baseline is extremely low. Districts with a similar baselines may have been better controls. During our site visits to Elk Grove and Hesperia, district officials reported gains in test scores since implementation and shared column charts showing how District-level test scores have gone up since 1998. However, neither District has completed an assessment of the

Methodology

statistical significance of the gains or determined conclusively that the increase in scores is associated with the treatment. Through our quantitative analysis, we hoped to find that student achievement in the model districts showed statistically significant greater gains than the achievement reported in the control districts and in Pasadena. We will provide the reader with a rough background and history of the problem these collaborative service delivery models are meant to address before moving on to our comprehensive analyses.

Background and History

3. BACKGROUND AND HISTORY

A significant trend in American education since the late seventies has been the increase in the number of students with mild disabilities receiving special education services.iv Since 1976, the number of students identified as having a specific learning disability (See Figure 1 for statutory language) increased more than 300 percent.6 Students with specific learning disabilities (SLD) represent over half of the six million children in special education in the United States today. Of students labeled with a specific learning disability, it is estimated that 80 percent have qualified because of reading difficulties as a result of poor or nonexistent instruction, not necessarily because of a true learning disability.v

The first federal legislation for students with special needs, The Education for All Handicapped Children Act (EAHCA or PL 94-142), was passed in 1975. The definition of specific learning disability first appears in this law. 11

Background and History

Figure 1 â&#x20AC;&#x201C; What is a specific learning disability?vi Specific learning disability is defined in the Individuals with Disabilities Education Act of 1997 (reauthorization of PL 94-142, EAHCA of 1975) in Sec. 602(26) as the following: (A) IN GENERAL- The term 'specific learning disability' means a disorder in one or more of the basic psychological processes involved in understanding or in using language, spoken or written, which may manifest itself in imperfect ability to listen, think, speak, read, write, spell, or do mathematical calculations. (B) DISORDERS INCLUDED- The term includes such conditions as perceptual disabilities, brain injury, minimal brain dysfunction, dyslexia, and developmental aphasia. (C) DISORDERS NOT INCLUDED- The term does not include a learning problem that is primarily the result of visual, hearing, or motor disabilities, of mental retardation, of emotional disturbance, or of environmental, cultural, or economic disadvantage. (a) A team may determine that a child has a specific learning disability if: (1) The child does not achieve commensurate with his or her age and ability levels in one or more of the areas listed in paragraph (a)(2) of this section, when provided with learning experiences appropriate for the childâ&#x20AC;&#x2122;s age and ability levels; and (2) The team finds that a child has a severe discrepancy between achievement and intellectual ability in one or more of the following areas: (i) oral expression; (ii) listening comprehension; (iii) written expression; (iv) basic reading skill; (v) reading comprehension; (vi) mathematics calculation; or (vii) mathematics reasoning. Source: Assistance to States for Education of Children with Disabilities Program and Preschool Grants for Children with Disabilities Final Rule, 34 C.F.R. pts. 300, 301 (1992); see also note 2.

Students identified as having a specific learning disability comprise the largest category of special education students in Pasadena. Specific learning disabled students account for almost 43% of the special education population. The second largest category of disability is speech and language impairment. The severe disabilities we commonly think of when referring to special education, such as mental retardation and autism, account for less than 15% of the special education population. Figure 2 depicts the incidence of disability categories in Pasadena.

Background and History

Figure 2 – Disability Categories7 SLD Most Impacted Disability Category in PUSD 42.6%

SLD 29.6%

SLI ED

8.8%

All Other

7.8%

7.3%

AUT 0.0%

4.0% 20.0%

40.0%

60.0%

80.0%

100.0%

Eligibility Labels MR = Mental Retardation SLI = Speech or Language Impairment ED = Emotional Disturbance SLD = Specific Learning Disability AUT = Autism All Other = Deaf, Hard of Hearing, Visual Impairment, Orthopedic Impairment, Other Health Impairment, DeafBlindness, Multiple Disability, Traumatic Brain Injury

Source: CDE Reporting Cycle – December 2001. Based on 2,864 total special education population. http://data1.cde.ca.gov/dataquest/

Special education students in Pasadena are placed along a continuum of services ranging from “most restrictive” to “least restrictive.” For example, the least restrictive environment is the general education classroom with typically developing peers and a more restrictive environment is a smaller class with other special education students only. The placement options are detailed on the following page in Figure 3. Federal and state law favor placing students in the least restrictive environment that will meet their needs as designated on the student’s Individualized Education Plan (IEP).

See Appendix C for a description of each disability category. 13

Background and History

Figure 3 – Continuum of Special Education Placements in Pasadena Least Restrictive

Setting General education classroom Resource Specialist Program Modified inclusion Blended full inclusion Non-severely handicapped Special Day Class Severely handicapped Special Day Class

Most Restrictive

Non-public school

Description Student’s needs are met within the general education classroom. Student is in general education classroom for majority of the day; may receive pull out services from Resource Specialist Teacher less than 50% of the time. Up to 5 special education students are placed in a general education classroom with a general education teacher and instructional assistant. Up to 10 students are included in a general education classroom with one general education teacher, one special education teacher, and an instructional assistant. Up to 12 students with mild to moderate disabilities are in a special education class with one special education teacher and one instructional assistant. Up to 12 students with moderate to severe disabilities are in a special education class with one special education teacher and one instructional assistant. Student with severe academic, behavioral, or emotional needs is placed in a non-public school for special education students.

Like many districts, Pasadena follows the “wait to fail” model of identification for specific learning disability that is supported in state and federal statute (See Figure 4 for a flowchart of the process).vii In current practice, if a student is performing poorly in school, a teacher or a parent can request a Student Study Team (SST) meeting. The SST is typically comprised of the student’s classroom teacher, the site Resource Specialist Teacher, the principal, and the parents. Any other adults who may be working with the child would also be included. The purpose of the SST is two-fold. First, it serves as a viable way for school staff and parents to get together and collaborate on helping the student in need. Second, the SST process is used as a “gatekeeper” to special education by requiring staff to exhaust all possible ways to support a

Background and History

struggling student within the general education system before referring him for a special education evaluation. The SST identifies the student’s area(s) of weakness and brainstorms strategies to address the problem(s). A period of time passes by which the teacher implements the suggestions brainstormed at the team meeting. The SST can meet several times to revisit the concerns and assess gains. Unfortunately, the main focus of the SST process seems to be on gatekeeping, rather than on collaboration. Teachers and parents know that in the current system, students receive interventions only if they are identified as special education. Also, since the SST is typically led by the site’s Resource Specialist (a special education teacher), there is a perception that the SST process simply represents a series of hoops parents and teachers must go through to get a child into special education. When the SST participants feel that all resources and strategies have been exhausted and the student is still not progressing, they forward the SST paperwork to the Special Education Department to refer the student for a psycho-educational assessment.8 A program specialist in the Special Education Department reviews the SST paperwork and either returns it to the school for further general education intervention and documentation or forwards them to a school psychologist. A psychologist reviews the documents and drafts an assessment plan for the student. The assessment plan is sent to the parents for their signature. The psychologist has fifty days from the date of the parents’ signature to assess the student. At the completion of the assessment, the psychologist determines if there is a two or more year discrepancy between the student’s ability and achievement. If there is such a discrepancy, the student is given an eligibility statement for special education. The team, comprised of parents and teachers, holds an initial Individualized

After exhausting all general education resources, the SST can also refer a student to AB3632 for mental health services or for a 504 Plan for behavioral concerns if the primary issue is not necessarily a suspicion of a disability. 15

Background and History

Education Plan meeting and determines the extent of services and placement that best meets the students’ needs. However, if there is no discrepancy or it is not big enough, the student simply remains in general education. As one can see from this identification model, much time is lost between the initial identification of a struggling student and the application of much needed specialized interventions provided by special education. There are cases where students do not have a significant discrepancy the first time they are assessed by a psychologist, but after one to two more years of failure, they do qualify for special education. Bill Tollestrup, Director of Special Education for Elk Grove, reports that between 40-60% of students who have initial evaluations do not qualify. This is why the process has been dubbed the “wait to fail” identification model. Figure 4 – Pasadena’s Current SLD Identification Process Step 1 – The Student Study Team (1-2 years) Teacher and/or parent notices student struggling and not achieving.

Teachers, parents, and principal hold a Student Study Team meeting.

Teachers try strategies brainstormed in the SST meeting.

This process can be repeated several times over a 1-2 year period.

Step 2 – Referral to Special Education (several months to a year) The SST feels they have exhausted all general education resources.

The team forwards SST documentation to Special Education Dept.

A specialist reviews it.

SST goes to psychologist. Go to step 3.

Step 3 – Eligibility Determination (2 – 6 months) Psychologist sends an assessment plan to parent for signature.

SST sent back to school. Go back to step 1.

Psychologist determines if there is a 2-year discrepancy between ability and achievement.

Once parent signature obtained, psychologist has 50 days to assess.

If yes, special ed.

If no, return to gen. ed.

Background and History

There must be a more efficient way to utilize professional expertise and resources to serve students. Since specific learning disabled students account for the greatest proportion of the special education population and the discrepancy model has been sharply criticized in the literature and in practice, it makes sense to target interventions at the students at risk of being identified with a specific learning disability. This is where reform could potentially have the greatest impact. Through our investigation of current best practices, we discovered that the concepts underpinning these models are indeed validated in the current learning disability and reading intervention research literature.

Literature Review

4. LITERATURE REVIEW

Our research summary shows that virtually every child can succeed in the early grades in principle. The number who will succeed in fact depends on the resources we are willing to devote to ensuring success for all and our willingness to reconfigure the resources we already devote to remedial and special education and related services. The key issue for at risk students is not if additional costs will be necessary, but when they should be provided.viii This statement by Professors Robert Slavin, Nancy Karweit, and Barbara Wasik of the Center for Research on the Education of Students Placed At Risk at Johns Hopkins University illustrates three very important points that validate the educational approach of Elk Grove and Hesperia. First, the public and its school systems already spend an enormous amount on academically ineffective and economically inefficient models of special and remedial education that result in significant numbers of children being left behind. Second, this investment can be made on the front-end, early in elementary school, or at a higher cost on the back-end, as students are diverted into special education, retained, drop out, or engage in other behaviors costly to society. Third, in the absence of additional funding, the first step in education reform is reconfiguring the management and use of resources the public already devotes to teaching all students to read and succeed early in school. This is precisely what our model districts have

Literature Review

attempted to accomplish. ix The research on learning disabilities and at-risk students spans forty-one years. Dr. Samuel Kirk developed the concept of â&#x20AC;&#x153;learning disabilitiesâ&#x20AC;? in 1962. Dr. Kirk created a model to conceptualize a group of students who unexpectedly under-perform despite exhibiting average or adequate mental capacity. He further asserted that a learning disability represents a discrepancy between a childâ&#x20AC;&#x2122;s achievement and his or her capacity to learn.x Dr. Kirk, a psychologist, used IQ tests to measure this discrepancy, a practice that the school system adopted. A vast body of research has developed around this conception of learning disabilities, which has been expanded to include those students at-risk of becoming learning disabled or failing in school. Among the most important works are those of Professor Robert E. Slavin and Dr. G. Reid Lyon. These and other scholars examined learning disabilities, reading disabilities, and reading acquisition as well as the many models and types of early intervention. In general, this literature yields four important points with respect to our inquiry: 1. Reading intervention and prevention efforts in grades K-2 work better than remediation, especially when delivered using a collaborative, systemic educational model. 2. Using the discrepancy model to identify and serve students is ineffective and may allow students to fail for several years before receiving help, thus seriously threatening the educational and life chances of these students. 3. Other policy alternatives or delivery models do not produce effects as large or longlasting as collaborative early intervention programs. 4. Criticisms of early intervention and prevention models do not prove that the costs outweigh the benefits of such programs nor recognize the failure of the status quo. 4.1 Early Intervention Works In our review of the research literature, we find substantial support for collaborative early intervention programs. Researchers found that early intervention programs produce large, long-

Literature Review

term positive effects on student reading proficiency and achievement on tests. Specifically, such programs may substantially reduce the number of children that might otherwise become eligible for special education services.xi The most successful models for preventing early reading failure of at-risk students had five features in common. First, they incorporated school-site collaborative teams that crafted education interventions for individual students by collecting and reviewing a range of student data, including periodic classroom assessments of observable academic skills. Second, the programs utilized teachers as tutors instead of over-relying on aides in the classroom or referring a student for special education testing when he or she had fallen behind significantly. Third, the programs were highly structured and involved specific curricula, materials, and teaching methods. Fourth, implementation required additional teacher and staff training, coaching and collaboration with other school professionals. Finally, the programs targeted reading in the early grades, especially fundamental reading skills such as letter recognition, sound-symbol relationships, blending, word recognition, and comprehension. In the most effective programs, intervention, even one-on-one tutoring, was available to all students who showed specific deficits on classroom diagnostic assessments.xii 4.2 The Discrepancy Model is Flawed and Delays Service to Students Recent research has discredited the traditional discrepancy model used to identify students with learning disabilities, a model the currently used by a majority of districts. First, scholars note that the discrepancy model is conceptually flawed. According to Dr. G. Reid Lyon and the Presidentâ&#x20AC;&#x2122;s Commission on Excellence in

â&#x20AC;&#x153;Particularly sobering is the finding that over 70 percent of the group identified as RD [reading disabled] in grade 3 was still identified as RD in grade 12.â&#x20AC;?

Special Education, the model relies on the comparison of IQ tests

Literature Review

to academic assessments to establish a discrepancy between a child’s learning potential and current performance. As Dr. Lyon states, “IQ tests reflect primarily a gross estimate of current general cognitive functioning and should not be used as a measure of learning potential.”xiii Comparing IQ and academic achievement scores is fraught with psychometric, statistical, and conceptual problems that may render such comparisons useless.xiv The President’s Commission report notes, “There is little justification for the ubiquitous use of the IQ tests for children with high-incidence disabilities, except when mild mental retardation is a consideration, especially given their cost and the lack of evidence indicating that IQ test results are related meaningfully to intervention outcomes.” Thus, the President’s Commission recommends “eliminating IQ tests from the identification process.”xv Second, the process of identifying, documenting, and testing a child’s discrepancy between IQ and achievement can take too long to be of use. In many cases, students fall behind 2 to 3 grade levels by the time they receive services. Such students are likely to be unmotivated, anxious about reading, and develop poor self-esteem as they fall behind peers.xvi Furthermore, remediation does not work very well, particularly for students in later elementary grades.xvii The traditional special education model focuses on eligibility, restricting access to services, and due process rather than instruction, early intervention, and prevention. Students determined to be ineligible for service must wait to fail further and those qualified for service have failed too long. Slavin, Karweit, and Wasik conclude, “Longitudinal studies find that at-risk 3rd graders who have failed one or more grades and are reading below grade level are extremely unlikely to complete high school.”xviii Dr. Lyon states, “Particularly sobering is the finding that over seventy percent of the group identified as RD (reading disabled) in grade 3 was still identified as RD in grade 12.”xix

Literature Review

Finally, the current special education systemâ&#x20AC;&#x2122;s eligibility and testing model is expensive and inefficient. The eligibility evaluation process can cost from $800 to $8,000 dollars per student and take hundreds of staff hours to complete.xx The Elk Grove Special Education Local Plan Area (SELPA) estimates that initial evaluations of special education eligibility cost $1,000 dollars per student on average.9 The total fiscal impact on school districts is significant. 4.3 Research on Alternative Strategies Research has pointed to universal best practices to improve student learning. Cognitive development and family support programs for children 0 to 3 years old, preschool, full-day Kindergarten, class size reduction, and collaborative early intervention all have positive effects on student academic achievement and other life outcomes. In the ideal world, all of these strategies would be funded and integrated into the child development and educational systems. However, given scarce resources, the question to ask is: which program yields the largest, most sustained gains in student achievement at the lowest cost? Research demonstrates that, taken individually, alternative strategies produce moderate gains in test scores. However, alternatives such as early childhood development and preschool mostly impact variables such as IQ, retention, drop out, and teen pregnancy rates. The gains in academic performance linked to preschool disappear in the first few years of school. Alternatives such as full-day Kindergarten and class-size reduction do improve student test scores, but the effects wear off by the end of 1st grade for full-day Kindergarten and by the end of 4th grade the gains are very small for class-size reduction.

A SELPA, created by California legislation, serves as a local education agency for special education students. Typically a SELPA serves 2000-4000 special education students. There can be single-District SELPAs, such as Elk Grove and Pasadena, or multi-District SELPAâ&#x20AC;&#x2122;s, such as the Desert/Mountain SELPA of which Hesperia is a member. The SELPA is responsible for a Local Plan for how students with special needs will be served. 22

Literature Review

4.3.1 Early Childhood Development Slavin, Wasik and Karweit report that child and family-based interventions in the first 3 years of life substantially and permanently increase students’ IQ test scores. For example, the Milwaukee Project increased student performance by paying for a licensed caregiver to spend 35 hours per week helping a mother with a mild mental disability and her at-risk child. The program provided high-quality preschool later on. The caregiver and parent improved the child’s school readiness by focusing on healthy cognitive development and by engaging the child in reading, speaking, listening and questioning activities.

In addition, the program provided parenting classes,

vocational education, and other services to participating parents. Even at age 10, the participating children had significantly higher IQs than a control group of at-risk students who did not receive the treatment. The Carolina Abecedarian Project produced similar results.xxi 4.3.2 Preschool Immediately following preschool, Kindergarten students show higher IQ and language skills than students who did not attend preschool. However, Karweit finds that the academic performance gains do not last beyond the early grades.xxii The most significant, long-term benefits of preschool are lower retention in grade and placement in special education rates. In addition, preschool is associated with lowering student dropout rates and delinquency. Slavin et al note that retention and special education placement are highly correlated with dropping out, which suggests that preschool impacts student dropout rates through keeping more students out of special education or from repeating a grade.10

Slavin, Karweit and Wasik report that grade retention and “transitional 1st grade” alternatives are not beneficial to students in the long run. They state that the experience of spending another year in school before 2nd grade has no long-term benefits and that students retained before third grade are at a higher risk of dropping out than students who were not retained. 23

Literature Review

4.3.3 Full-Day Kindergarten Currently, most school districts provide students with a half-day Kindergarten program. To boost student language ability and general readiness for 1st grade, some schools have opted to provide full-day programs. Research finds positive outcomes for students in full-day programs that use structured, sequenced literacy, and pre-reading programs. However, the treatmentâ&#x20AC;&#x2122;s impact on reading performance does not last beyond the first or second grades.xxiii 4.3.4 Class Size Reduction Between 1985 and 1989, the State of Tennessee conducted a scientifically controlled experiment to evaluate the impact of smaller class sizes on student achievement. The â&#x20AC;&#x153;Tennessee Experimentâ&#x20AC;? reduced class sizes from 25 students per class to 15 students per class for 6,500 students in grades K to 3. In this experiment, students were randomly assigned to one of three classroom types: 25 students with a teacher, 25 students with a teacher and a classroom aide, and 15 students with a teacher. The classes of 25 students without an aide served as the control group to assess the impact of classrooms with an aide and smaller classes. First grade students in smaller classes gained 0.23 standard deviations in reading and 0.27 in math test scores.xxiv These figures translate into roughly a 10 percentage point gain on the norm referenced ranking scale. Students in larger classes with an aide only demonstrated one twelfth of a standard deviation gain (5 percent gain in ranking) as compared to the scores of students in larger classes (the control). This reveals more gains in test scores for students in small classes than in regular classes with or without an aide. Of particular note, minority students earned scores that were, on average, 0.4 standard deviations higher in reading and 0.6 better in math. This represents a 16% gain in reading and 24% gain in math percentile ranks. In sum, students in smaller classes scored higher than the control group.

Literature Review

However, the small class size effect appears to fade considerably by the end of 3rd grade. The magnitude of students’ gain on reading and math test scores was positive but smaller in grades 4 to 8. Alan Krueger and Dianne Whitmore evaluated the Tennessee Experiment and demonstrated that students in grades 4 through 8 who were previously in small classes, earned 5% higher marks on tests than students who did not attend smaller classes. This advantage remained larger for minority children. They also reported that 4th to 8th grade students who were in K-3 classes with instructional aides made no statistically significant gains. In conclusion, alternative interventions produce positive academic and social outcomes for students. However, early childhood development, preschool, full-day Kindergarten and class size reduction require substantial monetary and staff inputs. For example, in 1996 California spent $1 billion dollars to reduce grades 1 to 3 from roughly 29 students per class to 20 students. This cost is recurring and grows as the student population and labor costs increase. In contrast, the collaborative early intervention and prevention model creates far fewer demands for additional space and staff. However, the collaborative models require staff time to 1) conduct inclass diagnostic assessments at least three times a year; 2) collaborate with other teachers and specialists; and 3) participate in professional development and school-site visits of model programs. 4.4 Criticisms of the Collaborative Model The main criticism of the early intervention model holds that some students will be misidentified as at-risk for academic failure and provided interventions they do not need. Dr. Lyon responds that the costs of delaying intervention are greater than intervening early. The wait to fail model is far more costly in terms of dollars and student outcomes than giving a misidentified student additional phonics instruction. Secondly, the “wait to fail” model’s eligibility testing routinely

Literature Review

denies service to roughly 50 percent of those students who are tested for special education. And yet, many of the students deemed ineligible do actually require interventions. Thousands of dollars are spent on IQ and other technical tests rather than on interventions for students who demonstrate learning gaps. The direct assessment and service costs as well as the opportunity costs of the existing system are much higher. Some might see the scaffolding element seen in some of the early intervention models as â&#x20AC;&#x153;ability tracking.â&#x20AC;? Tracking refers to the practice of putting all the academically low students in the same class. The scaffolding, used correctly, does not place students on permanent, segregated tracts. Collaborative programs rely on a minimum of three student assessments per year administered by the classroom teacher. These assessments are reviewed several times a year by a team of professionals. Students are assigned to reading levels, support, or intervention based on demonstrated need and, as progress is made, move through the system. On the other hand, researchers report that the average exit rate from special education to regular education is about 5% per year. Other criticisms revolve around the additional time, administrative leadership, and more complicated professional collaboration that the prevention and intervention model requires. It is true, collaborative early intervention will most likely require more staff time. The biggest time costs will be experienced during the first year or two of implementation as staff learn new skills and form grade-level and site teams. This time must be weighed against the additional time teachers, psychologists, specialists and district administrators currently spend assessing and serving students under the old model, keeping in mind that students often wait two to three years for service.

Literature Review

Finally, this collaborative model is personality and relationship dependent by nature. Collaborative interventions and scaffolding require teamwork, communication of detailed information, and systemic change. Fostering a new system of professional collaboration and intervention that produces academic growth is a considerable challenge for staff and students. We will address methods to handle these substantive, critical concerns in the implementation section of this report. 4.5 Conclusion In sum, the body of literature and current research provides strong support for the collaborative, early intervention and prevention model such as the ones used by Elk Grove and Hesperia. Multiple authorities point to the collaborative early intervention system focused on teaching all children to read by the end of 2nd grade.xxv In addition, research-based practices focused on reading acquisition in the early grades seem to yield the greatest, long-term gains in student performance. The President’s Commission on Excellence in Special Education and the National Research Council conclude, “…early screening followed by effective interventions in the classroom prevented many disabilities. Most impressive were the results of large-scale clinical trials indicating that early intervention of reading skills in conjunction with positive behavior programs resulted in improved academic achievement and reduction in behavioral difficulties of high-risk, predominantly minority children.”xxvi

Elk Groveâ&#x20AC;&#x2122;s Collaborative Model

5. ELK GROVEâ&#x20AC;&#x2122;S COLLABORATIVE MODEL

5.1 The Population Elk Grove Unified School District serves 49,970 students in 46 regular elementary, middle, and high schools. Not only has the student population in Elk Grove increased astronomically over the past ten years, the demographic shifts have also been dramatic. Once a moderately-sized, mostly white suburb of Sacramento, Elk Grove has received an influx of new residents, which has diversified the population (See Figure 5). The total student population of Elk Grove increased over 150% between the 1987/88 and 2001/02 school years. Significant population growth occurred among African American, Latino, and Asian students. The white population has continued to grow with the overall trend, but not at the same rate as the total population. Elk Grove is impacted with special populations (See Figure 6). Over a third of the students in the District receive free or reduced price meals and about twenty percent are not proficient in English. Elk Grove faces a challenge in addressing the instructional needs of English Language Learners because there are almost one hundred languages represented among 28

Elk Grove’s Collaborative Model

the student population. About fifteen percent of children aged 5 to 17 who live in the area served by the District are receiving CalWORKs welfare assistance. Elk Grove is ethnically and linguistically diverse. Figure 5 Ethnic Breakdown

Figure 6 Special Populations EGUSD Special Populations

EGUSD Now Ethnically Diverse 50% 40% 30% 20% 10% 0%

37% 19%

15.2%

CalWORKS

18%

17%

37.6%

er th O

hi W

As i

o ti n La

20.3%

ELL

ric

ic a

Meals

0.0%

10.0%

20.0%

30.0%

40.0%

50.0%

Source: California Department of Education, Educational Demographics Unit, CBEDS, The Language Census. www.ed-data.k12.ca.us Based on 2001/02 enrollment.

Like the national average, students with a specific learning disability account for just over half of all the students in Elk Grove who are identified as receiving special education services. Students with speech and language impairments are the second largest category of special education students. See Figure 7 for a breakdown of Elk Grove’s special education categories. Figure 7 SLD Accounts for Over Half of Special Education Population in EGUSD SLD

56.1%

SLI MR

20.9% 8.4%

6.0%

All Other

5.2%

AUT 0.0%

3.4% 20.0%

40.0%

60.0%

80.0%

100.0%

Source: CDE Reporting Cycle – December 2001. Based on 4,421 total special education population. http://data1.cde.ca.gov/dataquest/

Elk Grove’s Collaborative Model

5.2 Implementation In 1991 Marty Cavenaugh, the Director of Pupil Personnel Services and Special Education, determined that special education in Elk Grove had reached a crisis state. At that time, sixteen percent of their total District population was identified as having a disability. This large and growing population was significantly above the ten percent state-wide average for special education and proved costly for the District, as evidenced by special education’s large encroachment into the general fund. Cavenaugh approached the Superintendent at the time, Robert Trigg, and suggested the District devise a plan to address the crisis (See Figure 8). The Superintendent agreed and for one year, a group of nearly one hundred District stakeholders including general education and special education teachers, administrators, other personnel, and parents met to investigate the problem and plan a solution. The team focused on reforming the educational service delivery model for all students. Through the planning process, they identified a key problem: the state’s rigid, categorical funding model at the time did not allow for flexibility with respect to how personnel were utilized. Increasing the flexibility of funds, the team proposed, would allow the District to use the existing personnel more efficiently. Figure 8 – Elk Grove’s Implementation Process Director of Special Education initiated.

Group of 100 District stakeholders met for 1 year to devise plan.

Principals were given choice to implement Neverstreaming.

Five schools chose to pilot the program.

Elk Grove approached the State Board of Education and obtained a waiver from the funding model at the time, referred to as J50. The waiver allowed them to blend categorical funds together. In addition, the State agreed to base Elk Grove’s special education funding on ten percent of the District’s average daily attendance (ADA) rather than on the number of students identified as special education. This removed the monetary incentive for teachers or 30

Elk Grove’s Collaborative Model

administrators to refer students to special education and the disincentive to exit students from special education. The State also committed to fund the District at 10 percent of ADA even if they were successful in reducing the caseload below ten percent. Elk Grove operated under this waiver for three years, after which the State Legislature passed AB602. The new legislation essentially adopted this new funding model for the state and abandoned the old J50 funding model. Consequently, all districts in the state currently have the opportunity to blend categorical funds. In 1992/93, Elk Grove began the Neverstreaming

“We never issued a mandate. It was all by convincing and consensus. You can’t mandate a change of this nature.”

collaborative service delivery model created by the team of stakeholders in five Title 1 pilot schools.11 After the first year, the staff compared test scores from the Comprehensive Test of Basic Skills (CTBS) in the pilot schools and non-pilot schools. The Neverstreaming schools had higher test scores. This information was presented to the School Board and the principals. The site administrators were given the choice to implement Neverstreaming or not. There was no directive from the Superintendent or a specific time frame given for implementation at all sites. The District leadership felt strongly it could not mandate a change of this nature.12 Special Education Program Specialist, Terry DeBoer, reported; In an environment where principals live and die by test scores, it’s hard to ignore a program that improves test scores. The principals still had the option to implement or not, but Neverstreaming became a very obvious choice. Principals who held off had to explain why they were not going to do it.13 In the 1993/94 school year, all middle and high school and six more elementary principals chose

Elk Grove pilot schools included Prairie Elementary, Mack Elementary, Jackman Middle, Leinbach Elementary, and Reese Elementary schools. Note, through the various interviews and documents we received from Elk Grove, we identified conflicting information with respect to the number of pilot schools in the first three years of implementation. 12 Interview with current Bill Tollestrup, Director of Special Education for Elk Grove, 3/12/03. 13 Interview with Program Specialist, Terry DeBoer, 3/10/03. 31

Elk Groveâ&#x20AC;&#x2122;s Collaborative Model

to implement Neverstreaming. By 1997/98 all principals were convinced and every school in the District was implementing Neverstreaming to some degree. School sites that chose to implement Neverstreaming received support from the District in many forms. First, the District provided staff development and in-service training for both general and special educators. They offered training in assessment and collaboration. These trainings were often held in off contract hours. Through the Professional Development Department, the District offered direct incentives for teachers. Participating in training earned salary schedule credits at a fixed ratio, allowing staff to climb the Districtâ&#x20AC;&#x2122;s pay scale. Second, the District provided additional personnel in the beginning. In the elementary schools that piloted Neverstreaming, the District funded one full time Academic Coach to assist with reading and math interventions. In year two of implementation, the District provided one full time Literacy Coach to all middle and high schools (eight total). Third, the District provided funding for substitutes to release teams of teachers from one site to visit other Elk Grove schools that were farther along in implementation. This cross-school collaboration allowed teachers to gather best practices and better develop a plan for their own school. Finally, and many teachers involved would argue most importantly, the District and the School Board pledged not to reduce special education staff of Neverstreaming schools if they were successful in reducing special education caseloads. The District committed to not reduce special education staff if schools were successful in reducing special education caseloads using the Neverstreaming program. This proved to be crucial in adjusting the incentives teachers had to refer students to special education. Previously, a high caseload ensured a special education teacher her job. As the collaborative service delivery model in Elk Grove has evolved over the years, the District now bases special education staffing on a schoolâ&#x20AC;&#x2122;s average daily attendance

Elk Grove’s Collaborative Model

(ADA).14 By 1997/98, all schools in the District were implementing Neverstreaming in some form. It was difficult to assess the depth and quality of the Neverstreaming program at each site, which created an evaluation problem. The District developed a rubric for evaluating the implementation depth and quality at each school. Principals and a collaborative team completed the implementation rubric separately. This assessment process was used to identify weaknesses in the school’s program or implementation. The District and school site could then follow up with training or an adjustment in the program to address the weaknesses identified by the assessment. Early on in the District implementation, Elk Grove created a District-level collaborative team of key administrators for three main reasons (See Figure 9 for a list of team members). First, they realized that the District leadership had to have a consistent message and goals for Neverstreaming implementation. The current Director of Special Education, Bill Tollestrup, stated, “We had to keep the same target in sight.” Second, they also recognized that the leadership had to collaborate in the same way that they were encouraging school staffs to collaborate. Third, in order to affect change in the District and have enough power to continue the model, key players needed to be involved to address the challenges that developed. The team met once a month during the early years of implementation. Informal meetings among small groups of top leaders were also incredibly important to the implementation process. Often, ad hoc groups would meet to discuss problems, devise solutions, and adjust the programs at schools.

Figure 9 14

Interview with Bill Tollestrup, Director of Special Education in Elk Grove, 2/14/03. 33

Elk Grove’s Collaborative Model

Elk Grove District-Level Implementation Team Assistant Superintendent for Student Services Director of Special Education Director of Elementary and Secondary Curriculum and Reading Director of Elementary Education Director of Psychological Services Director of State and Federal Programs Director of Title 1 (at the time, it was a different position than the one above) Elk Grove’s Neverstreaming underwent several evaluations in the first years. The District was required to report to the State Board of Education for the first three years as a result of obtaining the funding model waiver. In 1994, Far West, an independent educational research firm now known as WestEd, developed an evaluation plan for Elk Grove to carry out. A California Department of Education Review Team also issued an informational report on the Neverstreaming program in the fall of 1995. In recent years, District-level administration recognized that teachers viewed Neverstreaming as a special education program. The word “neverstreaming” was meant to communicate that students would never be removed from mainstream education. However, the word “neverstreaming” reminded education professionals of the word “mainstreaming,” which is a term used exclusively in special education to describe placing special education students in specific general education classes for portions of the day. The collaborative service delivery model adopted by Elk Grove is a general education model for two main reasons. First, the model is designed to address the needs of all students, not just special education students. Second, all special education students are general education students first.xxvii Consequently, the District decided to change the name Neverstreaming to Collaborative Academic Support Teams (CAST) to remove the special education connotation from the title. From this point forward in the report, we will refer to Elk Grove’s service delivery model by its current name, CAST.

Elk Grove’s Collaborative Model

5.3 Key Components of CAST Elk Grove defines CAST as an intervention/prevention service delivery model that incorporates all educational resources available to serve at-risk students and their families. The goals of CAST are to provide a comprehensive, seamless educational model to prevent school failure and to provide supportive services to students and families exhibiting academic and/or socialemotional needs.15 The CAST model focuses exclusively on reading and literacy interventions. Through our interviews and site visits, we were able to identify the key components of the Collaborative Academic Support Team model, including ongoing diagnostic assessments, collaboration between general and special educators, professional development, learning centers, and regional teams. 5.3.1 Assessments Addressing students’ needs in a preventative manner requires knowing what their needs are. Assessments are used in Elk Grove early and often. General education teachers conduct a host of reading assessments with each student at the beginning of the year and periodically throughout the school year. This allows them to set a baseline and identify specific skill areas of weakness. The frequent reassessments give teachers the information needed to adjust intervention services as a student’s ability changes. Figure10 provides a list of “quick and dirty” assessments commonly used in Elk Grove. The assessments should not be used in isolation, as they only represent one or two aspects of literacy. This is why a battery of assessments is necessary in order to obtain a complete picture of a child’s reading ability.

Elk Grove’s Collaborative Model

Figure 10 – Primary, Intermediate, and Secondary Language Arts Assessments Name Type Buck County Assessment Provides a quick overview of student’s instructional reading level. The Names Test Assesses decoding skills. Spache Diagnostic Reading Scales Word analysis and phonics test. San Diego Quick Assessment Assesses student’s ability to read words in isolation. Slosson Oral Reading Test Assesses oral reading of words in isolation. Beginning/Advanced Phonics Skills Tests Informal test of high-utility, spelling-sound relationships for reading using single-syllable words for the BPST and multi-syllabic words for the APST. Reading Fluency and Grade Level Scales Assesses reading oral speed and accuracy. Reading Comprehension passages. Assesses comprehension of literal and inferential questions about a reading passage. Word Analysis Test Tests the student’s knowledge in short vowels, silent final e, consonant digraphs, consonant blends, vowel digraphs, dipthongs, r, l, w, controlled vowels, prefixes, suffixes, and syllabication. 5.3.2 Collaboration After each battery of assessments is administered, the school site Collaborative Academic Support Team meets with each general education teacher individually to discuss each student in a CAST Conference. The CAST Conference team is comprised of the classroom teacher, specialists, an administrator, a regional support team member, and other categorical staff (such as a language development teacher or a special educator). It is within this context that the collaborative team evaluates a student’s performance and designs an intervention program based on her needs. The format of these collaborative meetings varies from site to site. Some schools provide a roving substitute for two to three days during the meetings times. Other schools schedule the meetings over a series of days before and after school so as not to interrupt instructional time.

Presentation given by Terry DeBoer, Program Specialist in Elk Grove, 2/14/03. 36

Elk Grove’s Collaborative Model

5.3.3 Professional Development The Elk Grove Unified School District has an extensive Professional Development Department. It is essential to the implementation and ongoing evolution of CAST for the Professional Development Department to be involved as an active partner. The Department offers such extensive training and professional development that often teachers do not need to go outside of the District to obtain required credits in order to maintain their credential or advance up the pay scale. The Department is able to offer incentives to teachers to participate such as stipends or credits toward a pay raise. This Department focuses solely on staff development and hence, has the ability to research, network, and bring resources to the District in the form of workshops and experts. Through the Professional Development Department, teachers were trained in the necessary concepts and interventions underpinning the CAST model. 5.3.4 Learning Center A key component in Elk Grove’s CAST model is the Learning Center. Elk Grove’s Demonstration Program Application states, “Learning Centers provide a cost effective natural bridge to maintain and support students at risk.”16 In the original plan for Neverstreaming, the focus of the learning center model was providing the least restrictive environment for special education students. Students traditionally assigned to Special Day Classes, self-contained special education classrooms, are placed to general education classrooms and spend an appropriate portion of the day in the Learning Center. General education students are also served in the Learning Center based on the interventions designated by the CAST team consistent with the results of ongoing assessments.

Elk Grove’s Neverstreaming Demonstration Program Application, document obtained from Pasadena Special Education Department, 11/02. 37

Elk Grove’s Collaborative Model

Therefore, the Learning Center provides students at risk of school failure immediate access to intervention and prevention services.17 Based on regularly administered assessments, students can receiving instruction in the Learning Center the very next day. This shifts the focus of education from determining eligibility for special education, which may take years, to providing services as soon as students demonstrate the need for help. 5.3.5 Regional Teams While regionalization and regional teams are not a direct component of CAST, it is an essential piece of Elk Grove’s collaboration efforts. The District is divided into regions each composed of one high school, one middle school, and four to six elementary schools. The elementary and middle schools are “feeder” schools for the high school. The result of this regionalization is that this cluster of schools can serve an entire family throughout their children’s K-12 education. The Regional Team may be comprised of the staff from elementary, middle and high schools, Healthy Start, counseling services, and child welfare/attendance offices. These professionals collaborate to address the needs of a family consistently as a unit. Regional Teams address the needs of at-risk families and students by connecting them with preventative services. These teams allow a family to be linked with teachers and support staff who have established rapport with all of the siblings in the region’s schools. Through the Regional Teams, Elk Grove has established relationships with over forty local community service organizations addressing areas such as mental health and medical services. One example is a partnership between Elk Grove and the UC Davis Medical Center. Regional Teams are able to refer uninsured students to the Medical Center for preventative care, which in turn reduces UC Davis’ expenses.

Presentation given by Terry DeBoer, Program Specialist in Elk Grove, 2/14/03. 38

Elk Groveâ&#x20AC;&#x2122;s Collaborative Model

5.4 Reported Results Elk Grove Unified School District reports the following outcomes since the initial implementation of the collaborative service delivery model in 1992/93: 1) improved academic achievement, 2) increased average daily attendance, 3) fewer due process hearings, 4) fewer initial assessments for special education, 5) decreased special education caseloads, and 6) expenditures on special education are increasing at a minimal rate (not commensurate with population growth). 5.4.1 Academic Gains The District compared the CAT5 test scores of the first five pilot schools with similar non-pilot schools in 1992/93 and reported that the Neverstreaming schools performed better.18 Proponents used this information to convince the School Board, other principals, and the State Board of Education in the early years of implementation that the model was a success. They also compared CAT5 test scores in 1992/93 with SAT9 test scores in 2000/01 and noted an increase in national percentile rank in reading, language, and math for grades two through six.19 The District linked this increased academic success to the Neverstreaming program. 5.4.2 Increased Attendance The District also reports that in the last five years, on average, each student is attending school two days more than in previous years. Since schools in California are funded based on average daily attendance (ADA), this translates into increased revenue to the general fund. Bill Tollestrup, Director of Special Education, reported that the District receives approximately $26

18 19

Interview with Bill Tollestrup, Director of Special Education in Elk Grove, 3/12/03. Presentation given by Terry DeBoer, Program Specialist in Elk Grove, 2/14/03. 39

Elk Groveâ&#x20AC;&#x2122;s Collaborative Model

per student per day.20 Using total district enrollment in 2001 of 49,970, the increase in ADA earned the District approximately $1,299,220 during the calendar year. 5.4.3 Fewer Fair Hearings The Special Education Department reports that in the past ten years, they have had only one fair hearing with the family of a special education student.21 A fair hearing, or due process proceeding, occurs when the parents of a special education student file a complaint against the District with the State of California. If the complaint cannot be resolved and/or mediation is unsuccessful, a state hearing officer presides over a fair hearing. This represents a significant cost savings. In comparison, Pasadena had 12 due process hearings during the 2001/02 school year and has had 9 hearings to date during this school year. Each hearing costs the District approximately $2,500 per day or on average, $30,000 per hearing (4 to 6 days in length). If we assume that these two school years are typical for Pasadena, then the District is spending about $360,000 per year for due process proceedings. This estimate does not include additional costs incurred should the District lose the hearing, such as tuition to a special non-public school for the student. 5.4.4 Fewer Initial Evaluations Between 1996 and 1999, the number of initial psycho-education evaluations completed by school psychologists in order to determine eligibility for special education services decreased.22 The Elk Grove SELPA estimates that an initial assessment costs the District roughly $1,000. This cost includes such things as the time of the psychologist, classroom teacher and other District professionals, the testing materials and protocols, and substitutes if the Individualized

Interview with Bill Tollestrup, Director of Special Education Elk Grove, 3/18/03; Exact figures include $26.75 per student at a year-round school, $26.31 per student at traditional year school. 21 Presentation given by Terry DeBoer, Program Specialist in Elk Grove, 2/14/03. 22 Ibid. 40

Elk Groveâ&#x20AC;&#x2122;s Collaborative Model

Education Plan meeting is held during school time. Furthermore, Bill Tollestrup, Director of Special Education for Elk Grove, reports that of the initial assessments, only about 40-60% of the students qualify for special education.23 Figure 11 â&#x20AC;&#x201C; Number of Initial Assessments Declined in Elk Grove School Actual # of Total Estimated # of Difference Year Initial Enrollment Initial between # of Assessments Growth Assessments Initials in 96/97 Rate from based on and Estimated 96/9724 Growth Rate 96/97 1329 --1329 --97/98 976 6.38% 1414 438 98/99 539 5.69% 1494 955 Total Savings:

Estimated Cost Savings to District --$438,000 $955,000 $1,393,000

Source: Actual assessments reported by Bill Tollestrup. Population data obtained from CDE website, www.data1.cde.ca.gov/dataquest/, 3/14/03. Estimated assessments and cost savings based on analysis by authors.

Using the reported number of initial assessments in the 1996/97 school year as a baseline, we can estimate the cost savings over the two following years (See Figure 11). The total enrollment grew by 4,697 students from 1996/97 to 1998/99.25 Multiplying the number of initial assessments in 1996/97 by the total student population growth rate, we can estimate the amount of initial assessments that would have occurred using the reported number of assessments in 1996/97 as a baseline. Between 1997 and 1999, the District saved an estimated $1,393,000 in initial evaluation costs. This savings is translated into more efficient use of resources and time. For example, traditionally the time of a school psychologist is spent only on psycho-educational evaluations (both initials and triennials). Elk Grove asserts that by reducing the number of initial assessments, the CAST model allows the psychologist to spend time actually working with students in providing direct services. 5.4.5 Special Education Population

Interview with Bill Tollestrup, Director of Special Education, 3/12/03. EGUSD 1996/97 population = 37,787; 1997/98 population = 40,197; 1998/99 population = 42,484. 25 Source of population increase, CDE DataQuest, <www.data1.cde.ca.gov/dataquest/>, 3/12/03. 24

Elk Grove’s Collaborative Model

Within the first two years of Neverstreaming, the total district population increased by 3,357 students, but Elk Grove reduced their special education population almost in half, from 16% to just under 9%. As of the 2001 December count, Elk Grove’s special education population remains low at 8.85%. This reduction in the special education caseload had a significant fiscal impact for the District. Bill Tollestrup, Director of Special Education for Elk Grove, reported that the District spends approximately $3,900 more per student in the Resource Specialist Program, $7,000 more per student in a non-severely handicapped Special Day Class, and $16,000 more per severely handicapped student than what the District pays for a general education student. Between 1993/94 and 2001/02 the total district population increased 48% while the special education population increased only 41%. Tollestrup reported that over these years the severely handicapped population (especially students who are autistic, emotionally disturbed, and mentally retarded) has increased, while the non-severely handicapped population, including specific learning disabled students, has decreased. 5.4.6 Special Education Expenditures In October of 1995, a review team from the California Department of Education conducted onsite observations, a review of documents, and interviews to assess Neverstreaming and provide feedback to the District.26 The team reported the findings that the special education students as a percentage of the total population decreased and that the expenditures on special education programs increased at a minimal rate of 0.23% between 1994/95 and 1995/96. The report summarizes that while it cannot be conclusively determined that the collaborative model reduced the special education encroachment on the general fund, “Fiscally, it appears Neverstreaming is a successful program.”

Elk Grove’s Collaborative Model

5.5 Lessons Learned Teacher, school site administrators, and District-level administrators report that the collaborative educational service delivery model evolved and changed each year over the past ten years. With staff changes, student demographic changes, and population growth, the challenges school sites face vary from year to year. School site teams often adjust components of CAST depending on changes in site leadership, teacher turnover, and incoming student bodies. For example, Prairie Elementary currently does not scaffold students in each grade level. However, in past years, they have. Through our examination of CAST, we have identified lessons to be learned from Elk Grove’s experience in implementing and administering the model; 1) not using a top down approach worked for them, 2) the vision and commitment of the site administrator is key to success at the school level, 3) parent involvement has proven to be a factor in schools more successfully implementing CAST, 4) collaboration among teaching professionals is key and may be lacking in some Elk Grove schools, and 5) the experience in Elk Grove demonstrates the need for an evaluation plan. 5.5.1 Not a Top-Down Approach Stakeholders in Elk Grove feel strongly that this systemic shift of educational service delivery cannot be mandated. The interviewees reported that it was best that the program was not a Superintendent directive. This allowed for consensus building among the rank and file. School sites were presented with data and information that ultimately convinced them that the program was successful and worthwhile. Had it been a directive, the buy-in process may have been more difficult and fraught with more set backs.

“Principals are crucial to the implementation of CAST.”

Report provided to authors by Terry DeBoer, Program Specialist in Elk Grove, 3/11/03.

Elk Grove’s Collaborative Model

5.5.2 Principal is Key to Success Through our observations and interviews, we determined that the vision and leadership of the principal is a key factor in the success or failure of CAST implementation. One teacher reported, “Principals are crucial to the implementation of CAST. They set the tone for the purpose [of the program] for the whole school.”27 Throughout the past ten years, there have been a few principals who viewed CAST as a “back door” to special education. This interpretation of the concept is not consistent with the principles underpinning a collaborative service delivery model. The consequences have been that a site administrator may pressure special education staff to serve inappropriately high numbers of general education students in order to remove them from the general education classroom. This abuse of the CAST model has an impact on the quality of the program because it increases teacher to student ratios for interventions, undermines collaboration between teachers, and takes time away from serving identified special education students. Additionally, many of the “back-door” referral students exhibit behavioral or social problems, not academic problems requiring interventions. 5.5.3 Collaboration May be Lacking We did not observe consistent evidence of ongoing collaboration among staff in the two schools we visited. The collaboration consisted mainly of the CAST Conferences held three to four times during the school year and informal one on one consultation between teachers. There is no District mandated time for frequent collaboration among staff. The implications of this include a continued lack of understanding on the part of the general education teachers about the purpose of CAST and their role in working with special educators and a lack of coordination among grade level teams and between general and special educators. A few of the teachers we spoke to were frustrated with the disconnect that still exists between the general and special educators.

Elk Groveâ&#x20AC;&#x2122;s Collaborative Model

5.5.4 Parent Involvement Another component teachers and administrators identified as a key factor in success is parent involvement. Our interviewees reported that schools with high levels of parent involvement (which is correlated with income level) had the most successful Learning Centers and CAST implementation. One example given was Foulks Ranch Elementary school, which is located in a higher socio-economic status area. In general, Foulks Ranch Elementary was cited as having an extremely successful CAST program due to a dynamic site administrator, team of teachers committed to CAST, a great deal of general education teacher training, a stronger collaboration model between general and special educators, high parent involvement, and a stronger sense of teamwork.28 This provides evidence that implementation varies greatly from site to site. 5.5.5 Note Regarding Ventura Unified During our investigation of Elk Groveâ&#x20AC;&#x2122;s collaborative service delivery model, we obtained information from two special educators and one former psychologist in Ventura Unified School District, one of our control districts for Hesperia. In the past two years, individual teachers within schools have learned about the CAST model and have started using some of its principles with their students.29 Special education teachers at ten of the elementary schools are seeing atrisk, non-identified students for interventions on an ad-hoc basis.30 The teachers at these particular schools have instigated this change. Individual teachers who were excited about the model approached the District leadership for support in the hopes that like Elk Grove, the administration would take the lead in planning and implementing the 27

Interview with Elk Grove special education teacher, 2/14/03. Interview with Terry DeBoer, Program Specialist in Elk Grove, 2/14/03. 29 Interview with Kris Banning, teacher in Ventura Unified, 2/14/03. 28

Elk Groveâ&#x20AC;&#x2122;s Collaborative Model

model. The District leadership told them they could do what they want, but was not interested in implementing a District-wide program, nor did they commit to ensuring special education teachersâ&#x20AC;&#x2122; positions should the special education caseloads decrease. The teachers we spoke with were frustrated with the lack of support at the District-level. One of the teachers we spoke with will actually lose her position after this school year because her caseload decreased. The school site staff was avidly looking to fund her position in some other way, possibly with Title 1 money. For the purpose of our study, the implications of this information are two-fold. First, the experience of the teachers in Ventura further supports our finding that top District leadership is crucial to the overall success of implementation. Second, for the purposes of our quantitative analysis, a few of the Ventura schools serving as controls may be somewhat confounded with the treatment. We address the implications of this in the quantitative analysis section (See Section 8.2.1).

See Appendix D for a list of Ventura elementary schools reported to be serving at-risk students with special education resources. 46

Hesperiaâ&#x20AC;&#x2122;s Collaborative Model

6. HESPERIAâ&#x20AC;&#x2122;S COLLABORATIVE MODEL

6.1 The Population Hesperia Unified School District serves 15,683 students in sixteen elementary, middle and high schools. Hesperia is located in San Bernardino County off the 15 Freeway south of Barstow. They have experienced a student population growth of 65% since the 1987/88 school year. The white student population has been declining since the 1991/92 school year. The Latino and African American populations have increased roughly 240% since the 1987/88 school year. The student population in Hesperia is currently over fifty percent white and a third Latino. African American students and all other ethnicity categories account for less than 10% of the school population (See Figure 12).

Hesperiaâ&#x20AC;&#x2122;s Collaborative Model

Figure 12 Ethnic Breakdown

Figure 13 Special Populations Half of HUSD Students Receive Free or Reduced Price Meals

HUSD Majority White and Latino Student Population 54%

60%

CalWORKS

50%

11.4%

36%

40% 30%

Meals

50.1%

20% 10%

ELL

0% African American

Latino

White

Other

10%

20%

30%

40%

50%

60%

Source: CDE Educational Demographics Unit 1/25/03. Based on 2001/02 enrollment. http://data1.cde.ca.gov/dataquest/

Hesperia is impacted with special populations, especially students of low income or poverty level families. Fifty percent of Hesperiaâ&#x20AC;&#x2122;s students qualify for free or reduced price meals. In the areas of Hesperia served by the school district, about eleven percent of minors aged 5 to 17 are receiving CalWORKS welfare assistance. Less than ten percent of Hesperia students are considered English Language Learners (See Figure 13). Serving populations from low income or poverty level families as well as students who are not proficient in English poses instructional challenges to schools. Special education students in Hesperia are overwhelmingly identified as having a specific learning disability as compared to Elk Grove and Pasadena (See Figure 14). Over three-quarters of the special education population in Hesperia have specific learning disability as their eligibility label. The next largest group, students with speech and language impairment, accounts for only 11% of special education students.

Hesperia’s Collaborative Model

Figure 14 Over 80% of Special Education Students are Labeled SLD in HUSD SLD

81.8%

SLI

11.1%

All Other

2.7%

2.0%

AUT

0.3%

0.0%

20.0%

40.0%

60.0%

80.0%

100.0%

Source: CDE Reporting Cycle – December 2001. Based on 1,562 total special education population. http://data1.cde.ca.gov/dataquest/

6.2 Implementation The implementation process in Hesperia began with a thought process about serving at-risk students. In 1996, the Desert/Mountain Special Education Local Plan Area (SELPA) identified the types of students who were at risk for school failure. Among the characteristics they associated with being at-risk were low socio-economic status, a primary language other than English, limited English proficiency, and living in poverty. The increase in students with these characteristics in recent years in the classroom pose unprecedented challenges to the resources of general education. More and more, students who are doing poorly are referred to special education for services to address gaps in learning. The Director of Special Education in Hesperia, Dr. Jim Huckeba, listed three main reasons the typical special education model is not working: 1. It’s inefficient and expensive, 2. It’s fragmented, and 3. It’s overregulated.

Hesperia’s Collaborative Model

Dr. Huckeba quoted author Jim Grant in saying, “If you’re riding a horse and it dies, get off.” It was time to make a change.31 With this information, the Desert/Mountain SELPA Board of Directors presented a challenge to its member districts – “If you could start with a fresh sheet of paper and totally redesign special education, what would it look like?” Hesperia Unified School District took this challenge seriously and began investigating strategies that incorporated research based interventions and service delivery models with proven

“If you could start with a fresh sheet of paper and totally redesign special education, what would it look like?”

effectiveness. The Superintendent convened a team of District staff, teachers, parents, and Board members for a six-month research and planning period. After attending a conference at Johns Hopkins University to learn about Success For All, holding focus groups of administrators and teachers, consulting with researchers at UC Riverside, and observing Elk Grove’s Neverstreaming intervention model, Hesperia coined their own hybrid intervention model – ExCEL, Excellence: A Commitment to Every Learner (See Figure 15). Hesperia describes ExCEL as a “framework of strategies that build upon the philosophy that all children can learn.”32 ExCEL focuses on using strategies to address reading deficits, behavior problems, and lack of parental involvement – all factors in the reasons children fail in school. ExCEL is meant to address the needs of at-risk children in such a way that produces improved academic progress and increased educational opportunities for all students.

31 32

Interview with Dr. Jim Huckeba, 1/25/03. Email from Desert/Mountain SELPA to other California State SELPA directors 9/11/01. 50

Hesperia’s Collaborative Model

Figure 15 – Hesperia’s Implementation Process for ExCEL SELPA Board challenged Districts to recreate special education.

1996

Group of District administrators, principals, and teachers researched best practices.

1st 6 months

HUSD Superintendent gave directive to all principals.

Next 6 months

Schools given site-specific flexibility to implement principals of ExCEL.

1997/98 School Year

The District introduced this new model with a top-down approach. The Superintendent gave a directive to the principals in Hesperia to implement the concepts of the model at their schools. Each school was able to send a team of teachers and the principal to a four-day training on the concepts and details of the intervention model components offered by CalSTAT. After the four days of training, each site drafted an implementation plan and set goals specific to its school. While the impetus for change was a directive from the highest level of administration, the Superintendent, the schools were given site specific flexibility in pacing, scope, and details of implementing the concepts and principles of the collaborative service delivery model. The teachers’ union was included in this process as well. Prior to issuing the directive, the Superintendent met with the union president and “sold her the program” with information showing the academic achievement results in other districts such as Elk Grove. The concept and program shift was largely accepted by the union president because the District’s test scores were so low. The Superintendent and the union president then traveled to each school together to educate the principals and staff of the model. This sent a powerful message of support from both top administration and union leadership.33 While some individual teachers (mostly older veterans) were initially skeptical and resistant, there was no formal union opposition. The District did not put the collaborative model to a vote within the union or at individual school sites. Three years into the implementation, the District and the union adopted a memorandum of

Interview with Juniper Elementary School principal, 3/14/03. 51

Hesperia’s Collaborative Model

agreement regarding the Collaborative Wednesday banked time. The School Board was involved in the planning and implementation phases from the beginning. One principal described the Board members as being generally trustful of the Superintendent and said they had a good working relationship.34 The Superintendent and Director of Special Education made a presentation to the Board early on in the process. Board members were informed about why a change was needed and the goals of the program. The issues that received the most attention from the board were special education’s increasing encroachment on the general fund and the increasing specific learning disabled population. Members were invited to planning meetings and a few participated in the initial study team who visited Elk Grove to gather information.35 The School Board together with the Superintendent pledged to not reduce special education teaching staffs in the event that special education caseloads were reduced. With the directive to implement the model came a promise that if ExCEL was successful in reducing the number of students identified as special education, the District would not reduce special education staff members. As in Elk Grove, this proved to be a key component of the “buy-in” process. Without this commitment, there would have been a disincentive to implement the model – special educators could have worked themselves out of a job. Now special education staffing is based on the size of the school Since schools had site-specific flexibility, ExCEL looks slightly different in each school Superintendent advised, “Fake it till you feel it.”

34 35

and the timeline for implementation varied across the District. Some individual schools started using the concepts and model at their entire site at the start of the next school year. Some schools started

Interview with Maple Elementary School principal, 1/25/03. Interview with School Board Member (also a parent of six students in District), 1/25/03. 52

Hesperia’s Collaborative Model

on a smaller scale and only implemented it among the most interested teachers, or among only one grade level. Some schools that had recently experienced a change in administration or high teacher turnover were uncomfortable with the directive and took a significant period of time to bring key people together and plan thoroughly before changing their educational service delivery system. The schools were not forced to implement on a District timeline; they were provided additional training and support in their implementation efforts. However, it was made clear by top administration that schools were expected to be moving forward and implementing components of the program. The Superintendent advised struggling principals and skeptical teachers to “Fake it till you feel it.”36 All schools were implementing ExCEL in some form within three years of the directive. Cottonwood Elementary was among the first to fully implement in 1997/98. In an effort to assist schools in various stages of the implementation process, the District hosted a series of monthly meetings for principals and key teaching staff from their schools. Schools that were

“Cross-school collaboration is very important.”

successfully implementing all or some of the program elements gave presentations about their successes and lessons learned. While the meetings were not mandatory, one principal described it as “something you didn’t miss. You were expected to be there.”37 These peer led training sessions offered a powerful incentive and functioned as positive peer pressure. Teams at schools who were ahead in implementation offered their help to other schools within the District. This cross-school collaboration continues regularly today, five years after initial implementation. School teams continue to schedule visits to other schools to see first hand how ExCEL has evolved and been improved at different sites. This cross-school

36 37

Interview with Dr. Jim Huckeba, Director of Special Education in Hesperia, 1/25/03. Interview with Maple Elementary School principal, 1/25/03. 53

Hesperiaâ&#x20AC;&#x2122;s Collaborative Model

collaboration has been a crucial component of the ongoing improvement process of the model at each site.38 6.3 Key Components of ExCEL Hesperia feels that ExCEL summarizes the belief that â&#x20AC;&#x153;all children can learn and we can ensure the success of every child.â&#x20AC;?39 Essential components of ExCEL include 1) special and general educators working together in classrooms to provide supports for all learners, 2) a place for intense small group or one on one interventions to occur, 3) a means to connect with families in a proactive and positive way, and 4) time for general and special educators to meet and consider the learning needs to each student individually. Through our site visits, observations of ExCEL in action, and interviews, we identified several program elements that Hesperia uses to support these essential components. These program elements include ongoing diagnostic assessments, collaboration among staff, ongoing professional development, individualized learning plans for every student, research based intervention strategies targeted at students based on identified need, the learning center model, and scaffolding.40 6.3.1 Assessments Like in Elk Grove, a battery of diagnostic assessments is completed for each child four times a year by her general education classroom teacher. Teachers are given the opportunity to request a half-day substitute at the beginning of the school year and during each trimester while they assess their students individually. Assessments at Cottonwood include the following:41

Interview with Juniper Elementary School principal, 3/14/03 Email from Desert/Mountain SELPA to all SELPA directors, 9/11/01. 40 Information presented at site visit, 1/25/03. 41 The staff at Maple did not provide us with a list of specific assessments used. Each site can choose their own battery of assessments. Many of them are the same. 39

Hesperiaâ&#x20AC;&#x2122;s Collaborative Model

Figure 16 â&#x20AC;&#x201C; Assessments Used by Cottonwood Elementary Primary Upper Grades Concepts of Print Basic Phonics Skills Phonemic Awareness San Diego Quick Basic Phonics Skills Accuracy and Fluency Accuracy and Fluency Spelling Inventory Spelling Inventory Math Assessment Math Assessment Writing Prompt Writing Prompt Source: Presentation given at Cottonwood site visit, 1/25/03.

These assessments are used to evaluate the prevention and early intervention strategies each student needs to be successful in school. 6.3.2 Collaboration Collaboration among staff is a critical component of ExCEL. Principals and teaching staffs commit a great deal of time to ongoing meaningful collaboration within and across grade levels and between special and general educators. There are three types of collaboration meetings built into the District and school schedules (See Figure 17). In the beginning of the year, The ExCEL team leads, usually the special educators at the site and a site administrator, collaborate with each teacher to discuss the students in his/her class in a co-op meeting. The co-op meeting is similar to the CAST Conference used in Elk Grove. The teacher presents findings from the most recent battery of assessments and the team discusses and agrees upon a level of service needed by the child. They determine if specific interventions are required, if the student is meeting grade level benchmarks, or if the student requires enrichment opportunities for acceleration. These meetings occur over a three-day period in which roving substitutes are provided for the teachers as they meet with the team. It takes about 30 minutes for a primary teacher to discuss each child and about 45 minutes for the upper grade teachers. At the first co-op of the year, the team develops an individualized learning plan (ILP) for each student, and updates the ILP at subsequent co-op meetings.

Hesperia’s Collaborative Model

Grade level teams also have collaborative meetings three times a year following the coop meetings. This provides an opportunity for the staff to address teacher assignment for scaffolding, develop trimester team goals and grade level goals. Figure 17 – District and Schools Build in Time to Collaborate in Hesperia Type of Meeting Description -At beginning of year and once each trimester. CO-OP Meeting -Over a period of three days. -Teachers discuss each student’s assessment results with ExCEL team leads. -ILP developed or updated. -At beginning of year and once each trimester. Grade Level Team Meeting -Teachers assignment to ability level classes. -Set grade level team goals. -Set trimester goals. -Every Wednesday during their banked time, approximately Collaborative Wednesdays 60-90 minutes. -Teachers collaborate on strategies, individual student cases, professional development. -Grade level teams complete week’s lesson plans together. Source: Presentation given by Vice Principal at Cottonwood Elementary, 1/25/03, and phone interview with Juniper Elementary principal, 3/12/03.

In addition to the co-op and grade level meetings that occur each trimester, teachers meet once a week on Collaborative Wednesdays. In Hesperia, Wednesday is a “banked day.” This means that during the other days of the week, the school day is extended a few minutes and on Wednesdays, the students leave early. Banked days provide a natural way to build in collaborative or professional development opportunities for teachers without extending beyond their contract time. The District and the teacher’s union have a memorandum of understanding regarding the contract change caused by banked days. The banked days are used for teachers within and across grade levels and for special and general educators to collaborate about individual students’ needs. Teachers can share strategies and best practices as they continually improve their instruction.

Hesperiaâ&#x20AC;&#x2122;s Collaborative Model

6.3.3 Professional Development Teachers received training in current, research-based curricular and instructional strategies from the RESULTS program from California Reading Professional Development Institute. They also received training on effective collaboration practices, early prevention/intervention strategies, and school and classroom behavior management strategies. Much of the professional development was provided by the SELPA and other state resources, such as CalSTAT.42 In cases where the District need to pay for additional training, they used funds already budgeted for professional development at the District and school level were used. 6.3.4 Individualized Learning Plans An individualized learning plan (ILP) is developed for every student and updated each trimester in the co-op meeting based on assessments. This document details the amount and level of intervention that the student needs. Parents receive a letter explaining the services that their student will receive based on the ILP. The ILP is not a rigid document. The process of revisiting the ILP at least three times a year allows for adjustments to be made as a student progresses or if it becomes clear that he needs additional interventions. 6.3.5 Research-Based Intervention Strategies Teachers in Hesperia use several intervention programs to supplement the District-adopted curricula. Staff, teachers and assistants, are trained in each program they use with a group of students. These programs are intended to meet studentsâ&#x20AC;&#x2122; instructional needs that may not be met by the District-adopted curricula. They provide an opportunity for more intense instruction in specific skill areas where a student may be lacking, such as fluency or decoding. In addition, the interventions may use an alternative approach, such as multi-sensory phonemic awareness

California Services for Technical Assistance and Training, a project of the Special Education Division of the CDE. 57

Hesperia’s Collaborative Model

instruction, rather than phonics drills. These alternative instructional approaches may work better for students who have different learning styles and strengths. The intervention programs that may be used include the following: Figure 18 – Intervention Programs Used in Hesperia Reading Math Read Naturally Touch Math Mountain Language Mountain Math Accelerated Reading Accelerated Math Earobics Excel Math (not the same as ExCEL) SRA Corrective Reading Soundabet Zoo Phonics Thinking Maps Reading Recovery Source: Presentation given at Cottonwood site visit, 1/25/03.

6.3.6 Learning Center Like in Elk Grove, the learning center concept is a key component to the ExCEL model. The Coyote Learning Center at Cottonwood Elementary is housed in three full size classrooms with the dividing folding walls between them pushed back to make one giant room. The large open area is dotted with kidney tables where small groups of six or fewer students are working with an adult for an intense short period of time within the school’s reading or math instructional block. The Resource Specialist Teacher, Special Day Class teacher, and their special education instructional assistants work out of this room. Other trained adults, such as parent or community volunteers, sometimes work with small groups of students in the Learning Center as well. The students who come to the Learning Center require the most intense level of remediation in basic skills. These students benefit from the smallest student to teacher ratio, no more than six students per adult. Based on the assessments completed by their classroom teachers during each trimester and the results of the co-op meeting, the student’s ILP designates if a student needs to come to the Learning Center during the school-wide math and/or reading

Hesperiaâ&#x20AC;&#x2122;s Collaborative Model

block. The math block is one hour long and the reading block lasts one hour and ten minutes for grades 1-3 and one hour and fifteen minutes for grades 4-6. The Learning Center is also used outside of the block time for students who need additional short term, intensive interventions throughout the day in reading, written language, math, and behavior and social/emotional needs. The Learning Center is not just for students identified as special education. Any student, regardless of a label, who requires this level of support to meet the California State Standards can be served in the Learning Center. At Maple Elementary the learning center is called the Enrichment Center. It is located in what used to be the Resource Specialist room and the Special Day Class room with the folding wall between them pushed back. In addition to using the Enrichment Center as Cottonwood does for the reading and math blocks, the special education teachers have designed support targeted specifically at the upper grades at Maple. Grades four and five are grossly overcrowded, often accommodating 40 students per class, and have fewer teachers. At Maple, the Enrichment Center is used in the afternoon to offer social studies and science blocks to upper grades. This gives the intermediate level classroom teacher the opportunity to have a reduced class size for a period as well as provides more remedial instruction to those students who need it. Throughout the day, the Enrichment Center serves 120 non-identified students through the scaffolding system. Not every school site had adequate space to house a learning center. At Juniper Elementary, for example, the site is over forty years old and a large central room for a learning center was not available. This limitation forced the school site staff to creatively design the ExCEL model to meet their needs at the site. Special educators, assistants, and other adults are brought within the classrooms during the reading and math blocks to reduce ratios and provide

Hesperiaâ&#x20AC;&#x2122;s Collaborative Model

interventions.43 This model requires heightened logistical choreography, but it can be done successfully. 6.3.7 Scaffolding Another critical component to the collaborative service delivery model in Hesperia is the scaffolding provided to students within a grade level. Scaffolds are types of support provided by education professionals to assist students in bridging the gap between their current abilities and the intended goal.xxviii The scaffolding component of ExCEL is a way of organizing students for more targeted instruction. Organizing students by ability level for the block periods of time during the day provides the opportunity for more effective differentiated instruction than a teacher would be able to achieve by his/herself within his/her own classroom. The students requiring the most support benefit from the smallest student to teacher ratio. As ability level increases, so

In construction, a scaffold supports the workers â&#x20AC;&#x201C; temporarily providing the structure they need to complete their work.

does the ratio. An example of a typical primary grade level scaffolding organization is shown on the next page in Figure 19. For the reading block, the students are spread among the teachers at a particular grade level. The teacher assignments are determined at the grade level collaborative meeting each trimester. Teachers do not necessarily teach the same level all year and students can move among the levels as they make progress. The ratio is further decreased by additional instructional aides or teachers in the Learning Center/Enrichment Center or classroom.

Interview with Juniper Elementary School principal, 3/14/03. 60

Hesperia’s Collaborative Model

Figure 19 – Example Scaffolding Plan for Primary Grade Reading Block – Cottonwood Teacher Number of Students Served Student Reading Level Special Education Teachers 12 Lowest group of students (Learning Center) and Aides Teacher A and an Aide 12 Preemergent-Emergent Teacher B 15 Emergent Teacher C 15 Early-Early Fluent Teacher D 16 Early Fluent Teacher E 28 Fluent Source: Presentation given at Cottonwood site visit, 1/25/03.

6.4 Reported Results Hesperia Unified School District reports the following outcomes since the implementation of ExCEL District-wide in 1997/98: 1) improved academic achievement, 2) decreased special education caseloads leading to decreased expenses (not in personnel, but in time), 3) Improved staff and student morale, 4) improved attendance for teachers and students (increase in ADA), and 5) a decrease in behavior problems. 6.4.1 Academic Gains Hesperia reports gains in the Academic Performance Index (API) and their SAT9 test scores. The Maple Elementary principal reports that greater gains have been made in the primary grades than in the upper elementary grades. She suspects this may be because the first through third grades have smaller class sizes (20), which coupled with the ExCEL program, may produce the best results.44 The Juniper Elementary principal reported a 200 point gain on the API in the past three years.45 6.4.2 Special Education Population Hesperia also reports a 56% reduction in students typically identified as special education and in need of the Resource Specialist Program (RSP) since they began implementing ExCEL in 1997/98. Special education students served through the Resource Specialist Program are those 44

Interview with Maple Elementary principal, 1/25/03. 61

Hesperiaâ&#x20AC;&#x2122;s Collaborative Model

with specific learning disabilities and other mild handicaps. Hesperia spends approximately $3,307 more per RSP student than a typical general education student. Resource Specialist Programs typically overwhelmingly serve students with specific learning disability. As with the estimate of cost savings in Elk Grove, one should note that any cost savings associated with a reduction in special education caseloads may not be an exact translation into dollar cost savings, but actually reflective of a more efficient reallocation of resources that serve more students. Individual schools also report a shift of students to a less restrictive environment. For example, in the past 3 Â˝ years at Maple Elementary, the enrollment in the Special Day Class has dropped from 27 to 11 and six of those SDC students have moved from the Special Day Class to the Resource Specialist Program. 6.4.3 Fewer Fair Hearings Special Education Director, Dr. Jim Huckeba, also reports a decrease in due process hearings and mediations with parents of students with special needs. The few legal proceedings that the District has faced have been with families of students with severe disabilities, not those with mild to moderate disabilities, who are most served by the ExCEL model. As stated in the Elk Grove section of reported results, this represents a significant cost savings when compared to Pasadenaâ&#x20AC;&#x2122;s estimated current average annual cost for hearings, $360,000. 6.4.4 Other Measures of Success Interviewees in Hesperia also spoke to an increase in average daily attendance for students, increased teacher attendance, increased teacher and student morale, and a decrease in disciplinary referrals.

Interview with Juniper Elementary principal, 3/14/03. 62

Hesperia’s Collaborative Model

6.5 Lessons Learned Hesperia staff readily admitted that “ExCEL is not a cure all, a panacea, it does not serve every need.”46 One principal believes that the greatest marginal benefit from the collaborative model is gained by those students who are on the borderline, at the 30th to 40th percentile. ExCEL succeeds at targeting gaps and helps bring them up to speed. The program allows for greater differentiated instruction to these students in need. ExCEL is not about throwing out best instructional practices or existing intervention strategies; it simply makes adjustments to the delivery of those services. Through our interviews we identified five key variables that contributed to the success or lack of success at each site; 1) top-down approach to implementation, 2) principal willingness to lead by example, 3) teacher/staff willingness to trust colleagues with their students, 4) logistical flexibility at the school site, and 5) implications for Special Day Classes. 6.5.1 Yes to a Top-Down Approach Our interviewees in Hesperia stressed the importance of having the Superintendent and School Board support the change effort. Principals, teachers, parents, and students need to feel that the District is behind the effort. This also assisted principals when dealing with reluctant teachers. The teachers had less of a reason to be upset with their site administrator because they realized the ExCEL model was a directive from top administration.47 6.5.2 Principal Key to Success Every District administrator, principal and teacher we interviewed in both Hesperia and Elk Grove stated that the site administrator’s buyin and leadership were crucial components to successful

46 47

“The leader has to work harder, go overboard, be visible in change efforts.”

Interview with Maple Elementary principal, 1/25/03. Interview with Juniper Elementary principal, 3/14/03. 63

Hesperia’s Collaborative Model

implementation of a collaborative service delivery model. Dr. Huckeba acknowledged that ExCEL requires a great deal of commitment and ongoing hard work on the part of the teachers. In many ways the frequent assessments and collaboration with other professionals take more time and effort than when a teacher used to be able to go into his/her room, close the door, and teach largely in a vacuum. The principal needs to support the efforts of the staff to truly work collaboratively, while being respectful of valid concerns teachers may have. Figure 20 – Principal Goes Above and Beyond When the California State Standards were implemented, there was a shift in the focus of lesson planning. This principal began requiring that teachers cite the standard(s) being addressed in each lesson in their plan book for each day, each week. The principal let the teachers know that she would read everyone’s plan book once a week to verify that they were doing this. Having to check each teacher’s plan book each week was an additional requirement for the principal, just as asking them to document standards addressed added to their workload. Every Tuesday, the principal came to school at 4:30 a.m. with a flashlight. She visited each teacher’s classroom, read their plan book, and left individualized notes for them. This sent the message that she was serious, committed, and willing to go the same extra mile that she was asking of them. 6.5.3 Collaboration and Mutual Trust Part of the teacher buy-in that is required for successful implementation of any new educational delivery model is mutual trust. The scaffolding piece of ExCEL requires that grade level teachers have immense trust in each other. If students are to be distributed among the grade level teachers by ability, the grade level team must be a cohesive group that trusts one another and communicates with one another. Maple Elementary just began scaffolding in their upper grades this year. Previously there were two teachers on the grade level team who were perceived as weak and ineffective. Student test scores and general student success are attributed to the student’s homeroom classroom teacher. If a teacher does not trust his colleagues, he will not be willing to give up significant amounts of instructional time with his students. Another example of how this trust is crucial occurred at Maple this year at report card

Hesperia’s Collaborative Model

time. Teachers discovered they could not complete their students’ report cards alone. They needed to collaborate with their grade level team members to accurately report student progress in all subjects. Another example of teachers building trust is how Juniper’s teachers often use this time together to complete lesson plans for the week collaboratively. Since scaffolding requires teachers to share students, lesson plans must be consistent. 6.5.4 School Site Flexibility School site flexibility is also a key factor in the success of implementing this program. Providing flexibility to implement the concepts of ExCEL not only encourages site staff buy-in but recognizes that the assets and challenges at each site are different. Cottonwood Elementary is unique in that they were able to dedicate three full class size rooms to their Learning Center. Other campuses are strapped for space. At Juniper Elementary School, the scaffolding and reduction of student to teacher ratio is achieved by sending professionals into the general education classrooms, rather than pulling the students out to a Learning Center. 6.5.5 Implications for Current Special Education Placements One problem identified with the transition to ExCEL involved the status of non-severe Special Day Classes (SDC) at some elementary schools. With the implementation of ExCEL, some schools chose to disband their SDC(s). These students were returned to general education homerooms and served in the Learning Center as needed like all other students. However, problems arose when the staff realized that a number of students who had been placed in these SDCs still required a small class setting all day long. They were not succeeding in the general education classroom even with the supports offered by the Learning Center. Schools like Maple chose to keep their SDC and use it as homeroom for those students who need a smaller class setting all day for academic or behavioral reasons.

Model Program Comparison

7. MODEL PROGRAM COMPARISON

While Elk Grove pioneered the collaborative service delivery model in California and Hesperia borrowed heavily from their program components, the two models look different in many ways (See Figure 21 for comparison table). Elk Grove’s evolution is marked with an innovative beginning, since they designed the model from scratch, and followed with adjustments and improvements implemented year after year. Hesperia had the advantage of investigating several existing programs, the current research, and consulting with education professionals and scholars. Hesperia has been able to apply the pieces of Elk Grove’s model blended with the scaffolding component of Success for All and create a unique program. 7.1 Top-Down or Bottom-Up? One stark difference between the implementation of the collaborative models in Elk Grove and Hesperia is the Superintendents’ roles. The District administrators we interviewed in Elk Grove were emphatic that this type of systemic change cannot be mandated. Elk Grove staff felt strongly that consensus needs to be built among the rank and file. Principals needed to be 66

Model Program Comparison

persuaded and make their own decisions about implementation. This is reflected in their implementation because principals were able to choose whether or not to adopt Neverstreaming. Conversely, principals and District administrators we interviewed in Hesperia cited the importance of the Superintendentâ&#x20AC;&#x2122;s directive in their implementation efforts. School site staffs saw the directive as a firm commitment to the model from top administration. When skeptical teachers did not want to implement the program, principals were able to point to the Superintendent and say, â&#x20AC;&#x153;We have to do it. Fake it till you feel it.â&#x20AC;?48 This directive was softened by the flexibility awarded to schools to implement in their own way at their own pace. 7.2 District Commitment Despite the difference in top-down versus bottom-up approaches to implementation, in both Districts the Superintendent and the School Board committed to not decrease special education positions if caseloads dropped. Prior to the implementation of the collaborative service delivery model, the Districts funded special educator positions based on the caseload of identified students. If a caseload decreased below a designated number, a special education teacher would have to split his time between more than one school, or could lose his job completely. We cannot overstate the importance of this commitment in adjusting the incentives for the collaborative service delivery model. The purpose of the collaborative model is to serve atrisk students early based on need, rather than waiting to assess eligibility. To maintain a staffing policy based on caseload while trying to implement a collaborative model would be counter productive. 7.3 Program Elements The program elements in both Districts are largely similar. However, there are two main differences that are worth pointing out. Elk Grove does not use scaffolding extensively nor do

Model Program Comparison

they have weekly collaborative meetings among the teachers. The Resource Specialist Teachers at the schools we visited in Elk Grove seemed overwhelmed with serving the needs of the lowest general education students in addition to the RSP students. The grade levels in the schools we visited did not scaffold the remaining students into ability levels to offer further differentiated instruction. There seemed to be a disconnect between what was going on in the general education classroom and the Learning Center. There also seemed to be a lack of regular and ongoing collaboration among the staff at individual schools. Time to do so is not built into their schedules. The fact that Hesperia explicitly includes these two components into ExCEL ensures that the staff at individual schools are working together as a cohesive unit to serve the needs of all learners. 7.4 Principal Key Factor Teachers in both Districts reported that the site administrator has a great impact on the success or failure of the program. In Hesperia we were able to see two schools where the principals were both visionary leaders who were committed to the philosophy of ExCEL. Consequently, the model truly looked ideal at their school sites. In Elk Grove, teachers shared with us that there is a negative impact on teacher morale when principals use CAST as a â&#x20AC;&#x153;back-doorâ&#x20AC;? to special education services without respect to the integrity and process of the CAST model. 7.5 Measuring Outcomes Finally, both Districts use test scores, API gains, average daily attendance, special education referrals, initial evaluations, special education caseloads, and teacher satisfaction as measures of success in evaluating the models. Although both districts collect data and report trends, neither has commissioned a statistically and methodologically valid evaluation.

Interview with Juniper Elementary principal, 3/14/03. 68

Model Program Comparison

7.6 Conclusion In summary, our investigation has led us to the following broad conclusions: 1. The top-down versus bottom-up approach depends on the political and organizational culture of an individual district. 2. A District commitment to maintain special education teacher positions despite reductions in caseloads is crucial. 3. Two components that Hesperia institutes District-wide are key for fostering meaningful collaboration, Collaborative Wednesdays and scaffolding. 4. The site administrator is a key factor in the success of implementation at the school level. 5. In order to assess the true impact of any new service delivery model, a district should develop an evaluation plan prior to implementation. 6. School site flexibility in implementation is crucial in that each school needs to be able to adjust their service delivery in their own way and at their own pace to ensure teacher buyin. 7. Many district stakeholders need to be involved in the process of systemic change.

Model Program Comparison

Figure 21 – Model Program Comparison

Impetus for Change

How Started Approach Implementation

Key Model Components

Current Situation

Evaluation

Elk Grove

Hesperia

16% of student population identified as special education, Special Education Director felt this was a crisis to be addressed.

-Low test scores throughout District. -Increasing ELL and SLD population. -Challenge from SELPA Board of Directors.

Initiated by Director of Special Education, gathered group of 100 stakeholders to meet for 1 year. Started from scratch, totally redesigned delivery system.

District team of stakeholders (both general and special education) convened to investigate options. Gathered information, consulted professionals, looked at research, visited other programs. -Implemented in 92/93 in 5 -Superintendent gave directive in District selected pilot schools. 1997 to all principals, provided -Six more elementary schools training for school teams. began implementing in 93/94, -Schools given flexibility to principals given choice to adopt or implement on own time frame and in not. details of program. -All schools implementing in some -All schools had implemented in form by 97/98. some form within 3 years. Assessments early and often, Assessments early and often, addressing students’ individual addressing students’ individual needs needs, Learning Center, (ILP), Learning Center, scaffolding, co-op collaborative meetings co-op collaborative meetings 4x/yr, 3x/yr, regional family services grade level collaborative meetings, coordination. Collaborative Wednesday meetings 1x/week for teachers, parent education. Model implementation varies from Model implementation varies from school to school. Success and school to school, dependent on teacher satisfaction largely principal leadership. Attitude of dependent on site administrator. commitment to model with an Some teachers frustrated. understanding of its limitations. District submitted reports, including test scores and other Internal examination of test scores, outcomes data, to State Board of special education caseloads, Education for first 3 years of attendance, and disciplinary referrals. program to justify funding model waiver.

Quantitative Analysis

8. QUANTITATIVE ANALYSIS

8.1 District Descriptions No two districts in California are exactly alike, but the districts we examined are similar in some key respects (See Figure 22). Pasadena is demographically unique in a number of ways. First, Pasadena has a mix of ethnicities that is not comparable to any of the other districts included in our study. Pasadena boasts an unusually high population of African American students and a significantly low proportion of white students.49 Also, the Latino population in Pasadena is higher than in any of the other districts. Elk Grove is demographically unique in that is has large proportions of each major ethnic group, rather than a clear majority of one ethnicity. Over nineteen percent of the student population is Asian, which is included in the â&#x20AC;&#x153;otherâ&#x20AC;? category. Hesperia, Ventura, and Redlands have similar demographic breakdowns with small percentages of African Americans, about a third Latino, and majority white students. Pasadena is particularly challenged with respect to students who are not proficient in

Los Angeles County reports 11% African American and 18.2% White students for the 2001/02 school year. The state reports 8.3% African American and 34.8% White students for the same school year. 71

Quantitative Analysis

English and students receiving free or reduced price meals. Since Latino ethnicity and English Language Learners are highly correlated (0.67), it is not surprising that given Pasadenaâ&#x20AC;&#x2122;s high Latino population, there would also be a high ELL population. English Language Learners account for about a fourth of Pasadena students. Elk Grove has a similar proportion of English Language Learners, who account for about a fifth of their student population. This is not surprising considering that over 100 languages are represented among the families in their District. Over 60% of Pasadenaâ&#x20AC;&#x2122;s students qualify for free or reduced price meals. Hesperia has the next largest free and reduced price meal population with half of their students receiving the meal program. Of all the districts included in the study, Pasadena has the lowest percentage of teachers with full credentials. A review of Pasadenaâ&#x20AC;&#x2122;s student demographic variables reveals that the District has higher proportions of students with characteristics traditionally associated with being at risk for school failure. These include poverty, limited English language proficiency, and minority status. The implication is that there is significant room for academic growth in Pasadena. A collaborative service delivery model may have a greater marginal impact on students at risk of failure.

Quantitative Analysis

Figure 22 â&#x20AC;&#x201C; Demographic Variables for Each District 2001/02 School Year Non-Model Districts Model Districts Pasadena Redlands Ventura Hesperia Elk Grove 23,447 19,892 17,632 15,683 49,970 Enrollment Ethnicity 28.6 7.8 3.5 6.2 19.2 % African American 52.2 32.5 35.9 35.5 18.3 % Latino/Hispanic 15.4 46.2 54 54.1 36.6 % White/Non-Hispanic 3.8 13.5 6.6 4.2 25.9 % Other Special Populations 26.6 10.5 13.1 9 20.3 % English Learners 62.8 40.1 30.5 50.1 37.6 % Free/Reduced Meals* 21.5 9.7 5.5 11.4 15.2 % CalWORKS* Teacher Characteristics 72 93.6 94.8 89.2 97.1 % Teachers w/ Full Credential Source: CDE Educational Demographics Unit 1/25/03. http://data1.cde.ca.gov/dataquest/ * The most current data for these two variables is for the 2000/01 school year. http://www.ed-data.k12.ca.us/

Since there are many differences between the treatment and non-treatment districts, we looked at school-level data rather than District-level data. We did this for two main reasons. First, using schools instead of districts as our observations considerably increased our sample size. Ideally, we would have included many more non-treatment schools and a few other treatment schools identified by CalSTAT. However, this extensive data collection was beyond the time frame and scope of our project. Second, when we examined the student demographics and teacher characteristics of individual schools, they are more comparable between treatment and non-treatment schools than when we examined the same variables at the District level. We also examined the similarities and differences among the districts in regards to their special education populations. Pasadena is serving a higher proportion of students in special education as a percentage of total district enrollment (See Figure 23). In addition, Pasadena has the lowest proportion of students with specific learning disabilities compared to the other districts. There are some factors unique to Pasadena that may account for these differences. First, the Special Education Department in Pasadena serves all students with special needs,

Quantitative Analysis

whereas the model districts serve a higher proportion of students with mild to moderate disabilities. Many of the students with moderate to severe disabilities are served through county or SELPA programs in Elk Grove and Hesperia. Second, the geographic area served by the District (includes the cities of Pasadena and Sierra Madre, and the township of Altadena,) contains a disproportionately high number of Licensed Childrensâ&#x20AC;&#x2122; Institutions (LCIs), juvenile group homes, and foster homes.50 Students with the disability of emotional disturbance often come from these care facilities. Third, Pasadena has an extensive infants/toddlers and preschool identification system for students with special needs. Consequently, under the Child Find piece of the federal statute, Pasadena identifies students with disabilities other than specific learning disability early. This is especially true for students with speech and language impairments, which become noticeable as toddlers develop language acquisition skills. Finally, Pasadena is unique in their offerings of specific inclusion programs at various schools throughout the District. In a few cases, middle to upper class families with children with autism moved into the District so that their children can benefit from these programs. Elk Grove, Ventura, and Pasadena are all relatively close to the state average of students with specific learning disabilities, which is fifty-two percent. Sixty-seven percent of special education students in Redlands are identified as specific learning disabled, significantly higher than the state average. Hesperia, however, has an unusually high proportion of students with specific learning disability, over eighty percent.

Interview with Judith Barhydt, Director of Special Education in Pasadena, 10/02. 74

Quantitative Analysis

Figure 23 â&#x20AC;&#x201C; Special Education Populations for Each District 2001/02 School Year Non-Model Districts Model Districts Pasadena Redlands Ventura Hesperia Elk Grove Special Education Data* 2,864 1,856 1,669 1,562 4,421 District SE Enrollment Special Education as % of Total District Enrollment Disability Categories % Mentally Retarded (MR) % Speech/Language Impaired (SLI) % Emotionally Disturbed (ED) % Specific Learning Disability (SLD) % Autistic (AUT) % Other51 *

12.02

9.33

9.47

9.96

8.85

7.3 29.6

3.5 18.8

2.8 26

2.0 11.1

8.4 20.9

8.8 42.6

2.2 67.4

4.1 55.6

2.0 81.8

6.0 56.1

4 7.8

.48 7.6

2.9 8.6

0.3 2.7

3.4 5.2

Source: CDE Educational Demographics Unit 1/25/03. http://data1.cde.ca.gov/dataquest/. Enrollment differs from that reported on the Special Education Data Report of Key Performance Indicators because of timing of CDE data collection.

8.2 The Statistical Analysis In order to quantify the impact of collaborative programs on student achievement, we analyzed scores in reading and math subtests of the SAT9 between 1998 and 2002. Although we collected data for all five years of the SAT9 between 1998 and 2002, we dropped 1998 scores from our analysis since 1997/98 was the first year Hesperia began implementation of the ExCEL program, and the majority of Hesperia schools did not implement fully during the first year or implemented at different times of the year to varying degrees. Since collaborative programs such as CAST and ExCEL tend to focus preventative efforts at the elementary level, we limited our analysis to elementary schools.52 We gathered school-level score data for grades 2 through 5 for a total of 87 elementary schools in the five

Others includes Deaf, Hard of Hearing, Visual Impairment, Orthopedic Impairment, Other Health Impairment, Deaf-Blindness, Multiple Disability, and Traumatic Brain Injury. 52 See Appendix E for a complete list of schools by district. 75

Quantitative Analysis

districts (see Figure 24).53 We did not include sixth grade score data because Ventura and Redlands placed sixth grade classrooms in middle schools. In addition, not all of the elementary schools in Pasadena have sixth grade classrooms, as the District is in the process of transitioning sixth grade to the middle school level. Figure 24 – A Total of 87 Elementary Schools in Five Districts District

Treatment

Number of Elementary Schools

Elk Grove Hesperia Pasadena Redlands Ventura

yes (CAST) yes (ExCEL) no no no

25 12 22 13 15 Total = 87

8.2.1 Treatment and Control We identified the schools within the model districts of Elk Grove and Hesperia as “treatment” schools and the schools within Pasadena, Ventura, and Redlands districts as “controls.”54 Because we were working with a small number of districts, each with 12 to 25 elementary schools, we combined districts into treatment and control groups in order to increase our power to detect a difference in score gains between those schools with the collaborative model and those with the traditional service delivery models. After combining them into two groups, we had a sample size of 37 treatment schools and 50 control schools. Although Elk Grove and Hesperia do differ slightly in regards to implementation, program elements, the collaborative nature, and site-specific flexibility of each program, we did not feel that combining the two districts together would wash out the treatment effect because both Districts serve all students collaboratively based on frequent diagnostic assessments. It was 53

See Appendix F for a list of data sources.

Quantitative Analysis

difficult to measure the extent of the treatment because the program elements varied slightly from school to school. Ideally, we would have liked to perform extensive ethnographies within each school to determine the level of treatment. However, since this was beyond the scope of our project, we labeled each school as being a part of the treatment or control districts. As noted in Section 5.5.6, Ventura Unified was somewhat confounded with the treatment. We justified our decision to include them as control schools because of the lack of formality in their collaborative efforts and their very recent use of these concepts. Using the fifteen elementary schools in Ventura also boosted the number of observations, which in turn increased statistical power for the purpose of regression analysis. The result may be that we underestimated the true impact of the treatment because the Ventura control does not represent a complete absence of the treatment. 8.2.2 Score Gains We focused our analysis on the percent of students scoring at or above the 50th National Percentile Rank (NPR), which is the level established for students to demonstrate achievement at or above grade level by the California Department of Education.55 Our objective was to determine if treatment schools were more successful in raising test scores than schools in control districts. However, since the SAT9 was first administered after the implementation of Elk Groveâ&#x20AC;&#x2122;s program and concurrently with Hesperiaâ&#x20AC;&#x2122;s implementation, our analysis lacked the ideal before and after design by which to evaluate performance changes. Instead, we based our analysis of improvement on whether treatment schools achieved greater score gains than control schools during the years of the SAT9 administration. 54

Schools in Ventura were coded as controls because they have not formally adopted a collaborative service delivery model District-wide, even though some individual schools are serving at-risk students with special education resources.

Quantitative Analysis

We evaluated score gains in two ways.56 First, we evaluated gains by grade level. Our hypothesis was that treatment schools within Elk Grove and Hesperia would improve their implementation of the collaborative model over time. Therefore, we would expect to see score gains by grade level over time. For instance, if 50% of the second grade class scored at or above the 50th NPR in 1999 in Hesperia or Elk Grove, we would expect the next group of second graders in 2000 to score higher than 50%. Second, we evaluated score gains by cohort. Our hypothesis was that individual students are improving within the collaborative model as they progress through school and receive more treatment. Therefore, we would expect to see greater score gains by cohort from year-to-year within the model districts. For example, if 50% of the cohort of second graders in 1999 scored at or above the 50th NPR, we would expect greater than 50% of the same group of students to score at or above the 50th NPR as third graders in 2000. The following presentation of findings will include: 1. A presentation of data on score gains within treatment and control schools, which includes a descriptive comparison of reading and math gains both by grade level and by cohort. 2. A regression analysis on score gains controlling for those difference between schools that could account for noted variations in test scores found between treatment and control schools in the descriptive presentation. 8.3 Grade Level Descriptive Statistics57 We began our analysis by comparing score gains from 1999 to 2002 for treatment and control schools by grade level. We hoped to find quantitative evidence that treatment schools were improving on implementation of collaborative measures over time, thereby increasing score 55

See Appendix G for further information regarding the SAT9. We also performed the same analyses using mean scaled scores and found similar findings, unless otherwise noted. 56 We also performed an analysis to determine if the program had a greater marginal impact on high poverty or low poverty schools, but found no statistically significant effects.

Quantitative Analysis

gains by grade level at a faster rate than control schools with traditional identification and service delivery models. Figure 25 does suggest that treatment schools achieved greater gains in grades 2 through 4 in reading and in all grades in math. However in order to determine whether those differences were in fact real, we performed a paired t test comparing gains for treatment and control districts by grade level. Our hypothesis was that gains in reading and math scores were the same in treatment and control schools for all grade levels. The t test results confirmed that treatment schools did achieve greater gains in reading at the third and fourth grade levels, and in math in grades 2, 3 and 5. However, for grades 2 and 5 in reading and for grade 4 in math, we could not reject our hypothesis that there were no differences between treatment and control gains.58 Figure 25 â&#x20AC;&#x201C; Score Gains Greater for Treatment Schools by Grade Reading Score Gains Math Score Gains Betw een 1999 and 2002, Grades 2 - 5

Percent Gain

Betw een 1999 and 2002, Grades 2 - 5

15 10 5 0

Grade Treatment

Grade Control

Treatment

Control

Source: Standardized Testing and Reporting Division, California Department of Education, 11/02 <http://www.star.cde.ca.gov>

In summary, the descriptive analysis by grade level revealed the following differences in gains between treatment and control schools (see Figure 26): â&#x20AC;˘

Reading score gains were 4.34 percent higher in the third grade and 3.41 percent higher in the fourth grade for schools with collaborative models than in schools with traditional

See Appendix H for figures showing grade level and cohort descriptive data by district. When we performed the same analysis using mean scaled scores instead of percent scoring at or above 50 NPR, results in fourth grade reading were no longer statistically significant.

Quantitative Analysis

service delivery models. â&#x20AC;˘

Math score gains were 5.68 percent higher in the second grade, 6.90 percent higher in the third grade, and 5.68 higher percent in the fifth grade for schools with collaborative models than in schools with the traditional model.

Figure 26 â&#x20AC;&#x201C; Gains Achieved in Reading and Math for Treatment and Control Schools Grade Treatment Control Difference t value Reading 2nd 15.24 9.64 5.60 1.43 rd 3 7.62 3.28 4.34* 3.13 th 4 11.81 8.40 3.41* 3.34 5th 6.46 7.00 -0.54 0.35 Math 2nd 18.86 13.18 5.68* 3.05 rd 3 18.24 11.34 6.90* 2.41 4th 21.10 17.08 4.03 0.88 th 5 20.46 14.78 5.68* 3.04 *Statistically significant at the 0.05 level. Source: Standardized Testing and Reporting Division, California Department of Education, 11/02 <http://www.star.cde.ca.gov>

Although the descriptive analysis supports the existence of differences in score gains between treatment and control schools, we still need to determine whether these observed differences are in fact associated with the treatment. In order to determine this relationship, we performed a regression analysis controlling for differences between schools.59 The regression analysis enabled us to compare the effects of the treatment among schools which were similar in variables such as size, ethnicity, free or reduced meal participation, percent of English language learners, and teacher experience. 8.4 Cohort Descriptive Statistics We compared score gains between 1999 and 2002 for treatment and control schools by cohort. We focused our analysis on the cohort of students who started second grade in 1999 and finished

See Section 8.3.3 for the regression analysis findings. 80

Quantitative Analysis

fifth grade in 2002.60 Our expectation was that test scores for students in schools with a collaborative model would improve more over time than those students in schools with a traditional educational model. Figure 27 suggests that studentsâ&#x20AC;&#x2122; scores in treatment schools declined slightly in reading but showed greater gains in math in comparison to control schools. In fact, treatment schools appeared to decrease by 1.43 percent in reading from 1999 to 2002, while math scores appeared to increase by 8.98 percent. Again, we used a paired t test to determine whether the observed differences between treatment and control schools were real. None of the differences were statistically significant, and therefore, we could not reject the hypothesis that cohort gains in reading and math were the same for treatment and control schools. Figure 27 â&#x20AC;&#x201C; Score Gains Greater in Math Only for Treatment Schools by Cohort Reading and Math Score Gains Between 1999 and 2002 by Cohort

Percent Gain

10 5 0 Reading

Math

-5 Subject Treatment

Control

Source: Standardized Testing and Reporting Division, California Department of Education, 11/02 <http://www.star.cde.ca.gov>

Even though the descriptive analysis did not show the existence of real differences in score gains by cohort between treatment and control schools, a relationship may still exist between collaboration models and test scores. Descriptive statistics do not account for the 60

We also evaluated scores for the cohort of students who started the 2nd grade in 1998 and found similar results; we could not reject the hypothesis that there was no difference between treatment and control score gains. 81

Quantitative Analysis

differences between individual schools or for other characteristics within a school that can distort or hide the effect of the treatment, and therefore, we performed a regression analysis with the cohort data, controlling for the same demographics used in the grade level analysis regression. 8.5 Regression Model on Score Gains61 We hoped that the regression analysis would confirm that programs such as CAST and ExCEL were positively associated with score gains in both reading and math by grade level and by cohort between 1999 and 2002, suggesting that implementation of these programs improved over time and that student achievement improved at a rate faster in schools with collaborative models than in those with traditional educational models. We developed two simple regression models to determine whether the statistically significant differences we found in the grade level comparison held up after controlling for other demographic differences between schools. We also hoped the two models would unmask an association between the treatment and score gains by cohort, although our descriptive analysis found no significant differences. 8.5.1 Regression Variables We collected data on school-level demographics that we believed influence variations in student test scores and that are also commonly used as controls in educational studies in the literature (see Figure 28). We were especially interested in the free or reduced meal and English language learner variables, as we hypothesize that students identified within these categories are likely to be impacted the most through collaborative models such as CAST and ExCEL. Although we collected information on each variable for 1998 through 2002, we again dropped 1998 from our analysis to be consistent with our previous descriptive analyses. We worked with two forms of each variable, either the average over the four years or the change

We adjusted the standard errors for the non-independence of schools within districts. 82

Quantitative Analysis

between 1999 and 2002. Our dependent variables for reading and math were both changes in test scores between 1999 and 2002.

Figure 28 – Independent Variables Used in Regression Analysis Variables Description School Enrollment Measured as the number of students enrolled in school (enroll avg) on “Information Day” in early October. Number of Students Tested Measured as the percent of students enrolled taking the (∆ tested 02-99) tests in grades 2 – 5. Ethnicity Measured as percent African American, white, Latino (black avg, white avg, latino avg) or other. Free or Reduced Meal Program Measured as the percent of students at a school 2 (meals avg, ∆ meals 02-99, meals avg ) enrolled in the program. English Language Learners (ELL) Measured as the percent of students for whom there is 2 (el avg, ∆ el 02-99, el avg ) a report of a primary language other than English on the state-approved Home Language Survey and who on the basis of state approved oral language assessment procedures have been determined to lack the clearly defined English language skills of listening comprehension, speaking, reading, and writing necessary to succeed in the school’s regular instructional programs. Measured as the average number of years teaching. Teacher Experience62 (teach avg) 8.5.2 Developing the Model Since all of the independent variables most likely account for a portion of the variation observed in student test scores, we ideally would have liked to include all of these components within our regression model. However, we were working with a small number of schools, and therefore were limited in the size of our regression equation. We were also limited by the fact that a number of the variables were highly correlated and could not be included within the same equation without distorting our results.63 Taking into consideration that free and reduced meals 62

We also collected data on teacher credentials, but felt that years of experience was a better measure of teacher quality. 63 See Appendix I for a list of the highly correlated independent variables. 83

Quantitative Analysis

and English language learners were two of the most highly correlated variables (0.784), we determined that the best regression analysis would include two separate models, one controlling specifically for free or reduced meals and the other for English language learners. 1. Free or Reduced Meals Model (Meals) read02-99 or math02-99 = α + β1(treatment) + β2 (∆ tested 02-99) + β3 (enroll avg) + β4 (black avg) + β5 (other avg) + β6 (meals avg) + β7 (∆ meals 02-99) + [β8 teach avg)] 2. English Language Learners Model (ELL) read02-99 or math02-99 = α + β1(treatment) + β2 (∆ tested 02-99) + β3 (enroll avg) + β4 (black avg) + β5 (other avg) + β6 (el avg) + β7 (∆ el 02-99) + [β8 teach avg)] With respect to ethnicity, the white variable was highly correlated with a majority of the other independent variables and the Latino variable was highly correlated with both the free or reduced meals and English language learner variables. Therefore, we only included African American and other ethnicities in our models. We first ran our regression models without teacher experience. We then added the variable for teacher experience to the model to see how our results regarding the treatment changed. Finally, we tried to account for non-linear relationships by adding curvilinear terms. Therefore, we also ran our regression equations with a squared term for both free and reduced meals and English language learners. 8.5.3 Regression Findings by Grade Level and by Cohort Figures 29 and 30 present the significant findings of both the free or reduced meals and English language learners models by grade level. In regards to the free or reduced meals model, differences in score gains between treatment and control schools were significant only in fifth grade reading and second grade math. Therefore, among schools with the same enrollment, change in percent of students tested between 1999 and 2002, percent black, percent other, percent receiving free or reduced meals and change in percent receiving free or reduced meals,

Quantitative Analysis

the effect of having the treatment was associated with a decrease in score gains of -3.91 percent (p<0.10) in fifth grade reading and an increase of 9.99 percent (p<0.01) in second grade math.64 Figure 29 – Free and Reduced Lunch Equation: Treatment Associated with Score Gains in Grade 5 in Reading and Grade 2 in Math Reading Grade

Treatment

5 (t)

-3.910 (-2.44)*

Math Grade

Treatment

∆Percent Tested 02-99 -28.740 (-1.33) ∆Percent Tested 02-99 -36.416 (-1.33)

2 9.994 (t) (5.31)*** * Significant at the 0.10 level. *** Significant at the 0.01 level.

Enrollment Avg

Black Avg

Other Avg

Meals Avg

∆Meals 02-99

_cons

0.018 (2.17)*

2.213 (0.51)

-11.798 (-0.71)

2.127 (0.61)

-20.795 (-1.15)

-4.643 (-1.49)

Enrollment Avg

Black Avg

Other Avg

Meals Avg

∆Meals 02-99

_cons

-0.013 (-7.67)***

30.548 (5.63)**

-4.939 (-0.53)

-5.575 (-0.77)

-12.662 (-0.85)

18.916 (4.72)***

In regards to the English language learners model, differences in score gains between treatment and control schools were significant in grades 2 and 4 in reading and in grades 2 and 5 in math. Therefore, among those schools with the same enrollment, change in percent of students tested between 1999 and 2002, percent black, percent other, percent English language learners and change in percent English language learners, the effect of having the treatment was associated with an increase in score gains of 7.80 percent (p<0.05) in second grade reading and 3.47 percent (p<0.01) in fourth grade reading. In math, after controlling for the factors listed above, the effect of having the treatment was associated with an increase of 9.74 (p<0.05) in second grade and 5.62 in fifth grade (P<0.01).65

When we added teacher experience and meals squared variables to the equation, the coefficient on treatment in fifth grade reading was no longer statistically significant. For second grade math, the treatment coefficient increased to 10.237 (p<0.05) when teacher experience was added and 10.343 (p<0.01) when meals squared was added. 65 The coefficient on treatment in fourth grade reading was no longer statistically significant with the addition of teacher experience, however, adding ELL squared increased the treatment effect to 4.11 (p<0.10). The addition of both teacher experience and ELL squared variables increased coefficients on treatment, which remained statistically significant, in second grade reading, second grade math and fifth grade math an average of 0.5 percent. Finally, the addition of the teacher experience variable changed the association of the treatment in third grade reading to a statistically significant 5.80 (p<0.10).

Quantitative Analysis

Figure 30 – English Language Learners Equation: Treatment Associated with Score Gains in Grades 2 and 4 in Reading and Grades 2 and 5 in Math ∆Percent Tested 02-99 2 7.801 -47.632 (t) (3.39)** (-1.76) 4 3.469 2.787 (t) (2.15)* (0.21) ∆Percent Math Treatment Tested Grade 02-99 2 9.725 -31.340 (t) (3.36)** (-1.18) 5 5.623 -38.138 (t) (7.45)*** (-1.75) * Significant at the 0.10 level. ** Significant at the 0.05 level. *** Significant at the 0.01 level. Reading Grade

Treatment

Enrollment Avg

Black Avg

Other Avg

EL Avg

∆EL 02-99

_cons

0.003 (0.26) -0.001 (-0.18)

10.278 (0.55) 10.327 (2.32)*

-27.402 (-0.95) 10.684 (3.55)**

-12.370 (-0.49) -2.221 (-0.36)

-17.600 (-0.56) -18.043 (-1.16)

9.948 (1.46) 6.683 (6.47)***

Enrollment Avg

Black Avg

Other Avg

EL Avg

∆EL 02-99

_cons

-0.012 (-4.70)*** 0.005 (1.14)

31.488 (7.20)*** 0.921 (0.25)

-5.104 (-0.32) 12.714 (1.72)

-11.347 (-0.78) 14.080 (7.17)***

-3.000 (-0.07) -47.760 (4 94)***

17.635 (6.18)*** 7.558 (5.57)***

Neither of the models revealed any significant association between the treatment and score gains by cohort in reading and math. 8.6 Limitations Although we made every effort to gather the appropriate data as accurately as possible, the information we collected did present certain limitations, which influenced the significance of our results. We were particularly concerned with the measurement error associated with our dependent variable, changes in test scores, and the independent variables, free or reduced meals program and percent of students taking the test. The accuracy and effectiveness of student standardized test scores has been the subject of a number of educational studies. Kane and Staiger find that when using school-level data, one of the main sources of test score imprecision is sampling variation.xxix They note that this problem is particularly evident in many elementary schools, which contain a relatively small number of students per grade. The authors assert the amount of variation among students’ test scores is

Quantitative Analysis

often large relative to the total amount of variation observed between schools. Kane and Staiger also note that measuring changes in mean test score levels from one year to the next are unreliable measures. The authors continue, Schools differ very little in their rate of change in test scores or in their mean valueadded-- certainly much less than they differ in their mean test score levels. Moreover, those differences that do exist are often non-persistent--either due to sampling variation or other causes.xxx Since we have analyzed student achievement by measuring the change in test score data over a period of years, it is possible that a great proportion of the variance seen in the gains is due to sampling variation or other transient factors. We generated the percent of students receiving free or reduced meals by dividing the free or reduced price meal program count by the enrollment count for each school. Enrollment counts are collected on â&#x20AC;&#x153;Information Dayâ&#x20AC;? in early October and therefore only represent the number of students enrolled in school on that day. Free or reduced price meal program counts include students that may not have been attending the Local Education Agency. As a result, some of the students included in the free or reduced meals program count may not have been included in the enrollment count for the school. Therefore, we may have overestimated the percent of students participating in the free or reduced meal program, and thereby increased the measurement error of our analysis. We also generated the percent of students taking the test by dividing the number of students who took the test in grades 2 through 5 by the enrollment count. The enrollment count includes the total number of students enrolled for an entire elementary school, including in some cases grades K through 6. Therefore, by not removing the count of students in kindergarten, first grade and sixth grade, we again introduced more measurement error.

Quantitative Analysis

8.7 Conclusion In summary, the regression analysis by grade level revealed the following associations between the treatment and score gains: •

Collaborative programs were associated with an increase of approximately 10 percent in students scoring at or above the 50th percentile between 1999 and 2002 in second grade math scores (relationship found in both the meals and ELL models).

•

Collaborative programs were associated with an increase of approximately 7.5 percent in students scoring at or above the 50th percentile between 1999 and 2002 in second grade reading scores (relationship found in ELL model only).

•

Collaborative programs were associated with an increase of approximately 5.5 percent in students scoring at or above the 50th percentile between 1999 and 2002 in fifth grade math scores (relationship found in ELL model only).

•

Collaborative programs appear to also be associated with small score gains in third, fourth and fifth grade reading, however these findings were not consistent in both regression models.

8.8 Discussion We expected the collaborative programs to be positively associated with score gains in reading and math for all grades and by cohort. However, our quantitative analysis only provided evidence for significant gains in reading and math in the second grade and math in the fifth grade. One explanation of this finding is that the districts are not testing the same group of students. We hypothesize that control schools are not testing as many special education students as the treatment schools. Many special education students’ IEP teams exempt students from taking all or part of the test. Since we expect the model districts, Elk Grove and Hesperia, to identify fewer students as having specific learning disabilities, we expect more of their students

Quantitative Analysis

to be taking the SAT9 exam. Figure 31 confirms this theory, as both Elk Grove and Hesperia tested a greater percentage of their special education students in 2002. Figure 31 â&#x20AC;&#x201C; Special Education Students Taking SAT9

Percent of Special Education Students Taking SAT9 in Spring 2002 100.0% 80.0% 60.0% 40.0% 20.0% 0.0% Elk Grove

Hesperia

Pasadena Reading

Ventura

Redlands

Math

Since these students typically, though not always, score below the 50th NPR, we believe this influenced our findings in the higher grades. We would not have expected to see similar results in the second grade, since these students have yet to be identified as requiring special education services. We believe that class-sized reduction in the lower grades is another possible explanation of our quantitative findings. We theorized that the collaborative intervention coupled with lower class size produced higher achievement gains in the second grade than in those grades without class-size reduction. Although second and third grade classrooms in California have a ratio of 20 to 1, many districts did not begin class-size reduction in the third grade until after 1999/00, which may be one explanation of why we did not note an effect of the treatment in the third grade.

Quantitative Analysis

Another possible explanation of our results stems from the transition in curriculum for students between the primary and higher elementary grades. In the primary grades, students are expected to learn to read, whereas in the intermediate grades, they are expected to read to learn. As students progress through the elementary grades, the gaps widen, and the time required for successful intervention increases. A successful intervention at the kindergarten level can be performed in a twenty-minute period, while the same intervention at the sixth grade level has been estimated to take two hours in order to achieve the same level of success.66 Another explanation of our results that has been widely supported in the literature is the lasting impact of intervention alternatives. As we described extensively within the literature review, intervention models such as early childhood development, preschool and full-day kindergarten have been associated with greater marginal impacts on test score gains in the primary grades, however most of the cases test score effects wore off or decreased as students progressed through the upper grades. The fact that we found a significant effect in fifth grade in math even with our data limitations, suggests that perhaps our analysis underestimated the impact of the program, particularly in grades 3, 4 and 5.

Interview with Terry DeBoer, Program Specialist in Elk Grove, 2/14/03. 90

Implementation Analysis

9. IMPLEMENTATION ANALYSIS

After examining the two model programs in depth and speaking with representatives of the Pasadena Unified School District, we provide an analysis of the implementation efforts necessary to maximize success specifically designed to address the wealth of resources as well as particular challenges PUSD faces. 9.1 Existing Resources The collaborative service delivery model does not necessarily require additional funding. It is a more efficient way of using existing resources. Nor does the District need to purchase a curriculum. The collaborative service delivery model does not replace existing materials and interventions; the model simply reorganizes how services are delivered within the schools. The model does require time to collaborate, professional development, and the use of a panel of student data to develop specific interventions for each learner. The District already has several existing resources that would dovetail with a collaborative educational service delivery model. First, Pasadena has fully implemented the 91

Implementation Analysis

Open Court curriculum in September 2002. This program has a battery of assessments that the classroom teachers must administer at the beginning of the year, at the end of each week (after one story), and a Lion’s Assessment every five or six weeks at the end of each unit. Using existing assessment tools will eliminate the need for compiling a battery of assessments and requiring teachers to do new or additional work. For example, the Open Court Lion’s Assessments could be used in the collaborative, early intervention model and form the basis of information to scaffold students, by skill and need, into appropriate intervention settings. Pasadena teachers currently use the Lion’s Assessments to monitor each student’s language proficiency and progress in the reading curriculum. This assessment ranks students from 1 to 10 and places students into one of three groups: a rank of 1 to 3 is for students who need “intensive” intervention, a rank of 4 or 5 is for students who require “strategic” intervention, and a rank of 6 to 10 is reserved for students who are meeting “benchmark” performance expectations. Using Pasadena’s existing assessment system, teachers could meet with the school-site collaborative team each trimester, present student data, and design interventions together. The collaborative team can use the data to scaffold students into the appropriate class settings. Second, Pasadena implemented a banked day beginning in September 2002. School sites use this differently across the District. However, this day might be used more effectively and efficiently for collaborative meeting time comparable to Collaborative Wednesdays in Hesperia. This would focus teacher and school collaboration on positive student outcomes, especially increasing reading and math proficiency. Currently, there are schools using this time for staff meetings, which seems less than efficient.

Implementation Analysis

Third, special education staff and instructional assistants throughout the District have been trained in a variety of intervention programs used to supplement core curricula. Non special education students at-risk of failure who require intense remediation and intervention can benefit from instruction in these specialized programs delivered by trained professionals. Current reading and math intervention resources available in the District include: 1. 2. 3. 4. 5. 6.

Herman Reading Program LiPS multi-sensory phonics reading program Language! Zoo Phonics Earobics Touch Math Fourth, PUSDâ&#x20AC;&#x2122;s schools are already rich with specialized personnel who can be used

more efficiently to target the needs of individual students. Curriculum Resource Teachers, Language Development Resource Teachers, Literacy Aides, Language Development Aides, and support staff funded by Title 1 money offer a wide range of expertise and abilities that can be incorporated into the collaborative service delivery model. Fifth, training and technical assistance for administrators and staff as well as financial support for visiting the model districts can be funded by CalSTAT. In conjunction with Association of California School Administrators (ACSA) and Schwab Learning, CalSTAT identified the model districts and schools in the state who are implementing collaborative service delivery models. Finally, several school sites in Pasadena are independently showing interest in a collaborative model in which all students are served based on need. The Resource Specialist Teachers at Noyes and Field Elementary have designed ways to serve general education students in collaboration with their classroom teachers. The principal and vice principal at Longfellow Elementary are piloting an Individualized Learning Plan for each students attending summer

Implementation Analysis

school in 2003. These innovative practices are similar to assessing and meeting the needs for all students under CAST and ExCEL. Representatives from these three sites are interested in learning more about the collaborative service delivery model.67 9.2 Potential Costs If a collaborative model proves successful, there is one potential fiscal implication involving personnel. This implication does not involve adding to the budget, but rather keeping personnel expenses constant. Both Elk Grove and Hesperia Superintendents and the School Boards made a commitment to the principals and teachers not to reduce special education and support staff levels in the schools even if special education caseloads were reduced over time. Other potential costs are providing for substitutes during trainings, professional development, periodic collaboration time, and during assessment periods. A substitute costs about $100 per day. Professional development programs are basically fixed costs, services that districts already fund and staff. Pasadena already provides school staff with banked time for staff development, which is currently loosely defined. Furthermore, since all teachers have already been trained in Open Court, further staff development regarding assessments may not be required. These cost are minimal are in comparison to the potential gains for students. 9.3 Potential Savings The collaborative service delivery model may actually save the District money by reducing special education caseloads, initial evaluations, and the number of due process hearings. Both Elk Grove and Hesperia have seen a sharp reduction in due process hearings and mediations, which cost Pasadena approximately $360,000 per year.68 Despite population growth and a constant severely handicapped population, Hesperiaâ&#x20AC;&#x2122;s reduction in students identified with mild 67

Interviews with RSTs at Noyes and Field; vice principal at Longfellow, 2/21/03.

Implementation Analysis

to moderate disabilities has had a small impact in reducing the special education encroachment into the general education budget. In Elk Grove, the percent of students identified as special education has decreased since 1991 by almost 50% while their total District population has increased by 150%. Figure 32 – Potential Savings Based on Results in Model Districts Outcome Estimated Savings Reducing non severe special education $3,000-3,500 per student caseloads Reducing number of initial evaluations $1000 per evaluation Reducing number of fair hearings $2,500 per day; $30,000 average per hearing Increased Average Daily Attendance (ADA) $26 per day, per student Considering Pasadena’s existing resources and the potential costs and benefits associated with a collaborative service delivery model, implementation is feasible.

Email with Judith Barhydt, Director of Special Education for Pasadena, 3/19/03. Estimate based on 2001/02 calendar year – 12 hearings at an average cost of $30,000 per hearing. 95

Implementation Recommendations

10. IMPLEMENTATION RECOMMENDATIONS

We have identified the crucial next steps Pasadena Unified School District needs to take to increase the chance of successful implementation of a collaborative educational service delivery model. Borrowing from the lessons learned in Hesperia, there is a clear implementation strategy that would enhance Pasadenaâ&#x20AC;&#x2122;s chances for success in adopting a collaborative educational service delivery model. This broad, systemic change must be initiated, supported, and continually reinforced by the Superintendent Dr. Percy Clark, Jr., Deputy Superintendent Kathy Duba, the Assistant Superintendents, especially Jacqueline Cochran who oversees the Special Education Department, and the School Board. Several other key stakeholders in the District need to be informed and involved in the planning stages. For this model to be successful, buy-in from all levels of staff is essential. The best way to gain buy-in is through informing stakeholders and including them in the planning process. The following sections outline recommendations for a course of action (See Figure 34 at the end of this section). We recommend devoting at least six months to this District level investigation and planning period. 96

Implementation Recommendations

10.1 Create Teams, Define the Problem, Set Goals The first step is to convene a District-wide stakeholder planning team to steer the efforts in implementing a new model. This team should include District level administrators and representatives from the principals, teachers, Special Education Department, and parents. This groupâ&#x20AC;&#x2122;s first task is to further investigate the programs in Hesperia and Elk Grove and the other model districts identified by CalSTAT to see the results for themselves. As a result of being recognized by CalSTAT, these districts are open to providing information and training opportunities to any other district that is interested. In addition to the planning team, the District may want to consider planning for the following teams to support the implementation efforts at various levels: Figure 33 â&#x20AC;&#x201C; Creating Teams Team District-wide Stakeholder Team

District Administrator Team

Pilot School Teams

Members and Purpose -Members include teacher, parent, site administrator, classified, and District personnel representatives from across the District. -Purpose is to research the collaborative models and design a program that fits PUSDâ&#x20AC;&#x2122;s priorities, capabilities, and constraints. -Members include top level District administrators including an assistant superintendent, the director of special education. -Purpose is to drive systemic change efforts at the Districtlevel and keep the same target in sight. -Members include site administrator, special education teacher(s), grade level teacher representatives, and other staff. -Purpose is to design and implement the school site plan incorporating the collaborative model as outlined by the District.

10.2 Assemble the Evidence and Devise a Program Funding is available through technical assistance from CalSTAT. One of the key areas they support with technical assistance is collaborative effort. They can provide for key people from districts with successful collaborative educational service delivery models to come to PUSD and

Implementation Recommendations

present a four-day training session. The District can utilize this professional development as Hesperia did and send the principal and team of teachers from each school to this training. Technical assistance funding is also available to pay for Pasadena administrators and staff to visit model districts and see the program in action. Once the team of District stakeholders has gathered adequate information, we recommend that the stakeholder team drafts a District-wide collaborative educational service delivery model program and implementation plan, including the program elements best suited to Pasadenaâ&#x20AC;&#x2122;s priorities, capabilities, and constraints. 10.3 Get Buy-in and Commitment The Superintendent and other representatives from the District-wide planning team should share the plan with key District entities, such as the School Board, the PTA Council, the Community Advisory Committee, principals, teachers, and the community. Implementation efforts should not move forward if a majority of the School Board is not fully committed. The Boardâ&#x20AC;&#x2122;s support is crucial to ensure the special education staff positions will not be cut if numbers decrease. Using data and research based concepts will assist the District in convincing skeptical members of the Pasadena community. 10.4 Start Small â&#x20AC;&#x201C; Pilot Schools Begin Planning The District planning team will need to discuss and decide the best course of action for implementing the program in elementary schools. There are four options: 1. The District administration could select a handful of pilot schools based on variables attributing to success (i.e., strong principal, young staff, etc.). 2. Solicit volunteers for a small number of pilot schools, thus ensuring commitment on the part of the principal and staff. 3. The Superintendent could give a directive to all principals, like in Hesperia, to begin implementing the program, but provide for flexibility in site-specific details.

Implementation Recommendations

4. Start with the elementary schools grouped in one cluster so as to ensure consistency with top level administration since the schools would all have the same Assistant Superintendent. Option two has promise for the following reasons. First, if principals and staffs volunteer to pilot the delivery model, it signals a higher level of buy-in and therefore increases the chances of success. When dealing with systemic change of a large entity such as a District, it will help to have a few early successes to convince others of the benefits of the program. Second, starting with a few pilot schools as opposed to having all elementary schools implement on their own time frame will provide for a natural way to evaluate effectiveness. The measures of success can be compared between pilot and non-pilot schools after a period of time. Third, principals and site teams will benefit from cross-school collaboration opportunities between first adopters and others. This will serve to further convince staff and administrators. This cross-collaboration, much like the monthly presentations held in Hesperia, can serve as an open forum where problems can be raised and honestly dealt with. Fourth, the process of soliciting volunteers reinforces support for program innovators by highlighting successes. Since one of the most crucial key factors of success is principal leadership and commitment to the program concepts, those who volunteer will most likely be more motivated and dedicated to solid implementation of the program. 10.5 Monitor and Evaluate Effectiveness In order to assess the success of a collaborative service delivery model, an evaluation plan must be devised prior to implementation. A solid evaluation of the program will allow the District to attribute positive outcomes to the collaborative model. The following section (Section 11) reviews recommendations for an evaluation plan.

Implementation Recommendations

Figure 34 â&#x20AC;&#x201C; Implementation Process Step 1: Create teams, Define the problem, and Set goals.

Month 1 Assemble a broad based planning team of district stakeholders including top level administrators, principals, teachers, parents, and community members. Identify the problem; establish goals and consistent message.

Step 2: Assemble the evidence and Devise a program.

Months 2 to 4

Step 3:

Step 4:

Step 5:

Get buy-in and commitment.

Start small â&#x20AC;&#x201C; Pilot schools begin planning.

Monitor and Evaluate effectiveness

Months 4 to 5 Months 4 to 6 Key DISTRICT-Level Activities

Collect information needed to draft a plan.

Present plan to school board and Superintendent.

Outline key concepts and program elements that PUSD requires. Outline program elements and methods that school sites can determine.

Secure commitment from Superintendent and School Board that special education personnel will not be reduced if caseloads decrease.

Present concept at a principalsâ&#x20AC;&#x2122; meeting to gather input and to inform.

Select or solicit volunteer schools for piloting. Need strong principal and cadre of supportive teachers who wish to innovate.

Consult district budget staff to model fiscal impact of program changes.

Meet with Union leadership to get buy-in.

Months 1 to 6

Provide administrative support, data, training, and budgeted resources to the School Team.

Develop a data collection and evaluation plan that is useful to schools and evaluators.

Inform Superintendent and School Board about progress on a regular basis. For example presentation or status report once a month.

Collect pre-program data for later evaluation. Compare pre and post test score data on pilot schools after first year. Compare data on other desired outcomes. Use results may bolster support for program among skeptics.

100

Implementation Recommendations

Step 1: Create teams, Define the problem, and Set goals.

Month 1

Participate in Districtlevel planning as requested.

Step 2: Assemble the evidence and Devise a program.

Months 2 to 4

Step 3:

Step 4:

Step 5:

Get buy-in and commitment.

Start small – Pilot schools begin planning.

Monitor and Evaluate effectiveness

Months 4 to 5 Key SCHOOL-Level Activities

Collect and discuss schoollevel information. Provide input and feedback to District level team with regard to required vs. flexible elements of the program.

Determine if school site is ready, based on key variables such as principal leadership/commitment, enthusiastic staff, and cohesive teaching team.

Months 4 to 6

Months 1 to 6

Pilot school team participates in the four-day training offered by CalSTAT.

Consult test score and other outcome measure data after the first year to adjust the program.

After training, pilot schools draft a site-specific plan for implementation.

Pilot schools collect pre and post data on indicators such as attendance, literacy, and behavior.

District and School Teams adopt the school-site plan. Begin to implement elements of the plan.

Survey principal, teacher and parent satisfaction.

Key Processes District planning team meets once per month and informally as necessary. Seek input from all District stakeholders.

Teams visit and investigate model programs first hand. Invite school board members, union officials and parents

Publicize the program: press release, radio talk show appearance, district newsletter and website article.

Formally report findings to stakeholders.

Pilot schools communicate successes and challenges with other schools in middle of year 1.

ĤĀ $ $

$ $

Share results of evaluation with School Board, Supt. and all stakeholders. Publicize the results of the evaluation.

101

Evaluation Plan

11. EVALUATION PLAN

Program evaluation literature underscores the importance of employing solid methodology, including controlled experiments, pre and post treatment data, and time-series analysis. A quality evaluation will have the power to validate the program, win support of stakeholders, and document compliance with state and Federal laws, particularly “No Child Left Behind.” Most importantly, an insightful evaluation will provide detailed data to identify and correct implementation problems at individual school sites, thus improving educational service delivery to students. In our own evaluation of the collaborative, early intervention programs in Elk Grove and Hesperia, the importance of following these research methods is clear. Cobbling together many sources of information, we constructed our own quantitative analysis, reviewed districts’ internal data, and analyzed evaluation recommendations for Elk Grove’s program by an education research firm, Far West. In each of these inquiries, our ability to draw strong conclusions about the programs’ true impact or quality was obfuscated by differential implementation at individual

102

Evaluation Plan

school sites and by methodologically inadequate evaluation plans within districts. As discussed in the Methodology Section 2.3 of this report, the primary weaknesses of district evaluation efforts were: 1. Both within and between districts, the experiment was not controlled; 2. Districts lacked the ability to evaluate pre and post treatment effects due to a change in standardized test used, in the case of Elk Grove, and the lack of standardized testing prior to treatment in the case of Hesperia; 3. The evaluations relied on school level data, not student data, consisting of the percentage of students scoring below, at or above a nationally norm-referenced group of students on a standardized test; and 4. The definition of implementation and application of the “treatment” differed from site to site, making evaluation difficult. With regard to meaningful evaluation, we learned a central lesson from our site observations, personal interviews, and statistical analysis. Evaluation and implementation are inextricably linked. The quality of any evaluation is directly impacted by two critical factors: a. The development, adoption, and execution of a data collection and evaluation plan cognizant of program design before and during implementation; and b. A clear, consistent definition of the desired outcomes, the treatment, and adequate implementation used by program staff, administrators and evaluators. Building on these observations, our evaluation recommendations to Pasadena Unified School Districts are drawn directly from the challenges in implementing and evaluating Elk Grove and Hesperia’s programs. 11.1 Recommendation 1 – Design a Controlled Experiment A controlled experiment in this context would involve matching treatment schools with demographically similar non-treatment schools within the District. Panel data on the desired outcome measures from treatment and control schools should be compared to determine the program’s effectiveness in meeting district goals.

103

Evaluation Plan

In addition to comparing schools with a collaborative service delivery model to those with the traditional model, the treatment schools can be evaluated based on pre and post data. This provides the District with the appropriate base-line data to track program impact over time. 11.2 Recommendation 2 â&#x20AC;&#x201C; Identify Goals and Desired Outcomes A solid evaluation plan should clearly define the districtâ&#x20AC;&#x2122;s goals and desired outcomes for the program in order to measure progress. The Far West evaluation plan recommendation detailed many desired outcomes for Elk Grove. An evaluation plan must identify the major and minor measurable goals of the program. For example, the Superintendent in Hesperia defined one overarching goal when he called upon the District to raise student SAT9 means scaled scores to an average of 800 by 2010. This target established a clear benchmark and programmatic goal for all staff. The administrators and teachers in Elk Grove and Hesperia established academic, fiscal, and other desired outcomes. Academic goals included measures such as the number of at-risk students served quickly and adequately to prevent further academic set-backs, the percentage of students who read proficiently at grade level, the number of students referred for initial assessments for special education, and the percentage of students identified and served by special education. Fiscal goals included measures of financial success such as decreasing the number of initial assessments, reducing the special education population, preventing expensive late interventions and due process hearings. Other desired outcomes included increases in student attendance, decreased disciplinary referrals, as well as improved student and family mental and physical health. For each of the desired outcomes, an evaluation plan should establish a reasonable numerical target, a completion date, and specify the staff members who are

104

Evaluation Plan

responsible for meeting each goal. The results should be reported regularly to stakeholders throughout the district. 11.3 Recommendation 3 â&#x20AC;&#x201C; Use Data to Inform Decisions For the final program evaluation, analysts should collect student level data, in addition to school level data. Student level data will allow for a larger sample size, reduce standard errors in tests of statistical significance, increase statistical power, control

Attendance, behavior, health, family, and academic performance data should drive programmatic decisions.

for more variables, and account for student movement into and out of schools. It is vital to understand that the collaborative early intervention model is data-driven. Data collection, analysis and dissemination impacts program decisions and the quality of evaluation on two levels: in the District and within individual schools. Fortunately, Pasadena Unified School District currently operates a computerized data system(s) to track attendance, student program participation (i.e. regular education, special education, and speech and hearing students), behavioral referrals, and standardized test scores. District administrators and ITS staff should ensure the data is accessible and useful to program administrators and school teams helping students. In this regard, it might be beneficial for the District leaders charged with implementation to consult and engage the Districtâ&#x20AC;&#x2122;s Planning, Research and Evaluation Department to identify the systemâ&#x20AC;&#x2122;s data capabilities and gaps, to train school staff and to design an evaluation plan. Hopefully, the existing system can be used for implementation and evaluation purposes. Within individual schools, teachers and school professionals need quality, up-to-date data to make intervention decisions for individual students and judge the impact of the program

105

Evaluation Plan

overall. According to Elk Grove Unified School District’s measure of program implementation quality, only collecting student standardized test scores and teacher observations is not adequate to create student interventions directly related to his or her specific learning needs. Based on the experience of model districts, we recommend that school collaborative teams collect and use a panel of student data that includes attendance, behavior, student health, family, and academic performance data. Additionally, school staff should be able to generate student, class, grade, and school reports in designing interventions and planning program or curricular changes. 11.4 Recommendation 4 – Define Implementation An ideal evaluation plan would define the “treatments” the program offers to students and adequate implementation at a school site. In implementation, it needs to be clear what pieces of the model are a part of its major underlying concepts and what pieces of the model can be determined at the site level. This involves defining how student assessment data are used, the professional collaboration methods, and prevention and early intervention strategies that are research based. A tool such as the implementation rubric developed by Elk Grove will assist in uniformly identifying successful implementation at each site. Finally, the process of creating the rubric will induce planners to clearly define specific aspects of a successful implementation. 11.5 Conclusion As in any endeavor, the quality of work one does and the ability to measure success usually depends on planning, organizing and executing. This is true of high-quality evaluation plans. First, a solid evaluation plan clearly defines measurable program and evaluation goals, outlines key problems, identifies resources, creates a data collection system, and engages staff in executing the plan. Second, a solid evaluation plan follows proven methodology such as controlled experiments, pre and post treatment data, and appropriate data collection. Finally, the

106

Evaluation Plan

evaluators keep in mind the ultimate uses of the final evaluation: 1) to judge the true impact of the program, all else being equal, and 2) to inform decision-makers and service providers about more effective and efficient educational practices.

107

Conclusion

12. CONCLUSION

Experts and practitioners, including administrators and teachers interviewed in Pasadena Unified School District, overwhelmingly recognize that the current general and special education models are failing a considerable number of at-risk students. These students desperately need immediate, specialized instruction. However, students often are allowed to struggle for several years in general education settings before qualifying for the services available in the special education system. The system literally makes many students who do not show a two-year learning discrepancy wait to fail further. Clear evidence for this failure exists. Eighty percent of the diagnosed learning disabled students qualify because of reading difficulties, academic deficits that may have been caused by inadequate instruction not by true learning disabilities. Thankfully, researchers and educators have not merely cataloged the failure of our current model; they point to models that work for all students. Among these are early childhood development, full-day Kindergarten, pre-school, class size reduction, and collaborative intervention and prevention. In Pasadenaâ&#x20AC;&#x2122;s current fiscal climate and considering its recent implementation of Open Court, the collaborative intervention model stands out as a viable 108

Conclusion

alternative. This model allows the District to use existing curriculum, resources, and staff more efficiently to help all students succeed. The District will need to explore appropriate ways to work within the Open Court program while implementing a collaborative model. As some sources suggested, the District may wish to wait two to three years into the Open Court adoption until launching a major new initiative.69 However, this consideration does not forestall Pasadena from starting the planning process. Through our investigation, we have found that systematic collaboration between general and special education professionals that focuses on early intervention in literacy contributes to reducing special education caseloads, increasing schoolsâ&#x20AC;&#x2122; average daily attendance, and possibly raising student test scores. We recommend that Pasadena investigate the possibility of adopting a collaborative service delivery model to further strengthen the quality education the District already offers its students.

Interview with Pasadena site administrator, 2/21/03. 109

Endnotes

Endnotes i

Robert E. Slavin, Nancy Kareit, and Barbara Wasik, “Preventing Early School Failure: What Works?” Educational Leadership 50(4), December 1992 and January 1993, <http://www.ascd.org/readingroom/edlead/9212/slavin.html> (9 January 2003). ii

W.S. Barnett and C.M. Esobar, “The Economics of Early Educational Intervention: A Review,” Review of Educational Research 57 (1987): 387. iii

Stuart Biegel, “Paragraph 44 Independent Review, Report No. 19, The Annual Report of the San Francisco Consent Decree Monitoring Team,” 31 July 2002, 3, <ww.gseis.ucla.edu/courses/edlaw/sfrepts.htm> (18 March 2003). iv

Nettie Legters and Edward L. McDill, Schools and Students at Risk: Context and Framework for Positive Change, ed. Robert J. Rossi (New York: Teachers College Press, 1994), 24. v

President’s Commission on Excellence in Special Education, A New Era: Revitalizing Special Education for Children and their Families, Section 2 “Assessment and Identification,” 1999, 2. <http://www.ed.gov/inits/commissionsboards/whspecialeducation/reports/two.html> (8 January 2003). vi

G. R. Lyon et al, “Rethinking Learning Disabilities,” Rethinking Special Education for a New Century, (Washington, D.C.: Fordham Foundation and Progressive Policy Institute, 2001): 263. vii

Lyon et al, 264.

viii

Slavin, Karweit, and Wasik, 1.

Samuel Kirk, Educating Exceptional Children. (Boston, MA: Houghton Mifflin, 1962), 263. xi

J.K. Torgesen, “Individual Responses to Early Interventions in Reading: the Lingering Problem of Treatment Resisters,” Learning Disabilities Research and Practice 15 (2000): 55-64. xii

Slavin et al, 6; Lyon et al, 276

xiii

Lyon et al, 266.

xiv

Lyon et al, 266.

110

Endnotes

President’s Commission, Section 2, 3.

xvi

Slavin, Karweit, and Wasik, 2.

xvii

Lyon et al, 272.

xviii

Slavin, Karweit, and Wasik, 2.

xix

Lyon et al, 271.

President’s Commission, Section 2, 4.

xxi

Slavin, Karweit, Wasik, 3.

xxii

Slavin, Karweit, and Wasik, 4.

xxiii

P. Freyd and J. Lytle, “Corporate Approach to the 2 R’s: A Critique of IBM’s Writing to Read Program, Educational Leadership, 47, 6 (1990): 83-89. xxiv

F. Mosteller, “The Tennessee Study of Class Size in the Early School Grades,” The Future of Children, Critical Issues for Children and Youths, 5 (1995): 113-127. xxv

Slavin, Karweit, and Wasik, 3.

xxvi

President’s Commission, Section 2, 3.

xxvii

President’s Commission, Introduction, 3.

xxviii

A.M. Palinscar and A.L. Brown, “Reciprocal Teaching of Comprehension-Fostering and Comprehension Monitoring Activities,” Cognition and Instruction 2 (1984): 17-175. xxix

T.J. Kane and D.O. Staiger, “Volatility in Student Test Scores: Implications for TestBased Accountability Systems,” (2001): 1. xxx

Kane et al, 9.

111

Bibliography

Bibliography Barnett, W.S., and C. M. Escobar. “The Economics of Early Educational Intervention: A Review.” Review of Educational Research 57 (1987): 387-414. Biegel, Stuart. “Paragraph 44 Independent Review, Report No. 19, The Annual Report of the San Francisco Consent Decree Monitoring Team.” 31 July 2002. www.gseis.ucla.edu/courses/edlaw/sfrepts.htm (18 March 2003). Freyd, P. and J. Lytle. “Corporate Approach to the 2 R’s : A Critique of IBM’s Writing to Read Program.” Educational Leadership 47(6)(1990): 83-89. Gabriel, R.M. et al, “Studying the Sustained Achievement of Chapter I Students,” Washington, D.C.: U.S. Department of Education, 1985. Garcia, Eugene E. Making a Difference for Students At Risk: Trends and Alternatives. Edited by Margaret C. Wang and Maynard C. Reynolds. Thousand Oaks, CA: Corwin Press, Inc., 1995. Kane, T. J. and D.O. Staiger. Volatility in School Test Scores: Implications for Test-Based Accountability Systems. Washington, DC: Brookings Institute, 2001. Kirk, S. Educating Exceptional Children. Boston, MA: Houghton Mifflin, 1962. Legters, Nettie and Edward L. McDill. Schools and Students at Risk: Context and Framework for Positive Change. Edited by Robert J. Rossi. New York: Teachers College Press, 1994. Lyon, G.R. “Rethinking Learning Disabilities,” Rethinking Learning Disabilities for a New Century. Washington, DC: Progressive Policy Institute, 1991. Manning, M.L. and Leroy G. Baruth. Students at Risk. Needham Heights, Massachusetts: Allyn and Bacon, 1995. Montgomery, Alesia F. and Robert J. Rossi. Schools and Students at Risk: Context and Framework for Positive Change. Edited by Robert J. Rossi. New York: Teachers College Press, 1994. Mosteller, F. “The Tennessee Study of Class Size in the Early Grades.” The Future of Children, Critical Issues for Children and Youth. 5(1995): 113-127. Mostert, M.P. Interprofessional Collaboration in Schools. Needham Heights, Massachusetts: Allyn and Bacon, 1998.

Bib-1

Bibliography

Palinscar, A.M., A.L. Brown. “Reciprocal Teaching of Comprehension-Fostering and Comprehension Monitoring Activities.” Cognition and Instruction 2 (1984): 17-175. Paris, S.G., K.K. Wixson, and S.S. Palinscar. “Instructional Approaches to Reading Comprehension.” In Review of Research in Education. Edited by E.Z. Rothkof. Washington, D.C.: American Educational Research Association, 1986. Pugach, Marleen C. Making a Difference for Students At Risk: Trends and Alternatives. Edited by Margaret C. Wang and Maynard C. Reynolds. Thousand Oaks, CA: Corwin Press, Inc., 1995. Simmons, Deborah C. et al. “Implementation of a Schoolwide Reading Improvement Model: “No One Ever Told Us It Would Be This Hard!”” Learning Disabilities Research & Practice 15(2) (2000): 92-100. Slavin, Robert E., Nancy Karweit, and Nancy Madden, Effective Programs for Students at Risk, Massachusetts: Allyn and Bacon, 1989. Slavin, Robert E. et al. “Preventing Early School Failure: What Works?” Educational Leadership 50(4) (December 1992 and January 1993). <http://www.ascd.org/readingroom/edlead/9212/slavin.html> (16 November 2002). Tomasi, Susan and Sharon L. Weinberg “Classifying Children as LD: An Analysis of Current Practice in an Urban Setting,” Learning Disability Quarterly 22(1) (1999): 31-42. Torgesen, J.K. “Individual Responses to Early Interventions in Reading: The Lingering Problem of Treatment Registers.” Learning Disabilities Research and Practice. 15(2000): 55-64. Wang, Margaret C. et al. Making a Difference for Students At Risk: Trends and Alternatives. Edited by Margaret C. Wang and Maynard C. Reynolds. Thousand Oaks, CA: Corwin Press, Inc., 1995. U.S. Department of Education Office of Special Education and Rehabilitative Services, A New Era: Revitalizing Special Education for Children and Their Families, Washington, DC, 2002. <http://www.ed.gov/inits/commissionsboards/whspecialeducation/> Wood, D.J., J.S. Bruner, and G. Ross. “The Role of Tutoring in Problem Solving.” Journal of Psychology and Psychiatry 17 (1976): 89-100.

Bib-2

Appendix A

Appendix A List of Interviewees and Site Visits In some cases we have listed the position held, rather than the full name, in order to protect the individualsâ&#x20AC;&#x2122; privacy. Hesperia

Elk Grove

Pasadena

Interviews Dr. James Huckeba Director of Special Education Cara Bergen Principal, Juniper Elementary Andrea Metz Vice Principal Sharon Orrell Principal Special Education Teacher School Board member Bill Tollestrup Director of Special Education Terry deBoer Program Specialist 2 Resource Specialist Teachers

Site Visits District Office

Resource Specialist Teacher

Rutter Middle School

Judith Barhydt Director of Special Education Terry deBoer Program Specialist 2 Resource Specialist Teachers

District Office

Resource Specialist Teacher

Rutter Middle School

udith Barhydt Director of Special Education La Vonne Knedel Program Coordinator Nancy Yandle Resource Specialist Teacher Stephan Brown Vice Principal Anne Oyama Resource Specialist Teacher RSP Aide

District Office

A-1

Cottonwood Elementary Maple Elementary

District Office

Prairie Elementary

Noyes Elementary Longfellow Elementary Field Elementary

Appendix B

Appendix B Demographic Information on Schools Visited Source: CDE Educational Demographics Unit 1/25/03. http://data1.cde.ca.gov/dataquest/ Enrollment and Ethnicity 2001/02 School Year Enrollment % African % Latino American Hesperia Cottonwood 697 2.9 39.5 Maple 627 7.5 41.8 Elk Grove Prairie 1135 34.5 29.5 Rutter 1357 25.9 19.3 Pasadena Field 516 21.7 61.8 Longfellow 843 32.3 56.3 Noyes 495 47.7 31.5

Special Populations 2001/02 School Year % CalWorks % Free/Reduced Meals Hesperia USD Cottonwood 14 Maple 7 Elk Grove USD Prairie 25 Rutter 34 Pasadena USD Field 24 Longfellow 28 Noyes 11

% White/ Non-Hispanic

% Other

54.1 44.7

3.5 6

8.1 19.4

27.9 35.4

11.8 8.9 16

4.7 2.5 4.8

% English Language Learners (ELL)

59 58

7.6 14.7

78 61

42.3 28.4

68 71 48

36.8 29.1 16.2

Appendix B

Teacher Characteristics 2001/02 School Year % Teachers with Full Credential Hesperia USD Cottonwood 94 Maple 100 Elk Grove USD Prairie 95 Rutter Pasadena USD Field 79 Longfellow 86 Noyes 61

Average Years Teaching 9.4 13.3 9.9 15 12.8 12

Appendix C

Appendix C Disability Category Descriptions California Department of Education The thirteen disability categories recognized by state and federal law include: •

Mental Retardation (MR): Mental Retardation means significantly sub-average general intellectual functioning existing concurrently with deficits in adaptive behavior, and manifested during the developmental period, which adversely affects a child's educational performance. (34 CFR Part 300.5).

•

Hard of Hearing (HH): Hard of Hearing means a hearing impairment, whether permanent or fluctuating, which adversely affects a child's educational performance but which is not included under the definition of "deaf" in this section. (34 CFR Part 300.5).

•

Deafness (DEAF): Deafness means a hearing impairment which is so severe that the child is impaired in processing linguistic information through learning, with or without amplification, which adversely affects educational performance. (34 CFR Part 300.5).

•

Hearing Impairment (HI): Hearing Impairment is a federal category of disability which includes both hard of hearing and deaf individuals as defined above.

•

Speech or Language Impairment (SLI): Speech and Language Impairment means a communication disorder such as stuttering, impaired articulation, language impairment, or a voice impairment, which adversely affects a child's educational performance. (34 CFR Part 300.5).

•

Visual Impairment (VI): Visually Impaired means a visual impairment that, even with correction, adversely affects a child's educational performance. The term includes both partially seeing and blind children. (34 CFR Part 300.5).

•

Emotional Disturbance (ED): Emotional Disturbance means a condition exhibiting one or more of the following characteristics over a long period of time and to a marked degree, which adversely affects educational performance: A. An inability to learn which cannot be explained by intellectual, sensory, or health factors; B. An inability to build or maintain satisfactory interpersonal relationships with peers and teachers; C. Inappropriate types of behavior or feeling under normal circumstances; D. A general pervasive mood of unhappiness or depression; or

C-1

Appendix C

E. A tendency to develop physical symptoms or fears associated with personal or school problems. The term (ED) includes children who are schizophrenic. The term does not include children who are socially maladjusted, unless it is determined that they exhibit one or more of the characteristics listed above. (34 CFR Part 300.5). •

Orthopedic Impairment (OI): Orthopedic Impairment means a severe orthopedic impairment which adversely affects a child's educational performance. The term includes impairments caused by congenital anomaly (e.g., clubfoot, absence of some member, etc.), impairments caused by disease (e.g., poliomyelitis, bone tuberculosis, etc.), and impairments from other causes (e.g., cerebral palsy, amputations, and fractures or burns which cause contractures). (34 CFR Part 300.5).

•

Other Health Impairment (OHI): Other Health Impairment means having limited strength, vitality or alertness, due to chronic or acute health problems such as a heart condition, tuberculosis, rheumatic fever, nephritis, asthma, sickle cell anemia, hemophilia, epilepsy, lead poisoning, leukemia, or diabetes, which adversely affects a child's educational performance (34 CFR Part 300.5)

•

Specific Learning Disability (SLD): Specific Learning Disability means a disorder in one or more of the basic psychological processes involved in understanding or in using language, spoken or written, which may manifest itself in an imperfect ability to listen, think, speak, read, write, spell, or to do mathematical calculations. The term includes such conditions as perceptual handicaps, brain injury, minimal brain dysfunction, dyslexia, and developmental aphasia. The term does not include children who have learning problems that are primarily the result of visual, hearing, or motor handicaps, of mental retardation of emotional disturbance or of environmental, cultural, or economic disadvantage. (34 CFR Part 300.5).

•

Deaf-Blindness (DB): Deaf-Blindness means concomitant hearing and visual impairments, the combination of which causes such severe communication and other developmental and educational problems that they cannot be accommodated in special education programs solely for deaf or blind children. (34 CFR Part 300.5).

•

Multiple Disabilities (MD): Multiple Disabilities means concomitant impairments (such as mental retardation, blindness, mental retardation, orthopedic impairment, etc.,) the combination of which causes such severe educational problems that they cannot be accommodated in special education programs solely for one of the impairments. The term does not include deaf-blind children. (34 CFR Part 300.5).

•

Autism (AUT): Autism means a developmental disability significantly affecting verbal and non-verbal communication and social interaction, generally evident before age three, that adversely affects educational performance.

C-2

Appendix C

Characteristics of autism include -- irregularities and impairments in communication, engagement in repetitive activities and stereotyped movements, resistance to environmental change or change in daily routines, and unusual responses to sensory experiences. The term does not include children with characteristics of the disability serious emotional disturbance (SED). If a child manifests characteristics of the disability category "autism" after age three, that child still could be diagnosed as having "autism" if the criteria in the above paragraph are satisfied. (34 CFR Part 300.5). â&#x20AC;˘

Traumatic Brain Injury (TBI): Traumatic Brain Injury means an injury to the brain caused by an external physical force or by an internal occurrence such as stroke or aneurysm, resulting in total or partial functional disability or psychosocial maladjustment that adversely affects educational performance. The term includes open or closed head injuries resulting in mild, moderate, or severe impairments in one or more areas, including cognition; language memory; attention; reasoning; abstract thinking; judgment; problem-solving; sensory, perceptual and motor abilities; psychosocial behavior; physical functions; information processing; and speech. The term does not include brain injuries that are congenital or degenerative, or brain injuries induced by birth trauma. (34 CFR Part 300.5)

C-3

Appendix D

Appendix D Ventura Schools School Elmhurst Foster Loma Vista Montalvo Mound Poinsettia Portola Reynolds Saticoy Sunset

Number of General Education Students Receiving Services 13 41 48 19 23 13 36 15 36 30

D-1

Appendix E

Appendix E Complete List of Schools by District

E-1

Appendix F

Appendix F Data Sources Data for our analysis was derived from the following California Department of Education resources: 1) DataQuest provided information on district and school demographic variables such as enrollment, free or reduced price meals program, English language learners, ethnicity and teacher experience <http://data1.cde.ca.gov/dataquest/>. 2) Ed-Data provided further information on fiscal, demographic and performance data on schools <http://www.ed-data.k12.ca.us/>. 3) Standardized Testing and Reporting (STAR) provided SAT9 test score data for 1998 to 2002 <http://star.cde.ca.gov>. 4) Education Demographics Unit provided databases with enrollment and student demographic variables http://cde.ca.gov/.

F-1

Appendix G

Appendix G Stanford Achievement Test, Ninth Edition, Form T (Stanford 9) In November 1997, the State Board of Education, as required by statute, designated the Stanford 9 (SAT 9) as the achievement test for the California Standardized Testing and Reporting (STAR) Program. The test was first administered to all California students during the spring 1998 and since then has been administered every spring term. School districts are required to administer the test to all students in grades 2 â&#x20AC;&#x201C; 11, with the exception of students receiving special education services with Individual Education Plans (IEPs) that specify an alternate assessment or students whose parents or guardians submit written requests for exemption. Tests are administered to all students during a period that includes 10 instructional days before and 10 instructional days after the day on which 85% of each school, program or trackâ&#x20AC;&#x2122;s instructional year is completed. The Stanford 9 is a national norm-referenced multiple choice achievement test. Because the questions and scoring are the same from year-to-year, results from the 2002 administration can be compared with the results from any of the previous four years. Scores are reported by grade level as national percentile rank (NPR) in three categories: percent scoring above 75th NPR, percent scoring at or above 50th NPR, and percent scoring above 25th NPR. Mean scaled scores are also reported. We focused our analyses on the percent of students scoring at or above the 50th NPR, as this is the percentage of students purported to have demonstrated achievement at or above grade level on this particular test. We also performed our analyses using mean scaled scores, but found no significant differences in the results. We reported our findings according to the gains in percent of students scoring at or above the 50th NPR.

G-1

Appendix H

Appendix H Grade Level and Cohort Analyses Descriptive Statistics

Grade Level Hesperia and Elk Grove with Greater or Equal Gains in Reading in Grades 2 through 4

Percent Gain

District Reading Score Gains Between 1999 and 2002, Grades 2 - 5 30 20 10 0 Hesperia

Elk Grove

Pasadena

Redlands

Ventura

District 2nd

3rd

4th

5th

Elk Grove with Greater Gains than Control Districts in Math in All Grades

Percent Gain

District Math Score Gains Between 1999 - 2002, Grades 2 - 5 30 20 10 0 Hesperia

Elk Grove

Pasadena

Redlands

District 2nd

3rd

H-1

4th

5th

Ventura

Appendix H

Cohort Elk Grove has Greatest Cohort Gain in Math

Percent Gain

District Reading and Math Score Gains Between 1999 and 2002 by Cohort 15 10 5 0 -5 Hesperia

Elk Grove

Pasadena

Redlands

District Reading

H-2

Math

Ventura

Appendix I

Appendix I Correlation Matrix of Regression Variables

Variables ∆ CalWORKS 02-99 / CalWORKS avg ∆ Emergency Cred 02-99 / Emergency avg ∆ Emergency Cred 02-99 / Full Cred avg Full Cred avg / Emergency Cred avg Latino avg / Emergency Cred avg Other avg / ∆ Enrollment 02-99 EL avg / CalWORKS avg EL avg / Latino avg White avg / Black avg White avg / CalWORKS avg White avg / Latino avg Meals avg / CalWORKS avg Meals avg / ∆ CalWORKS 02-99 Meals avg / Latino avg White avg / EL avg Meals avg / EL avg Treatment / Number Tested avg Meals avg / White avg

I-1

Corrlation 0.7155 -0.8236 0.7506 -0.8519 0.6038 0.7042 0.7240 0.6742 -0.7466 -0.7644 -0.6242 0.8672 -0.6545 0.7212 -0.8138 0.7839 0.7540 -0.8158

Appendix F MASTER SCHOOL LIST County County Name District District Name School School Name 34 Sacramento 67314 Elk Grove Unified 3430170 Valley High 34 Sacramento 67314 Elk Grove Unified 3430311 Calvine High (Cont.) 34 Sacramento 67314 Elk Grove Unified 3430329 Rio Cazadero High (Cont.) 34 Sacramento 67314 Elk Grove Unified 3430352 Las Flores High (Alter.) (2-11) 34 Sacramento 67314 Elk Grove Unified 3430477 Florin High 34 Sacramento 67314 Elk Grove Unified 3430527 Insights High (Cont.) 34 Sacramento 67314 Elk Grove Unified 3430535 Transition High (Cont.) 34 Sacramento 67314 Elk Grove Unified 3430592 Laguna Creek High 34 Sacramento 67314 Elk Grove Unified 3430618 Sheldon High 34 Sacramento 67314 Elk Grove Unified 3430725 Elk Grove Community Day 34 Sacramento 67314 Elk Grove Unified 3430733 Capital Community Day 34 Sacramento 67313 Elk Grove Unified 3430741 Advanced Instruction 34 Sacramento 67314 Elk Grove Unified 3430873 Franklin 34 Sacramento 67314 Elk Grove Unified 3432002 Daylor (William) High (Cont.) 34 Sacramento 67314 Elk Grove Unified 3432572 Elk Grove High 34 Sacramento 67314 Elk Grove Unified 6032981 Kirchgater (Anna) Elementary 34 Sacramento 67314 Elk Grove Unified 6032999 Baker (Jessie) Elementary 34 Sacramento 67314 Elk Grove Unified 6033005 Mack (Charles E.) Elementary 34 Sacramento 67314 Elk Grove Unified 6033013 Cosumnes River Elementary 34 Sacramento 67314 Elk Grove Unified 6033021 Reese (David) Elementary 34 Sacramento 67314 Elk Grove Unified 6033039 Dillard Elementary 34 Sacramento 67314 Elk Grove Unified 6033047 Elk Grove Elementary 34 Sacramento 67314 Elk Grove Unified 6033054 Florin Elementary 34 Sacramento 67314 Elk Grove Unified 6033062 Franklin Elementary 34 Sacramento 67314 Elk Grove Unified 6033088 McKee (James A.) Elementary 34 Sacramento 67314 Elk Grove Unified 6033096 Pleasant Grove Elementary 34 Sacramento 67314 Elk Grove Unified 6033104 Kennedy (Samuel) Elem. 34 Sacramento 67314 Elk Grove Unified 6033112 Sierra-Enterprise Elementary 34 Sacramento 67314 Elk Grove Unified 6059174 Rutter (James) Middle 34 Sacramento 67314 Elk Grove Unified 6061808 Kerr (Joseph) Middle 34 Sacramento 67314 Elk Grove Unified 6077291 Leimbach (Herman) Elementary 34 Sacramento 67314 Elk Grove Unified 6098743 Markofer (Florence) Elementary 34 Sacramento 67314 Elk Grove Unified 6098750 Prairie Elementary 34 Sacramento 67314 Elk Grove Unified 6101844 Feickert (Ellen) Elementary 34 Sacramento 67314 Elk Grove Unified 6106355 Jackson (Isabelle) Elementary 34 Sacramento 67314 Elk Grove Unified 6107700 Foulks Ranch Elementary

Notes Not used - high school only Not used - high school only Not used - high school only Not used - data not available for all years Not used - high school only Not used - high school only Not used - high school only Not used - high school only Not used - high school only Not used - high school only Not used - high school only Not used - data not available for all years Not used - high school only Not used - high school only Not used - high school only No data available, for kids with severe disabilities

Not used - middle school only Not used - middle school only

1 Master School List

Appendix F MASTER SCHOOL LIST County County Name District District Name School School Name 34 Sacramento 67314 Elk Grove Unified 6107718 Union House Elementary 34 Sacramento 67314 Elk Grove Unified 6107916 Reith (John) Elementary 34 Sacramento 67314 Elk Grove Unified 6109516 Butler (Arthur C.) Elem. 34 Sacramento 67314 Elk Grove Unified 6109821 Jackman (Samuel) Middle 34 Sacramento 67314 Elk Grove Unified 6110118 Ehrhardt (John) Elementary 34 Sacramento 67314 Elk Grove Unified 6110985 Tsukamoto (Mary) Elementary 34 Sacramento 67314 Elk Grove Unified 6112031 Eddy (Harriet G.) Middle 34 Sacramento 67314 Elk Grove Unified 6112106 Donner (Elitha) Elem. 34 Sacramento 67314 Elk Grove Unified 6112254 Elk Grove Charter (3-11) 34 Sacramento 67314 Elk Grove Unified 6113179 Morse Elementary 34 Sacramento 67314 Elk Grove Unified 6113187 Beitzel Elementary 34 Sacramento 67314 Elk Grove Unified 6113831 Smedberg (T. R.) Middle 34 Sacramento 67314 Elk Grove Unified 6116818 Sims (Joseph) Elementary 34 Sacramento 67314 Elk Grove Unified 6117063 Crossroads #1 34 Sacramento 67314 Elk Grove Unified 6117071 Crossroads #2 34 Sacramento 67314 Elk Grove Unified 6117089 Las Flores Community Day 34 Sacramento 67314 Elk Grove Unified 6118046 Case Elementary 34 Sacramento 67314 Elk Grove Unified 6118053 Stone Lake Elementary 34 Sacramento 67314 Elk Grove Unified 6120000 Johnson, Toby (Middle School) 34 Sacramento 67314 Elk Grove Unified 6120018 West, Irene B. 34 Sacramento 67314 Elk Grove Unified 6120026 Fite 34 Sacramento 67314 Elk Grove Unified 6120034 Elliott Ranch Elementary 36 36 36 36 36 36 36 36 36 36 36 36 36

San Bernardino San Bernardino San Bernardino San Bernardino San Bernardino San Bernardino San Bernardino San Bernardino San Bernardino San Bernardino San Bernardino San Bernardino San Bernardino

75044 75044 75044 75044 75044 75044 75044 75044 75044 75044 75044 75044 75044

Hesperia Unified Hesperia Unified Hesperia Unified Hesperia Unified Hesperia Unified Hesperia Unified Hesperia Unified Hesperia Unified Hesperia Unified Hesperia Unified Hesperia Unified Hesperia Unified Hesperia Unified

3630407 3630472 3630746 3630811 3630944 3631132 6035943 6035950 6035968 6059547 6089643 6100937 6103337

Hesperia High Mojave High (Cont.) Sultana High Desert Trails High (Alt) (2-11) Hesperia Community Day Crosswalk Joshua Circle Elementary Juniper Elementary Eucalyptus Elementary Hesperia Junior High Mesa Grande Elementary Kingston Elementary Maple Elementary

Notes

Not used - data not available for all years Not used - middle school only

Not used - middle school only Not used - data not available for all years

Not used - middle school only Not used - data not available for all years Not used - data not available for all years Not used - data not available for all years Not used - data not available for all years Not used - data not available for all years Not used - data not available for all years Not used - middle school only Not used - data not available for all years Not used - data not available for all years Not used - data not available for all years Not used - high school only Not used - high school only Not used - high school only Not used - data not available for all years Not used - high school only Not used - high school only

Not used - middle school only

2 Master School List

Appendix F MASTER SCHOOL LIST County 36 36 36 36 36 36 36

County Name San Bernardino San Bernardino San Bernardino San Bernardino San Bernardino San Bernardino San Bernardino

District 75044 75044 75044 75044 75044 75044 75044

District Name Hesperia Unified Hesperia Unified Hesperia Unified Hesperia Unified Hesperia Unified Hesperia Unified Hesperia Unified

School 6105498 6106454 6108112 6108120 6109359 6111751 6114680

School Name Cottonwood Elementary Lime Street Elementary Hollyvale Elementary Carmel Elementary Ranchero Middle Topaz Elementary Mesquite Trails (Elem)

Notes

19 19 19 19 19 19 19 19 19 19 19 19 19 19 19 19 19 19 19 19 19 19 19 19 19 19 19 19

Los Angeles Los Angeles Los Angeles Los Angeles Los Angeles Los Angeles Los Angeles Los Angeles Los Angeles Los Angeles Los Angeles Los Angeles Los Angeles Los Angeles Los Angeles Los Angeles Los Angeles Los Angeles Los Angeles Los Angeles Los Angeles Los Angeles Los Angeles Los Angeles Los Angeles Los Angeles Los Angeles Los Angeles

64881 64881 64881 64881 64881 64881 64881 64881 64881 64881 64881 64881 64881 64881 64881 64881 64881 64881 64881 64881 64881 64881 64881 64881 64881 64881 64881 64881

Pasadena Unified Pasadena Unified Pasadena Unified Pasadena Unified Pasadena Unified Pasadena Unified Pasadena Unified Pasadena Unified Pasadena Unified Pasadena Unified Pasadena Unified Pasadena Unified Pasadena Unified Pasadena Unified Pasadena Unified Pasadena Unified Pasadena Unified Pasadena Unified Pasadena Unified Pasadena Unified Pasadena Unified Pasadena Unified Pasadena Unified Pasadena Unified Pasadena Unified Pasadena Unified Pasadena Unified Pasadena Unified

1931062 1931674 1932409 1936103 1936806 1936822 1996248 6021497 6021505 6021539 6021547 6021554 6021562 6021570 6021588 6021612 6021620 6021638 6021653 6021661 6021679 6021687 6021703 6021711 6021729 6021737 6021752 6021760

Blair High Marshall Fundamental (6) Coombs (Norma) Alt (K-8) Muir High Rose City High (Cont.) Pasadena High Pasadena Community Prep Allendale Elementary Altadena Elementary Burbank Elementary Cleveland Elementary Don Benito Fundamental Edison Elementary Field Elementary Franklin Elementary Hamilton Elementary Jackson Elementary Jefferson Elementary Linda Vista Elementary Loma Alta Elementary Longfellow Elementary Madison Elementary Noyes Elementary Roosevelt Elementary San Rafael Elementary Sierra Madre Elementary Washington Middle Webster Elementary

Not used - high school only Not used - middle school

Not used - middle school only

Not used - high school only Not used - high school only Not used - high school only No data available

Not used - middle school

3 Master School List

Appendix F MASTER SCHOOL LIST County County Name District District Name School School Name 19 Los Angeles 64881 Pasadena Unified 6021778 Willard Elementary 19 Los Angeles 64881 Pasadena Unified 6119549 Washington Accelerated 19 Los Angeles 64881 Pasadena Unified 6058465 Eliot Middle 19 Los Angeles 64881 Pasadena Unified 6058499 Wilson Middle 19 Los Angeles 64881 Pasadena Unified 6119549 Washington Accelerated 19 Los Angeles 64881 Pasadena Unified 6120265 McKinley 56 56 56 56 56 56 56 56 56 56 56 56 56 56 56 56 56 56 56 56 56 56 56 56 56 56 56 56 56

Ventura Ventura Ventura Ventura Ventura Ventura Ventura Ventura Ventura Ventura Ventura Ventura Ventura Ventura Ventura Ventura Ventura Ventura Ventura Ventura Ventura Ventura Ventura Ventura Ventura Ventura Ventura Ventura Ventura

72652 72652 72652 72652 72652 72652 72652 72652 72652 72652 72652 72652 72652 72652 72652 72652 72652 72652 72652 72652 72652 72652 72652 72652 72652 72652 72652 72652 72652

Ventura Unified Ventura Unified Ventura Unified Ventura Unified Ventura Unified Ventura Unified Ventura Unified Ventura Unified Ventura Unified Ventura Unified Ventura Unified Ventura Unified Ventura Unified Ventura Unified Ventura Unified Ventura Unified Ventura Unified Ventura Unified Ventura Unified Ventura Unified Ventura Unified Ventura Unified Ventura Unified Ventura Unified Ventura Unified Ventura Unified Ventura Unified Ventura Unified Ventura Unified

5630264 5630272 5630280 5630298 5630348 5630793 5637822 6056030 6056055 6056063 6056089 6056097 6056105 6056113 6056121 6056147 6056154 6056162 6056170 6056188 6056204 6056212 6056238 6060370 6060388 6062145 6062152 6097034 6115687

Buena Vista High (Cont.) Pacific High (Cont.) (6-11) Ventura Islands High El Camino High (Alter). Foothill Technology High Buena High Ventura High Sunset Element Blanche Reynolds Elementary Foster (E.P.) Elementary Elmhurst Elementary Juanamaria Elementary Serra (Junipero) Elementary Lincoln Elementary Loma Vista Elementary Montalvo Elementary Mound Elementary Oak View Elementary Pierpont Elementary Poinsettia Elementary Saticoy Elementary Sheridan Way Elementary Will Rogers Elementary Balboa Middle Cabrillo Middle Anacapa Middle De Anza Middle Portola Elementary Homestead (Alternative)

Notes Not used - data not available for all years Not used - middle school Not used - middle school Not used - middle school No data available Not used - high school only Not used - data not available for all years Not used - high school only Not used - high school only Not used - high school only Not used - high school only Not used - high school only Not used - data not available for all years

Not used - data not available for all years

Not used -middle school only Not used -middle school only Not used -middle school only Not used -middle school only Not used - data not available for all years 4 Master School List

Appendix F MASTER SCHOOL LIST County County Name District 56 Ventura 72652 36 36 36 36 36 36 36 36 36 36 36 36 36 36 36 36 36 36 36 36

San Bernardino San Bernardino San Bernardino San Bernardino San Bernardino San Bernardino San Bernardino San Bernardino San Bernardino San Bernardino San Bernardino San Bernardino San Bernardino San Bernardino San Bernardino San Bernardino San Bernardino San Bernardino San Bernardino San Bernardino

67843 67843 67843 67843 67843 67843 67843 67843 67843 67843 67843 67843 67843 67843 67843 67843 67843 67843 67843 67843

District Name Ventura Unified Redlands Unified Redlands Unified Redlands Unified Redlands Unified Redlands Unified Redlands Unified Redlands Unified Redlands Unified Redlands Unified Redlands Unified Redlands Unified Redlands Unified Redlands Unified Redlands Unified Redlands Unified Redlands Unified Redlands Unified Redlands Unified Redlands Unified Redlands Unified

School School Name 6116040 Citrus Glen 6108179 6111132 6059414 6059422 6036479 6036487 6036495 6036503 6036511 6067060 6036537 6036545 6036552 6036560 6061881 3634995 3630779 3635042 6036586 6036594

Arroyo Verde Elementary Bryn Mawr Elementary Clement Middle Cope Middle Crafton Elementary Cram Elementary Fallsvale Elementary Franklin Elementary Kimberly Elementary Kingsbury Elementary Lugonia Elementary Mariposa Elementary McKinley Elementary Mentone Elementary Moore Middle Orangewood High (cont.) Redlands East Valley High Redlands Senior High Smiley Elementary Victoria Elementary

Notes Not used - data not available for all years

Not used - middle school Not used - middle school

Not used - data not available for all years

Not used - middle school Not used - data not available for all years Not used - high school only Not used - high school only

5 Master School List