EDUCATION Reading school report cards for all they’re worth

BY LAUREN ANDERSON, for BizTimes

IF YOU WANT TO KNOW how Milwaukee schools are doing, your first stop might be to pull up the school report cards compiled and released by the Wisconsin Department of Public Instruction.

The DPI’s report cards annually publish the performance of traditional public schools and districts, charter schools, and private schools participating in the parental choice program. As the bedrock of the state’s accountability system, the report cards offer a means of comparing schools across Milwaukee’s three school sectors.

But for those outside the education realm, reading and interpreting the report cards can be confusing. In recent years, changes to the computational system underpinning them and pandemic-related interruptions to the data have further complicated matters. More than a plain reading is required to understand the value – and limitations – of the report cards.

WHAT DO THE REPORT CARDS MEASURE?

The report cards are an aggregation of a variety of school performance measures largely drawn from students’ performance on state and federally mandated standardized tests.

Each school and district is assigned an overall score on a scale of 0 to 100, corresponding to five rating categories that range from “Significantly Exceeds Expectations” to “Fails to Meet Expectations.” Those categories are also associated with a five-star rating system.

The overall score is determined by the weighted average of four factors: a school’s achievement

(how much students know, based on state testing); growth (how much students progressed over the course of a year, including the pace of improvement); target group outcomes (how the lowest-scoring students are performing on achievement, growth, absenteeism, attendance and graduation); and how well students are reaching certain milestones that predict whether they will graduate on time.

Academic outcomes on the report cards are based on K-8 students’ performance on the annual Forward exam and 9-12 grade students’ performance on the ACT/ACT Aspire exams.

HOW FAR DOES THE DATA GO BACK?

Public schools were first issued report cards in the 2011-‘12 school year and private schools with at least 20 parental choice program students were added in 2015-‘16, as required by state law. One benefit of the report cards is the creation of a consistent reporting system that has allowed for more meaningful comparison among choice, charter and traditional public schools.

For a city with a fragmented education system, having data for all three sectors is essential for accountability, said Abby Andrietsch, president and chief executive officer of St. Augustine Preparatory Academy, a private voucher school in Milwaukee.

“We have more clear information today than, for sure, we had 15 years ago across all three sectors,” said Andrietsch, who also founded and formerly led Milwaukee education nonprofit Schools That Can Milwaukee. “For voucher schools and

Reading a school report card

Department of Public Instruction school report cards contain a lot of information. Here are a few tips for making sense of the data:

• Start with the top-line overall accountability score, but make sure you look beyond that number and whether a school meets expectations.

• Look at both the achievement and growth scores. The former tells you what students know; the latter is a measure of progress.

• Dig into how subgroups of students perform on achievement and growth. Are there gaps in proficiency between white and Black students?

• Exercise caution in making comparisons to prior years. The COVID-19 pandemic altered test participation and formula changes have been made.

“There is no substitute to going in, talking to teachers, talking to school leaders, getting a feel for a building, hearing and seeing what’s going on,” said Colleston Morgan, vice president of strategy and policy for City Forward Collective. “There’s no amount of data or numbers on a paper that will substitute for the hard work and the little bit of magic, quite frankly, inside the school building and also the context of the challenges in which the school is operating.” charter schools to hold ourselves accountable to quality in the same way we’re asking public schools to hold themselves accountable is really important.”

Beginning in the 2019-‘20 school year, the pandemic brought significant disruptions, resulting in a pause in testing. No school report cards were produced that year. The following year saw historically low participation rates – with about 50% of students citywide taking the tests – which further obscured year-over-year comparisons.

DPI also rolled out changes to the report cards in 2020-‘21, including alterations to the formula that determines overall scores and cosmetic changes to the layout of the cards. Notably, the overall score ranges that determine a school’s star rating were adjusted. Previously, an overall score of 63 or higher was needed to “meet expectations.” With the change, that threshold was lowered to 58. The net result: despite higher overall scores on the report cards, student proficiency rates have not actually increased over the past six years.

All of this makes it difficult to hold up schools’ overall scores from 2021 forward against previous years’ scores.

“Essentially, if you’re looking at report cards today, you can’t compare these report cards post-pandemic with report cards from before the pandemic,” said Spencer Schien, senior manager of data and analytics for City Forward Collective. “It’s really not a great comparison, especially if you’re looking at overall scores.”

WHY ARE REPORT CARDS CONTROVERSIAL?

One of the most controversial aspects of the report cards is methodological, particularly the formula that determines a school’s overall score.

The weighting of the achievement and growth components vary, depending on the percentage of economically disadvantaged students in a school. For schools with 65% or more low-income students, growth is weighted at nine times the rate of achievement. In most Milwaukee schools, the achievement score counts for 5% of a school’s overall score, while growth counts for 45%.

By comparison, in neighboring Whitefish Bay, where 1.4% of students are economically disadvantaged, that weighting is reversed: achievement is weighted at 45% of the overall score, while growth is weighted at 5%.

Variable weighting attempts to recognize the more challenging work high-poverty districts –where students are much more likely to begin the school year behind grade level – have in making up large achievement gaps among students within a year’s time, said Colleston Morgan Jr., vice president of strategy and policy for City Forward Collective.

“Both things matter. I’m a parent; I care about how much my children learn in a year, but I also care that my kids are on grade level,” Morgan said. “I think the intention was a good one, to make sure we were both recognizing that students are starting at different places and schools have different levels of challenge in, based on that starting point, in bringing students up to grade level.”

Compared to neighboring states, Wisconsin is an outlier in how heavily growth is weighted versus proficiency among low-income schools and districts.

Highlighting the disconnect, a 2022 City Forward Collective report found six out of 10 Milwaukee students are enrolled in schools that meet expectations, though fewer than one in five are able to read, write and do math on grade.

“Because it’s so high, we ended up in a place where you can have schools that have done good work – and it’s well recognized in the growth measure – but the vast majority of students are not meeting grade level expectations in math or in ELA (English Language Arts). And yet those schools are receiving ratings of ‘meeting expectations,’ despite the fact that maybe only a handful of students in the school are reading and writing and doing math at grade level,” Morgan said.

Some have called for a recalibration of the growth/proficiency weighting. The Metropolitan Milwaukee Association of Commerce and City Forward Collective have each put forth policy recommendations this year seeking a more accurate picture of student outcomes, advocating for a better balance of the report card’s achievement and growth components.

“We have, at this point, almost a decade’s worth of data from Wisconsin, and we have examples from other states to be able to better calibrate the weighting of those two measures in a way that can both recognize the importance of proficiency and growth,” said Morgan.

WHAT REPORT CARDS DON’T SHOW

Neither purely quantitative nor purely qualitative assessments fully capture the picture of a school’s performance, education leaders argue. Test scores offer a snapshot of how much students are learning but can’t address questions of, for example, how safe they feel in the classroom. Subjective testimonials, on the other hand, can’t answer how many students are at grade level for reading and writing.

As founding board chair of the National Alliance for Public Charter Schools in 2005, education reform advocate Howard Fuller was on the ground floor of conversations about tying test scores to school performance and accountability for the growing school choice movement.

“What happened was … test scores became the dominant and consummate way to understand the value of a school. And I believe that was a mistake,” he said. “Because we know we are in a situation where people look at test scores, but they don’t necessarily look at value added.”

As founder of Dr. Howard Fuller Collegiate Academy, a public charter school in Milwaukee in

Top-ranked Milwaukee private, public and charter schools by DPI report cards

which 80% of students are economically disadvantaged, Fuller said report cards hold schools accountable for factors outside their purview.

“For example, how do you hold schools responsible for attendance? There’s an argument that, if the school is really doing great things, kids will come. That’s not a real thing. Because if a parent wakes up and says, ‘I’m not sending my kid to school today because I want her to take care of her younger brothers,’ you have no control over that as a school, but yet you’re held responsible for attendance,” Fuller said.

With its students coming in multiple grades behind, DHFCA is focused on getting students on grade level and accepted into college. School report cards, Fuller said, aren’t able to fully capture this work.

“If you look at our school, I don’t think our school is a high-performing school,” he said. “But I’m not trying to be a high-performing school. I’m trying to have a school that will change children’s lives, that will put them on a trajectory that gives them a chance to be socially and economically productive citizens.”

Report cards can lend themselves to questions about a school’s outcomes, but others are also worth considering, Morgan said.

“Do I see positive interactions between teachers and families? Between teachers and students? Between principal and others?” he said. “Those types of qualitative data points matter immensely in any field, but they really matter when we’re talking about children and families and their educational journey.” n

What We Do

What is M3 ? It’s a partnership among Milwaukee’s three largest public education institutions dedicated to transforming the future of our city through education. In 2015, these three institutions came together to build seamless connections from high school to college, beginning with goals for college credit while still in high school, campus visits and tours, parent support, and financial aid navigation. M3 is witnessing the rewards Milwaukee students are gaining through three key impact areas: Connecting Learning, Transitioning, and Driving Completion.

Connecting Learning

• Every student builds an academic and career plan beginning in elementary school.

• Educators from MPS, MATC and UWM collaborate to build connections and make transitions seamless from elementary school through higher education.

• Joint professional development enhances culturally responsive teaching practices so instructors better understand our students and can deliver learning that is personally meaningful.

Transitioning

• MPS students engage in experiences and programs at MPS, MATC and UWM that support the transition to college.

• The College Connections dual enrollment program, which began in 2019, has enabled about 400 MPS students to earn more than 5,000 college credits while still in high school – a combined cost savings of $1.5 million.

• The Smart Start transition program, launched in 2022, engages with MPS students prior to graduation, provides summer programs on the MATC or UWM campus, and offers support through the first year of college.

Driving Completion

• Students learn how to succeed through advising, support services, and opportunities for learning outside the classroom.

• Student college and career fairs help students start thinking about life after high school and after higher education.

• A vigorous campaign of events and supports helps students complete the Free Application for Federal Student Aid (FAFSA).

M3 (pronounced M-cubed) is transforming the future of our young people and Milwaukee communities through the power of public education by multiplying the impact of Milwaukee Public Schools, Milwaukee Area Technical College, and the University of Wisconsin-Milwaukee. FOR MORE INFORMATION, VISIT UWM.EDU/M-CUBED OR EMAIL M-CUBED@UWM.EDU