Appraisal Bias? The Inconvenient Truth of a Flawed Report

by Isaac Peck, Publisher

You can’t always judge a book by its cover, or a report by its summary results. At least not without some scrutiny.

In the summer of 2022, in Baltimore, Maryland, the National Community Reinvestment Coalition (NCRC) conducted a “mystery shopper experiment” with the item being a residential appraisal. It concluded that appraisers are racially biased. Here’s what a lazy analysis failed to find.

The NCRC’s experiment had biracial couples engage local appraisers (who knew nothing of the experiment) to perform an appraisal of their home as a way of testing how appraisers would value the home based on race. In half the tests, the Black spouse answered the door and acted as if they needed an appraisal for a divorce or loan. The other half had the white spouse answer the door. In addition to a Black or white spouse, each house was “blackwashed” or “whitewashed” (as the NCRC puts it), meaning family photographs and cultural objects were removed to make the home racially neutral. Twelve fully-completed “matched pair” appraisal reports (from three homes) were ordered and delivered as part of the experiment.

aways of the data, NCRC writes, is that appraisers assigned higher values to homes when they believe the owners are white ($7,000 higher per property). The report states that the two “most glaring” examples are 1) a home valued 12.9 percent higher when the white spouse presented ($40,000 variance), and 2) a home valued 9.1% higher with a white spouse ($46,000 variance).

What the Report Doesn’t Say

One key data point that the NCRC does not disclose is that the two most glaring examples of racial bias were performed by the same appraiser. We might call this appraiser an “outlier,” which in statistics is defined as “an observation that lies an abnormal distance from other values in a random sample from a population.”

In fact, if we remove this one appraiser as a data point, the data flips the other way! The results of NCRC’s experiment actually show that appraisers (on average) favor Black homeowners and assign their homes higher values.

But perhaps more glaring is that the one “outlier” appraiser, who was so pivotal in skewing the data, is Black!

The results from NCRC’s experiment were revealed in an October 2022 report titled Mystery-Shopper Testing In Home Appraisals

Exposes Racial Bias

Undermining Black Wealth, which has been widely cited and reported on by both local and national news outlets.

The conclusions the NCRC draws (https://bit.ly/ncrcmystery ) have been shared by the media and are obvious in the title: home appraisals are racially biased and undermine Black wealth. Specifically, one of the key take-

[Name Redacted], the appraiser who “undervalued” the first house in the NCRC’s “most glaring” example, and who “overvalued” the second house in the NCRC’s other key example, is a Black appraiser.

Blind to Data

Whether the NCRC failed to dig deeper or ignored what they found, this reveals a flawed conclusion. Here too, the NCRC filed a discrimination complaint with the Department of Housing and Urban Development (HUD) against [Name Redacted] as a result of this experiment.

The HUD complaint against the Black appraiser reads: “There is a significant disparity between how Respondent-valued a home that was presented as Black-owned, and how Respondent valued a home that was presented as White-owned on multiple occasions. This disparity constitutes a significant difference in treatment based on race.” (https://bit.ly/HUD903form )

HUD has not responded to WRE as of this writing. The Black appraiser in question performed a total of two appraisals as part of the experiment.

Service Issues

Besides the NCRC’s data analysis issues explained above, the report also indicates that Black homeowners experienced “unprofessional treatment from appraisers,” while white homeowners did not. “One appraiser refused to complete an assignment for a Black homeowner. Another took 11 weeks to complete a report for a Black homeowner,” the NCRC writes.

Here too, the NCRC filed a discrimination complaint with HUD against the appraiser who took 11 weeks to complete a report for a Black homeowner (actor).

When Working RE investigated, we learned that the slow appraiser called out in the report had a pretty good reason: he had experienced a medical emergency—which affected all of his reports during this time frame. The appraiser also reported that the homeowner/actor (and the NCRC) never asked him why the report was late before filing a HUD complaint against him.

It is also worth noting that this appraiser who turned in the report late actually valued the Black (actor) homeowner’s home at a higher value than his appraiser peers. This is another inconvenient truth the NCRC fails to include in their report.

As to the appraiser who refused to complete the report, they may have seen something fishy and decided to steer clear. Not a bad idea, given that the NCRC filed discrimination complaints against two out of six appraisers who participated in the study (a 33 percent “hit rate,” according to WRE’s simple math).

Total Variance

Setting aside the fact that the two most egregious examples of racial bias were performed by the same (Black) appraiser and that with his data removed from the analysis the results actually show favoritism toward Black homeowners, the total variance of the basic data NCRC collected is worth looking at.

First, the NCRC reports that out of its twelve matched paired appraisals (six tests, two appraisals per test), the total of the valuations provided to the white homeowners is $2,418,000 and the total of the valuations provided to the Black homeowners is $2,377,000.

This is a 1.7 percent variance in valuations of the same homes, which is within the historic 5 percent variance range for appraisals. Regardless of the facts, NCRC writes that the results are indicative of “the severity of racial bias” within the profession and supports their argument that “sweeping and aggressive action” is needed from policymakers and the industry itself.

Conclusion

The NCRC’s report and experiment is a great example of how selectively presenting data and filtering statistics can misrepresent the reality of a situation.

The fact that NCRC has “more than 20 years of experience executing mystery shopper testing in housing and lending markets to probe for discriminatory business practices,” and has failed to disclose that both of the “most glaring” appraisal examples of racial bias in their experiment were performed by one appraiser—an outlier—and that this appraiser was in fact a Black appraiser, raises very serious questions about the NCRC’s credibility.

The NCRC, however, considers the experiment a glowing success. In an online webinar, Director of Fair Housing at NCRC, Tracy McCracken, touted this as the first experiment of its kind and said its testing methodology was a promising tool to root out “racist” appraisers.

In the same webinar, Jake Lilien, NCRC’s Counsel for Fair Housing Enforcement, recommended additional testing specifically in Alabama, Georgia, Louisiana, Mississippi, North Carolina, and South Carolina.

“There are certain things I don’t want to say publicly about how we did the testing, because who is to say we’re done testing appraisers. Other groups may try to use our same practices and we don’t want appraisers to know what these practices are. So, I can’t share specific details about how we contacted appraisers, what excuses we gave, etc. But if you work for a Fair Housing organization, reach out to me and I’d love to talk to you,” Lilien said. Their methodology is so good, they’re eager to share it.

In March 2023, HUD issued over $40 million in grants dedicated to “testing” for fair housing complaints, with a specific note about testing for “appraisal bias.” NCRC received $425,000.

Stay safe out there! WRE

IMPORTANT: Now, more than ever, appraisers need a team with deep expertise in their corner. Specializing in appraiser insurance and defense for over 20 years, OREP provides up to $500,000 of discrimination claim coverage for appraisers. Visit OREP.org/ appraisers to get a quote today.