Is your diversity training getting the results that you want — and that your employees (and prospective employees) deserve? Do you even have a system for reviewing the effectiveness of your training?

Unfortunately, for all the resources that organizations are putting behind these commendable efforts, too often we really don’t know how well they are working.

That’s the troubling conclusion reached by Betsy Levy Paluck, and she is in a position to know. A behavioral scientist and professor of psychology at Princeton University who studies prejudice and behavior change, Paluck in 2021 led a team that reviewed the academic literature on the reduction of prejudice. What the team found was not encouraging.

U.S. companies have spent billions of dollars on diversity, equity and inclusion (DEI) programs since 2020, but they are still trying to figure out how to make sure the programs they are funding actually help. Paluck says American companies paid $3.4 billion to outside firms in 2020 to develop and implement DEI programs. Writing in Fortune, Joan Williams of the University of California’s Center for WorkLife Law reports that companies spend some $8 billion a year on DEI programs, but with “remarkably few results” that can be quantified.

Paluck, in The Washington Post, says this absence of quantifiable improvements in the areas of DEI for all these expenditures leaves companies vulnerable to political attack, “often by people who believe those shouldn’t be goals in the first place.” This pushback of course can threaten the existence of the programs themselves.

The problem seems to be one of insufficiently grounded evaluations of these programs, and the methods of evaluation can be improved.

In its review of the literature, Paluck’s team looked at studies of programs

that used random assignment and control groups, as you would use to check the effectiveness and safety of a drug. Out of hundreds of studies evaluating prejudice reduction programming from the past decade, only two large studies tracked the effects of diversity training. Most diversity training evaluations look like customer satisfaction surveys (“How much did you appreciate this?”) or elementary school worksheets (“Tell me what you learned today about stereotyping”).

‘Untested Drugs’

Few new studies have been done since Paluck’s review, and these have shed almost no light on the problem and how to fix it. Bottom line: “We’re treating a pandemic of discrimination and racial and religious resentment with untested drugs.”

The way forward, of course, is not to allow the pandemic to continue but to test the drugs. And you don’t need to wait for comprehensive global studies of what others are doing to begin your own testing of your own initiatives.

Paluck recommends rolling out programs over time to divisions of the company that are randomly chosen, “comparing trained vs. untrained workers while eventually providing it to everyone. You can track whether the training is followed by more hiring, promotion and retention of underrepresented workers.”

You can even measure the frequency with which minority and female employees speak up in meetings — and to what degree their contributions are listened to — by tracking these exchanges using online meeting software.

Finally, you can survey employees “about racial or gender attitudes among their colleagues, rather than asking them to report their own biases.”

Williams, author of Bias Interrupted: Creating Inclusion for Real and for Good, also believes small changes can have big impacts. Simple “evidence-based tweaks” to existing programs “can make significant DEI gains in short order.” When employees’ performance evaluation forms are improved, for example, the results can be significant and almost immediate.

‘Change Business Systems’

A bigger step, she writes, is giving DEI heads power and not just a title; too often they are paid well but are not positioned for success. They need to be able “to work with human resources to change business systems in order to interrupt bias and level the playing field.”

But make no mistake. Effective DEI programs are not nice add-ons. They are requirements, and it is urgent that we find out what works and what doesn’t. Because, Paluck warns, “if we don’t study what works when it comes to diversity initiatives, we know what will almost surely follow: another crime of hate, followed by a surge in diversity trainings that might not help at all.”