Scientific American column on Duarte et al

Research psychologist Piercarlo Valdesolo wrote a nice guest column on our BBS paper over at Scientific American.

Scientific American doesn't let me comment on their website, rejecting it as spam, even on a new account. This has been true for months – I'm not sure what's triggering the spam flag. Maybe length. I've pasted a greatly expanded version of my comment on the Valdesolo column below:

-------------------------------------------------

Joe Duarte here. Thanks Piercarlo for the generous coverage.

I agree that pitting competing biases against each other is not the right approach, and I regret the line from our paper where we say "but we can diversify the field to the point where individual viewpoint biases begin to cancel each other out."

That doesn't quite capture what we mean. Mostly we mean peer review. If politically biased researchers knew that a conservative, libertarian, or even an alert liberal were likely to review their paper, I think it would change things.

That said, I don't think scientists need to have political identities, and I think it might be better if fewer of us did. I regret that we even need to discuss the political ideologies of researchers in a scientific field, but the biases in social psychology are quite evident and force our hand. And in a world like ours, certainly in an environment like contemporary academia, researchers who don't have notable political identities will still make all sorts of implicit assumptions that shape and frame their research questions, and those assumptions might be traced to a political ideology.

Ideological bias seems to be a special kind of bias. It runs deep and rides on fierce tribal mechanisms. We can see it in some of the comments here – people don't seem to understand the nature of political bias in social science, and how profoundly it can affect what people think is true, what people consider "facts."

For example, one commenter said: "Social psychology research shows that political liberals tend to be more open-minded and less enamoured of authority..."

That's a perfect example. When people consider whether social psychology is biased, they really ought to zoom back and think about how constructs are created and labeled, how ideological and cultural biases can be embedded in measures. In this case, the measures we use for the variables the commenter invoked are themselves deeply biased, and I think, simply invalid.

The "openness to experience" scale asks people whether they consider themselves "sophisticated in the arts, music, and literature", "inventive", and whether they "like to play with ideas."

On its face, this is a roundabout way of asking whether someone is an academic, perhaps a measure of urban sophistication. It's culturally biased, and many laypeople would see this right away. It's unclear how people from rural communities are supposed to show up on this scale. They'd likely be embarrassed to call themselves "sophisticated", seeing it as arrogant, and I wouldn't be surprised if that item correlates with narcissism. Rural people probably score low on "openness" because we don't give them any items they could relate to, on which a valid personality trait of openness trait could be measured in their case. (And we do see that people in developing countries score lower, and some researchers have questioned its validity as a result of the psychometrics there.) To the extent the measure marginalizes rural communities, this could partly or fully explain conservatives' lower scores. We'll need more research with rural samples to find out, but I would never use that scale in a sincere effort to measure openness. (The effect on conservative scores in extant research could be explained by college student participants who are from rural communities, small towns, and even small cities with few outlets for "sophistication" in the arts – I never see this background info reported.)

The popular Right-Wing Authoritarianism scale is invalid. It's a caricature scale teeming with cartoonish items meant to make conservatives look bad. One item reads "Our country desperately needs a mighty leader who will do what has to be done to destroy the radical new ways and sinfulness that are ruining us."

This is not serious. Notably, conservatives don't actually endorse RWA on average, nor do they tend to endorse the Social Dominance Orientation (SDO) scale, another invalid but popular caricature scale. The reported effects, the links between conservatism and these awful sounding views, are based on linear correlations that ride the fact that liberals are at the floor and conservatives are at — or slightly below — the midpoint. They're not endorsing the items, which means they don't agree with them, yet these invalid correlations are reported as though conservatives are "high" on these constructs, implying positive agreement. Here we've also got an issue of how we do statistics, how we misuse linear correlation, often ignoring actual placement on scales and the substantive meaning therein. The fact that correlation, and all our GLM methods, are based on deviation from sample means, rather than substantive placement, endorsement, agreement, disagreement, etc., is a huge vulnerability for us. I explore these issues in another paper.

These scales have been in use for decades. If there were more than a couple of conservatives or libertarians in the field, these scales would likely have been exposed as biased and invalid back in the 1990s. It's also possible for leftists to catch these issues. Part of the issue here is that we have an incomplete account of methodological validity. We don't learn about these issues in graduate school. The nature of a caricature scale is a new insight, and the limits of the validity of correlation tests on scale measures are not well explored. With advances in our understanding of validity, even biased researchers can catch these issues. There is also some cool research adding texture to things like openness. See Brandt, et al's new paper.

Another issue I see in the comments and in the reaction by some people in the field is a striking lack of knowledge of conservative, libertarian or other non-leftist thought. This is a predictable product of the insular effects of homogeneity. Ideology is powerful in shrinking reality, shrinking the intellectual landscape. Some people are assuming that their contemporary American academic leftist ideology is a set of descriptive truths, but they don't seem to have much justification for such beliefs or awareness of competing schools. They often take left-wing value judgments for granted, like the idea that income inequality is unjust, and never consider that people might genuinely, fundamentally, and benevolently disagree with this. Lots of people don't find any reason to care about variance in incomes as a morally relevant dimension. As sociologist Chris Martin put it in a guest post on Jon Haidt's blog:

For instance, liberals often talk about inequality as a synonym for unfairness. They then describe conservatives as tolerant of inequality. However, inequality (in itself) may simply not be salient for people who aren’t liberals. It’s not that these people don’t care about fairness, but rather that they don’t think that inequality of outcomes necessarily implies unfairness. People (and groups) may differ in how hard they work, or in how valuable their contributions are in the current economy.

(See John Tamny for an articulate example of a pro-income-inequality position. Note that whether someone opposes or favors income inequality isn't the key issue – people might not see any reason to care about this dimension to begin with, as Martin notes.)

Now, these are philosophical issues, and as such they have no apparent relevance to an empirical field like social psychology. However, social psychologists often frame their research around ideological positions, and it can undermine the validity of our research. We give several examples in the paper. I was alarmed to see an SPSP symposium this year framed with this statement:

"Economic inequality is at historic highs. The wealthiest 1% own 40% of the nation’s wealth. This staggering inequality raises the question..."

That's remarkably biased, with the "staggering" bit and the collectivist premise behind "the nation's" wealth. It's disturbing that social psychologists are comfortable issuing ideological proclamations in their research, in their official business so to speak. I think it's fine to have a political identity, to believe that leftist ideology is correct, but ideological positions have no place in our work.

Some number of people are coming into social psychology with a political agenda – their research and careers are driven by left-wing ideology. A very common pattern is to seek to find out why non-leftists believe what they do, what's wrong with them, and how we can change them. This leads to pathologizing conservatives, and lately, non-environmentalists. It sometimes looks like dissonance reduction on the part of researchers. If you believe your ideology is true, but look out upon the world and see that large numbers of people disagree with it, well there must be something wrong those people. So the next step is to inventory the reasons why people don't embrace your ideology, the ideology you just know is true and noble. System justification theory is the canonical example. There's a heavy effort to find out how people can possibly justify "the system" or the "status quo", as contemporary leftists put it. The framing is often something like: Obviously it's an unjust capitalist system, so why aren't people revolting? How can the poor support a system that disadvantages them?

These questions are loaded with a number of ideological assumptions, and it's noteworthy that the field has not policed this ideological bias and allows such ideological content in its peer-reviewed papers. The assumptions include: that our "system" does in fact "disadvantage" the poor; that the poor (the left nth of the bell curve) in America are victims of injustice; perhaps that poverty (again, the always present left side of the always present bell curve) is inherently unjust and/or morally relevant; that the poor would be better off or their interests better served in some other kind of system (socialism? communism? a 20% chunkier welfare state, a la Denmark?); perhaps implicitly that the rest of society would not be worse off in that system; that our current system should be seen principally in terms of its capitalist or market elements and not by the abundant socialistic and regulatory elements that have emerged since the 1930s; that any harms or suboptimal outcomes are caused by the capitalistic or market elements and not by the socialistic, tax, or regulatory elements; that some sort of materialist, collectivist utilitarianism, perhaps a Marxist variety, is how we should evaluate societies; and that social psychologists should base their research around such philosophical-ideological premises.

Beyond system justification, a lot of research focuses on why people aren't leftist in their outlook, how they can "tolerate" or "rationalize" income inequality, why they don't care about the things leftists care about, whether they are "pro-environmental" and how to make them more "pro-environmental". Environmentalism is a rather new political ideology, and possibly a religion or a substitute for traditional religion, and it's alarming that social psychologists are promoting it and trying to convert people to it. Embracing new, abstract, and somewhat ambiguous values like "nature" and "the environment" is just assumed to be equivalent to rationality or something. Environmentalist values are contested by scholars all over the place (though not so vigorously within academia), but the field seems unaware of this, and unaware of their status as values, as ideological tenets, as opposed to descriptive beliefs about the world.

There's no equivalent conservative or libertarian bias in the field, probably because there are virtually no conservatives or libertarians in the field, and if they framed their research around similar ideological agendas it would be an easy catch for a leftist field. For example, if a conservative researcher framed his research by talking about the "staggering" number of abortions, he'd be run out of town.

I was also stunned to see SPSP diversity travel award recipients – most of whom were not underrepresented minorities – start their bios with statements like "I am an activist." Not a scientist. An activist. We know what kind of activist they will be, what ideology they will be trying to advance. And it was depressing to see virtually all the minorities focus their research on prejudice and stereotyping against their own minority groups. It's like as soon as a minority steps into the field, they go into their corner and do the ideologically-biased and approved mesearch on prejudice. This marginalizes the very few minorities we have, and somewhat weakens the benefits of diversity, since they're not attending to core social psychology research and the cultural biases therein.

So people should be cautious in taking findings at face value. In a field where the left dominates, it would be strange to simply accept findings of conservative flaws -- it's worth digging deeper. To a large extent, social science is made of words. The semantic and lexical flexibility researchers enjoy, and how political and cultural biases can exploit this flexibility, pose large risks to the validity of research. This reality is underexposed. Some of what we see in social psychology is equivalent to what would happen if we gave people in the humanities some stats software. They would treat their extremely high-level, abstract, and ideologically-loaded concepts as descriptive realities and correlate them with each other. Over the next decade or two, I think we'll see make substantial methodological progress and avoid a lot of these issues.