On fraud

I've never had to think about scientific fraud until the last few months. It's been an interesting journey. I've got a paper coming soon that focuses on this issue. I'm of two minds about the need to think about it. It raises interesting ethical and epistemological questions, and I'm deeply interested in both ethics and epistemology, but it also feels parasitical, an exquisite way to distract people from their research. So on that count, part of me resents even having to think about fraud, write about fraud, or deal with people who engage in fraud.

I've talked to a number of people about the recent cases, and I want to highlight something. A couple of people seemed to think that fraud = data fabrication, that they're isomorphic. In that scenario, fraud is inherently hidden, and can only be uncovered by authorities opening up a lab freezer and pulling out the stem cells or doing forensics on someone's data.

The definition of scientific fraud is itself a focus of ongoing scholarly inquiry and discussion. There are lots of definitions in the literature. I'm not aware of any that define fraud so narrowly as to restrict it to fabrication. Data fabrication, or what I sometimes call "spreadsheet fraud", is but one class of fraud.

We probably tend to think of recent salient cases of fraud and treat them as the prototypical form of fraud, perhaps the only form, in a manner similar to the availability heuristic. The most famous of the recent cases in social psychology is probably that of Diederik Stapel, who admitted fabricating research.

My former program chair at UNC, Larry Sanna, evidently engaged in similar misconduct, though much less information has been made publicly available there than in the Stapel case. One thing we know for sure is that he destroyed the careers of his graduate students – and my friends – before they had even begun. That is really something.

Lewandowsky said this:

"NASA Faked the Moon Landing—Therefore (Climate) Science Is a Hoax: An Anatomy of the Motivated Rejection of Science"

...when only three participants out of 1145 in his blog-posted web survey held those two beliefs.

He also said:

"Endorsement of free markets also predicted the rejection of other established scientific findings, such as the facts that HIV causes AIDS and that smoking causes lung cancer."

...when 95% and 96% of free market endorsers agreed that HIV causes AIDS, and that smoking causes lung cancer, respectively.

I think that's going to have to be fraud any day of the week. The key element of the concept of scientific fraud – and any definition we'd find – is deception or misrepresentation. Note that the definitions commonly include falsification, in addition to and distinct from fabrication. The definition of falsification commonly includes misrepresenting the results of statistical analyses or omitting data or results such that the findings are misrepresented or the reader misled. For an example of the former element in a definition, see here.

At no point did Lewandowsky et al. inform the reader that there were only three moon-climate hoax people in a sample of 1145, or that only ten people endorsed the moon hoax to begin with. At no point did the authors disclose that only 16 of 1145 people disputed that HIV causes AIDS, 11 disputed the smoking--lung cancer link, or that only 5% and 4% of free market endorsers disputed those facts. Such trivial numbers cannot be used draw inferences or run correlations – these could be errant keystrokes, sticky keys, or a few felines. There was no data in this study to support the authors' claims. (Also, the effect proclaimed in the title is not reported in the paper.)

As you can see, in the Lewandowsky case, we don't need to open up anyone's lab freezer. We don't need a committee or authority to tell us this is fraud. We don't need any further information – the numbers I gave above are not in dispute. Anyone can just look at the (stripped) data file the authors released. Our mortal eyes and brains are sufficient to validate my claims or any other claims about the data. It's just an Excel spreadsheet with some survey responses, one of the simplest datasets we'll ever see.

The epistemological structure here is that Lewandowsky et al made claims A and B. I then argue that claims A and B are false by reference to their own data (and have the potential to harm millions of innocent people who would be linked to beliefs they clearly do not hold.) I point to the data, produce the numbers. I further argue this kind of conduct – making false claims of the sort they made – comfortably fits within the definition of scientific fraud, not just mine, but many extant definitions and almost certainly the common person's definition. I take it to be a safe assumption that the authors knew their data, had looked at it and so forth. They knew there were only three people who could possibly fit the effect they asserted in their title, meaning there was no effect. They certainly knew this for at least a year before I stumbled on this case, and they've done nothing to correct the record, nor have they retracted the paper. They've done more the opposite of that, and Lewandowsky's explanation for his title belongs in a museum of scientific fraud.

The nature of my claim is such that it is based on their claims and data – no frozen tissue samples or university officials will be relevant, logically or epistemically, to what I am saying. Again, the numbers are not in dispute. My claim rests on those numbers and the claims the authors made. There is no other evidence that is relevant or necessary to my particular argument – nothing needs to be uncovered. In that respect, this is an unusual fraud case. People might not be used to cases of this sort, with this structure, but we don't need a novel definition of fraud to enfold cases like these. Existing definitions seem quite adequate, though social psychology as a field hasn't had rigorous conversations about how fraud should be defined, nor do we have institutions or mechanisms to deal with it.

I assumed we did, and I realize now that I took the wrong inference from the Stapel, Smeesters, and Hauser cases. In every one of those cases, which are about the only cases I knew of, it looked as though the host universities did an admirable job of thoroughly investigating the fraud or misconduct (I'm not clear on what the Hauser case turned out to be.) The Levelt Committee at the University of Tilburg did a commendable job of investigating Stapel, and produced an excellent report.

I now realize that all those cases started with a member of the university community walking into an ethics office or whatever and reporting either fraud or that something wasn't right. As a social psychologist, things are starting to make a bit more sense now. It's much easier for officials and institutions to ignore e-mails from an outsider alleging that one of their researchers committed fraud than it is to ignore a member of their own community sitting in front of them. I should have seen the implications of these factors long ago. In fact, Nature has repeatedly discussed how difficult it can be to get universities to investigate fraud.

That's not an excuse for institutions like the University of Western Australia or Queensland. Their officials should be energetic about investigating such cases, should have a fundamentally different orientation toward outsiders reporting any kind of misconduct. The people in those positions should be selected for their eagerness to pursue such cases, for their integrity, and their non-investment in maintaining the university's public image. Such people exist, but I think it's hard to find them if you're not looking very specifically for them. However, these institutions' behavior is not as strange as I used to think, folding in the above factors, just from a descriptive social psychology standpoint. (I never actually contacted Queensland given that they tried to hide or destroy the evidence of the 97% fraud by sending legal threat letters to the whistleblower who released said evidence. In a move I've never heard of, they even told him that he could not divulge receipt of the letters, claiming that the legal threat letters were themselves copyrighted.)

I hope that clears some things up. There seems to be a subculture in science that carries a sense of extraordinary entitlement and privilege with respect to fraud accusations, a degree of entitlement that would not be found outside of academia. I think Australian philosopher Brian Martin is right in arguing that scientific elites have a strong vested interest in defining fraud as narrowly as possible. I think it's clear that we're going to have a conflict of interest, as a vocation, in how vigorously we want to define fraud. I think many scientists will agree with this – it's an almost trivial insight given everything we know about human nature, and this reality does not require that most scientists be fraudsters, or that even ten percent of them are. And I think it's certainly reasonable to want to err on the side of false negatives, rather than false positives (this assumes that our options truly map to differentiable rates of false negatives and false positives – I'm not sure that's true.)

In the Lewandowsky case, some dispute might come down to a researcher's right to say X predicts Y1 when there is no one at Y1, but there is a Pearson correlation between X and Y with a sufficiently low p-value. In this case, I think it's pretty clear-cut because the HIV-AIDS variable was an opposite variable – there were two levels of disagreement and two levels of agreement. Anytime we have that kind of scale, where one side is the substantive opposite of the other, we know that we can get a positive or negative correlation even if scores are clustered exclusively on one side of the scale.

For example, we can easily have a negative correlation between free market endorsement and an HIV-AIDS item even if no one disputes that HIV causes AIDS. A negative correlation does not at all imply that free market views predict rejection of that basic scientific fact – not to anyone who is familiar with the formula for correlation. Neither positive nor negative correlations imply any particular placement on the scale. This phenomenon is made possible by having a scale with more than two points, and is made more and more possible the richer the scale (e.g. a seven-point scale.) This is basic stats. And disagreeing with me on our license to link people to views they do not hold simply because we have an inferential statistic driven by variance on the other side of the scale will do nothing to justify the title, so I think you'd have your work cut out for you. You'd also have to argue that we can make any inferences about any of these variables with data from a survey posted on environmentalist blogs, open to anyone in the world including fakers, and where the authors either stripped or never collected age, gender, and nationality.

There's a core logic problem with any notion that fraud should only be defined as data fabrication, that nothing we say can be fraud, that fraud only relates to numbers and not words. If we can say anything we want, proclaim any effect in our titles and papers regardless of whether we have data to back them up, then no one would need to fabricate data. If words can't be fraud, then all future fraud can be shifted from data to words.

My outrage in this case – and it really is outrage – is that millions of people were falsely linked to beliefs that could be incredibly damaging to them. We can never take it back. It's out there now, because one of our own put it out there. It could harm people for years to come. One day Okcupid and similar services might be able to quantify the harm – would you want to date someone, to have sex with someone, who disputes that HIV causes AIDS? Is there any doubt that having read in the NYT that free market endorsement predicts rejection of the HIV-AIDS link might bias someone – even non-consciously – against the conservatives, libertarians, and economists they see on a dating site or meeting in day-to-day life? This case is probably the most vivid consequence I've seen of the political homogeneity and bias of the field.

I doubt it will ever affect me personally, that anyone would think someone with a PhD and several books to his name might dispute that HIV causes AIDS (in some future scenario where I found myself single.) But it could clearly affect millions of others. I've never understood social psychology to be a vehicle of mass harm. All our vaunted research ethics and IRBs count for nothing if we can't manage to police people, editors, journals, and bodies like APS when they plant false and unbelievably harmful links that could impact millions of innocent people. That breaks my heart. It really breaks my heart that anyone could do this, that I could be associated with a field that does things like this. It will further break my heart if I come to find that other social psychologists' hearts don't bleed upon processing what happened here.


Fake Rolexes

A couple of people have e-mailed me trying to get me to accuse Cook or Lewandowsky of mundane economic/criminal fraud, perhaps to expose me to a lawsuit. One of these people identified himself as Brendan Montague, a person who has reputation as some sort of environmentalist operative (try google.) In both cases, these people, having failed to hook me, have argued that we simply cannot use the concept of scientific fraud because any use of the word fraud must mean economic criminal fraud, like selling fake Rolexes.

This is easily the most specious, witless, and sinister argument I've ever heard. The idea that we simply cannot speak of such a well-established, ancient concept as scientific fraud is the only use I have ever had for the word execrable. Scientific fraud is, as I mentioned, a topic of explicit scholarship and inquiry. There are countless journal articles contributing definitions, examining causes, estimating rates, and so on. There are countless cases of it in the news, all over the world. When we speak of scientific fraud, no one thinks we're talking about selling someone a pound of beef that weighs eleven ounces. Not being able to speak of scientific fraud is tantamount to not being able to speak, and I think we might be facing a rather broad assault on freedom of speech from the modern left. Concepts like "hate speech" and "offensive" are thrown all over the place, and I'm sure how they're defined, whether their application consistently fits any coherent definition, or whether they are categorically destructive concepts to have in a theory of free speech.

Note that the concept of journalistic fraud is very similar to scientific fraud, well-established, and widely used. When the story broke that that Jayson Blair had fabricated all those stories at the New York Times, no one would have taken the topic to be economic criminal fraud. As far as I recall, no one was talking about criminal charges at all. Scientists and journalists are somewhat privileged in that respect – fraud generally has no criminal justice implications.

I don't know much about the Michael Mann lawsuit, but if this is an actual plaintiff's argument in that case, we might want to pay more attention to the case. It seems revolutionary that we'd be banned from using well-established concepts if they happen to have a different use in criminal law. I'm not sure we've ever heard such an argument. Perhaps this is why all those groups across the political spectrum filed amicus briefs siding with the defendants (I forget who all the defendants are.) Gavin Schmidt made a ridiculous argument on Twitter in support of Mann, something to the effect that a charge of (scientific) fraud is defamation per se, which rests on the assumption that we're talking about criminal fraud, an assumption we can dismiss with prejudice. It also rests on the assumption that any such allegation is false, a core feature of defamation. It's so self-serving for scientists try to silence anyone who would call out scientific fraud, to maintain some sort of aristocratic privilege.

(Tone edits above: I'm struggling to find the appropriate tone with some of these posts. It disappoints me that I still drop to words like stupid and idiotic. There are uses for those words, but I don't think they accomplish much. The idea of not being able to use concepts like scientific or journalistic fraud is so ludicrous and dangerous that I get real snappy. Nothing bothers me more than these assaults on freedom of speech. Without that, I'm not sure what we'd have left.

Also edited the "If this is a plaintiff's argument, Michael Mann is a threat to freedom of speech..." clause, since someone rightly pointed out it was based on a conditional, and we probably shouldn't make sweeping statements of that sort based on conditionals. I think I have a tone problem. On the one hand, I think these issues merit a forceful tone, but on the other hand, "forceful" is a continuum, and sometimes my tone seems excessive even to me. I'll have to think about this some more.)

Moreover, I think this new theory of non-contextual word use is incredibly dark. Can you imagine a reality where we were not allowed to speak of scientific fraud unless some authority told us we could? (Or journalistic fraud.) We know that universities and these "fact-finding" bodies have profound conflicts of interest. Ask Nature. It's so strange to place fraud investigations in the hands of such conflicted parties – we don't see that kind of rigged setup outside of academia. We know that we lack the institutions, mechanisms, and anti-corruption measures we would need to reliably police fraud. We don't even have independent investigators. (FYI, I used to help companies comply with the Sarbanes-Oxley Act.) We know that the scientific community is likely to have some conflicts of interest here. We know that academia in our era is intensely politicized, and that one ideology dominates and discriminates. Model in your head what would happen to the public's trust in science if the term scientific fraud were legally removed from our vocabulary, where fraud-fighters were silenced. If you think the public doesn't trust science now, I think you might want to get a stopwatch to see how long it takes for taxpayers to defund science in that scenario.

Imagine we were talking about plagiarism, and we said someone stole someone else's work. Would Schmidt and other climate activists say that we must be accusing the plagiarist of grand theft auto? What other words are we not allowed to use? What other words are used to mean different things in other walks of life? Should we be banned from using any word that is used in criminal law, no matter how well-established it is in our non-criminal-law domain? This is as intellectually bankrupt as anything we're going to see in a normal human lifespan.

These people definitely need to check their privilege.

(I'm not sure how to adjudicate falsity where a political commentator is assessing a scientist's work, as in the Mann case, especially given that the scientist has clearly done some sloppy work. There might be issues of satire or similar license. There might be some presumption of the writer's right to interpret and judge scientists' work, some range of reasonable and expected divergence. No one should be beholden to investigations by institutions that have such obvious conflicts of interest, or really to any authorities or powers that be. It's demonstrably the case that authorities cannot be consistently relied upon to investigate fraud. People should be free to see, think, and speak for themselves. I don't know the details of what Steyn said, though I remember something about the word "bogus", in addition to "fraud". I can't imagine a universe, certainly not one that features the First Amendment, where writers aren't allowed to assess someone's work as bogus. I don't think Mann has ever been accused of doing anything remotely like what Lewandowsky did (I'm not sure anyone has ever done what Lewandowsky did), but I haven't followed his career. He's the last climate scientist I would trust, given his awful behavior, the way he treats Judith Curry, and how much he politicizes climate science.)

In Conclusion

People seem to be worried about my career, discrimination and so forth. I know full well that all this is extremely poorly timed with respect to my academic career, that calling out fraud, advocating a more credible and scientifically consensual definition of fraud, or even just using the word fraud will lead some people to discriminate against me in the job market. I know it could kill my academic career. But if people seriously think that this would deter me, or that it ought to deter me, I think they have failed to build in themselves the classic scientific virtues. I also think they're heartless with respect to the potential victims of this garbage. It would never occur to me to sacrifice basic principles of integrity, ethics, or beneficence in order to get an academic job, or tenure. The thought is alien to me. I don't know how to live like that. 

I'm going to be discriminated against anyway: 37.5% of social psychologists explicitly stated that they would discriminate against a conservative job candidate (maybe libertarians would face less discrimination – I'm not sure.) 44.1% said they thought their departmental colleagues would discriminate. These numbers might be catastrophic for the career prospects of anyone like me. They might be more than sufficient, given some math on opportunities, iterations, and network effects, to lock people out. They might, in effect, confer herd immunity for a politically biased and tribal field. The discrimination will touch people when they submit to journals, apply for grants, and so forth. The math just might be catastrophic. Perhaps there's literature on the math of discrimination and the thresholds necessary to confer herd immunity.

My timescale seems to be different from the hecklers'. I don't care about the short-term. I'm happy to be right fifty or a hundred years from now, in terms of what other people think, and I think it's pretty obvious that these cases are only going to look worse and worse as time passes. Still, I think it's unrealistic to expect that these cases are going to turn out well for the researchers even in the short-term. I used to help companies comply with Sarbanes-Oxley. I know what the standards are in the world at large, certainly in America at large.

Presumably, calling out political bias as an issue will spark even more discrimination, as will disclosing past experiences of discrimination, such as the fact that the University of Arizona social psychology program denied me admission after probing my views of Jimmy Carter, of all people. I decided to do that after hearing some horror stories and seeing some new data. The discrimination in social psychology is really a disgrace, and these "scholars" don't seem to have any grasp of the complexity of human thought, the range of perspectives a thinker might take or even create. They seem to have no sense of time and place, don't understand our small place in the grand sweep, give far too much primacy to the political identities of our day. People who would discriminate against someone for their momentary views of a distant President do not understand the nature of the enterprise.

Add my anti-fraud activism against old, tenured white men to the basket, and the fact that I'm a first-generation American, first-generation college graduate, English-as-a-second-language Mexican-American from a copper-mining town with some nice pubs might not save me. I am not unaware of this fact. But I'll be a social scientist regardless of whether American academia chooses to discriminate against me again. It will look terrible for me to be out there, locked out of academic social psychology or philosophy of science, as the publications and research pile up and up and up. It will be ridiculous, will make social psychology and academia look absurdly unethical and discriminatory. Be that as it may, it's never been easier to be a social scientist outside of academia, what with the internet, crowdfunding of research, the fact that social science has never been expensive, etc. I'm not going to look the other way on this garbage, not when we're talking about "findings" that can be so harmful to so many people. I've read the books about Enron, and I'm, well, a social psychologist. I know what bias does, what tribalism does. Whatever happens, happens. This is simply the universe as I find it.