Jose L. Duarte
  • Home
  • Blog
  • Media Tips
  • Research
  • Data
  • An example
  • Haiku
  • About

Sam Harris, Jon Haidt, and Bias

5/21/2015

4 Comments

 
I was fascinated by my colleague and mentor Jon Haidt's analysis of the rate of certainty words in Sam Harris' books, and the reaction it sparked.

I recently came upon Daniel Miessler's post, where he defends Harris thusly:

Haidt makes a major mistake here by thinking (and saying) that because individuals are bad at judging their own objectivity it must mean that using science to gauge happiness and suffering is a fools errand.

The error should be obvious: Science isn’t based on individuals making judgements. It’s based on evidence that has been validated objectively by many.

Harris isn’t proposing that he, or any other individual, sits down and divines a solution to complex world problems using his logic Ouija board. He’s saying that science should be used to show how policy changes affect human happiness and suffering. Big difference.


This gets at some things that have been coming up a lot lately. I see very similar arguments from social scientists who deny political bias in the field. Ultimately I think we need to do a better job of explaining what we mean by bias, what bias looks like, the different kinds of bias that can arise, etc.

But let's focus on the Miessler passage. He thinks bias is inevitably avoided or expunged by scientists "objectively" validating evidence.

This could only be true if we knew everything there was to know about bias and how to catch it.

Another way of putting this: It could be only true if we knew all the forms of bias we were vulnerable to, and how to catch them.

We do not have any such knowledge. Our knowledge and understanding of bias is a domain of discovery, scientific discovery. We haven't sorted it all out yet. In fact, I think it would be more accurate to say that we're just getting started. I'd venture that there are biases in science, whether social science or other sciences, that we won't know about for fifty years. We know about more about bias today – and different forms of bias – than we knew in 1960. In 2060, we'll know even more.

So it's not possible for scientists to confidently say "we're not biased." Some scientists and some fields will be more justified in worrying less about bias, but social science is the most vulnerable. Social science is mostly made of words. We can string together some words, have people respond to those words, give the variable a label, correlate it with responses to other words we've strung together, give that a label, and make sweeping declarations. We can give awful sounding labels to our variables, loaded with an ideological sword, like "Social Dominance Orientation", and we can say that huge swaths of society are high in this "Social Dominance Orientation", even if they aren't. We do that all the time. We routinely link groups to beliefs they do not in fact hold (most conservatives do not endorse SDO, and researchers routinely conceal this.) This is a huge scientific and ethical problem. That kind of bias leads us to say things that are false, so bias can be very costly. This kind of bias hasn't been rooted out yet. We can say it will be, sticking to a "science is self-correcting" mantra, but for that mantra to be valid, science needs to be self-correcting at a reasonable rate of speed, and it isn't.

Note that it is social science that will provide our knowledge of well-being and happiness, the kind of knowledge Harris anticipates. We can add neuroscience and biomedical research to the mix, but that won't change much. There could be all sorts of biases in our measures of well-being and happiness. We could be stuck with a profound error, an error rooted in bias, and not know it for decades. This isn't necessarily a dealbreaker for positive psychology and the study of well-being. I'm sympathetic to Harris' project and I've always admired his work. I think he has criticized positive psychology for perhaps similar reasons. I'm nervous about his confidence in brain scans. I wouldn't talk about brain scans as a definitive measure of well-being without qualifying it as a distant future possibility. I would expect the first generation or two of neuroimaging to lead us into all sorts of errors of method and inference.

Another important point: Bias will not be caught if everyone has the same biases. Or even if a sufficiently large majority of a field has the same biases. Those kinds biases are only caught by history.

Bias will definitely not be caught if, in response to claims of bias, scientists simply say "science is self-correcting." That's a good recipe for non-self-correction, for reinforcing bias. Although in such cases I think perhaps we need to more clearly communicate what bias is, or what kinds of biases we're talking about.

In social science, many leftists acknowledge the political bias of the field, but some minority simply respond by declaring that the field is not biased. I have yet to see anyone engage the examples of biased research my colleagues and I have offered, or that I have offered separately. There's nothing but silence in response to the substantive examples. I'm not sure what's going on there, but I think in some cases they have no schema at all for social science being politically biased. They don't know what that would look like, have no account of that kind of bias as category of bias, and they also tend not to see leftist ideology as an ideology – only the other side is ideological.

In these cases they'll try to argue that reality simply has a leftist bias. It hasn't yet occurred to them that when a field is biased, people are expected to make that argument. They haven't lingered on the fact that their perception of reality being left-friendly is compatible with two realities: reality having a leftist bias, and the field having a leftist bias. Nor have they meditated on how to go about finding out which it is. I'm always dumbfounded that any social scientist would not understand that political ideology can profoundly shape and mediate the "reality" we see, and that being in a field dominated by fellow leftists could have a profound impact on their construal of said reality. That's an elementary observation, one that is intuitive to lots of carpenters, nurses, and baristas. We of all people have to understand it.

I know a few academics who think conservatives are inherently malevolent. That type of cartoon universe is probably unmovable – we'll just have to keep walking, focus on people whose minds can be engaged. It's unfortunate that academia has become so amenable to cartoon universes – it's incompatible with good scholarship. Scholars of that quality shouldn't be paid to be scholars – they're not good enough to be in academia, but we have lots of them. I think the poor quality of so much of modern scholarship outside of the physical sciences is a big problem, and one that will impose serious costs on our society for a long time.

In any case, bias comes in many forms, and we don't know about all of them. In the meantime, I think we need to do a better job of explaining what we mean by bias. We need a full account of political bias, its nature and operation. We don't have that yet.

Regarding Harris and Haidt, I think their views are closer than is assumed. I think they differ more in style than in substance. For example, I think Harris would agree with lots of things Jon says here. The difference in style comes down to the fact that Harris is a hardass about the obviousness of certain kinds of truths, and Jon is the opposite. I also think it's extremely important to always know when people are talking about morality descriptively vs. prescriptively. Harris' project is ultimately a unification of both, but I've known lots of intellectuals who seem incapable of thinking of morality prescriptively. Jon's good at distinguishing the two, but people following these debates sometimes forget what's what.

Even though Sam's a hardass, I think he deserves a lot of respect for sharing his journey. He's as open and transparent as he can possibly be. He shares his struggles, his exasperation, his inner processes, like no other scholar I've seen. He takes us on his journey, his dialogues and debates (see his recent encounter with Chomsky.) I feel for him sometimes. His exasperation and pain is quite evident as he grapples with bizarre arguments and unconscionable misrepresentations of his views. It will be interesting to see where he is in ten years, what ground he's covered. I often wonder about where I'll be, intellectually, in ten years, and Sam sparks the same curiosity in me.

4 Comments
MikeR
5/22/2015 01:23:47 am

Well, I enjoy some of Sam Harris's writing, but I think this bias and certainty and obliviousness to contrary evidence is pretty glaring in them. Same with most of the New Atheists. Christopher Hitchens, may he rest in peace, was maybe the worst, but Harris is similar. They may well claim that rationalism is responsible for a lot of the good in today's world. But that they vehemently and repeatedly deny that religion often makes people better - in a way that rationalism may not be able to replace - is to me obvious bias, and a willful refusal to deal with reality.
Anecdotal evidence: I know what I was like at age twenty, and I know what I was like at age 22, after I became religious. I became a better person in a number of very tangible ways. Not at all perfect, but better. More honest: there was an incident that stands out in my memory where I suddenly realized that I wasn't going to do something that I would have done before - specifically because of my religion. More kind: various sorts of behaviors helping others that I started to do regularly that I hadn't been doing at all - because of my religion. More moral: some stuff I stopped doing, some stuff I started doing like getting married and raising a family - because of my religion.
Indeed, part of what attracted me to my religion was that it was so obvious to me that the people who were exposing me to it were in some ways clearly better people than the ones I grew up with.

I remember a debate a while back between Harris and Dennis Prager. Harris did well, but his refusal to acknowledge what was to me transparent reality made it hard for me to think that he understands his subject.

Reply
tlitb1
5/22/2015 11:25:37 am

I agree that Harris is being open and transparent. I think he regards himself as being up for it, warts and all. I saw his debate with Chomsky.

I think Harris came out better in that and yes, he knew. However, if you are a Chomsky partisan, I bet you could successfully parse it for your own benefit too. ;)

I do find it strange that Harris seems so dogmatic in thinking there is a scientifically discoverable human moral landscape within some timeline that would make it useful to humanity. I mean how do we not know that environments will change in a way that dictates we could evolve into digger wasps while our brains adapt to that atrocity while never caring? ;)

I've read Harris's The Moral Landscape and have great sympathy with Haidt's criticism about the prevalence of certainty in it. I was constantly struck with that certainty throughout while I read it, it seem to be prone immediate anachronism.

David Deutsch The Beginning of Infinity: Explanations that Transform The World, is a book that seems more rational on that score about open ended development, mostly because it doesn't seem to depend on anecdotes about arguments with people who blew his mind ;)

Harris' is obviously enamoured with the idea of his own field of cognition garnering the same respect as medicine, another field that has open ended doubt but a current empirical field of testing and obvious utility that his claimed scientific morality doesn't have chance of touching yet .

BTW Speaking as a total intellectual hick, but still speaking: If anyone want's to persuade Harris's thesis even though the deadlines gone...

I think a thesis about the development of human societies from hunter gatherer through farming to today is the way to go. Ref Jared Diamond and Tom Standage.

Short version: this condemned humans desperate inequality for thousands of years until about 150 years ago. Every welfare and fairness equality indicator says that humans should have stayed hunter gatherers.

Reply
tlitb1
5/24/2015 10:13:35 am

Re-reading my last comment I realise I went off on a Sam Harris riff :) To try to be more on topic: Here's another riff :)

I was actually initially struck by this great idea above about the reality of the current knowledge of the forms of bias:

"We do not have any such knowledge. Our knowledge and understanding of bias is a domain of discovery, scientific discovery. We haven't sorted it all out yet."

This seems very true. From my layman point of view I do see there is much discussion of bias in the literature, Ionaddis seems a step in the right direction, but otherwise strong studies like his seem to be quietly ignored or used like window dressing.

I think now probably the only pertinent thing in my last comment was the reference to David Deutsch who (if I remember it well) argues against the idea of thinking there will be stepping stone incremental demarcated areas of settled scientific certainty.

Not read it for a while but I took away the idea that Deutsch argues powerfully, IMHO, that there will *always* be discoveries to be made in scientific knowledge, and that 'settled' sub-sections will break down into new areas, and the new ideas will always be profound and different.

I do think that the idea of bias being adequately handled by scientists i.e. saying that almost by definition being described as a scientist means you *can't* be biased - is the one of the insidious grandest delusions popularity used today in the media. An idea that if broken could be very destructive to a lot of policy infrastructures that interact with science - hence the strong maintenance of that (self?) illusion.

The idea that science could be both powerful, useful, and also riven with bias - actually maybe often *needing* bias - seems to have successfully been made an anathema to consider.

This reminds me of the debate about "un-biased" journalism. There is a bizarre tacit acceptance of this ideal between most political enemies.

They all want to claim they are reporting the truth from the frontline unbiased.

IMO A real ideal journalist should acknowledge that he has his ideology and want to see how his ideology beats others by being best at finding "new" factual stuff, knowing he will always be seeing it through his specfic lens wating for it to be challenged.

I think there are far more examples of journalists who do that, and are better and far braver than many posturing scientists who want to be acclaimed as passionate campaigners who are alos "un-biased" just er, because they're scientists.




Reply
Michael
8/4/2015 07:38:41 am

this is a good article on psychological biases in science:

http://nautil.us/issue/24/error/the-trouble-with-scientists

"Psychologist Brian Nosek of the University of Virginia says that the most common and problematic bias in science is “motivated reasoning”: We interpret observations to fit a particular idea. Psychologists have shown that “most of our reasoning is in fact rationalization,” he says. In other words, we have already made the decision about what to do or to think, and our “explanation” of our reasoning is really a justification for doing what we wanted to do—or to believe—anyway. Science is of course meant to be more objective and skeptical than everyday thought—but how much is it, really?"

Reply



Leave a Reply.

    José L. Duarte

    Social Psychology, Scientific Validity, and Research Methods.

    Archives

    July 2017
    December 2015
    October 2015
    September 2015
    August 2015
    June 2015
    May 2015
    April 2015
    March 2015
    February 2015
    January 2015
    November 2014
    October 2014
    September 2014
    August 2014
    July 2014
    June 2014

    Categories

    All

    RSS Feed

Powered by Create your own unique website with customizable templates.
  • Home
  • Blog
  • Media Tips
  • Research
  • Data
  • An example
  • Haiku
  • About