Back when I was reading through the sequences I noticed that several times after Eliezer Yudkowsky had explained some example of people being irrational a commenter, most commonly Robin Hanson, would say that it is possible that the behaviour is actually not irrational at all.
My favorite example is a post (which I can’t find right now) claiming that when people are confronted with arguments against their position they end up being more certain of the beliefs they already hold and that this is irrational because they are not properly updating when receiving new evidence. In the comments someone said that the subjects could be observing that the argument against their position is weak and be reasoning that in a world where their beliefs were wrong, they would expect there to be better arguments for the true position, so he weakness of the arguments is evidence that their position is correct.
This seems possible but unlikely to me. I think this is valid reasoning and if I was observing for the first time arguments against a belief I had and those arguments were very weak (especially if they were coming from people who I have observed make strong arguments in the past and I know have put a lot of thought in to the issue) I would definitely update towards my beliefs being more likely. But despite that, I still think it is psychologically unrealistic to think this is what is happening in the majority of people’s brains when they are presented with evidence against a belief and end up being even more confident*.
There are other psych experiments where we can apply the same reasoning. For example in the Asch’s conformity experiment, we could reason that when subjects conform they are actually updating on the evidence of other people in the room apparently having different views about the length of the line.
Two possible models that can be used to explain observations of human irrationality are, firstly, “Yes, that’s because humans are irrational, which is exactly what we would expect form what we know about evolution” and secondly “What appears to be human irrationality is actually people behaving irrationality but, for example, trying to achieve different goals then they appear to be. ”
My prior from the inside is that the first is much more likely and that it is exactly what I would predict if I had not observed human behaviour but was told about evolution. The only reason I can think to have a prior that favors second model is a belief in the Neoclassical models of perfect rational self-interested human agents**. The first model also seems simpler so gets Occam’s Razor/Solomonoff’s Induction points.The worst part is that these models are usually used to explain the same observation so it is hard to think of evidence that would be more likely to exist if one was true and not the other.
The second model seems to be connected (conceptually in my head, if not in the reality) with both the idea of revealed preferences and the signalling model of human behaviour. I am skeptical of both of these concepts and will hopefully be writing posts in the future about why.
* Taking the outside view, its possibly that this is just elitism of the “Well I am smart enough to reason like that, but most other people aren’t ” Because of this thought, I am going to update slightly away from the “People Are Irrational” Model.
** Either the extremely unrealistic Econ 101 version or the more nuanced version held my more knowledge Neoclassical economists