Scientific polarization · mistrust weighted by disagreement

Twocamps

Everyone here is a Bayesian. Everyone reports their results honestly. Nobody has a funder, a side, or a motive. One thing is added: a researcher takes evidence less seriously when it comes from somebody whose view already differs sharply from their own. That is enough.

The rules

The one assumption that does all the work

A community is choosing between an established action, whose success rate everybody knows, and a new one that might be better. They experiment, they publish, they update. Nothing about that produces disagreement that lasts.

Now suppose you read a result from somebody whose conclusions are wildly at odds with yours. Discounting it is not obviously unreasonable: a large disagreement is some evidence that one of you has a problem with their methods, and it need not be you. That is the entire modification, and it is applied symmetrically to everybody.

The trap underneath it is the same one from the page next door: a researcher who concludes the new action is worse stops testing it, and therefore stops producing any evidence about it. Nothing further can reach them, because there is nothing further to arrive.

Start from
Round
0
not started
Furthest apart
—
of a hundred
Still testing it
—
the better action
Stopped looking
—
and cannot be reached
Every researcher, over time one line each
Ends up thinking the new action is better Ends up thinking it is not
How much they listen
Spread and abandonment drag or arrow keys to replay
How far apart the community is Still testing the new action
Where the threshold sits
Things worth trying

Four runs and a sweep

Press Everyone listens
Textbook. Credences fan out at first, then close, and the community settles on the better action. This is what pooling honest evidence is supposed to do.
Press The same community, wary
Same researchers, same problem, same seed. The lines fork instead of closing. Nobody changed their standards of evidence; they only stopped counting evidence from people who disagreed with them.
Press Sweep the mistrust setting
The collapse is sharp and it sits between 1.0 and 1.5 — exactly where a researcher stops hearing anybody more than half a credence away. Below it, 95% of communities agree and are right. Above it, 5%.
Press Wary, with an obvious answer
It still splits. This is where the page differs from the one next door, where making the problem easy killed the effect outright. Here an obvious answer barely helps: even when the new action is forty points better, more than a quarter of communities still fracture and fewer than half reach the right answer. An obvious answer is no use to somebody who has stopped looking.
Watch the stopped looking figure while a wary run plays
It only ever goes up. Once a researcher drops below the halfway line they stop testing the new action, stop producing evidence about it, and become permanently unreachable. That is the mechanism, not the mistrust itself.
The argument

Why anyone cares

The usual explanation for a polarized public is that people are irrational, dishonest, or badly informed. This model has none of those and polarizes anyway, which makes it a much less comfortable result.

What is and is not assumed

Every agent updates by Bayes' rule. Every agent reports what they found. No agent has a preference about which answer is true, and no agent is exposed to different evidence by design. The single addition is Jeffrey conditioning on an uncertain report: when somebody's conclusions are far from yours, you treat their data as less likely to be sound.

O'Connor and Weatherall (2018) argued that this is not even a departure from rationality. If you have good reason to think your own methods are sound, then somebody reaching a very different conclusion is some evidence that theirs are not. Discounting them is what a careful person does.

What it produces

Below a threshold, nothing much: the community converges and gets it right nineteen times in twenty. Above it, consensus collapses almost completely. The characteristic failure is not that the community agrees on the wrong thing — that stays rare. It is that it stops agreeing at all, and that a large fraction of it stops looking.

They did not stop reasoning. They stopped generating the evidence that would have changed their minds.

That last part is the mechanism and it is worth separating from the mistrust. Mistrust slows the spread of evidence. What makes the split permanent is that a researcher who has concluded the new action is worse no longer tests it, so no further evidence about it exists to be discounted in the first place. The community is not stuck because it is stubborn. It is stuck because it has stopped producing the thing that would unstick it.

Why an easy question does not rescue it

This is where the page parts company with the one next door. There, making the better option clearly better destroyed the effect completely: every network structure finds an obvious answer, so there is nothing for connectivity to spoil. The natural expectation is that the same should apply here.

It does not, and the difference is instructive. Raise the gap from five points to forty and wary communities still fracture a third of the time. The reason is that the two models fail differently. Zollman's community fails by converging too early on evidence that was not enough; a clearer signal fixes that directly. This community fails by ceasing to generate evidence at all, and no amount of obviousness helps a researcher who has stopped running the experiment. An answer being easy to find only matters to somebody still looking for it.

Why this is not a story about bad actors

Models of polarization that assume propaganda, motivated reasoning or bad faith explain the phenomenon by putting it in the premises. This one does not, and that makes it harder to dismiss and harder to fix. You cannot address it by improving anybody's honesty or reasoning, because both are already perfect. What would have to change is either how readily people discount those they disagree with, or whether anybody is still testing the unfashionable option.

O'Connor and Weatherall extended this line of work into a general account of how misinformation spreads through communities that are behaving reasonably. It is worth reading alongside the more common assumption that polarization requires somebody to be lying.

What this model is not

There is one question, two options, and no politics. Nobody chooses who to read, nobody has an identity tied to an answer, and nothing outside the community is pushing on it. Real polarization involves group membership, motivated reasoning, and actors with an interest in the outcome, none of which appear here. The claim is only that those ingredients are not necessary, which is a narrow claim and a surprising one.

References

Sources

  1. Bramson, A., Grim, P., Singer, D. J., Berger, W. J., Sack, G., Fisher, S., Flocken, C., & Holman, B. (2017). Understanding polarization: Meanings, measures, and model evaluation. Philosophy of Science, 84(1), 115–159.
  2. Hegselmann, R., & Krause, U. (2002). Opinion dynamics and bounded confidence models, analysis, and simulation. Journal of Artificial Societies and Social Simulation, 5(3), 1–33.
  3. Jeffrey, R. C. (1965). The logic of decision. University of Chicago Press.
  4. Kelly, T. (2008). Disagreement, dogmatism, and belief polarization. The Journal of Philosophy, 105(10), 611–633.
  5. O'Connor, C., & Weatherall, J. O. (2018). Scientific polarization. European Journal for Philosophy of Science, 8(3), 855–875.
  6. O'Connor, C., & Weatherall, J. O. (2019). The misinformation age: How false beliefs spread. Yale University Press.
  7. Weatherall, J. O., & O'Connor, C. (2021). Conformity in scientific networks. Synthese, 198(8), 7257–7278.