Lewis signaling game · reinforcement learning

Wordless

One agent can see the world but cannot act. The other can act but cannot see. Between them sit a handful of noises that mean nothing at all, because nothing means anything yet and there is nobody to decide otherwise. Neither agent reasons, plans, or knows the other exists. Run it. A vocabulary shows up.

The rules

One exchange, start to finish

The world falls into one of several states. The sender sees which, and picks a noise. The receiver hears the noise, never the state, and does something. If what it did matches how the world was, both are paid. Otherwise nothing happens.

Where the difficulty lives

The sender has no idea what any noise conveys, because that depends entirely on what the receiver does with it. The receiver has no idea either, for the mirror reason. There is no fact about what signal three means that is settled before they start, and no way to agree on one, because agreeing would already take a language.

Neither one is choosing. Each keeps a tally: when something works, do more of it. That is the whole of the learning rule, and it is the reason this counts as an answer rather than a restatement.

Start from
Exchanges
0
not started
Getting through
—
if they stopped learning
Signal carries
—
bits about the state
Vocabulary
—
noises in real use
The exchange thickness is how often
The lexicon what each side has learned
Left: how often each state gets each noise. Right: what the receiver does when it hears one. Both start flat. A language is these two going sharp at the same time, in a way that lines up.
Getting through, over time drag or arrow keys to replay
Information carried
The ceiling on the lower chart is how many bits it would take to name every state exactly once. Reaching it means the noises have become names.
Across many runs
What went wrong, when it did
Things worth trying

Five runs that each show something

Two states, two noises
It gets there essentially every time. With only two things to say, reinforcement cannot really fail.
Three states, three noises
About one run in five jams: two states end up sharing a noise and one of them can never get through again. Nothing went wrong; it just settled early.
Three states, two noises
Now it is arithmetic. Success tops out around 67% because one state has no name available. The ceiling is the vocabulary, not the learning.
Raise the cost of failure
Jams drop from roughly 19% to 12%. Being punished for a bad pairing breaks the tie that a purely reward-driven pair gets stuck on.
Let it coin new noises
Jamming nearly vanishes and communication reaches the high nineties, but the vocabulary bloats: three states end up with five or six names between them. It works and it is not tidy.
Six states, six noises
Harder every time you add one. More things to name means more chances to settle on the wrong pairing before the right one gets a turn.
The argument

Why anyone cares

This is a small model of a large problem: how a noise comes to mean something when there was no earlier meaning available to establish it with.

Quine's problem

The natural account of language is that words mean what we agree they mean. Quine's objection is that this explains nothing, because an agreement is itself a thing you have to express, and expressing it needs a language you do not yet have. Convention cannot be the foundation of meaning if reaching a convention already presupposes meaning. Every attempt to ground language in agreement runs into the same circle.

Lewis's answer

Lewis (1969) built the game above to break the circle. His point is that a convention does not need to be agreed to. It needs only to be a stable arrangement that nobody has reason to depart from while everyone else keeps to it. Two people meeting regularly at a particular corner have a convention without ever having discussed corners.

Lewis's sender and receiver were rational agents with common knowledge of each other's rationality, which lets them reason their way to an equilibrium. That is a real answer to Quine, and it comes with a real cost: it explains the origin of meaning by assuming agents sophisticated enough to have most of what meaning was supposed to provide.

Skyrms's answer

Skyrms (2010) removed the rationality. The agents above cannot reason, cannot model each other, cannot represent the problem they are in. They keep a tally and do more of whatever paid last time. And they still arrive.

Nobody assigns the signals any meaning. Nobody could. Meaning is what is left once the process settles.

That matters because it drops the price of Lewis's answer to almost nothing. If a signalling system can assemble itself out of reinforcement, then the story does not need sophisticated agents at the start, and can plausibly apply to animals, cells, and populations that plainly do not deliberate.

Where it jams, and why that is the interesting part

Run three states enough times and roughly one pair in five ends up in partial pooling: two states share a noise, so the receiver cannot tell them apart and is right about one of them at best. Communication is real, information is genuinely being carried, and the arrangement is stable. It is simply worse than it could have been, and neither party can fix it, because departing from it unilaterally makes things worse before it makes them better.

This is the model's most honest contribution. A story that always succeeded would prove too much. What emerges here is not the best possible language but whichever one the history of accidents happened to reach first.

Synonyms

Switch on invention and jamming nearly disappears, because a sender that can always try a fresh noise is never trapped by an early bad pairing. The cost is that states accumulate several names each. By Lewis's strict definition, which requires exactly one name per state, that is not a signalling system at all. By any functional measure it works perfectly. Skyrms treats the synonyms as an ordinary product of the process rather than a defect, and the model above reports them as their own outcome for that reason.

What this model is not

There are two agents, fixed roles, no population, no network, and no mechanism by which a signal could come to be about anything more complicated than which of a handful of states obtains. Nothing here addresses syntax, compositionality, or reference to absent things. Skyrms and others extend signalling games in all of those directions; this is the base case, which is worth seeing on its own because the base case is the one that has to work first.

References

Sources

  1. Argiento, R., Pemantle, R., Skyrms, B., & Volkov, S. (2009). Learning to signal: Analysis of a micro-level reinforcement model. Stochastic Processes and Their Applications, 119(2), 373–390.
  2. Barrett, J. A. (2007). Dynamic partitioning and the conventionality of kinds. Philosophy of Science, 74(4), 527–546.
  3. Godfrey-Smith, P. (2011). Review of Signals: Evolution, learning, and information by Brian Skyrms. Mind, 120(480), 1288–1297.
  4. Huttegger, S. M., Skyrms, B., Smead, R., & Zollman, K. J. S. (2010). Evolutionary dynamics of Lewis signaling games: Signaling systems vs. partial pooling. Synthese, 172(1), 177–191.
  5. Lewis, D. (1969). Convention: A philosophical study. Harvard University Press.
  6. Quine, W. V. (1936). Truth by convention. In O. H. Lee (Ed.), Philosophical essays for A. N. Whitehead (pp. 90–124). Longmans.
  7. Roth, A. E., & Erev, I. (1995). Learning in extensive-form games: Experimental data and simple dynamic models in the intermediate term. Games and Economic Behavior, 8(1), 164–212.
  8. Skyrms, B. (1996). Evolution of the social contract. Cambridge University Press.
  9. Skyrms, B. (2010). Signals: Evolution, learning, and information. Oxford University Press.
  10. Zollman, K. J. S. (2005). Talking to neighbors: The evolution of regional meaning. Philosophy of Science, 72(1), 69–85.