Language is probably humanity’s most distinguishing attribute, but we still don’t wholly comprehend many of its fundamental properties, for instance, how it evolved to be such an efficient system. Facebook Artificial Intelligence (AI) models attempted to recreate the exact circumstances and communication processes to imitate natural behavior to better understand this.
Let’s dig into what they did.
Two neural networks (NNs) were devised, a speaker and a listener. They were assigned the task of playing the “Communication Game”. So the round starts as the Speaker sees a color chip from a continuous color space and creates a distinct sound that can be considered equivalent to the human word. Now it’s the listener’s turn to interpret the correct meaning. The listener is provided with two color options, out of which one is correct, and the listener attempts to choose the correct one.
At first, the Speaker generated terms at random, but these gradually represent regions of the color space. They repeated this experiment several times, changing the task complexity by rendering the goal and distractor chips more or less alike. These differences resulted in a variety of distinct color-naming “vocabularies.”
They got this result by using game performance as the primary training signal and not putting any constraints on the evolving code other than the requirement that it comprises a single distinct symbol. Human naming schemes are effective in the sense that they maximize the accuracy/complexity trade-off. A virtually identical trade-off was observed for the NNs as natural human languages when they graphed the observations.
Evaluating the Accuracy/Complexity Trade-Off
Minimizing the Information Bottleneck objective function approximates the ideally optimal trade-offs between complexity and accuracies.
The communication model of Zaslavsky et al. was adopted to compare Neural networks and human naming systems, which represented each target color chip as a Gaussian distribution of overall colors centered at that color.
To read further about the Information Bottleneck theory, click here.
There are yet some areas of language cognizance that are unexplained to human knowledge. However, the AI models’ two contributions were highly relevant to this topic. First, since utility and ambiguity avoidance often characterize the code evolved by communicating Neural Networks (NNs), these considerations cannot be reasoned away by biological agents’ least-effort factors. Second, NNs must share a discrete signal. Discreteness is a striking, perhaps unique feature of human language, and it is often used as a prerequisite for the combinatorial infinity of speech that characterizes it.