The origins of Zipf's meaning-frequency law

In his pioneering research, G. K. Zipf observed that more frequent words tend to have more meanings, and showed that the number of meanings of a word grows as the square root of its frequency. He derived this relationship from two assumptions: that words follow Zipf's law for word frequencies (a power law dependency between frequency and rank) and Zipf's law of meaning distribution (a power law dependency between number of meanings and rank). Here we show that a single assumption on the joint probability of a word and a meaning suffices to infer Zipf's meaning-frequency law or relaxed versions. Interestingly, this assumption can be justified as the outcome of a biased random walk in the process of mental exploration.

Introduction G. K. Zipf (1949) investigated many statistical regularities of language. Some of them have been investigated intensively such as Zipf's law for word frequencies (Fedorowicz, 1982;Ferrer-i-Cancho & Gavaldà, 2009;Font-Clos, Boleda, & Corral, 2013;Ferrer-i-Cancho, 2016a) or Zipf's law of abbreviation (Strauss, Grzybek, & Altmann, 2006;Ferrer-i-Cancho et al., 2013). Some others, such as Zipf's law of meaning distribution have received less attention. In his pioneering research, Zipf (1945) found that more frequent words tend to have more meanings. The functional dependency between µ, the number of meanings of a word, and f , the frequency of a word, has been approximated with (Ilgen & Karaoglan, 2007;Baayen & Moscoso del Prado Martín, 2005;Zipf, 1945) where δ is a constant such that δ ≈ 1/2. Eq. 1 defines Zipf's meaning-frequency law. Equivalently, the meaning-frequency law can be defined as G. K. Zipf derived the meaning-frequency law assuming two laws, the popular Zipf's law for word frequencies and the law of meaning distribution. On the one hand, Zipf's law for word frequencies states the relationship between the frequency of a word and its rank (the most frequent word has rank i = 1, the 2nd most frequent word has rank i = 2 and so on) as where α ≈ 1 is a constant (Zipf, 1945(Zipf, , 1949. On the other hand, the law of meaning distribution (Zipf, 1945(Zipf, , 1949 states that where γ ≈ 1/2. Notice that i is still the rank of a word according to its frequency. The constants α, γ and δ can be estimated applying some regression method as in Zipf's pioneering research (Zipf, 1945(Zipf, , 1949. Sometimes, power-laws such as those described in Eqs. 1-4 are defined using asymptotic notation. For instance, random typing yields f = Θ(i −α ), i.e. for sufficiently large i, one has that (Conrad & Mitzenmacher, 2004) where a 1 and a 2 are constants such that a 1 ≤ a 2 . Eq. 5 can be seen as relaxation of Eq. 3. Similarly, Heaps' law on the relationship between V , the number of types, as function of T , the number of tokens, is defined as V = Θ(T β ) with 0 < β < 1 (Baeza-Yates & Navarro, 2000), a relaxed version of V ∝ T β (see Font-Clos and Corral (2015) for a critical revision of the power law model for Heaps' law). The meaning-frequency law (Eq. 1) and the law of meaning distribution (Eq. 4) predict the number of meanings of a word using different variables as predictors. The meaning-frequency law has been confirmed empirically in various languages: directly through Eq. 1 in Dutch and English (Baayen & Moscoso del Prado Martín, 2005) or indirectly through Eq. 4 and the assumption of Zipf's law (Zipf, 1945;Ilgen & Karaoglan, 2007) in Turkish and English. Qualitatively, the meaning-frequency law defines a positive correlation between frequency and the number of meanings. Using a proxy of word meaning the qualitative version of the law has been found in dolphin whistles (Ferrer-i-Cancho & McCowan, 2009) and in chimpanzee gestures (Hobaiter & Byrne, 2014). Thus, the law is a candidate for a universal property of communication. Zipf (1945) argued that Eq. 1 with δ = 1/2 follows from Eq. 3 with α = 1 and Eq. 4 with γ = 1/2. Indeed, it has been proven that Eq. 1 with δ = γ/α follows from Eqs. 1 and 4 (Ferrer-i-Cancho, 2016b). Here we consider alternative derivations of Eq. 1 or relaxed versions of Eq. 1 from the assumption of a biased random walk (Sinatra, Gómez-Gardeñes, Lambiotte, Nocosia, & Latora, 2011;Gómez-Gardeñes & Latora, 2008) over word-meaning associations. The remainder of the article is organized as follows.
First, we will present the mathematical framework. Second, we will present a minimalist derivation of the meaning-frequency law (Eq. 1) with δ = 1/2 law that is based on just one assumption on the joint probability of a word and a meaning. Suppose a word s that is connected to µ meanings and a meaning r that is connected to ω words. Assuming that the joint probability of s and r is proportional to µ if s and r are connected and zero otherwise, suffices to obtain Eq. 1 with δ = 1/2. A problem of the argument is that the definition is somewhat arbitrary and theoretically superficial.
Third, we will replace this simplistic assumption by a more fundamental assumption, namely that the joint probability of s and r is proportional to µω if s and r are connected and zero otherwise. This assumption is a more elegant solution for two reasons: it corrects the arbitrariness of the assumption of the minimalist derivation, fits into standard network theory and it can be embedded into a general theory of communication. From this deeper assumption we derive the meaning-frequency law following three major paths. The 1st path consists of assuming the principle of contrast (Clark, 1987) or the principle of no synonymy (Goldberg, 1995, p. 67), namely, ω ≤ 1 for all words, which leads exactly to Eq. 1. The 2nd path consists of assuming that meaning degrees are mean independent of word degrees, which leads to a mirror of the meaning-frequency law (Eq. 2) where E[p|µ] is the expectation of p, the probability of a word, knowing that its degree is µ. Notice that p is linked with f as p ≈ f /L, where L is the length of a text in tokens, i.e. total number of tokens (the sum of all type frequencies) (Moreno-Sànchez, Font-Clos, & Corral, 2016). The 3rd path makes no assumption to obtain a relaxed version of the meaning-frequency law, namely the number of meanings is bounded above and below by two power-laws over f , where b 1 and b 2 are constants such that b 1 ≤ b 2 . The result can be summarized as µ = Θ(f δ ).
Put together, these three paths strongly suggest that languages are channeled to reproduce Zipf's meaning-frequency law. Fourth, we will review a family of optimization models of communication that was put forward to investigate the origins of Zipf's law for word frequencies (Ferrer-i-Cancho & Díaz-Guilera, 2007) but that has recently been used to shed light on patterns of vocabulary learning and the mapping of words into meanings (Ferrer-i-Cancho, 2016c). Interestingly, models from that family give Eq. 1 with δ = 1 (Ferrer-i-Cancho, 2016b). Crucially, however, the true exponent is δ ≈ 1/2 (Zipf, 1945;Ilgen & Karaoglan, 2007). The mismatch should not be surprising.
Imagine that a speaker has to choose a word for a certain meaning. Those models assume that given a meaning, all the words with that meaning are equally likely (Ferrer-i-Cancho & Díaz-Guilera, 2007). However, this simple assumption is not supported by psycholinguistic research (Snodgrass & Vanderwart, 1980). We will show how to modify their definition so that the words that are used for a certain meaning do not need to be equally likely and one can obtain Eq. 1 with δ = 1/2 or relaxed versions. Finally, we will discuss the results, highlighting the connection with biased random walks, and indicate directions for future research.

A mathematical framework
As in the family of models of communication above, we assume a repertoire of n words, s 1 ,...,s i ,...s n and a repertoire of m meanings, r 1 ,...,r i ,...,r m . Words and meanings are associated through an n × m adjacency matrix A = {a ij }: a ij = 1 if s i and r j are associated (a ij = 0 otherwise). A defines the edges of an undirected bipartite network of word-meaning associations. The degree of the i-th word is while the degree of the j-th meaning is In human language, the relationship between sound and meaning has been argued to be arbitrary to a large extent (Saussure, 1916;Hockett, 1966;Pinker, 1999). That is, there is no intrinsic relationship between the word form and its meaning. For example the word "car" is nothing like an actual automobile. An obvious exception are onomatopoeias, which are relatively rare in language. However, despite the immense flexibility of the world's languages, some sound-meaning associations are preferred by culturally, historically, and geographically diverse human groups (Blasi, Wichmann, Hammarström, Stadler, & Christiansen, 2016). The framework above is agnostic concerning the type of association between sound and meaning. By doing that, we are borrowing the abstract perspective of network theory, that is a priori neutral concerning the nature or the origins of the edges (Newman, 2010;Barthélemy, 2011). Our framework could be generalized to accommodate Peirce's classic types of reference, i.e., iconic, indexical and symbolic (Deacon, 1997), or the state-of-the-art on the iconicity/systematicity distinction (Dingemanse, Blasi, Lupyan, Christiansen, & Monaghan, 2015). An crucial reason to remain neutral is that the distinctions above were not made when defining the laws of meaning that are the target of this article.
The framework allows one to model lexical ambiguity: a lexically ambiguous word is a word such that its degree is greater than one. Although the model starts from a flat hierarchy of concepts (by default all concepts have the same degree of generality), a word with an abstract meaning could be approximated either as a word linked to a single abstract concept or as a word linked to the multiple specific meanings it covers (Ferrer-i-Cancho, 2016c). As for the latter approach, the word for vehicle would be linked to the meanings for car, bike, ship, airplane,... Suppose that p(s i , r j ) is the joint probability of the unordered pair formed by s i and r j . By definition, The probability of s i is and the probability of r i is Our model shares the assumptions of distributional semantics that the meaning of a word is represented as a vector of the weights of different concepts for that word (Lund & Burgess, 1996). In our framework, the meaning of the word s i , is represented by the m-dimensional vector The joint probabilities p(s i , r j ) for all words and meanings defines a weighted matrix of the same size of A. In the coming sections, we will derive the meaning frequency-law defining p(s i , r j ) as a function of A. Put differently, we will derive the law from a weighted undirected bipartite graph that is build from the unweighted undirected graph defined by A. This organization in two graphs (one unweighted and the other weighted) instead of a single weighted graph is borrowed from successful models of communication (Ferrer-i-Cancho, 2016c) and allows one to apply the theory of random walks (Sinatra et al., 2011;Gómez-Gardeñes & Latora, 2008) as we will see later on.

A minimalist derivation of the law
The law of meaning distribution can be derived by making just one rather simple assumption, i.e.
Applying Eq. 10, one obtains where c is a normalization constant defined as Notice that c is not a parameter and its value if determined by the definition of probability in Eq. 10. Applying Eq. 11 to Eq. 14 gives namely Eq. 1 with δ = 1/2. Our derivation of the strong and relaxed version of the meaning-frequency law is simpler than that of Zipf's in the sense that it requires assuming a smaller number of equations (we are assuming only Eq. 13 while Zipf assumed Eqs. 3 and 4). However, the challenge of our approach is the justification of Eq. 13.

A theoretical derivation of the law
The definition of p(s i , r j ) in Eq. 13 suffices as a model but not for the construction of a real theory of language. Eq. 13 is simple but somewhat arbitrary: the degree of the word, µ i , contributes raised to 1 but the degree of the meaning, ω j has no direct contribution, or one may say that it contributes raised to 0. Therefore, a less arbitrary equation would be where φ is a positive parameter (φ ≥ 0). Applying Eq. 10 to Eq. 15, one obtains with c = 1/M and Notice that φ is the only parameter of the model given n and m. Applying Eq. 11 to Eq. 16, one obtains Eq. 15 is theoretically appealing for various reasons. If p(s i , r j ) is regarded as the weight of the association between s i and r j , it defines the general form of the relationship between the weight of and edge and the product of the degrees of vertices at both ends that is found in real networks (Barrat, Barthélemy, Pastor-Satorras, & Vespignani, 2004). For this reason, a unipartite version of Eq. 15 is assumed to study dynamics on networks (Baronchelli, Castellano, & Pastor-Satorras, 2011). When φ = 0, it matches the definition of models about the origins of Zipf's law for word frequencies (Ferrer-i-Cancho, 2005b), the variation of the exponent of the law (Ferrer-i-Cancho, 2005a, 2006 and vocabulary learning (Ferrer-i-Cancho, 2016c). When φ = 1, it defines an approximation to the stationary probability of observing a transition involving s i and r j in a random walk on a network that is biased to maximize the entropy rate of the walks (Appendix A), thus suggesting that the meaning-frequency law could be a manifestation of a particular random walk process on semantic memory.
Two equivalent linguistic principles, the principle of contrast (Clark, 1987) and the principle of no synonymy (Goldberg, 1995, p. 67) can be implemented in our model as ω j ∈ {0, 1}. From an algebraic standpoint, the condition ω j ∈ {0, 1} is equivalent to orthogonality of the word vectors of the matrix A. If A i * indicates the row vector of A for the i-th word, A i * and A k * are orthogonal if and only if A k * · A k * = 0, where the dot indicates the scalar product of two vectors. To simplify matters, we assume that there is no row vector of A that equals 0, a vector that has 0 in all components. So far we have used µ i and ω j to refer, respectively, to the degree of the i-th word and the j-th meaning. We define µ e i and ω e i as the degree of the word and the degree of the meaning of the i-th edge. µ e i and ω e i are components of the vectors µ e i and ω e i , respectively. We have µ e i · ω e i > 0 because µ e i , ω e i ≥ 1 by definition. A deeper insight can be obtained with the concept of remaining degree, the degree at one end of the edge after subtracting the unit contribution of the edge (Newman, 2002). The vectors of remaining degrees are then The condition ω j ∈ {0, 1} is equivalent to ω ′e i = 0. ω j ∈ {0, 1} leads to µ ′e i ω ′e i = 0 but trivially for being ω ′e i null. The assumption φ = 1 and ω j ∈ {0, 1} (orthogonality of row vectors of A), transform Eq. 18 into Eq. 13 because a ij = 0 and ω j = 0 are equivalent when ω j does not exceed 1. In general, Eq. 18 combined with the principle of contrast gives When φ = 1, we get δ = 1/2 again. Interestingly, the principle of contrast follows from the principle of mutual information maximization, a more fundamental principle that allows one to predict vocabulary learning in children and that can be combined with the principle of entropy minimization to predict Zipf's law for word frequencies (Ferrer-i-Cancho, 2016c). With Eq. 15, we follow Bunge (Bunge, 2013, pp. 32-33) preventing scientific knowledge from becoming "an aggregation of disconnected information" and aspiring to build a "system of ideas that are logically connected among themselves". It is possible to obtain a relaxed meaning frequency-law under more general conditions. In particular, we would like to get rid of the heavy constraint that meaning degrees cannot exceed one. Suppose that d is a constant such that 0 < d ≤ n. Some obvious but not very general conditions are ω j ∈ {0, d} for all j or ω j = d for all j. It is easy to see that they lead again to Eq. 13 when φ = 1. A more general condition can be defined as follows. First, we define E[ω φ |µ] as the conditional expectation of ω φ given µ for an edge. Here µ and ω are the degrees at both ends of an edge. Then suppose that A is given and that (Kolmogorov, 1956;Poirier, 1995), then the expectation of p(s i ) (as defined in Eq. 18) knowing µ i is which can be seen as a regression model (Ritz & Streibig, 2008) for the meaningfrequency law (Eq. 1) with word degree as predictor. Notice that mean independence is a more general condition than mutual or statistical independence but a particular case of uncorrelation (Ferrer-i-Cancho, Hernández-Fernández, Baixeries, Dębowski, & Mačutek, 2014). So far, we have seen ways of obtaining the meaning-frequency law from Eq. 15 making further assumptions. It is possible to obtain a relaxed version of the meaning-frequency law making no additional assumption (except Eq. 15 or the biased random walk that justifies it). Eq. 18 can be expressed as with Assuming that or equivalently .
Recalling p ≈ f /L, these results can be summarized using asymptotic notation as f = Θ(µ φ+1 ) or µ = Θ(f 1/(φ+1) ). The power of the bounds above depends on the gap between T min and T max . The gap can be measured with the ratio where ω min and ω max are the minimum and the maximum meaning degree, respectively. The principle of mutual information maximization between words and meanings, a general principle of communication (Ferrer-i-Cancho, 2016c), puts pressure for concordance with the meaning-frequency law. To see it, we consider two cases: n ≤ m and m ≤ n. When n ≤ m, its maximization predicts ω j ≤ 1 (Appendix B). As unlinked meanings are irrelevant (they do not belong to the support set), we have ω min = 1. As pressure for mutual information maximization increases, ω max tends to 1 and thus T max /T min tends to 1. Put differently, the gap between the upper and the lower bound in Eq. 20 reduces as pressure for mutual information maximization increases. When n ≥ m, mutual information maximization predicts that ω j = d, where d is an integer such that d ∈ [1, ⌊n/m⌋] (Appendix B). We have seen above that one obtains the meaning-frequency law (Eq. 1) immediately from Eq. 15 when ω j is constant. We conclude that the chance of observing the meaning-frequency law increases as pressure for mutual information maximization increases.

A family of optimization models of communication
Here we revisit a family of optimization models of communication (Ferrer-i-Cancho & Díaz-Guilera, 2007) in light of the results of the previous sections. These models share the assumption that the probability that a word s i is employed to refer to meaning r j is proportional to a ij , i.e.
Applying n i=1 p(s i |r j ) = 1 to Eq. 21, we obtain We adopt the convention p(s i |r j ) = 0 when ω j = 0. Eq. 22 defines the probability of transition of a standard (unbiased) random walk to a word (Noh & Rieger, 2004), i.e. given a meaning, all related words are equally likely. This is unrealistic in light of picture naming norms (Snodgrass & Vanderwart, 1980;Duñabeitia et al., 2017). Consider the picture-naming norms compiled by Snodgrass and Vanderwart (1980), who simply asked participants to name 260 black-and-white line drawings of common objects. For some objects (e.g., balloon, banana, sock, star) there was 100% agreement among the participants for the word used to name the pictured object. However, for other objects there was considerable variability in the word used to name the pictured object. Important for the present argument, the other words that were used in such cases were not selected with equal likelihood. For example, the picture of a wineglass had 50% agreement, with the word glass (36% of the responses) and the word goblet (14% of the responses) also being used to name the object, showing that all the words that could be used for a given meaning are not equally likely. Although subjects tend to provide more specific responses when the concept is presented in textual form with respect to a visual form presentation (Tversky & Hemenway, 1983), we used the visual case simply to challenge the assumption of an unbiased random walk in general and justify a more realistic approach.
In contrast to Eq. 22, the fundamental assumption in Eq. 15 leads to namely the transition probabilities of a biased random walk when φ > 0 (Sinatra et al., 2011;Gómez-Gardeñes & Latora, 2008). To see it, notice that the combination of Eq. 12 and 16 produces Recalling the definition of conditional probability and applying Eq. 16 again, one obtains Eq. 23.
Recalling the definition of ω j in Eq. 9, it is easy to realize that Eq. 22 is a particular case of Eq. 23 with φ = 0. While the family of models above stems from a concrete definition of a conditional probability, i.e. p(s i |r j ) in Eq. 22, the general model that we have presented in this article is specified by a definition of the joint probability, i.e. p(s i , r j ) in Eq. 15.
Models within that family are generated through assuming an unbiased random walk from a meaning to a word (Eq. 22) and making different assumptions on p(r j ).
If one assumes that all meanings are equally likely (p(r j ) = 1/m with ω j ≥ 1) one obtains the 1st model (Ferrer-i-Cancho & Solé, 2003). If one assumes that the probability of a meaning is proportional to its degree (p(r j ) ∝ ω j ) one obtains the 2nd model (Ferrer-i-Cancho, 2005b). While in the 2nd model p(r j |s i ) defines an unbiased random walk from s i to r j (all r j 's connected to s i are equally likely), this is not necessarily the case for the 1st model (Ferrer-i-Cancho & Díaz-Guilera, 2007). Therefore, the 2nd model defines a pure unbiased random walk while the 1st model is unbiased from meaning to words but biased from words to meanings. Now we will introduce a generalized version of the family of models above consisting of replacing Eq. 22 by Eq. 23 and generating the corresponding variants of the 1st and the 2nd model applying the same procedure as in the original family, namely via Eq. 25. Notice that Eq. 23 defines the probability of reaching s i from r j in a biased random walk when φ > 0.
Concerning the 1st model, suppose that the probabilities of the meanings are given a priori (they are independent from the A matrix), e.g., all meanings are equally likely. Then it is easy to show that the model yields a relaxed version of the meaning frequency law, namely µ i = Θ(p(s i ) δ ), the number of meanings is bounded above and below by two power-laws, i.e. (Appendix C) where b 1 and b 2 are constants (b 1 ≤ b 2 ) and δ = 1/(φ + 1). Eq. 26 defines non-trivial bounds when δ = 1 (Appendix C). The case δ = 1 matches that an optimization model of Zipf's law for word frequencies (Ferrer-i-Cancho, 2005b, 2016b.
To generate a variant of the 2nd model, recall that Eq. 23 comes from Eq. 15. Eqs. 12 and 16 produce Eq. 24. This variant of the 2nd model derives all probability definitions from Eq. 15. We have shown above that this variant is able to generate the meaning-frequency law.

Discussion
We have seen that it is possible to obtain the meaning-frequency law (Eqs. 1 and 2) from Eq. 15 making certain assumptions. We have also seen that a relaxed version of the law (Eq. 7 can be obtained from Eq. 15 without making any further assumption. Our findings suggest that word probabilities are channeled somehow to manifest the meaning-frequency law. We have seen that the principle of mutual information maximization contributes to the emergence of the law. Our derivation is theoretically appealing for various reasons. First, it is more parsimonious than G. K. Zipf's concerning the number of equations that are assumed (we only need Eq. 15 while Zipf involved Eqs. 3 and 4). Second, it can help a family of optimization models of language to reproduce the meaning-frequency law. Therefore, a crucial assumption is Eq. 16, that we have justified as the outcome of a random walk that is biased to maximize the entropy rate of the paths (Appendix A). A random walk is the correlate in network theory of the concept of mental exploration (navigation without a target or nonstrategic search) in cognitive science and related fields (Baronchelli, Ferrer-i-Cancho, Pastor-Satorras, Chatter, & Christiansen, 2013). Semantic memory processes can be usefully theorized as searches over a network (Thompson & Kello, 2014;Abbott, Austerweil, & Griffiths, 2015) or some semantic space (Smith, Huber, & Vul, 2013). These approaches support the hypothesis of a Markov chain process for memory search (Bourgin, Abbott, Griffiths, K.A., & Vul, 2014), provide a deeper understanding of creativity (Kenett & Austerweil, 2016a) and help to develop efficient navigation strategies (Capitán et al., 2012).
A random walk in a unipartite word network of word-word associations has been argued to underlie Zipf's law for word frequencies (Allegrini, Gricolini, & Palatella, 2004). Here we contribute with a new hypothesis linking random walks with a linguistic law: that the meaning-frequency law would be an epiphenomenon of a biased random walk over a bipartite network of word-meaning associations in the process of mental exploration. The bias consists of exploiting local information, namely the degrees of first neighbours (Sinatra et al., 2011). Transitions to nodes with higher degree are preferred. Our model shows that it is possible to approximate the optimal solution to a problem (maximizing the entropy rate of the paths) following an apparently nonstrategic search (Hills, Jones, & Todd, 2012;Abbott et al., 2015).
The probability of a word in Eq. 18 defines the probability that a random walker visits the word in the long run. This probability is what the PageRank algorithm estimates in the context of a standard (non-biased) random walk (Page, Brin, Motwani, & Winograd, 1998). The assumption of a random walk with the particular bias above could help to improve random walk/PageRank methods to predict the prominence in memory of a word (Griffiths, Steyvers, & Firl, 2007) or the importance of a tag (Jäschke, Marinho, Hotho, Schmidt-Thieme, & Stumme, 2007). A virtue of our biased random walk is that it predicts an uneven conditional probability of a word given a meaning (Eq. 23) as it happens in real language (Snodgrass & Vanderwart, 1980). A standard (uniform) random walk cannot explain this fact and for that reason the family of optimization models of language revisited above fails to reproduce the meaningfrequency law with δ = 1/2.
Although biased random walks have already been used to solve information retrieval problems (see Duarte Torres, Hiemstra, Weber, and Pavel (2014) and references therein), a bias based on the degree of neighbours has not been considered as far as we know. We hope that our results stimulate further research on linguistics laws and biased random walks in the information sciences. Specifically, we hope that our article becomes the fuel of future empirical research.

Acknowledgements
We are specially grateful to R. Pastor-Satorras and Massimo Stella for helpful comments and insights. We also thank S. Semple, M. Gentili and E. Bastrakova for helpful discussions. This research was supported by the grant AP-COM (TIN2014-57226-P) from MINECO (Ministerio de Economía y Competitividad) and the grant 2014SGR 890 (MACDA) from AGAUR (Generalitat de Catalunya).

A Random walks
We will show that Eq. 15 defines the probability of observing a transition between s i and r j in any direction in a biased random walk. We will proceed in two steps. First, we will summarize some general results on biased random walks on unipartite networks and then we will adapt them to bipartite networks.
Suppose a unipartite network of n nodes that is defined by an n×n adjacency matrix B = {b ij } such that b ij = 1 if the i-th and the j-th node are connected and b ij = 1 otherwise. Let k i be the degree of the i-th node, namely, Suppose a random walk over the vertices of a network where p(j|i) is the probability of jumping from i to j. A first order approximation to the p(j|i) that maximizes the entropy rate is (Sinatra et al., 2011) We choose a generalization (Gómez-Gardeñes & Latora, 2008) that gives Eq. 27 with φ = 1. The stationary probability of visiting the i-th vertex in the biased random walk defined by Eq. 28 is (Gómez-Gardeñes & Latora, 2008) where and Now we adapt the results above to a bipartite graph of word-meaning associations. As the graph is bipartite, the random walker will be alternating between words and meanings. The probability that the vertex visited is a word is 1/2 (the same probability for a meaning). Suppose that there are n words and m meanings. Recall that the bipartite network of word-meaning associations is defined by an n × m adjacency matrix A = {a ij } such that a ij = 1 if the i-th word and the j-th meaning are connected and a ij = 1 otherwise. µ i is the degree of the i-th word is (Eq. 8) whereas ω j is the degree of the j-th meaning (Eq. 9). The probability of jumping from r j to s i becomes (recall Eq. 28) The probability of jumping from s i to r j is The stationary probability of visiting the word s i becomes (recall Eq. 29 and 30) where M v corresponds to T in Eq. 29. Adapting Eqs. 31 and 30, one obtains where M is defined as in Eq. 17. Applying Eq. 33, it is easy to see that as expected.
The combination of Eqs. 32 and 33 allows one to derive the probability of observing the transition from s i to r j as where c v = 1/(2M ). Similarly, the probability of observing the transition from r j to s i is Therefore the stationary probability of observing a transition between s i and r j in any direction (from s i to r j or from r j to s i ) is with c = 1/M , as we wanted to show. Finally, we will link p(s i ), the probability of a word that is used in the main text to derive the meaning-frequency law, with p v (s i ). Notice that p(s i ) = p v (s i |S), the latter being the probability of visiting vertex s i knowing that it belongs to the partition S, the partition of words. Since the graph is bipartite, p v (S), probability that the random walk is visiting a vertex of partition S, is 1/2. The joint probability of visiting vertex s i and that the vertex belongs to S is Then p(s i ) is the stationary probability of visiting s i in a biased random walk knowing that the vertex is in S.

B Mutual information maximization
Suppose that I(S, R) is the mutual information between words (S) and meanings (R), that can be defined as where H(S) is the entropy of words and H(S|R) is the conditional entropy of words given meanings. For the case φ = 0, the configurations that maximize I(S, R) when n ≤ m are defined by two conditions (Ferrer-i-Cancho, 2016c) 2. µ i ∈ {0, 1} for i = 1, 2, ..., n.
Here we will show that the configurations that maximize I(S, R) are the same as in the case φ = 0 when φ is a positive and finite real number (φ ≥ 0). By symmetry, it suffices to show it for the case n ≤ m. We will proceed in three steps. First, deriving the configurations minimizing H(S|R). Second, showing that the configurations above yield maximum I(S, R). Third, showing that they are the only configurations.
Step 2: notice that the 2nd condition of the case n ≤ m above implies H(S|R) = 0 (recall Step 1). The 2nd condition transforms Eq. 17 into and then H(S) = log n (as all words are equally likely). Thus, H(S) is taking its maximum possible value whereas H(S|R) is taking its minimum value. As I(S, R) = H(S) − H(S|R), it follows that I(S, R) is maximum.
Step 3: notice that • If the 2nd condition fails, then H(S|R) > 0 and thus I(S, R) < log n even if H(S) is maximum because of Eq. 34. Thus, the 2nd condition is required to maximize I(S, R).
• If the 1st condition fails (while the 2nd condition holds), then words are not equally likely as the probability of a word is proportional to a power of its degree (Eq. B). Then one has that H(S) < log n and it follows that I(S, R) is not maximum because I(S, R) ≤ H(S).

C New models
recalling the definition of µ i in Eq. 8. Equivalently, with δ = 1 φ + 1 , namely a relaxed version of the meaning-frequency law when φ = 1. Notice that Eqs. 36 and 37 define non-trivial bounds in the sense that they are not expected from bounds on join-probability alone. If the range of variation of p(s i , r j ) satisfies π min ≤ p(s i , r j ) ≤ π max when p(s i , r j ) > 0, then Eq. 11 gives m j=1 a ij π min ≤ p(s i ) ≤ m j=1 a ij π max and then π min µ i ≤ p(s i ) ≤ π max µ i .
Therefore, the finding that where b 1 and b 2 are constants is trivial when δ = 1.