We use cookies to give you a better experience. Carry on browsing if you're happy with this, or read our cookies policy for more information.

Skip main navigation

Zipf’s law

In this video we discuss Zipf's law: a curious relation that connects populations of cities and distributions of words to inverse relations.
12.1
Inverse proportionalities turn out to have interesting applications in surprising directions. One such was noted by an American linguist, George Zipf, who discovered that if you looked at words in the English language, they satisfied an interesting pattern in terms of frequency and rank. It was basically an inverse proportionality that he was seeing in the words, the most popular words, of the English language. After his discovery, the same pattern was surprisingly discovered in a lot of other different seemingly unrelated phenomena, including populations of cities, numbers of corporations, income levels, and so on. So there’s quite a lot of surprising connections, the inverse relationship manifesting itself in surprising ways. So what is Zipf’s law?
73
So named after George Kingsley Zipf, who lived from 1902 to 1950. And here’s his law, that in many classical texts, the frequency of a given word is inversely proportional to its rank in the frequency table. What does that mean? He’s talking about the most popular or most frequently used words in the English language. So if you take a big book and you count the number of times each word appears, then the word “the” will almost certainly appear more than any other word. That’s the most popular word in the English language in terms of its usage. And in fact, generally speaking, it occurs about 7% of the time. So 7% of words in a typical text are the word “the”.
122.5
The next most popular word is “of”, which generally speaking is around the 3.1% level, followed by “to” at 2.7%; “and” at 2.6%; “in” at 1.8%; “is”, 1.2%; “for”, 1.0%; and “that”, 0.8%. So what did Zipf observe about these steadily decreasing numbers? He observed that the 3.1% is roughly half of the top entry, 6.8%. He observed at the next one, 2.7%, is roughly 1/3 of the top entry, 6.8%, that 2.6% is roughly 1/4 of 6.8%, that 1.8% is roughly 1/5 of 6.8%, and so on. So we can capture this relation mathematically by saying that the frequency times the rank seems to be roughly constant at around 6.8%. So here is rank. Here is frequency in that direction.
201.6
And the product of the two of them is roughly 6.8%, which we could kind of represent with an inverse proportional graph like that.
In this video we discuss Zipf’s law: a curious relation that connects distributions of words and populations of cities to inverse relations. It was first noticed by George Kingsley Zipf, an American linguist, when looking at the relative frequencies of words in a large text, like the book Moby Dick.

The most common words in English

Here are the most frequent words in the English language, along with the rough percentages of how often that word occurs in written texts. For example, the most common word, ‘the’, appears roughly \(\normalsize{6.8\%}\) of the time.
Of the \(\normalsize{92}\) words in the two previous paragraphs, I counted \(\normalsize{9}\) uses of the word ‘the’. That is therefore somewhat above average.
RankWordPercentage
1the6.8
2of3.1
3to2.7
4and2.6
5in1.8
6is1.2
7for1.0
8that0.8
Zipf noticed that the second most common word ‘of’ occurs about half as often as the most common word ‘the’. While the third most common word ‘to’ occurs about a third as often as ‘the’. And so on. The seventh most common word ‘for’ occurs about one seventh as often as ‘the’.
More generally, the frequency of the \(\normalsize{n\text{th}}\) most common word is about \(\normalsize{\frac{1}{n}}\) times the frequency of the most common word.
So a graph of the frequencies of the most common words looks roughly like this:
Graph of frequency in terms of rank of word
This distribution, remarkably, is quite stable over many different publications. For example, data provided by http://norvig.com/mayzner.html support this claim.
Furthermore it turns out that less than \(\normalsize{200}\) words account for more than half of all the written words in English.

The most common words in Russian

Frequencies for the most common words in the Russian language looks roughly like this:
RankWordFrequencyTranslation
1и7.1and, though
2в5.5in, at
3не4.0not
4на3.2on, it, at, to
5что2.6what, that, why
6я2.3I
7с2.2with, and, from, of
8он2.1he
9а1.5while, and, but
8как1.4how, what, as, like

Discussion

If you speak a language other than English or Russian, what are the most common words in that language? Is there a similar pattern as what Zipf noticed?
Is there any sense in applying Zipf’s law to simplify the process of learning a foreign language at a basic level? How about reading books in foreign languages?
Please let us know in the comments what you think.
This article is from the free online

Maths for Humans: Linear, Quadratic & Inverse Relations

Created by
FutureLearn - Learning For Life

Our purpose is to transform access to education.

We offer a diverse selection of courses from leading universities and cultural institutions from around the world. These are delivered one step at a time, and are accessible on mobile, tablet and desktop, so you can fit learning around your life.

We believe learning should be an enjoyable, social experience, so our courses offer the opportunity to discuss what you’re learning with others as you go, helping you make fresh discoveries and form new ideas.
You can unlock new opportunities with unlimited access to hundreds of online short courses for a year by subscribing to our Unlimited package. Build your knowledge with top universities and organisations.

Learn more about how FutureLearn is transforming access to education