What frequency analysis shows
Counting how often each character or word appears reveals structure that is invisible when reading. The distribution is remarkably stable across large samples of a language — stable enough that it has practical uses in cryptography, compression, linguistics, and text forensics.
In English, the most common letters run roughly E, T, A, O, I, N, S, H, R, with E appearing in about 12% of all letters and Z in under 0.1%. The vowels A, E, I, O, and U together account for around 38% of letters.
Classical cryptanalysis
Frequency analysis is the technique that broke substitution ciphers, and it is the reason they are no longer used seriously.
A simple substitution cipher replaces each letter consistently with another. It preserves the frequency distribution exactly — so if the most common ciphertext letter is Q, it very likely stands for E. Working through the distribution, plus common patterns like double letters and single-letter words, recovers the plaintext without needing the key.
The technique is generally credited to the 9th-century Arab polymath Al-Kindi and has been rediscovered many times since. It is why every serious modern cipher is designed to produce output that is statistically indistinguishable from random noise: any detectable pattern in ciphertext is a foothold.
Compression
Frequency analysis is the foundation of most lossless compression. Huffman coding assigns short bit sequences to frequent symbols and longer ones to rare symbols, so common characters cost fewer bits than uncommon ones.
In English text, giving E a 3-bit code and Z a 12-bit code produces a substantial net saving over fixed 8-bit encoding — typically 40–60% for plain prose. Arithmetic coding and the dictionary methods used in gzip and Brotli refine this considerably, but the underlying insight is the same: exploit the non-uniform distribution.
This is also why already-compressed data does not compress further. Once redundancy has been removed, the distribution is close to uniform and there is nothing left to exploit.
Other practical uses
Language identification. Letter frequency profiles differ enough between languages to identify them from a short sample. Spanish is vowel-heavy; German has a distinctive frequency for certain consonant clusters.
Keyboard layout design. Dvorak and Colemak were designed from frequency data, placing common letters on the home row. QWERTY was designed around mechanical constraints in 19th-century typewriters, which is why it performs poorly on this measure.
Authorship analysis. Function word frequencies — the, of, and, but — are surprisingly consistent per author and largely unconscious, making them useful in stylometry and disputed authorship cases.
Word frequency in your own writing reveals unintended repetition, filler words, and overused constructions in a way that reading rarely does.
Zipf's law
Word frequency in natural language follows a strikingly regular pattern. Zipf's law states that the frequency of a word is roughly inversely proportional to its rank: the most common word appears about twice as often as the second, three times as often as the third, and so on.
In English, the accounts for roughly 7% of all words, of about 3.5%, and about 2.8%. The pattern holds across languages, across text types, and even in some non-linguistic data.
A practical consequence is that a small vocabulary covers most text. The 100 most common English words make up around half of all written material, and 1,000 words cover roughly 85% — which is why frequency-ordered vocabulary lists are so effective for language learning.
Frequently asked questions
What is the most common letter in English?
E, appearing in roughly 12% of all letters. The usual order is E, T, A, O, I, N, S, H, R, and vowels together account for around 38% of letters.
How does frequency analysis break ciphers?
Simple substitution ciphers preserve the frequency distribution, so the most common ciphertext letter is very likely E. Working through the distribution recovers the plaintext without the key.
What is Zipf's law?
Word frequency is roughly inversely proportional to rank - the most common word appears about twice as often as the second. In English, the 100 commonest words make up about half of all written text.
Why does already-compressed data not compress further?
Compression works by exploiting non-uniform frequency. Once redundancy has been removed the distribution is close to uniform, leaving no pattern to exploit.