# Why the logarithm?

A companion to ML006: derive surprise from three rules, count yes/no questions, rule out simpler candidates, see why fractional bits are averages and build entropy as average surprise.

Every number is exact: 1 in 8 is 3 questions, a die is 2.585 bits, and the ½ ¼ ⅛ ⅛ bag averages 1.75 bits, matching its question tree.

Based on Claude Shannon, A Mathematical Theory of Communication (1948). Original examples and numbers by 21IFM.

Canonical page: https://21ifm.com/ai/entropy-why-logarithm

## Explore

- [Entropy and cross-entropy (ML006)](https://21ifm.com/ai/entropy)
- [Exponentials and logarithms](https://21ifm.com/basics/exponentials-logarithms)