Real World Maths Busters

The Maths Behind Everyday Life

The story, the formula and a worked example — see exactly where this maths shows up in the world around you.

Back to all stories
Technology 3 min read

A Billion Parameters in Your Pocket

GPT-3 has 175 billion adjustable numbers that encode language.

AI language models learn by adjusting billions of numerical parameters during training. GPT-3’s 175 billion parameters store patterns of how words relate to each other.

Storage Maths

Each parameter is typically stored as a 16-bit (2-byte) number. 175 billion parameters = 175 × 10⁹ × 2 bytes = 350 GB just for the weights.

Storage=Parameters×Bytes per param\text{Storage} = \text{Parameters} \times \text{Bytes per param}

175 billion parameters at 16-bit precision

  1. 1

    Bytes

    175×109×2=3.5×1011 bytes175 \times 10^9 \times 2 = 3.5 \times 10^{11} \text{ bytes}
  2. 2

    In GB

    3.5×1011109=350 GB\frac{3.5 \times 10^{11}}{10^9} = 350 \text{ GB}
Answer

350 GB just for the model weights — plus memory for computation

The takeaway

An AI model is just a vast grid of numbers — each one nudged during training to capture patterns in language.

Back to all stories