Real World Maths Busters

The Maths Behind Everyday Life

The story, the formula and a worked example — see exactly where this maths shows up in the world around you.

Back to all stories
Technology 4 min read

Why Training GPT-4 Cost $100 Million

Trillions of calculations at pennies per billion.

Training a large language model requires an astronomical number of floating-point operations on thousands of GPUs for months.

Cost estimation

Cloud GPU cost is roughly $0.005 per petaFLOP.

cost=FLOPs1015×$0.005\text{cost} = \frac{\text{FLOPs}}{10^{15}} \times \$0.005

2 × 10²⁵ FLOPs

  1. 1

    PetaFLOPs

    2 × 10¹⁰

  2. 2

    Cost

    2 × 10¹⁰ × $0.005 ≈ $100M

Answer

Roughly $100 million in compute

The formula behind it

Scientific Notation

a×10na \times 10^n

The takeaway

Training AI is expensive because the maths is simple but the scale is mind-boggling.

Back to all stories