Machine Learning & AI

Advanced

Bellman Equation

Expresses the value of a state as the best immediate reward plus discounted future value.

Formula

V(s)=\max_a\Big[R(s,a)+\gamma\sum_{s^{\prime}}P(s^{\prime}\mid s,a)V(s^{\prime})\Big]

Variables

V(s)State value

RReward

\gammaDiscount factor

PTransition probability

Example

gamma near 1 values the long-term future

Did You Know?

Richard Bellman coined "dynamic programming" — and the "curse of dimensionality" — in the 1950s.

reinforcement learning bellman value

Share this formula

More in Machine Learning & AI

Linear Regression Model

Predicts a continuous value as a weighted sum of input features plus a bias.

Gradient Descent Update

Iteratively moves parameters in the direction that most reduces the loss.

ReLU Activation

Rectified Linear Unit: outputs the input if positive, else zero.

Leaky ReLU

A ReLU variant that lets a small gradient flow for negative inputs.

Tanh Activation

Squashes input to the range (-1, 1); zero-centred activation.

Softmax

Turns a vector of scores into a probability distribution that sums to 1.