Machine Learning & AI

Advanced

Q-Learning Update

Model-free reinforcement learning rule that learns action values from experience.

Formula

Q(s,a):=Q(s,a)+\alpha\big[r+\gamma\max_{a^{\prime}}Q(s^{\prime},a^{\prime})-Q(s,a)\big]

Variables

\alphaLearning rate

rReward

\gammaDiscount

s^{\prime}Next state

Example

The bracket is the temporal-difference error

Did You Know?

Deep Q-Networks used this rule to learn Atari games directly from raw pixels in 2013.

reinforcement learning q-learning td

Share this formula

More in Machine Learning & AI

Linear Regression Model

Predicts a continuous value as a weighted sum of input features plus a bias.

Gradient Descent Update

Iteratively moves parameters in the direction that most reduces the loss.

ReLU Activation

Rectified Linear Unit: outputs the input if positive, else zero.

Leaky ReLU

A ReLU variant that lets a small gradient flow for negative inputs.

Tanh Activation

Squashes input to the range (-1, 1); zero-centred activation.

Softmax

Turns a vector of scores into a probability distribution that sums to 1.