Search Results

1 result for “adam”

Formulae (1)

\theta := \theta - \alpha\,\dfrac{\hat{m}_t}{\sqrt{\hat{v}_t}+\epsilon}

Adaptive optimizer combining momentum and per-parameter learning rates.