Search Results

3 results for “activation”

Formulae (3)

\sigma(x) = \frac{1}{1 + e^{-x}}

Maps any real number to the range (0, 1); used in logistic regression.

\sigma(z_i) = \frac{e^{z_i}}{\sum_{j=1}^{K} e^{z_j}}

Converts a vector of scores into a probability distribution.

f(x) = \max(0, x)

Rectified Linear Unit: outputs the input if positive, else zero.