Search Results

7 results for “entropy”

Formulae (7)

L = -\frac{1}{n}\sum[y\log(\hat{y}) + (1-y)\log(1-\hat{y})]

Loss function for binary classification.

H = -\sum_{i=1}^{n} p_i \log_2 p_i

Measures the uncertainty or information content of a distribution.

L = -\sum_{i=1}^{K} y_i \log(\hat{y}_i)

Loss for multi-class classification with one-hot labels.

\Delta S_{universe} \ge 0

The entropy of an isolated system never decreases.

\Delta S_{universe} \geq 0

Total entropy of an isolated system never decreases.

\Delta S = \frac{Q}{T}

Reversible entropy change at temperature T.

H = -\sum_i p_i \log_2 p_i

Average information content of a random source (bits).