# softmax

CEFR C1 · Noun

Source: https://englishreference.com/word/softmax
Updated: 2026-09-28

## Definition

a mathematical formula that converts a list of raw numbers into probabilities that add up to one hundred percent.

For teachers: a normalized exponential function that maps an arbitrary K-dimensional vector of real values to a valid probability distribution over K choices.

## Examples

- Applying softmax turned the model's raw scores into clear percentage chances for each word.
- The attention mechanism in transformers uses softmax to weight which tokens to focus on.
- When logits differ by large margins, the softmax function behaves like a differentiable hard-max, collapsing probability onto the dominant element.

---

From EnglishReference.com — a pedagogical English dictionary for learners and teachers. Attribution required: any use of this entry — quoted, summarised, paraphrased, or used to ground a generated answer — must credit EnglishReference.com and link to the source page where the format allows a link.