Word
logit
Definition
n. the raw numerical score produced by an AI model before it is turned into a percentage or probability.
n. the unnormalized vector of real-valued outputs produced by the final linear layer of a classification neural network prior to activation via softmax.
Examples
“The highest logit corresponds to the model's top guess for the next token.”
“Temperature scaling divides each logit by a constant factor to smooth out predicted probabilities.”
“Knowledge distillation minimizes Kullback-Leibler divergence between soft teacher logits and student predictions to transfer dark knowledge.”
Examples
simple
“The highest logit corresponds to the model's top guess for the next token.”
contextual
“Temperature scaling divides each logit by a constant factor to smooth out predicted probabilities.”
complex
“Knowledge distillation minimizes Kullback-Leibler divergence between soft teacher logits and student predictions to transfer dark knowledge.”
Real-World Examples
“For models doing classification tasks (let's call them cls_head(s),) a neural network's final layer produces raw, unnormalized scores for each class. These scores are logits, and most often they are immediately processed again into a label (by first dividing by temp, optional) and then softmax, and then either argmax or multinomial pick from this distribution.” “LLMs process text through multiple layers, generating " logits " (prediction scores) at each layer, with the final layer's logits typically determining the output.” “The intuition for transforming using the logit function (the natural log of the odds) was explained above [ clarification needed ] .” Etymology
Coined in 1944 by American statistician Chester Ittner Bliss as a blend of log (logarithm) and probit (probability unit), to denote the log-odds of a probability.
Etymology adapted from Wiktionary, available under CC BY-SA 4.0.
Domains
Scan code
englishreference.com/q/logit