Word
regularization
Phoneticsreg·u·lar·iza·tion
Definition
n. a set of programming techniques that stops an AI model from memorizing its training data too closely, helping it perform better on fresh examples.
n. the introduction of auxiliary constraints, penalties, or stochastic noise into an optimization problem to prevent overfitting and encourage generalizable solutions.
Examples
“Applying weight regularization prevented the language model from memorizing the training sentences word for word.”
“Dropout is a popular form of regularization that randomly silences half of the neurons during each training step.”
“L2 weight decay provides isotropic quadratic regularization, pulling overparameterized weight trajectories toward the origin during gradient updates.”
Examples
simple
“Applying weight regularization prevented the language model from memorizing the training sentences word for word.”
contextual
“Dropout is a popular form of regularization that randomly silences half of the neurons during each training step.”
complex
“L2 weight decay provides isotropic quadratic regularization, pulling overparameterized weight trajectories toward the origin during gradient updates.”
Real-World Examples
“L2 regularization is a technique used to reduce model complexity and prevent overfitting by penalizing large weights.” “Generalization is good — so good that, to achieve it, we incentivize machines not to memorize, through “regularization”.” “Dropout is a technique for improving neural networks by reducing overfitting. Standard backpropagation learning builds up brittle co-adaptations that work for the training data but do not generalize to unseen data. Random dropout breaks up these co-adaptations by regularization.” Etymology
From regular (Latin regularis, 'containing rules for guidance'), from regula ('rule, ruler'). Introduced mathematically by Andrey Tikhonov in 1963.
Etymology adapted from Wiktionary, available under CC BY-SA 4.0.
Domains
Scan code
englishreference.com/q/regularization