Word
inductive bias
Definition
n. the built-in assumptions or shortcuts an AI algorithm uses to prioritize one explanation over another.
n. the set of structural and algorithmic assumptions a learning system uses to predict outputs on unseen inputs before observing any data.
Examples
“Assuming that smoother lines are more likely than jagged lines is a common inductive bias.”
“Convolutional networks possess a strong inductive bias toward translation invariance, making them ideal for image analysis.”
“While strong inductive bias enables rapid learning from scarce data, unconstrained architectures like transformers eventually dominate when compute and data scale up.”
Examples
simple
“Assuming that smoother lines are more likely than jagged lines is a common inductive bias.”
contextual
“Convolutional networks possess a strong inductive bias toward translation invariance, making them ideal for image analysis.”
complex
“While strong inductive bias enables rapid learning from scarce data, unconstrained architectures like transformers eventually dominate when compute and data scale up.”
Real-World Examples
“The no-free-lunch theorem for machine learning [34,57] basically says that some set of preferences (or inductive bias) over the space of all functions is necessary to obtain generalization, that there is no completely general-purpose learning algorithm, that any learning algorithm will generalize better on some distributions and worse on others.” “Cognitive scientists and machine learning researchers often focus on the architecture of a neural network as a source of inductive bias.” Etymology
Introduced by computer scientist Tom M. Mitchell in 1980 in 'The Need for Biases in Learning Generalizations', formalizing the philosophical principle that learning requires priors.
Etymology adapted from Wiktionary, available under CC BY-SA 4.0.
Domains
Scan code
englishreference.com/q/inductive-bias