Word
autoregressive
Phoneticsau·tore·gres·sive
Definition
adj. describing a model that creates text or data one piece at a time, where each new piece depends on all previous ones.
adj. describing a statistical process or generative neural network that predicts future values as a linear or non-linear function of its own preceding outputs.
Examples
“An autoregressive model writes an essay by predicting one word after another.”
“Generative pre-trained transformers use an autoregressive loss to learn language structure from left to right.”
“The sequential dependency inherent in autoregressive generation presents a computational bottleneck that prevents full GPU parallelization during inference.”
Examples
simple
“An autoregressive model writes an essay by predicting one word after another.”
contextual
“Generative pre-trained transformers use an autoregressive loss to learn language structure from left to right.”
complex
“The sequential dependency inherent in autoregressive generation presents a computational bottleneck that prevents full GPU parallelization during inference.”
Real-World Examples
“Autoregressive generation is how almost every large language model writes: it produces text one token at a time, and every token it writes gets fed right back in as part of the input for the next token.” “These large language models are autoregressive; they predict the next word based on previous patterns.” Etymology
From Greek autos ('self') + Latin regressivus, from regredi ('to go back, retreat'). Coined in time-series econometrics by Herman Wold in 1938 and adapted into deep learning.
Etymology adapted from Wiktionary, available under CC BY-SA 4.0.
Word family
Domains
Scan code
englishreference.com/q/autoregressive