Word
sparsity
Phoneticsspar·si·ty
UK/spˈɑːsɪti/
Definition
n. having only a few active elements or numbers in a large set, with most values being zero.
n. the mathematical or computational property of matrices, vectors, or representations where the vast majority of elements are equal to zero.
Examples
“Due to sparsity in the desert, towns were separated by hundreds of miles of empty land.”
“Activation sparsity allows sparse autoencoders to isolate clear features without interference.”
“Enforcing L1 weight sparsity during pruning significantly reduces model memory footprints while preserving downstream classification accuracy.”
Examples
simple
“Due to sparsity in the desert, towns were separated by hundreds of miles of empty land.”
contextual
“Activation sparsity allows sparse autoencoders to isolate clear features without interference.”
complex
“Enforcing L1 weight sparsity during pruning significantly reduces model memory footprints while preserving downstream classification accuracy.”
Real-World Examples
“For many models, most of the parameters—the weights and activations—are actually zero, or so close to zero that they could be treated as such without losing accuracy. This quality is known as sparsity.” “Neurons in the brain exhibit two types of sparsity; they are sparsely interconnected and sparsely active. These two types of sparsity, called weight sparsity and activation sparsity, when combined, offer the potential to reduce the computational cost of neural networks by two orders of magnitude.” Etymology
From Latin sparsus, past participle of spargere ('to scatter, strew, sprinkle'). Coined in English statistical mathematics in the mid-20th century.
Etymology adapted from Wiktionary, available under CC BY-SA 4.0.
Word family
Domains
Scan code
englishreference.com/q/sparsity