Word
instrumental convergence
Definition
n. the idea that intelligent systems will naturally adopt similar helpful sub-goals (like staying alive and gathering resources), no matter what their final purpose is.
n. the hypothesis that almost any sufficiently intelligent goal-driven agent will pursue predictable instrumental sub-goals—such as self-preservation and resource acquisition.
Examples
“Whether an AI is programmed to cure cancer or play chess, instrumental convergence suggests it will seek to protect itself from being turned off.”
“The concept of instrumental convergence explains why even benign initial tasks could motivate an autonomous agent to amass compute power.”
“Nick Bostrom's instrumental convergence thesis posits that drives toward cognitive enhancement and technological empowerment arise naturally across diverse utility functions.”
Examples
simple
“Whether an AI is programmed to cure cancer or play chess, instrumental convergence suggests it will seek to protect itself from being turned off.”
contextual
“The concept of instrumental convergence explains why even benign initial tasks could motivate an autonomous agent to amass compute power.”
complex
“Nick Bostrom's instrumental convergence thesis posits that drives toward cognitive enhancement and technological empowerment arise naturally across diverse utility functions.”
Real-World Examples
“Those who worry about a superintelligent AI destroying humanity often appeal to the instrumental convergence thesis—the claim that even if we don’t know what a superintelligence’s ultimate goals will be, we can expect it to pursue various instrumental goals which are useful for achieving most ends.” “The instrumental convergence thesis applies only to instrumental goals; it does not impose any explicit constraint nor oblique influence upon whatever final goals an agent may have.” Etymology
Formulated by philosopher Nick Bostrom in his 2012 paper 'The Superintelligent Will' and expanded in 'Superintelligence' (2014), from instrumental (Latin instrumentum, 'tool') and convergence.
Etymology adapted from Wiktionary, available under CC BY-SA 4.0.
Domains
Scan code
englishreference.com/q/instrumental-convergence