Self-attention is the mechanism within transformer models that allows each token in a sequence to attend to every other token, computing relevance scores that determine how much each word should influence the representation of every other word. It is what lets a model understand that in «the cat sat on the mat», «it» refers to «the cat» and not «the mat». Self-attention replaced the sequential processing of RNNs with a parallelizable approach that scales efficiently.