"Attention is All You Need" Paper

In more detail

Published by Vaswani et al., introduced self-attention and the Transformer. Enabled training on longer sequences efficiently. Foundation for GPT, BERT, T5, and modern language models. One of most cited ML papers.

Examples

• Transformer architecture
• All modern LLMs
• BERT
• GPT series
• Vision Transformers

Related terms