Attention Is All You Need

Paper that revolutionized deep learning AI and introduced transformers.

Main idea: allow AI to read entire sentences and develop context about the relationships between words. Attention represents the relative weight given to each unit within the sentence.

Example: “The black cat walks across the street.”

AI understands that “black” refers to the cat, not the street. It also understands that the cat is walking, not the street.