Knowledge Graph — Coursera Notes › Academic disciplines › Information Technology / Computer Science › Artificial Intelligence › Deep Learning

Transformer

concept · part of Deep Learning

Underlying architecture of modern LLMs

They use self-attention mechanisms to capture intricate relationships within data sequences, enabling human-like language generation. Examples include GPT for text generation.

Transformers process all tokens in parallel rather than sequentially, which enables efficient training on large datasets. They consist of an encoder and decoder stack, with each layer containing multi-head self-attention and feed-forward neural networks. Positional encodings are added to input embeddings to retain sequence order information.

Inside Transformer (2)

• BERT — A Transformer variant used for text classification, translation, summarization, and generation.
• Self-attention — Mechanism in transformers that weighs the importance of different words in a sequence.

Connections

• Alternative to RNN
• Related to Deep Learning
• Related to BERT
• Related to GPT
• Related to Self-attention
• Related to BERT
• Related to GPT

This is the text view of an interactive 3D knowledge graph — open this page with JavaScript enabled to explore it visually.