Attention

Today we are going to talk about Attention. This is third part of the NLP series. See summary and other posts here.

Transformer & BERT

Attention

Transformer based architectures are built around idea of Attention. In the paper “Attention is All You Need”

Reference

• CS229 winter 2019 course website , Lecture ()

• Attention is All You Need

• The Annotated Transformer

• blog post [Learning Word Embedding] (https://lilianweng.github.io/lil-log/2017/10/15/learning-word-embedding.html) by Lilian Weng

• Up Next

I will post more learning notes about NLP. Stay tuned!