Sequences and Natural Language Processing

How machines read: turning text into numbers, bag of words and TF-IDF, word embeddings, recurrent networks, LSTMs and the attention idea that led to transformers.

6 lessonsabout 75 min

Text is the most common data most teams have: support tickets, reviews, search queries, contracts. Before a model can use any of it, the words have to become numbers.

You will start with preprocessing and tokenization, then represent documents as word counts and TF-IDF weights, the fast baselines that still win many text tasks. Next come word embeddings, where meaning becomes geometry. Then you will process text as a sequence with recurrent networks, see why LSTMs and GRUs fixed their memory problems, and finish with sequence-to-sequence models and attention, the bridge to the transformers of the AI track.

Lessons

ML-10.01 Text preprocessing and tokenization basics Text must be turned into numbers. Beginner 10 min
ML-10.02 Bag of words and TF-IDF Bag of words represents a document by counting each word, ignoring order. Beginner 10 min
ML-10.03 Word embeddings Word embeddings represent words as dense vectors learned from context: words that appear in similar contexts get similar vectors. Intermediate 14 min
ML-10.04 Recurrent neural networks A recurrent neural network processes a sequence one element at a time, carrying a hidden state that summarizes what it has seen so far. Intermediate 14 min
ML-10.05 LSTM and GRU LSTMs add a separate cell state and gates that learn what to forget, what to store and what to output, which lets information and gradients flow across long sequences. Intermediate 14 min
ML-10.06 Seq2seq and attention Sequence-to-sequence models encode an input sequence and decode an output sequence, as in translation. Intermediate 14 min

Start with lesson 1