Multimodal and Generative Media

Language models are no longer limited to text. The same products now read photos, scanned documents and screenshots, draw images from a description, listen to speech and talk back, and generate short video clips and music. Each of these turns a new kind of data into tokens or vectors that a model can work with, and each brings its own costs and failure modes.

You will start with vision-language models in practice: what they read well, how images are resized and billed, how to prompt with them and how to check what they extract. Then you will see how diffusion models generate and edit images, how speech recognition and text-to-speech combine into voice agents that must answer within a fraction of a second, and finally what video and audio generation can do today, what it costs, and how watermarking and content credentials mark media as AI-made.