snlp-2
Large language models: generation, evidence, and tools
Overview
We've seen how neural networks learn representations and how attention connects positions in a sequence. Now we'll use language models to generate text, make predictions, retrieve evidence, and request tools.
In case you missed the news, Earth has recently gained access to the Galactic Archives of the Zarbonian Empire. We'll build an archival assistant to help us navigate this newly available information. Along the way, we'll inspect what supports its answers and equip it with tools (like a clock. Interstellar archival access does not include a wristwatch).
Yes, the setting is fictional and a bit ridiculous, but we can use it to explore how language models use information, where their answers come from, and what makes an answer worth trusting.
As you progress through this unit, keep the following question in mind: How could the model know that, and what supports its answer? Training can change model parameters. Instructions, examples, and reference material can change what it does in a particular context. Retrieval and tools can make additional information available. Our fictional setting can help remind us that none of these makes a fluent answer automatically reliable. We'll inspect the evidence at each step.
Outcomes
By the end of this unit, you should be able to…
- connect tokenization and next token prediction to autoregressive generation
- distinguish training objectives from the behavior they encourage
- design prompting comparisons using separate development and evaluation examples
- distinguish valid output structure from correct predictions
- trace evidence through retrieval and locally executed tools
- compare applications by inspecting their failures as well as their scores
Work through these in order
| Lesson | What we'll inspect |
|---|---|
| 0a. Model Gateway basics | Setup before the first API exercise. Use Class Maestro's current project and model ID. |
| 0b. Transformers and attention | Optional review of transformer architecture and attention. |
| 1. How LLMs See Text: Tokenization | Tokens, bytes, Unicode, and learned merge rules |
| 2. How models generate text | N-gram horizons, attention, decoding, and stopping |
| 3. Training, context, and evidence | Training objectives, in-context learning, prompting experiments, and evaluation |
| 4. Answering with evidence | Retrieve relevant information, produce structured answers, and check claims against the source material. |
| 5. Giving language models tools | Connect a model to functions in your code, control repeated calls, and evaluate whether the tools help. |