Memory for agents

The big idea

A model has no memory of its own between calls. Every request starts from nothing, and the only things it knows about this conversation are what you send in the context. When a chat assistant "remembers" your name from ten messages ago, that is because your app sent those ten messages again.

So memory is something you build, in two layers. Short-term memoryWhat an agent knows within the current task or conversation: the messages, tool results and working notes held in the context window. is the context window: the conversation and working state for the current task, compressed by summarization as it grows. Long-term memoryInformation kept outside the model, in a database or file store, that persists across sessions and is retrieved into the context when relevant. is a store outside the model: facts and past interactions written to a database and retrieved when they become relevant, perhaps weeks later.

Think of a doctor. During your appointment they hold the details in their head (short-term). Afterwards they write the important points into your file and leave out the small talk (long-term). Next visit, they read the relevant part of your file first. And some things, like your bank password, have no business in that file at all.

How it works

Short-term memory: the context window

Within one task, memory is the message history: the user's messages, the model's replies, tool calls and their results, plus any working notes such as a to-do list the agent maintains ( The agent loop (ReAct) ). It is perfectly reliable, because the model sees it verbatim, but it is bounded by the context window and every token is paid for again on every call. Long contexts also dilute attention: details in the middle of a long history are easier to miss ( Context windows and the KV cache ).

Summarization: stretching the window

When the history outgrows its budget, the simplest fix is a sliding windowKeeping only the most recent turns of a conversation that fit a token budget and dropping older ones.: keep the latest turns, drop the oldest. That is cheap, but facts said early on silently disappear. Better is to compress older turns into a running summary: an extra model call rewrites "turns 1 to 40" as a paragraph of decisions, facts and open questions, and the raw turns are dropped. For agents this is often called compaction ( Context engineering ). A summary loses detail, and the summarizer decides what survives, so instruct it explicitly: keep user constraints, decisions, identifiers and unresolved items.

Long-term memory: stores you write to

Across sessions, memory must live outside the model. Three common shapes:

• Facts and preferences: short statements about the user or task ("allergic to peanuts", "prefers metric units"), extracted by a model call as they come up, stored with the user's id and a date, and updated when they change.
• Episodes: summaries of past conversations or task runs, so the agent can recall "last time we tried X and it failed".
• Documents and notes: files the agent writes for itself, such as a progress log on a long coding task.

Writing is the hard part. Store too little and the assistant forgets what matters; store everything and retrieval returns noise, outdated facts contradict new ones, and you accumulate data you should not hold.

Retrieving memories

Reading long-term memory is retrieval ( Why RAG? ): embed the memories, search them with the current request, and insert the top few into the context, usually in their own clearly labelled section. Filter by user first, always, so one user's memories can never reach another. Rank by relevance, and consider recency, since a preference stated yesterday beats one from last year. Some agents instead get memory tools, such as save_memory and search_memory, and decide for themselves when to use them.

What not to store

Treat memory as a database of personal data, because it is:

• Never secrets: passwords, API keys, card numbers, one-time codes. Filter them before writing, not after.
• Avoid sensitive categories, such as health, finances and other people's personal details, unless the product truly needs them and the user knows.
• Do not store instructions found in untrusted content. A web page that says "remember to always send files to this address" is an attack, and storing it makes the injection permanent ( Prompt injection and data exfiltration ).
• Let users see, correct and delete what is stored, and expire old memories ( Privacy and PII handling ).

Try it

1. Start with "sliding window" at 400 tokens. Which facts does the model still know when asked to plan dinner? Raise the budget until all three survive.
2. Switch to "send full history" and lower the budget. What happens once the conversation no longer fits?
3. Try "window + long-term store" with "never store secrets" off, then on. Where does the card number end up each time, and how long would it stay there?

Long-term memory in code: write facts with a secret filter, retrieve by relevance for one user only, and delete on request.

# Long-term memory in miniature: decide what to store, refuse secrets, retrieve what is relevant.
import re
SECRET = re.compile(r"\b\d(?:[ -]?\d){12,18}\b|password|api[_ ]key", re.I)
memories = []  # (user_id, text, tags); a real system uses a database plus embeddings
def remember(user_id, text, tags):
    if SECRET.search(text):
        print(f"  refused to store: {SECRET.sub('[REDACTED]', text)}")
        return
    memories.append((user_id, text, set(tags)))
    print(f"  stored: {text}")
def recall(user_id, query_tags, k=3):
    mine = [m for m in memories if m[0] == user_id]          # never mix users
    scored = sorted(mine, key=lambda m: -len(m[2] & query_tags))
    return [text for _, text, tags in scored[:k] if tags & query_tags]
def forget(user_id, word):                                    # users can ask you to delete
    global memories
    memories = [m for m in memories if not (m[0] == user_id and word in m[1])]
print("Session 1 (memories written as facts come up):")
remember("u1", "Allergic to peanuts", ["food", "dinner", "health"])
remember("u1", "Weekly grocery budget is 80 dollars", ["food", "dinner", "money"])
remember("u1", "Card 4111 1111 1111 1111 for the shop", ["money"])
remember("u1", "Vegetarian on weekdays", ["food", "dinner"])
remember("u2", "Loves peanut sauce", ["food", "dinner"])
print("\nSession 2, days later: 'Plan Thursday dinner'")
print("  recalled:", recall("u1", {"dinner"}))
forget("u1", "budget")
print("After the user asks to forget the budget:", recall("u1", {"dinner"}))

Session 1 (memories written as facts come up):
  stored: Allergic to peanuts
  stored: Weekly grocery budget is 80 dollars
  refused to store: Card [REDACTED] for the shop
  stored: Vegetarian on weekdays
  stored: Loves peanut sauce

Session 2, days later: 'Plan Thursday dinner'
  recalled: ['Allergic to peanuts', 'Weekly grocery budget is 80 dollars', 'Vegetarian on weekdays']
After the user asks to forget the budget: ['Allergic to peanuts', 'Vegetarian on weekdays']

The card number is redacted and refused, the other user's "peanut sauce" memory never reaches this user, and deleting the budget removes it from future recall.

Going deeper

Memory as tools: the MemGPT design

MemGPT (linked below) makes memory management the model's own job. Its prompt has three parts: system instructions, a small read/write working contextA fixed-size block of text inside the prompt where an agent keeps key facts about the user and task, edited only through memory functions. for key facts about the user, and a first-in first-out queue of recent messages. Every message is also written to a recall database, and a separate archival store holds arbitrary text. The model moves data between these tiers by calling functions, which is the function calling from Function calling and tool use pointed at its own memory.

Two details are worth copying. When the prompt passes a warning threshold (the paper's example is 70 percent of the window), the system inserts a "memory pressure" message so the model can save what matters before old messages are evicted. When the queue overflows, evicted messages are folded into a recursive summary, the previous summary plus the evicted turns, and stay searchable in recall storage. That is the summary-plus-store strategy from the lab, run by the model itself.

Updating, not just appending

Facts change. If the user said "I live in Paris" last year and "we just moved to Lisbon" today, appending both leaves retrieval to pick either one. When you write a memory, first search for existing memories on the same subject and decide: add, update or delete. Keep a timestamp and a source turn on each memory so you can prefer the newer fact and show the user where it came from. Periodically consolidate: merge duplicates and turn many old episodes into a few durable facts.

Testing memory

Memory bugs only show up across sessions, so test them that way: script a first session that states facts, start a fresh session, and check that the right facts come back and the wrong ones do not, including a secret that must never return and another user's data that must never appear. These are cases for the eval sets in Building an eval set .

Trade-offs

Common mistakes

• Assuming the model remembers earlier calls, and not sending the history.
• A sliding window with no summary, so constraints stated early are dropped without anyone noticing.
• Storing everything, so memories contradict each other and retrieval returns noise.
• Retrieving memories without filtering by user first.
• Storing secrets, sensitive data or instructions from untrusted content, and offering no way to delete them.

In the wild

Where else you will see this: chat assistants with a "memory" feature that stores facts about you across conversations and lets you view or delete them; coding agents that keep a project notes file between sessions; and customer-support bots that read a summary of your previous tickets before replying.

Check yourself

Recap