AI Agent Memory in Python: Short-Term, Long-Term, and Persistent State (2026)

You shipped an agent that remembers the user's name for the length of one conversation. Then a user comes back the next day, says "as I mentioned yesterday, I'm on the enterprise plan", and your agent cheerfully asks what plan they're on. Nothing broke. Your agent simply has no memory that outlives the process.

This is the most common gap between an LLM demo and an LLM product. The model API is stateless by design — every request carries its own context, and anything you want remembered has to be put back in by you. This guide walks through four levels of agent memory, from a ten-line conversation buffer up to vector-backed long-term recall, with Python you can run today and the token math to keep it affordable.

1. Why Your Agent Forgets Everything

An LLM completion call has no session. You send messages in, you get text out, and the server keeps nothing for next time. Any sense of continuity your agent appears to have is an illusion you manufactured by re-sending prior messages with each request.

That has three consequences worth internalising:

• Memory costs tokens. Every remembered turn is re-billed on every call. A 40-turn conversation replayed verbatim is not free — it is a steadily growing tax on every response.
• Memory competes for the context window. Even with large windows, long contexts degrade. Models attend less reliably to material buried in the middle of a very long prompt, so dumping the whole history eventually makes the agent worse, not better.
• Memory lives in your process unless you persist it. A Python list dies on restart, on deploy, and on the second pod of your autoscaling group.

So "adding memory" is really three separate engineering decisions: what to keep, how long to keep it, and where to store it. Get those right and the rest is plumbing.

2. The Four Types of Agent Memory

Before writing code, name what you actually need. Most teams say "memory" and mean one of four very different things:

The mistake almost everyone makes is building episodic memory and calling it done. Conversation history solves "what did we just talk about" and nothing else. If a user tells your agent "I prefer Python over JavaScript for examples" and you only store messages, that preference is gone the moment the session ends no matter how good your history handling is.

3. Level 1: Conversation Buffer

The baseline. Keep a list, append each turn, send the whole thing every call. It is genuinely correct for short interactions and it is what you should ship first.

class ConversationBuffer:
    def __init__(self, system: str, max_turns: int = 20):
        self.messages = [{"role": "system", "content": system}]
        self.max_turns = max_turns

    def add(self, role: str, content: str) -> None:
        self.messages.append({"role": role, "content": content})
        self._trim()

    def _trim(self) -> None:
        # keep system + the last max_turns messages
        if len(self.messages) - 1 > self.max_turns:
            self.messages = [self.messages[0]] + self.messages[-(self.max_turns):]

    def as_prompt(self) -> list[dict]:
        return list(self.messages)

Two details that save pain later. Always keep the system message at index 0 when trimming — trimming it off silently changes your agent's behaviour mid-conversation. And trim by turns, not characters, so you never cut a message in half and produce a malformed alternating history.

This is where most internal tools should stop. If your agent handles a single short task, a buffer is simpler, faster, and easier to debug than anything below.

4. Level 2: Sliding Window and Summarization

Buffers break when conversations get long. Two fixes, and they solve different problems.

Sliding window keeps the last N turns and drops the rest. Cheap, predictable, and it forgets older context completely — fine for support triage, bad for a project assistant that references decisions from an hour ago.

Summarization compresses the dropped turns into a running summary that stays in context. More expensive (one extra LLM call per compaction) but it preserves the gist indefinitely:

SUMMARY_PROMPT = """Summarize the conversation so far for continuity.
Keep: user goals, decisions made, constraints stated, open questions.
Drop: greetings, repetition, tool output detail already resolved.
Return plain prose under 200 words."""

class SummarizingMemory:
    def __init__(self, client, model: str, keep_recent: int = 6):
        self.client = client
        self.model = model
        self.summary = ""
        self.recent: list[dict] = []
        self.keep_recent = keep_recent

    def maybe_compact(self) -> None:
        if len(self.recent) <= self.keep_recent * 2:
            return
        to_compact, self.recent = self.recent[:-self.keep_recent], self.recent[-self.keep_recent:]
        transcript = "\n".join(f"{m['role']}: {m['content']}" for m in to_compact)
        resp = self.client.chat.completions.create(
            model=self.model,
            messages=[
                {"role": "system", "content": SUMMARY_PROMPT},
                {"role": "user", "content": f"Existing summary:\n{self.summary}\n\nNew turns:\n{transcript}"},
            ],
        )
        self.summary = resp.choices[0].message.content

    def as_prompt(self, system: str) -> list[dict]:
        out = [{"role": "system", "content": system}]
        if self.summary:
            out.append({"role": "system", "content": f"Summary of earlier conversation:\n{self.summary}"})
        return out + self.recent

Note the compaction trigger is 2 × keep_recent, not keep_recent. Compacting on every boundary means an LLM call on every turn; waiting until you have twice the window halves the summarization cost with no practical loss.

Keep the summary in a system message rather than prepending it to the user turn. It reads as context the model should weigh but not answer, and it stops the model from treating the summary as a fresh instruction to reply to.

5. Level 3: Persistent Checkpoints

Everything so far lives in RAM. The moment you run two workers, deploy, or restart, your agent's memory is gone. Persistence means writing state somewhere durable keyed by a stable conversation ID.

If you are orchestrating with LangGraph, this is a one-line swap — the checkpointer handles it:

from langgraph.checkpoint.sqlite import SqliteSaver
from langgraph.graph import StateGraph

builder = StateGraph(AgentState)
# ... add nodes and edges ...

with SqliteSaver.from_conn_string("agent_state.db") as checkpointer:
    graph = builder.compile(checkpointer=checkpointer)

config = {"configurable": {"thread_id": "user-8412-conv-7"}}
graph.invoke({"messages": [("user", "continue where we left off")]}, config)

thread_id is the whole game. It is your conversation key, and the same ID on a different machine resumes the same state. MemorySaver is the in-process equivalent — useful for tests and notebooks, worthless in production because it evaporates on restart. PostgresSaver is the drop-in when you have more than one app server.

Without a framework, the same idea is about thirty lines of stdlib:

import sqlite3, json, datetime

def init_db(path="memory.db"):
    con = sqlite3.connect(path)
    con.execute("""CREATE TABLE IF NOT EXISTS threads (
        thread_id TEXT, seq INTEGER, role TEXT, content TEXT, ts TEXT,
        PRIMARY KEY (thread_id, seq))""")
    con.execute("""CREATE TABLE IF NOT EXISTS summaries (
        thread_id TEXT PRIMARY KEY, summary TEXT, updated TEXT)""")
    return con

def append(con, thread_id: str, role: str, content: str) -> None:
    con.execute(
        "INSERT INTO threads VALUES (?,?,?,?,?)",
        (thread_id, next_seq(con, thread_id), role, content, datetime.datetime.utcnow().isoformat()),
    )
    con.commit()

def load(con, thread_id: str, limit: int = 50) -> list[dict]:
    rows = con.execute(
        "SELECT role, content FROM threads WHERE thread_id=? ORDER BY seq DESC LIMIT ?",
        (thread_id, limit),
    ).fetchall()
    return [{"role": r[0], "content": r[1]} for r in reversed(rows)]

Write through on every turn, not on session end — a crashed worker should not cost the user their history. And scope every query by thread_id: the most common data leak in agent apps is a missing WHERE clause that lets one user's thread read another's.

6. Level 4: Semantic Long-Term Memory

Transcripts answer "what did we say". They do not answer "what do we know about this user". For that you need to extract durable facts and store them independently of any conversation.

The pattern is a background extraction step: after a turn (or every few turns), ask a cheap model to pull out facts worth keeping, then save them with an embedding for later retrieval.

EXTRACT_PROMPT = """Extract durable facts about the user from this exchange.
Include: preferences, role, stack, plan/tier, constraints, project names.
Exclude: anything specific to this one task, greetings, transient state.
Return JSON: {"facts": ["..."]} — empty list if nothing durable."""

def extract_facts(client, turns: list[dict]) -> list[str]:
    resp = client.chat.completions.create(
        model="gpt-4o-mini",
        response_format={"type": "json_object"},
        messages=[
            {"role": "system", "content": EXTRACT_PROMPT},
            {"role": "user", "content": json.dumps(turns[-6:])},
        ],
    )
    return json.loads(resp.choices[0].message.content)["facts"]

Use a small cheap model here, not your main one. Extraction is a classification task and gpt-4o-mini-class models handle it fine at a fraction of the cost. Run it asynchronously after responding — the user should never wait on memory writes.

Store facts with a per-user namespace and a timestamp. Facts go stale, and "the user is on the free plan" from eight months ago is worse than no fact at all. Record when you learned it and let old entries decay or get re-verified.

7. Building a Memory Manager

Here is a compact manager that combines all four levels. It is about ninety lines and covers most real applications:

class AgentMemory:
    """Episodic (recent turns) + semantic (persistent facts) memory."""

    def __init__(self, con, thread_id: str, user_id: str, embed_fn, keep_recent: int = 8):
        self.con, self.thread_id, self.user_id = con, thread_id, user_id
        self.embed = embed_fn
        self.keep_recent = keep_recent

    def record(self, role: str, content: str) -> None:
        append(self.con, self.thread_id, role, content)

    def learn(self, fact: str) -> None:
        self.con.execute(
            """INSERT INTO facts (user_id, fact, embedding, learned)
               VALUES (?,?,?,?)
               ON CONFLICT(user_id, fact) DO UPDATE SET learned=excluded.learned""",
            (self.user_id, fact, json.dumps(self.embed(fact)),
             datetime.datetime.utcnow().isoformat()),
        )
        self.con.commit()

    def recall_facts(self, query: str, k: int = 5) -> list[str]:
        q = self.embed(query)
        rows = self.con.execute(
            "SELECT fact, embedding FROM facts WHERE user_id=?", (self.user_id,)
        ).fetchall()
        scored = sorted(
            ((cosine(q, json.loads(r[1])), r[0]) for r in rows), reverse=True
        )
        return [f for score, f in scored[:k] if score > 0.35]   # relevance floor

    def build_prompt(self, system: str, user_input: str) -> list[dict]:
        facts = self.recall_facts(user_input)
        history = load(self.con, self.thread_id, limit=self.keep_recent)
        blocks = [{"role": "system", "content": system}]
        if facts:
            blocks.append({
                "role": "system",
                "content": "Known about this user:\n" + "\n".join(f"- {f}" for f in facts),
            })
        return blocks + history + [{"role": "user", "content": user_input}]

The relevance floor in recall_facts matters more than it looks. Without it you always inject the top 5 facts even when nothing is relevant, which drags in noise and quietly steers the model. A cosine floor of roughly 0.3–0.4 (tune against your embedder) means "no relevant memory" is a valid outcome the code handles gracefully.

Embed the query, not the full conversation, when recalling. Retrieving on the user's latest message finds facts relevant to what they are asking right now.

8. Retrieving the Right Memory

Having stored memory is not the hard part; pulling the right slice at the right time is. Three techniques worth the effort:

• Recency-weighted retrieval. Blend similarity with age — score = 0.7 * cosine + 0.3 * recency. A fact learned today usually beats a marginally better match from last year.
• Hybrid search. Pure vector search misses exact tokens: product names, error codes, ticket IDs. Combine embedding similarity with BM25 keyword matching, or just do a substring pre-filter for known identifiers.
• Progressive disclosure. Do not inject everything you know. Put a one-line index of available memory in the prompt and let the agent request detail through a tool. This is what the better coding assistants do, and it keeps the context clean while still making deep recall possible.

Progressive disclosure is the highest-leverage of the three once you pass a few hundred stored facts, because it converts a fixed per-request cost into a variable one you only pay when the memory is actually needed.

9. Approaches Compared

They compose rather than compete. A realistic production agent uses a checkpoint for durability, a sliding window over recent turns, and a vector store for durable facts. Start with the row you actually need today and add the next one when you feel the pain.

10. Budgeting Tokens and Cost

Memory is a line item. Rough math for a support agent on a mid-tier model at about $0.15 per million input tokens:

• Verbose buffer, 40 turns — ~8,000 tokens of history per call. At 500 calls/day that is 4M input tokens daily, roughly $180/month spent almost entirely on replaying old text.
• Sliding window, 8 turns — ~1,600 tokens per call. Same traffic: ~0.8M tokens, about $36/month.
• Window + 5 recalled facts — ~1,750 tokens per call, plus one summarization call per compaction. Lands around $45/month with far better continuity than the bare window.

The point is not the exact numbers, it is that going from naive to managed memory is routinely a 5x cost reduction and a quality improvement. Replaying a long transcript is the most expensive way to remember something.

If you are already caching, note the interaction: prompt caching only pays off when the prefix is stable. Appending to a growing history changes the prefix every turn and defeats it. Put your static content (system prompt, recalled facts) at the front and volatile content (latest user message) at the end so the cacheable prefix stays stable — our prompt caching guide covers the mechanics.

11. Common Mistakes

• Storing everything as one transcript. Conversation history is not a knowledge base. Facts buried in a transcript are expensive to retrieve and impossible to update — extract them.
• Using MemorySaver in production. It is in-process RAM. Your memory feature disappears on every deploy and every restart. Use SqliteSaver at minimum, PostgresSaver with more than one server.
• No thread scoping. Forget the WHERE thread_id = ? and users read each other's conversations. Treat thread_id as a tenant key and enforce it in the data layer, not in the prompt.
• Injecting irrelevant facts. Always pulling top-k with no relevance floor guarantees noise on every call. Return nothing when nothing matches.
• Writing memory synchronously. Fact extraction and embedding add latency the user pays for. Fire it off after the response is sent.
• Never expiring anything. Stale facts are worse than missing ones because they are confidently wrong. Timestamp everything and re-verify or decay old entries.
• Letting the model manage its own memory. Asking "what should I remember?" in the prompt works until it doesn't, and you cannot audit it. Do extraction in deterministic code with a schema and keep the full transcript as your source of truth so you can rebuild when the extraction logic improves.
• No tests for recall. Memory regressions are silent — the agent just gets vaguer. Add a pytest case that teaches a fact, starts a fresh thread, and asserts the fact comes back.

Bottom Line

Memory is not a feature you bolt on, it is a storage decision you make early. Pick what you need to remember, decide how long it stays true, and put it somewhere that outlives the process. Everything after that is choosing the cheapest mechanism that retrieves it reliably.

If you have an afternoon, work in this order. First, give every conversation a stable ID and persist turns to SQLite — that alone converts your agent from a demo into something users can return to. Second, cap the replay window at about eight turns and watch both cost and answer quality improve at the same time. Third, add fact extraction with a cheap model writing to a namespaced table, retrieved with a relevance floor so irrelevant memory stays out of the prompt. Fourth, write the pytest case that teaches a fact and reads it back in a fresh thread, because memory bugs never announce themselves.

The failure mode to avoid is the transcript trap: faithfully storing every message forever, then paying to replay all of it on every call while your agent still forgets the user's name next week. Store facts, not just words.