Why Memory Matters
Every LLM has a context window — a fixed amount of text it can "see" at once. For short conversations, stuffing everything into context works. But real agents work on long tasks across many sessions and need to remember facts about users and past experiences. A thoughtful memory architecture is critical.
The Four Layers of Agent Memory
1. In-Context (Working) Memory
Everything currently in the prompt. Fast and immediately available, but limited by the context window and lost when the session ends.
2. Episodic Memory
A record of past interactions — raw messages, compressed summaries, or structured event logs. Enables continuity across sessions.
3. Semantic Memory (Vector Store)
Facts and knowledge stored as vector embeddings, retrieved via similarity search:
from langchain_openai import OpenAIEmbeddings
from langchain_community.vectorstores import Chroma
vs = Chroma(embedding_function=OpenAIEmbeddings())
vs.add_texts(["User prefers dark mode and Python over JavaScript"])
docs = vs.as_retriever(search_kwargs={"k": 5}).get_relevant_documents("user preferences")4. Procedural Memory
The agent's skills — system prompt, tool definitions, fine-tuned capabilities. Updated rarely through prompt engineering or fine-tuning.
Summarisation Strategy
from langchain.memory import ConversationSummaryBufferMemory
memory = ConversationSummaryBufferMemory(
llm=llm,
max_token_limit=2000, # Summarise when history exceeds limit
return_messages=True
)Good rule of thumb: keep the last 10 raw messages and a running summary of everything older. Balances recency with long-term continuity.
