Agent Memory Explained

Three layers

1. Short-term (conversation)

The last N messages in context. Summarize older turns when the window fills.

Cost: tokens every request.
Implementation: message array + rolling summary system prompt.

2. Long-term (user memory)

Facts extracted from past sessions: "prefers bullet points", "uses Python", "company is Acme Corp".

Cost: extraction LLM call + storage + retrieval embedding.
Implementation: Mem0, custom Postgres JSONB, or vector store per user.

3. External knowledge (RAG)

Not memory — documents, tickets, codebase. Retrieved by query relevance.

Implementation: RAG Explained, vector DB.

When to add long-term memory

Add when users say:

• "Remember that I..."
• "Like last time..."
• "You already know my stack"

Skip when:

• Single-session tasks (classify this email)
• RAG on docs covers the knowledge need
• Privacy policy forbids storing inferred traits

Memory extraction pattern

After each session:
  → LLM extracts candidate facts (structured JSON)
  → Deduplicate against existing memory
  → Store with source session ID and timestamp

On new session:
  → Embed user query
  → Retrieve top-k memory facts
  → Inject into system prompt (labeled as user memory)

Show users what is stored. Let them delete.

Production concerns

Tools (2026)

Coming to directory: Mem0. For now see Top GitHub Repos.

Practical default

v1: 20-turn history + RAG on user docs
v2: Summarize sessions older than 1 hour
v3: Long-term memory for power users who opt in

Memory is a product feature with privacy cost — not an infrastructure requirement on launch day.