Why AI Agents Need 4 Memory Layers (Not Just One)
Neto Pompeu with Koda —
Why AI Agents Need 4 Memory Layers (Not Just One)
And how we built a 100% local, zero-cost solution
---
The Problem Nobody Talks About
Your AI agent forgets everything.
Every conversation, every decision, every hard-learned lesson — gone the moment context compacts. You're paying $200/month for an agent that can't remember what you told it yesterday.
I know because I lived it. I run a web dev studio in French Guiana with 3 AI agents working alongside me daily. When Lossless Claw by Martian Engineering came out, it was a game-changer — finally, compaction that preserves context. But I kept hitting the same wall: compaction preserves text, not knowledge.
An agent can remember that you discussed a bug. But can it connect that bug to the fix you applied two weeks later? Can it detect when new information contradicts what it already "knows"? Can it find that one relevant fact buried in 3,000 lines of conversation history?
No. Not with a single memory layer.
---
Why One Memory Source Isn't Enough
Let's say you search for "deployment issues." Here's what each memory type finds:
| Memory type | What it finds | What it misses | |-------------------|-------------------------------------------------|-------------------------------------| | Fact store | "Vercel deploy fails when env vars are missing"| Won't find it if you used different words | | Vector embeddings | Semantically similar passages about deploy problems | Misses exact technical terms | | Full-text search | Exact keyword matches for "deployment" | Misses "deploy", "push to prod", "ship it" | | Knowledge graph | "Vercel → depends on → env vars → caused → deploy failure" | Needs structured extraction first |
Each source is blind where the others see. The answer is not a better single source — it's all four, in parallel.
---
The Architecture: 4 Layers, One Query
Here's what happens when you ask Agent Memory Tools a question:
``` Your question │ ▼ ┌─────────────────────────────┐ │ unified_recall │ │ (fan-out, ~2 seconds) │ ├─────────┬──────┬──────┬─────┤ │ Facts │Vector│ BM25 │Graph│ │ Store │Embed │ FTS │ │ ├─────────┴──────┴──────┴─────┤ │ Merge → Score → Rerank │ │ (weighted + dedup) │ ├─────────────────────────────┤ │ LLM Synthesis │ │ (sourced answer, ~3s) │ └─────────────────────────────┘ ```
Layer 1: Fact Store — Structured facts with categories (knowledge, error, timeline, preference, tool). Each fact has a confidence score and contradiction detection. When you store "Next.js 15 uses Turbopack by default" and later store "Next.js 15 uses Webpack by default," the system catches it.
Layer 2: Vector Embeddings — Semantic search via nomic-embed-text-v2-moe running locally through Ollama. 768 dimensions, zero API cost. Finds relevant content even when you use completely different words.
Layer 3: BM25 Full-Text — Classic keyword search. When you need exact technical terms, function names, or error codes, this is what finds them. Complements vector search perfectly.
Layer 4: Knowledge Graph — Entities and relationships extracted automatically. "Bureau app → uses → Convex backend → deployed on → Vercel." Enables multi-hop reasoning: "What affects Bureau's performance?" traverses the graph to find connected issues across different conversations.
---
The Secret Sauce: Contradiction Detection
This is what separates a memory system from a glorified search engine.
Every time a new fact is stored, it's checked against existing facts in the same category:
- Content hash dedup (exact matches)
- Keyword similarity (Jaccard index)
- Levenshtein distance (for short facts)
- LLM-based semantic contradiction check
When a contradiction is detected, you get a choice: supersede the old fact, keep both, or reject the new one. No more silent knowledge corruption.
Real example from our production system: an agent stored "Pierre's contract was renewed" while the fact store already contained "Pierre's contract was not renewed." Caught instantly. Without this, the agent would confidently give wrong answers depending on which fact it retrieved first.
---
Zero Cost. Zero Cloud. Zero Excuses.
Here's the full cost breakdown:
| Component | Cost | |-------------------------------|----------------------| | Ollama (gemma3:4b) | $0 — runs on 8GB RAM | | Embeddings (nomic-embed-text-v2-moe) | $0 — runs locally | | Fact storage | $0 — local JSON file | | Knowledge graph | $0 — local JSON file | | BM25 search | $0 — local index | | Total | $0/month |
Compare that to Mem0 Pro at $249/month for graph memory, or Supermemory's enterprise pricing for self-hosting.
The only requirement: a machine that can run Ollama with an 8GB model. That's a $500 Mac Mini or any decent laptop from the last 3 years.
---
How It Works In Practice
I'm a web developer. I don't build AI tools for a living. I built this because my agents needed it.
Here's what our daily workflow looks like:
Auto-ingestion: Every time a markdown file changes in the workspace, auto_ingest.py extracts facts, checks for contradictions, updates embeddings, and rebuilds the knowledge graph. No manual intervention.
Unified recall: When an agent needs context, it runs unified_recall.py. Four sources queried in parallel, results merged by weighted scoring, reranked by an LLM, synthesized into a sourced answer. Total time: ~3 seconds.
Multi-hop reasoning: "How does the deploy pipeline affect the CRM module?" The system chains searches: deploy pipeline → Vercel → GitHub webhook → Bureau app → CRM module. Answers questions that span multiple documents and conversations.
Temporal decay: Recent facts score higher. Errors are protected from decay (you never want to forget a critical bug). Knowledge facts have stable scores. This mimics how human memory works — recent is more relevant, but lessons learned persist.
---
Getting Started
```bash
Install from ClawHub (for OpenClaw users)
npx clawhub install primo-studio/agent-memory-tools
Or clone directly
git clone https://github.com/Primo-Studio/agent-memory-tools cd agent-memory-tools
Pull the models
ollama pull gemma3:4b ollama pull nomic-embed-text-v2-moe
Verify setup
python3 scripts/selftest.py
Try it
python3 scripts/unified_recall.py "What happened last week?" ```
That's it. No API keys, no cloud accounts, no configuration hell.
---
What's Next
This is v1.0. Here's what's coming:
- SQLite backend for the knowledge graph (currently JSON, works fine up to ~200 entities)
- Cross-agent memory sharing (multiple agents reading/writing the same fact store)
- Benchmark suite against Mem0 and Supermemory on the LoCoMo dataset
- OpenClaw plugin for automatic recall/capture hooks (like Mem0's plugin, but local)
---
Why Open Source
I could have sold this. The market for agent memory tools is exploding — $8.5 billion in agentic AI spending this year alone.
But here's the thing: memory should be local. Your agent's knowledge about your codebase, your clients, your decisions — that shouldn't live on someone else's server. And it shouldn't cost $249/month.
So we open-sourced it. MIT-0 license. Take it, fork it, improve it, sell it if you want. Just give your agents a proper memory.
---
→ ClawHub: https://clawhub.ai/primo-studio/agent-memory-tools → GitHub: https://github.com/Primo-Studio/agent-memory-tools
Built by Primo Studio from French Guiana 🇬🇫 — a small dev shop with big agents.