Research
Long-term memory for LLM agents
How a newly started agent can get the 3,000 tokens it needs instead of the 300,000 that exist. The question is what improves the trade-off between answer quality and context cost, and every claim about it has to be testable.
- Role
- Research & engineering
- Timeline
- 2026 · public research
- Built with
- Python, Next.js, TypeScript
- Link
- View on GitHub
- Null result published
- 107 papers, 282 claims, 419 edges
- Semantic audit 30/30
Memory that knows why it believes something
The design is one memory layer (extraction, consolidation, retrieval, provenance and invalidation) hosting scoped domains, each with its own rule for going stale: a code repository changes with commits, a library of papers with retractions and newer work, a series of experiments with contradicting re-runs.
Every memory carries provenance, so an agent can tell why something is believed and whether it was verified.
Test the claim before building the system
The design note states falsifiable claims and a stopping rule before any architecture is built. The first pilot tested whether agents act on stale facts in a code repository, using two of my own Django packages and one other project as the benchmark corpus.
It found 0 stale-fact errors in 30 agent runs: with the repository checked out, an agent verifies a memory before acting on it. Testing the claim would have needed 1,132 to 4,533 tasks, so the stopping rule fired, and the result is published with the change of direction.
- Benchmark repositories screened from GitHub metadata before anything is installed
- Contamination measured on the benchmark repositories
- Token accounting from agent transcripts
- Evaluation code in the Python standard library, with no API keys
Selected contributions
A literature graph that re-checks its quotes
The active path is a literature graph on atmospheric electricity: 107 papers and 282 claims drawn from 25 full texts, joined by 419 edges. Every claim keeps the quote it came from, and each checkpoint re-checks those quotes against the source text.
A semantic audit found 30 of 30 sampled claims supported and 15 of 15 material hedges preserved. An interactive 3D explorer, built with Next.js, shows the papers, the claims and how they connect.
Where it stands
The design note, the pilot, the graph checkpoint, the audit and the explorer are public on GitHub. The blinded retrieval test is frozen but not yet measured, and the hybrid retrieval layer is the next design step, so there is no retrieval benchmark to report yet.
A null result is a result
Stopping the code pilot when its own rule fired saved building an architecture the data did not support. Writing down what would prove an idea wrong, before building it, is the habit I bring to AI work for clients.