Overview
Status: in progress. Memory Lab is a developer-focused project exploring how persistent memory affects the performance of AI coding agents on software engineering tasks.
The central question is whether giving a coding agent access to relevant experience from previous tasks helps it perform better on future tasks.
The project combines two parts. A memory engine preserves and retrieves useful experience. An evaluation platform measures whether that experience actually improves the agent's performance.
The problem
Coding agents can solve individual tasks, but they may not consistently retain useful project knowledge and lessons from previous work. Each new task can start from scratch, repeating investigation that has already been done.
Memory is not a free fix. It introduces its own questions:
- What should be stored? Not every detail of a past task is worth keeping.
- How should it be represented? The same lesson can be stored as a note, a rule, an example or a summary.
- When should it be retrieved? Memory that arrives at the wrong moment is noise.
- Does it actually help? Retrieved memory can be irrelevant, out of date or misleading.
Research question
How does persistent memory affect the performance of coding agents on real software engineering tasks?
Approach
The plan is to compare three memory strategies against the same set of software engineering tasks:
- No persistent memory. The agent starts each task without access to previous experience.
- Static project instructions. The agent receives a fixed set of written project guidance.
- Dynamically retrieved memories. The agent receives memories selected for the current task from previous experience.
Using the same tasks for all three strategies keeps the comparison fair: the only thing that changes is how the agent gets its context.
Current implementation
What exists today:
- a Python command-line interface;
- SQLite-backed memory storage;
- memory categorisation and retrieval;
- context-building workflows that assemble memories for a task;
- memory assessment functionality;
- tests and reproducible development workflows.
Kinds of memory
The project explores different kinds of memory, borrowing terms from how memory is often described:
- Working memory: what the agent is holding for the current task.
- Episodic memory: records of specific past tasks and what happened.
- Semantic memory: general facts about a project or codebase.
- Procedural memory: how to do things, such as the steps to run or release a project.
- Preference memory: conventions and preferences to follow.
Evaluation questions
The evaluation platform is being built to answer questions such as:
- Does memory improve task performance?
- Does the retrieved memory actually help with the current task?
- Does memory reduce repeated investigation?
- When does memory become irrelevant or misleading?
- Which memory strategies provide the greatest benefit?
Results
The project is still in progress. Results will be added here once the first experiments are complete.
What's next
- Finish the evaluation platform so the three strategies can be run against the same tasks.
- Define how task performance and memory usefulness will be measured.
- Publish the method, and later the results, as research notes on this site.
The project does not have a public release or repository yet.