A paper gave chat agents an OS-style memory hierarchy

What happened

On 12 October 2023, researchers from UC Berkeley posted a paper titled MemGPT: Towards LLMs as Operating Systems. It proposes managing a language model's limited context window the way an operating system manages limited physical memory: keep the most relevant material in a small, fast "main context" the model reads directly, and move everything else to a larger, slower "external context" the model can retrieve on demand. The system decides what to move using function calls the model itself can issue, described in the paper as analogous to memory paging.

What the documents show

The paper reports that this design lets a model work with documents and conversation histories longer than its native context window, by paging relevant material in and out rather than holding everything at once, and it demonstrates the approach on document analysis and on multi-session conversations where a simulated character's memory persists and develops across sessions. The project's code repository, now maintained under the name Letta, describes itself as building "stateful agents: AI with advanced memory that can learn and improve over time" and states plainly that it was "f.k.a. MemGPT," confirming the research continued into an actively maintained open-source project rather than remaining a one-off academic demonstration.

The mechanism

The mechanism is retrieval, not an expanded context window. Nothing about this approach gives a model a larger working memory in the sense of more parameters holding more information simultaneously; the model's own context limit is unchanged. What changes is that a surrounding system decides, turn by turn, what to fetch from storage and insert into that limited window before the model generates a reply. This is the mechanistic reality behind most claims that a companion app "remembers" a user: a database or vector store holds past statements, a retrieval step selects which ones look relevant to the current message, and those selected fragments are inserted into the prompt. The model itself does not carry memory between sessions; the surrounding software does that work.

What it leaves open

The paper is a systems and evaluation contribution, not a specification any particular companion app is shown to follow; it does not describe how any named consumer product implements memory, and this record does not claim it does. Whether a given app's retrieval step selects accurately, how often it retrieves the wrong or outdated fact, and what a user can see or delete from that external store are product-specific questions the paper does not address.

• Is "remembering" implemented as retrieval from stored past messages, or as something else entirely?
• Can a user view, export or delete what the retrieval store holds about them?
• Does the app disclose how it decides which stored fragments to insert into a given conversation?

MemGPT is useful precisely because it names the plumbing. Once memory is understood as retrieval into a bounded window, a marketing claim that a companion "never forgets" becomes a testable statement about what a retrieval system stores and surfaces, not a claim about the model's own capacity.