What this teaches
Most agents remember documents in one of two ways: they carry the chat history, or they search raw chunks of the documents again for every question. Wiki memory is a third way. Andrej Karpathy described it in April 2026 as the “LLM Wiki”: the agent keeps a small set of markdown pages, one per project, person, system or policy, and updates them each time a new document arrives. Questions are answered from those pages, which already reflect everything read so far.
This example teaches the idea with one question and five short, synthetic documents about a fictional project, Ledger Export.
The problem, on a real example
The question is: “How long are customer exports kept?”
Plain retrieval (BM25 over paragraph chunks of the five documents) returns three passages:
- Help-centre FAQ draft, 5 May 2026: “Your export is kept for 7 days.”
- Kickoff notes, 2 March 2026: “Finished export files are kept for 30 days, then deleted.”
- Chat thread, 20 April 2026: “From 1 May, export files for Enterprise plans are kept for 90 days. All other plans stay at 30 days.”
Three numbers, from three documents written at different times, and nothing in the passages says which one is current. Retrieval finds text; it doesn’t know which text is still true.
What wiki memory is
Each time a new document arrives, the agent reads it once and updates its pages: it writes down what is new, rewrites what changed, and crosses out what is no longer true, with the document that replaced it. When a question comes in, it reads the one or two pages that cover it. The work moves from question time to the moment a document arrives, and it is done once.
Asked the same question after five documents, the wiki answers from one page:
Since 1 May 2026, export files are kept for 90 days on Enterprise plans and 30 days on all other plans. The emailed download link expires after 7 days; customers can request a new link while the file exists. The draft help-centre FAQ says 7 days, which is being corrected.
How the walkthrough works
The app plays the five documents in order: kickoff notes (2 March), design doc (16 March), incident report INC-482 (7 April), a chat thread (20 April) and a draft FAQ (5 May). For each one you see the document on the left and the wiki page it changes on the right. Each passage that drives an edit is highlighted with a number, and the edit it caused carries the same number. The passage lights up first, then the edit applies as tracked changes: added text in green, old values crossed out with the reason.
After the five documents, a check (lint) reads every page at once and finds two contradictions and one link to a page that doesn’t exist. No single update could have seen them, because each one only opened the pages its document was about. After a person approves the fixes, lint finds 0 problems. The finished wiki has 15 pages and 51 links.
It plays by itself, with a speed control (0.5× to 1.5×), and you can pause or step through at any time.
How conflicts are resolved
Real documents disagree all the time, so the agent writes each fact down with its scope, source and date, and plain code applies one rule:
- Scope first. A fact for all plans applies to each plan, so “90 days” and “30 days” are two scopes, not a conflict.
- Then source rank: approved decision > design doc > incident report or runbook > meeting notes or chat > draft.
- Then date: within the same rank, the newer document wins.
- Same rank, same date, different values: disputed, and a person decides.
What the resolver (src/resolve.ts) returns on the example’s facts:
- Retention, Enterprise: 90 days, from Amara’s decision on 20 April. The design doc’s and the kickoff’s 30 days and the draft’s 7 days all rank lower.
- Retention, other plans: 30 days, from the same decision. The design doc and kickoff agree and are kept as supporting sources; the draft’s 7 days ranks lower.
- Longest date range in one export: disputed, 24 or 12 months. Two runbooks, both dated 12 May, disagree. The FAQ draft’s 24 months ranks lower and can’t settle it, so the question goes to a person, and any answer about it states both values.
The ranking is a design choice, not a law. Your organisation’s order will differ.
Who does what
- The model reads each document, decides which pages it affects, writes and rewrites page text, crosses out old facts with the reason, spots contradictions that need understanding, and writes answers from the pages.
- Code stores the files with Git history, rebuilds the index and log, checks links, compares fact values, applies the ranking rule, and checks that every quoted sentence exists verbatim in its source.
- A person approves sensitive edits and decides disputed facts.
Taking it to production, briefly
The demo has five short documents; real systems have thousands of large ones that change and conflict. The app and the article cover eight areas as design guidance (the demo does not test them): big documents (route to the affected pages first, process section by section, split large pages), a wiki too big to read (hub pages and search over the wiki), conflicting facts (structured facts, the ranking rule, a review queue), mistakes and drift (verified citations, raw documents as the source of truth, regression questions), many documents at once (a queue, locks, one Git commit per ingest), cost (cheap model to route, strong model to edit, reprocess only changed documents), deleted documents and permissions (provenance, inherited access rules), and combining with RAG and knowledge graphs.
Run it
Node.js 22.12 or later. No API keys.
git clone https://github.com/AIEngineerHelp/applied-ai-engineering.git
cd applied-ai-engineering/agents/wiki-memory
npm ci
npm start # http://127.0.0.1:4331
npm test # 22 tests
npm run check # TypeScript type check
Code: applied-ai-engineering/agents/wiki-memory at main · AIEngineerHelp/applied-ai-engineering · GitHub
Live demo: https://wiki-memory.vercel.app
Limitations: this is a scripted teaching demo with no LLM running. The wiki pages, the plain-language notes and the extracted facts were written by hand to show what a careful model should produce; everything derived from them (tracked changes, links, index and log, lint, the retrieval comparison and the conflict resolution) is computed by tested code. Five documents is not a benchmark.