An LLM-Compiled Wiki Instead of RAG, for One Person's Notes

The standard recipe for letting a model answer questions from your own documents is retrieval-augmented generation. Chunk the documents, embed the chunks, store the vectors, fetch the nearest ones at question time. It works, and past a certain size it is the right tool.

For one person's knowledge base I think it is the wrong first move. My vault answers questions from compiled markdown and a generated index, with no embeddings anywhere. I want to lay out why that is enough at my size and where I expect it to stop being enough.

The pattern

The idea is Andrej Karpathy's, from a post in April 2026. I know it mainly through Chase AI's breakdown. Raw sources land in a staging folder. An LLM compiles them into wiki articles organized by concept and keeps index files and short summaries current. To answer a question, the agent reads an index, chooses a few articles, and reads those.

By Chase AI's account, Karpathy expected to need RAG and found the model was good enough at maintaining its own indexes that he did not. Retrieval becomes a walk down a known path: master index, then one wiki's index, then an article.

Nate Herk draws the contrast in a way I find fair to both sides. RAG looks for chunks that sound similar to the question. A wiki follows links that somebody built on purpose. His comparison chart gives each a weakness: the wiki does not scale to millions of documents, and RAG loses the structure between them.

What mine looks like

55-raw/               captures and transcripts, left as they arrived
50-research/          one note per source, claims carry timestamps
10-cortex/            topic articles compiled from those notes
_retrieval-index.md   generated from note frontmatter

Three pieces do the work. A yt-research skill writes a note for each video, with a timestamp anchor on every claim, and will not finish until the anchors verify. A cortex-compile skill folds those notes into topic articles, one section per pass so a compile fits inside the context window. The retrieval index is generated from frontmatter.

That last piece matters most. Every note declares in its frontmatter what question it answers, which entities it covers and its key terms. The index is those fields gathered into one table. An agent reads the table, picks rows, and opens only those files.

I chose not to add a vector database. At the end of June the library was about 470 notes. None of this is clever. It is frontmatter and a table.

Why it holds up at this size

The clearest argument I have seen is in a conference talk I took notes on this week, by Paul Iusztin and Louis-Francois Bouchard, about their open-source research system. Theirs has the same shape as mine: immutable raw sources, an index, and LLM-written wiki pages. They keep the index in a single index.yaml where I generate mine from each note.

Their token argument is a waterfall. The agent reads the index summary first, then the source's wiki page, then derived pages, and opens the raw source only if it still lacks the answer. Summaries are written once at ingest, so most questions end early and the context window stays small. Their advice for a personal system is to "forget the infrastructure you think you need."

They do not call vector databases wrong. For retrieval over everything, at scale, inside a product, they still recommend RAG. Their objection is specific to personal use: it needs infrastructure, it is hard to inspect by hand, and everything else has to be built around it.

Inspection is the argument that convinces me. When an answer is wrong I can open the index row, the article and the source note, and see which of the three was wrong. With an embedding store, asking why a chunk came back is a harder question to answer by reading.

A caveat on the source: the talk ends with a course pitch, the demos were run ahead of time, and the daily-use claims are self-reported. The architecture is public and checkable.

Where it stops working

Scale is the obvious limit, and my sources are vague about it. Nate Herk puts the wiki's range at hundreds of pages. He and Chase AI both put RAG's at millions of documents. Nobody in my notes measured where the crossover falls, and the gap between those two numbers is four orders of magnitude. The figure my notes carry for Karpathy's own wiki is about 100 articles and 400,000 words, and I have not traced it back to his post. My library already holds several times that many notes, so the index, not the context window, is what has to carry it.

The index is the next thing to break. One index.yaml is comfortable when a wiki is scoped to a single project, which is how Iusztin and Bouchard use it. Whether it survives ten thousand sources is not shown. Generating mine from frontmatter means no one maintains it by hand, but it still gains a row per note. At some point reading the index becomes the expensive step, and I expect to need a second level of index or plain text search before I need embeddings.

The compile step is a trust problem. An LLM wrote the articles, so they are only as good as the gate in front of them. In Tonbi's AI Garage's build of this pattern, a question the wiki cannot answer triggers a web search and the result is filed straight back as permanent pages, with no verification. Errors compound along with everything else. Iusztin and Bouchard list their own open problems here: telling strong sources from stale ones, catching contradictions, and compacting memory.

Last, some questions are not lookups. Chase AI points to graph RAG for relationship queries across a very large corpus, and a product with many users is a different problem from one person's notes.

Key Takeaways

At personal scale, an LLM-compiled wiki plus an index answers questions without embeddings, a vector store or a chunking pipeline.
The index does the retrieval. Make each note declare what question it answers, then generate the index from that.
The crossover point to RAG is asserted, not measured, in every source I have. Watch index size as the early warning.
A compiled wiki needs a verification gate, or it accumulates confident errors.

Sources

The idea I want to try next is Iusztin and Bouchard's: let a question leave a new note behind, but send that note through the same compile gate as everything else.