Karpathy’s LLM Wiki: How to Build a Mini Brain That Actually Remembers

Why RAG Never Learns

Most “chat with your documents” systems work the same way: you upload files, the model retrieves relevant chunks at query time, and generates an answer. This is useful but stateless. The model rediscovers knowledge from scratch on every question. It never accumulates understanding.

The result is a ceiling most teams hit quickly. Ask a question that requires synthesizing five documents, and the model has to find and piece together the fragments every time. Cross-references are not stored. Contradictions are not tracked. Synthesis is not preserved.

Andrej Karpathy’s LLM Wiki pattern changes this. The idea is to build a persistent, interlinked knowledge base that the LLM maintains over time. The knowledge compounds. Future reasoning becomes cheaper, deeper, and more contextually aware.

What Is an LLM Wiki?

An LLM Wiki is a structured collection of markdown files that sits between you and your raw sources. It is not an embedding index. It is an evolving knowledge substrate.

When you add a new source, the LLM does not just index it. It reads it, extracts the key information, and integrates it into the existing wiki. It updates entity pages, revises summaries, notes contradictions, and strengthens the evolving synthesis. The cross-references are already there. The contradictions have already been flagged.

The wiki is the persistent, compounding artifact. You read it. The LLM writes it.

The Three-Layer Architecture

Layer	What It Holds	Who Owns It
Raw sources	Articles, papers, transcripts, data files	You curate it
The wiki	Summaries, entity pages, concept pages, comparisons, synthesis	The LLM writes and maintains it
The schema	Conventions, workflows, and structure for the wiki	You co-evolve it with the LLM

Raw Sources

Keep your source files immutable. They are the source of truth. The LLM reads them but never edits them. This separation is what makes the wiki maintainable. If the LLM writes something wrong, you can always trace it back to the raw source.

The Wiki

The wiki is a directory of markdown files. Common page types include:

Entity pages — people, companies, products, characters
Concept pages — key ideas, frameworks, definitions
Summaries — condensed versions of sources
Comparisons — side-by-side analysis of related ideas
Synthesis — the evolving big-picture view
Open questions — gaps and contradictions to resolve
Index — a catalog of every page
Log — a chronological record of ingests and changes

The Schema

The schema is a CLAUDE.md or AGENTS.md file that tells the LLM how to maintain the wiki. It defines the directory structure, naming conventions, page formats, and workflows for ingest, query, and lint. A good schema turns a generic chatbot into a disciplined wiki maintainer.

How to Build and Maintain the Wiki

There are three core operations: ingest, query, and lint.

Ingest

Drop a new source into the raw collection and ask the LLM to process it. A typical ingest flow:

Read the source.
Discuss key takeaways with you.
Write a summary page.
Update the index.
Update relevant entity and concept pages.
Note contradictions with existing pages.
Append an entry to the log.

A single source can touch 10–15 wiki pages. The first time you do this, the LLM will guess a lot. After a few iterations, the schema captures the patterns and the process becomes routine.

Query

You ask questions against the wiki, not against the raw documents. The LLM reads relevant pages, follows cross-references, and synthesizes an answer with citations. The answer can take many forms: a markdown page, a comparison table, a slide deck, a chart, or a canvas.

The critical habit: file good answers back into the wiki. A comparison you asked for, a connection you discovered, an analysis you generated — these are valuable. Do not let them disappear into chat history. This is how exploration compounds just like ingested sources.

Lint

Periodically, ask the LLM to health-check the wiki. Look for:

Contradictions between pages
Stale claims superseded by newer sources
Orphan pages with no inbound links
Important concepts that lack their own page
Missing cross-references
Gaps that could be filled with a web search

The LLM is good at suggesting new questions to investigate. This is what keeps the wiki alive as it grows.

RAG vs. LLM Wiki

RAG	LLM Wiki
Retrieves raw chunks at query time	Maintains synthesized pages over time
Stateless — each query starts fresh	Persistent — knowledge compounds
Cross-references computed on demand	Cross-links already written and maintained
Contradictions handled per query	Contradictions tracked and resolved across the wiki
Good for ad-hoc questions	Good for deep, evolving knowledge domains
Cheap to set up	Requires schema discipline
Hits a ceiling on synthesis	Gets richer with every source

RAG is not obsolete. It is the right tool for quick lookups. The LLM Wiki is the right tool for building a second brain.

Why This Works Now

The hard part of maintaining a knowledge base has always been the bookkeeping. Updating links, keeping summaries current, noting contradictions, maintaining consistency across dozens of pages. Humans abandon wikis because the maintenance burden grows faster than the value.

LLMs do not get bored. They do not forget to update a cross-reference. They can touch 15 files in one pass. The cost of maintenance drops toward zero. Once maintenance is cheap, the architecture changes entirely.

The human’s job becomes curation, direction, and asking the right questions. The LLM’s job is the grunt work.

Practical Implementation

You do not need custom infrastructure. The simplest setup is a git repo of markdown files, an LLM agent, and a note-taking app like Obsidian.

Directory Structure

llm-wiki/
├── raw/
│   ├── articles/
│   ├── papers/
│   └── transcripts/
├── wiki/
│   ├── entities/
│   ├── concepts/
│   ├── summaries/
│   ├── comparisons/
│   ├── synthesis.md
│   ├── index.md
│   └── log.md
├── CLAUDE.md
└── .gitignore

Example Schema Snippet

# Wiki Schema

## Directory conventions
- Entity pages: `wiki/entities/<slug>.md`
- Concept pages: `wiki/concepts/<slug>.md`
- Source summaries: `wiki/summaries/<source-slug>.md`

## Ingest workflow
1. Read source from `raw/`.
2. Write a summary in `wiki/summaries/`.
3. Update or create relevant entity and concept pages.
4. Add cross-links using `[[page name]]`.
5. Update `wiki/index.md`.
6. Append to `wiki/log.md`.

## Writing style
- Use short paragraphs.
- Include a one-line TL;DR at the top of each page.
- Cite sources explicitly with `([source](raw/path))`.
- Flag contradictions with a `> ⚠️ Contradiction:` callout.

Tools That Help

Obsidian — markdown IDE with graph view, backlinks, and Dataview plugin
Obsidian Web Clipper — converts web articles to markdown
Marp — markdown-based slide decks
Dataview — query page frontmatter dynamically
qmd — local search engine for markdown with BM25 and vector search
Git — version history and collaboration

What I Learned About Knowledge Systems at Google

At Google, I saw the same problem at scale. Internal documentation, design docs, wikis, and code comments all decay. The knowledge is there, but it is scattered, stale, and disconnected. The bottleneck is not the amount of information. It is the maintenance.

The LLM Wiki pattern is the first approach that attacks the maintenance problem directly. It does not just organize information. It keeps the organization current. That is the difference between a static knowledge base and a compounding one.

Frequently Asked Questions

Is an LLM Wiki just a fancy RAG system?

No. RAG retrieves raw text at query time. An LLM Wiki stores synthesized, interlinked knowledge that the LLM maintains over time. The wiki answers from understanding, not from chunks.

Do I need Obsidian?

No. Any markdown editor or git repo works. Obsidian is useful because of its graph view, backlinks, and plugins, but the pattern is tool-agnostic.

How is this different from a regular wiki?

The LLM writes and maintains it. A human wiki rots because maintenance is expensive. An LLM wiki stays current because the LLM does the tedious work.

What kinds of projects is this best for?

Any domain where knowledge accumulates over time. Research, competitive analysis, business documentation, book notes, personal health tracking, course notes, or team knowledge bases.

Can I use this for a team?

Yes. The wiki is a git repo, so it supports collaboration and pull reviews. Humans can review the LLM’s changes just like they review code.

How do I prevent the LLM from making bad edits?

Keep sources immutable, maintain a schema, and review edits. The wiki is a proposal. You are the curator. Start with small ingests and build trust before handing over larger updates.

References & Further Reading

Karpathy on LLM Wiki — Andrej Karpathy’s writings on AI and knowledge systems
Obsidian — markdown-based knowledge base with graph view and plugins
qmd — local search engine for markdown files
Vannevar Bush’s Memex — the original vision of associative knowledge trails
Anthropic Claude Code — terminal-based agent for maintaining the wiki