Hal

An experiment in how far 24GB of consumer GPU actually goes. I pointed Claude Code at Qwen3-Coder — the best coding model that fits that VRAM budget — and used it for real work to find where it fell over. What it needed next was somewhere to keep what it learned, and I already had an Obsidian vault, so it grew into something that both organises the vault and reads from it.

The vault is the point. Dump anything into the inbox and Hal files it into a cross-linked, indexed knowledge base — which then becomes the thing it searches when you ask it something.

A 32K context window is small, and it’s shared between input and output. Everything you load as background is context the actual work doesn’t get, so the design question throughout is: what is the least context that makes this answer good?

That produced a memory system with three tiers rather than one — durable facts in a capped CLAUDE.md that archives itself past ~280 lines, session summaries rather than transcripts, and a resume command that loads three logs by default rather than everything.

And retrieval instead of stuffing: the vault is embedded, only the closest passages are retrieved, and the model is constrained to answer from those passages only, citing each as an Obsidian [[wikilink]] — or to say plainly that the notes don’t cover it. That constraint is what makes the answers trustworthy. Without it a small model blends your notes with its training data and you cannot tell which is which.

The single biggest quality improvement was a content decision, not a model one. Indexing everything made results worse: dashboards, index shells and empty stubs are short and mention every topic in the vault, so they score well against almost any query while containing nothing useful. They crowded real notes out of the top-k. The indexer now skips them, and that single change did more for answer quality than any model or embedding swap.

The embeddings and the index always run on your machine, in every profile — the vault is read, chunked and searched locally, and there is no hosted retrieval step. On the two GPU profiles nothing leaves the machine at all, because the model answering you is the one on your own GPU. On the no-GPU profile the answer is written by Claude Code, so the retrieved passages are sent along with the question: search stays local, the write-up does not. hal ask --no-llm returns the retrieved passages only.

Works: local answering with citations on both GPU profiles and CPU-only retrieval without one; the twelve slash commands, the memory tiers and the morning brief; and an MCP server usable from any MCP client.

Deliberately deferred: it’s built around Claude Code as the harness. That was the fastest route to something usable and it couples the project to a specific CLI — a purpose-built agent loop would give more control over exactly what enters the context window, and that’s the most likely direction for the next version.

Known rough edges: answers are only as good as your notes, and retrieval can’t rescue a vault full of empty stubs — garbage notes retrieve as garbage. Importing a large existing vault is slow even batched. A local 30B is not a frontier model: very capable at code, search and note-wrangling, and it will lose to a cloud model on hard multi-file reasoning. Linux only.