QMD

Query Markup Documents

On-device search for everything
you need to remember

By Tobi Lutke — github.com/tobi/qmd

16k+ stars · TypeScript · MIT License · Created Dec 2025

Part I

ELI5

The simple version anyone can understand

Imagine a super-smart librarian
who lives inside your computer

You have hundreds of notes, meeting transcripts, docs scattered everywhere.

QMD is like hiring a librarian who:

  • Reads every single document you give it
  • Remembers what each one is about, not just the words
  • Finds answers instantly when you ask questions
  • Never phones home — works 100% offline

Three ways to find your stuff

1. Keyword Search

Like Ctrl+F on steroids. Finds exact words.

"Find every doc that says budget Q4"

2. Meaning Search

Understands what you mean, not just what you type.

"Find docs about spending plans for year-end"

3. Hybrid (the magic one)

Combines both + an AI judge picks the best results.

# Keywords
$ qmd search "budget Q4"

# Meaning
$ qmd vsearch "spending plans"

# Both + AI reranking
$ qmd query "what did we spend?"

Why does this matter?

The problem

  • Your notes and docs are scattered across folders
  • Ctrl+F only finds exact words — misses synonyms and meaning
  • Cloud search (Google, Notion) sends your data to someone else's servers
  • Setting up local AI search normally requires a PhD

QMD's answer

  • One command: npm install -g @tobilu/qmd
  • Point it at folders, it indexes everything
  • Uses small AI models (2GB total) that run on your laptop
  • Plugs directly into Claude, Cursor, or any MCP-capable AI

The big idea: State-of-the-art hybrid search that used to require Elasticsearch + OpenAI + a server farm, now runs on your MacBook with zero cloud dependencies.

How AI agents use QMD

QMD speaks the Model Context Protocol (MCP) — the universal language for AI tools.

You ask Claude: "What were the action items from last Tuesday?"
↓
Claude calls QMD's MCP server with a hybrid query
↓
QMD searches your local notes — no internet needed
↓
Claude reads the results and gives you a real answer

Your data never leaves your machine. The AI just gets the relevant snippets.

Part II

Full Technical Deep Dive

Architecture, algorithms, and implementation details

Architecture Overview

QMD is a TypeScript monolith (~260KB of source) built around a single SQLite database with three access patterns:

LayerTechnology
CLINode.js parseArgs + tsx
MCP Serverstdio + HTTP/SSE transports
SDKcreateStore() library API
StorageSQLite (better-sqlite3 / bun:sqlite)
Full-textFTS5 with Porter stemming
Vectorssqlite-vec (cosine distance)
LLM Runtimenode-llama-cpp (GGUF models)
ConfigYAML synced to SQLite

Source layout

src/
  store.ts    154KB  core engine
  cli/
    qmd.ts    115KB  CLI commands
    formatter  13KB  output formats
  llm.ts       50KB  model layer
  collections  13KB  YAML config
  mcp/
    server.ts  30KB  MCP protocol
  db.ts         3KB  SQLite compat
  index.ts     17KB  SDK exports
finetune/     query expansion model

SQLite Schema — The Single Database

Everything lives in ~/.cache/qmd/index.sqlite:

-- Content-addressable storage
CREATE TABLE content (
  hash TEXT PRIMARY KEY,
  doc  TEXT NOT NULL,
  created_at TEXT
);

-- Document index (soft-delete)
CREATE TABLE documents (
  collection TEXT,
  path       TEXT,
  title      TEXT,
  hash       TEXT REFERENCES content,
  active     INTEGER DEFAULT 1,
  UNIQUE(collection, path)
);

-- Full-text search
CREATE VIRTUAL TABLE documents_fts
  USING fts5(
    filepath, title, body,
    tokenize='porter unicode61'
  );
-- Vector embeddings (sqlite-vec)
CREATE VIRTUAL TABLE vectors_vec
  USING vec0(
    hash_seq   TEXT PRIMARY KEY,
    embedding  float[N]
      distance_metric=cosine
  );

-- Chunk metadata
CREATE TABLE content_vectors (
  hash TEXT, seq INTEGER,
  pos  INTEGER, model TEXT,
  PRIMARY KEY(hash, seq)
);

-- LLM response cache
CREATE TABLE llm_cache (
  hash TEXT PRIMARY KEY,
  result TEXT,
  created_at TEXT
);

Key insight: Content-addressable storage means identical documents across collections are stored once. The hash (SHA256) is the universal join key.

Cross-Runtime SQLite — db.ts

A compatibility layer that works under both Bun and Node.js:

const isBun = typeof globalThis.Bun !== "undefined";

if (isBun) {
  // Dynamic import prevents tsc errors
  const mod = "bun:" + "sqlite";
  _Database = (await import(mod)).Database;

  // macOS: swap in Homebrew SQLite
  // (Apple's omits LOAD_EXTENSION)
  if (process.platform === "darwin") {
    BunDatabase.setCustomSQLite(
      "/opt/homebrew/opt/sqlite/lib/..."
    );
  }
} else {
  _Database = (await import(
    "better-sqlite3"
  )).default;
}

Why this matters

  • Apple's system SQLite disables LOAD_EXTENSION
  • sqlite-vec is a native extension — needs full SQLite
  • Bun has built-in SQLite but needs the Homebrew swap on macOS
  • Node uses better-sqlite3 with compatible API
  • If sqlite-vec fails to load, vector search is disabled but BM25 still works

Graceful degradation: QMD works without vectors. BM25 search is always available.

Three Local GGUF Models

All models run via node-llama-cpp and auto-download from HuggingFace on first use:

Model Purpose Size Base
EmbeddingGemma Vector embeddings — converts text to float arrays for semantic similarity ~300MB (Q8) 300M params
Qwen3-Reranker Re-ranks candidate results by relevance. Scores doc-query pairs. ~640MB (Q8) 600M params
qmd-query-expansion Expands user query into keyword + semantic + hypothetical doc variants ~1.1GB (Q4_K_M) Qwen3-1.7B (fine-tuned)

~2GB total RAM for all three. Models cache locally and use inactivity timeouts (5 min) to free VRAM when idle. Promise guards prevent duplicate loads.

The Hybrid Query Pipeline

What happens when you run qmd query "what did we decide about pricing?"

Step 1: Query Expansion (LLM)
↓
BM25 / FTS5
lex: variants
+
Vector / sqlite-vec
vec: + hyde: variants
↓
Step 3: Reciprocal Rank Fusion (RRF)
↓
Step 4: LLM Re-ranking (top 40 candidates)
↓
Step 5: Score Blending & Final Ranking

Step 1: Query Expansion

A fine-tuned 1.7B model transforms your query into typed retrieval variants:

Input: "what did we decide about pricing?"

Output (3 line types):
lex: pricing decision meeting notes
lex: pricing strategy final
vec: discussions about product pricing
       and strategy decisions
vec: revenue model and price changes
hyde: After reviewing the market data,
  the team decided to move forward
  with the tiered pricing approach
  starting Q2...

Three query types

  • lex: Keyword variants for BM25 — short, no filler words
  • vec: Natural language phrases for vector search
  • hyde: Hypothetical Document Embedding — a ~100-word synthetic answer that the real answer should be near in vector space

Strong signal shortcut: If BM25 alone returns a result with score ≥ 0.85 and a gap ≥ 0.15 from second place, expansion is skipped entirely.

Fine-tuning the Query Expansion Model

The repo includes the full fine-tuning pipeline in finetune/:

Training setup

Base modelQwen3-1.7B
MethodLoRA (rank 16, alpha 32)
TargetsAll q/k/v/o/gate/up/down projections
Training data~2,290 examples
Epochs5
Batch size16 (4 x 4 grad accum)
LR scheduleCosine from 2e-4
HardwareA10G (24GB VRAM)

Results

Train accuracy: 97.4%

Eval accuracy: 93.8%

Scoring dimensions (max 140)

  • Format compliance (0–30)
  • Diversity of expansions (0–30)
  • HyDE quality (0–20)
  • Content quality (0–20)
  • Entity preservation (–45 to +20)
  • Think mode bonus (0–20)

BM25 Full-Text Search via FTS5

Query parsing

// User syntax -> FTS5 query
buildFTS5Query("budget Q4 -sports")
// Returns: "budget"* "Q4"* NOT "sports"*

buildFTS5Query('"exact phrase" docs')
// Returns: "exact phrase" "docs"*

// Features:
// - Prefix matching with * suffix
// - Quoted phrase = exact match
// - Negation via NOT binary op
// - Porter stemming + Unicode

Score normalization

// BM25 returns negative values
// Normalize: |bm25| / (1 + |bm25|)
score = -10 → 0.91  // strong
score =  -2 → 0.67  // medium
score = -0.5 → 0.33  // weak

FTS5 configuration

  • Three indexed columns: filepath, title, body
  • Filepath weighted 10x vs body 1x
  • Porter stemming catches inflections (running/ran/runs)
  • Unicode61 tokenizer for international text

Why BM25? It is the gold standard for keyword retrieval. SQLite FTS5's built-in BM25 is fast, battle-tested, and requires zero infrastructure beyond the database file.

Vector Search via sqlite-vec

Two-step query (avoids sqlite-vec JOIN hang)

-- Step 1: KNN from vector table
SELECT hash_seq, distance
FROM vectors_vec
WHERE embedding MATCH ?
  AND k = ?

-- Step 2: Join back to documents
SELECT d.path, d.title, cv.pos
FROM content_vectors cv
JOIN documents d ON d.hash = cv.hash
WHERE cv.hash || '_' || cv.seq
  IN (...)
  AND d.active = 1

Score = 1 − cosine_distance. Deduplicated by filepath (best chunk wins).

Embedding model

  • EmbeddingGemma (300M params, Q8 quantized)
  • Cosine distance metric in sqlite-vec
  • Dynamic dimension detection on first embed
  • Supports custom models for multilingual (CJK, etc.)

Embedding format

  • Queries: task-prefixed (search_query: ...)
  • Documents: search_document: title: ... text: ...
  • Qwen3 models use different prefix format

Known quirk: sqlite-vec JOINs hang indefinitely. The two-step pattern is intentional.

Reciprocal Rank Fusion (RRF)

Merging results from BM25 and vector search into a single ranked list:

The algorithm

// RRF formula per document:
RRF(d) = Σ for each query q:
  weight(q) / (k + rank(d, q))

// where k = 60 (standard constant)
// weight(q) depends on query type:
//   original query = 2x weight
//   expansion variants = 1x weight

Position-aware blending

// Top ranks: trust retrieval more
rank 1-5:  75% retrieval + 25% reranker
rank 6-15: 65% retrieval + 35% reranker
rank 16+:  40% retrieval + 60% reranker

Why RRF?

  • Rank-based, not score-based — no need to normalize across different scoring systems
  • Original query gets double weight so user intent stays dominant
  • Top-ranking docs get bonus points for appearing in multiple retrievers
  • Handles the incommensurability problem: BM25 scores and cosine distances are on totally different scales

Smart confidence: Top results likely matched keywords exactly, so retrieval signals are trusted more (75%). Lower results may benefit from the LLM reranker's semantic understanding (60%).

LLM Re-ranking with Qwen3-Reranker

How it works

  • Top 40 candidates from RRF are sent to the reranker
  • Qwen3-Reranker (600M, Q8) scores each query-document pair
  • Outputs a relevance probability per candidate
  • Documents are deduplicated before reranking
  • Parallel evaluation splits work across CPU/GPU cores

Score interpretation

ScoreMeaning
0.8 – 1.0Highly relevant
0.5 – 0.8Moderately relevant
0.2 – 0.5Somewhat relevant
0.0 – 0.2Low relevance

Final blend formula

// Position-dependent blending
if (rank <= 5) {
  final = 0.75 * retrieval
        + 0.25 * reranker;
} else if (rank <= 15) {
  final = 0.65 * retrieval
        + 0.35 * reranker;
} else {
  final = 0.40 * retrieval
        + 0.60 * reranker;
}

// Intuition:
// Top hits: retrieval was confident
//   -> trust the original ranking
// Lower hits: less confident
//   -> let the reranker decide

Use --explain flag to see the full score trace.

Smart Document Chunking

Documents are split at ~900 tokens per chunk with 15% overlap. The algorithm prioritizes natural semantic boundaries.

Break point scores

PatternScore
# H1 heading100
## H2 heading90
Code block boundary ```80
Horizontal rule ---60
Blank line (paragraph)20
List item - item5
Any newline1

Decay function

// Prefer splits near the target size
// but accept earlier good boundaries

normalizedDist = distance / window;
multiplier = 1.0 - (dist**2) * 0.7;
finalScore = bp.score * multiplier;

// At target:  1.00x
// 25% back:   0.96x (gentle)
// 50% back:   0.83x (moderate)
// 75% back:   0.61x (steep)
// Edge:       0.30x

Code fence protection: Never splits inside a ```code block```. Fences are detected and excluded from candidate break points.

Embedding Generation Pipeline

Batch processing flow

Identify docs needing embeddings
↓
Build batches (64 docs / 64MB max)
↓
Chunk each doc (~900 tokens)
↓
Embed in sub-batches of 32
↓
Store vectors in sqlite-vec

Implementation details

  • Each chunk gets formatDocForEmbedding(text, title)
  • Sub-batches of 32 chunks processed via session.embedBatch()
  • Context truncation prevents GGML crashes from oversized inputs
  • Parallel contexts split across CPU cores / GPU
  • Progress callback reports chunks, bytes, and errors in real time

Document IDs

Each doc gets a 6-char hash ID (#abc123) from the first 6 characters of its SHA256 content hash. Used throughout for retrieval: qmd get #abc123

Dimension auto-detection: the vectors_vec table is created on first embed with the model's actual output dimensions.

LLM Resource Management

Running three models locally requires careful memory management:

Session lifecycle

// Scoped session pattern
await withLLMSession(llm, async (session) => {
  // session has abort signal
  // and duration limits
  const embeddings = await
    session.embedBatch(texts);
});
// resources freed on exit

// Reference counting tracks active
// sessions per model
class LLMSessionManager {
  refs: Map<model, count>
  canUnload(): boolean
}

Key patterns

  • Inactivity timeout: 5-min idle unloads contexts (keeps models warm in memory)
  • Promise guards: Prevent concurrent model loads that duplicate VRAM
  • No greedy decoding: Prevents generation loops
  • Context size bounds: Truncation before inference prevents crashes
  • ETag caching: Model downloads check freshness via HTTP ETag, skip re-download
  • Deterministic disposal: Cleanup with timeouts, not just GC

MCP Server — AI Agent Integration

QMD exposes 4 tools via the Model Context Protocol, with both stdio and HTTP transports:

ToolPurpose
query Multi-type search (lex/vec/hyde). The flagship tool. Accepts 1-10 typed sub-queries.
get Single doc by path or docid. Supports :line ranges. Suggests similar files on 404.
multi_get Batch retrieval via glob or CSV. 10KB default max per file.
status Index health: doc counts, embedding coverage, collection metadata.

Resource: qmd:// URIs

Read-only markdown access via qmd://{+path} resource templates. Discovery is search-only (no directory listing).

Two transports

# Stdio (Claude Code, Cursor)
$ qmd mcp

# HTTP (shared, long-lived)
$ qmd mcp --http --port 8181

# Background daemon
$ qmd mcp --http --daemon
$ qmd mcp stop

Dynamic instructions

On MCP initialize, the server builds a system prompt including:

  • Collection inventory with doc counts
  • Vector index availability warnings
  • Query type examples and strategies
  • Retrieval workflow documentation

Collections & Hierarchical Context

Collection model

# YAML config (~/.config/qmd/index.yml)
collections:
  meeting-notes:
    path: ~/Documents/meetings
    pattern: "**/*.md"
    ignore:
      - "**/drafts/**"
    context:
      /: "Weekly team meeting notes"
      /2025: "Current year meetings"
      /2025/q1: "Q1 planning sessions"
  docs:
    path: ~/projects/api/docs
    pattern: "**/*.{md,mdx}"

Virtual paths

Universal addressing across collections:

# Full virtual path
qmd://meeting-notes/2025/standup.md

# Context resolution (longest prefix)
qmd://meeting-notes/2025/q1/jan.md
  → matches /2025/q1 context
  → "Q1 planning sessions"

Context hierarchy

  • Path-prefix matching (longest wins)
  • Global context applies to all collections
  • Attached to search results for LLM context
  • qmd context check finds missing descriptions

CLI Commands & Output Formats

Core commands

# Index your files
$ qmd collection add ~/notes --name notes
$ qmd embed

# Three search modes
$ qmd search "budget Q4"       # BM25
$ qmd vsearch "spending plans"  # vector
$ qmd query "what did we spend" # hybrid

# Retrieve documents
$ qmd get meeting.md:50       # from line 50
$ qmd get "#abc123"           # by docid
$ qmd multi-get "journals/2025*"

# Maintenance
$ qmd update --pull   # git pull + reindex
$ qmd status          # index health
$ qmd cleanup         # vacuum + prune

Output formats

FlagFormatUse Case
(default)Colorized CLIHumans
--jsonStructured JSONAI agents
--csvCSVSpreadsheets
--mdMarkdownDocumentation
--xmlXMLPipelines
--filesFile listScripting

Key flags

  • -c collection — restrict to collection
  • --min-score 0.5 — filter weak results
  • --explain — show full score trace
  • --no-rerank — skip LLM reranking
  • --full — show full document content

SDK / Library Mode

Use QMD as an embedded search engine in your own Node.js/Bun apps:

import { createStore } from '@tobilu/qmd';

// Initialize from YAML, inline config, or DB path
const store = await createStore({
  dbPath: './my-index.sqlite',
  collections: {
    docs: { path: './content', pattern: '**/*.md' }
  }
});

// Hybrid search
const results = await store.search({
  queries: [
    { type: 'lex', query: 'API rate limits' },
    { type: 'vec', query: 'throttling and quotas' }
  ],
  limit: 10,
  minScore: 0.3
});

// Document retrieval
const doc = await store.get('api/auth.md');
const body = await store.getDocumentBody(
  doc.filepath, { fromLine: 1, maxLines: 50 }
);

await store.close();

QMDStore interface

  • search() — Full hybrid with expansion + reranking
  • searchLex() — BM25 only
  • searchVector() — Vector only
  • expandQuery() — Manual expansion
  • get() / multiGet() — Retrieval
  • addCollection() / removeCollection()
  • addContext() / listContexts()
  • update() / embed() — Indexing
  • getStatus() / getIndexHealth()
  • close() — Lifecycle

Config sources: Inline config, YAML file, or existing DB path. Supports lazy-loaded LLM models and write-through sync between YAML and SQLite.

QMD in a Nutshell

WhatOn-device hybrid search engine for markdown/knowledge bases
HowBM25 + vectors + LLM reranking, all in SQLite
Models3 GGUF models, ~2GB total, via node-llama-cpp
IntegrationCLI, MCP server, and npm SDK
Secret sauceFine-tuned query expansion + position-aware score blending

github.com/tobi/qmd  ·  MIT License  ·  npm install -g @tobilu/qmd