George RAG Component
Implementation Reference

Chapter 01 Executive Summary

The problem: George's original story system loaded the user's entire story file into every LLM prompt as a raw text dump, truncated hard at 6,000 characters. All context was always identical regardless of what the user asked, the file size was capped regardless of how much lore the user wrote, and multiple source files were impossible.

The solution: A TF-IDF vector index with an HTTP sidecar server. Story files are chunked into passages, each passage is embedded into a sparse vector, and at query time only the top-3 most semantically relevant passages are retrieved and injected. The result is targeted context, no file size limit, multi-file support, sub-millisecond queries, and zero new Swift dependencies.

Chapter 02 Problem Statement & Prior Art

2.1 The Old System (storyBible)

Before this PR, the story pipeline in GeorgeController.swift read the entire file synchronously, truncated it at 6,000 chars, and pasted it verbatim into every prompt — no relevance filtering whatsoever.

Swift// GeorgeController.swift — BEFORE this PR
private var storyBible: String = ""

func loadStoryFile(at path: String, announce: Bool = true) {
    let raw = try String(contentsOfFile: path, encoding: .utf8)
    // PROBLEM 1: hard cap — files >6,000 chars are silently truncated
    storyBible = raw.count > 6000 ? String(raw.prefix(6000)) : raw
}

private func buildStoryPrompt(topic: String) -> String {
    // PROBLEM 2: always paste the ENTIRE file, no matter what topic was asked
    if !storyBible.isEmpty {
        sections.append("=== STORY SOURCE FILE ===\n\(storyBible)\n=== END ===")
    }
    // A query about "the Hollow King" gets identical context to
    // a query about "the town market". No relevance filtering at all.
}

2.2 Identified Failure Modes

Chapter 03 Solution Design

3.1 Architecture Overview

The RAG system is a sidecar architecture: a Python HTTP server runs alongside George's existing MLX inference server, exposing two endpoints. George's Swift code calls these endpoints instead of managing text directly.

Architecture┌─────────────────────────────────────────┐
│ George macOS Process                    │
│                                         │
│  GeorgeController.swift                 │
│    │ loadStoryFile() → POST /index      │
│    │ buildStoryPrompt() → POST /query   │
│    ▼                                    │
│  RAGClient.swift ──────────────────────►│
│                       HTTP localhost:8181│
└────────┬────────────────────────────────┘
         │
         ▼
┌─────────────────────────────────────────┐
│ rag_server.py (sidecar process)         │
│                                         │
│  POST /index ──► chunk_text()           │
│               ──► TfidfVectorizer.fit() │
│               save to ~/.george/rag_*   │
│                                         │
│  POST /query ──► vectorizer.transform() │
│               ──► cosine_similarity()   │
│               return top-K chunks       │
└─────────────────────────────────────────┘

3.2 Technology Decisions

Decision 1: TF-IDF over Neural Embeddings

Decision 2: HTTP Sidecar over Swift Native

An alternative design embedded retrieval logic directly in Swift. This was rejected because scikit-learn's TF-IDF uses heavily optimised sparse matrix arithmetic that would take weeks to replicate. The HTTP API is also the stable contract — the Python backend can be swapped for BM25, dense retrieval, or neural embeddings without touching a single line of Swift.

Decision 3: Refit-from-Scratch on Every Index Operation

When a new file is indexed, the TfidfVectorizer is refitted from scratch across all chunks in the corpus. IDF weights are global — they reflect how rare a word is across the entire corpus. An incremental vectorizer would produce incorrect IDF weights if two files both contain the same character name. Measured cost: 4ms for a 12,000-character corpus (60 chunks). Acceptable.

Chapter 04 Repository Structure

This PR adds one new directory to the george_sadface repo:

Shellgeorge_sadface/
├── Sources/George/
│   ├── GeorgeController.swift  ← 3-line patch (see §7)
│   ├── RAGClient.swift         ← NEW: Swift HTTP client + lifecycle
│   └── Resources/
│       └── rag_server.py       ← NEW: Python RAG server
├── Tests/
│   └── test_rag.py             ← NEW: 35-test suite
├── README.md                   ← Updated: RAG quick-start added
└── Package.swift               ← Unchanged (no new Swift deps)

Chapter 05 rag_server.py — Deep Dive

5.1 Text Chunking

Chunking converts a raw story file into retrieval units. The design goal is chunks large enough to contain a complete thought but small enough that a single query surfaces only the relevant portion.

Pythondef chunk_text(text: str) -> List[str]:
    # Phase 1: split on paragraph boundaries (double newline)
    raw_paragraphs = re.split(r'\n\s*\n', text.strip())
    for para in raw_paragraphs:
        if len(para) <= MAX_CHARS:
            chunks.append(para)   # short paragraph: keep as-is
        else:
            start = 0
            while start < len(para):
                chunk = para[start : start + MAX_CHARS]
                # Phase 2: prefer breaking at sentence boundary
                for sep in ('. ', '! ', '? '):
                    last = chunk.rfind(sep)
                    if last > MAX_CHARS // 2:
                        chunk = chunk[:last + 1]
                        break
                chunks.append(chunk.strip())
                start += max(len(chunk) - OVERLAP_CHARS, 1)
    # Phase 3: discard degenerate chunks
    return [c for c in chunks if len(c.strip()) >= 20]

5.2 TF-IDF Vectorizer Configuration

Each parameter is a deliberate choice tuned for user-authored fantasy/fiction content:

Pythonself.vectorizer = TfidfVectorizer(
    ngram_range=(1, 2),   # unigrams + bigrams
    # "hollow king" scores as a single high-IDF term
    min_df=1,             # include ALL words — fantasy names appear once
    max_df=0.95,          # drop words in >95% of chunks ("the", "and")
    sublinear_tf=True,    # log(1 + tf) prevents frequency flooding
    strip_accents="unicode",
    analyzer="word",
)

5.3 RAGIndex — Core Data Structure

RAGIndex is a class (not a module-level dict) for encapsulation and testability. Three persistence files are written to ~/.george/ on every index operation:

5.4 Cosine Similarity Search

Pythondef query(self, query_text: str, top_k: int = 3) -> List[Dict]:
    q_vec = self.vectorizer.transform([query_text])  # sparse [1, vocab]
    # cosine_similarity handles L2 normalisation internally
    scores = cosine_similarity(q_vec, self.matrix)[0]  # [N]
    top_idx = np.argsort(scores)[::-1][:top_k]
    return [
        {"rank": int(r+1), "score": round(float(scores[i]), 4),
         "chunk": self.chunks[i], "source": Path(self.sources[i]).name}
        for r, i in enumerate(top_idx)
    ]

Chapter 06 RAGClient.swift — Deep Dive

6.1 Class Design

RAGClient is a @MainActor singleton, consistent with GeorgeController itself. All async HTTP calls cross to background automatically via URLSession, then return results on MainActor — no manual DispatchQueue needed.

6.2 indexFile() — Replace storyBible Loading

Swiftfunc indexFile(path: String) async -> String {
    var req = URLRequest(url: baseURL.appendingPathComponent("index"))
    req.httpMethod = "POST"
    req.setValue("application/json", forHTTPHeaderField: "Content-Type")
    req.httpBody = try? JSONSerialization.data(withJSONObject: ["path": path])
    let (data, _) = try await URLSession.shared.data(for: req)
    let resp = try JSONDecoder().decode(RAGIndexResponse.self, from: data)
    if resp.ok {
        return "Got it. I indexed \(resp.file ?? "the file") into \(resp.chunks ?? 0) passages."
    } else {
        return "Sorry, I could not index that file. \(resp.error ?? "")"
    }
}

6.3 query() — Replace storyBible Injection

Swiftfunc query(_ queryText: String, topK: Int = 3) async -> String? {
    var req = URLRequest(url: baseURL.appendingPathComponent("query"))
    req.httpMethod = "POST"
    req.setValue("application/json", forHTTPHeaderField: "Content-Type")
    req.httpBody = try? JSONSerialization.data(withJSONObject: [
        "query": queryText, "top_k": topK
    ])
    let (data, _) = try await URLSession.shared.data(for: req)
    let resp = try JSONDecoder().decode(RAGQueryResponse.self, from: data)
    guard resp.ok, !resp.results.isEmpty else { return nil }
    return resp.context  // pre-assembled chunks joined with "---" separators
}

6.4 Server Lifecycle — launchRAGServer()

The sidecar is launched as a child process of George. It searches a priority-ordered list of candidate paths for rag_server.py, mirroring the same pattern used by GamePlayer.swift for the ArkanoidMac binary.

Chapter 07 GeorgeController.swift — The Three-Line Patch

Only three locations in GeorgeController.swift change. The diff is intentionally minimal — the goal was to augment the existing story pipeline, not rewrite it.

Change 1 — boot(): launch the sidecar

Swift// AFTER (one new line):
func boot() async {
    openCommandReference()
    gamePlayer = GamePlayer(george: self)
    RAGClient.launchRAGServer()  // ← NEW: start Python sidecar
    autoLoadStoryBible()
    await startInferenceServer()
}

Change 2 — loadStoryFile(): index instead of read

Swiftfunc loadStoryFile(at path: String, announce: Bool = true) {
    Task { @MainActor in
        let msg = await RAGClient.shared.indexFile(path: path)
        self.storyBiblePath = path
        self.storyBible = "RAG_INDEXED"  // sentinel: non-empty = indexed
        if announce { self.speakOneShot(msg) }
    }
}

Change 3 — buildStoryPrompt(): query instead of dump

Swiftif storyBible == "RAG_INDEXED" {
    let queryText = topic.isEmpty ? "story narrative characters world" : topic
    if let ctx = await RAGClient.shared.query(queryText, topK: 3) {
        sections.append("=== RELEVANT STORY CONTEXT (RAG) ===\n\(ctx)\n=== END CONTEXT ===")
    }
    // nil return = server down or empty index — prompt continues without story context
}

Chapter 08 Test Suite — test_rag.py

8.1 Test Architecture

8.2 Accuracy Tests — The Most Important Section

Six named queries are fired against the Ashenvale story corpus. Each query specifies keywords that must appear in the retrieved chunks. Pass condition: ANY keyword appears in the top-2 results. Threshold: ≥80% required. Actual: 100%.

Pythonaccuracy_tests = [
    ("Elara cartographer maps",              ["elara", "cartograph", "map"]),
    ("Hollow King Caeden underworld bargain", ["hollow", "caeden", "king", "void"]),
    ("Wren mechanical owl whispering",        ["wren", "owl", "whisper"]),
    ("iron law sorcerer exile",              ["iron", "sorcer", "exile", "law"]),
    ("Velmoor city memory keepers",          ["velmoor", "memory", "city", "ring"]),
    ("Elara brass compass spirit",           ["compass", "spirit", "brass", "elara"]),
]

8.3 Test Isolation

Every test section that touches RAGIndex uses a tempfile.TemporaryDirectory() and monkey-patches the three class-level path constants before instantiating the index. Tests never write to the real ~/.george/ directory and never interfere with each other.

Chapter 09 Benchmark Results

All benchmarks measured on Apple M-series (arm64), macOS Sonoma. Test corpus: 12,000 characters (60 chunks) — 10× the typical user story file.

Chapter 10 Known Limitations & Future Work

10.1 Current Limitations

10.2 Future Work

• BM25 retrieval — consistently better than TF-IDF for short-passage retrieval. rank-bm25 is a drop-in Python replacement.
• Hybrid retrieval — combine TF-IDF lexical score with a cross-encoder re-ranker for semantically distant queries.
• Neural embeddings (optional path) — the HTTP API is stable; swap rag_server.py internals for sentence-transformers without touching RAGClient.swift.
• Automatic re-index on file change — use FSEvents (macOS) to watch story file path and trigger a silent re-index when modified.
• Web UI for index inspection — a simple local HTML page served by rag_server.py showing all indexed chunks and retrieval scores for a test query.

Chapter 11 PR Review Checklist

Correctness

• chunk_text() produces non-overlapping unique chunks for all test inputs
• Re-indexing the same file replaces chunks, does not duplicate
• Multi-file indexing accumulates without cross-contamination
• Index persists to disk and reloads correctly across process restart
• query() returns nil gracefully when server is down — no crash
• buildStoryPrompt() fallback works when RAG context is unavailable

Tests

• All 35 tests pass with zero failures
• HTTP tests verify all status codes (200, 400, 404)
• Retrieval accuracy is 100% on the 6-query named-entity test suite
• Accuracy threshold check (≥80%) is enforced as a test assertion
• Test isolation — no writes to real ~/.george/ during test runs
• Performance tests enforce <10s index time, <200ms query latency

Integration

• GeorgeController diff is exactly 3 locations, all clearly marked
• Fallback graceful when rag_server.py not found — warning logged, George continues
• Existing "what's in your story file" voice command still works via sentinel check
• Chess, Arkanoid, notes, and all other subsystems are unaffected

Security

• Server binds to 127.0.0.1 only — not exposed on LAN
• No user-controlled input reaches subprocess arguments
• File path in POST /index is validated by Python's Path.read_text() — permission errors surface as ok=False
• No credentials, API keys, or sensitive data stored