Beyond Google Search: Understand and Build Your Personal AI with 'RAG Builder'
Have you ever felt like your most valuable resource—your personal notes, journals, and daily diaries—are just... sitting there? Hundreds of brilliant ideas, insights, and lessons are trapped in files, never to be easily recalled or connected. I certainly did. That feeling is exactly what led me to create RAG Builder, a simple, local-first system that turns your scattered thoughts into a powerful, queryable knowledge base. Whether you're a developer curious about local LLM applications or an AI enthusiast looking to truly leverage your data, this post is for you.
The Origin Story: From Substack Scroll to Code Sprint
It all started with a late-night scroll. I was reading a fascinating blog post on Substack about Retrieval-Augmented Generation (RAG) and the concept clicked instantly. Instead of a large language model (LLM) trying to guess an answer, what if it could fetch the most relevant information from a trusted source first?
That's when the "aha!" moment hit me: My notes are my most trusted source.
I immediately dropped everything and jumped into the code. Leveraging the power of modern AI tools like Cursor and a pure flow state (what I call "vibe coding"), I put the pedal to the metal. The entire core system—from splitting documents into chunks to setting up the local vector database and connecting the local LLM—came together in a furious three-day sprint. It was fast, fun, and incredibly rewarding. Now, the project is stable, robust, and live on GitHub, ready for you to clone and experiment with.
What Exactly is RAG? (And Why You Need It)
Before diving into the nitty-gritty of the implementation, let's quickly cover Retrieval-Augmented Generation (RAG). This is the core magic of RAG Builder.
Think of it like a brilliant, privacy-focused research assistant:
The Library (Your Notes): First, RAG Builder takes all your notes, journals, and diaries and processes them into small, searchable "chunks." It then stores the meaning of these chunks in a vector database (this is like an ultra-smart, meaning-based index).
The Search (Retrieval): When you ask a question (e.g., "What were my key takeaways from the conference last month?"), The system doesn't just search for keywords. It searches the vector database for the chunks that are most related to the meaning of your question.
The Answer (Generation): It takes these retrieved, related chunks (the context) and hands them to a local LLM (the brain), which uses only that information to formulate a coherent and accurate response.
The result? Highly personalized answers, grounded in your own knowledge, and completely private because everything runs locally!
How RAG Builder Works: A Technical Deep Dive
RAG Builder is a sophisticated system that transforms a collection of Markdown files (like an Obsidian vault) into a searchable knowledge base. The entire system is designed to run locally, ensuring your data remains private. It leverages powerful open-source tools like Ollama for running LLMs, Hugging Face Transformers for generating embeddings, and ChromaDB for persistent vector storage.
The project is split into two main pipelines: the Indexing Pipeline (which builds the knowledge base) and the Query Pipeline (which answers your questions).
High-Level Architecture
The following modules work together to create the RAG system:
Document Loader: Reads your Markdown files from the local filesystem.
Text Splitter: Breaks down large documents into smaller, manageable chunks.
Embedding Model: Converts the text chunks into numerical representations (vectors) that capture their semantic meaning.
Vector Store: A specialized database (ChromaDB) that stores these vectors and allows for efficient searching.
Retriever: Searches the vector store using an advanced hybrid search to find the most relevant document chunks.
LLM: A local language model (via Ollama) that receives the question and the retrieved context to generate a final answer.
Interfaces: The system provides both a Command-Line Interface (CLI) and a modern web interface for interaction.
The Indexing Pipeline: How We Build the Knowledge Base
This indexing pipeline runs the first time you start the application or when you manually trigger a refresh.
Step 1: Loading Your Documents
The process begins in the documentLoader.js module, which recursively scans the directory you provide (your note vault path) for all .md files. It intelligently ignores directories like .obsidian to skip metadata files. For each file, it extracts content and creates a structured document object with essential metadata.
Step 2: Splitting Text into Smart Chunks
The loaded documents are passed to a text splitter. The project uses LangChain's RecursiveCharacterTextSplitter, which is specifically configured to understand Markdown syntax. It attempts to split the text along logical boundaries (headings, paragraphs, sentences). Crucially, each chunk has a slight overlap with the previous one to maintain context between them.
Step 3: Generating Semantic Embeddings
Using the HuggingFaceTransformersEmbeddings library, specifically the popular all-MiniLM-L6-v2 model, each chunk is converted into a vector (a list of numbers). This process runs entirely on your local machine. Chunks with similar topics will have vectors that are "close" to each other in mathematical space, representing the text's meaning.
Step 4: Storing in the Vector Database
The chunks and their corresponding vectors are then stored in a ChromaDB database. Because ChromaDB is persistent, you only need to complete this process once. Subsequent startups will be much faster unless you explicitly decide to refresh the data.
The Query Pipeline: How We Answer Your Questions
When you ask a question, the query pipeline is activated to find the most accurate answer from your notes.
Step 1: Advanced Search and Retrieval
RAG Builder employs a sophisticated, multi-stage retrieval process:
Query Expansion: Your initial question is automatically expanded with synonyms and related terms to ensure a wider search (e.g., searching for "learning" might also search for "studying").
Hybrid Search: The system performs a hybrid search combining:
Semantic Search: Finds chunks that are conceptually related to your query.
Keyword Search: Looks for exact keyword matches within the document content.
Reranking: The combined results are then reranked using a specialized algorithm. This logic boosts the score of results that contain exact phrase matches or have keywords in their title, significantly improving accuracy.
Step 2: Building the Context
The top-ranked, most relevant chunks are then formatted into a single block of text called the "context." The system carefully assembles this context, including metadata like the source file and relevance score.
Step 3: Prompt Engineering and Generation
This context, along with your original question, is inserted into a carefully crafted prompt template. This engineered prompt gives the LLM very specific instructions, such as:
Answer only based on the provided context.
If the information isn't in the context, state that clearly.
Do not make up information (hallucinate).
Finally, this complete prompt is sent to the local LLM running via Ollama. The LLM processes the request and generates the final answer, which is then streamed back to you in the interface.
Interfaces and Usability
RAG Builder provides two simple ways to interact with your powerful knowledge base:
CLI (
cli.js): A straightforward command-line interface for users who prefer working in the terminal.Web Interface (
webServer.js): A modern, responsive web application built with Express.js. It provides a much richer user experience, with real-time search, source attribution, and the ability to refresh the knowledge base directly from the UI. It also exposes a full REST API for all its operations.
Conclusion
The journey of building RAG Builder has been a perfect illustration of what's possible with local AI and a weekend of "vibe coding." I hope this project inspires you not only to organize your personal knowledge but also to dive into building your own RAG application.
Now it's your turn!
Clone it: Check out the full source code and documentation on GitHub: https://github.com/ps011/rag-builder
Try it: Follow the setup guide and start asking your notes questions!
Share your insights: What are you going to build next with RAG? Let me know in the comments!
