---
name: build-rag-search-over-docs
description: Builds retrieval-augmented generation (RAG) so an AI answers from your documents - split docs into chunks, embed them into a vector store, retrieve the best matches, and answer with citations or say it does not know. Use when the user wants a chatbot that knows their docs, AI search over a knowledge base, or answers grounded in their content.
license: MIT
metadata:
  category: backend
  tags: "rag,chatbot,docs,llm"
  author: johnny-bravo
  version: "1.0"
  updated: "2026-11-02"
  status: published
  aliases: "rag, vector search, embeddings, chat with your docs"
  source_url: "https://github.com/wshobson/agents/tree/7e2fb559cf4b824b27f1fec49fe88f7511066d26/plugins/llm-application-dev/skills/rag-implementation"
  source_license: MIT
  copyright: "Copyright (c) 2024 Seth Hobson"
  original_author: "Seth Hobson (wshobson/agents, rag-implementation)"
  modified: "Adapted by TheSkillVault: rewritten as a framework-free build order for beginners with an original answer prompt; LangGraph quick start and long catalogues dropped; added privacy notes, gotchas and done-when."
  imported_on: "2026-11-02"
---

# Build RAG search over your docs

RAG means: find the most relevant passages first, then ask the model to answer only from them. It cuts made-up answers and lets you cite sources.

## When to use
- You want a help bot that answers from your docs, policies or notes.
- Plain chat answers are wrong because the model never saw your content.
- You need answers with links back to the source page.

## When not to use
- You only need a general chatbot. Use build-ai-chatbot-api-route.

## Steps
1. Collect the source docs and keep a URL or file path for each.
2. Split them into chunks of about 300-800 words with a small overlap, keeping headings with each chunk.
3. Create embeddings for every chunk with one embedding model, and store them with the text and source in a vector store (pgvector, a hosted vector database, or a local index).
4. At question time, embed the question and fetch the top 5-10 chunks. Add keyword search too if exact names or codes matter.
5. Optionally rerank the results and keep the best 3-5.
6. Send the model the question plus those chunks, with the prompt below.
7. Return the answer with its sources listed.
8. Build a small test set of 20 real questions with known answers and check it after every change.
9. Re-embed changed docs when content updates.

```text
Answer the question using only the sources below. Cite sources as [1], [2].
If the sources do not contain the answer, say "I don't know based on these docs."
Sources:
{{numbered chunks with titles}}
Question: {{question}}
```

## Gotchas
- Do not index private or sensitive documents into a bot that the public can query.
- Keep API keys for the model and the vector store on the server, never in browser code.
- Chunks that are too small lose context; too large and retrieval gets vague. Test both.
- Changing the embedding model means re-embedding everything.

## Portability
Works with any model provider and vector store. If you use LangChain or LlamaIndex, they wrap these same steps; a plain Postgres table with pgvector is an equivalent option for small projects.

## Example prompt
"Build a help chatbot that answers from the Markdown files in /docs, cites the page it used, and says when it does not know."

## Done when
- Answers cite the chunks they used.
- Off-topic questions get "I don't know" rather than a guess.
- A saved test set of real questions passes at an agreed rate.

---
Adapted from "rag-implementation" in wshobson/agents (MIT, Copyright (c) 2024 Seth Hobson). See LICENSE.txt.