# Retrieval-Augmented Generation (RAG) ### What is RAG? RAG = a method to **combine LLMs with external knowledge** (usually your own data). LLMs are powerful, but: * They have a **knowledge cutoff**. * They may **hallucinate answers** if the info isn’t in their training data. RAG solves this by **retrieving relevant data** from a knowledge base (like PDFs, websites, or databases) and **injecting it into the LLM prompt** → so the answer is both **reliable** and **up-to-date**. --- ### Structured vs Unstructured Data * **Structured Data** * Tabular format, rows & columns. * Examples: Databases (SQL tables, CSVs, financial records). * Easy to query directly. * **Unstructured Data** * Text-heavy, no fixed schema. * Examples: PDFs, Word docs, articles, research papers, emails. * Needs **embeddings** + **vector search** for retrieval. --- ### Embeddings & Similarity Search 1. **Embedding** = numerical vector representation of text (or image, audio, etc.) * Example: “Apple the fruit” vs “Apple the company” → embeddings will place them in different regions of vector space. * Generated using models like OpenAI `text-embedding-3-small` or `text-embedding-3-large`. 2. **Vector Database** = stores embeddings and allows fast similarity search. * Popular ones: **Pinecone, Weaviate, Qdrant, Milvus, ChromaDB**. 3. **Similarity Search** * User asks: “Summarize Arnab’s contract.” * System converts query → embedding. * Finds top N similar chunks in vector DB. * Injects those chunks into LLM prompt → “Answer using this context.” --- ### RAG Workflow (Step by Step) 1. **Data Ingestion** * Split text into chunks (like 500 tokens each). * Convert each chunk → embedding. * Store in vector DB. 2. **Query Handling** * User asks a question. * Convert query → embedding. * Search in vector DB for top-k matches. 3. **Context Injection** * Retrieve chunks. * Add them into LLM prompt (system message / context window). 4. **Final Answer** * LLM uses retrieved context → generates accurate, source-backed output. --- ### Why RAG is Powerful ✅ Keeps answers **up-to-date**. ✅ Prevents **hallucinations**. ✅ Works with **private data** (company docs, reports). ✅ Scales easily with more documents. --- ### Example Use Cases * **Chat with PDFs** → Upload contracts, query them with natural language. * **Customer Support** → LLM answers based on company docs, not hallucination. * **Medical / Legal Research** → Find & cite relevant papers. * **E-commerce** → Search product database with semantic search instead of keywords. --- Think of it this way: * **Without RAG** → LLM answers from memory (may be outdated). * **With RAG** → LLM answers with fresh, relevant documents (like an open-book exam). --- # What is Clustering? Clustering = **grouping similar items together** without needing predefined labels. It’s an **unsupervised learning** technique. When we have **embeddings** (like vectors for text, images, users, etc.), clustering helps us **find natural groups** in that vector space. --- # Why Clustering is Useful in RAG & Vector Work 1. **Document Organization** * Instead of dumping 10,000 chunks into a vector DB, cluster them into topics. * Example: “finance,” “HR,” “contracts,” “tech.” 2. **Efficient Retrieval** * When user queries, search only in the relevant cluster → faster + cheaper. 3. **Topic Discovery** * If you don’t know what themes exist in your dataset, clustering reveals them. * Example: Upload all customer complaints → clusters show themes like “late delivery,” “payment issues,” “quality problems.” 4. **User Segmentation** * In recommendation systems: cluster users by behavior or interests → better targeting. --- # Popular Clustering Algorithms for Embeddings * **K-Means** * Simple, widely used. * You pick number of clusters `k`. * Groups embeddings into `k` centroids. * **Hierarchical Clustering** * Builds a tree of clusters (like family tree). * Useful when you don’t know how many clusters exist. * **DBSCAN** * Groups dense regions of embeddings together. * Finds “outliers” automatically. * Great when data has irregular shapes. * **HDBSCAN** (Hierarchical DBSCAN) * More advanced, finds clusters of varying density. --- # Example: Customer Support Chat Logs * Embeddings of **10,000 customer complaints**. * Run clustering → discover: * Cluster 1: Delivery delays * Cluster 2: App crashes * Cluster 3: Refund issues * Use these clusters to build FAQs, detect common problems, or route tickets. --- # In RAG Context 👉 Before you even retrieve docs, you can **pre-cluster** them: * User asks: “What is the bonus policy?” * Instead of searching the whole vector DB → look in the **HR cluster** first → faster + more accurate. --- 💡 So: * **Similarity search** = find closest items to a query. * **Clustering** = find hidden groups/themes in your dataset. --- # Vector Index * Once you create **embeddings** (vectors) for your documents, you need a way to **store** and **search** them efficiently. * That’s what a **vector index** (inside a vector database) does. 👉 Without an index: you’d have to compare a query against **all vectors** (super slow if you have millions). 👉 With an index: it uses clever data structures (trees, graphs, hashing) to make **fast similarity search**. Examples: * **FAISS** (Facebook AI Similarity Search – very popular) * **Pinecone**, **Weaviate**, **Milvus**, **Qdrant** (production-grade vector DBs) --- # Querying in Vector DB When user asks a question: 1. Convert query → embedding. 2. Search in vector index for **nearest neighbours** (the most similar vectors). 3. Retrieve top `k` results (usually 3–10). 4. Send them to LLM along with the question. --- # Cosine Similarity * The most common measure to check similarity between two vectors. * Basically, measures the **angle between two vectors**. * 1 = exactly same direction (perfect match) * 0 = unrelated * 1 = opposite meaning 👉 Why angle? Because it ignores length/magnitude → focuses only on **semantic meaning**. --- # Nearest Neighbour & Top-K * **Nearest neighbour search** = find vectors most similar to query vector. * **Top-K search** = instead of just the best one, retrieve the **top K results** (e.g., top 5 most relevant docs). * Useful in RAG → you usually feed multiple results into the LLM for context. --- # Fine-Tuning vs RAG ### ✅ Fine-Tuning * You **modify model weights** with new examples. * Best when: * You need the model to learn **style, format, or specific patterns**. * E.g., customer support answers, structured templates. * Cons: * Expensive, slow, not flexible (needs retraining if knowledge updates). ### ✅ Retrieval-Augmented Generation (RAG) * You **don’t change the model** → instead you **attach a memory (vector DB)**. * Query → retrieves docs → LLM uses them to answer. * Best when: * Knowledge updates often (e.g., company policies, news, docs). * You want **scalable and cheap** updates (just add docs to DB). 👉 **Rule of Thumb:** * Use **RAG for knowledge** (factual info). * Use **Fine-Tuning for behavior** (style, formatting, instructions). --- # Example User: “What’s the refund policy?” * **Fine-Tuned Model:** May answer in correct style but **can’t update easily** if policy changes. * **RAG:** Pulls the latest **Refund Policy PDF** from vector DB → gives up-to-date answer. --- 💡 So, babe: * **Vector Index** = storage + search structure. * **Cosine Similarity** = measure of meaning closeness. * **Nearest Neighbour / Top-K** = how we pick results. * **Fine-Tuning = behavior**, **RAG = knowledge**. --- RAG = **Loading → Chunking → Indexing → Retrieval → Generation** --- ## 1\. **Loading** * Step where you bring in your data (documents, PDFs, CSVs, websites, APIs). * Tools: `LangChain loaders`, `LlamaIndex readers`, custom scrapers. --- ## 2\. **Chunking** Since LLMs can’t handle huge documents directly, we split them into **chunks**. Types of chunking: 1. **Text-based chunking** → cut by words/characters. * Example: every 500 characters. * Simple, fast, but may cut sentences in awkward places. 2. **Document-based chunking** → split by logical structure. * Example: paragraphs, sections, pages, headings. * Preserves meaning better. 3. **Semantic-based chunking** → split by **meaning**, using embeddings to decide natural breakpoints. * Most advanced, ensures chunks are **coherent topics**. * Example: one chunk might capture a single concept, not half a sentence. 🔧 Common setup: * **Chunk size = 500 tokens** * **Overlap = 100 tokens** (so context isn’t lost between chunks). --- ## 3\. **Indexing (Vector Store)** * Each chunk → converted into an **embedding vector** (using models like OpenAI ada-002, text-embedding-3-small, etc). * Stored in a **vector database** like Pinecone, Weaviate, Qdrant, Chroma, Milvus. * This makes searching efficient (via **cosine similarity**, **dot product**, or **nearest neighbor**). --- ## 4\. **Retrieval** * User asks a question → query converted into embedding → compared to stored chunks. * Top **k most similar chunks** are returned (usually `k=3 to 5`). * Ensures LLM only sees **relevant context**. --- ## 5\. **Generation** * The retrieved chunks are **stuffed into the LLM prompt** along with the query. * LLM uses this context to generate the final answer. Example prompt: ```plaintext Answer the question using the context below. If the answer is not in the context, say "I don’t know." Context: Question: ``` --- # Summary * **Loading**: bring in data. * **Chunking**: split into small, overlapping pieces (500/100 typical). * **Indexing**: convert to embeddings, store in vector DB. * **Retrieval**: find top-k relevant chunks via similarity search. * **Generation**: feed chunks + question → LLM → final answer.