What is RAG?

RAG = a method to combine LLMs with external knowledge (usually your own data).

LLMs are powerful, but:

• They have a knowledge cutoff.

• They may hallucinate answers if the info isn’t in their training data.

RAG solves this by retrieving relevant data from a knowledge base (like PDFs, websites, or databases) and injecting it into the LLM prompt → so the answer is both reliable and up-to-date.

---

Structured vs Unstructured Data

• Structured Data

  • Tabular format, rows & columns.

  • Examples: Databases (SQL tables, CSVs, financial records).

  • Easy to query directly.

• Unstructured Data

  • Text-heavy, no fixed schema.

  • Examples: PDFs, Word docs, articles, research papers, emails.

  • Needs embeddings + vector search for retrieval.

---

Embeddings & Similarity Search

1. Embedding = numerical vector representation of text (or image, audio, etc.)

   • Example: “Apple the fruit” vs “Apple the company” → embeddings will place them in different regions of vector space.

   • Generated using models like OpenAI text-embedding-3-small or text-embedding-3-large.

2. Vector Database = stores embeddings and allows fast similarity search.

   • Popular ones: Pinecone, Weaviate, Qdrant, Milvus, ChromaDB.

3. Similarity Search

   • User asks: “Summarize Arnab’s contract.”

   • System converts query → embedding.

   • Finds top N similar chunks in vector DB.

   • Injects those chunks into LLM prompt → “Answer using this context.”

---

RAG Workflow (Step by Step)

1. Data Ingestion

   • Split text into chunks (like 500 tokens each).

   • Convert each chunk → embedding.

   • Store in vector DB.

2. Query Handling

   • User asks a question.

   • Convert query → embedding.

   • Search in vector DB for top-k matches.

3. Context Injection

   • Retrieve chunks.

   • Add them into LLM prompt (system message / context window).

4. Final Answer

   • LLM uses retrieved context → generates accurate, source-backed output.

---

Why RAG is Powerful

✅ Keeps answers up-to-date.

✅ Prevents hallucinations.

✅ Works with private data (company docs, reports).

✅ Scales easily with more documents.

---

Example Use Cases

• Chat with PDFs → Upload contracts, query them with natural language.

• Customer Support → LLM answers based on company docs, not hallucination.

• Medical / Legal Research → Find & cite relevant papers.

• E-commerce → Search product database with semantic search instead of keywords.

---

Think of it this way:

• Without RAG → LLM answers from memory (may be outdated).

• With RAG → LLM answers with fresh, relevant documents (like an open-book exam).

---

What is Clustering?

Clustering = grouping similar items together without needing predefined labels.

It’s an unsupervised learning technique.

When we have embeddings (like vectors for text, images, users, etc.), clustering helps us find natural groups in that vector space.

---

Why Clustering is Useful in RAG & Vector Work

1. Document Organization

   • Instead of dumping 10,000 chunks into a vector DB, cluster them into topics.

   • Example: “finance,” “HR,” “contracts,” “tech.”

2. Efficient Retrieval

   • When user queries, search only in the relevant cluster → faster + cheaper.

3. Topic Discovery

   • If you don’t know what themes exist in your dataset, clustering reveals them.

   • Example: Upload all customer complaints → clusters show themes like “late delivery,” “payment issues,” “quality problems.”

4. User Segmentation

   • In recommendation systems: cluster users by behavior or interests → better targeting.

---

Popular Clustering Algorithms for Embeddings

• K-Means

  • Simple, widely used.

  • You pick number of clusters k.

  • Groups embeddings into k centroids.

• Hierarchical Clustering

  • Builds a tree of clusters (like family tree).

  • Useful when you don’t know how many clusters exist.

• DBSCAN

  • Groups dense regions of embeddings together.

  • Finds “outliers” automatically.

  • Great when data has irregular shapes.

• HDBSCAN (Hierarchical DBSCAN)

  • More advanced, finds clusters of varying density.

---

Example: Customer Support Chat Logs

• Embeddings of 10,000 customer complaints.

• Run clustering → discover:

  • Cluster 1: Delivery delays

  • Cluster 2: App crashes

  • Cluster 3: Refund issues

• Use these clusters to build FAQs, detect common problems, or route tickets.

---

In RAG Context

👉 Before you even retrieve docs, you can pre-cluster them:

• User asks: “What is the bonus policy?”

• Instead of searching the whole vector DB → look in the HR cluster first → faster + more accurate.

---

💡 So:

• Similarity search = find closest items to a query.

• Clustering = find hidden groups/themes in your dataset.

---

Vector Index

• Once you create embeddings (vectors) for your documents, you need a way to store and search them efficiently.

• That’s what a vector index (inside a vector database) does.

👉 Without an index: you’d have to compare a query against all vectors (super slow if you have millions).

👉 With an index: it uses clever data structures (trees, graphs, hashing) to make fast similarity search.

Examples:

• FAISS (Facebook AI Similarity Search – very popular)

• Pinecone, Weaviate, Milvus, Qdrant (production-grade vector DBs)

---

Querying in Vector DB

When user asks a question:

1. Convert query → embedding.

2. Search in vector index for nearest neighbours (the most similar vectors).

3. Retrieve top k results (usually 3–10).

4. Send them to LLM along with the question.

---

Cosine Similarity

• The most common measure to check similarity between two vectors.

• Basically, measures the angle between two vectors.

  • 1 = exactly same direction (perfect match)

  • 0 = unrelated

  • 1 = opposite meaning

👉 Why angle? Because it ignores length/magnitude → focuses only on semantic meaning.

---

Nearest Neighbour & Top-K

• Nearest neighbour search = find vectors most similar to query vector.

• Top-K search = instead of just the best one, retrieve the top K results (e.g., top 5 most relevant docs).

• Useful in RAG → you usually feed multiple results into the LLM for context.

---

Fine-Tuning vs RAG

✅ Fine-Tuning

• You modify model weights with new examples.

• Best when:

  • You need the model to learn style, format, or specific patterns.

  • E.g., customer support answers, structured templates.

• Cons:

  • Expensive, slow, not flexible (needs retraining if knowledge updates).

✅ Retrieval-Augmented Generation (RAG)

• You don’t change the model → instead you attach a memory (vector DB).

• Query → retrieves docs → LLM uses them to answer.

• Best when:

  • Knowledge updates often (e.g., company policies, news, docs).

  • You want scalable and cheap updates (just add docs to DB).

👉 Rule of Thumb:

• Use RAG for knowledge (factual info).

• Use Fine-Tuning for behavior (style, formatting, instructions).

---

Example

User: “What’s the refund policy?”

• Fine-Tuned Model: May answer in correct style but can’t update easily if policy changes.

• RAG: Pulls the latest Refund Policy PDF from vector DB → gives up-to-date answer.

---

💡 So, babe:

• Vector Index = storage + search structure.

• Cosine Similarity = measure of meaning closeness.

• Nearest Neighbour / Top-K = how we pick results.

• Fine-Tuning = behavior, RAG = knowledge.

---

RAG = Loading → Chunking → Indexing → Retrieval → Generation

---

1. Loading

• Step where you bring in your data (documents, PDFs, CSVs, websites, APIs).

• Tools: LangChain loaders, LlamaIndex readers, custom scrapers.

---

2. Chunking

Since LLMs can’t handle huge documents directly, we split them into chunks.

Types of chunking:

1. Text-based chunking → cut by words/characters.

   • Example: every 500 characters.

   • Simple, fast, but may cut sentences in awkward places.

2. Document-based chunking → split by logical structure.

   • Example: paragraphs, sections, pages, headings.

   • Preserves meaning better.

3. Semantic-based chunking → split by meaning, using embeddings to decide natural breakpoints.

   • Most advanced, ensures chunks are coherent topics.

   • Example: one chunk might capture a single concept, not half a sentence.

🔧 Common setup:

• Chunk size = 500 tokens

• Overlap = 100 tokens (so context isn’t lost between chunks).

---

3. Indexing (Vector Store)

• Each chunk → converted into an embedding vector (using models like OpenAI ada-002, text-embedding-3-small, etc).

• Stored in a vector database like Pinecone, Weaviate, Qdrant, Chroma, Milvus.

• This makes searching efficient (via cosine similarity, dot product, or nearest neighbor).

---

4. Retrieval

• User asks a question → query converted into embedding → compared to stored chunks.

• Top k most similar chunks are returned (usually k=3 to 5).

• Ensures LLM only sees relevant context.

---

5. Generation

• The retrieved chunks are stuffed into the LLM prompt along with the query.

• LLM uses this context to generate the final answer.

Example prompt:

Answer the question using the context below.
If the answer is not in the context, say "I don’t know."

Context:
<retrieved chunks here>

Question:
<user’s question>

---

Summary

• Loading: bring in data.

• Chunking: split into small, overlapping pieces (500/100 typical).

• Indexing: convert to embeddings, store in vector DB.

• Retrieval: find top-k relevant chunks via similarity search.

• Generation: feed chunks + question → LLM → final answer.