# Tool Calling with LLMs

## 🔹 What is Tool Calling?

* **Definition**: Tool calling is when LLMs (Large Language Models) are given access to **external functions, APIs, or databases**.
    
* Instead of **hallucinating answers**, the model decides:
    
    👉 “Do I know this? Or should I call a tool to get the real answer?”
    
* **Tools = just functions** that LLMs can request during a conversation.
    

✅ Example:

* Without tools → “Who won the 2025 IPL final?” → Model fails (knowledge cutoff).
    
* With tool calling → Model calls a **web search API** → gets fresh data → replies correctly.
    

---

## 🔹 Why Use Tool Calling? (Use Cases)

1. **Knowledge Beyond Cutoff**
    
    * LLMs are trained only up to a fixed date (knowledge cutoff).
        
    * Tools let them fetch **real-time info**.
        
    * Example: “What’s the weather in Kolkata today?”
        
2. **Web Search**
    
    * Search APIs: **Tavily, Serper, Brave, Bing**.
        
    * Use case: live news, sports results, product availability.
        
3. **Database Lookups**
    
    * Fetch user records, inventory, product prices.
        
    * Example: “Show me my last 5 transactions.”
        
4. **Math & Specialized Tasks**
    
    * Calculator, unit conversion, finance APIs, weather APIs.
        
    * Example: “Convert $100 to INR at today’s rate.”
        
5. **Automation & Actions**
    
    * LLMs can **execute tasks**, not just reply with text.
        
    * Example: Book tickets, send an email, update a calendar.
        
6. **Data Enrichment**
    
    * Mix reasoning + tool data.
        
    * Example: “List top 5 coffees under ₹500 in India” → LLM calls product API → formats neatly.
        

---

## 🔹 Popular Web Search Tools

| Tool/API | Best Use Case | Production Ready? | Notes |
| --- | --- | --- | --- |
| **Serper** | Hobby/small side projects | ❌ No | Wrapper around Google. Quick, but limited. |
| **Brave API** | Privacy-focused search | ❌ No | Good for experiments, slower than Google. |
| **Tavily** | Real-world apps | ✅ Yes | Most reliable, structured JSON responses, AI-optimized. |
| **Bing Search** | Microsoft ecosystem apps | ✅ Yes | Works well, but sometimes noisy results. |

👉 **Best choice for prototyping/production** = **Tavily**.

---

## 🔹 Tool Calling Flow

1. **User asks a question.**
    
2. **LLM decides**: Do I have enough knowledge? If not → request a tool.
    
3. **LLM generates a tool\_call object.**
    
    ```json
    {
      "tool": "webSearch",
      "arguments": { "query": "latest cricket world cup winner" }
    }
    ```
    
4. **Backend intercepts & executes the tool** (API call, function).
    
5. **Tool response fed back into the conversation.**
    
6. **LLM uses result** → generates final human-friendly answer.
    

🔁 Repeat if multiple tools needed until final response is ready.

---

## 🔹 Example Code (Web Search Tool)

```jsx
tools: [
  {
    type: "function",
    function: {
      name: "webSearch",
      description: "Search for latest information",
      parameters: {
        type: "object",
        properties: {
          query: { type: "string", description: "The search query" }
        },
        required: ["query"]
      }
    }
  }
],
tool_choice: "auto"
```

* `tool_choice: "auto"` → LLM decides when to call the tool.
    
* Backend captures `tool_call` → executes API → returns results.
    
* LLM integrates tool output → gives final reply.
    

---

## 🔹 Knowledge Cutoff

* Every model has a **knowledge cutoff date** (last training data).
    
* Example: GPT-4 cutoff → Oct 2023.
    
* If asked about 2025 events → must use tool calling (e.g., web search).
    

---

## 🔹 Streaming Responses

* Normally: wait until the full answer is ready.
    
* With streaming: **receive tokens live as they’re generated**.
    
* Benefits:
    
    ✅ Feels instant & interactive.
    
    ✅ Great for chatbots, coding assistants, writing apps.
    

⚠️ Limitation:

* `response_format: json_object` ❌ does not support streaming.
    
    * Because streaming JSON risks **broken/incomplete JSON mid-way**.
        

---

## 🔹 Best Practices

* Use **one reliable search API** (Tavily recommended).
    
* Always **validate tool input/output** (use schema validation).
    
* Add **timeouts, retries, and error handling** in production.
    
* Prevent infinite loops if LLM keeps calling the tool.
    

---

✨ **In short:**

Tool calling makes LLMs **more powerful and trustworthy** by connecting them to **real data, APIs, and actions**. It bridges the gap between **static knowledge** and **dynamic, real-world needs**.