+-----------------------------------------------------------------------------------------+
|                  Berkeley Function-Calling Leaderboard (BFCL V4)                        |
+-----------------------------------------------------------------------------------------+

TRADITIONAL CHAT BENCHMARKS (MMLU / GSM8k)
- Evaluates: Factual trivia recall, multiple-choice math questions.
- Flaw: Completely irrelevant for evaluating if an agent can invoke a Stripe API!

BFCL V4 AGENTIC BENCHMARK (The 2026 Gold Standard)
- Evaluates: Real-world API invocation across Python, Java, JavaScript, and REST.
- AST-Based Evaluation: Compares the Abstract Syntax Tree (AST) of the generated call
  against valid schema trees (Zero false penalties for formatting/spacing differences!).
- Multi-Turn & Parallel Calling: Evaluates 1-to-many and multi-hop sequential tools.

+-----------------------------------------------------------------------------------------+
|                  BFCL V4 Leaderboard & Throughput Matrix (2026)                         |
+-----------------------------------------------------------------------------------------+

\text{Sequential Latency} = 4 \times 1,200\text{ms} = 4,800\text{ms}

+-----------------------------------------------------------------------------------------+
|                  Sequential vs Parallel Tool Invocation Pipeline                        |
+-----------------------------------------------------------------------------------------+

SEQUENTIAL (Legacy - 4.8s Total):
[LLM Call 1] ---> [Fetch AAPL] ---> [LLM Call 2] ---> [Fetch MSFT] ---> [LLM Call 3] ...

PARALLEL (2026 Standard - 1.1s Total):
[LLM Call: Emits 4 Tool Calls in Single Array]
       |
       +-------> [Worker 1: Fetch AAPL (Promise.all)] ----+
       +-------> [Worker 2: Fetch MSFT (Promise.all)] ----+---> [Final Synthesis]
       +-------> [Worker 3: Fetch NVDA (Promise.all)] ----+
       +-------> [Worker 4: Fetch GOOG (Promise.all)] ----+

// agent/toolExecutor.ts
import OpenAI from "openai";
import { z } from "zod";
import { zodToJsonSchema } from "zod-to-json-schema";

const openai = new OpenAI();

// 1. Define Tools using Zod Schemas
const StockPriceSchema = z.object({
  ticker: z.string().describe("Stock ticker symbol (e.g. AAPL, MSFT)"),
});

const CurrencyExchangeSchema = z.object({
  from: z.string().length(3),
  to: z.string().length(3),
  amount: z.number().positive(),
});

// Tool Definitions for OpenAI API
const tools: OpenAI.ChatCompletionTool[] = [
  {
    type: "function",
    function: {
      name: "get_stock_price",
      description: "Get real-time stock price and market capitalization.",
      parameters: zodToJsonSchema(StockPriceSchema) as any,
    },
  },
  {
    type: "function",
    function: {
      name: "convert_currency",
      description: "Convert financial amounts between international currencies.",
      parameters: zodToJsonSchema(CurrencyExchangeSchema) as any,
    },
  },
];

// Mock API implementations
async function executeTool(name: string, args: any) {
  if (name === "get_stock_price") {
    // Fast mock API
    return { ticker: args.ticker, price: (Math.random() * 200 + 50).toFixed(2), currency: "USD" };
  }
  if (name === "convert_currency") {
    return { convertedAmount: (args.amount * 1.08).toFixed(2), targetCurrency: args.to };
  }
  throw new Error(`Unknown tool: ${name}`);
}

// 2. High-Speed Parallel Execution Handler
export async function handleUserPrompt(userPrompt: string) {
  const messages: OpenAI.ChatCompletionMessageParam[] = [
    { role: "system", content: "You are a financial intelligence agent. Use parallel tools when multiple assets are requested." },
    { role: "user", content: userPrompt },
  ];

  // Turn 1: LLM selects tools and parameters
  const response = await openai.chat.completions.create({
    model: "gpt-4o",
    messages,
    tools,
    tool_choice: "auto", // Allows parallel calling
  });

  const message = response.choices[0].message;
  messages.push(message);

  // If the model emitted tool calls, execute them in PARALLEL!
  if (message.tool_calls && message.tool_calls.length > 0) {
    console.log(`[Agent] Executing ${message.tool_calls.length} tool calls in PARALLEL...`);

    // Execute all tools simultaneously using Promise.all
    const toolResults = await Promise.all(
      message.tool_calls.map(async (toolCall) => {
        const args = JSON.parse(toolCall.function.arguments);
        const result = await executeTool(toolCall.function.name, args);

        return {
          role: "tool" as const,
          tool_call_id: toolCall.id,
          content: JSON.stringify(result),
        };
      })
    );

    // Append all tool responses back to conversational state
    messages.push(...toolResults);

    // Turn 2: LLM synthesizes final answer using tool data
    const finalResponse = await openai.chat.completions.create({
      model: "gpt-4o",
      messages,
    });

    return finalResponse.choices[0].message.content;
  }

  return message.content;
}

+-----------------------------------------------------------------------------------------+
|                  Common Tool-Calling Failure Modes & Defenses                           |
+-----------------------------------------------------------------------------------------+

1. SCHEMA DRIFT / TYPE ERRORS (e.g. Model sends integer instead of string)
   - Defense: Enforce runtime Zod schema validation BEFORE executing API calls.
   - If invalid, return an error tool message: "Invalid parameter 'id': expected UUID string."

2. HALLUCINATED TOOLS (Model invents tool: 'send_bitcoin_payment')
   - Defense: Strict whitelist validation; reject un-registered tools immediately.

3. CONTEXT BLOAT FROM GIANT JSON RESPONSES (API returns 5MB of raw JSON)
   - Defense: Filter API outputs with jq / JSON selectors before passing back to LLM!

       +-------------------------------------------------------------+
       |             End-to-End Latency for 4 Tool Calls (Seconds)   |
       +-------------------------------------------------------------+
 Sequential Tool Invocation (1-by-1) | ==================================== [5.20s]
 Parallel Tool Execution (Promise.all)| ======== [1.15s] (78% Latency Reduction!)
                                      +-------------------------------------+
                                      0s      1.3s    2.6s    3.9s    5.2s