Mastering Function Calling and Tool Use in OpenAI APIs

Large Language Models (LLMs) are incredibly smart, but they have a fundamental limitation: they are frozen in time. They cannot access your private database, they do not know today's weather, and they cannot execute real-time calculations. To solve this, OpenAI introduced Function Calling (also known as Tool Use).

Function calling turns an LLM into an intelligent coordinator. Instead of trying to guess an answer, the model can decide to call an external tool (a function written in your Java application, an external API, or a database query), receive the result, and use that result to formulate a highly accurate response.

Understanding the Core Concept

A common misconception is that the OpenAI API executes your code directly on its servers. This is incorrect. The model does not run any code. Instead, the interaction follows a structured, multi-step loop:

• Step 1: You send a prompt to the model, along with a list of "tools" (descriptions of your local functions written in JSON Schema format).
• Step 2: The model analyzes the prompt. If it needs a tool to answer, it responds with a structured JSON object containing the name of the tool and the arguments to pass to it.
• Step 3: Your local application intercepts this request, runs the actual Java method or API call, and gets the result.
• Step 4: Your application sends the result back to the model.
• Step 5: The model reads the result and presents a natural language response to the user.

The Function Calling Lifecycle

+--------------+                 +------------+                 +-------------+
|  User Query  | --------------> | OpenAI API | --------------> |  Your App   |
+--------------+                 +------------+                 +-------------+
                                        |                              |
                                        | (Requests Tool Execution)    |
                                        v                              |
                                 [JSON Arguments] -------------------->|
                                                                       | (Executes Java Code)
                                                                       v
                                 [Tool Result]  <----------------------|
                                        |
                                        v
+--------------+                 +------------+
| Final Answer | <-------------- | OpenAI API |
+--------------+                 +------------+
    

Step-by-Step Java Implementation

Let us build a practical Java example. Imagine we want the LLM to tell us the current inventory level of a specific product in our warehouse. Since the LLM does not have access to our database, we will provide it with a tool called getProductStock.

Step 1: Define the Local Java Method

This is the actual code running on your secure server that queries your database.

public class InventoryService {
    public static int getProductStock(String productId) {
        // In a real application, this would query your SQL database
        if ("PROD-100".equals(productId)) {
            return 42;
        } else if ("PROD-200".equals(productId)) {
            return 0;
        }
        return -1;
    }
}
    

Step 2: Define the Tool Schema

We must describe our Java method to OpenAI using a JSON Schema. This tells the model what the function does and what parameters it expects. In Java, you can construct this JSON using libraries like Jackson or Gson.

{
  "type": "function",
  "function": {
    "name": "getProductStock",
    "description": "Get the current warehouse stock level for a given product ID",
    "parameters": {
      "type": "object",
      "properties": {
        "productId": {
          "type": "string",
          "description": "The unique product identifier, e.g., PROD-100"
        }
      },
      "required": ["productId"]
    }
  }
}
    

Step 3: The Complete Execution Loop in Java

Here is how you orchestrate this entire workflow in Java. This code sends the prompt and tool definition, detects if the model wants to call the tool, executes the local Java method, and sends the result back to OpenAI.

import java.util.ArrayList;
import java.util.List;

public class OpenAIApp {

    public static void main(String[] args) {
        // 1. Initialize conversation history
        List<Message> messages = new ArrayList<>();
        messages.add(new Message("user", "How many units of PROD-100 do we have left?"));

        // 2. Define tools available to the model
        List<Tool> tools = getAvailableTools();

        // 3. First call to OpenAI
        ChatResponse response = OpenAIClient.chatCompletion(messages, tools);

        // 4. Check if the model wants to call a tool
        if (response.hasToolCalls()) {
            ToolCall toolCall = response.getToolCall();
            String functionName = toolCall.getName();
            String arguments = toolCall.getArguments();

            System.out.println("Model requested tool: " + functionName);
            System.out.println("With arguments: " + arguments);

            if ("getProductStock".equals(functionName)) {
                // Parse arguments (e.g., {"productId": "PROD-100"})
                String productId = parseProductId(arguments);
                
                // Execute local Java method
                int stock = InventoryService.getProductStock(productId);

                // Add assistant's tool call request to history
                messages.add(new Message("assistant", toolCall));

                // Add tool execution result to history
                messages.add(new Message("tool", "The stock level is: " + stock, toolCall.getId()));

                // 5. Second call to OpenAI with the tool result included
                ChatResponse finalResponse = OpenAIClient.chatCompletion(messages, tools);
                System.out.println("Final Answer: " + finalResponse.getContent());
            }
        } else {
            System.out.println("Answer: " + response.getContent());
        }
    }
}
    

Real-World Use Cases

• Live Data Retrieval: Fetching current weather, stock prices, or shipping tracking information from third-party REST APIs.
• Database Operations: Safely querying or updating user records in a relational database based on natural language commands.
• System Actions: Triggering actions like sending an email, creating a Jira ticket, or scheduling a calendar event.
• Mathematical Computations: Offloading complex math formulas to a dedicated calculator tool to avoid LLM calculation errors.

Common Mistakes and How to Avoid Them

• Poor Tool Descriptions: The model relies entirely on the description field to decide whether to call a tool. If your description is vague (e.g., "Gets data"), the model will either ignore it or call it incorrectly. Be explicit: "Retrieves the current stock level for a product using its alphanumeric ID."
• Skipping Input Validation: Treat arguments generated by the LLM as untrusted user input. Always validate and sanitize the arguments (e.g., prevent SQL injection or path traversal) before passing them to your database or system.
• Infinite Loops: If your local tool returns an error message, the model might try to call the tool again with the same arguments, creating an infinite loop. Implement a counter to limit the maximum number of tool execution retries.
• Ignoring Token Limits: Tool definitions and tool outputs consume tokens. Sending massive database payloads back to the model can quickly exceed context window limits and increase costs.

Interview Notes for Developers

• What is the purpose of tool_choice? By default, the model decides automatically whether to call a tool (tool_choice: "auto"). You can force the model to always call a specific tool by setting tool_choice: {"type": "function", "function": {"name": "my_function"}}, or disable tool use entirely by setting it to "none".
• How do you handle multiple tool calls? Modern OpenAI models support parallel tool calling. The model can request to call multiple tools at once (e.g., checking stock for three different products in a single turn). Your application should be designed to loop through and execute all requested tool calls in parallel or sequence.
• What is the role of the "tool" message role? In the conversation history, messages have roles like "user", "assistant", or "system". When returning the output of a function, you must use the "tool" role and provide the matching tool_call_id so the model knows which tool call the result corresponds to.

Summary

Function calling bridges the gap between static language generation and dynamic application logic. By describing your Java methods as tools, you empower OpenAI models to query databases, call external APIs, and execute secure calculations. Remember to write clear tool descriptions, treat LLM-generated arguments as untrusted input, and handle parallel tool calls gracefully to build robust, production-ready AI applications.