Function Calling Patterns Across LLM Providers

What Is Function Calling

Function calling lets an LLM decide when to invoke external tools and what arguments to pass. Instead of generating a text response, the model outputs a structured function call that your code executes. You then feed the result back to the model, which uses it to generate a final answer.

This pattern is the foundation of LLM agents, RAG pipelines, and any application where the model needs to interact with the real world: querying databases, calling APIs, running calculations, or controlling software.

How It Works Across Providers

The concept is the same everywhere. You define functions with JSON schemas, send them alongside your prompt, and the model either responds with text or requests a function call. The differences are in the API structure.

OpenAI

OpenAI calls them “tools” with type “function”.

from openai import OpenAI

client = OpenAI()

tools = [
    {
        "type": "function",
        "function": {
            "name": "get_weather",
            "description": "Get the current weather for a city",
            "parameters": {
                "type": "object",
                "properties": {
                    "city": {
                        "type": "string",
                        "description": "City name, e.g. San Francisco",
                    },
                    "units": {
                        "type": "string",
                        "enum": ["celsius", "fahrenheit"],
                        "description": "Temperature unit",
                    },
                },
                "required": ["city"],
            },
        },
    }
]

response = client.chat.completions.create(
    model="gpt-4o",
    messages=[{"role": "user", "content": "What's the weather in Tokyo?"}],
    tools=tools,
)

# Check if the model wants to call a function
message = response.choices[0].message
if message.tool_calls:
    tool_call = message.tool_calls[0]
    print(f"Function: {tool_call.function.name}")
    print(f"Arguments: {tool_call.function.arguments}")

Anthropic

Anthropic uses a tools parameter with input_schema instead of parameters.

import anthropic

client = anthropic.Anthropic()

tools = [
    {
        "name": "get_weather",
        "description": "Get the current weather for a city",
        "input_schema": {
            "type": "object",
            "properties": {
                "city": {
                    "type": "string",
                    "description": "City name, e.g. San Francisco",
                },
                "units": {
                    "type": "string",
                    "enum": ["celsius", "fahrenheit"],
                    "description": "Temperature unit",
                },
            },
            "required": ["city"],
        },
    }
]

response = client.messages.create(
    model="claude-sonnet-4-20250514",
    max_tokens=1024,
    tools=tools,
    messages=[{"role": "user", "content": "What's the weather in Tokyo?"}],
)

# Anthropic returns tool_use blocks in the content array
for block in response.content:
    if block.type == "tool_use":
        print(f"Function: {block.name}")
        print(f"Arguments: {block.input}")
        print(f"Tool use ID: {block.id}")

Google (Gemini)

Google uses function_declarations within a tools parameter.

import google.generativeai as genai

get_weather = genai.protos.FunctionDeclaration(
    name="get_weather",
    description="Get the current weather for a city",
    parameters=genai.protos.Schema(
        type=genai.protos.Type.OBJECT,
        properties={
            "city": genai.protos.Schema(type=genai.protos.Type.STRING),
            "units": genai.protos.Schema(
                type=genai.protos.Type.STRING,
                enum=["celsius", "fahrenheit"],
            ),
        },
        required=["city"],
    ),
)

model = genai.GenerativeModel(
    "gemini-2.0-flash",
    tools=[genai.protos.Tool(function_declarations=[get_weather])],
)

response = model.generate_content("What's the weather in Tokyo?")

The Tool Use Loop

In all providers, function calling requires a loop: send the message, check for tool calls, execute them, feed results back, and repeat until the model responds with text.

import json

def run_tool_loop(client, messages, tools, available_functions):
    """Generic tool use loop for OpenAI."""
    while True:
        response = client.chat.completions.create(
            model="gpt-4o",
            messages=messages,
            tools=tools,
        )

        message = response.choices[0].message
        messages.append(message)

        if not message.tool_calls:
            return message.content

        for tool_call in message.tool_calls:
            fn_name = tool_call.function.name
            fn_args = json.loads(tool_call.function.arguments)

            # Execute the function
            fn = available_functions[fn_name]
            result = fn(**fn_args)

            # Append the result
            messages.append({
                "role": "tool",
                "tool_call_id": tool_call.id,
                "content": json.dumps(result),
            })

# Usage
available_functions = {
    "get_weather": get_weather_impl,
    "get_forecast": get_forecast_impl,
}

messages = [{"role": "user", "content": "Compare weather in Tokyo and London"}]
answer = run_tool_loop(client, messages, tools, available_functions)
print(answer)

Parallel Function Calls

When the model needs multiple pieces of information, it can request several function calls at once. OpenAI returns multiple items in tool_calls. Anthropic returns multiple tool_use blocks.

# OpenAI parallel calls - the model might return two tool_calls at once
# for "Compare weather in Tokyo and London"

message = response.choices[0].message
if message.tool_calls:
    # Execute all calls in parallel
    import concurrent.futures

    def execute_call(tool_call):
        fn_name = tool_call.function.name
        fn_args = json.loads(tool_call.function.arguments)
        fn = available_functions[fn_name]
        result = fn(**fn_args)
        return tool_call.id, result

    with concurrent.futures.ThreadPoolExecutor() as executor:
        futures = [executor.submit(execute_call, tc) for tc in message.tool_calls]
        for future in concurrent.futures.as_completed(futures):
            call_id, result = future.result()
            messages.append({
                "role": "tool",
                "tool_call_id": call_id,
                "content": json.dumps(result),
            })

Designing Good Function Schemas

The schema is the contract between your code and the model. Clear schemas produce reliable function calls.

Use Descriptive Names and Descriptions

# Bad - vague name, no description
{"name": "query", "parameters": {"type": "object", "properties": {"q": {"type": "string"}}}}

# Good - clear name, helpful description
{
    "name": "search_products",
    "description": "Search the product catalog by keyword. Returns up to 10 matching products with name, price, and availability.",
    "parameters": {
        "type": "object",
        "properties": {
            "query": {
                "type": "string",
                "description": "Search keywords, e.g. 'wireless headphones under $50'",
            },
            "category": {
                "type": "string",
                "enum": ["electronics", "clothing", "home", "sports"],
                "description": "Filter results by product category",
            },
            "max_results": {
                "type": "integer",
                "description": "Maximum number of results to return (1-10)",
                "default": 5,
            },
        },
        "required": ["query"],
    },
}

Use Enums for Constrained Values

Enums prevent the model from inventing invalid values. If a parameter can only be one of a few options, list them explicitly.

Keep Schemas Focused

Each function should do one thing. A function that queries a database, formats results, and sends an email is hard for the model to use correctly. Split it into three functions.

Error Handling

Functions can fail. Network errors, invalid arguments, rate limits. Your tool loop needs to handle these gracefully.

def safe_execute(fn, fn_args):
    """Execute a function with error handling."""
    try:
        result = fn(**fn_args)
        return {"status": "success", "data": result}
    except ValueError as e:
        return {"status": "error", "message": f"Invalid input: {str(e)}"}
    except TimeoutError:
        return {"status": "error", "message": "Request timed out, please try again"}
    except Exception as e:
        return {"status": "error", "message": f"Unexpected error: {str(e)}"}

When you return an error to the model, it can often recover by adjusting its arguments or trying a different approach. Including a clear error message helps the model self-correct.

Forcing Function Calls

Sometimes you want the model to always call a specific function rather than responding with text.

# OpenAI - force a specific function
response = client.chat.completions.create(
    model="gpt-4o",
    messages=messages,
    tools=tools,
    tool_choice={"type": "function", "function": {"name": "search_products"}},
)

# Anthropic - force tool use
response = client.messages.create(
    model="claude-sonnet-4-20250514",
    messages=messages,
    tools=tools,
    tool_choice={"type": "tool", "name": "search_products"},
)

Use tool_choice="auto" (the default) to let the model decide. Use tool_choice="none" to prevent function calls entirely.

Multi-Step Chains

Complex tasks require multiple function calls in sequence, where the output of one call feeds into the next.

# The model might:
# 1. Call search_products("wireless headphones")
# 2. Call get_product_reviews(product_id=123)
# 3. Call check_inventory(product_id=123, warehouse="west")
# 4. Generate a final recommendation

# The tool loop handles this naturally - each iteration
# processes one round of function calls and feeds results back

The key insight is that you do not need to orchestrate this. The model decides the sequence based on the task and available functions. Your tool loop just keeps running until the model responds with text.

Cost Considerations

Function schemas count toward your input token usage. Each function definition adds tokens to every request. With 20 functions, each with detailed schemas, you might add 2000-3000 tokens per request.

Strategies to reduce cost:

• Only include functions relevant to the current task
• Use concise but clear descriptions
• Group related operations into fewer functions with a mode parameter
• Cache responses for repeated identical function calls

Summary

Function calling follows the same pattern across providers: define schemas, send them with the prompt, execute returned calls, and feed results back. OpenAI uses tools with parameters, Anthropic uses tools with input_schema, and Google uses function_declarations. Design schemas with clear names, descriptions, and enums. Handle errors gracefully so the model can self-correct. Use parallel execution when the model requests multiple calls. Keep your tool loop running until the model produces a text response.