Function calling and tool use

The big idea

A user asks your assistant "Do I need an umbrella in Oslo today?". The model was trained months ago and has no idea what the sky over Oslo looks like right now. It cannot browse, it cannot call an API, it can only produce text. But you can tell it: "if you need the weather, ask me for it in this exact format, and I will look it up".

That is function callingA model feature where the model, instead of answering in prose, returns a structured request to call one of the functions you described, with arguments as JSON. Also called tool use.. You describe each tool with a name, a purpose and a parameter schema. The model decides whether a tool would help and, if so, replies with a structured call: the tool name and JSON arguments. Your code runs the tool and sends the result back. The model then continues, perhaps calling more tools, until it can answer.

The key point: the model never executes anything. Think of a manager with a phone and an assistant. The manager says "call the weather service and ask about Oslo". The assistant decides whether that call is allowed, makes it, and reads back the answer. The manager only ever talks. Your code is the assistant, and it holds all the keys.

How it works

Step 1: Describe the tools

A tool schemaThe description of a tool given to the model: a name, a plain-language purpose, and a JSON Schema for its parameters. has three parts: a name (get_weather), a description that says what the tool does and when to use it, and a JSON Schema for its parameters (city, a required string). The model sees only this text. It never sees your code, so the description is the whole interface: a vague one gets the tool called at the wrong times or with the wrong arguments.

Tool definitions are sent with every request and count as input tokens, so dozens of long tool descriptions cost money and attention on every turn.

Step 2: The model proposes a call

On each request the model decides between answering in text and calling a tool. If it calls one, the reply contains one or more tool callsA structured request from the model to run a named tool with JSON arguments, carrying an id that the matching result must refer to., each with an id, the tool name and arguments as JSON, and the reply is marked as stopped for a tool call. Models are trained to produce these calls, and many APIs can enforce the schema, but treat the arguments like any other model output: parse and validate them before use.

Step 3: Your code executes it

Your code looks up the function by name, validates the arguments, checks the action is allowed for this user, and runs it. This is where every safety decision lives. A model can be talked into proposing a harmful call, for example by instructions hidden in a web page it read ( Prompt injection and data exfiltration ), so dangerous actions such as payments or deletes need permission checks or a human approval step, never the model's say-so.

Step 4: Return the result

Append the result to the conversation as a tool result that carries the same call id, then call the model again with the full history: the question, the model's tool call and the result. The model is stateless ( Context engineering ), so it only "remembers" the call because you sent it back. If the tool failed, return the error as the result, with a message that helps the model recover, instead of crashing.

Step 5: Parallel calls and repeats

One reply can contain several calls. "Is it warmer in Oslo or Rome?" needs two independent lookups, so the model can emit both at once. These are parallel tool callsSeveral tool calls returned by the model in a single reply, which your code can run concurrently before returning all the results together.. Run them concurrently if they are independent, then return one result per call id, together in the next request. The model may also call tools again after seeing results, which is the start of the agent loop in The agent loop (ReAct) .

Try it

1. With "one tool call", press Step until step 3. What did the model send, and has anything run yet? Count the model calls by the end.
2. Switch to "parallel calls" and step to the tool results. How many results go back, how are they matched to their calls, and how many model calls did the whole question take?
3. Switch to "tool error". What does your app send the model at step 6, and how would the answer change if the app had crashed or sent an empty result?
4. Try "no tool needed". The tool is offered every time; who decides it is not needed?

The same loop in code, with a scripted stand-in for the model so it runs without an API key:

# The tool-calling loop with a scripted stand-in for the model (no API key needed).
# The "model" only proposes calls; this code validates and runs them, then sends results back.
import json
TOOLS = [{"name": "get_weather", "description": "Current weather for a city. Use for questions about today's weather.",
          "parameters": {"type": "object", "properties": {"city": {"type": "string"}}, "required": ["city"]}}]
FAKE_WEATHER = {"Oslo": {"temp_c": 6, "rain_mm": 4.2}, "Rome": {"temp_c": 21, "rain_mm": 0.0}}
def get_weather(city):
    if city not in FAKE_WEATHER:
        raise ValueError(f"unknown city {city!r}; pass a city name such as 'Oslo'")
    return FAKE_WEATHER[city]
def fake_model(messages, tools):
    last = messages[-1]
    if last["role"] == "user":  # decide which tools to call (a real model reads the schemas)
        cities = [c for c in ("Oslo", "Rome", "Atlantis") if c in last["content"]]
        return {"role": "assistant", "tool_calls": [
            {"id": f"call_{i}", "name": "get_weather", "arguments": json.dumps({"city": c})}
            for i, c in enumerate(cities, 1)]}
    results = [m["content"] for m in messages if m["role"] == "tool"]
    return {"role": "assistant", "content": "Answer written from: " + "; ".join(results)}
def run(question, max_steps=5):
    messages = [{"role": "user", "content": question}]
    for _ in range(max_steps):
        reply = fake_model(messages, TOOLS)
        messages.append(reply)
        if not reply.get("tool_calls"):
            return reply["content"]
        for call in reply["tool_calls"]:  # parallel calls: run each, one result per call id
            args = json.loads(call["arguments"])
            try:
                out, err = json.dumps(get_weather(**args)), False
            except (ValueError, TypeError) as e:
                out, err = f"error: {e}", True
            print(f"  ran {call['name']}({args}) -> {out}")
            messages.append({"role": "tool", "tool_call_id": call["id"], "content": out, "is_error": err})
    return "stopped: too many steps"
for q in ["Umbrella in Oslo today?", "Compare Oslo and Rome.", "Weather in Atlantis?"]:
    print(q); print("  final:", run(q))

Umbrella in Oslo today?
  ran get_weather({'city': 'Oslo'}) -> {"temp_c": 6, "rain_mm": 4.2}
  final: Answer written from: {"temp_c": 6, "rain_mm": 4.2}
Compare Oslo and Rome.
  ran get_weather({'city': 'Oslo'}) -> {"temp_c": 6, "rain_mm": 4.2}
  ran get_weather({'city': 'Rome'}) -> {"temp_c": 21, "rain_mm": 0.0}
  final: Answer written from: {"temp_c": 6, "rain_mm": 4.2}; {"temp_c": 21, "rain_mm": 0.0}
Weather in Atlantis?
  ran get_weather({'city': 'Atlantis'}) -> error: unknown city 'Atlantis'; pass a city name such as 'Oslo'
  final: Answer written from: error: unknown city 'Atlantis'; pass a city name such as 'Oslo'

Everything the model does is return a dictionary. Your code parses the arguments, runs the function, turns the failure for "Atlantis" into an error result instead of an exception, and stops after max_steps no matter what.

Trade-offs

Common mistakes

• Believing the model "ran" the tool. It only asked; if your code did not execute it, nothing happened.
• Executing arguments without validation or permission checks, especially for actions that write, pay or delete.
• Forgetting to send the model's tool call back in the history, or returning a result with the wrong call id.
• Raising an exception on tool failure instead of returning an error result the model can read and recover from.
• Vague names and descriptions, such as do_stuff(data), so the model guesses when and how to call it.

In the wild

Where you will see this: every chat assistant that searches the web, reads your calendar or runs code is doing function calling under the hood, as are customer-support bots that look up an order or open a ticket. Coding assistants call tools to read files, edit them and run tests. Natural-language-to-SQL features call a query tool with generated SQL. And the Model Context Protocol ( Model Context Protocol (MCP) ) is a standard way to package tools so any compatible app can offer them to a model.

Check yourself

Recap