Every AI agent, chatbot that books a flight, and coding assistant that edits a file rests on one unglamorous mechanism: function calling. The term sounds like the model is executing software. It is not. A language model can only produce text. Function calling is a formal convention that turns a specific kind of text, a JSON object naming a function and its arguments, into an instruction your application can act on.
That distinction is the whole idea. The model proposes. Your code executes.
The handshake, step by step
The mechanics, as documented in OpenAI's function calling guide, work like this:
- You send the model a normal prompt, plus a list of tool definitions. Each definition is a small JSON record with a
name, adescription, and aparametersfield containing a JSON Schema that describes the arguments the function accepts. - The model reads the conversation and the catalog. If it decides a tool is needed, its reply is not prose. It is a structured call: the function name plus a JSON object of arguments.
- Your application actually makes the call. The API provider runs nothing. Your server queries the weather service, the database, or the ticketing system.
- You send the result back to the model as a new message. The model folds it into the conversation and answers the user, or asks for another tool.
Repeat step four until the model stops requesting tools, and you have an agent loop. The model is the scheduler; your code is the hand.
A concrete definition, in the shape OpenAI's documentation uses:
{
"type": "function",
"function": {
"name": "get_weather",
"description": "Get current weather for a location",
"parameters": {
"type": "object",
"properties": {
"location": { "type": "string" },
"unit": { "type": "string", "enum": ["celsius", "fahrenheit"] }
},
"required": ["location"]
}
}
}
Anthropic's tool use documentation describes the same handshake with a flatter envelope: a tool has name, description, and an input_schema whose top-level type must be an object. The model returns a tool_use block and the API run ends with a tool_use stop reason until you feed back a tool_result. Google's Gemini API and every major open-source serving engine, vLLM and SGLang among them, expose equivalents. The wire formats differ; the contract is identical.
The model was never executing anything
Before June 2023, developers who wanted structured output from a model did it by prompt engineering: beg for JSON, parse what comes back, retry when the model wraps it in markdown or adds a polite preamble. When OpenAI shipped function calling on June 13, 2023 alongside gpt-4-0613 and gpt-3.5-turbo-0613, it was fine-tuned into the models: the system prompt was augmented with the function descriptions, and the model's output was validated against the schema before being handed back.
The important nuance, which the name obscures: "function calling" is a misnomer at the API layer. The model generates arguments. Calling is your job. This is why the security properties of an agent are entirely up to the application. The model can be instructed by text it did not write, and there is no architectural firewall between a malicious document and a persuasive tool request. That failure mode, prompt injection, is a separate explainer; the point here is that function calling gives a model reach, and reach needs guardrails you build yourself.
Two related misconceptions are worth killing:
- The model does not see the function's code, only its description. The
descriptionfield is prompt, not documentation. Models pick tools by reading it, so vague descriptions produce wrong tool choices. - Nothing has been verified until it executes. A schema-valid call can still be wrong: right format, wrong city, wrong ticket ID. The API can guarantee grammar. Only your system can guarantee meaning.
Schema enforcement: from "usually" to "always"

The original weakness was that models could still violate the schema they were asked to follow, especially on nested or exotic definitions. OpenAI's structured outputs announcement on August 6, 2024 put numbers on the gap. On internal evals of complex JSON schema following, gpt-4o-2024-08-06 with strict mode scored a perfect 100%, while gpt-4-0613 scored less than 40%. Strict mode works by constraining decoding: the token sampler is limited to tokens that keep the output parseable against the supplied JSON Schema, so invalid structure becomes impossible rather than unlikely.
Anthropic added a comparable strict: true option to tool definitions, with a narrower supported schema subset when enabled. In practice this means the argument layer of tool calling is a solved problem on the frontier hosted APIs. What was not solved is everything above it: choosing the right tool, passing correct values, and knowing when not to call anything.
How anyone knows this works: BFCL
The standard yardstick is the Berkeley Function-Calling Leaderboard, built by UC Berkeley's Gorilla team. The BFCL paper presented at ICML 2025 describes the method: given a user request and a set of typed function schemas, a model must emit the right call, and BFCL grades it two ways. Abstract Syntax Tree (AST) matching parses the generated call and checks the function name, parameters, values, and types against ground truth without executing anything. Executable evaluation actually runs the call against real or simulated APIs and compares return values.
The benchmark has grown in lockstep with the field: v1 covered simple, multiple, and parallel calls; v2 added enterprise and open-source contributed functions; v3 added multi-turn, multi-step interactions; v4 covers holistic agentic settings including web search, memory management, and format sensitivity. The authors' own conclusion after evaluating a wide range of models is the honest summary of the state of tool calling in 2026: state-of-the-art models excel at single-turn calls, while memory, dynamic decision-making, and long-horizon reasoning remain open challenges.
That matches how the technology behaves in production. Filling in get_weather(location="Paris", unit="celsius") is reliable. Choosing correctly among 200 tools, sequencing four calls with data dependencies, and noticing that the fifth call should not happen because the third one failed: that is where agents still stumble, and why the boring loop-and-check code around the model is where most agent engineering actually happens.
Why it matters
Three consequences of this small API convention deserve emphasis.
First, function calling is the substrate of the entire agent economy. Computer use, Model Context Protocol servers, browser automation, coding assistants that run tests: all of them are function calling with a bigger vocabulary of tools. The 2026 tool ecosystem standardized on top of this handshake rather than inventing a new one.
Second, the interface shape explains most tool-design advice. Because the model only sees names and descriptions, tool design is prompt design. A tool named exec that does six things will be used worse than three tools that each do one.
Third, the trust boundary never moves. No matter how capable the model, the decision about what a call is allowed to do lives in your code. Models have gotten dramatically better at proposing actions since June 2023. The part where a human-written program decides whether to carry one out has not changed at all, and it is the part that keeps systems safe.
Sources
- OpenAI API documentation: Function calling
- OpenAI: Function calling and other API updates (June 13, 2023)
- OpenAI: Introducing Structured Outputs in the API (August 6, 2024)
- Anthropic: Tool use overview
- Berkeley Function Calling Leaderboard (BFCL) V4
- Patil et al., The Berkeley Function Calling Leaderboard, ICML 2025
- Gorilla repository, UC Berkeley
