Deploying autonomous AI agents and automated API workflows requires reliable structured output generation and tool invocation. While general-purpose language models can be prompted to output JSON tool invocations, specialized function-calling fine-tunes provide higher schema compliance, lower parameter hallucination rates, and support for parallel tool invocation.
Four open-source model families have emerged as prominent architectures for local and private function calling: Gorilla OpenFunctions-v2, Nous Research Hermes 3, MeetKai Functionary, and Salesforce xLAM. These models differ across training pipelines, serialization protocols, multi-turn state tracking, and serving parser compatibility.
The Production Function-Calling Problem
Function calling requires an autoregressive language model to solve multiple simultaneous constraints:
- Tool Selection: Identifying the correct function name from a set of N candidate tools based on user intent.
- Schema Parsing: Generating valid JSON or Python AST argument trees matching strict typed signatures.
- Hallucination Prevention: Deciding when not to invoke any tool if none are relevant.
- Parallel Execution: Emitting multiple independent tool calls in a single generation step.
- Multi-Turn Execution: Incorporating tool execution results back into context and continuing multi-step problem solving.
Ad-hoc prompt engineering on generic instruction-tuned models frequently fails in high-throughput environments due to syntax errors, unescaped string delimiters, and hallucinated parameter keys. Function-calling models address this by baking specialized serialization schemas into supervised fine-tuning (SFT) and reinforcement learning datasets.
Architectural Paradigms and Serialization Schemas
Each open-source model family implements a distinct token serialization protocol to represent tool definitions and generated invocations:
1. Gorilla OpenFunctions-v2
Developed by UC Berkeley and the Gorilla LLM project, Gorilla OpenFunctions-v2 is trained on executable API definitions across multiple programming languages, including Python, Java, JavaScript, and REST APIs.
OpenFunctions-v2 formats function calls using a Python-like abstract syntax tree (AST) call structure or standard JSON objects. By learning direct argument mapping over diverse code signatures, the model generalizes across polyglot backend endpoints.
2. Nous Research Hermes 3
The Hermes 3 series (built on Llama 3.1 in 8B, 70B, and 405B sizes) utilizes ChatML formatting with explicit XML wrapping tags. Tool definitions are supplied in system messages, and invocations are emitted using <tool_call> containers:
<tool_call>
{"name": "fetch_user_record", "arguments": {"user_id": 1042, "include_history": true}}
</tool_call>When receiving execution outputs, Hermes expects <tool_response> tags containing the returned payload. This structured enclosure makes regex parsing and streaming token extraction straightforward in production gateways.
3. MeetKai Functionary
MeetKai designed Functionary (available across Small and Medium sizes based on Llama architectures) to mimic OpenAI function-calling mechanics natively. Functionary uses custom special tokens such as <|start_header_id|> and dedicated plugin call tokens rather than generic prompt wrappers.
By separating function names and arguments into distinct generation phases, Functionary allows serving engines to stream arguments dynamically to downstream HTTP clients while validating parameters in real time.
4. Salesforce xLAM (Large Action Models)
Salesforce released the xLAM family (including xLAM-1B-FC, xLAM-7B-FC, and xLAM-8x22B-MoE), trained on the APIGen dataset of 60,000 verified multi-domain function-calling pairs.
xLAM enforces JSON schema compliance with strict parameter typing, optimizing small parameter footprints (1B and 7B) for high-concurrency enterprise pipelines.
Evaluation on the Berkeley Function-Calling Leaderboard (BFCL)
The Berkeley Function-Calling Leaderboard (BFCL) provides standard benchmarking across AST correctness, parameter type accuracy, multi-turn state retention, and relevance filtering:
- Simple AST: Measures exact signature matching on single function invocations.
- Multiple and Parallel Calling: Evaluates the ability to select among multiple candidate tools and emit batched invocations simultaneously.
- Irrelevant / No-Tool Detection: Measures whether the model correctly avoids calling functions when user prompts do not require tool use.
- Multi-Turn Execution: Tests state persistence and intermediate result ingestion across sequential action loops.
On BFCL evaluations, xLAM-7B achieves an overall accuracy of 88.24%, while Gorilla OpenFunctions-v2 reaches 84.71% overall accuracy. Hermes 3 70B and Functionary Medium models maintain over 80% to 85% accuracy across parallel and multi-step evaluation suites, outperforming earlier baseline models such as GPT-3.5 Turbo.
Serving Infrastructure and Engine Parsers
Integrating function-calling models into production inference clusters requires native parser support in engines like vLLM and SGLang.
vLLM provides built-in tool call parsers configured via CLI arguments:
- Hermes models:
--tool-call-parser hermesautomatically parses<tool_call>XML blocks into OpenAI-compatibletool_callsresponse structures. - Llama 3.1 / 3.2 models:
--tool-call-parser llama3_jsonhandles JSON formatted tool blocks. - Mistral series:
--tool-call-parser mistralprocesses[TOOL_CALLS]token syntax. - Custom parsers: Serving operators can register custom parser plugins for Functionary or proprietary schemas.
When serving with constrained decoding engines such as XGrammar, Outlines, or llguidance, schema enforcement can be combined with model generation to guarantee valid JSON syntax at the token masking level.
Model Selection Guidelines for Production
Choosing between these open-source tool-use models depends on deployment constraints:
- Low-Latency Edge and Microservice Deployments: xLAM-1B-FC and xLAM-7B-FC offer high parameter efficiency and low memory footprints for high-concurrency single-turn API routing.
- Multi-Step Agentic Loops and Complex Reasoning: Hermes 3 (70B / 405B) provides comprehensive multi-turn conversational reasoning, tool handling, and system prompt steering.
- Direct OpenAI API Drop-In Compatibility: MeetKai Functionary aligns closely with OpenAI client SDK schemas and streaming behaviors.
- Polyglot Backend Services: Gorilla OpenFunctions-v2 offers broad cross-language support for codebases spanning Python, Java, JavaScript, and REST endpoints.
