Best open models for tool calling in November 2026: benchmark scores, parser flags and hardware fit

As of 7 November 2026, the open models that call tools most reliably on independent tests are GLM-5.3 (50.3% on Artificial Analysis's tau3-Banking), Qwen3.8 2.4T A95B (49.1%) and Qwen3.8-27B (48.0%); the 27B runs on a 32 GB GPU at 4 bits. On Sierra's older three-domain tau2-bench, Qwen3.5-397B-A17B averages 87.9%, ahead of every closed model with a full three-domain run. Whichever you pick, the serving engine must use the right parser: in vLLM 0.33.0 that means --enable-auto-tool-choice plus the family's --tool-call-parser, such as qwen3_coder for Qwen3.8.

Pass^1 in percent unless noted. AA = Artificial Analysis per-evaluation data, read 7 November 2026; its tau2-bench scores come from earlier index versions, which is why only older models have them. Sierra runs from taubench.com: tau2 rows dated 26 February to 2 March 2026 (gpt-5.2 user simulator, 4 trials), tau3-Banking rows dated 4 August 2026. MCPMark from its verified board, updated 20 July 2026. Sizes are Hugging Face file totals; the Qwen3.6-35B-A3B size is from our Qwen3.6-35B-A3B page.

Which benchmarks measure tool calling, and how current are they?

tau-bench is the one with fresh open-model results; BFCL and MCPMark are useful mainly for older models. They test different things.

• tau-bench (Sierra) simulates a customer and scores whether the agent completes a support task through tools while following a policy: retail, airline and telecom in tau2, and a knowledge-retrieval banking domain in tau3. Sierra runs some models itself and accepts vendor submissions, which it flags as unverified. Artificial Analysis runs tau3-Banking itself for many current models, so where the AA column has a value it is the newest like-for-like comparison. Kimi K3.1 Pro, DeepSeek V4.1 Flash 1017 and MiniMax-M3.1-Flash have no AA tau3 score yet.
• BFCL V4 checks whether calls are well-formed and correctly chosen across single, parallel, multi-turn, web-search and memory tasks. Its page says "Last Updated: 2026-04-12", and the best open entry is GLM-4.6 (FC, thinking) at 72.38% overall against 77.47% for Claude Opus 4.5. Kimi-K2-Instruct scores 59.06% and Qwen3-235B-A22B-Instruct-2507 52.15%. No 2026 open release is on it.
• MCPMark runs 127 tasks against real MCP servers (filesystem, GitHub, Notion, Playwright, Postgres). On the board updated 20 July 2026, kimi-k3-max scores 96.06% and leads gpt-5-5-xhigh at 92.91%, but the site's own data does not mark it as an open model, and the board predates the autumn releases.

The striking result is how well small models do on tau3-Banking. Qwen3.8-27B at 48.0% sits 2.3 points behind GLM-5.3, a model 28 times its size, and ahead of the closed Qwen3.8 Max (0902) at 47.8% in AA's runs. My reading is that tool use rewards following a format and a policy more than raw knowledge; the gap on coding benchmarks is much larger (see the coding model rankings).

How reliable are the models across repeated attempts?

Much less reliable than pass^1 suggests: on tau-bench, the share of tasks solved on all four tries is 36% to 71% of the single-try score. Sierra reports pass^k, the share of tasks a model solves in every one of k independent trials. From its submission files, read 7 November 2026:

Sierra-run submissions on taubench.com; the last column is my division. Kimi K3 and GLM-5.2 tie at pass^1 but separate at pass^4.

For a production agent, pass^4 is the number to plan around. Two models with the same headline score can differ by almost a quarter in how often they get the same task right every time, and only repeated runs on your own tasks will show which one you have.

Which parser flags does each model family need?

Each family writes tool calls in its own format, so the engine needs the matching parser, and most current families also need a matching reasoning parser. The table lists what the model's card or vLLM's recipe uses, checked 7 November 2026 against vLLM 0.33.0 (released 2 November) and SGLang v0.5.23 (2 November).

vLLM always also needs --enable-auto-tool-choice. SGLang parser names are from its tool parser docs and the parser registry in its source; blank cells mean I found no named parser for that engine.

A Qwen3.8-27B launch that matches the recipe looks like this:

vllm serve Qwen/Qwen3.8-27B-FP8 \
  --enable-auto-tool-choice \
  --tool-call-parser qwen3_coder \
  --reasoning-parser qwen3 \
  --max-model-len 65536

vLLM's docs add one setting worth turning on: without a schema constraint, "arguments may occasionally be malformed" or break the parameter schema. Set "strict": true on a tool's function definition, or start the server with --tool-strict-level function, and vLLM enforces the argument schema with a structural-tag grammar. The glm47 parser applies that level by default. For DGX Spark-specific install steps, see the vLLM on DGX Spark guide.

What about llama.cpp and Ollama?

Both work, with less per-model configuration and less control.

llama.cpp (latest build b12189, 7 November 2026) parses tool calls from the model's Jinja chat template. The function-calling doc still says to start llama-server with --jinja, but the current server README lists it as enabled by default. Native handlers exist for Llama 3.x, Hermes 2/3 and Qwen 2.5, Mistral Nemo, Functionary, Command R7B, DeepSeek R1 and gpt-oss (Harmony), among others; any other template falls back to a generic handler, which the doc says may use more tokens. Two cautions from the same page: parallel tool calls are off unless the request sets "parallel_tool_calls": true, and extreme KV quantization such as -ctk q4_0 can substantially degrade tool calling. The site's llama.cpp guide covers the build for a Spark.

Ollama (v0.40.15, 3 November 2026) takes a tools array on /api/chat and handles the format inside the model's template, so there is no parser flag to set. Its tool calling docs cover parallel calls and a multi-turn agent loop, and for streaming they say to collect thinking, content and tool calls and send them all back with the tool results. That works well for models with a clean template in the Ollama library; for a release from last week, vLLM or SGLang with the vendor's parser is the safer route.

What goes wrong with tool calling, and how do you fix it?

Most failures are configuration mismatches, not model limits:

• Calls arrive as plain text. The parser does not match the model's format, or --enable-auto-tool-choice is missing. Use the parser from the recipe table above.
• Calls vanish inside the thinking block. No reasoning parser, or the wrong one, so the call stays in reasoning text. Set the family's --reasoning-parser.
• Multi-turn agents lose the plot. Kimi K3.1 Pro's card requires each assistant message to be sent back complete, including reasoning_content and tool_calls. Frameworks that keep only content break this.
• No chat template. DeepSeek-V4.1-Flash-1017 ships without a Jinja template, so engines that rely on one (llama.cpp, plain Transformers) need DeepSeek's own encoder; vLLM handles it with --tokenizer-mode deepseek_v41.
• Empty content with gpt-oss on SGLang. SGLang's docs say its gpt-oss parser drops the analysis channel, so explanatory text can come back empty; return tool results as role="tool" to complete the round.
• Valid JSON, wrong values. A grammar enforces shape, not meaning. The structured output guide covers testing against your own schemas.

Which tool-calling model fits your hardware?

For most self-hosters the answer is Qwen3.8-27B, because it is near the top of the independent results and small enough to run anywhere.

• 32 GB GPU: Qwen3.8-27B at 4 bits (17.6 GB, Apache 2.0). Keep reasoning at medium or above: AA measured 47.4% at Medium and 32.2% at Low.
• One 128 GB Spark: Qwen3.8-27B in FP8 (30.9 GB) with room for long agent traces, or GLM-5.3-Flash at 2 bits if you also need its coding strength. gpt-oss-120b (65.2 GB) is the mature option, with native tool-call support in vLLM, SGLang, llama.cpp and Ollama, but scores lower on these tests.
• Two Sparks: GLM-5.3-Flash in NVIDIA's 204.4 GB NVFP4 build.
• GPU server: GLM-5.3, the top open model on tau3-Banking.

My recommendation: start with Qwen3.8-27B on vLLM using the recipe flags, set strict on every tool, and measure pass^4 on twenty of your own tasks before trusting it. Where it ranks overall is in the open-weight leaderboard snapshot. A Spark to run that test on rents at $0.79/hour, billed per minute (pricing).

FAQ

Sources

Every figure above is attributed to its source in the sentence or table where it appears. Pages were read on 7 November 2026 unless a date says otherwise.

• LLM Leaderboard (per-evaluation scores, tau3-Banking and tau2-bench), Artificial Analysis
• tau-bench: Benchmarking AI Agents on Real-World Tasks, Sierra
• Berkeley Function Calling Leaderboard (BFCL) V4, UC Berkeley Gorilla, 2026-04-12
• MCPMark Verified Leaderboard, MCPMark, 2026-07-20
• Tool Calling, vLLM documentation
• vLLM releases (v0.33.0), GitHub, 2026-11-02
• vLLM recipe: Qwen/Qwen3.8-27B, vLLM Recipes
• vLLM recipe: zai-org/GLM-5.3-Flash, vLLM Recipes
• vLLM recipe: deepseek-ai/DeepSeek-V4.1-Flash-1017, vLLM Recipes
• vLLM recipe: moonshotai/Kimi-K3.1-Pro, vLLM Recipes
• Tool Parser, SGLang documentation
• SGLang releases (v0.5.23), GitHub, 2026-11-02
• llama.cpp function-calling.md, ggml-org on GitHub, 2026-10-09
• llama.cpp releases (b12189), GitHub, 2026-11-07
• Tool calling, Ollama documentation
• Ollama releases (v0.40.15), GitHub, 2026-11-03
• nvidia/NVIDIA-Nemotron-3.5-Super-VL-120B-A12B-BF16 model card, Hugging Face, 2026-10-22
• moonshotai/Kimi-K3.1-Pro model card, Hugging Face, 2026-11-05