At Watt AI Lab, under Shanda Group, I led the tool-calling roadmap and post-training work connecting open-weight models to real application workflows. Our team released watt-tool-70B and watt-tool-8B, fully fine-tuned from Llama 3.3-70B-Instruct and Llama 3.1-8B-Instruct for function calling and multi-turn tool use.
On 29 December 2024, the 70B model reached No. 1 on the Berkeley Function-Calling Leaderboard V3, with the 8B model at No. 4. I shared the result and our training approach in a LinkedIn announcement. This was a team achievement; my role was to lead the tool-calling roadmap and model post-training.
BFCL V3 results
First place, 29 December 2024
Last archived first-place snapshot, 27 February 2025
| Date | Milestone |
|---|---|
| 29 Dec 2024 | First archived No. 1 for watt-tool-70B (74.24 overall accuracy); watt-tool-8B ranked No. 4 (67.94). |
| 27 Feb 2025 | Last archived screenshot here showing watt-tool-70B still at No. 1 (74.31); watt-tool-8B still ranked No. 4 (67.98). |
| 25 Apr 2025 | The 70B model’s period at No. 1 ended. It moved to No. 3, with Salesforce’s xLAM-2-70b-fc-r ahead of it. |
These are historical BFCL V3 results. The two screenshots document the first and last archived No. 1 snapshots in this project; the April milestone follows my project history. The live BFCL leaderboard continues to evolve.
Training approach
The work focused on choosing the right tool, supplying appropriate arguments, and recognizing when no available tool was relevant. We combined full fine-tuning with iterative data refinement and reasoning-path construction.
Data coverage and tool irrelevance
We rebalanced the distribution of tool usage and combined several sources: annotations from the Lupan platform, including social-media APIs; the Glaive and xLAM function-calling datasets; and ToolACE. We also added our own tool-irrelevance dataset to train decisions about when a tool should not be called, with the aim of reducing hallucinated or unnecessary calls.
Iterative dataset refinement
We used function masking, introduced similar tools as hard negatives, and varied the wording of tool descriptions, parameter definitions, and user questions. These iterations aimed to improve discrimination between plausible tool choices and make the training data less dependent on a single phrasing.
Reasoning paths with MCTS
Inspired by rStar, we used Monte Carlo Tree Search (MCTS) to construct reasoning paths, exploring up to five reasoning strategies for choosing the next action from the current state. The original announcement describes this alongside the data-refinement work.
The released model cards also document supervised fine-tuning, chain-of-thought data synthesis, and Direct Multi-Turn Preference Optimization (DMPO) for multi-turn agent tasks.
From models to application workflows
The models were designed for tool selection, parallel function calls, and multi-turn dialogue in workflow-building applications such as Lupan. This connected model training to the practical task of executing a user’s request across multiple tools.
The surrounding Watt workflow supported more than 200 tools. In a separate 1,000-task multi-step evaluation, task completion improved from 70% to 90%. These are workflow-level results from my project experience, separate from the BFCL scores above.