Google has launched Tool Retrieval in the Gemini API, now in preview in the Interactions API. Gemini API Tool Retrieval lets an agent keep a large tool library available. It does not send every parameter schema in every prompt. The model loads a schema only when a task needs that tool. For agents wired to dozens of functions or MCP servers, it targets a real cost: tool definitions. You can check the deployment options and specs here through our GPULeaderboard.com chrome extension.
TL;DR
- Tool: An API feature with no stated tool cap. Google advises static loading under 20 stable tools.
- Runs on: Google’s servers (server-side search) or your own registry (client-side). Interactions API,
v1betaonly. - Performance: Search, schema load and function call resolve in 1 API request in server mode.
- Best: 0 extra round trips and no index to host for server-side retrieval.
What is Gemini API Tool Retrieval?
Tool Retrieval is a Gemini API feature that defers tool parameter schemas until the model asks for them. Google’s documentation states the problem plainly. A few dozen function declarations can consume tens of thousands of tokens on every request.
With deferral, the model sees only each tool’s name and a short description. It does not see the parameter schema until retrieval.
How does defer_loading work in the Gemini Interactions API?
You add {"type": "tool_search"} to the tools array. Then you set defer_loading: true on each function to hold back. The @googledevs launch post shows a deferred issue_refund function on gemini-4-hydrogen. Its code comment mentions 50+ other deferred functions.
When the model needs a deferred tool, the response contains 3 steps:
tool_search_call: the model requests a tool by name.tool_search_result: the API returns the full schema.function_call: the model emits the call your code executes.
The API resolves the first 2 steps itself. Your app only runs the function_call and returns a function_result.
Deferred functions can carry an optional short_description. It is shown during retrieval and is valid only when defer_loading is true.
Server-side vs client-side retrieval
Server-side is the default. Gemini runs the search, so there is no index to build or host.
Client-side mode uses "execution": "client". Your app runs its own lookup, such as an embedding index, registry or permission filter. It returns matching declarations in function_result.result. Those tools can join mid-conversation, even if absent from the initial tools array. Each client-side search costs 1 extra round trip.
The 2 modes cannot be combined in 1 request. Only 1 tool_search entry is allowed per request.
Does it work with MCP servers and prefix caching?
Yes on both, with caveats. Setting defer_loading: true on an mcp_server entry defers every tool that server exposes. Deferred MCP tools still execute remotely through the Gemini API MCP support.
On caching, deferred schemas stay out of the prompt prefix. Loaded schemas append inline, so implicit caching still applies. The docs add one catch: deferring tools can shrink a prompt below the minimum cacheable size.
What does Tool Retrieval cost?
There is no separate fee. Tokens bill at the model’s standard input, cached and output rates. Each server-side search triggers an extra model call over the full context. Those tokens appear in total_tool_use_tokens, billed at the input rate.
Each loaded schema then stays billed as input for the rest of the conversation. The docs say static loading can cost less for small libraries or per-turn searches.
How does it compare with Claude and OpenAI tool search?
Anthropic and OpenAI already ship similar tool search features, with published savings numbers.
| Feature | Gemini API Tool Retrieval | Anthropic Tool Search Tool | OpenAI tool search |
|---|---|---|---|
| Status | Preview, v1beta | Not disclosed on docs page | Not disclosed on docs page |
| API surface | Interactions API only | Messages API; Bedrock InvokeModel | Responses API |
| Models | Gemini 4 Carbon, Neon, Hydrogen, Argon; 3.8 Flash; 3.6 Flash | Claude 5 series, Opus 4.5+, Sonnet 4.5+, Haiku 4.5 | gpt-5.4 and later |
| Search modes | Server or client | Regex, BM25 or custom | Hosted or client-executed |
| Max deferred tools | Not disclosed | 10,000 | Not disclosed |
| Results per search | Not disclosed | 5 by default | Not disclosed |
| MCP deferral | Per mcp_server entry | Per mcp_toolset | On MCP server tool |
| Separate fee | No | No | Not disclosed |
| Published token savings | None | 85% (77K to 8.7K tokens) | 47% on 250 MCP Atlas tasks |
| Published accuracy | None | Opus 4.5: 79.5% to 88.1% | Not disclosed |
Anthropic and OpenAI figures are self-reported in their own tests. They are not comparable to any Gemini result.
Key Takeaways
- Gemini API Tool Retrieval defers tool schemas until the model requests them.
- Server mode resolves search, load and call in 1 API request.
- Client mode supports custom registries, at 1 extra round trip per search.
- No extra fee, but loaded schemas stay billed and cannot be unloaded.
- Google published no savings data; Anthropic reports 85%, OpenAI 47%.
Check out the Google blog post, the Tool Retrieval docs and the MCP function calling docs. You can check the deployment options and specs here through our GPULeaderboard.com chrome extension.
Also, feel free to follow us on Twitter and donโt forget to join our 150k+ML SubReddit and Subscribe to our Newsletter. Wait! are you on telegram? now you can join us on telegram as well.
Need to partner with us for promoting your GitHub Repo OR Hugging Face Page OR Product Release OR Webinar etc.? Connect with us
Michal Sutter is a data science professional with a Master of Science in Data Science from the University of Padova. With a solid foundation in statistical analysis, machine learning, and data engineering, Michal excels at transforming complex datasets into actionable insights.








