classifiers.refresh_tool_embeddings module

Refresh embeddings for tools whose descriptions have changed.

Compares each registered tool’s live description against the stored metadata in Redis. Tools with mismatched descriptions are “stale”. Tools whose tool_index_data.json entry is missing or does not have the expected number of non-empty string synthetic_queries are “invalid”. Both sets are refreshed (union) unless --invalid-only is passed.

Redis: This script does not delete or flush the embedding hash. It only HSETs keys for tools that successfully produced an embedding. If query generation or embedding fails for every target, Redis is left unchanged.

Index file: If tool_index_data.json exists but fails to parse, this module used to load {} and could overwrite the file with a partial index. That is now blocked (see load guard below).

Run python -m classifiers.refresh_tool_embeddings (see --help) with optional arguments:

  • --force — Re-embed every tool regardless of whether its description changed (useful after switching embedding models).

  • --tools — Comma-separated list of specific tool names to refresh.

  • --tools-dir — Tool scripts directory (default: tools).

  • --embed-only — Skip Gemini query generation; recompute embeddings from existing tool_index_data.json only (use after build_tool_index).

  • --invalid-only — Only tools with invalid/missing index queries (skip stale-description check). Ignored with --force or --embed-only.

Synthetic query generation uses exponential backoff, rotates Gemini models (gemini-2.5-flash, gemini-3-flash-preview after the default), then OpenRouter google/<primary> if all Gemini calls fail (key: OPENROUTER_QUERY_GEN_API_KEY or OPENROUTER_API_KEY; 429 backoff). See classifiers/build_tool_index.generate_synthetic_queries and GEMINI_QUERY_GEN_* / GEMINI_QUERY_GEN_429_* env vars.

async classifiers.refresh_tool_embeddings.find_stale_tools(registered, redis_client)[source]

Return names of tools whose live descriptions differ from Redis.

Detects “stale” tools: those whose currently registered description no longer matches the description captured the last time the tool was embedded. This is the cheap change-detection step that lets a refresh re-embed only the tools that actually changed instead of every tool.

Reads the stored metadata once via HGETALL on the TOOL_METADATA_HASH_KEY Redis hash, JSON-decodes each entry, and compares its description field against the live description attribute on the registered tool definition. Tools absent from the hash are skipped (nothing to compare against); tools whose stored metadata is unparseable JSON are treated as stale so they get rebuilt. This function only reads from Redis and does not mutate it.

Called by refresh_tool_embeddings() (twice: the --embed-only and the default stale/invalid paths) and by the standalone classifiers.update_changed_tool_embeddings script.

Parameters:

  • registered (dict[str, Any]) – Mapping of tool name to the registered tool definition object, as returned by discover_tools; each value must expose a description attribute.

  • redis_client (Redis) – Async Redis client used to read the stored tool metadata hash.

Return type:

list[str]

Returns:

The list of tool names whose live description differs from (or has no valid stored counterpart for) the value in Redis.

async classifiers.refresh_tool_embeddings.refresh_tool_embeddings(*, force=False, embed_only=False, invalid_only=False, tool_names=None, tools_dir='tools')[source]

Re-embed the tools that changed and write their vectors back to Redis.

The end-to-end refresh routine for the vector tool classifier. It decides which tools to refresh, (re)generates their synthetic queries via the LLM unless embed_only is set, recomputes centroid embeddings for them, and stores those vectors plus metadata in Redis so the classifier can route to the changed tools. The default mode is incremental (only stale or invalid tools); flags widen or narrow that target set.

This orchestrates several collaborators: discover_tools to enumerate registered tools, find_stale_tools() and discover_invalid_query_index_tools to pick targets, load_index_file / save_index_file for tool_index_data.json (backed up to a .bak copy before write), generate_synthetic_queries to call Gemini generateContent (with OpenRouter fallback) for query text, the OpenRouterEmbeddings client plus compute_tool_centroids_bulk() to turn those queries into one vector per tool, and store_tool_embedding_hash to index each vector. It touches three Redis structures: it reads TOOL_METADATA_HASH_KEY (via find_stale_tools()) and TOOL_EMBEDDINGS_HASH_KEY (for the force path), then HSETs both hashes plus the per-tool RediSearch index entries. Redis is left untouched when no embedding is produced. It also reads Config / env vars for the API key and Redis URL, opens its own httpx.AsyncClient for query generation, bounds generation concurrency with a semaphore of 3, and always closes the Redis client in a finally block.

Called by the module’s main() (CLI entry point) and invoked programmatically by the write_python_tool and import_mcp_tool tools after a new tool is created, so the classifier learns the new tool.

Parameters:

  • force (bool) – Re-embed every tool that already has an embedding (or, with embed_only, every tool with synthetic queries in the index), ignoring the stale-description check.

  • embed_only (bool) – Recompute embeddings from the existing tool_index_data.json only, skipping LLM synthetic-query generation. Used after build_tool_index.

  • invalid_only (bool) – Refresh only tools whose index entry is missing or has a bad synthetic_queries list, skipping the stale check. Ignored with force or embed_only.

  • tool_names (list[str] | None) – Explicit list of tool names to refresh; unknown names are warned about and skipped. Overrides the automatic target selection.

  • tools_dir (str) – Directory of tool scripts to discover (default tools).

Return type:

bool

Returns:

True on success (including the no-op case where nothing needed refreshing), False on a fatal precondition failure such as a missing API key, an unreadable index file, or no valid targets.

Raises:

RuntimeError – If a target tool has fewer than the expected number of synthetic queries in the index when embeddings are computed.

async classifiers.refresh_tool_embeddings.main()[source]

Parse CLI arguments and drive a single refresh_tool_embeddings() run.

The command-line entry point for python -m classifiers.refresh_tool_embeddings. It builds the argparse parser for the --force, --tools, --tools-dir, --embed-only, and --invalid-only flags, normalises the comma-separated --tools value into a list, and forwards everything to refresh_tool_embeddings().

The whole run’s exit status is derived here: it calls sys.exit(0) on a truthy result, sys.exit(1) on a falsy result, and also sys.exit(1) after logging the traceback if the refresh raises. Output goes to the module logger configured at import time.

Called only by the if __name__ == "__main__" guard via asyncio.run(main()); it is the process entry point and has no in-repo callers.

Return type:

None