threadweave

Threadweave persistent knowledge system.

Provides the DNA Vault (file-backed archive with Redis embedding index), Persistent Weave (channel/category/guild-scoped enforcement pointers), Shadow Memory (hidden per-user memory store), and Weave Exceptions (per-channel overrides for DNA pointers).

All Redis keys live under the stargazer:threadweave namespace for backward compatibility with the old codebase.

class threadweave.ThreadweaveManager(redis_client, openrouter, embedding_model='google/gemini-embedding-001', admin_user_ids=None, dna_vault_path='data/dna_vault')

Bases: object

Core threadweave infrastructure.

Parameters:

• redis_client (Any) – Async Redis client (same one used by MessageCache).

• openrouter (OpenRouterClient) – OpenRouterClient used for generating embeddings.

• embedding_model (str) – Model identifier for embedding generation.

• admin_user_ids (set[str] | None) – Set of user IDs authorised to wield threadweave tools.

• dna_vault_path (str) – Directory for file-backed DNA vault storage.

__init__(redis_client, openrouter, embedding_model='google/gemini-embedding-001', admin_user_ids=None, dna_vault_path='data/dna_vault')

Initialize the instance.

Parameters:

• redis_client (Any) – Redis connection client.

• openrouter (OpenRouterClient) – The openrouter value.

• embedding_model (str) – The embedding model value.

• admin_user_ids (set[str] | None) – The admin user ids value.

• dna_vault_path (str) – The dna vault path value.

Return type:

None

require_admin(user_id)

Gate threadweave actions behind the Prime Architect allow-list.

Checks whether user_id belongs to self.admin_user_ids and, when it does not, produces a ready-to-return denial payload. This is the single chokepoint that keeps the destructive threadweave capabilities (vaulting DNA, planting Shadow Memories, editing the Persistent Weave) restricted to authorised operators.

It performs no I/O — it is a pure in-memory set membership test against the admin set seeded at construction time. Called by nearly every handler in tools/threadweave_tools.py (e.g. the excise, shadow-memory, and weave-exception tools), each of which calls tw.require_admin(ctx.user_id) first and short-circuits if the return value is truthy.

Parameters:

user_id (str) – Platform sender id to authorise; coerced to str before comparison.

Returns:

None when the user is authorised; otherwise a dict with an in-character error message and a status of "denied" that the caller can surface directly to the LLM/user.

Return type:

dict[str, str] | None

async vault_dna(origin_user_id, excised_by, content, post_links, thread_color, channel_id='', category_id='', guild_id='', attached_files=None)

Archive an excised conversational thread into the DNA Vault.

Persists the full “excised” content both as a file-backed record and as a searchable index entry, so that future conversations can be cross-checked against past offences. This is the durable write side of the DNA Vault: it mints a fresh dna_id, builds a short human summary, writes the complete payload to a .dna JSON file, embeds the summary for semantic search, and registers the entry in Redis.

It touches several subsystems. The file write goes to {dna_vault_path}/{dna_id}.dna via _write_dna_file (run in a worker thread so the event loop is not blocked). The summary is embedded through _embed (the OpenRouterClient embedding API); if embedding raises, the failure is logged and an empty vector is stored so the record still exists. The index entry is written to the Redis hash stargazer:threadweave:dna_index keyed by dna_id. Called by the threadweave excision tools in tools/threadweave_tools.py (the various excise/vault handlers around lines 101-511), which then typically register a Persistent Weave pointer for the new id.

Parameters:

• origin_user_id (str) – Id of the user whose content is being excised.

• excised_by (str) – Id of the admin performing the excision.

• content (str) – Full text being archived as the thread DNA.

• post_links (list[str]) – Source message/permalink references for the content.

• thread_color (str) – Severity/category tag (e.g. red, black) recorded and surfaced in the short description.

• channel_id (str) – Originating channel id, stored for scoping.

• category_id (str) – Originating category id, stored for scoping.

• guild_id (str) – Originating guild id, stored for scoping.

• attached_files (list[str] | None) – Optional list of associated file references.

Returns:

A summary of the write containing dna_id, the absolute-ish file_path, the short_description, and the full index_entry that was stored in Redis.

Return type:

dict[str, Any]

async read_dna(dna_id)

Fetch a single DNA Vault entry by id, index plus full payload.

Resolves a vaulted thread back into its two halves: the lightweight Redis index record (metadata and embedding) and the complete file-backed payload (the original content and links). This is the read counterpart to vault_dna and the canonical way other code rehydrates an archived excision.

It reads the index hash stargazer:threadweave:dna_index via HGET keyed by dna_id; if present, it loads the corresponding .dna JSON file from disk (path taken from the index, falling back to {vault_path}/{dna_id}.dna). Called by the web admin layer (web/threadweave_api.py) and by several threadweave tool handlers in tools/threadweave_tools.py that need to inspect or edit an existing DNA record before acting on it.

Parameters:

dna_id (str) – Unique id of the DNA Vault entry to read.

Returns:

None if no index entry exists for the id; otherwise a dict with index (the Redis metadata) and full_payload (the parsed file contents, or None if the backing file is missing).

Return type:

dict[str, Any] | None

async delete_dna(dna_id)

Permanently delete a DNA Vault entry from disk and the index.

Fully retires an archived thread: it removes both the file-backed record and its searchable index entry so the DNA no longer surfaces in vault searches or context injection. This is the destructive inverse of vault_dna.

It first reads the index hash stargazer:threadweave:dna_index via HGET to locate the backing file, deletes that .dna file from disk when it exists, then removes the field from the index hash via HDEL. Called by the web admin endpoint in web/threadweave_api.py and by the DNA-deletion tool handler in tools/threadweave_tools.py (around line 601, which typically also clears any Persistent Weave pointers for the same id).

Parameters:

dna_id (str) – Unique id of the DNA Vault entry to delete.

Returns:

True if an index entry existed and was removed; False if no entry was found for the id.

Return type:

bool

async search_dna_vault(query, top_k=5, user_filter=None, query_embedding=None)

Rank DNA Vault entries by semantic similarity to a query.

Provides the retrieval-augmented side of the vault: given a natural language query (or a precomputed query embedding), it finds the most relevant archived excisions so they can be injected into prompt context or shown to an operator. It powers the “DNA Vault RAG” section assembled by get_context_for_prompt.

When no query_embedding is supplied it embeds query via _embed (the OpenRouterClient embedding API), returning an empty list on embedding failure. It then loads every entry from the Redis hash stargazer:threadweave:dna_index via HGETALL, optionally filters by origin_user when user_filter is set, skips entries with no stored embedding, and scores the rest in one vectorised pass with utils.cosine.cosine_batch(). Called internally by get_context_for_prompt; no external callers were found.

Parameters:

• query (str) – Natural-language search text. Ignored when query_embedding is provided.

• top_k (int) – Maximum number of best-matching entries to return.

• user_filter (str | None) – When set, only entries whose origin_user equals this value are considered.

• query_embedding (list[float] | None) – Optional precomputed query vector to reuse, avoiding a redundant embedding call.

Returns:

Up to top_k index entries ordered by descending similarity; empty if embedding failed or no embedded entries matched.

Return type:

list[dict[str, Any]]

async add_persistent_weave_pointer(dna_id, origin_user_id, short_description, thread_color, channel_id='', category_id='', guild_id='')

Register a DNA pointer into one or more Persistent Weave scopes.

Promotes an archived DNA entry into “active enforcement” by writing lightweight pointers at whichever of the channel, category, and guild scopes are supplied. Pointers in the Persistent Weave are what get_context_for_prompt reads back to remind the bot which excised-thread patterns are local law in the current context.

For each non-empty scope id it writes the same JSON pointer (id, origin user, short description, thread color, timestamp) via HSET into the matching Redis hash: stargazer:threadweave:pweave:channel, ...:pweave:category, or ...:pweave:guild (each suffixed with the scope id), keyed by dna_id. It logs the scopes written but performs no embedding or file I/O. Called by the threadweave excision tools in tools/threadweave_tools.py (the various vault/excise handlers around lines 112-523), usually right after vault_dna.

Parameters:

• dna_id (str) – Id of the DNA Vault entry the pointer references.

• origin_user_id (str) – Id of the user the excised thread belongs to.

• short_description (str) – Human-readable summary stored in the pointer.

• thread_color (str) – Severity/category tag carried on the pointer.

• channel_id (str) – When non-empty, writes a channel-scoped pointer.

• category_id (str) – When non-empty, writes a category-scoped pointer.

• guild_id (str) – When non-empty, writes a guild-scoped pointer.

Returns:

A dict with dna_id and scopes_written, the list of scope labels (e.g. "channel:123") that received a pointer.

Return type:

dict[str, Any]

async remove_persistent_weave_pointer(dna_id)

Tear down a DNA pointer from every Persistent Weave scope.

Globally deactivates enforcement for a DNA id by deleting its pointer wherever it may live, without the caller needing to know which channel, category, or guild scopes it was registered under. This is the inverse of add_persistent_weave_pointer.

It scans Redis with SCAN across all three pointer prefixes (stargazer:threadweave:pweave:channel / ...:category / ...:guild, each pattern ...:*) and issues an HDEL for dna_id on every matching hash, collecting the keys it actually removed from. Note this does not remove the DNA from the vault itself. Called by the web admin endpoint in web/threadweave_api.py and by the DNA-deletion tool handler in tools/threadweave_tools.py (around line 602), typically alongside delete_dna.

Parameters:

dna_id (str) – Id of the DNA pointer to remove from all scopes.

Returns:

A dict with dna_id and removed_from, the list of Redis hash keys from which a pointer was actually deleted.

Return type:

dict[str, Any]

async get_all_persistent_weave(channel_id, category_id='', guild_id='')

Collect Persistent Weave pointers across channel/category/guild.

Gathers the raw, unfiltered set of active DNA pointers that apply to a location, grouped by the scope they were registered at. It is the broad fan-out read that get_filtered_persistent_weave builds on before applying per-channel exceptions.

For each non-empty scope id it reads the corresponding Redis hash via _get_weave_pointers (which issues HGETALL and JSON-decodes the values) against stargazer:threadweave:pweave:channel / ...:category / ...:guild suffixed with the id. It does not deduplicate or apply exceptions. Called internally by get_filtered_persistent_weave; no external callers were found.

Parameters:

• channel_id (str) – When non-empty, reads channel-scoped pointers.

• category_id (str) – When non-empty, reads category-scoped pointers.

• guild_id (str) – When non-empty, reads guild-scoped pointers.

Returns:

A dict with channel, category, and guild keys, each mapping to the list of pointer dicts found at that scope (empty lists where the scope id was blank or had no pointers).

Return type:

dict[str, list[dict[str, Any]]]

async get_filtered_persistent_weave(channel_id, category_id='', guild_id='')

Resolve the effective Persistent Weave pointers for a channel.

Produces the deduplicated, exception-aware list of DNA pointers that should actually be enforced in a given channel. This is the read used to drive prompt injection: a pointer that has a per-channel weave exception is suppressed here so the bot stops enforcing that thread in the channel where it was waived.

It first fans out via get_all_persistent_weave to gather pointers from every applicable scope, then walks them keeping each dna_id only once. When a channel_id is provided it consults get_weave_exceptions (Redis set stargazer:threadweave:exception per id) and drops any pointer whose exception set contains this channel. Called internally by get_context_for_prompt; no external callers were found.

Parameters:

• channel_id (str) – Channel being rendered; also used to apply per-channel weave exceptions.

• category_id (str) – Category scope to include when non-empty.

• guild_id (str) – Guild scope to include when non-empty.

Returns:

The active pointer dicts, deduplicated by dna_id and with channel-excepted pointers removed.

Return type:

list[dict[str, Any]]

async add_weave_exception(dna_id, channel_id)

Add weave exception.

Parameters:

• dna_id (str) – The dna id value.

• channel_id (str) – Discord/Matrix channel identifier.

Returns:

True on success, False otherwise.

Return type:

bool

async remove_weave_exception(dna_id, channel_id)

Delete the specified weave exception.

Parameters:

• dna_id (str) – The dna id value.

• channel_id (str) – Discord/Matrix channel identifier.

Returns:

True on success, False otherwise.

Return type:

bool

async get_weave_exceptions(dna_id)

Retrieve the weave exceptions.

Parameters:

dna_id (str) – The dna id value.

Returns:

The result.

Return type:

list[str]

async add_shadow_memory(target_user_id, description, created_by, source_dna_id='')

Plant a hidden, embedded Shadow Memory attached to a user.

Records a private note about a user that informs the bot’s behaviour but is never disclosed to the user it concerns. Each memory is embedded at write time so it can later be retrieved by semantic relevance during prompt assembly.

It mints a shadow_id, embeds description via _embed (the OpenRouterClient embedding API; on failure the error is logged and an empty vector is stored so the memory still persists), and writes the full entry via HSET into the Redis hash stargazer:threadweave:shadow:{target_user_id} keyed by the new shadow id. Called by the shadow-memory tool handlers in tools/threadweave_tools.py (around lines 187 and 802), often seeded from a freshly vaulted DNA id.

Parameters:

• target_user_id (str) – Id of the user the memory is attached to.

• description (str) – The hidden note text; also the source for the embedding.

• created_by (str) – Id of the admin who created the memory.

• source_dna_id (str) – Optional DNA Vault id this memory derives from.

Returns:

The stored entry, including shadow_id, the target/creator ids, the description, the embedding (possibly empty), source_dna_id, and an ISO timestamp.

Return type:

dict[str, Any]

async delete_shadow_memory(target_user_id, shadow_id)

Delete the specified shadow memory.

Parameters:

• target_user_id (str) – The target user id value.

• shadow_id (str) – The shadow id value.

Returns:

True on success, False otherwise.

Return type:

bool

async get_shadow_memories(target_user_id)

Retrieve the shadow memories.

Parameters:

target_user_id (str) – The target user id value.

Returns:

The result.

Return type:

list[dict[str, Any]]

async clear_all_shadow_memories(target_user_id)

Nuke ALL Shadow Memories for a user in one shot. # 💀🔥

Deletes the entire Redis hash stargazer:threadweave:shadow:{target_user_id}. Returns the number of individual shadow entries that existed before deletion.

Parameters:

target_user_id (str) – Id of the user whose shadow memories are being purged.

Returns:

Count of shadow entries that were destroyed.

Return type:

int

async search_shadow_memories(target_user_id, query, top_k=5, query_embedding=None)

Semantically rank a target user’s Shadow Memories against a query.

Embeds query (unless a precomputed query_embedding is supplied), loads every stored Shadow Memory for target_user_id via get_shadow_memories(), keeps only entries that carry an embedding, and returns the top_k most similar by cosine similarity (cosine_batch()). Read-only; an embedding failure or an empty corpus yields an empty list rather than raising.

Called by ThreadweaveManager’s relationship-context assembly (threadweave.py:1099) to surface the most relevant shadow recollections about another user.

Parameters:

• target_user_id (str) – The user whose Shadow Memories are searched.

• query (str) – Natural-language text to match against; embedded on demand.

• top_k (int) – Maximum number of memories to return (default 5).

• query_embedding (list[float] | None) – Optional precomputed embedding for query that skips the embedding call.

Returns:

The up-to-top_k most similar memory entries, most-similar first; empty if embedding fails or no embedded entries exist.

Return type:

list[dict[str, Any]]

async store_pending_approval(dna_id, approval_type, draft_content, requested_by)

Store pending approval.

Parameters:

• dna_id (str) – The dna id value.

• approval_type (str) – The approval type value.

• draft_content (str) – The draft content value.

• requested_by (str) – The requested by value.

Returns:

Result string.

Return type:

str

async get_pending_approvals(approval_type)

Retrieve the pending approvals.

Parameters:

approval_type (str) – The approval type value.

Returns:

The result.

Return type:

list[dict[str, Any]]

async get_context_for_prompt(channel_id, category_id='', guild_id='', user_ids=None, query='', query_embedding=None)

Build the threadweave context string.

Gathers: 1. Filtered Persistent Weave pointers 2. Shadow Memories for each user 3. DNA Vault RAG results for the query

Return type:

str

Parameters:

• channel_id (str)

• category_id (str)

• guild_id (str)

• user_ids (list[str] | None)

• query (str)

• query_embedding (list[float] | None)