atropa_lore_semantic

Validated, local semantic retrieval for the static ATROPA lore corpus.

The runtime path in this module performs no network I/O. It consumes a prebuilt, normalized NumPy matrix and caller-supplied query embedding, then returns bounded excerpts with source provenance. A missing or invalid index fails closed to the existing read-only lexical corpus search (or to an empty list when that fallback is disabled).

The index is treated as untrusted data even when it ships beside the service: its manifest, file digests, model, dimensions, dtype, metric, row count, normalization, and every record’s provenance are validated before use.

exception atropa_lore_semantic.AtropaIndexValidationError

Bases: ValueError

Raised when an ATROPA semantic index fails integrity validation.

class atropa_lore_semantic.AtropaSemanticIndex(records, embeddings, manifest)

Bases: object

A validated record table and normalized memory-mapped vector matrix.

Parameters:
records: tuple[dict[str, Any], ...]
embeddings: ndarray
manifest: dict[str, Any]
async atropa_lore_semantic.asearch_atropa_lore(query, *, query_embedding, max_results=5, min_score=0.3, max_excerpt_chars=1800, index_dir=PosixPath('/home/ubuntu/stargazer-v3/stargazer-v3/data/knowledge/atropa_semantic'), corpus_path=PosixPath('/home/ubuntu/stargazer-v3/stargazer-v3/data/knowledge/atropa_corpus.jsonl'), expected_model='google/gemini-embedding-001', expected_dimensions=3072, lexical_fallback=True)

Run validated local ATROPA retrieval without blocking the event loop.

Return type:

list[dict[str, Any]]

Parameters:
async atropa_lore_semantic.asearch_holder_ecosystem_sources(query, *, query_embedding, max_results=5, min_score=0.3, max_excerpt_chars=1800, expected_model='google/gemini-embedding-001', expected_dimensions=3072, atropa_index_dir=PosixPath('/home/ubuntu/stargazer-v3/stargazer-v3/data/knowledge/atropa_semantic'), atropa_corpus_path=PosixPath('/home/ubuntu/stargazer-v3/stargazer-v3/data/knowledge/atropa_corpus.jsonl'), dysnomia_index_dir=PosixPath('/home/ubuntu/stargazer-v3/stargazer-v3/data/knowledge/dysnomia_core_semantic'), dysnomia_corpus_path=PosixPath('/home/ubuntu/stargazer-v3/stargazer-v3/data/knowledge/dysnomia_core_corpus.jsonl'))

Run the combined local ecosystem search off the event loop.

Return type:

list[dict[str, Any]]

Parameters:
atropa_lore_semantic.clear_index_cache()

Drop the process-local validated-index cache (primarily useful in tests).

Return type:

None

atropa_lore_semantic.load_semantic_index(index_dir=PosixPath('/home/ubuntu/stargazer-v3/stargazer-v3/data/knowledge/atropa_semantic'), *, expected_model='google/gemini-embedding-001', expected_dimensions=3072, source_corpus_path=None)

Load and strictly validate an ATROPA semantic index.

Successful indexes are cached by their artifact file signatures plus the optional source-corpus signature. The NumPy matrix is opened read-only with mmap_mode='r' so workers do not each need a private copy of the shipped vector data.

Return type:

AtropaSemanticIndex

Parameters:

  • index_dir (Path)

  • expected_model (str)

  • expected_dimensions (int)

  • source_corpus_path (Path | None)

atropa_lore_semantic.search_atropa_lore(query, *, query_embedding, max_results=5, min_score=0.3, max_excerpt_chars=1800, index_dir=PosixPath('/home/ubuntu/stargazer-v3/stargazer-v3/data/knowledge/atropa_semantic'), corpus_path=PosixPath('/home/ubuntu/stargazer-v3/stargazer-v3/data/knowledge/atropa_corpus.jsonl'), expected_model='google/gemini-embedding-001', expected_dimensions=3072, lexical_fallback=True)

Return local cosine-similarity matches for a caller-supplied embedding.

No embedding provider is called here. If the query vector or validated index is unavailable, the function optionally falls back to the existing deterministic lexical corpus search.

Return type:

list[dict[str, Any]]

Parameters:
atropa_lore_semantic.search_holder_ecosystem_sources(query, *, query_embedding, max_results=5, min_score=0.3, max_excerpt_chars=1800, expected_model='google/gemini-embedding-001', expected_dimensions=3072, atropa_index_dir=PosixPath('/home/ubuntu/stargazer-v3/stargazer-v3/data/knowledge/atropa_semantic'), atropa_corpus_path=PosixPath('/home/ubuntu/stargazer-v3/stargazer-v3/data/knowledge/atropa_corpus.jsonl'), dysnomia_index_dir=PosixPath('/home/ubuntu/stargazer-v3/stargazer-v3/data/knowledge/dysnomia_core_semantic'), dysnomia_corpus_path=PosixPath('/home/ubuntu/stargazer-v3/stargazer-v3/data/knowledge/dysnomia_core_corpus.jsonl'))

Search both inert ecosystem snapshots with bounded source diversity.

Each source keeps its own digest-bound semantic index and independently falls back to its sanitized lexical corpus. Results are round-robin merged so a high-frequency term in one codebase cannot starve the other.

Return type:

list[dict[str, Any]]

Parameters:
atropa_lore_semantic.validate_index_record(record)

Validate and reduce one index record to the runtime-safe field set.

Return type:

dict[str, Any]

Parameters:

record (Any)