mirror of
https://github.com/semantica-agi/semantica.git
synced 2026-08-29 04:26:20 +00:00
557e29ee14
* fix(explorer): repair /api/enrich/extract and the /api/decisions routes Two Explorer API endpoints fail on every install. /api/enrich/extract imported extract_entities and extract_relations from semantic_extract.methods, where neither name is defined — that module ships only the per-strategy variants (extract_entities_ml, extract_relations_regex, ...), and nothing re-exports a plain facade. The resulting ImportError was caught and reported as "semantic_extract module not available. Ensure spacy and transformers are installed.", so a wiring bug looked like a missing dependency. The route now calls NamedEntityRecognizer and RelationExtractor directly, the classes the README documents, and feeds the extracted entities into relation extraction rather than re-deriving them. The 503 branch stays for a genuinely absent module. Every /api/decisions* route returned 500 once the graph held a decision: record_decision() stores timestamp as datetime.now().timestamp(), a float, while DecisionResponse types the field as str, so pydantic rejected the value the library itself wrote. A before-mode field validator on DecisionResponse normalizes float, int and datetime inputs to ISO-8601, covering every route that builds the model instead of only the list endpoint. The existing tests missed both: test_extract accepted 503 as a pass, and the decision fixtures are hand-built nodes carrying no timestamp at all. Both are tightened, and a TestRecordedDecisions class exercises the routes against decisions created through record_decision(). Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> * perf(semantic_extract): cache spaCy models instead of loading one per call extract_entities_ml(), extract_relations_similarity() and extract_relations_dependency() called spacy.load() on every invocation, so the model was re-read from disk and re-initialized per call. On a short sentence that is ~120 ms of loading around ~2 ms of work, and successive calls never got cheaper. The path is reachable from the CLI, the MCP extract_entities tool, the pipeline ner_extract step and POST /api/enrich/extract, and process_batch() multiplies it by the number of documents. The module already had a cached loader for one code path — get_nlp_model() and its _nlp_cache global — but the extraction functions bypassed it. Adds load_spacy_model(), a process-level cache keyed by model name behind a lock so concurrent callers do not each start a load, and routes the five call sites through it. Errors are left uncached and propagate unchanged, so the existing OSError fallbacks to pattern extraction still fire. get_nlp_model() keeps its own entry: it loads with disable=["parser", "ner", "lemmatizer"] for similarity work, so its model is not interchangeable with the NER one. Cache entries record the spacy module object they came from. Several tests patch methods.spacy with a mock and assert on load calls; without that guard a name-keyed cache would hand a previous test's mock to a later one. Measured on the same sentence, Python 3.12.13 / spacy 3.8.15 / en_core_web_sm: extract_entities_ml() median 132 ms before, 2.1 ms after, identical entities. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> * fix(explorer): harden extraction and timestamp handling * fix(explorer): catch OverflowError/OSError in decision timestamp validator DecisionResponse._normalize_timestamp only guarded against NaN/inf via math.isfinite(), but datetime.fromtimestamp() raises OverflowError or OSError for finite epoch values outside the platform's representable range (e.g. milliseconds stored where seconds were expected). Those exceptions escaped the pydantic validator unhandled, reintroducing an unhandled 500 on /api/decisions* for exactly the bug class this PR closes. Also exclude bool from the numeric branch, since bool is an int subclass and was being silently coerced to epoch 0/1. * docs: add changelog entry for PR #886 (explorer extract/decisions fixes) Documents the extraction 503, decisions timestamp 500, and folded-in spaCy caching fixes, plus the review-round hardening from Sameer6305 and the timestamp overflow/bool fix from this follow-up commit. --------- Co-authored-by: joseedson18jc <joseedson18jc@users.noreply.github.com> Co-authored-by: Claude Opus 5 <noreply@anthropic.com> Co-authored-by: Sameer Kadam <sskadam6305@gmail.com> Co-authored-by: KaifAhmad1 <kaifahmad087@gmail.com> Co-authored-by: Mohd Kaif <98801504+KaifAhmad1@users.noreply.github.com>