mirror of
https://github.com/semantica-agi/semantica.git
synced 2026-08-29 04:26:20 +00:00
fix: prevent fallback recursion and write proper Parquet in embed generate command
Fixes #994: 1. Prevent self-recursion in methods.py: generate_embeddings, embed_text, calculate_similarity, and pool_embeddings all registered themselves as custom methods, causing infinite self-calls when dispatch invoked them without explicitly passing method parameter. Fix: check custom_method is not the function itself before recursing. 2. Fix embed generate --output corrupt output: the CLI wrote json.dumps(result, default=str) which produced plaintext repr of numpy arrays (e.g. '[1.49e-01 4.85e-02 ...]') instead of proper Parquet. Fix: detect .parquet extension (case-insensitive), convert numpy array to pandas DataFrame with dim_* columns and id index, use to_parquet(). Non-parquet extensions fall back to JSON with clear ImportError message. 3. Add pyarrow>=14.0.0 to core dependencies (previously only in ingest-parquet/ingest-arrow optional extras). The documented quick-start flow of embed generate --output ... requires pyarrow out of the box. (Note: pandas>=1.3.0 is already a core dependency; pyarrow is the missing piece.) Note: .github/workflows/* files are excluded from this PR as they require a token with workflow scope. Upstream workflows are unchanged.
This commit is contained in:
+2
-1
@@ -85,7 +85,8 @@ dependencies = [
|
||||
"loguru>=0.7.3",
|
||||
"structlog>=22.1.0",
|
||||
"gensim>=4.4.0",
|
||||
"httpx<0.29.0"
|
||||
"httpx<0.29.0",
|
||||
"pyarrow>=14.0.0"
|
||||
]
|
||||
|
||||
[project.urls]
|
||||
|
||||
Reference in New Issue
Block a user