mirror of
https://github.com/semantica-agi/semantica.git
synced 2026-08-30 04:40:16 +00:00
Sameer Kadam
4bf525d409
feat(ingest): add Salesforce ingestor Adds first-class Salesforce ingestion support, following the existing Connector + Data + Ingestor architecture already used by the Snowflake and Databricks integrations: SalesforceConnector / SalesforceData / SalesforceIngestor, exposed lazily from semantica.ingest so the base install stays unaffected. SalesforceConnector supports both auth landscapes Salesforce actually uses in practice: username + password + security token (SOAP login, on-prem/sandbox), and session_id + instance_url for reusing an existing authenticated session. Production and sandbox are selected through domain, credentials can come from environment variables, and the connector never intentionally puts credential material into logs, exceptions, or its own repr. SalesforceIngestor covers ingest_sobject(), ingest_query(), list_sobjects(), get_sobject_schema(), and export_as_documents(), against standard sObjects, custom objects (__c), custom metadata objects (__mdt), platform events (__e), namespaced objects, and relationship-field traversal (Owner.Name). Pagination follows nextRecordsUrl/query_more() automatically and stops once a caller's limit is satisfied rather than continuing to fetch full pages past it. Dynamically constructed SOQL is validated before it's sent: sObject names, field names, relationship paths, ORDER BY expressions, and numeric limits are checked, and WHERE fragments are screened against common injection primitives after masking quoted string literals so a value like status = 'union' doesn't false-positive. Raw SOQL passed directly to ingest_query() stays intentionally caller-controlled, since that method is documented as the advanced/unvalidated escape hatch. Salesforce-specific attributes metadata is stripped from returned records before they're handed to the rest of the pipeline, while relationship data, normal field values, and datetime normalization are preserved. export_as_documents() uses the Salesforce Id as the stable document identifier and keeps the source record in document metadata for provenance. Wired into the unified ingestion API via ingest_salesforce() and ingest(source_type="salesforce", ...), registered with MethodRegistry under sobject/query/list_sobjects/schema/documents. Isolated behind the semantica[db-salesforce] extra (simple-salesforce>=1.12.0), included in db-all. JWT Bearer authentication and Bulk API 2.0 are intentionally out of scope for this first connector; both are documented as deliberate follow-ups rather than gaps. fix(ingest): address Salesforce review findings - limit now validates as a non-negative integer before use; negative, string, and float values raise ValidationError instead of silently returning an empty result, raising a bare TypeError, or building an invalid LIMIT 0 query - fields is validated as a non-empty list of strings; a bare string (e.g. "Id") no longer gets iterated character-by-character into nonsense field names, and an empty list no longer builds a syntactically invalid SELECT - the generic connection-failure path now raises with `from None` instead of chaining the original exception, so credential or request detail from the underlying library can't surface through a traceback - the unified ingest() dispatch no longer coerces a non-dict source into None and silently falling back to environment credentials; an invalid source now raises - _validate_order_by rewritten to validate each dot-separated component through _validate_field_name, rejecting malformed fragments like "Name." or "Owner..Name" that the previous regex let through - CI conflicts from parallel merges resolved; upstream markdown dependency changes preserved test(ingest): add Salesforce JWT coverage Adds construction and connect() coverage for the JWT Bearer auth path (consumer_key + privatekey/privatekey_file), the one auth mode that had no dedicated tests despite handling private key material. Also removes _SAFE_ORDER_RE, left behind as dead code once _validate_order_by was rewritten to use _validate_field_name per component, and fixes a test-isolation leak where an earlier test left SALESFORCE_AVAILABLE=True behind for a later test that expected it False when simple-salesforce isn't installed.
309 lines
7.8 KiB
JSON
309 lines
7.8 KiB
JSON
{
|
|
"$schema": "https://mintlify.com/docs.json",
|
|
"theme": "mint",
|
|
"name": "Semantica",
|
|
"description": "The Accountability and Context Layer for AI — Context Graphs · Decision Intelligence · Full Provenance",
|
|
"colors": {
|
|
"primary": "#10B981",
|
|
"light": "#10B981",
|
|
"dark": "#10B981"
|
|
},
|
|
"favicon": "/assets/img/semantica-logo.png",
|
|
"logo": {
|
|
"light": "/assets/img/semantica-logo.png",
|
|
"dark": "/assets/img/semantica-logo.png"
|
|
},
|
|
"font": {
|
|
"headings": {
|
|
"family": "Space Grotesk",
|
|
"weight": 700
|
|
},
|
|
"body": {
|
|
"family": "Inter",
|
|
"weight": 400
|
|
}
|
|
},
|
|
"appearance": {
|
|
"default": "dark",
|
|
"strict": true
|
|
},
|
|
"css": "/assets/custom.css",
|
|
"background": {
|
|
"color": {
|
|
"dark": "#080C10",
|
|
"light": "#080C10"
|
|
}
|
|
},
|
|
"chat": {
|
|
"enabled": true
|
|
},
|
|
"feedback": {
|
|
"thumbsRating": true,
|
|
"suggestEdit": true,
|
|
"raiseIssue": true
|
|
},
|
|
"metadata": {
|
|
"og:title": "Semantica — Accountability & Context Layer for AI",
|
|
"og:description": "Build explainable, auditable knowledge graphs with full provenance. Open source. MIT licensed.",
|
|
"og:image": "/assets/img/semantica-logo.png",
|
|
"twitter:card": "summary_large_image",
|
|
"twitter:site": "@BuildSemantica"
|
|
},
|
|
"navigation": {
|
|
"tabs": [
|
|
{
|
|
"tab": "Overview",
|
|
"groups": [
|
|
{
|
|
"group": "Overview",
|
|
"pages": [
|
|
"index"
|
|
]
|
|
},
|
|
{
|
|
"group": "Core Concepts",
|
|
"pages": [
|
|
"concepts",
|
|
"modules",
|
|
"choose-your-module",
|
|
"glossary"
|
|
]
|
|
},
|
|
{
|
|
"group": "Guides",
|
|
"pages": [
|
|
"architecture",
|
|
"learning-more",
|
|
"guides/graphrag",
|
|
"guides/ingest",
|
|
"guides/llm-integrations",
|
|
"guides/pipeline",
|
|
"guides/multi-agent",
|
|
"guides/decision-intelligence",
|
|
"guides/context-graphs",
|
|
"guides/agent-memory",
|
|
"guides/provenance",
|
|
"guides/semantic-extraction",
|
|
"guides/ontology",
|
|
"guides/reasoning",
|
|
"guides/export",
|
|
"guides/deduplication",
|
|
"guides/mcp-server",
|
|
"guides/shacl-validation",
|
|
"guides/conflict-resolution",
|
|
"guides/change-management",
|
|
"guides/policy-engine",
|
|
"guides/visualization",
|
|
"guides/distance-intelligence",
|
|
"guides/graph-analytics"
|
|
]
|
|
},
|
|
{
|
|
"group": "Integrations",
|
|
"pages": [
|
|
"integrations/agno",
|
|
"integrations/crewai",
|
|
"integrations/langchain",
|
|
"integrations/docling",
|
|
"integrations/snowflake",
|
|
"integrations/databricks",
|
|
"integrations/salesforce"
|
|
]
|
|
},
|
|
{
|
|
"group": "Graph Stores",
|
|
"pages": [
|
|
"graph_stores/apache_age"
|
|
]
|
|
},
|
|
{
|
|
"group": "Vector Stores",
|
|
"pages": [
|
|
"vector_stores/pgvector"
|
|
]
|
|
}
|
|
]
|
|
},
|
|
{
|
|
"tab": "Quick Start",
|
|
"groups": [
|
|
{
|
|
"group": "Get Started",
|
|
"pages": [
|
|
"installation",
|
|
"getting-started",
|
|
"quickstart",
|
|
"choose-your-module",
|
|
"cli-setup",
|
|
"explorer-setup"
|
|
]
|
|
}
|
|
]
|
|
},
|
|
{
|
|
"tab": "Modules",
|
|
"groups": [
|
|
{
|
|
"group": "Context & Intelligence",
|
|
"pages": [
|
|
"guides/graphrag",
|
|
"guides/ingest",
|
|
"guides/llm-integrations",
|
|
"guides/pipeline",
|
|
"guides/multi-agent",
|
|
"guides/decision-intelligence",
|
|
"guides/context-graphs",
|
|
"guides/agent-memory",
|
|
"guides/provenance",
|
|
"guides/semantic-extraction",
|
|
"guides/ontology",
|
|
"guides/reasoning",
|
|
"guides/export",
|
|
"guides/deduplication",
|
|
"guides/mcp-server",
|
|
"guides/shacl-validation",
|
|
"guides/conflict-resolution",
|
|
"guides/change-management",
|
|
"guides/policy-engine",
|
|
"guides/visualization",
|
|
"guides/distance-intelligence",
|
|
"guides/graph-analytics",
|
|
"reference/context",
|
|
"reference/kg",
|
|
"reference/temporal",
|
|
"reference/distance",
|
|
"reference/semantic_extract",
|
|
"reference/reasoning",
|
|
"reference/ontology"
|
|
]
|
|
},
|
|
{
|
|
"group": "Storage",
|
|
"pages": [
|
|
"reference/vector_store",
|
|
"reference/graph_store",
|
|
"reference/triplet_store"
|
|
]
|
|
},
|
|
{
|
|
"group": "Data Pipeline",
|
|
"pages": [
|
|
"reference/ingest",
|
|
"reference/parse",
|
|
"reference/split",
|
|
"reference/normalize",
|
|
"reference/embeddings",
|
|
"reference/pipeline"
|
|
]
|
|
},
|
|
{
|
|
"group": "Quality & Provenance",
|
|
"pages": [
|
|
"reference/deduplication",
|
|
"reference/conflicts",
|
|
"reference/provenance",
|
|
"reference/change_management"
|
|
]
|
|
},
|
|
{
|
|
"group": "Output",
|
|
"pages": [
|
|
"reference/export",
|
|
"reference/visualization",
|
|
"reference/explorer"
|
|
]
|
|
},
|
|
{
|
|
"group": "Utilities",
|
|
"pages": [
|
|
"reference/llms",
|
|
"reference/seed",
|
|
"reference/evals",
|
|
"reference/utils",
|
|
"reference/core",
|
|
"reference/mcp_server"
|
|
]
|
|
}
|
|
]
|
|
},
|
|
{
|
|
"tab": "Cookbook",
|
|
"groups": [
|
|
{
|
|
"group": "Cookbook",
|
|
"pages": [
|
|
"cookbook"
|
|
]
|
|
}
|
|
]
|
|
},
|
|
{
|
|
"tab": "FAQ",
|
|
"groups": [
|
|
{
|
|
"group": "FAQ",
|
|
"pages": [
|
|
"faq"
|
|
]
|
|
},
|
|
{
|
|
"group": "Community",
|
|
"pages": [
|
|
"community",
|
|
"community-projects",
|
|
"contributing-guide",
|
|
"governance",
|
|
"citation",
|
|
"project-license"
|
|
]
|
|
}
|
|
]
|
|
},
|
|
{
|
|
"tab": "Changelog",
|
|
"href": "https://github.com/semantica-agi/semantica/releases"
|
|
}
|
|
]
|
|
},
|
|
"navbar": {
|
|
"links": [
|
|
{
|
|
"label": "Discord",
|
|
"href": "https://discord.gg/sV34vps5hH",
|
|
"icon": "discord"
|
|
},
|
|
{
|
|
"label": "GitHub",
|
|
"href": "https://github.com/semantica-agi/semantica",
|
|
"icon": "github"
|
|
},
|
|
{
|
|
"label": "PyPI",
|
|
"href": "https://pypi.org/project/semantica/",
|
|
"icon": "python"
|
|
},
|
|
{
|
|
"label": "Follow on X",
|
|
"href": "https://x.com/BuildSemantica",
|
|
"icon": "x-twitter"
|
|
}
|
|
],
|
|
"primary": {
|
|
"type": "button",
|
|
"label": "pip install semantica",
|
|
"href": "https://pypi.org/project/semantica/"
|
|
}
|
|
},
|
|
"search": {
|
|
"prompt": "Search docs...",
|
|
"location": "topbar"
|
|
},
|
|
"footer": {
|
|
"socials": {
|
|
"github": "https://github.com/semantica-agi/semantica",
|
|
"discord": "https://discord.gg/sV34vps5hH",
|
|
"x": "https://x.com/BuildSemantica"
|
|
}
|
|
}
|
|
}
|