docs: clarify Snowflake OAuth auth, add ArrowIngestor and non-re-exported ingestors (PR #808)

This commit is contained in:
Sameer6305
2026-07-28 15:00:02 +05:30
parent b105b8ea97
commit ce914b396a
3 changed files with 8 additions and 3 deletions
+4 -2
View File
@@ -382,16 +382,18 @@ table_lineage = databricks.get_table_lineage("customers", catalog="main", schema
snowflake = SnowflakeIngestor(
account="myaccount",
user="myuser",
password="mypassword", # or private_key / token for key-pair / OAuth auth
password="mypassword", # or private_key=... for key-pair; use authenticator="oauth", token=... for OAuth
warehouse="COMPUTE_WH",
database="MYDB",
)
orders = snowflake.ingest_table("ORDERS", limit=10_000)
```
> **Security Note:** Never hardcode credentials (`token`, `password`, `private_key`) in production code; pass them via environment variables (e.g., `DATABRICKS_TOKEN`, `SNOWFLAKE_PASSWORD`) or a secrets manager.
**Supported sources:** Local files (PDF, DOCX, PPTX, HTML, TXT, CSV, JSON, YAML, Excel, XML) · Web pages · RSS/Atom feeds · REST APIs · Databases (PostgreSQL, MySQL, SQLite, Oracle, SQL Server) · Parquet datasets · Databricks (Unity Catalog + Delta Lake) · Snowflake · Git repositories · Email (IMAP/POP3) · Message streams (Kafka, RabbitMQ, Kinesis, Pulsar) · MCP resources · Apache Arrow/Feather/IPC (`ArrowIngestor`)
Elasticsearch and Google Drive ingestion also ship (`ElasticIngestor`, `GDriveIngestor`) but aren't re-exported from the top-level `semantica.ingest` namespace yet — import them directly: `from semantica.ingest.elastic_ingestor import ElasticIngestor`.
DuckDB, Elasticsearch, Google Drive, HuggingFace, MongoDB, and Pandas ingestion also ship (`DuckDBIngestor`, `ElasticIngestor`, `GDriveIngestor`, `HuggingFaceIngestor`, `MongoIngestor`, `PandasIngestor`) but aren't re-exported from the top-level `semantica.ingest` namespace yet — import them directly: `from semantica.ingest.duckdb_ingestor import DuckDBIngestor`.
</details>
+3 -1
View File
@@ -340,7 +340,7 @@ from semantica.ingest import SnowflakeIngestor
snowflake = SnowflakeIngestor(
account="myaccount",
user="myuser",
password="mypassword", # or private_key / token for key-pair / OAuth auth
password="mypassword", # or private_key=... for key-pair; use authenticator="oauth", token=... for OAuth
warehouse="COMPUTE_WH",
database="ANALYTICS",
schema="PUBLIC",
@@ -380,6 +380,8 @@ print(f"Enterprise data graph: {graph.stats()['node_count']} nodes")
For authentication details (PAT vs. OAuth M2M for Databricks; password vs. key-pair vs. OAuth for Snowflake), schema/catalog introspection, and troubleshooting, see the dedicated [Databricks Integration](../integrations/databricks) and [Snowflake Integration](../integrations/snowflake) guides.
> **Security Note:** Never hardcode credentials (`token`, `password`, `private_key`) in production code; pass them via environment variables (e.g., `DATABRICKS_TOKEN`, `SNOWFLAKE_PASSWORD`) or a secrets manager.
## Combining All Five Sources
Once you have text from each source, `AgentContext.store()` accepts a flat list of strings. Semantica embeds and indexes them together — the context graph has no concept of which string came from which source unless you add metadata explicitly.
+1
View File
@@ -29,6 +29,7 @@ icon: "database"
| `SnowflakeIngestor` | Snowflake data warehouse queries and table exports |
| `DatabricksIngestor` | Databricks Unity Catalog metadata, Delta table queries, and lineage |
| `ParquetIngestor` | Apache Parquet files and partitioned datasets with column selection |
| `ArrowIngestor` | Apache Arrow IPC and Feather file processing |
| `XMLIngestor` | XXE-safe XML parsing with optional XSD schema validation |
| `EmailIngestor` | IMAP/POP3 email ingestion with attachment extraction |
| `OntologyIngestor` | OWL/RDF/Turtle ontology file ingestion |