mirror of
https://github.com/semantica-agi/semantica.git
synced 2026-09-12 04:01:35 +00:00
05aec1975d
* feat: add Redshift ingestor * fix(redshift): resolve 5 code-review issues in RedshiftIngestor Fix #1 — restore autocommit on caller-owned connections All four methods (ingest_table, ingest_query, get_table_schema, list_tables) now capture the connection's prior autocommit value before enabling it for read-only ingestion and restore it in the finally block when the connection was already open. Transient connections are still closed normally. Fix #2 — batch fetching: process rows per-batch, not after accumulating ingest_query's batch path now converts and extends data[] directly inside the fetch loop so raw tuples from each batch are eligible for GC before the next fetch. The documentation is updated to accurately state that batch_size controls driver round-trip size, not total memory. Fix #3 — schema env-var now honoured RedshiftIngestor.__init__ changes schema default from 'public' to None so the existing os.getenv('REDSHIFT_SCHEMA', 'public') fallback in the body is reached when no explicit argument is supplied. Fix #4 — db-redshift included in db-all aggregate pyproject.toml db-all now references db-redshift alongside the other database extras. Fix #5 — duplicate column labels no longer silently drop values _disambiguate_columns() appends _1, _2, ... suffixes to repeated cursor labels before dict(zip) so every value is retained. A new _make_row_dicts() helper encapsulates the zip+disambiguate pattern used by both _fetch_all and ingest_query. Tests: 22 new targeted tests added covering all five fixes. --------- Co-authored-by: Sameer Kadam <sameerkadam@Mac.lan> Co-authored-by: Kaif <98801504+KaifAhmad1@users.noreply.github.com>