feat(export): implement Neo4j Bulk CSV Exporter and update registry docs (#261) (#665)

* feat(export): implement Neo4j Bulk CSV Exporter and update registry docs (#261)

* fix(export): address review bugs in Neo4j CSV exporter

- _write_csv: filter **options to known csv.writer dialect params only,
  preventing TypeError when callers pass kwargs like delimiter= or encoding=
  that would reach csv.writer twice or as unknown arguments
- export_neo4j_csv: split kwargs into constructor-level init_params vs
  per-call call_kwargs before forwarding, eliminating the double-pass that
  caused dialect params to collide inside _write_csv
- _prepare_export: remove dead node_id_lookup dict that was built but never
  consumed by any caller
- export_knowledge_graph dispatch: drop the ambiguous "neo4j" format alias
  (kept "neo4j_csv" and "neo4j-csv"); "neo4j" conflicts with the codebase's
  established meaning of the live Bolt/Cypher store backend; add inline
  comment clarifying that file_path is treated as an output directory for
  this format
- export_usage.md: fix all three wrong API examples — constructor params
  node_label_sep/strict_validation corrected to label_separator/strict,
  non-existent nodes_path/rels_path kwargs removed, convenience-method
  example updated to show the correct positional output_dir argument

Co-Authored-By: KaifAhmad1 <kaif2208@gmail.com>

* docs(changelog): add Neo4j Bulk CSV Export entry for PR #665

Documents the new Neo4jCSVExporter feature contributed by @Luffy2208
and the five follow-up bug fixes (TypeError on dialect kwargs,
double-pass kwargs split, dead node_id_lookup removal, ambiguous
format="neo4j" alias removal, and wrong API examples in docs).

Co-Authored-By: KaifAhmad1 <kaif2208@gmail.com>

---------

Co-authored-by: KaifAhmad1 <kaifahmad087@gmail.com>
Co-authored-by: KaifAhmad1 <kaif2208@gmail.com>
This commit is contained in:
Luffy2208
2026-06-23 21:15:25 +05:30
committed by GitHub
co-authored by KaifAhmad1 KaifAhmad1
parent a450f9eddc
commit f3dc2a449d
7 changed files with 1410 additions and 14 deletions
+33
View File
@@ -11,6 +11,39 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
### Added
- **Neo4j Bulk CSV Export** (#665) by @Luffy2208
- Added `Neo4jCSVExporter` for generating Neo4j bulk-import CSV files compatible with `neo4j-admin database import`
- Produces deterministic `nodes.csv` and `relationships.csv` with stable node IDs — reuses existing graph IDs or derives reproducible SHA-256 content-based IDs when none are present
- Multi-label support via Neo4j `:LABEL` convention with configurable `label_separator` (default `;`)
- Alphabetically sorted property columns and deterministic row ordering for reproducible output across permuted inputs
- Relationship endpoint resolution: aliases (`name`, `text`, `label`) automatically mapped to stable node IDs
- Nested property serialisation to canonical JSON; flat scalar values written directly
- `dry_run()` method for pre-flight CSV validation without writing files
- `validate_export()` for post-write integrity checks (unique `:id`, consistent column widths, valid endpoint references)
- `export_nodes()` and `export_relationships()` for partial exports
- `strict=True` mode raises `ValidationError` on unresolved relationship endpoints
- `export_neo4j_csv()` convenience function and `format="neo4j_csv"` / `format="neo4j-csv"` dispatch in `export_knowledge_graph()`
- Registry integration under the `neo4j_csv` task namespace
- Documentation added to `semantica/export/export_usage.md` with usage examples, mapping assumptions, and `neo4j-admin` import command
- 13 tests covering headers, node/relationship CSV structure, multi-label, missing properties, deterministic output, CSV quoting/escaping, Unicode, empty graphs, dry-run, duplicate ID detection, ambiguous alias handling, nested property serialisation, and `KnowledgeGraph` integration
### Fixed
- **Neo4j CSV exporter `_write_csv` crashed with `TypeError` on dialect kwargs** (#665) by @KaifAhmad1
- Passing `delimiter=`, `encoding=`, or any caller kwarg to `export_neo4j_csv` caused `csv.writer` to receive unknown or duplicate keyword arguments; `_write_csv` now whitelists only valid `csv.writer` dialect params (`quotechar`, `doublequote`, `skipinitialspace`, `escapechar`, `strict`)
- **`export_neo4j_csv` double-passed kwargs to both the constructor and `export()`** (#665) by @KaifAhmad1
- Constructor-level settings (`node_file_name`, `relationship_file_name`, `encoding`, `delimiter`, `label_separator`, `strict`) were merged into config for the constructor then re-forwarded as `**kwargs` to `export_knowledge_graph`, causing dialect params to collide; kwargs are now split into `init_kwargs` and `call_kwargs` before forwarding
- **Dead `node_id_lookup` dict removed from `_prepare_export`** (#665) by @KaifAhmad1
- The `{original_index → stable_id}` mapping was built on every export but never consumed; removed to avoid misleading future readers
- **Dropped ambiguous `format="neo4j"` alias from `export_knowledge_graph` dispatch** (#665) by @KaifAhmad1
- `"neo4j"` is used throughout the codebase to identify the live Bolt/Cypher graph store backend; routing it silently to the offline bulk-CSV exporter would have confused callers; only `"neo4j_csv"` and `"neo4j-csv"` are accepted
- **`export_usage.md` documented non-existent constructor and function parameters** (#665) by @KaifAhmad1
- Examples showed `node_label_sep` (correct: `label_separator`), `strict_validation` (correct: `strict`), and `nodes_path`/`rels_path` kwargs that do not exist; all three examples corrected to match the actual API
- **Public API Ingestion Support** (#602) by @Luffy2208
- Added `PublicAPIIngestor` class built on top of `RESTIngestor` for credential-free REST endpoints
- Added `PublicAPIExample` and `PublicAPIExamples` catalog with 6 pre-configured no-auth examples:
+14 -1
View File
@@ -61,6 +61,14 @@ CSV Export:
- Multi-file Export: Knowledge graph split into multiple CSV files
(entities, relationships)
Neo4j Bulk CSV Export:
- Neo4j Admin Import CSV: Deterministic nodes.csv and relationships.csv
files compatible with neo4j-admin database import
- Stable Node IDs: Reuse existing graph IDs or derive deterministic IDs
from node contents
- Label and Property Mapping: Multi-label :LABEL values and sorted
property columns
Graph Export:
- GraphML Serialization: GraphML format generation for graph visualization tools
- GEXF Serialization: GEXF format generation for Gephi and similar tools
@@ -124,6 +132,7 @@ Main Classes:
- RDFExporter: RDF format export (Turtle, RDF/XML, JSON-LD)
- JSONExporter: JSON and JSON-LD format export
- CSVExporter: CSV format export for tabular data
- Neo4jCSVExporter: Neo4j bulk import CSV export
- ParquetExporter: Parquet format export for analytics and data warehousing
- GraphExporter: Graph format export (GraphML, GEXF, DOT)
- YAMLExporter: YAML format export for semantic networks
@@ -162,8 +171,8 @@ License: MIT
"""
from .arango_aql_exporter import ArangoAQLExporter
from .distance_exporter import DistanceExporter
from .config import ExportConfig, export_config
from .distance_exporter import DistanceExporter
try:
from .arrow_exporter import ArrowExporter
@@ -188,6 +197,7 @@ from .methods import (
export_graph,
export_json,
export_lpg,
export_neo4j_csv,
export_owl,
export_parquet,
export_rdf,
@@ -197,6 +207,7 @@ from .methods import (
get_export_method,
list_available_methods,
)
from .neo4j_csv_exporter import Neo4jCSVExporter
from .owl_exporter import OWLExporter
try:
@@ -228,6 +239,7 @@ __all__ = [
"NamespaceManager",
"JSONExporter",
"CSVExporter",
"Neo4jCSVExporter",
"ParquetExporter",
"GraphExporter",
"SemanticNetworkYAMLExporter",
@@ -249,6 +261,7 @@ __all__ = [
"export_owl",
"export_vector",
"export_lpg",
"export_neo4j_csv",
"export_arango",
"generate_report",
"get_export_method",
+68 -6
View File
@@ -13,6 +13,8 @@ This guide demonstrates how to use the export module for exporting knowledge gra
7. [OWL Export](#owl-export)
8. [Vector Export](#vector-export)
9. [LPG Export](#lpg-export)
- [Cypher Query Export](#cypher-query-export)
- [Neo4j Bulk CSV Export](#neo4j-bulk-csv-export)
10. [Report Generation](#report-generation)
11. [Knowledge Graph Export](#knowledge-graph-export)
12. [Using Methods](#using-methods)
@@ -442,6 +444,63 @@ exporter.export_knowledge_graph(kg, "graph.cypher")
# mgconsole < graph.cypher
```
### Neo4j Bulk CSV Export
The `Neo4jCSVExporter` generates node and relationship CSV files specifically structured for Neo4j's high-performance bulk import utility (`neo4j-admin database import`).
#### Basic Usage
```python
from semantica.export import Neo4jCSVExporter
# Initialize the exporter
exporter = Neo4jCSVExporter(
label_separator=";", # Separator for multi-label nodes (default: ";")
strict=True # Raise on unresolved relationship endpoints (default: True)
)
# Export a knowledge graph — output_dir receives nodes.csv and relationships.csv
exporter.export_knowledge_graph(kg, "neo4j_import/")
```
#### Convenience Method
You can also use the convenience wrapper `export_neo4j_csv`:
```python
from semantica.export.methods import export_neo4j_csv
export_neo4j_csv(kg, "neo4j_import/")
```
Pass `validate=True` to run a post-export integrity check before returning:
```python
export_neo4j_csv(kg, "neo4j_import/", validate=True)
```
#### Importing into Neo4j
Once the CSV files are generated, they can be imported into a new Neo4j database using the `neo4j-admin database import` command:
```bash
neo4j-admin database import full \
--nodes=nodes.csv \
--relationships=relationships.csv \
neo4j
```
#### Mapping Assumptions & Rules
- **Header Specification**:
- Node CSV headers are generated as `:id` (node identifier) and `:LABEL` (labels).
- Relationship CSV headers are generated as `:START_ID` (source node), `:END_ID` (target node), and `:TYPE` (relationship type).
- Custom attributes/properties are written as standard columns.
- **Node Labels**: Supports multiple labels per node. Labels are serialized into a single string column using the configured `label_separator` (default `;`).
- **Stable Node IDs**: Reuses `id` (or `entity_id` / `node_id`) from node dictionaries. If a node is missing an ID, a stable content-derived ID is generated via a SHA-256 hash of its properties, ensuring reproducibility.
- **Relationship Endpoint Resolution**: If a relationship refers to nodes by their `name`, `text`, or `label` rather than their exact `id`, the exporter resolves these aliases automatically to their stable node IDs to guarantee that `:START_ID` and `:END_ID` strictly match existing node IDs.
- **Property Serialization**: Flat values (strings, numbers, booleans) are serialized directly. Nested properties (e.g. dicts, lists, sets) are serialized into a canonical JSON representation.
## Report Generation
### HTML Report
@@ -553,7 +612,8 @@ from semantica.export.methods import (
export_yaml,
export_owl,
export_vector,
export_lpg
export_lpg,
export_neo4j_csv
)
# RDF export
@@ -579,6 +639,9 @@ export_vector(vectors, "vectors.json", format="json")
# LPG export
export_lpg(kg, "graph.cypher", method="cypher")
# Neo4j CSV bulk export
export_neo4j_csv(kg, nodes_path="nodes.csv", rels_path="relationships.csv")
```
## Using Registry
@@ -799,16 +862,16 @@ from semantica.export.methods import export_json
def custom_jsonld_export(data, file_path, **kwargs):
"""Custom JSON-LD export with specific context."""
from semantica.export import JSONExporter
exporter = JSONExporter(format="json-ld")
# Add custom context
if isinstance(data, dict) and "@context" not in data:
data["@context"] = {
"@vocab": "http://example.org/vocab#",
"ex": "http://example.org/ns#"
}
exporter.export(data, file_path, format="json-ld", **kwargs)
# Register custom method
@@ -921,7 +984,7 @@ for format_name, file_path, export_func, kwargs in export_configs:
2. **Parallel Export**: Export to multiple formats in parallel
```python
from concurrent.futures import ThreadPoolExecutor
with ThreadPoolExecutor() as executor:
executor.submit(export_json, kg, "output.json")
executor.submit(export_rdf, kg, "output.ttl", format="turtle")
@@ -954,4 +1017,3 @@ for format_name, file_path, export_func, kwargs in export_configs:
with gzip.open("output.json.gz", "wb") as f_out:
f_out.writelines(f_in)
```
+76 -2
View File
@@ -47,6 +47,9 @@ LPG Export:
- "cypher": Cypher query format for Neo4j/Memgraph
- "lpg": Labeled Property Graph format
Neo4j Bulk CSV Export:
- "neo4j_csv": CSV files for ``neo4j-admin database import``
Report Generation:
- "html": HTML report format
- "markdown": Markdown report format
@@ -168,6 +171,7 @@ from .csv_exporter import CSVExporter
from .graph_exporter import GraphExporter
from .json_exporter import JSONExporter
from .lpg_exporter import LPGExporter
from .neo4j_csv_exporter import Neo4jCSVExporter
from .owl_exporter import OWLExporter
try:
@@ -693,6 +697,63 @@ def export_lpg(
raise
def export_neo4j_csv(
knowledge_graph: Any,
output_dir: Union[str, Path],
method: str = "default",
**kwargs,
) -> Dict[str, Path]:
"""
Export knowledge graph to Neo4j bulk-import CSV files.
This wrapper writes deterministic ``nodes.csv`` and ``relationships.csv``
files that can be imported with ``neo4j-admin database import``.
Args:
knowledge_graph: Knowledge graph dictionary or object with graph attributes.
output_dir: Output directory for Neo4j CSV files.
method: Export method (default: "default").
**kwargs: Additional options passed to ``Neo4jCSVExporter``.
Returns:
Mapping with ``"nodes"`` and ``"relationships"`` output paths.
"""
custom_method = method_registry.get("neo4j_csv", method)
if custom_method and custom_method is not export_neo4j_csv:
try:
return custom_method(knowledge_graph, output_dir, **kwargs)
except Exception as e:
logger.warning(
f"Custom method {method} failed: {e}, falling back to default"
)
try:
config = export_config.get_method_config("neo4j_csv")
# Split kwargs: constructor-level settings vs per-call export() options.
# Passing all kwargs to both the constructor AND export_knowledge_graph
# causes dialect params like delimiter to reach csv.writer twice.
_init_params = frozenset({
"node_file_name",
"relationship_file_name",
"encoding",
"delimiter",
"label_separator",
"strict",
"config",
})
init_kwargs = {k: v for k, v in kwargs.items() if k in _init_params}
call_kwargs = {k: v for k, v in kwargs.items() if k not in _init_params}
config.update(init_kwargs)
exporter = Neo4jCSVExporter(**config)
return exporter.export_knowledge_graph(knowledge_graph, output_dir, **call_kwargs)
except Exception as e:
logger.error(f"Failed to export Neo4j CSV: {e}")
raise
def export_arango(
knowledge_graph: Dict[str, Any],
file_path: Union[str, Path],
@@ -713,12 +774,18 @@ def export_arango(
- vertex_collection: Override default vertex collection name
- edge_collection: Override default edge collection name
- batch_size: Batch size for INSERT operations
- include_collection_creation: Whether to include collection creation statements
- include_collection_creation: Whether to include collection
creation statements
Examples:
>>> from semantica.export.methods import export_arango
>>> export_arango(kg, "output.aql")
>>> export_arango(kg, "graph.aql", vertex_collection="nodes", edge_collection="links")
>>> export_arango(
... kg,
... "graph.aql",
... vertex_collection="nodes",
... edge_collection="links",
... )
"""
# Check for custom method in registry
custom_method = method_registry.get("arango", method)
@@ -872,6 +939,10 @@ def export_knowledge_graph(
export_owl(knowledge_graph, file_path, format=format, method=method, **kwargs)
elif format == "cypher":
export_lpg(knowledge_graph, file_path, method=method, **kwargs)
elif format in ["neo4j_csv", "neo4j-csv"]:
# file_path is treated as the output directory; nodes.csv and
# relationships.csv are written inside it.
export_neo4j_csv(knowledge_graph, file_path, method=method, **kwargs)
elif format == "aql":
export_arango(knowledge_graph, file_path, method=method, **kwargs)
elif format == "parquet":
@@ -944,6 +1015,9 @@ method_registry.register("vector", "json", export_vector)
method_registry.register("vector", "numpy", export_vector)
method_registry.register("lpg", "default", export_lpg)
method_registry.register("lpg", "cypher", export_lpg)
method_registry.register("neo4j_csv", "default", export_neo4j_csv)
method_registry.register("neo4j_csv", "neo4j_csv", export_neo4j_csv)
method_registry.register("neo4j_csv", "neo4j-csv", export_neo4j_csv)
method_registry.register("arango", "default", export_arango)
method_registry.register("arango", "aql", export_arango)
method_registry.register("report", "default", generate_report)
+905
View File
@@ -0,0 +1,905 @@
"""
Neo4j bulk CSV export for ``neo4j-admin database import``.
This module writes Semantica knowledge graphs to the two CSV files expected by
Neo4j's offline bulk importer:
``nodes.csv``::
:id,:LABEL,name,type
1,Person;Engineer,Alice,user
``relationships.csv``::
:START_ID,:END_ID,:TYPE,since
1,2,KNOWS,2024
Example import command::
neo4j-admin database import full \\
--nodes=nodes.csv \\
--relationships=relationships.csv \\
neo4j
The exporter accepts the canonical Semantica ``{"entities": ..., "relationships": ...}``
shape, graph-style ``{"nodes": ..., "edges": ...}`` dictionaries, and objects exposing
``.entities``/``.relationships`` or ``.nodes``/``.edges`` attributes. Existing node IDs
are reused when present; otherwise deterministic IDs are derived from node contents.
"""
from __future__ import annotations
import csv
import hashlib
import json
from dataclasses import asdict, is_dataclass
from pathlib import Path
from typing import Any, Dict, Iterable, List, Optional, Sequence, Union
from ..utils.exceptions import ProcessingError, ValidationError
from ..utils.helpers import ensure_directory
from ..utils.logging import get_logger
from ..utils.progress_tracker import get_progress_tracker
class Neo4jCSVExporter:
"""
Export knowledge graphs to Neo4j bulk-import CSV files.
The exporter creates deterministic ``nodes.csv`` and ``relationships.csv`` files
under the requested output directory. Node property columns and relationship
property columns are generated from the graph data and sorted alphabetically.
Args:
node_file_name: File name for node rows (default: ``nodes.csv``).
relationship_file_name: File name for relationship rows
(default: ``relationships.csv``).
encoding: Output file encoding (default: ``utf-8``).
delimiter: CSV delimiter (default: ``,``).
label_separator: Neo4j label separator for ``:LABEL`` values
(default: ``;``).
strict: If true, raise on relationships that cannot be connected to an
exported node. If false, unresolved endpoint values are written as-is.
config: Optional configuration dictionary, merged with keyword overrides.
Example:
>>> from semantica.export import Neo4jCSVExporter
>>> kg = {
... "entities": [{"id": "1", "labels": ["Person"], "name": "Alice"}],
... "relationships": [],
... }
>>> Neo4jCSVExporter().export_knowledge_graph(kg, "neo4j_import")
"""
NODE_ID_HEADER = ":id"
NODE_LABEL_HEADER = ":LABEL"
REL_START_HEADER = ":START_ID"
REL_END_HEADER = ":END_ID"
REL_TYPE_HEADER = ":TYPE"
_NODE_ID_KEYS = ("id", "entity_id", "node_id", "_id", "key", "uid")
_REL_ID_KEYS = ("id", "relationship_id", "edge_id", "_id", "key", "uid")
_SOURCE_KEYS = (
"source",
"source_id",
"from",
"from_id",
"subject",
"start",
"start_id",
)
_TARGET_KEYS = (
"target",
"target_id",
"to",
"to_id",
"object",
"end",
"end_id",
)
_REL_TYPE_KEYS = (
"type",
"relationship_type",
"relation_type",
"predicate",
"label",
)
_NODE_LABEL_KEYS = ("labels", "node_labels")
_NODE_STRUCTURAL_KEYS = set(_NODE_ID_KEYS) | set(_NODE_LABEL_KEYS)
_REL_STRUCTURAL_KEYS = (
set(_REL_ID_KEYS) | set(_SOURCE_KEYS) | set(_TARGET_KEYS) | set(_REL_TYPE_KEYS)
)
def __init__(
self,
node_file_name: str = "nodes.csv",
relationship_file_name: str = "relationships.csv",
encoding: str = "utf-8",
delimiter: str = ",",
label_separator: str = ";",
strict: bool = True,
config: Optional[Dict[str, Any]] = None,
**kwargs: Any,
) -> None:
self.logger = get_logger("neo4j_csv_exporter")
self.config = config or {}
self.config.update(kwargs)
self.node_file_name = self.config.get("node_file_name", node_file_name)
self.relationship_file_name = self.config.get(
"relationship_file_name", relationship_file_name
)
self.encoding = self.config.get("encoding", encoding)
self.delimiter = self.config.get("delimiter", delimiter)
self.label_separator = self.config.get("label_separator", label_separator)
self.strict = bool(self.config.get("strict", strict))
self._validate_file_name(self.node_file_name, "node_file_name")
self._validate_file_name(self.relationship_file_name, "relationship_file_name")
if not self.label_separator:
raise ValueError("label_separator cannot be empty")
if not self.delimiter:
raise ValueError("delimiter cannot be empty")
self.progress_tracker = get_progress_tracker()
if not self.progress_tracker.enabled:
self.progress_tracker.enabled = True
self.logger.debug(
"Neo4j CSV exporter initialized: nodes=%s, relationships=%s",
self.node_file_name,
self.relationship_file_name,
)
def export(
self,
knowledge_graph: Any,
output_dir: Union[str, Path],
validate: bool = False,
**options: Any,
) -> Dict[str, Path]:
"""
Export a graph to Neo4j bulk CSV files.
Args:
knowledge_graph: Graph dictionary or object with graph attributes.
output_dir: Directory where ``nodes.csv`` and ``relationships.csv``
will be written.
validate: If true, validate written files and raise ``ValidationError``
if they do not meet basic Neo4j import expectations.
**options: Optional overrides for ``node_file_name``,
``relationship_file_name``, ``strict``, or CSV writer options.
Returns:
Mapping with ``"nodes"`` and ``"relationships"`` output paths.
"""
output_dir = Path(output_dir)
ensure_directory(output_dir)
node_file_name = options.pop("node_file_name", self.node_file_name)
relationship_file_name = options.pop(
"relationship_file_name", self.relationship_file_name
)
strict = bool(options.pop("strict", self.strict))
self._validate_file_name(node_file_name, "node_file_name")
self._validate_file_name(relationship_file_name, "relationship_file_name")
nodes_path = output_dir / node_file_name
relationships_path = output_dir / relationship_file_name
tracking_id = self.progress_tracker.start_tracking(
file=str(output_dir),
module="export",
submodule="Neo4jCSVExporter",
message=f"Exporting graph to Neo4j CSV directory: {output_dir}",
)
try:
prepared = self._prepare_export(knowledge_graph, strict=strict)
self._write_csv(
nodes_path,
[self.NODE_ID_HEADER, self.NODE_LABEL_HEADER]
+ prepared["node_columns"],
prepared["node_rows"],
**options,
)
self._write_csv(
relationships_path,
[
self.REL_START_HEADER,
self.REL_END_HEADER,
self.REL_TYPE_HEADER,
]
+ prepared["relationship_columns"],
prepared["relationship_rows"],
**options,
)
exported = {"nodes": nodes_path, "relationships": relationships_path}
if validate:
validation = self.validate_export(
output_dir,
node_file_name=node_file_name,
relationship_file_name=relationship_file_name,
)
if not validation["valid"]:
raise ValidationError(
"Neo4j CSV export validation failed",
errors=validation["errors"],
)
self.progress_tracker.stop_tracking(
tracking_id,
status="completed",
message=(
f"Exported {len(prepared['node_rows'])} node row(s) and "
f"{len(prepared['relationship_rows'])} relationship row(s)"
),
)
self.logger.info("Exported Neo4j CSV files to: %s", output_dir)
return exported
except Exception as exc:
self.progress_tracker.stop_tracking(
tracking_id, status="failed", message=str(exc)
)
if isinstance(exc, (ValidationError, ProcessingError)):
raise
raise ProcessingError(f"Failed to export Neo4j CSV files: {exc}") from exc
def export_knowledge_graph(
self,
knowledge_graph: Any,
output_dir: Union[str, Path],
**options: Any,
) -> Dict[str, Path]:
"""Convenience wrapper around :meth:`export`."""
return self.export(knowledge_graph, output_dir, **options)
def export_nodes(
self,
nodes: Sequence[Any],
output_dir: Union[str, Path],
**options: Any,
) -> Dict[str, Path]:
"""Export node rows with an empty relationship file."""
return self.export(
{"entities": list(nodes), "relationships": []}, output_dir, **options
)
def export_relationships(
self,
relationships: Sequence[Any],
output_dir: Union[str, Path],
nodes: Optional[Sequence[Any]] = None,
**options: Any,
) -> Dict[str, Path]:
"""Export relationship rows, optionally with the node rows they reference."""
return self.export(
{"entities": list(nodes or []), "relationships": list(relationships)},
output_dir,
**options,
)
def dry_run(self, knowledge_graph: Any, **options: Any) -> Dict[str, Any]:
"""
Prepare and validate CSV content without writing files.
Returns a summary containing headers, row counts, and validation errors.
This is useful in CI or before running ``neo4j-admin database import``.
"""
strict = bool(options.pop("strict", self.strict))
prepared = self._prepare_export(knowledge_graph, strict=strict)
validation = self._validate_prepared(prepared)
return {
"valid": validation["valid"],
"errors": validation["errors"],
"node_header": [self.NODE_ID_HEADER, self.NODE_LABEL_HEADER]
+ prepared["node_columns"],
"relationship_header": [
self.REL_START_HEADER,
self.REL_END_HEADER,
self.REL_TYPE_HEADER,
]
+ prepared["relationship_columns"],
"node_count": len(prepared["node_rows"]),
"relationship_count": len(prepared["relationship_rows"]),
}
def validate_export(
self,
output_dir: Union[str, Path],
node_file_name: Optional[str] = None,
relationship_file_name: Optional[str] = None,
) -> Dict[str, Any]:
"""
Validate written CSV files against core Neo4j import expectations.
The validation checks required headers, consistent row widths, unique node
IDs, non-empty relationship endpoints/types, and relationship endpoint
references to exported node IDs.
"""
output_dir = Path(output_dir)
nodes_path = output_dir / (node_file_name or self.node_file_name)
relationships_path = output_dir / (
relationship_file_name or self.relationship_file_name
)
errors: List[str] = []
node_ids: set = set()
if not nodes_path.exists():
errors.append(f"Missing node CSV file: {nodes_path}")
else:
try:
with open(
nodes_path, "r", encoding=self.encoding, newline=""
) as handle:
reader = csv.reader(handle, delimiter=self.delimiter)
rows = list(reader)
if not rows:
errors.append("Node CSV is empty")
else:
header = rows[0]
if len(header) < 2 or header[:2] != [
self.NODE_ID_HEADER,
self.NODE_LABEL_HEADER,
]:
errors.append("Node CSV header must start with ':id,:LABEL'")
for row_number, row in enumerate(rows[1:], start=2):
if len(row) != len(header):
errors.append(
f"Node row {row_number} has {len(row)} columns; "
f"expected {len(header)}"
)
continue
node_id = row[0]
if not node_id:
errors.append(f"Node row {row_number} has an empty :id")
elif node_id in node_ids:
errors.append(f"Duplicate node :id {node_id!r}")
else:
node_ids.add(node_id)
except csv.Error as exc:
errors.append(f"Node CSV parse error: {exc}")
if not relationships_path.exists():
errors.append(f"Missing relationship CSV file: {relationships_path}")
else:
try:
with open(
relationships_path,
"r",
encoding=self.encoding,
newline="",
) as handle:
reader = csv.reader(handle, delimiter=self.delimiter)
rows = list(reader)
if not rows:
errors.append("Relationship CSV is empty")
else:
header = rows[0]
expected = [
self.REL_START_HEADER,
self.REL_END_HEADER,
self.REL_TYPE_HEADER,
]
if len(header) < 3 or header[:3] != expected:
errors.append(
"Relationship CSV header must start with "
"':START_ID,:END_ID,:TYPE'"
)
for row_number, row in enumerate(rows[1:], start=2):
if len(row) != len(header):
errors.append(
f"Relationship row {row_number} has {len(row)} "
f"columns; expected {len(header)}"
)
continue
start_id, end_id, rel_type = row[:3]
if not start_id or not end_id:
errors.append(
f"Relationship row {row_number} has an empty endpoint"
)
if not rel_type:
errors.append(
f"Relationship row {row_number} has an empty :TYPE"
)
if node_ids and start_id and start_id not in node_ids:
errors.append(
f"Relationship row {row_number} references unknown "
f":START_ID {start_id!r}"
)
if node_ids and end_id and end_id not in node_ids:
errors.append(
f"Relationship row {row_number} references unknown "
f":END_ID {end_id!r}"
)
except csv.Error as exc:
errors.append(f"Relationship CSV parse error: {exc}")
return {"valid": not errors, "errors": errors}
def _prepare_export(self, graph: Any, strict: bool) -> Dict[str, Any]:
normalized = self._normalize_graph(graph)
node_infos = self._prepare_nodes(normalized["nodes"])
node_columns = sorted(
{key for node in node_infos for key in node["properties"].keys()}
)
alias_lookup = self._build_alias_lookup(node_infos)
node_id_set = {node["id"] for node in node_infos}
relationship_infos = self._prepare_relationships(
normalized["relationships"],
alias_lookup=alias_lookup,
node_id_set=node_id_set,
strict=strict,
)
relationship_columns = sorted(
{
key
for relationship in relationship_infos
for key in relationship["properties"].keys()
}
)
node_rows = []
for node in sorted(node_infos, key=lambda item: item["id"]):
node_rows.append(
[node["id"], self.label_separator.join(node["labels"])]
+ [
self._serialize_value(node["properties"].get(column))
for column in node_columns
]
)
relationship_rows = []
for relationship in sorted(
relationship_infos,
key=lambda item: (
item["start_id"],
item["end_id"],
item["type"],
self._canonical_json(item["properties"]),
),
):
relationship_rows.append(
[
relationship["start_id"],
relationship["end_id"],
relationship["type"],
]
+ [
self._serialize_value(relationship["properties"].get(column))
for column in relationship_columns
]
)
prepared = {
"node_rows": node_rows,
"relationship_rows": relationship_rows,
"node_columns": node_columns,
"relationship_columns": relationship_columns,
}
validation = self._validate_prepared(prepared)
if not validation["valid"]:
raise ValidationError(
"Neo4j CSV export data is invalid",
errors=validation["errors"],
)
return prepared
def _normalize_graph(self, graph: Any) -> Dict[str, List[Dict[str, Any]]]:
if isinstance(graph, dict):
nodes = graph.get("nodes") or graph.get("entities") or []
relationships = graph.get("edges") or graph.get("relationships") or []
else:
nodes = getattr(graph, "nodes", None)
if nodes is None:
nodes = getattr(graph, "entities", None)
relationships = getattr(graph, "edges", None)
if relationships is None:
relationships = getattr(graph, "relationships", None)
if nodes is None and relationships is None:
raise ProcessingError(
f"Cannot export object of type '{type(graph).__name__}': "
"expected a dict with 'entities'/'relationships' or "
"'nodes'/'edges', or an object exposing equivalent attributes."
)
return {
"nodes": [self._record_to_dict(node) for node in list(nodes or [])],
"relationships": [
self._record_to_dict(relationship)
for relationship in list(relationships or [])
],
}
def _prepare_nodes(self, nodes: Sequence[Dict[str, Any]]) -> List[Dict[str, Any]]:
explicit_ids: Dict[str, int] = {}
node_infos: List[Dict[str, Any]] = []
for index, node in enumerate(nodes):
explicit_id = self._first_value(node, self._NODE_ID_KEYS)
has_explicit_id = explicit_id is not None and str(explicit_id) != ""
base_id = (
str(explicit_id) if has_explicit_id else self._derived_node_id(node)
)
if has_explicit_id:
if base_id in explicit_ids:
raise ValidationError(
f"Duplicate node ID {base_id!r}",
first_index=explicit_ids[base_id],
duplicate_index=index,
)
explicit_ids[base_id] = index
node_infos.append(
{
"base_id": base_id,
"has_explicit_id": has_explicit_id,
"original_index": index,
"record": node,
"canonical": self._canonical_json(node),
"labels": self._extract_node_labels(node),
"properties": self._extract_properties(
node, structural_keys=self._NODE_STRUCTURAL_KEYS
),
}
)
used_ids = set(explicit_ids.keys())
generated_counts: Dict[str, int] = {}
for node in sorted(
node_infos,
key=lambda item: (
item["base_id"],
item["canonical"],
item["original_index"],
),
):
if node["has_explicit_id"]:
node["id"] = node["base_id"]
continue
base_id = node["base_id"]
generated_counts[base_id] = generated_counts.get(base_id, 0) + 1
suffix = generated_counts[base_id]
candidate = base_id if suffix == 1 else f"{base_id}_{suffix}"
while candidate in used_ids:
suffix += 1
generated_counts[base_id] = suffix
candidate = f"{base_id}_{suffix}"
used_ids.add(candidate)
node["id"] = candidate
return node_infos
def _prepare_relationships(
self,
relationships: Sequence[Dict[str, Any]],
alias_lookup: Dict[str, str],
node_id_set: set,
strict: bool,
) -> List[Dict[str, Any]]:
relationship_infos = []
for index, relationship in enumerate(relationships):
raw_start = self._first_value(relationship, self._SOURCE_KEYS)
raw_end = self._first_value(relationship, self._TARGET_KEYS)
if raw_start is None or raw_end is None:
raise ValidationError(
f"Relationship {index} is missing source or target",
relationship=relationship,
)
start_id = self._resolve_endpoint(raw_start, alias_lookup)
end_id = self._resolve_endpoint(raw_end, alias_lookup)
rel_type = self._extract_relationship_type(relationship)
unresolved = [
endpoint
for endpoint in (start_id, end_id)
if endpoint not in node_id_set
]
if unresolved and strict:
raise ValidationError(
f"Relationship {index} references unknown node ID(s): "
f"{', '.join(repr(endpoint) for endpoint in unresolved)}",
relationship=relationship,
)
relationship_infos.append(
{
"start_id": start_id,
"end_id": end_id,
"type": rel_type,
"properties": self._extract_properties(
relationship, structural_keys=self._REL_STRUCTURAL_KEYS
),
}
)
return relationship_infos
def _build_alias_lookup(
self, node_infos: Sequence[Dict[str, Any]]
) -> Dict[str, str]:
aliases: Dict[str, str] = {}
ambiguous = set()
for node in node_infos:
for alias in self._node_aliases(node["record"], node["id"]):
if alias in aliases and aliases[alias] != node["id"]:
ambiguous.add(alias)
else:
aliases[alias] = node["id"]
for alias in ambiguous:
aliases.pop(alias, None)
return aliases
def _node_aliases(self, node: Dict[str, Any], stable_id: str) -> Iterable[str]:
raw_aliases: List[Any] = [stable_id]
raw_aliases.extend(node.get(key) for key in self._NODE_ID_KEYS)
raw_aliases.extend(
node.get(key)
for key in ("name", "text", "label")
if not isinstance(node.get(key), (list, tuple, set, dict))
)
nested_properties = node.get("properties")
if isinstance(nested_properties, dict):
raw_aliases.extend(
nested_properties.get(key)
for key in self._NODE_ID_KEYS + ("name", "text", "label")
)
for alias in raw_aliases:
if alias is not None and str(alias) != "":
yield str(alias)
def _extract_node_labels(self, node: Dict[str, Any]) -> List[str]:
label_values: List[Any] = []
for key in self._NODE_LABEL_KEYS:
if key in node:
label_values.extend(self._coerce_sequence(node[key]))
label_field = node.get("label")
if isinstance(label_field, (list, tuple, set)):
label_values.extend(self._coerce_sequence(label_field))
if not label_values:
node_type = node.get("type") or node.get("entity_type")
label_values.extend(self._coerce_sequence(node_type))
labels: List[str] = []
seen = set()
for label in label_values:
token = self._sanitize_token(str(label), fallback="")
if token and token not in seen:
labels.append(token)
seen.add(token)
return labels or ["Entity"]
def _extract_relationship_type(self, relationship: Dict[str, Any]) -> str:
rel_type = self._first_value(relationship, self._REL_TYPE_KEYS)
return self._sanitize_token(str(rel_type or ""), fallback="RELATED_TO")
def _extract_properties(
self, record: Dict[str, Any], structural_keys: set
) -> Dict[str, Any]:
properties: Dict[str, Any] = {}
nested_properties = record.get("properties")
if isinstance(nested_properties, dict):
for key, value in nested_properties.items():
properties[str(key)] = value
for key, value in record.items():
if key in structural_keys or key == "properties":
continue
properties[str(key)] = value
return properties
def _resolve_endpoint(self, value: Any, alias_lookup: Dict[str, str]) -> str:
if isinstance(value, dict) or is_dataclass(value):
endpoint_record = self._record_to_dict(value)
endpoint_id = self._first_value(endpoint_record, self._NODE_ID_KEYS)
if endpoint_id is None:
endpoint_id = (
endpoint_record.get("name")
or endpoint_record.get("text")
or endpoint_record.get("label")
)
if endpoint_id is None:
endpoint_id = self._derived_node_id(endpoint_record)
value = endpoint_id
elif not isinstance(value, str) and hasattr(value, "__dict__"):
return self._resolve_endpoint(self._record_to_dict(value), alias_lookup)
endpoint = str(value)
return alias_lookup.get(endpoint, endpoint)
def _first_value(self, record: Dict[str, Any], keys: Sequence[str]) -> Any:
for key in keys:
if key in record and record[key] is not None:
return record[key]
return None
def _record_to_dict(self, record: Any) -> Dict[str, Any]:
if isinstance(record, dict):
return dict(record)
if is_dataclass(record):
return asdict(record)
if hasattr(record, "__dict__"):
return {
key: value
for key, value in vars(record).items()
if not key.startswith("_")
}
raise ValidationError(
f"Expected graph records to be dictionaries or objects, got "
f"{type(record).__name__}"
)
def _derived_node_id(self, node: Dict[str, Any]) -> str:
digest = hashlib.sha256(self._canonical_json(node).encode("utf-8")).hexdigest()
return f"n_{digest[:16]}"
def _canonical_json(self, value: Any) -> str:
return json.dumps(
self._to_jsonable(value),
sort_keys=True,
ensure_ascii=False,
separators=(",", ":"),
)
def _to_jsonable(self, value: Any) -> Any:
if isinstance(value, dict):
return {str(key): self._to_jsonable(val) for key, val in value.items()}
if isinstance(value, (list, tuple)):
return [self._to_jsonable(item) for item in value]
if isinstance(value, set):
return sorted(self._to_jsonable(item) for item in value)
if is_dataclass(value):
return self._to_jsonable(asdict(value))
if isinstance(value, (str, int, float, bool)) or value is None:
return value
return str(value)
def _serialize_value(self, value: Any) -> str:
if value is None:
return ""
if isinstance(value, bool):
return "true" if value else "false"
if isinstance(value, (dict, list, tuple, set)) or is_dataclass(value):
return self._canonical_json(value)
return str(value)
def _coerce_sequence(self, value: Any) -> List[Any]:
if value is None:
return []
if isinstance(value, str):
if self.label_separator in value:
return [part for part in value.split(self.label_separator) if part]
return [value]
if isinstance(value, set):
return sorted(value)
if isinstance(value, (list, tuple)):
return list(value)
return [value]
def _sanitize_token(self, value: str, fallback: str) -> str:
token = value.strip()
if not token:
return fallback
sanitized = []
for char in token:
if char.isalnum() or char == "_":
sanitized.append(char)
else:
sanitized.append("_")
result = "".join(sanitized).strip("_")
if not result:
return fallback
if not (result[0].isalpha() or result[0] == "_"):
result = f"_{result}"
return result
# Valid csv.writer dialect params other than delimiter/lineterminator/quoting
# (those three are always supplied explicitly below).
_CSV_WRITER_PARAMS = frozenset(
{"quotechar", "doublequote", "skipinitialspace", "escapechar", "strict"}
)
def _write_csv(
self,
file_path: Path,
header: List[str],
rows: List[List[str]],
**options: Any,
) -> None:
lineterminator = options.get("lineterminator", "\n")
quoting = options.get("quoting", csv.QUOTE_MINIMAL)
# Only forward recognised csv.writer dialect params so that arbitrary
# caller kwargs (e.g. encoding, validate, node_file_name) do not
# trigger TypeError: csv.writer() got unexpected keyword argument.
writer_options = {
key: value
for key, value in options.items()
if key in self._CSV_WRITER_PARAMS
}
with open(file_path, "w", encoding=self.encoding, newline="") as handle:
writer = csv.writer(
handle,
delimiter=self.delimiter,
lineterminator=lineterminator,
quoting=quoting,
**writer_options,
)
writer.writerow(header)
writer.writerows(rows)
def _validate_prepared(self, prepared: Dict[str, Any]) -> Dict[str, Any]:
errors: List[str] = []
node_ids = [row[0] for row in prepared["node_rows"]]
node_id_set = set(node_ids)
if len(node_ids) != len(node_id_set):
errors.append("Node rows contain duplicate :id values")
for row_number, row in enumerate(prepared["node_rows"], start=2):
expected = 2 + len(prepared["node_columns"])
if len(row) != expected:
errors.append(
f"Node row {row_number} has {len(row)} columns; expected {expected}"
)
if not row[0]:
errors.append(f"Node row {row_number} has an empty :id")
for row_number, row in enumerate(prepared["relationship_rows"], start=2):
expected = 3 + len(prepared["relationship_columns"])
if len(row) != expected:
errors.append(
f"Relationship row {row_number} has {len(row)} columns; "
f"expected {expected}"
)
continue
start_id, end_id, rel_type = row[:3]
if not start_id or not end_id:
errors.append(f"Relationship row {row_number} has an empty endpoint")
if not rel_type:
errors.append(f"Relationship row {row_number} has an empty :TYPE")
if node_id_set and start_id and start_id not in node_id_set:
errors.append(
f"Relationship row {row_number} references unknown :START_ID "
f"{start_id!r}"
)
if node_id_set and end_id and end_id not in node_id_set:
errors.append(
f"Relationship row {row_number} references unknown :END_ID "
f"{end_id!r}"
)
return {"valid": not errors, "errors": errors}
def _validate_file_name(self, file_name: str, param_name: str) -> None:
if not file_name:
raise ValueError(f"{param_name} cannot be empty")
path = Path(file_name)
if path.name != file_name:
raise ValueError(f"{param_name} must be a file name, not a path")
+11 -5
View File
@@ -9,6 +9,7 @@ Supported Registration Types:
* "rdf": RDF export methods
* "json": JSON/JSON-LD export methods
* "csv": CSV export methods
* "neo4j_csv": Neo4j Bulk CSV export methods
* "graph": Graph format export methods
* "yaml": YAML export methods
* "owl": OWL export methods
@@ -26,7 +27,8 @@ Algorithms Used:
Key Features:
- Method registry for custom export methods
- Task-based method organization (rdf, json, csv, graph, yaml, owl, vector, lpg, report, export)
- Task-based method organization (rdf, json, csv, neo4j_csv, graph,
yaml, owl, vector, lpg, report, export)
- Dynamic registration and unregistration
- Easy discovery of available methods
- Support for community-contributed extensions
@@ -43,7 +45,7 @@ Example Usage:
>>> available = method_registry.list_all("json")
"""
from typing import Any, Callable, Dict, List, Optional
from typing import Callable, Dict, List, Optional
class MethodRegistry:
@@ -53,6 +55,7 @@ class MethodRegistry:
"rdf": {},
"json": {},
"csv": {},
"neo4j_csv": {},
"graph": {},
"yaml": {},
"owl": {},
@@ -68,7 +71,8 @@ class MethodRegistry:
Register a custom export method.
Args:
task: Task type ("rdf", "json", "csv", "graph", "yaml", "owl", "vector", "lpg", "report", "export")
task: Task type ("rdf", "json", "csv", "neo4j_csv", "graph",
"yaml", "owl", "vector", "lpg", "report", "export")
name: Method name
method_func: Method function
"""
@@ -82,7 +86,8 @@ class MethodRegistry:
Get method by task and name.
Args:
task: Task type ("rdf", "json", "csv", "graph", "yaml", "owl", "vector", "lpg", "report", "export")
task: Task type ("rdf", "json", "csv", "neo4j_csv", "graph",
"yaml", "owl", "vector", "lpg", "report", "export")
name: Method name
Returns:
@@ -111,7 +116,8 @@ class MethodRegistry:
Unregister a method.
Args:
task: Task type ("rdf", "json", "csv", "graph", "yaml", "owl", "vector", "lpg", "report", "export")
task: Task type ("rdf", "json", "csv", "neo4j_csv", "graph",
"yaml", "owl", "vector", "lpg", "report", "export")
name: Method name
"""
if task in cls._methods and name in cls._methods[task]:
+303
View File
@@ -0,0 +1,303 @@
"""Tests for Neo4j bulk CSV export."""
import csv
from pathlib import Path
import pytest
from semantica.export import Neo4jCSVExporter
from semantica.export.methods import export_knowledge_graph, export_neo4j_csv
from semantica.kg.knowledge_graph import KnowledgeGraph
from semantica.utils.exceptions import ValidationError
def _read_csv(path: Path):
with open(path, "r", encoding="utf-8", newline="") as handle:
return list(csv.reader(handle))
def _sample_graph():
return {
"entities": [
{
"id": "2",
"labels": ["Person", "Engineer"],
"name": "Alice",
"type": "user",
"properties": {
"age": 30,
"email": "alice@example.com",
},
},
{
"id": "1",
"type": "Company",
"name": "Acme, Inc.",
"properties": {"founded": 2024},
},
{
"text": "No ID concept",
"type": "Concept",
"properties": {
"empty": "",
"quote": 'He said "hello"',
"unicode": "東京",
},
},
],
"relationships": [
{
"source": "2",
"target": "1",
"type": "WORKS_FOR",
"properties": {"role": "Engineer", "since": 2024},
},
{
"source": "Alice",
"target": "Acme, Inc.",
"relationship_type": "KNOWS",
"properties": {"note": 'line1\nline2, "quoted"'},
},
],
}
def test_header_correctness_and_node_csv_structure(tmp_path):
exporter = Neo4jCSVExporter()
exporter.export_knowledge_graph(_sample_graph(), tmp_path)
rows = _read_csv(tmp_path / "nodes.csv")
assert rows[0] == [
":id",
":LABEL",
"age",
"email",
"empty",
"founded",
"name",
"quote",
"text",
"type",
"unicode",
]
by_id = {row[0]: row for row in rows[1:]}
assert list(by_id) == sorted(by_id)
assert by_id["2"][1] == "Person;Engineer"
assert by_id["2"][2] == "30"
assert by_id["2"][6] == "Alice"
assert by_id["2"][9] == "user"
assert by_id["1"][1] == "Company"
assert by_id["1"][2] == ""
assert by_id["1"][5] == "2024"
derived_ids = [node_id for node_id in by_id if node_id.startswith("n_")]
assert len(derived_ids) == 1
derived_row = by_id[derived_ids[0]]
assert derived_row[4] == ""
assert derived_row[7] == 'He said "hello"'
assert derived_row[8] == "No ID concept"
assert derived_row[10] == "東京"
def test_relationship_csv_structure_and_properties(tmp_path):
exporter = Neo4jCSVExporter()
exporter.export(_sample_graph(), tmp_path)
rows = _read_csv(tmp_path / "relationships.csv")
assert rows[0] == [":START_ID", ":END_ID", ":TYPE", "note", "role", "since"]
assert rows[1] == ["2", "1", "KNOWS", 'line1\nline2, "quoted"', "", ""]
assert rows[2] == ["2", "1", "WORKS_FOR", "", "Engineer", "2024"]
def test_missing_properties_create_empty_cells(tmp_path):
exporter = Neo4jCSVExporter()
exporter.export(
{
"nodes": [
{"id": "a", "type": "Thing", "properties": {"optional": "present"}},
{"id": "b", "type": "Thing"},
],
"edges": [{"source": "a", "target": "b", "type": "RELATED"}],
},
tmp_path,
)
rows = _read_csv(tmp_path / "nodes.csv")
optional_index = rows[0].index("optional")
by_id = {row[0]: row for row in rows[1:]}
assert by_id["a"][optional_index] == "present"
assert by_id["b"][optional_index] == ""
def test_deterministic_output_with_permuted_graph(tmp_path):
graph_a = {
"entities": [
{"name": "Bob", "type": "Person"},
{"name": "Alice", "type": "Person"},
],
"relationships": [
{"source": "Alice", "target": "Bob", "type": "KNOWS"},
],
}
graph_b = {
"entities": list(reversed(graph_a["entities"])),
"relationships": list(reversed(graph_a["relationships"])),
}
exporter = Neo4jCSVExporter()
out_a = tmp_path / "a"
out_b = tmp_path / "b"
exporter.export(graph_a, out_a)
exporter.export(graph_b, out_b)
assert (out_a / "nodes.csv").read_text(encoding="utf-8") == (
out_b / "nodes.csv"
).read_text(encoding="utf-8")
assert (out_a / "relationships.csv").read_text(encoding="utf-8") == (
out_b / "relationships.csv"
).read_text(encoding="utf-8")
def test_csv_quoting_escaping_and_unicode(tmp_path):
exporter = Neo4jCSVExporter()
exporter.export(_sample_graph(), tmp_path)
nodes_text = (tmp_path / "nodes.csv").read_text(encoding="utf-8")
relationships_text = (tmp_path / "relationships.csv").read_text(encoding="utf-8")
assert '"Acme, Inc."' in nodes_text
assert '"He said ""hello"""' in nodes_text
assert "東京" in nodes_text
assert '"line1\nline2, ""quoted"""' in relationships_text
assert _read_csv(tmp_path / "nodes.csv")
assert _read_csv(tmp_path / "relationships.csv")
def test_empty_graph_export_writes_importable_headers(tmp_path):
exporter = Neo4jCSVExporter()
exporter.export({"entities": [], "relationships": []}, tmp_path)
assert _read_csv(tmp_path / "nodes.csv") == [[":id", ":LABEL"]]
assert _read_csv(tmp_path / "relationships.csv") == [
[":START_ID", ":END_ID", ":TYPE"]
]
validation = exporter.validate_export(tmp_path)
assert validation == {"valid": True, "errors": []}
def test_dry_run_and_written_validation(tmp_path):
exporter = Neo4jCSVExporter()
summary = exporter.dry_run(_sample_graph())
assert summary["valid"] is True
assert summary["node_count"] == 3
assert summary["relationship_count"] == 2
assert summary["node_header"][:2] == [":id", ":LABEL"]
assert summary["relationship_header"][:3] == [
":START_ID",
":END_ID",
":TYPE",
]
exporter.export(_sample_graph(), tmp_path, validate=True)
assert exporter.validate_export(tmp_path)["valid"] is True
def test_knowledge_graph_dataclass_and_convenience_wrappers(tmp_path):
kg = KnowledgeGraph(
entities=[
{"id": "a", "type": "Person", "name": "Alice"},
{"id": "b", "type": "Person", "name": "Bob"},
],
relationships=[{"source": "a", "target": "b", "type": "KNOWS"}],
)
result = export_neo4j_csv(kg, tmp_path / "direct")
assert result["nodes"].name == "nodes.csv"
assert result["relationships"].name == "relationships.csv"
assert result["nodes"].exists()
export_knowledge_graph(kg, tmp_path / "unified", format="neo4j_csv")
assert (tmp_path / "unified" / "nodes.csv").exists()
assert (tmp_path / "unified" / "relationships.csv").exists()
def test_invalid_relationship_endpoint_fails_strict_validation(tmp_path):
exporter = Neo4jCSVExporter()
with pytest.raises(ValidationError):
exporter.export(
{
"entities": [{"id": "a", "type": "Thing"}],
"relationships": [
{"source": "a", "target": "missing", "type": "RELATED"}
],
},
tmp_path,
)
def test_ambiguous_aliases_are_not_resolved(tmp_path):
exporter = Neo4jCSVExporter()
# Create two nodes with the same alias/name "Alice"
graph = {
"entities": [
{"id": "node1", "name": "Alice", "type": "Person"},
{"id": "node2", "name": "Alice", "type": "Person"},
],
"relationships": [{"source": "Alice", "target": "node1", "type": "KNOWS"}],
}
# Verify that strict export raises ValidationError because the alias is ambiguous
with pytest.raises(ValidationError):
exporter.export(graph, tmp_path)
def test_duplicate_node_ids_fail_validation(tmp_path):
exporter = Neo4jCSVExporter()
# Create two nodes with the same explicit ID
graph = {
"entities": [
{"id": "node1", "name": "Alice", "type": "Person"},
{"id": "node1", "name": "Bob", "type": "Person"},
],
"relationships": [],
}
# Verify duplicate explicit node IDs raise ValidationError
with pytest.raises(ValidationError):
exporter.export(graph, tmp_path)
def test_nested_properties_are_json_serialized(tmp_path):
exporter = Neo4jCSVExporter()
graph = {
"entities": [
{
"id": "node1",
"type": "Person",
"properties": {"nested_dict": {"k": "v"}, "nested_list": [1, 2, 3]},
}
],
"relationships": [],
}
exporter.export(graph, tmp_path)
rows = _read_csv(tmp_path / "nodes.csv")
assert rows[0] == [":id", ":LABEL", "nested_dict", "nested_list", "type"]
# Verify that the nested properties are serialized as deterministic JSON strings
by_id = {row[0]: row for row in rows[1:]}
assert by_id["node1"][2] == '{"k":"v"}'
assert by_id["node1"][3] == "[1,2,3]"