semantica/cookbook/introduction/07_Building_Knowledge_Graphs.ipynb

Building Knowledge Graphs

Overview

This notebook demonstrates how to build knowledge graphs from extracted entities and relationships using Semantica's graph building modules. You'll learn to use GraphBuilder and EntityResolver.

Documentation: API Reference

Learning Objectives

• Extract entity mentions and relations, and map them into graph records
• Use GraphBuilder to construct a graph whose edges come from the actual extracted relations
• Use EntityResolver to merge duplicate mentions and remap relationship endpoints
• Use the semantica.deduplication module and report the complete deduplicated entity set

Installation

Install Semantica from PyPI:

pip install semantica
# Or with all optional dependencies:
pip install semantica[all]

---

Step 1: Extract Entities and Relations

Extract entity mentions and relations from text. The sample text mentions Apple Inc. in two separate sentences, so we can later show how duplicate mentions are resolved into one canonical entity.

In [ ]:

%pip install semantica

# spaCy models are distributed separately from the spaCy library. This lesson
# relies on the English model to recognize standalone places such as Cupertino.
import sys
import subprocess
import spacy

try:
    spacy.load("en_core_web_sm")
except OSError:
    subprocess.check_call([sys.executable, "-m", "spacy", "download", "en_core_web_sm"])

In [ ]:

from semantica.semantic_extract import NERExtractor, RelationExtractor

text = (
    "Apple Inc. is headquartered in Cupertino, California. "
    "Tim Cook is the CEO of Apple Inc. "
    "The company is a technology company."
)

ner_extractor = NERExtractor()
relation_extractor = RelationExtractor()

mentions = ner_extractor.extract(text)
relations = relation_extractor.extract(text, mentions)

print("Entity mentions:")
for mention in mentions:
    print(f"  {mention.text!r:<13} {mention.label:<7} span=[{mention.start_char}:{mention.end_char}]")

print("\nExtracted relations:")
for rel in relations:
    print(f"  {rel.subject.text!r} --{rel.predicate}--> {rel.object.text!r}")

Step 2: Build the Knowledge Graph

Give every mention a graph ID, then translate each relation's subject and object into those IDs. Building edges from the actual relation endpoints — rather than guessing endpoints from list positions — is what keeps the graph faithful to the text.

In [ ]:

from semantica.kg import GraphBuilder

entities = []
span_to_id = {}
for i, mention in enumerate(mentions, 1):
    graph_id = f"e{i}"
    span_to_id[(mention.start_char, mention.end_char)] = graph_id
    entities.append({
        "id": graph_id,
        "type": mention.label,
        "name": mention.text,
        "properties": {},
    })

relationships = []
for rel in relations:
    source_id = span_to_id.get((rel.subject.start_char, rel.subject.end_char))
    target_id = span_to_id.get((rel.object.start_char, rel.object.end_char))
    if source_id is None or target_id is None:
        print(f"Skipping relation with unmapped endpoint: "
              f"{rel.subject.text!r} --{rel.predicate}--> {rel.object.text!r}")
        continue
    relationships.append({
        "source": source_id,
        "target": target_id,
        "type": rel.predicate,
        "properties": {},
    })

builder = GraphBuilder()
knowledge_graph = builder.build({"entities": entities, "relationships": relationships})

id_to_name = {entity["id"]: entity["name"] for entity in entities}

print(f"Graph entities ({len(knowledge_graph['entities'])}):")
for entity in knowledge_graph["entities"]:
    print(f"  {entity['id']}: {entity['name']} ({entity['type']})")

print(f"\nGraph relationships ({len(knowledge_graph['relationships'])}):")
for relationship in knowledge_graph["relationships"]:
    print(f"  {id_to_name[relationship['source']]} "
          f"--{relationship['type']}--> {id_to_name[relationship['target']]}")

edges = {
    (id_to_name[r["source"]], r["type"], id_to_name[r["target"]])
    for r in knowledge_graph["relationships"]
}
assert ("Apple Inc.", "located_in", "Cupertino") in edges
assert ("Tim Cook", "works_for", "Apple Inc.") in edges

Step 3: Entity Resolution

The graph currently contains two nodes for the same organization. EntityResolver merges duplicate mentions into one canonical entity and records which source IDs were merged (merged_from), so relationship endpoints can be remapped onto the canonical entity.

In [ ]:

from semantica.kg import EntityResolver

entity_resolver = EntityResolver()
resolved_entities = entity_resolver.resolve_entities(entities)

canonical_id = {}
for entity in resolved_entities:
    for source_id in entity.get("merged_from", [entity["id"]]):
        canonical_id[source_id] = entity["id"]
    if entity.get("merged_from"):
        print(f"Merged {entity['merged_from']} -> {entity['id']}: {entity['name']}")

print(f"\nMentions in: {len(entities)}, resolved entities out: {len(resolved_entities)}")

resolved_names = {entity["id"]: entity["name"] for entity in resolved_entities}
print("\nRelationships remapped onto canonical entities:")
for relationship in relationships:
    source = canonical_id[relationship["source"]]
    target = canonical_id[relationship["target"]]
    print(f"  {resolved_names[source]} --{relationship['type']}--> {resolved_names[target]}")

canonical_entities = {(entity["name"], entity["type"]) for entity in resolved_entities}
assert canonical_entities == {
    ("Apple Inc.", "ORG"),
    ("Tim Cook", "PERSON"),
    ("Cupertino", "GPE"),
    ("California", "GPE"),
}
assert len(resolved_entities) == 4

Step 4: Deduplication

The semantica.deduplication module gives finer control over the same problem. Note that merge_duplicates returns one MergeOperation per duplicate group — the complete deduplicated collection is those merged entities plus every entity that was not part of any group.

In [ ]:

from semantica.deduplication import DuplicateDetector, EntityMerger, MergeStrategy

detector = DuplicateDetector(similarity_threshold=0.8)
duplicate_groups = detector.detect_duplicate_groups(entities)
print(f"Duplicate groups: {len(duplicate_groups)}")
for group in duplicate_groups:
    print(f"  {[entity['name'] for entity in group.entities]} "
          f"(confidence={group.confidence:.2f})")

merger = EntityMerger()
merge_operations = merger.merge_duplicates(
    entities, strategy=MergeStrategy.KEEP_MOST_COMPLETE
)

merged_source_ids = {
    entity["id"] for op in merge_operations for entity in op.source_entities
}
untouched_entities = [e for e in entities if e["id"] not in merged_source_ids]
deduplicated_entities = untouched_entities + [
    op.merged_entity for op in merge_operations
]

print(f"\nMerge operations: {len(merge_operations)}")
print(f"Deduplicated entities ({len(deduplicated_entities)}):")
for entity in deduplicated_entities:
    print(f"  {entity['id']}: {entity['name']} ({entity['type']})")

assert len(merge_operations) == 1
assert len(deduplicated_entities) == 4

Summary

You've learned how to build knowledge graphs:

• Extraction to graph: map each mention to a graph ID and build edges from the actual Relation.subject / Relation.object endpoints
• GraphBuilder: construct knowledge graphs from explicit {"entities": ..., "relationships": ...} input
• EntityResolver: merge duplicate mentions into canonical entities and remap relationship endpoints
• Deduplication: combine MergeOperation results with untouched entities to get the complete deduplicated set

Next: Learn how to analyze graphs in the Graph_Analytics notebook.