semantica/docs/reference/core.md

Core

Framework infrastructure, lifecycle management, and plugin system.

---

🎯 Overview

• :material-cogs:{ .lg .middle } Semantica

  ---

  Main framework class coordinating all components and workflows

• :material-lifecycle:{ .lg .middle } Lifecycle Management

  ---

  Manage initialization, startup, shutdown, and state transitions

• :material-tune:{ .lg .middle } Configuration

  ---

  Unified configuration management via YAML and Environment variables

• :material-puzzle:{ .lg .middle } Plugin System

  ---

  Extensible plugin registry for adding custom modules and capabilities

• :material-console:{ .lg .middle } Method Registry

  ---

  Registry for custom orchestration methods and extensibility

!!! tip "When to Use" - Application Startup: Initializing the Semantica framework in your app - Configuration: Tuning global settings - Extension: Developing custom plugins or modules - Orchestration: Coordinating complex workflows across multiple modules

---

⚙️ Algorithms Used

Lifecycle Management

• State Machine: UNINITIALIZED -> INITIALIZING -> READY -> RUNNING -> STOPPING -> STOPPED
• Priority-based Hooks: Startup and shutdown hooks executed in priority order (lower = earlier)
• Graceful Shutdown: Ensuring all resources (DB connections, thread pools) are closed properly

Configuration

• Layered Loading: Defaults -> Config File -> Environment Variables (Priority order)
• Schema Validation: Validating config structure against defined schemas
• Nested Access: Dot notation for accessing nested configuration values

Plugin System

• Discovery: Auto-discovery of plugins via directory scanning
• Registration: Dynamic registration of classes and functions
• Dependency Resolution: Automatic loading of plugin dependencies

---

Main Classes

Semantica

The main framework class that coordinates all components.

Methods:

Method	Description
__init__(config=None, **kwargs)	Initialize framework with optional configuration
initialize()	Initialize all framework components
build_knowledge_base(sources, **kwargs)	Build knowledge base from data sources
run_pipeline(pipeline, data)	Execute a processing pipeline
get_status()	Get system health and status
shutdown(graceful=True)	Shutdown the framework gracefully

Example:

from semantica.core import Semantica

# Initialize framework
framework = Semantica()
framework.initialize()

# Build knowledge base
result = framework.build_knowledge_base(
    sources=["doc1.pdf", "doc2.docx"],
    embeddings=True,
    graph=True
)

# Check status
status = framework.get_status()
print(f"System state: {status['state']}")

# Shutdown
framework.shutdown()

ConfigManager

Manages global configuration loading, validation, and merging.

Methods:

Method	Description
load_from_file(file_path, validate=True)	Load config from YAML or JSON file
load_from_dict(config_dict, validate=True)	Load config from dictionary
merge_configs(*configs, validate=True)	Merge multiple configurations
get_config()	Get current configuration
set_config(config, validate=True)	Set current configuration
reload(file_path=None)	Reload configuration from file

Example:

from semantica.core import ConfigManager

manager = ConfigManager()
config = manager.load_from_file("config.yaml")

# Merge configurations
config1 = manager.load_from_file("base_config.yaml")
config2 = manager.load_from_file("override_config.yaml")
merged = manager.merge_configs(config1, config2)

Config

Configuration data class with validation and nested access.

Methods:

Method	Description
get(key_path, default=None)	Get nested configuration value by key path
set(key_path, value)	Set nested configuration value
update(updates, merge=True)	Update configuration with new values
validate()	Validate configuration settings
to_dict()	Convert configuration to dictionary

Example:

from semantica.core import Config, ConfigManager

manager = ConfigManager()
config = manager.load_from_dict({"processing": {"batch_size": 32}})

# Access nested values
batch_size = config.get("processing.batch_size", default=16)

# Update values
config.set("processing.batch_size", 64)
config.update({"quality": {"min_confidence": 0.9}})

# Validate
config.validate()

LifecycleManager

System lifecycle management with hooks and health monitoring.

Methods:

Method	Description
startup()	Execute startup sequence with registered hooks
shutdown(graceful=True)	Execute shutdown sequence
register_startup_hook(hook_fn, priority=50)	Register a startup hook
register_shutdown_hook(hook_fn, priority=50)	Register a shutdown hook
register_component(name, component)	Register component for health monitoring
health_check()	Perform comprehensive system health check
get_health_summary()	Get summary of system health
get_state()	Get current system state
is_ready()	Check if system is ready
is_running()	Check if system is running

Example:

from semantica.core import LifecycleManager

manager = LifecycleManager()

# Register hooks
def init_db():
    print("Initializing database...")

manager.register_startup_hook(init_db, priority=10)
manager.startup()

# Register component for health monitoring
class DatabaseComponent:
    def health_check(self):
        return {"healthy": True, "message": "Connected"}

db = DatabaseComponent()
manager.register_component("database", db)

# Check health
health = manager.health_check()
summary = manager.get_health_summary()

manager.shutdown(graceful=True)

PluginRegistry

Plugin registry and management system for dynamic plugin discovery and loading.

Methods:

Method	Description
__init__(plugin_paths=None)	Initialize with optional plugin paths for auto-discovery
register_plugin(plugin_name, plugin_class, version="1.0.0", **metadata)	Manually register a plugin
load_plugin(plugin_name, **config)	Load and initialize a plugin
unload_plugin(plugin_name)	Unload a plugin
list_plugins()	List all available plugins
get_plugin_info(plugin_name)	Get information about a plugin
is_plugin_loaded(plugin_name)	Check if a plugin is loaded
get_loaded_plugin(plugin_name)	Get loaded plugin instance

Example:

from semantica.core import PluginRegistry

# Auto-discover plugins
registry = PluginRegistry(plugin_paths=["./plugins"])

# Load plugin with configuration
plugin = registry.load_plugin("my_plugin", api_key="xxx")

# List all plugins
plugins = registry.list_plugins()
for plugin_info in plugins:
    print(f"{plugin_info['name']}: {plugin_info['version']}")

# Get plugin info
info = registry.get_plugin_info("my_plugin")

MethodRegistry

Registry for custom orchestration methods.

Methods:

Method	Description
register(task, name, method_func)	Register a custom orchestration method
get(task, name)	Get method by task and name
list_all(task=None)	List all registered methods
unregister(task, name)	Unregister a method
clear(task=None)	Clear all registered methods

Example:

from semantica.core import method_registry

def custom_kb_builder(sources, **kwargs):
    # Custom logic
    return {"knowledge_graph": {}}

method_registry.register("knowledge_base", "custom", custom_kb_builder)

# Use custom method
method = method_registry.get("knowledge_base", "custom")
result = method(sources=["doc.pdf"])

---

Orchestration Methods

Convenience functions for common orchestration tasks.

build_knowledge_base()

Build knowledge base from data sources.

from semantica.core.methods import build_knowledge_base

result = build_knowledge_base(
    sources=["doc1.pdf", "doc2.docx"],
    method="default",
    embeddings=True,
    graph=True
)

run_pipeline()

Execute a processing pipeline.

from semantica.core.methods import run_pipeline

result = run_pipeline(
    pipeline={"steps": ["parse", "extract"]},
    data="sample text",
    method="default"
)

initialize_framework()

Initialize Semantica framework.

from semantica.core.methods import initialize_framework

framework = initialize_framework(
    config={"llm_provider": {"name": "openai"}},
    method="default"
)

get_status()

Get system status.

from semantica.core.methods import get_status

status = get_status(framework=my_framework, method="detailed")

get_orchestration_method()

Get orchestration method by task and name.

from semantica.core.methods import get_orchestration_method

method = get_orchestration_method("knowledge_base", "custom")

list_available_methods()

List all available orchestration methods.

from semantica.core.methods import list_available_methods

all_methods = list_available_methods()
kb_methods = list_available_methods("knowledge_base")

---

Configuration

Environment Variables

Configuration can be loaded from environment variables with SEMANTICA_ prefix:

export SEMANTICA_PROCESSING_BATCH_SIZE=64
export SEMANTICA_LLM_PROVIDER_MODEL=gpt-4
export SEMANTICA_QUALITY_MIN_CONFIDENCE=0.8

YAML Configuration

llm_provider:
  name: openai
  model: gpt-4
  api_key: ${OPENAI_API_KEY}

embedding_model:
  name: openai
  model: text-embedding-ada-002

processing:
  batch_size: 32
  max_workers: 4

quality:
  min_confidence: 0.7

logging:
  level: INFO

plugins:
  my_plugin:
    enabled: true
    config_key: config_value

JSON Configuration

{
  "llm_provider": {
    "name": "openai",
    "model": "gpt-4"
  },
  "processing": {
    "batch_size": 32
  }
}

---

Integration Examples

Basic Usage

from semantica.core import Semantica, ConfigManager

# 1. Load configuration
config_manager = ConfigManager()
config = config_manager.load_from_file("config.yaml")

# 2. Initialize framework
framework = Semantica(config=config)
framework.initialize()

try:
    # 3. Build knowledge base
    result = framework.build_knowledge_base(
        sources=["doc1.pdf", "doc2.docx"],
        embeddings=True,
        graph=True
    )
    
    # 4. Check status
    status = framework.get_status()
    print(f"System state: {status['state']}")
    
finally:
    # 5. Shutdown gracefully
    framework.shutdown(graceful=True)

Custom Plugin

from semantica.core import PluginRegistry

class MyPlugin:
    def initialize(self):
        print("Plugin initialized")
    
    def execute(self, data):
        return {"processed": True}

registry = PluginRegistry()
registry.register_plugin(
    plugin_name="my_plugin",
    plugin_class=MyPlugin,
    version="1.0.0"
)

plugin = registry.load_plugin("my_plugin")
result = plugin.execute("sample data")

Lifecycle Hooks

from semantica.core import LifecycleManager

manager = LifecycleManager()

def init_database():
    print("Initializing database...")

def cleanup_database():
    print("Cleaning up database...")

manager.register_startup_hook(init_database, priority=10)
manager.register_shutdown_hook(cleanup_database, priority=10)

manager.startup()
# ... do work ...
manager.shutdown(graceful=True)

Custom Orchestration Method

from semantica.core import method_registry, Semantica

def fast_kb_builder(sources, **kwargs):
    framework = Semantica()
    framework.initialize()
    try:
        return framework.build_knowledge_base(
            sources=sources,
            embeddings=False,  # Skip for speed
            graph=True,
            **kwargs
        )
    finally:
        framework.shutdown()

method_registry.register("knowledge_base", "fast", fast_kb_builder)

# Use custom method
from semantica.core.methods import build_knowledge_base
result = build_knowledge_base(sources=["doc.pdf"], method="fast")

---

Best Practices

1. Always Initialize: Always call initialize() after creating a Semantica instance before using it.

2. Graceful Shutdown: Always call shutdown(graceful=True) in a finally block to ensure proper cleanup.

3. Configuration Management: Use ConfigManager for loading and managing configurations. Prefer YAML files for complex configurations.

4. Error Handling: Wrap framework operations in try-except blocks to handle ConfigurationError and ProcessingError appropriately.

5. Health Monitoring: Register components with LifecycleManager for health monitoring and use health_check() regularly.

6. Plugin Development: Follow the plugin interface (must have initialize() and execute() methods) when creating custom plugins.

7. Method Registration: Use MethodRegistry for extensibility. Register custom methods for knowledge base building, pipeline execution, etc.

8. Hook Priorities: Use appropriate priorities for lifecycle hooks. Lower numbers execute first.

9. Configuration Validation: Always validate configurations using config.validate() before using them.

10. Resource Cleanup: Ensure all resources are properly cleaned up in shutdown hooks.

---

See Also

• Core Usage Guide - Comprehensive usage guide with detailed examples
• Pipeline Module - Executed by the Semantica framework
• Utils Module - Shared utilities used by Core