1. Persistence Layer (PostgreSQL)
PostgreSQL serves as the primary source of truth for the platform, managing all structured data and knowledge metadata.- Requirements: PostgreSQL v15+ with the
pgvectorextension. - Core Data: User accounts, workspace configurations, flow definitions, agent settings, and knowledge base metadata.
- Vector Search: The
pgvectorextension enables high-performance semantic search and retrieval for AI knowledge bases. - Vector Database Flexibility: While
pgvectoris the default for platform-native Knowledge Bases, Noxus can be configured to support other common vector databases. Additionally, workflows and agents can connect to any externally hosted vector database during runtime for specialized search requirements. - Operational Tip: Use managed services like AWS RDS or GCP Cloud SQL with automated backups and multi-AZ failover for production environments.
2. Coordination Layer (Task Broker)
This layer powers real-time task queueing and message passing between Noxus services for asynchronous execution.- Default Engine: Redis is the default engine for asynchronous flow execution and service coordination.
- Alternative Engines: This layer is highly configurable to match your infrastructure:
- PostgreSQL: Can be used as the coordination engine for simpler, single-database architectures.
- RabbitMQ: Supported for high-throughput, enterprise-grade message queueing requirements.
- Security: Never expose the coordination layer to the public internet. Ensure it is only accessible via private networking from Noxus services.
3. Caching Layer
The caching layer is dedicated to accelerating access to frequently used resources and transient state.- Technology: Redis is always utilized as the high-speed caching layer.
- Usage: Stores session data, frequently accessed configuration, and temporary computation results to minimize latency.
- Operational Tip: Monitor memory saturation and eviction policies to ensure optimal performance during high-load bursts.
4. Object Storage Layer (S3/GCS/MinIO)
The object storage layer handles large-scale binary data, files, and artifacts generated or consumed by your AI workflows.- Supported Backends: S3-compatible storage (AWS S3, Google Cloud Storage, Azure Blob Storage, or on-premises MinIO).
- Core Data: Uploaded documents for knowledge bases, generated images/files from flows, and archived execution payloads.
- Lifecycle Management: Implement retention rules to automatically archive or delete old artifacts based on your organization’s data policy.
- Security: Enforce server-side encryption and use IAM roles/Service Accounts for access instead of long-lived access keys.
Liquid Data Management
Noxus features a Liquid Data architecture that autonomously manages the lifecycle of your information across these storage tiers.- Autonomous Movement: Data is automatically shifted between high-performance cache, persistence, and cold storage based on active usage patterns and access frequency.
- Policy-Driven Retention: Define granular retention policies that govern how long data remains in each tier before being archived or purged.
- Budget Optimization: By intelligently moving inactive data to lower-cost object storage, Noxus helps you maintain strict storage budgets without sacrificing performance for active workloads.
Storage Architecture Summary
| Layer | Technology | Primary Function |
|---|---|---|
| Persistence | PostgreSQL + pgvector | Source of truth, metadata, and semantic search. |
| Coordination | Redis / Postgres / RabbitMQ | Task queueing and service orchestration. |
| Caching | Redis | High-speed data access and session state. |
| Object Storage | S3 / GCS / MinIO | Large files, artifacts, and knowledge assets. |