Skip to main content
Ops events give deployment operators a structured record of lifecycle events and classified failures. They are disabled by default and are separate from tenant Phoenix/Langfuse tracing. Prometheus remains the metrics backend; enabling ops events does not create an OTel metrics pipeline.

​
Enable the exporter

Set these values on backend, worker, and relay processes, then restart them:
The endpoint is a base URL. The exporter appends /v1/logs and sends OTLP/HTTP protobuf. For an authenticated collector, set OPS_TELEMETRY_HEADERS through the existing secret configuration, for example Authorization=Bearer <collector-token>. Do not put credentials in the endpoint URL. Use TLS for traffic outside the trusted cluster network. Both platform Helm charts accept these settings in extraEnv; put authentication headers in secrets. With SECRETS_SOURCE=FILE, add the values to the encrypted internal configuration instead. In an environment-backed Docker Compose deployment, pass the same variables through each service’s environment section.
There is no collector or storage backend enabled by default. Setting the flag without a nonempty endpoint leaves the stream disabled. Configuration is loaded once per process; changing it requires restarting all producers.

​
Minimal collector for staging verification

Run an OpenTelemetry Collector reachable only inside the test namespace. A logs pipeline with the debug exporter is sufficient to inspect synthetic test events:
Expose port 4318 through a ClusterIP Service named ops-collector, apply the Helm settings above, and read the collector pod logs. For ongoing operation, replace debug with an exporter to your logs store and configure retention and restricted operator access. Event attributes include tenant, workspace, user, and resource identifiers; they must not be exposed through a public receiver or dashboard.

​
Event catalogue

Every event name supported by this implementation is listed below. All records use instrumentation scope noxus.ops, severity INFO, a UTC event timestamp, and an empty body. Resource metadata includes service.name (the deployment name), deployment.environment.name, and OpenTelemetry SDK metadata. Optional attributes are omitted when their source identity is unavailable.

​
Billing lifecycle

These events fire when BillingManager.register_event() accepts the registration attempt. They carry these attributes:

​
Case activity

A newly inserted CaseActivity row emits after its outer database transaction commits. Every event carries noxus.case.id and noxus.case.activity_type; it also carries noxus.user.id when the activity has an actor. Titles, comments, email contents, field values, and activity payloads are excluded. task_failed is a legacy accepted activity value. Current task writers use automation_failed; old stored rows are not replayed into the ops stream. The custom classification stays fixed and never becomes a tenant-defined event name.

​
Agent insights

A newly inserted InsightEventRow emits after its outer database transaction commits. Attributes are noxus.tenant_id, noxus.group_id, and noxus.insight.type, plus noxus.conversation.id and noxus.agent.id when present. Insight keys, labels, scores, analysis text, and payloads are excluded.

​
Classified failures and API keys

Credential permanence is a Boolean classification: true means reconnection is required; false means the failure was not classified as permanent. Integration and provider keys identify registered integration types, not customer credential names. Provider error text and tokens are excluded. The SQL denial reason is one of parse_error, multi_statement, not_select, not_write, schema_denied, table_denied, function_denied, or write_denied. SQL text and database error messages are excluded. Billing limit type is internal or license. Constraint type uses the enum member name: AI_OPERATIONS, MESSAGES_SENT, KB_FILES_UPLOADED, TOKENS, WORKFLOWS, TRIGGER_RUNS, NCS_SEARCH, or NCS_SCRAPE. Usage amounts and limits are excluded.

​
Coverage and delivery semantics

Run, node, model, and token usage remain in the existing Prometheus instruments and billing accounting. Successful table queries produce operation/row/duration metrics; only instrumented guard denials produce an ops event. Plugin installation failures, sandbox OOM, and model-health suspension/recovery are follow-up coverage; this implementation does not emit dedicated events for them. These are best-effort diagnostics, not a durable audit or accounting log. A crash or full exporter queue can lose events. Billing registration events describe the registration attempt, not successful database persistence; noxus.event.id supports consumer deduplication. Case/insight events are emitted only after the outer commit; rolled-back savepoints and transactions do not emit. Existing database history is not backfilled.

​
Verify delivery

  1. Enable the exporter and start the collector as described above.
  2. Create a temporary API key through the workspace’s API keys page.
  3. Read the collector pod logs and find noxus.api_key.created. Confirm the record has a current timestamp, an empty body, and the expected key and tenant IDs. The secret key value must be absent.
  4. Revoke the temporary key through the same page and confirm noxus.api_key.revoked arrives with the same key ID.
  5. Allow for the configured export interval and the collector’s batch delay before treating an event as missing.

​
Troubleshooting

  • No events: verify the flag and endpoint in every producer, restart after changes, and confirm the collector has a logs pipeline on port 4318.
  • Exporter errors: check collector health, network policy, HTTP path, TLS, and authentication. A trace-only receiver is insufficient.
  • Missing run traces: inspect the run span capture separately. It uses SimpleSpanProcessor and Redis/cold storage; the ops collector is unrelated.