Human-in-the-Loop - Noxus Documentation

Human-in-the-Loop (HITL) workflows combine automation efficiency with human judgment. Use HITL when automation can handle routine tasks but humans should review edge cases, make final decisions, or handle exceptions.

When to Use Human-in-the-Loop

Good Use Cases

1. High-Stakes Decisions

Financial approvals over threshold
Legal document review
Medical diagnoses
Customer service escalations
Contract negotiations

2. Edge Cases and Exceptions

Data quality issues AI can’t resolve
Ambiguous inputs requiring interpretation
Conflicting information needing reconciliation
Novel situations outside training data

3. Quality Assurance

Review AI-generated content before publishing
Verify data extraction accuracy
Approve automated actions
Validate classification results

4. Compliance and Governance

Regulatory approval requirements
Multi-level authorization workflows
Audit trail requirements
Policy enforcement

5. Learning and Improvement

Collect training data for AI models
Identify automation failure patterns
Refine business rules
Update decision criteria

Poor Use Cases

Don’t use HITL for:

Fully automatable tasks (adds unnecessary friction)
Time-critical operations (humans introduce latency)
High-volume, low-value decisions (doesn’t scale)
Tasks requiring instant response (humans aren’t available 24/7)

Human-in-the-Loop Node

The Human in the Loop node pauses workflow execution and requests human input.

Basic Configuration

Title: Display title for the approval request

Approve Invoice Payment

Description: Context and instructions for reviewer

Please review this invoice for accuracy:
- Vendor: ((vendor_name))
- Amount: $((amount))
- Invoice Number: ((invoice_number))

Verify against purchase order and approve if correct.

Actions: Buttons for user to click

- Approve
- Reject
- Request More Information

Inputs to Display: Show relevant data to the reviewer

- Invoice PDF (file)
- Invoice details (JSON object)
- Purchase order (text)
- Vendor history (text)

Assigned To: User or group who should review

- Specific user: [email protected]
- Group: Finance Team
- Role: Approvers

Outputs

Selected Action: Which button the user clicked

"Approve", "Reject", or "Request More Information"

User Comment (Optional): Text entered by reviewer

"Approved. Amount matches PO #12345."

Approved By: User who made the decision

{
  "email": "[email protected]",
  "name": "John Smith",
  "timestamp": "2024-01-15T10:30:00Z"
}

Basic HITL Patterns

Pattern 1: Simple Approval

Binary approve/reject decision:

Process Request
    ↓
Validate Data
    ↓
Human in the Loop:
  Title: "Approve Request"
  Description: "Review request from ((customer_name))"
  Actions: [Approve, Reject]
  Assigned To: Managers
  Outputs: action, comment
    ↓
Switch: ((action))
    ├─ Approve
    │   ↓
    │   Process Approval
    │   Send confirmation email
    └─ Reject
        ↓
        Log rejection reason: ((comment))
        Send rejection email

Pattern 2: Multi-Option Decision

Multiple paths based on human choice:

AI Classify Ticket
  Output: category, confidence
    ↓
Conditional: confidence < 0.7
    └─ True → Low confidence, need human
        ↓
        Human in the Loop:
          Title: "Classify Support Ticket"
          Description: "AI couldn't confidently classify. Please select category."
          Actions: [Bug Report, Feature Request, Question, Complaint]
          Display: ticket_text, customer_history
          Outputs: action (selected category)
        ↓
        Switch: ((action))
            ├─ Bug Report → Route to Engineering
            ├─ Feature Request → Route to Product
            ├─ Question → Route to Support
            └─ Complaint → Route to Customer Success

Pattern 3: Review and Correct

Human reviews and corrects AI output:

AI Extract Information from Resume
  Output: extracted_data
    ↓
Human in the Loop:
  Title: "Review Extracted Resume Data"
  Description: "Verify AI extracted this information correctly"
  Display:
    - Original resume (PDF)
    - Extracted data (JSON)
  Actions: [Approve, Correct]
  Input Fields:
    - Name (pre-filled: extracted_data.name)
    - Email (pre-filled: extracted_data.email)
    - Phone (pre-filled: extracted_data.phone)
    - Skills (pre-filled: extracted_data.skills)
  Outputs: action, corrected_data
    ↓
Switch: ((action))
    ├─ Approve → Use extracted_data
    └─ Correct → Use corrected_data
    ↓
Create Candidate Record

Pattern 4: Exception Escalation

Automation handles normal cases, escalates exceptions:

Process Order (continue on error: true)
    ↓
Conditional: order_amount > $10,000 OR processing_failed
    ├─ True → Exception, escalate
    │   ↓
    │   Human in the Loop:
    │     Title: "High-Value Order Requires Approval"
    │     Description:
    │       - Order total: $((order_amount))
    │       - Customer: ((customer_name))
    │       - Payment method: ((payment_method))
    │     Actions: [Approve, Reject, Investigate]
    │     Assigned To: Sales Manager
    │     Outputs: action, comment
    │   ↓
    │   Switch: ((action))
    │       ├─ Approve → Complete order
    │       ├─ Reject → Cancel and refund
    │       └─ Investigate → Create investigation ticket
    └─ False → Normal case, auto-approve
        ↓
        Complete order automatically

Advanced HITL Patterns

Pattern: Multi-Level Approval

Chain of approvals with escalation:

Submit Expense Report
  Amount: $5,000
    ↓
Level 1: Manager Approval
    ↓
    Conditional: amount > $1,000
        └─ True → Require manager approval
            ↓
            Human in the Loop:
              Title: "Approve Expense Report"
              Assigned To: ((employee_manager))
              Actions: [Approve, Reject]
              Outputs: manager_action
    ↓
    Conditional: manager_action equals "Approve" AND amount > $5,000
        └─ True → Escalate to director
            ↓
            Level 2: Director Approval
                ↓
                Human in the Loop:
                  Title: "Director Approval Required (Over $5K)"
                  Assigned To: Finance Director
                  Actions: [Approve, Reject, Request Justification]
                  Outputs: director_action
    ↓
    Conditional: all approvals received
        └─ True → Process reimbursement

Pattern: Parallel Approvals

Multiple approvers must agree:

Contract Requires Legal + Finance Approval
    ↓
Parallel Branches:

Branch A: Legal Review
    ↓
    Human in the Loop:
      Title: "Legal Review: Contract"
      Assigned To: Legal Team
      Actions: [Approve, Request Changes]
      Outputs: legal_decision
    ↓
    Store: legal_approved = (legal_decision == "Approve")

Branch B: Finance Review
    ↓
    Human in the Loop:
      Title: "Finance Review: Contract"
      Assigned To: Finance Team
      Actions: [Approve, Request Changes]
      Outputs: finance_decision
    ↓
    Store: finance_approved = (finance_decision == "Approve")

Merge Branches:
    ↓
    Conditional: legal_approved AND finance_approved
        ├─ True → Execute contract
        └─ False → Return for revisions

Pattern: Approval with Delegation

Allow approver to delegate to someone else:

Human in the Loop:
  Title: "Approve Budget Allocation"
  Assigned To: Department Head
  Actions: [Approve, Reject, Delegate]
  Outputs: action, comment, delegate_to
    ↓
Switch: ((action))
    ├─ Approve → Process allocation
    ├─ Reject → Cancel request
    └─ Delegate
        ↓
        Human in the Loop:
          Title: "Budget Allocation (Delegated)"
          Assigned To: ((delegate_to))
          Actions: [Approve, Reject]
          Outputs: delegated_action
        ↓
        Process based on delegated_action

Pattern: Timeout with Default Action

Auto-approve if no response within timeframe:

Human in the Loop:
  Title: "Review Content for Publication"
  Assigned To: Content Manager
  Timeout: 24 hours
  Actions: [Approve, Request Edits, Reject]
  Outputs: action
    ↓
Conditional: timeout occurred (no response)
    └─ True → Default action
        ↓
        Log: "Auto-approved after 24hr timeout"
        Approve and publish

Implementation (using scheduled workflow):

Main Workflow:
    Create approval request in database
    Store: approval_id, created_at
    Send notification to approver
    Output: approval_id

Scheduled Workflow (runs hourly):
    Query: pending approvals older than 24 hours
    ↓
    For each pending approval:
        Auto-approve
        Log timeout action
        Continue main workflow

Pattern: Collect Training Data

Use human decisions to improve AI:

AI Classify Document
  Output: predicted_category, confidence
    ↓
Conditional: confidence < 0.9 OR random_sample (10%)
    └─ True → Request human classification
        ↓
        Human in the Loop:
          Title: "Classify Document (Training Data)"
          Description: "AI predicted: ((predicted_category)) (((confidence * 100))% confidence)"
          Actions: [Category A, Category B, Category C, Category D]
          Display: document_preview
          Outputs: human_category
        ↓
        Store Training Data:
          document_id: ((doc_id))
          ai_prediction: ((predicted_category))
          ai_confidence: ((confidence))
          human_label: ((human_category))
          match: ((predicted_category == human_category))
        ↓
        Use human_category for downstream processing

UX Best Practices

Clear Context

Bad:

Title: "Review This"
Description: "Check if this is okay"

Good:

Title: "Approve Customer Refund Request"
Description: "
Customer ((customer_name)) purchased ((product_name)) on ((purchase_date)).

They are requesting a refund for the following reason:
\"((refund_reason))\"

Customer history:
- Total purchases: $((lifetime_value))
- Previous refunds: ((previous_refunds))
- Account age: ((account_age)) days

Please approve or deny this refund request.
"

Actionable Buttons

Bad:

Actions: [Yes, No]

Good:

Actions: [Approve Refund, Deny Refund, Request More Information]

Best (with expected outcomes):

Actions:
  - "Approve Refund ($((amount)))" → Refund processed immediately
  - "Deny Refund" → Customer notified, ticket closed
  - "Request More Information" → Customer receives follow-up email

Show Relevant Data

Only show data needed for decision: Too Little:

Display: customer_name
(Missing: order details, history, reason)

Too Much:

Display: entire_database_dump
(Overwhelming, hard to find relevant info)

Just Right:

Display:
  - Customer: ((customer_name)) - Account since ((join_date))
  - Order: #((order_id)) - $((amount)) - ((order_date))
  - Refund Reason: ((reason))
  - Customer LTV: $((lifetime_value))
  - Previous Refunds: ((refund_count))

Provide Decision Guidance

Help reviewers make consistent decisions:

Title: "Review Flagged Transaction"
Description: "
Transaction flagged by fraud detection system.

**Transaction Details:**
- Amount: $((amount))
- Card: ((card_last_4))
- Location: ((location))
- IP Address: ((ip_address))

**Fraud Indicators:**
- Risk Score: ((risk_score))/100
- Unusual location: ((unusual_location))
- High velocity: ((high_velocity))

**Decision Guidelines:**
- Score 0-30: Usually safe, approve unless other red flags
- Score 31-70: Review carefully, check customer history
- Score 71-100: High risk, deny unless verified by customer

**Recommended Action:** ((recommended_action))
"
Actions: [Approve, Deny, Contact Customer First]

Notification Strategies

Email Notifications

Send email when approval needed:

Human in the Loop Node (approval_pending)
    ↓
Send Email:
  To: ((assigned_approver_email))
  Subject: "Action Required: Approve ((request_type))"
  Body:
    "You have a pending approval request:

    Type: ((request_type))
    Requester: ((requester_name))
    Amount/Value: $((amount))

    Please review and approve at:
    ((approval_link))

    This request will auto-expire in ((timeout_hours)) hours."

Slack Notifications

Real-time notification with action buttons:

Human in the Loop Node (approval_pending)
    ↓
Send Slack Message:
  Channel: @((assigned_approver_slack))
  Message:
    "🔔 Approval needed: ((request_type))

    Details:
    • Requester: ((requester_name))
    • Amount: $((amount))
    • Priority: ((priority))

    [View Details](((approval_link)))"

Advanced: Slack Buttons (requires custom integration)

Send Slack Message with Buttons:
  Buttons: [Approve, Reject, View Details]
  On Button Click: Continue workflow with action

SMS for Urgent Approvals

Critical approvals that need immediate attention:

Conditional: priority equals "urgent"
    └─ True
        ↓
        Send SMS:
          To: ((manager_phone))
          Message: "URGENT: Approve $((amount)) payment? Reply APPROVE or DENY. Details: ((short_link))"
        ↓
        Wait for SMS Reply
        ↓
        Parse: reply contains "APPROVE"
            ├─ True → Approve
            └─ False → Deny or continue HITL

Tracking and Analytics

Approval Metrics

Track approval performance:

For Each Approval Request:
    Store Metrics:
      - request_id
      - assigned_to
      - created_at
      - decision_at
      - decision_time (decision_at - created_at)
      - action (approve/reject/other)
      - timeout_occurred

Analytics:
  - Average decision time by approver
  - Approval rate by request type
  - Timeout rate
  - Bottlenecks (requests waiting longest)

SLA Monitoring

Alert on approval delays:

Scheduled Workflow (hourly):
    ↓
    Query: pending approvals
    ↓
    For each approval:
        Calculate: age = now() - created_at
        ↓
        Conditional: age > SLA_threshold
            └─ True → SLA breach
                ↓
                Send Escalation:
                  To: ((approver_manager))
                  Message: "SLA breach: Approval pending for ((age)) hours"
                  Escalate: Assign to manager

Approval Audit Trail

Maintain compliance audit trail:

On Approval Decision:
    Store Audit Record:
      - approval_id
      - workflow_id
      - approved_by (email, name, IP)
      - decision (approve/reject)
      - decision_timestamp
      - comment
      - supporting_data (snapshot)
      - delegated_from (if delegated)

Error Handling

Approver Unavailable

Handle absent approvers:

Human in the Loop:
  Assigned To: Primary Approver
  Timeout: 4 hours
    ↓
Conditional: timeout occurred
    └─ True → Escalate
        ↓
        Human in the Loop:
          Title: "ESCALATED: ((original_title))"
          Assigned To: Backup Approver
          Description: "Primary approver ((primary)) didn't respond within 4 hours"
          Timeout: 4 hours
        ↓
        Conditional: second timeout
            └─ True → Final escalation
                ↓
                Human in the Loop:
                  Assigned To: Department Head
                  Description: "URGENT: Double escalation"

Invalid Input

Validate human input:

Human in the Loop:
  Actions: [Approve, Deny]
  Input Fields: approval_amount (number)
  Outputs: action, approval_amount
    ↓
Conditional: approval_amount <= 0 OR approval_amount > max_amount
    └─ True → Invalid amount
        ↓
        Human in the Loop:
          Title: "Invalid Amount Entered"
          Description: "Amount must be between $0 and $((max_amount)). Please re-enter."
          Input Fields: approval_amount

Lost Approvals

Recover from workflow interruptions:

On Workflow Start:
    Check: pending_approvals table
    ↓
    Conditional: approval record exists for this workflow
        └─ True → Resume from approval
            ↓
            Read: stored approval state
            Continue workflow from HITL node

Best Practices

Use Sparingly: Only add human steps where truly needed Provide Context: Give reviewers all info needed to decide Clear Actions: Use descriptive button labels Set Timeouts: Don’t let approvals languish forever Escalation Paths: Define what happens if primary approver unavailable Track Metrics: Monitor approval times and bottlenecks Audit Trail: Log all approval decisions for compliance Test Thoroughly: Simulate approvals in development Notification Strategy: Ensure approvers are notified promptly Mobile-Friendly: Approval interfaces should work on mobile Feedback Loop: Show approvers the outcome of their decisions Training: Provide guidelines for consistent decision-making

Human-in-the-Loop workflows combine the best of automation and human judgment, enabling scalable systems that handle edge cases gracefully.

Error Handling

Error escalation and recovery patterns

Conditionals

Branch workflow logic based on conditions

Triggers

Trigger workflows for approval requests

Integrations

Send notifications via email, Slack, etc.

Introduction

Core Concepts

Platform

Infrastructure

Documentation Index

​When to Use Human-in-the-Loop

​Good Use Cases

​Poor Use Cases

​Human-in-the-Loop Node

​Basic Configuration

​Outputs

​Basic HITL Patterns

​Pattern 1: Simple Approval

​Pattern 2: Multi-Option Decision

​Pattern 3: Review and Correct

​Pattern 4: Exception Escalation

​Advanced HITL Patterns

​Pattern: Multi-Level Approval

​Pattern: Parallel Approvals

​Pattern: Approval with Delegation

​Pattern: Timeout with Default Action

​Pattern: Collect Training Data

​UX Best Practices

​Clear Context

​Actionable Buttons

​Show Relevant Data

​Provide Decision Guidance

​Notification Strategies

​Email Notifications

​Slack Notifications

​SMS for Urgent Approvals

​Tracking and Analytics

​Approval Metrics

​SLA Monitoring

​Approval Audit Trail

​Error Handling

​Approver Unavailable

​Invalid Input

​Lost Approvals

​Best Practices

Error Handling

Conditionals

Triggers

Integrations

When to Use Human-in-the-Loop

Good Use Cases

Poor Use Cases

Human-in-the-Loop Node

Basic Configuration

Outputs

Basic HITL Patterns

Pattern 1: Simple Approval

Pattern 2: Multi-Option Decision

Pattern 3: Review and Correct

Pattern 4: Exception Escalation

Advanced HITL Patterns

Pattern: Multi-Level Approval

Pattern: Parallel Approvals

Pattern: Approval with Delegation

Pattern: Timeout with Default Action

Pattern: Collect Training Data

UX Best Practices

Clear Context

Actionable Buttons

Show Relevant Data

Provide Decision Guidance

Notification Strategies

Email Notifications

Slack Notifications

SMS for Urgent Approvals

Tracking and Analytics

Approval Metrics

SLA Monitoring

Approval Audit Trail

Error Handling

Approver Unavailable

Invalid Input

Lost Approvals

Best Practices