N8N Queue Mode Deployment Guide¶

Quick Start¶

This guide explains how to deploy N8N in distributed queue mode on Azure for parallel workflow execution.

Prerequisites¶

1. Azure CLI installed and authenticated
2. Access to Azure Container Apps environment
3. Required secrets (see below)

Required Secrets¶

Set these environment variables before deployment:

export REDIS_PASSWORD="your-redis-password"
export DB_PASSWORD="your-db-password"
export N8N_ENCRYPTION_KEY="your-encryption-key"  # CRITICAL: Must be same across all instances
export N8N_ADMIN_PASSWORD="your-admin-password"

You can find these values in your .env file or Azure Key Vault.

Deployment Methods¶

Method 1: Using GitHub Actions (Recommended)¶

1. Go to GitHub Actions: .github/workflows/deploy-n8n-queue-mode.yml
2. Click "Run workflow"
3. Select action:
4. deploy-all - Full deployment (main + workers)
5. deploy-main - Update main instance only
6. deploy-workers - Deploy workers only
7. rollback - Revert to regular mode
8. Set worker replicas (default: min=2, max=10)
9. Click "Run workflow"

Method 2: Using Deployment Script¶

# Export secrets first (get values from Azure Key Vault or .env)
export REDIS_PASSWORD="<your-redis-password>"
export DB_PASSWORD="<your-db-password>"
export N8N_ENCRYPTION_KEY="<your-encryption-key>"
export N8N_ADMIN_PASSWORD="<your-admin-password>"

# Optional: Set N8N version
export N8N_VERSION="1.123.20"

# Run deployment
./scripts/deploy-n8n-queue-mode.sh deploy-all

Script Commands¶

# Deploy everything (main + workers)
./scripts/deploy-n8n-queue-mode.sh deploy-all

# Deploy main instance only
./scripts/deploy-n8n-queue-mode.sh deploy-main

# Deploy workers only
./scripts/deploy-n8n-queue-mode.sh deploy-workers

# Verify deployment
./scripts/deploy-n8n-queue-mode.sh verify

# Rollback to regular mode
./scripts/deploy-n8n-queue-mode.sh rollback

Method 3: Manual Azure CLI Commands¶

See the full commands in Technical Documentation.

Architecture¶

Before (Regular Mode): - 1 N8N instance handling UI + workflow execution - No horizontal scaling - Limited parallel workflow execution

After (Queue Mode): - 1 N8N Main instance (UI, API, webhooks, scheduling) - 2-10 N8N Worker instances (workflow execution only) - Redis queue (DB /6) for job distribution - Auto-scaling based on CPU/memory

Verification¶

1. Check Deployment Status¶

# Check main instance
az containerapp show \
  --name slidefactory-n8n \
  --resource-group rg-slidefactory-prod-001 \
  --query "{status:properties.provisioningState, queueMode:properties.template.containers[0].env[?name=='EXECUTIONS_MODE'].value | [0]}"

# Check workers
az containerapp show \
  --name slidefactory-n8n-worker \
  --resource-group rg-slidefactory-prod-001 \
  --query "{status:properties.provisioningState, replicas:properties.template.scale}"

2. View Logs¶

# Main instance logs
az containerapp logs show \
  --name slidefactory-n8n \
  --resource-group rg-slidefactory-prod-001 \
  --follow

# Worker logs
az containerapp logs show \
  --name slidefactory-n8n-worker \
  --resource-group rg-slidefactory-prod-001 \
  --follow

3. Check Redis Queue¶

# Connect to Redis
redis-cli -h slidefactory-redis.redis.cache.windows.net \
  -p 6380 \
  -a "<your-redis-password>" \
  --tls \
  -n 6

# Check queue status
KEYS "bull:*"
LLEN "bull:n8n:waiting"
LLEN "bull:n8n:active"

4. Test Workflow Execution¶

1. Open N8N: https://slidefactory-n8n.thankfulsmoke-fef50a06.westeurope.azurecontainerapps.io
2. Create and execute a test workflow
3. Check worker logs to see which worker processed the job
4. Execute multiple workflows simultaneously to verify parallel execution

Scaling Operations¶

Scale Workers Manually¶

# Scale up
az containerapp update \
  --name slidefactory-n8n-worker \
  --resource-group rg-slidefactory-prod-001 \
  --min-replicas 5 \
  --max-replicas 20

# Scale down
az containerapp update \
  --name slidefactory-n8n-worker \
  --resource-group rg-slidefactory-prod-001 \
  --min-replicas 1 \
  --max-replicas 5

Auto-Scaling Rules¶

Workers automatically scale based on: - CPU: Scales up when CPU > 70% - Memory: Scales up when memory > 80% - Cool-down: 5 minutes between scale operations

Monitoring¶

Key Metrics¶

1. Worker Count: Number of active worker replicas
2. Queue Depth: Pending jobs in Redis (LLEN "bull:n8n:waiting")
3. Execution Time: Workflow completion time
4. Success Rate: Percentage of successful executions

View Active Workers¶

az containerapp revision list \
  --name slidefactory-n8n-worker \
  --resource-group rg-slidefactory-prod-001 \
  --query "[?properties.active].{name:name, replicas:properties.replicas}"

Troubleshooting¶

Workers Not Picking Up Jobs¶

Symptoms: Jobs stay in waiting queue, workers idle

Diagnosis:

# Check worker logs for errors
az containerapp logs show \
  --name slidefactory-n8n-worker \
  --resource-group rg-slidefactory-prod-001 \
  --tail 100 | grep -i error

# Verify Redis connection
redis-cli -h slidefactory-redis.redis.cache.windows.net -p 6380 -a <password> --tls -n 6 PING

Solution: - Verify all environment variables are set correctly - Ensure N8N_ENCRYPTION_KEY matches between main and workers - Restart workers: az containerapp restart --name slidefactory-n8n-worker --resource-group rg-slidefactory-prod-001

High Queue Depth¶

Symptoms: Many jobs in waiting state, slow execution

Diagnosis:

redis-cli -h <host> -p 6380 -a <password> --tls -n 6 LLEN "bull:n8n:waiting"

Solution: - Increase worker replicas - Check worker logs for errors - Verify workflows are not stuck or failing

Database Connection Issues¶

Symptoms: "Too many connections" errors

Solution: - Reduce worker count temporarily - Increase PostgreSQL max_connections - Add PgBouncer connection pooler

Rollback¶

If issues occur, rollback to regular mode:

# Using script
./scripts/deploy-n8n-queue-mode.sh rollback

# Or via GitHub Actions
# Select action: rollback

This will: 1. Delete worker instances 2. Remove queue mode configuration from main instance 3. N8N returns to single-instance regular mode

Cost Implications¶

Current Setup (Regular Mode): - 1 instance: ~$30/month

Queue Mode Setup:

Recommendation: Start with Min=1, Max=10 for production (no cold start, reasonable cost)

Cost Optimization: - Use Min=0 for development/staging environments (saves ~$75/month per environment) - Set min replicas to 0 during off-hours (schedule-based scaling) - Monitor and tune resource limits - Use Azure Reserved Instances for 30-50% discount on baseline workers

Detailed Cost Analysis: See Cost Optimization Guide

Additional Resources¶

• Technical Implementation Guide
• N8N Official Queue Mode Documentation

Last Updated: 2025-11-25