Skip to content

Deployment Checklist: Workflow Engine Registration System

Date: 2025-11-04 Feature: Workflow Engine Registration Backend (Phase 1) Target: Preview → Main (Azure)

Pre-Deployment Verification

✅ Local Testing

  • Migration created: 453c1befe40_add_workflow_engines_table.py
  • Migration applied locally: docker compose exec web alembic upgrade head
  • Multi-source loading tested
  • Environment variable fallback working
  • Encryption/decryption functional
  • Zero breaking changes confirmed

⚠️ Known Cosmetic Issue

  • Warning: N8N engine disabled: API key or URL not configured appears in logs
  • Status: Cosmetic only - engine loads and works correctly
  • Cause: N8NWorkflowEngine.init checks global settings, but config overrides work
  • Impact: None - system fully functional
  • Fix: Optional cleanup in future update

Step 1: Azure Preview Database Migration

1.1 Check Current Migration Status (Optional)

# Set environment variables for preview database
export PGHOST="slidefactory-db.postgres.database.azure.com"
export PGPORT="5432"
export PGDATABASE="slidefactory-preview"
export PGUSER="dbadmin"
export PGPASSWORD="esLVoH6UVweWRVU9yC*u"
export PGSSLMODE="require"

# Check current alembic version
psql "postgresql://$PGUSER:$PGPASSWORD@$PGHOST:$PGPORT/$PGDATABASE?sslmode=require" \
  -c "SELECT version_num FROM alembic_version;"

Expected: Should show 9c7deff4ac4a (current head before our migration)

Option A: Via Azure Container App (Recommended)

# Connect to preview web container and run alembic
az containerapp exec \
  --name slidefactory-web-preview \
  --resource-group <your-resource-group> \
  --command "alembic upgrade head"

Expected Output: 

INFO  [alembic.runtime.migration] Running upgrade 9c7deff4ac4a -> 453c1befe40, add workflow_engines table

Option B: Via Docker (If testing locally first)

# Test migration locally first (optional but recommended)
docker compose exec web alembic upgrade head

# Or test with preview database connection
docker compose exec web env \
  DATABASE_URL="postgresql+psycopg2://dbadmin:esLVoH6UVweWRVU9yC*u@slidefactory-db.postgres.database.azure.com:5432/slidefactory-preview?sslmode=require&connect_timeout=30" \
  alembic upgrade head

Option C: Manually via psql (Fallback only if Alembic not available)

# ⚠️ Only use this if Options A/B don't work
# This manually applies the migration SQL

psql "postgresql://$PGUSER:$PGPASSWORD@$PGHOST:$PGPORT/$PGDATABASE?sslmode=require" << 'EOF'
BEGIN;

-- Create workflow_engines table
CREATE TABLE workflow_engines (
    id SERIAL PRIMARY KEY,
    name VARCHAR(255) NOT NULL UNIQUE,
    engine_type VARCHAR(50) NOT NULL,
    api_url VARCHAR(500) NOT NULL,
    api_key_encrypted TEXT,
    active BOOLEAN NOT NULL DEFAULT true,
    config JSONB DEFAULT '{}',
    created_by INTEGER REFERENCES app_users(id),
    created_at TIMESTAMP NOT NULL DEFAULT now(),
    updated_at TIMESTAMP NOT NULL DEFAULT now()
);

-- Create indexes
CREATE INDEX ix_workflow_engines_id ON workflow_engines(id);
CREATE UNIQUE INDEX ix_workflow_engines_name ON workflow_engines(name);
CREATE INDEX ix_workflow_engines_engine_type ON workflow_engines(engine_type);
CREATE INDEX ix_workflow_engines_active ON workflow_engines(active);

-- Update alembic version
UPDATE alembic_version SET version_num = '453c1befe40';

COMMIT;
EOF

1.3 Verify Migration

# Check migration applied
psql "postgresql://$PGUSER:$PGPASSWORD@$PGHOST:$PGPORT/$PGDATABASE?sslmode=require" \
  -c "SELECT version_num FROM alembic_version;"

# Expected: 453c1befe40

# Check table exists
psql "postgresql://$PGUSER:$PGPASSWORD@$PGHOST:$PGPORT/$PGDATABASE?sslmode=require" \
  -c "SELECT COUNT(*) FROM workflow_engines;"

# Expected: 0 (empty table)

# Check table structure
psql "postgresql://$PGUSER:$PGPASSWORD@$PGHOST:$PGPORT/$PGDATABASE?sslmode=require" \
  -c "\d workflow_engines"

# Expected: Shows table with columns: id, name, engine_type, api_url, api_key_encrypted, active, config, created_by, created_at, updated_at

Step 2: Azure Main Database Migration

⚠️ Only proceed after preview migration successful and tested!

2.1 Run Migration on Main via Alembic

Option A: Via Azure Container App (Recommended)

# Execute migration in main web container
az containerapp exec \
  --name slidefactory-web-main \
  --resource-group <your-resource-group> \
  --command "alembic upgrade head"

Expected Output: 

INFO  [alembic.runtime.migration] Running upgrade 9c7deff4ac4a -> 453c1befe40, add workflow_engines table

Option B: Via Docker (If testing locally first)

# Test with main database connection
docker compose exec web env \
  DATABASE_URL="postgresql+psycopg2://dbadmin:esLVoH6UVweWRVU9yC*u@slidefactory-db.postgres.database.azure.com:5432/slidefactory?sslmode=require&connect_timeout=30" \
  alembic upgrade head

Option C: Manually via psql (Fallback only)

# ⚠️ Only use this if Options A/B don't work
export PGDATABASE="slidefactory"  # Changed from slidefactory-preview

# Same SQL as preview (Step 1.2 Option C)
psql "postgresql://$PGUSER:$PGPASSWORD@$PGHOST:$PGPORT/$PGDATABASE?sslmode=require" << 'EOF'
[Same CREATE TABLE SQL as in Step 1.2 Option C]
EOF

2.2 Verify Main Database

# Set database to main
export PGDATABASE="slidefactory"

# Check migration applied
psql "postgresql://$PGUSER:$PGPASSWORD@$PGHOST:$PGPORT/$PGDATABASE?sslmode=require" \
  -c "SELECT version_num FROM alembic_version;"

# Expected: 453c1befe40

# Check table exists
psql "postgresql://$PGUSER:$PGPASSWORD@$PGHOST:$PGPORT/$PGDATABASE?sslmode=require" \
  -c "SELECT COUNT(*) FROM workflow_engines;"

# Expected: 0 (empty table)

# Check table structure
psql "postgresql://$PGUSER:$PGPASSWORD@$PGHOST:$PGPORT/$PGDATABASE?sslmode=require" \
  -c "\d workflow_engines"

# Expected: Shows table structure

Step 3: Git Commit & Push to Preview

3.1 Review Changes

# Check all modified files
git status

# Expected files:
# New files:
#   app/workflowengine/db_models.py
#   app/workflowengine/crud.py
#   app/workflowengine/encryption.py
#   app/workflowengine/schemas.py
#   alembic/versions/453c1befe40_add_workflow_engines_table.py
#   test_workflow_engines.py
#   ../../ROADMAP.md
#   TESTING_GUIDE.md
#   DEPLOYMENT_CHECKLIST.md
#   ../reports/technical/2025-11-04_workflow_engine_registration_system.md
#
# Modified files:
#   app/workflowengine/registry.py
#   app/main.py
#   docker-compose.override.yml
#   .env.local

3.2 Review Diffs

# Review each modified file
git diff app/workflowengine/registry.py
git diff app/main.py
git diff docker-compose.override.yml
git diff .env.local

3.3 Commit Changes

# Add all changes
git add app/workflowengine/db_models.py
git add app/workflowengine/crud.py
git add app/workflowengine/encryption.py
git add app/workflowengine/schemas.py
git add alembic/versions/453c1befe40_add_workflow_engines_table.py
git add test_workflow_engines.py
git add ../../ROADMAP.md
git add TESTING_GUIDE.md
git add DEPLOYMENT_CHECKLIST.md
git add ../reports/technical/2025-11-04_workflow_engine_registration_system.md
git add app/workflowengine/registry.py
git add app/main.py
git add docker-compose.override.yml
git add .env.local

# Commit with descriptive message
git commit -m "feat: implement workflow engine registration system (Phase 1)

- Add database model for workflow engine registration
- Implement API key encryption (Fernet)
- Add CRUD operations with validation
- Implement multi-source engine loading (database + environment)
- Add Alembic migration for workflow_engines table
- Maintain backward compatibility with environment variables
- Update registry to support string-based engine keys
- Integrate with application startup (lifespan)
- Add Pydantic schemas for API
- Remove n8n-init container (no longer needed)
- Add comprehensive testing guide and roadmap

Changes:
- ✅ Zero breaking changes - Azure deployment unchanged
- ✅ Environment variable fallback working
- ✅ Foundation for multi-engine support (N8N, Prefect, Windmill)
- ✅ Encrypted API key storage

See [Workflow Engine Registration System](../archive/technical-reports/2025-11-04_workflow_engine_registration_system.md) for details (archived)
See TESTING_GUIDE.md for testing instructions

🤖 Generated with Claude Code
Co-Authored-By: Claude <noreply@anthropic.com>"

3.4 Push to Preview Branch

# Push to preview branch
git push origin preview

# This will trigger GitHub Actions workflow:
# - Build Docker image
# - Push to Azure Container Registry
# - Deploy to slidefactory-web-preview
# - Deploy to slidefactory-worker-preview

3.5 Monitor Deployment

# Watch GitHub Actions
# Go to: https://github.com/your-org/your-repo/actions

# Or via CLI (if gh CLI installed)
gh run watch

# Check deployment logs
az containerapp logs show \
  --name slidefactory-web-preview \
  --resource-group <your-resource-group> \
  --follow

3.6 Verify Preview Deployment

# Check preview logs for successful startup
az containerapp logs show \
  --name slidefactory-web-preview \
  --resource-group <your-resource-group> \
  --tail 100 | grep -i workflow

# Expected logs:
# - "Initializing workflow engine registry (multi-source)"
# - "Loading workflow engines from environment variables"
# - "Workflow registry initialized with X engines"
# - "Workflows  : X/X engines enabled"

Access preview URL and verify: - [ ] Application starts without errors - [ ] Login works - [ ] Workflows page accessible - [ ] No console errors

Step 4: Merge to Main

4.1 Final Preview Verification

Before merging, verify preview is stable: - [ ] Application running smoothly for 10+ minutes - [ ] No errors in logs - [ ] Login and authentication working - [ ] Workflows accessible - [ ] Database migration successful

4.2 Merge Preview to Main

# Switch to main branch
git checkout main

# Pull latest
git pull origin main

# Merge preview
git merge preview

# Resolve any conflicts if necessary
# (Should be clean merge if no parallel work)

# Push to main
git push origin main

4.3 Monitor Main Deployment

# Watch GitHub Actions for main deployment
gh run watch

# Check main deployment logs
az containerapp logs show \
  --name slidefactory-web-main \
  --resource-group <your-resource-group> \
  --follow

4.4 Verify Main Deployment

# Check main logs
az containerapp logs show \
  --name slidefactory-web-main \
  --resource-group <your-resource-group> \
  --tail 100 | grep -i workflow

# Expected: Same successful logs as preview

Access production URL and verify: - [ ] Application starts without errors - [ ] Login works - [ ] Workflows page accessible - [ ] No console errors - [ ] Environment variable engine loaded (if configured)

Rollback Plan (If Needed)

If Preview Deployment Fails:

# Rollback preview database migration
psql "postgresql://$PGUSER:$PGPASSWORD@$PGHOST:$PGPORT/slidefactory-preview?sslmode=require" << 'EOF'
BEGIN;

-- Drop table
DROP TABLE IF EXISTS workflow_engines CASCADE;

-- Rollback alembic version
UPDATE alembic_version SET version_num = '9c7deff4ac4a';

COMMIT;
EOF

# Revert preview branch
git checkout preview
git reset --hard origin/preview~1
git push -f origin preview

If Main Deployment Fails:

# Rollback main database migration
psql "postgresql://$PGUSER:$PGPASSWORD@$PGHOST:$PGPORT/slidefactory?sslmode=require" << 'EOF'
BEGIN;

-- Drop table
DROP TABLE IF EXISTS workflow_engines CASCADE;

-- Rollback alembic version
UPDATE alembic_version SET version_num = '9c7deff4ac4a';

COMMIT;
EOF

# Revert main branch
git checkout main
git reset --hard origin/main~1
git push -f origin main

Post-Deployment Verification

Smoke Tests

Preview: 

# Test environment variable engine loading
curl -s https://your-preview-url.azurewebsites.net/ | grep -i workflow

Main: 

# Test environment variable engine loading
curl -s https://your-main-url.com/ | grep -i workflow

Database Checks

# Check preview database
psql "postgresql://$PGUSER:$PGPASSWORD@$PGHOST:$PGPORT/slidefactory-preview?sslmode=require" \
  -c "SELECT COUNT(*) as engine_count FROM workflow_engines;"

# Check main database
psql "postgresql://$PGUSER:$PGPASSWORD@$PGHOST:$PGPORT/slidefactory?sslmode=require" \
  -c "SELECT COUNT(*) as engine_count FROM workflow_engines;"

# Both should show: 0 (no engines registered yet - env fallback working)

Success Criteria

✅ Deployment Successful If:

  • Both databases migrated successfully
  • Preview deployment healthy
  • Main deployment healthy
  • No errors in application logs
  • Environment variable engine loading works
  • Login and authentication functional
  • Workflows page accessible
  • Zero breaking changes confirmed

🎉 Congratulations!

Phase 1 (Backend) is now deployed to production!

Next Steps: - Monitor application for 24-48 hours - Review logs for any unexpected warnings

Notes

Environment Variable Configuration

Current Azure Setup: - N8N_API_URL and N8N_API_KEY set in GitHub Secrets - Engine loads as "n8n-default" from environment - No database registration needed - Backward compatible with existing setup

Future Phase 2 (Admin UI)

Once UI is implemented, users can: - Register additional engines via Admin UI - Manage multiple N8N instances - Add Prefect, Windmill, other engines - Database-registered engines take precedence

SECRET_KEY Requirement

⚠️ Important: Ensure SECRET_KEY is set in Azure Container App environment: - Preview: PREVIEW_SECRET_KEY GitHub Secret - Main: SECRET_KEY GitHub Secret - Used for API key encryption/decryption - Must remain consistent (changing it breaks encrypted keys)

Contact

For issues or questions during deployment: - Check logs first: az containerapp logs show --name <app-name> --resource-group <rg> --tail 200 - Review TESTING_GUIDE.md for troubleshooting - See Workflow Engine Registration System for technical details (archived)