Skip to content

Test Infrastructure Quick Fix - Service Connection Mocking

Date: 2025-11-13 Issue: Tests hanging due to import-time service connections Solution: Mock external services at import time in conftest.py Status: ✅ Implemented and Working

Problem Analysis

Root Cause

Tests were hanging indefinitely because the application attempts to connect to external services (Redis, MinIO) at module import time:

  1. Celery (app/celery_app.py): Attempts Redis connection at import
  2. MinIO (app/filemanager/storage/minio_client.py): Instantiates Minio client in constructor
  3. Test Environment: These services aren't available during test collection

Why Previous Fixes Failed

  • Environment variables were set in conftest.py, but too late
  • Modules import and connect before pytest fixtures run
  • Connection attempts block indefinitely when services unavailable

Solution Implemented

Approach: Import-Time Mocking (Quick Fix)

Added mocks in tests/conftest.py that execute before any app modules import:

Changes: 1 file, ~45 lines added

What Was Mocked

  1. Celery Mock (lines 36-43):
  2. Mocks app.celery_app module
  3. Provides pass-through decorators for @celery.task

  4. Prevents Redis connection attempts

  5. MinIO Mock (lines 45-77):

  6. Mocks minio.Minio class
  7. Provides MockMinioStorageClient with all required methods
  8. Returns realistic mock data for storage operations

Key Implementation Details

# Mock Celery before import
mock_celery = MagicMock()
mock_celery.task = lambda *args, **kwargs: lambda f: f  # Pass-through decorator
sys.modules['app.celery_app'] = MagicMock(celery_app=mock_celery)

# Mock MinIO before import
sys.modules['minio'] = MagicMock(Minio=MagicMock(return_value=mock_minio_client))

Critical: These mocks are inserted into sys.modules before any app.* imports occur.

Test Results

Before Fix

  • Tests hung indefinitely
  • Required manual CTRL+C to terminate
  • No test output

After Fix

# CLI tests
========================= 7 passed, 1 skipped in 0.78s =========================

# Unit tests
============================== 5 passed in 0.72s ===============================

Verification

timeout 10 .venv/bin/python -c "from tests import conftest; from app import main"
# ✓ Completes in <1 second (was hanging forever)

Scope of Changes

Metric Value
Files Changed 1
Lines Added ~45
Lines Modified 0
Production Code Changed No
Risk Level Very Low

Trade-offs

Advantages ✅

  • Minimal changes: Only 1 test file modified
  • Zero production risk: No production code touched
  • Fast implementation: ~5 minutes
  • Immediate results: Tests unblocked immediately
  • Safe: Mocks are only active during tests

Limitations ⚠️

  • Architectural issue not fixed: Services still connect at import time
  • Integration tests: Can't easily test real Redis/MinIO connections
  • Mock maintenance: Need to update mocks if service APIs change
  • Technical debt: Better solved with lazy initialization (future work)

Future Improvements (Not Urgent)

Option: Lazy Initialization Refactor

Convert service connections to lazy initialization (initialize on first use, not at import):

Scope: - 20 files touched - ~150 lines modified - Better architecture - Takes ~2 hours

Recommendation: Plan this as a separate refactoring task when time permits. The quick fix is sufficient for now.

Testing Strategy

The mocked services support: - ✅ Unit tests (no real services needed) - ✅ CLI tests (using mocks) - ⚠️ Integration tests (can override mocks with real services if needed)

For integration tests requiring real services: 

@pytest.mark.integration
def test_with_real_redis():
    # Can patch/unpatch to use real services
    pass

Conclusion

Status: Problem solved with minimal code changes.

The quick fix successfully unblocks all tests by mocking external services at import time. While this doesn't address the architectural issue (services shouldn't connect at import), it's a pragmatic solution that:

  1. ✅ Fixes the immediate problem (tests hanging)
  2. ✅ Minimal code changes (45 lines in 1 file)
  3. ✅ Zero production risk
  4. ✅ Allows tests to run successfully

The proper architectural fix (lazy initialization) can be planned as a future improvement when time permits.

  • Modified: tests/conftest.py (lines 12-77)
  • Production code: No changes required
  • Test results: All passing (7 CLI + 5 unit tests verified)