Slidefactory Core Migration - CLI Command Plan¶
Date: 2025-11-20 Status: Implementation Plan Related Documents: - 2025-11-18_repository_separation_quick_reference.md - 2025-11-18_repository_separation_implementation_guide.md
Prerequisites¶
- Current directory:
/home/cgast/Github/slidefactory/s5-slidefactory - Target directory:
/home/cgast/Github/slidefactory/slidefactory-core(exists, git initialized) - Both repositories should be on their respective branches before starting
Phase 1: Prepare Core Repository Structure¶
Step 1.1: Create Base Directory Structure¶
cd ../slidefactory-core
# Create src-based package structure
mkdir -p src/slidefactory
mkdir -p static
mkdir -p templates
mkdir -p packages
mkdir -p tests
mkdir -p .github/workflows
mkdir -p docs
Step 1.2: Create Core Package Files¶
# Create __init__.py for package
cat > src/slidefactory/__init__.py << 'EOF'
"""
Slidefactory Core - AI-driven presentation automation platform.
This package provides the core engine for AI-powered presentation generation,
workflow integration, and document processing.
"""
__version__ = "1.0.0"
__author__ = "Christian Gast"
# Export key components
from slidefactory.app.config import settings, Config
from slidefactory.app.main import app
__all__ = [
"__version__",
"settings",
"Config",
"app",
]
EOF
Step 1.3: Create pyproject.toml¶
cat > pyproject.toml << 'EOF'
[build-system]
requires = ["setuptools>=61.0", "wheel"]
build-backend = "setuptools.build_meta"
[project]
name = "slidefactory-core"
version = "1.0.0"
description = "AI-driven presentation automation platform - Core engine"
readme = "README.md"
requires-python = ">=3.11"
license = {text = "Proprietary"}
authors = [
{name = "Christian Gast", email = "cg@cgast.de"}
]
keywords = ["presentations", "automation", "ai", "n8n", "fastapi"]
classifiers = [
"Development Status :: 4 - Beta",
"Intended Audience :: Developers",
"Programming Language :: Python :: 3",
"Programming Language :: Python :: 3.11",
"Framework :: FastAPI",
]
dependencies = [
"fastapi>=0.104.0",
"uvicorn[standard]>=0.24.0",
"sqlalchemy>=2.0.0",
"psycopg2-binary>=2.9.9",
"alembic>=1.12.0",
"celery>=5.3.4",
"redis>=5.0.1",
"jinja2>=3.1.2",
"jinja2-fragments>=1.2.0",
"python-pptx>=0.6.23",
"pillow>=10.1.0",
"requests>=2.31.0",
"python-dotenv>=1.0.0",
"pydantic>=2.5.0",
"pydantic-settings>=2.1.0",
"httpx>=0.25.2",
"python-multipart>=0.0.6",
"python-jose[cryptography]>=3.3.0",
"passlib[bcrypt]>=1.7.4",
"fastapi-sessions>=0.3.2",
"sse-starlette>=1.8.2",
"apscheduler>=3.10.4",
"mistralai>=0.0.7",
"openai>=1.3.0",
"anthropic>=0.7.0",
"pgvector>=0.2.3",
"beautifulsoup4>=4.12.2",
"lxml>=4.9.3",
"markdown>=3.5.1",
"pypdf>=3.17.1",
"openpyxl>=3.1.2",
]
[project.optional-dependencies]
dev = [
"pytest>=7.4.3",
"pytest-asyncio>=0.21.1",
"pytest-cov>=4.1.0",
"pytest-xdist>=3.5.0",
"black>=23.12.0",
"flake8>=6.1.0",
"mypy>=1.7.1",
]
azure = [
"azure-storage-blob>=12.19.0",
"azure-identity>=1.15.0",
"msal>=1.26.0",
]
minio = [
"minio>=7.2.0",
]
[project.scripts]
slidefactory = "slidefactory.cli.main:main"
[project.urls]
Homepage = "https://github.com/cgast/slidefactory-core"
Repository = "https://github.com/cgast/slidefactory-core"
Issues = "https://github.com/cgast/slidefactory-core/issues"
[tool.setuptools]
packages = ["slidefactory"]
package-dir = {"" = "src"}
[tool.setuptools.package-data]
slidefactory = [
"templates/**/*.j2",
"templates/**/*.html",
"static/**/*",
"alembic.ini",
"alembic/**/*",
]
[tool.black]
line-length = 120
target-version = ['py311']
[tool.pytest.ini_options]
testpaths = ["tests"]
python_files = ["test_*.py", "*_test.py"]
python_classes = ["Test*"]
python_functions = ["test_*"]
markers = [
"unit: Unit tests",
"integration: Integration tests",
"e2e: End-to-end tests",
]
EOF
Phase 2: Copy Core Application Code¶
Step 2.1: Copy App Directory¶
cd ../slidefactory-core
# Copy entire app directory to src/slidefactory/app
cp -r ../s5-slidefactory/app src/slidefactory/
# Remove Python cache files
find src/slidefactory/app -type d -name "__pycache__" -exec rm -rf {} + 2>/dev/null || true
find src/slidefactory/app -type f -name "*.pyc" -delete
Step 2.2: Copy CLI¶
# Copy CLI directory
cp -r ../s5-slidefactory/cli src/slidefactory/
# Clean cache
find src/slidefactory/cli -type d -name "__pycache__" -exec rm -rf {} + 2>/dev/null || true
find src/slidefactory/cli -type f -name "*.pyc" -delete
Step 2.3: Copy Alembic (Database Migrations)¶
# Copy alembic directory
cp -r ../s5-slidefactory/alembic src/slidefactory/
# Copy alembic.ini
cp ../s5-slidefactory/alembic.ini src/slidefactory/
# Clean cache
find src/slidefactory/alembic -type d -name "__pycache__" -exec rm -rf {} + 2>/dev/null || true
Step 2.4: Copy Core Static Assets (Generic Only)¶
# Copy core CSS, JS, vendor files
cp -r ../s5-slidefactory/static/css static/
cp -r ../s5-slidefactory/static/js static/
cp -r ../s5-slidefactory/static/vendor static/
cp -r ../s5-slidefactory/static/fonts static/
# Copy generic images ONLY (not S5 branding)
mkdir -p static/img
# We'll manually filter this - need to identify generic vs S5-specific
Step 2.5: Manually Review and Copy Generic Images¶
# List current images in s5-slidefactory to identify generic ones
cd ../s5-slidefactory
ls -la static/img/
# Then manually copy ONLY generic ones to core
# Example (adjust based on actual generic files):
cd ../slidefactory-core
cp ../s5-slidefactory/static/img/logo.png static/img/ 2>/dev/null || echo "File not found, skip"
cp ../s5-slidefactory/static/img/favicon.ico static/img/ 2>/dev/null || echo "File not found, skip"
# DO NOT COPY:
# - sportfive_logo.png
# - sportfive_large.png
# - sf_logo_*
# - Any S5-specific branding
Step 2.6: Copy Core Templates (Generic Only)¶
cd ../slidefactory-core
# Copy all templates first
cp -r ../s5-slidefactory/templates/* templates/
# Review templates for S5-specific content
# We'll need to manually review and remove/genericize S5 branding references
Step 2.7: Copy N8N Package¶
Step 2.8: Copy Core Tests¶
# Copy tests directory
cp -r ../s5-slidefactory/tests .
# Clean cache
find tests -type d -name "__pycache__" -exec rm -rf {} + 2>/dev/null || true
find tests -type f -name "*.pyc" -delete
Step 2.9: Copy Requirements and Configuration Files¶
# Copy requirements.txt
cp ../s5-slidefactory/requirements.txt .
# Copy .env.example (if exists, or create one)
cp ../s5-slidefactory/.env.example .env.example 2>/dev/null || touch .env.example
# Copy init.sql for database initialization
cp ../s5-slidefactory/init.sql . 2>/dev/null || echo "No init.sql found"
# Copy pytest.ini if exists
cp ../s5-slidefactory/pytest.ini . 2>/dev/null || echo "No pytest.ini found"
Phase 3: Fix Import Paths¶
Step 3.1: Update All Import Statements¶
All imports need to change from from app. to from slidefactory.app.:
cd ../slidefactory-core
# Update imports in app/ directory
find src/slidefactory/app -type f -name "*.py" -exec sed -i 's/from app\./from slidefactory.app./g' {} +
find src/slidefactory/app -type f -name "*.py" -exec sed -i 's/import app\./import slidefactory.app./g' {} +
# Update imports in cli/ directory
find src/slidefactory/cli -type f -name "*.py" -exec sed -i 's/from app\./from slidefactory.app./g' {} +
find src/slidefactory/cli -type f -name "*.py" -exec sed -i 's/import app\./import slidefactory.app./g' {} +
# Update imports in alembic/ directory
find src/slidefactory/alembic -type f -name "*.py" -exec sed -i 's/from app\./from slidefactory.app./g' {} +
find src/slidefactory/alembic -type f -name "*.py" -exec sed -i 's/import app\./import slidefactory.app./g' {} +
# Update CLI-specific imports (from cli. to from slidefactory.cli.)
find src/slidefactory/cli -type f -name "*.py" -exec sed -i 's/from cli\./from slidefactory.cli./g' {} +
Step 3.2: Update Alembic env.py for New Structure¶
Manual edit required for src/slidefactory/alembic/env.py: - Change sys.path.insert(0, os.path.dirname(os.path.dirname(__file__))) to account for new structure - Update imports from app. to slidefactory.app.
Step 3.3: Update Template Paths in Code¶
Templates will still be in templates/ at root, but may need path adjustments:
# Search for template loading code and verify paths
grep -r "templates" src/slidefactory/app --include="*.py" | grep -i "jinja\|template"
# Manual review and fixes needed
Phase 4: Update Configuration for Core Package¶
Step 4.1: Remove Client-Specific Defaults¶
Edit src/slidefactory/app/config.py:
# Manual edits needed:
# - Remove any S5-specific defaults
# - Ensure CLIENT_NAME, CLIENT_LOGO, etc. use environment variables
# - Remove hardcoded Azure/deployment URLs
# - Make all client-specific settings configurable via env vars
Step 4.2: Create Generic .env.example¶
cat > .env.example << 'EOF'
# Slidefactory Core - Environment Configuration Template
# Application
VERSION=1.0.0
CLIENT_NAME=Slidefactory
CLIENT_LOGO=/static/img/logo.png
CLIENT_PRIMARY_COLOR=#007bff
# Database
DATABASE_URL=postgresql://postgres:postgres@localhost:5432/slidefactory
# Redis
REDIS_HOST=localhost
REDIS_PORT=6379
REDIS_PASSWORD=
CELERY_BROKER_URL=redis://localhost:6379/0
CELERY_RESULT_BACKEND=redis://localhost:6379/0
# Storage (minio or azure)
STORAGE_PROVIDER=minio
MINIO_CLIENT_URL=localhost:9000
MINIO_ACCESS_KEY=admin
MINIO_SECRET_KEY=adminadminadmin
# Azure Storage (if using azure provider)
# AZURE_STORAGE_ACCOUNT_NAME=
# AZURE_STORAGE_ACCOUNT_KEY=
# AI Providers
AI_PROVIDER=openai
AI_MODEL=gpt-4
OPENAI_API_KEY=
# Optional AI Providers
# AZURE_OPENAI_API_KEY=
# MISTRAL_API_KEY=
# ANTHROPIC_API_KEY=
# N8N Integration
N8N_API_URL=http://localhost:5678/api/v1
N8N_API_KEY=
# Jina AI (for document processing)
JINA_API_KEY=
# Security
SECRET_KEY=your-secret-key-here
# Deployment
DEPLOYMENT_PLATFORM=docker
SERVER_URL=http://localhost:8080
EOF
Phase 5: Create Docker Base Image¶
Step 5.1: Create Dockerfile.base¶
cat > Dockerfile.base << 'EOF'
FROM python:3.11-slim
# Set working directory
WORKDIR /code
# Install system dependencies
RUN apt-get update && apt-get install -y \
gcc \
g++ \
libpq-dev \
&& rm -rf /var/lib/apt/lists/*
# Copy requirements
COPY requirements.txt /code/
RUN pip install --no-cache-dir -r requirements.txt
# Copy package files
COPY pyproject.toml /code/
COPY src/ /code/src/
COPY static/ /code/static/
COPY templates/ /code/templates/
# Install package
RUN pip install -e .
# Expose port
EXPOSE 8080
# Default command (override in client repos)
CMD ["uvicorn", "slidefactory.app.main:app", "--host", "0.0.0.0", "--port", "8080"]
EOF
Phase 6: Create GitHub Actions Workflows¶
Step 6.1: Create Test Workflow¶
mkdir -p .github/workflows
cat > .github/workflows/test.yml << 'EOF'
name: Test Core
on:
pull_request:
branches: [main, develop]
push:
branches: [main, develop]
jobs:
test:
runs-on: ubuntu-latest
services:
postgres:
image: ankane/pgvector:latest
env:
POSTGRES_PASSWORD: postgres
POSTGRES_DB: slidefactory_test
options: >-
--health-cmd pg_isready
--health-interval 10s
--health-timeout 5s
--health-retries 5
ports:
- 5432:5432
redis:
image: redis:7-alpine
options: >-
--health-cmd "redis-cli ping"
--health-interval 10s
--health-timeout 5s
--health-retries 5
ports:
- 6379:6379
steps:
- uses: actions/checkout@v4
- name: Set up Python
uses: actions/setup-python@v4
with:
python-version: '3.11'
- name: Install dependencies
run: |
pip install -e ".[dev]"
- name: Run tests
env:
DATABASE_URL: postgresql://postgres:postgres@localhost:5432/slidefactory_test
REDIS_HOST: localhost
REDIS_PORT: 6379
run: |
pytest tests/ -v --cov=slidefactory --cov-report=xml
- name: Upload coverage
uses: codecov/codecov-action@v3
with:
file: ./coverage.xml
EOF
Step 6.2: Create Publish Workflow¶
cat > .github/workflows/publish.yml << 'EOF'
name: Publish to GitHub Packages
on:
release:
types: [published]
workflow_dispatch:
inputs:
version:
description: 'Version to publish (e.g., 1.0.0)'
required: true
jobs:
publish:
runs-on: ubuntu-latest
permissions:
contents: read
packages: write
steps:
- uses: actions/checkout@v4
- name: Set up Python
uses: actions/setup-python@v4
with:
python-version: '3.11'
- name: Install build tools
run: |
pip install build twine
- name: Update version
if: github.event_name == 'workflow_dispatch'
run: |
VERSION="${{ github.event.inputs.version }}"
sed -i "s/version = \".*\"/version = \"${VERSION}\"/" pyproject.toml
sed -i "s/__version__ = \".*\"/__version__ = \"${VERSION}\"/" src/slidefactory/__init__.py
- name: Build package
run: |
python -m build
- name: Publish to GitHub Packages
env:
TWINE_USERNAME: __token__
TWINE_PASSWORD: ${{ secrets.GITHUB_TOKEN }}
run: |
twine upload dist/* --repository-url https://upload.github.com/pypi/${{ github.repository_owner }}/
EOF
Phase 7: Create Documentation¶
Step 7.1: Create Core README¶
cat > README.md << 'EOF'
# Slidefactory Core
AI-driven presentation automation platform - Core engine.
## Overview
Slidefactory Core provides the foundational platform for AI-powered presentation generation, workflow orchestration, and document processing. This package is designed to be deployed by client-specific repositories.
## Features
- **Presentation Generation**: PowerPoint automation with Jinja2 templating
- **AI Provider Integration**: OpenAI, Azure OpenAI, Mistral, Anthropic, Ollama, OpenRouter
- **Workflow Orchestration**: N8N, Prefect, Windmill integration
- **Document Processing**: PDF/text ingestion, chunking, embedding, vector search
- **Multi-Cloud Storage**: MinIO, Azure Blob Storage
- **Database Migrations**: Alembic-based schema management
- **CLI Tool**: Complete management interface
## Installation
### From GitHub Packages
pip install slidefactory-core --index-url https://github.com/cgast/packages/pypi/simple/
### From Source (Development)
git clone https://github.com/cgast/slidefactory-core.git
cd slidefactory-core
pip install -e ".[dev]"
## Usage
### CLI Tool
# Initialize system
slidefactory init all
# Manage users
slidefactory user create-local user@example.com --name "John Doe"
# Manage API keys
slidefactory api-key create "My Key" --user-id 1
# Generate presentations
slidefactory presentation generate --template-id 123 --data-file data.json
## Configuration
All configuration via environment variables. See `.env.example`.
## Development
# Run tests
pytest tests/
# Run with coverage
pytest --cov=slidefactory --cov-report=html
## Versioning
Semantic versioning (MAJOR.MINOR.PATCH):
- **MAJOR**: Breaking changes
- **MINOR**: New features (backwards compatible)
- **PATCH**: Bug fixes
## License
Proprietary - Private use only.
## Client Deployments
This core package is used by:
- [s5-slidefactory](https://github.com/cgast/s5-slidefactory) - Azure Container Apps
- [eddy-slidefactory](https://github.com/cgast/eddy-slidefactory) - Docker/Coolify
## Contributing
Create feature branch from `main`, make changes with tests, create PR.
EOF
Step 7.2: Create CHANGELOG.md¶
cat > CHANGELOG.md << 'EOF'
# Changelog
All notable changes to slidefactory-core will be documented in this file.
The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/),
and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
## [Unreleased]
### Added
- Initial core package structure
## [1.0.0] - 2025-11-20
### Added
- Initial release
- Core FastAPI application
- AI provider integration (OpenAI, Azure, Mistral, Anthropic, Ollama)
- N8N workflow integration
- Document processing with vector search
- Multi-cloud storage (MinIO, Azure)
- CLI tool
- Database migrations with Alembic
EOF
Phase 8: Create .gitignore¶
cat > .gitignore << 'EOF'
# Python
__pycache__/
*.py[cod]
*$py.class
*.so
.Python
build/
develop-eggs/
dist/
downloads/
eggs/
.eggs/
lib/
lib64/
parts/
sdist/
var/
wheels/
*.egg-info/
.installed.cfg
*.egg
# Virtual environments
venv/
env/
ENV/
# IDE
.vscode/
.idea/
*.swp
*.swo
# Environment
.env
.env.local
.env.*.local
# Testing
.coverage
htmlcov/
.pytest_cache/
.tox/
# Logs
*.log
# Database
*.db
*.sqlite
# OS
.DS_Store
Thumbs.db
EOF
Phase 9: Verify and Test¶
Step 9.1: Verify Package Structure¶
cd ../slidefactory-core
# Verify structure
tree -L 3 -I '__pycache__|*.pyc|venv|.git'
# Expected structure:
# .
# ├── src/
# │ └── slidefactory/
# │ ├── __init__.py
# │ ├── app/
# │ ├── cli/
# │ └── alembic/
# ├── static/
# ├── templates/
# ├── packages/
# ├── tests/
# ├── pyproject.toml
# ├── requirements.txt
# ├── README.md
# └── ...
Step 9.2: Test Installation¶
# Create virtual environment
python3 -m venv venv
source venv/bin/activate
# Install package in editable mode
pip install -e ".[dev]"
# Verify installation
python -c "import slidefactory; print(slidefactory.__version__)"
# Verify CLI
slidefactory --help
Step 9.3: Run Tests¶
# Run tests (will need services running)
pytest tests/ -v
# Or just check if imports work
pytest tests/ -v --collect-only
Step 9.4: Test Import Paths¶
# Test that all imports resolve correctly
python << 'EOF'
try:
from slidefactory.app.main import app
from slidefactory.app.config import settings
from slidefactory.cli.main import main
print("✅ All imports successful")
except ImportError as e:
print(f"❌ Import error: {e}")
EOF
Phase 10: Commit Core Repository¶
Step 10.1: Initial Commit¶
cd ../slidefactory-core
# Stage all files
git add .
# Review what will be committed
git status
# Create initial commit
git commit -m "feat: initial slidefactory-core package structure
- Add src/slidefactory package structure
- Add FastAPI application core
- Add CLI tool
- Add Alembic database migrations
- Add tests
- Add pyproject.toml with dependencies
- Add GitHub Actions workflows
- Add documentation
This is the core package extracted from s5-slidefactory for multi-client deployment.
🤖 Generated with Claude Code"
# Push to remote (if remote is configured)
git push origin main
Phase 11: Cleanup Validation¶
Step 11.1: Document What Was Moved¶
cd ../s5-slidefactory
# Create a migration record
cat > docs/reports/technical/2025-11-20_core_files_moved.md << 'EOF'
# Core Files Moved to slidefactory-core
## Directories Moved
- `app/` → `src/slidefactory/app/`
- `cli/` → `src/slidefactory/cli/`
- `alembic/` → `src/slidefactory/alembic/`
- `tests/` → `tests/` (copied)
- `static/css/`, `static/js/`, `static/vendor/`, `static/fonts/` → `static/`
- `templates/` → `templates/` (reviewed for generic content)
- `packages/n8n-nodes-slidefactory/` → `packages/`
## Files Moved
- `requirements.txt`
- `alembic.ini`
- `init.sql` (if exists)
- `pytest.ini` (if exists)
## Not Moved (S5-Specific)
- `.github/workflows/` - Azure deployment workflows (S5-specific)
- `static/img/sportfive*` - S5 branding
- `docker-compose.yml` - S5 configuration
- `.env` - S5 environment
- Deployment scripts
## Next Steps
- Create s5_branding/ directory in s5-slidefactory
- Update s5-slidefactory to depend on slidefactory-core package
EOF
Summary of Commands (Quick Reference)¶
# Phase 1: Prepare structure
cd ../slidefactory-core
mkdir -p src/slidefactory static templates packages tests .github/workflows docs
# Phase 2: Copy core files
cp -r ../s5-slidefactory/app src/slidefactory/
cp -r ../s5-slidefactory/cli src/slidefactory/
cp -r ../s5-slidefactory/alembic src/slidefactory/
cp ../s5-slidefactory/alembic.ini src/slidefactory/
cp -r ../s5-slidefactory/static/{css,js,vendor,fonts} static/
cp -r ../s5-slidefactory/templates/* templates/
cp -r ../s5-slidefactory/packages/n8n-nodes-slidefactory packages/
cp -r ../s5-slidefactory/tests .
cp ../s5-slidefactory/requirements.txt .
# Phase 3: Fix imports
find src/slidefactory -type f -name "*.py" -exec sed -i 's/from app\./from slidefactory.app./g' {} +
find src/slidefactory -type f -name "*.py" -exec sed -i 's/from cli\./from slidefactory.cli./g' {} +
# Phase 4-8: Create configuration files (see detailed steps above)
# Phase 9: Test
python3 -m venv venv
source venv/bin/activate
pip install -e ".[dev]"
pytest tests/ --collect-only
# Phase 10: Commit
git add .
git commit -m "feat: initial slidefactory-core package"
git push origin main
Risk Mitigation¶
-
Backup First: Before starting, create a full backup of s5-slidefactory
-
Test in Stages: Don't copy everything at once. Test after each major phase.
-
Keep Original: Don't delete anything from s5-slidefactory until core is working.
-
Git Branches: Consider working on a branch in slidefactory-core first:
Expected Issues and Solutions¶
Issue 1: Import Errors After sed Replacements¶
Solution: Some imports might be double-replaced or incorrectly replaced. Review manually:
Issue 2: Template Paths Not Found¶
Solution: Check that templates are in the right location relative to where Jinja2 looks:
Issue 3: Alembic Can't Find Models¶
Solution: Ensure alembic/env.py imports all models and uses correct paths.
Issue 4: CLI Not Found After Installation¶
Solution: Verify [project.scripts] in pyproject.toml and reinstall:
Validation Checklist¶
- Package structure created (
src/slidefactory/) - All core code copied
- Import paths updated (
app.→slidefactory.app.) - pyproject.toml created with all dependencies
- Generic static assets copied (no S5 branding)
- Templates copied and reviewed
- Tests copied
- .gitignore created
- README.md created
- CHANGELOG.md created
- Dockerfile.base created
- GitHub Actions workflows created
- Package installs successfully (
pip install -e .) - CLI command works (
slidefactory --help) - Imports work (Python can import slidefactory modules)
- Tests can be collected (
pytest --collect-only) - Initial commit created
- Changes pushed to remote
Next Steps After Core Migration¶
- Test the core package thoroughly
- Create v1.0.0 release tag
- Publish to GitHub Packages (or test locally)
- Begin Phase 2: Migrate s5-slidefactory to client repository structure
- Update s5-slidefactory to depend on slidefactory-core package
Estimated Effort: 4-6 hours for careful execution with testing Complexity: Medium - requires attention to import paths and file organization