arbret/backend/tests/OPTIMIZATION_PLAN.md

Backend Test Optimization Plan

Overview

This plan implements three optimizations to speed up backend test execution:

1. Session-scoped role setup (#4)
2. Session-scoped schema + transaction rollback (#1)
3. Parallel test execution (#2)

Current baseline: 236 tests in ~110 seconds (~0.46s per test)

Implementation Steps

Step 1: Session-Scoped Role Setup (#4)

Goal: Create roles once per test session instead of 236 times.

Changes:

• Create @pytest.fixture(scope="session") for engine
• Create @pytest.fixture(scope="session") for roles setup
• Modify client_factory to use pre-created roles instead of calling setup_roles() each time

Benefits:

• Eliminates 236 role creation operations
• Roles are static data, safe to share across tests

Risks: Low - roles are read-only after creation

---

Step 2: Session-Scoped Schema Creation (#1)

Goal: Create database schema once per session instead of dropping/recreating 236 times.

Changes:

• Move schema creation (drop_all + create_all) to session-scoped engine fixture
• Schema created once at session start, cleaned up at session end
• Each test still gets a fresh database state via transaction rollback

Benefits:

• Eliminates 236 schema drop/create operations (major bottleneck)
• Expected 40-60% speed improvement

Risks: Medium - need to ensure proper cleanup and isolation

---

Step 3: Transaction Rollback Pattern (#1)

Goal: Use database transactions to isolate tests instead of dropping tables.

Approach:

• Each test runs inside a transaction
• After test completes, rollback the transaction (not commit)
• Next test starts with clean state automatically

Implementation Strategy:

1. Create a session-scoped connection pool
2. For each test:
   • Start a transaction (or use a savepoint)
   • Run test with all DB operations in this transaction
   • Rollback transaction after test
3. Override get_db() to yield sessions within the transaction context

Key Challenge: FastAPI's get_db dependency needs to work with transaction boundaries.

Solution Options:

• Option A: Use nested transactions (savepoints) - more complex but better isolation
• Option B: Use connection-level transactions - simpler, rollback entire connection state

Recommended: Option B (simpler, sufficient for test isolation)

Changes:

• Modify client_factory to use transaction-scoped sessions
• Update get_db_session() to work within transaction context
• Ensure all test DB operations happen within transaction

Benefits:

• Fast test isolation (rollback is much faster than drop/create)
• Maintains test independence

Risks: Medium - need to ensure:

• No commits happen during tests (or they're rolled back)
• Transaction boundaries are properly managed
• Async context managers work correctly

---

Step 4: Update Fixtures for New Architecture

Changes:

• Update client_factory to depend on session-scoped engine and roles
• Update get_db_session() to work with transaction rollback
• Ensure user fixtures (regular_user, admin_user, etc.) work with new pattern
• Update override_get_db() to yield sessions within transaction context

Testing: Run a subset of tests to verify fixtures work correctly

---

Step 5: Add pytest-xdist for Parallel Execution (#2)

Goal: Run tests in parallel across CPU cores.

Changes:

1. Add pytest-xdist to pyproject.toml dev dependencies
2. Update Makefile to use pytest -n auto for parallel execution
3. Ensure test isolation is maintained (transaction rollback ensures this)

Configuration:

• Use -n auto to auto-detect CPU cores
• Can override with -n 4 for specific core count
• Add pytest-xdist to dependency groups

Benefits:

• 2-4x speed improvement (depending on CPU cores)
• Works well with transaction isolation

Risks: Low - transaction rollback ensures tests don't interfere

Note: May need to adjust if tests have shared state (but transaction rollback should prevent this)

---

Step 6: Testing and Validation

Verification Steps:

1. Run full test suite: make test-backend
2. Verify all 236 tests pass
3. Measure execution time improvement
4. Check for any flaky tests (shouldn't happen with proper isolation)
5. Test parallel execution with pytest -n auto

Success Criteria:

• All tests pass
• Significant speed improvement (target: 50-70% faster)
• No test flakiness introduced
• Parallel execution works correctly

---

Implementation Order

1. ✅ Step 1: Session-scoped role setup (easiest, low risk)
2. ✅ Step 2: Session-scoped schema creation (foundation for #3)
3. ✅ Step 3: Transaction rollback pattern (core optimization)
4. ✅ Step 4: Update all fixtures (required for #3 to work)
5. ✅ Step 5: Add pytest-xdist (quick win, independent)
6. ✅ Step 6: Test and validate

---

Technical Details

Transaction Rollback Pattern

# Pseudo-code for transaction pattern
@pytest.fixture(scope="function")
async def db_transaction(engine):
    async with engine.connect() as conn:
        trans = await conn.begin()
        try:
            # Create session factory that uses this connection
            session_factory = async_sessionmaker(bind=conn, ...)
            yield session_factory
        finally:
            await trans.rollback()  # Always rollback, never commit

Session-Scoped Engine

@pytest.fixture(scope="session")
async def engine():
    engine = create_async_engine(TEST_DATABASE_URL)
    # Create schema once
    async with engine.begin() as conn:
        await conn.run_sync(Base.metadata.drop_all)
        await conn.run_sync(Base.metadata.create_all)
    yield engine
    await engine.dispose()

Role Setup

@pytest.fixture(scope="session")
async def roles(engine):
    session_factory = async_sessionmaker(engine)
    async with session_factory() as db:
        roles = await setup_roles(db)
        await db.commit()  # Commit roles once
    return roles

---

Rollback Plan

If issues arise:

1. Revert conftest.py changes
2. Remove pytest-xdist dependency
3. Restore original fixture structure

All changes are isolated to test files, no production code affected.