# Backend Test Optimization Plan

## Overview
This plan implements three optimizations to speed up backend test execution:
1. **Session-scoped role setup** (#4)
2. **Session-scoped schema + transaction rollback** (#1)
3. **Parallel test execution** (#2)

Current baseline: 236 tests in ~110 seconds (~0.46s per test)

## Implementation Steps

### Step 1: Session-Scoped Role Setup (#4)

**Goal**: Create roles once per test session instead of 236 times.

**Changes**:
- Create `@pytest.fixture(scope="session")` for engine
- Create `@pytest.fixture(scope="session")` for roles setup
- Modify `client_factory` to use pre-created roles instead of calling `setup_roles()` each time

**Benefits**: 
- Eliminates 236 role creation operations
- Roles are static data, safe to share across tests

**Risks**: Low - roles are read-only after creation

---

### Step 2: Session-Scoped Schema Creation (#1)

**Goal**: Create database schema once per session instead of dropping/recreating 236 times.

**Changes**:
- Move schema creation (`drop_all` + `create_all`) to session-scoped `engine` fixture
- Schema created once at session start, cleaned up at session end
- Each test still gets a fresh database state via transaction rollback

**Benefits**:
- Eliminates 236 schema drop/create operations (major bottleneck)
- Expected 40-60% speed improvement

**Risks**: Medium - need to ensure proper cleanup and isolation

---

### Step 3: Transaction Rollback Pattern (#1)

**Goal**: Use database transactions to isolate tests instead of dropping tables.

**Approach**: 
- Each test runs inside a transaction
- After test completes, rollback the transaction (not commit)
- Next test starts with clean state automatically

**Implementation Strategy**:
1. Create a session-scoped connection pool
2. For each test:
   - Start a transaction (or use a savepoint)
   - Run test with all DB operations in this transaction
   - Rollback transaction after test
3. Override `get_db()` to yield sessions within the transaction context

**Key Challenge**: FastAPI's `get_db` dependency needs to work with transaction boundaries.

**Solution Options**:
- **Option A**: Use nested transactions (savepoints) - more complex but better isolation
- **Option B**: Use connection-level transactions - simpler, rollback entire connection state

**Recommended**: Option B (simpler, sufficient for test isolation)

**Changes**:
- Modify `client_factory` to use transaction-scoped sessions
- Update `get_db_session()` to work within transaction context
- Ensure all test DB operations happen within transaction

**Benefits**:
- Fast test isolation (rollback is much faster than drop/create)
- Maintains test independence

**Risks**: Medium - need to ensure:
- No commits happen during tests (or they're rolled back)
- Transaction boundaries are properly managed
- Async context managers work correctly

---

### Step 4: Update Fixtures for New Architecture

**Changes**:
- Update `client_factory` to depend on session-scoped `engine` and `roles`
- Update `get_db_session()` to work with transaction rollback
- Ensure user fixtures (`regular_user`, `admin_user`, etc.) work with new pattern
- Update `override_get_db()` to yield sessions within transaction context

**Testing**: Run a subset of tests to verify fixtures work correctly

---

### Step 5: Add pytest-xdist for Parallel Execution (#2)

**Goal**: Run tests in parallel across CPU cores.

**Changes**:
1. Add `pytest-xdist` to `pyproject.toml` dev dependencies
2. Update `Makefile` to use `pytest -n auto` for parallel execution
3. Ensure test isolation is maintained (transaction rollback ensures this)

**Configuration**:
- Use `-n auto` to auto-detect CPU cores
- Can override with `-n 4` for specific core count
- Add `pytest-xdist` to dependency groups

**Benefits**:
- 2-4x speed improvement (depending on CPU cores)
- Works well with transaction isolation

**Risks**: Low - transaction rollback ensures tests don't interfere

**Note**: May need to adjust if tests have shared state (but transaction rollback should prevent this)

---

### Step 6: Testing and Validation

**Verification Steps**:
1. Run full test suite: `make test-backend`
2. Verify all 236 tests pass
3. Measure execution time improvement
4. Check for any flaky tests (shouldn't happen with proper isolation)
5. Test parallel execution with `pytest -n auto`

**Success Criteria**:
- All tests pass
- Significant speed improvement (target: 50-70% faster)
- No test flakiness introduced
- Parallel execution works correctly

---

## Implementation Order

1. ✅ **Step 1**: Session-scoped role setup (easiest, low risk)
2. ✅ **Step 2**: Session-scoped schema creation (foundation for #3)
3. ✅ **Step 3**: Transaction rollback pattern (core optimization)
4. ✅ **Step 4**: Update all fixtures (required for #3 to work)
5. ✅ **Step 5**: Add pytest-xdist (quick win, independent)
6. ✅ **Step 6**: Test and validate

---

## Technical Details

### Transaction Rollback Pattern

```python
# Pseudo-code for transaction pattern
@pytest.fixture(scope="function")
async def db_transaction(engine):
    async with engine.connect() as conn:
        trans = await conn.begin()
        try:
            # Create session factory that uses this connection
            session_factory = async_sessionmaker(bind=conn, ...)
            yield session_factory
        finally:
            await trans.rollback()  # Always rollback, never commit
```

### Session-Scoped Engine

```python
@pytest.fixture(scope="session")
async def engine():
    engine = create_async_engine(TEST_DATABASE_URL)
    # Create schema once
    async with engine.begin() as conn:
        await conn.run_sync(Base.metadata.drop_all)
        await conn.run_sync(Base.metadata.create_all)
    yield engine
    await engine.dispose()
```

### Role Setup

```python
@pytest.fixture(scope="session")
async def roles(engine):
    session_factory = async_sessionmaker(engine)
    async with session_factory() as db:
        roles = await setup_roles(db)
        await db.commit()  # Commit roles once
    return roles
```

---

## Rollback Plan

If issues arise:
1. Revert `conftest.py` changes
2. Remove `pytest-xdist` dependency
3. Restore original fixture structure

All changes are isolated to test files, no production code affected.