parallel tests
This commit is contained in:
parent
73a45b81cc
commit
139a5fbef3
GPG key ID:
883EDBAA726BD96C
5 changed files with 61 additions and 228 deletions
|
|
@ -1,206 +0,0 @@
|
|||
# Backend Test Optimization Plan
|
||||
|
||||
## Overview
|
||||
This plan implements three optimizations to speed up backend test execution:
|
||||
1. **Session-scoped role setup** (#4)
|
||||
2. **Session-scoped schema + transaction rollback** (#1)
|
||||
3. **Parallel test execution** (#2)
|
||||
|
||||
Current baseline: 236 tests in ~110 seconds (~0.46s per test)
|
||||
|
||||
## Implementation Steps
|
||||
|
||||
### Step 1: Session-Scoped Role Setup (#4)
|
||||
|
||||
**Goal**: Create roles once per test session instead of 236 times.
|
||||
|
||||
**Changes**:
|
||||
- Create `@pytest.fixture(scope="session")` for engine
|
||||
- Create `@pytest.fixture(scope="session")` for roles setup
|
||||
- Modify `client_factory` to use pre-created roles instead of calling `setup_roles()` each time
|
||||
|
||||
**Benefits**:
|
||||
- Eliminates 236 role creation operations
|
||||
- Roles are static data, safe to share across tests
|
||||
|
||||
**Risks**: Low - roles are read-only after creation
|
||||
|
||||
---
|
||||
|
||||
### Step 2: Session-Scoped Schema Creation (#1)
|
||||
|
||||
**Goal**: Create database schema once per session instead of dropping/recreating 236 times.
|
||||
|
||||
**Changes**:
|
||||
- Move schema creation (`drop_all` + `create_all`) to session-scoped `engine` fixture
|
||||
- Schema created once at session start, cleaned up at session end
|
||||
- Each test still gets a fresh database state via transaction rollback
|
||||
|
||||
**Benefits**:
|
||||
- Eliminates 236 schema drop/create operations (major bottleneck)
|
||||
- Expected 40-60% speed improvement
|
||||
|
||||
**Risks**: Medium - need to ensure proper cleanup and isolation
|
||||
|
||||
---
|
||||
|
||||
### Step 3: Transaction Rollback Pattern (#1)
|
||||
|
||||
**Goal**: Use database transactions to isolate tests instead of dropping tables.
|
||||
|
||||
**Approach**:
|
||||
- Each test runs inside a transaction
|
||||
- After test completes, rollback the transaction (not commit)
|
||||
- Next test starts with clean state automatically
|
||||
|
||||
**Implementation Strategy**:
|
||||
1. Create a session-scoped connection pool
|
||||
2. For each test:
|
||||
- Start a transaction (or use a savepoint)
|
||||
- Run test with all DB operations in this transaction
|
||||
- Rollback transaction after test
|
||||
3. Override `get_db()` to yield sessions within the transaction context
|
||||
|
||||
**Key Challenge**: FastAPI's `get_db` dependency needs to work with transaction boundaries.
|
||||
|
||||
**Solution Options**:
|
||||
- **Option A**: Use nested transactions (savepoints) - more complex but better isolation
|
||||
- **Option B**: Use connection-level transactions - simpler, rollback entire connection state
|
||||
|
||||
**Recommended**: Option B (simpler, sufficient for test isolation)
|
||||
|
||||
**Changes**:
|
||||
- Modify `client_factory` to use transaction-scoped sessions
|
||||
- Update `get_db_session()` to work within transaction context
|
||||
- Ensure all test DB operations happen within transaction
|
||||
|
||||
**Benefits**:
|
||||
- Fast test isolation (rollback is much faster than drop/create)
|
||||
- Maintains test independence
|
||||
|
||||
**Risks**: Medium - need to ensure:
|
||||
- No commits happen during tests (or they're rolled back)
|
||||
- Transaction boundaries are properly managed
|
||||
- Async context managers work correctly
|
||||
|
||||
---
|
||||
|
||||
### Step 4: Update Fixtures for New Architecture
|
||||
|
||||
**Changes**:
|
||||
- Update `client_factory` to depend on session-scoped `engine` and `roles`
|
||||
- Update `get_db_session()` to work with transaction rollback
|
||||
- Ensure user fixtures (`regular_user`, `admin_user`, etc.) work with new pattern
|
||||
- Update `override_get_db()` to yield sessions within transaction context
|
||||
|
||||
**Testing**: Run a subset of tests to verify fixtures work correctly
|
||||
|
||||
---
|
||||
|
||||
### Step 5: Add pytest-xdist for Parallel Execution (#2)
|
||||
|
||||
**Goal**: Run tests in parallel across CPU cores.
|
||||
|
||||
**Changes**:
|
||||
1. Add `pytest-xdist` to `pyproject.toml` dev dependencies
|
||||
2. Update `Makefile` to use `pytest -n auto` for parallel execution
|
||||
3. Ensure test isolation is maintained (transaction rollback ensures this)
|
||||
|
||||
**Configuration**:
|
||||
- Use `-n auto` to auto-detect CPU cores
|
||||
- Can override with `-n 4` for specific core count
|
||||
- Add `pytest-xdist` to dependency groups
|
||||
|
||||
**Benefits**:
|
||||
- 2-4x speed improvement (depending on CPU cores)
|
||||
- Works well with transaction isolation
|
||||
|
||||
**Risks**: Low - transaction rollback ensures tests don't interfere
|
||||
|
||||
**Note**: May need to adjust if tests have shared state (but transaction rollback should prevent this)
|
||||
|
||||
---
|
||||
|
||||
### Step 6: Testing and Validation
|
||||
|
||||
**Verification Steps**:
|
||||
1. Run full test suite: `make test-backend`
|
||||
2. Verify all 236 tests pass
|
||||
3. Measure execution time improvement
|
||||
4. Check for any flaky tests (shouldn't happen with proper isolation)
|
||||
5. Test parallel execution with `pytest -n auto`
|
||||
|
||||
**Success Criteria**:
|
||||
- All tests pass
|
||||
- Significant speed improvement (target: 50-70% faster)
|
||||
- No test flakiness introduced
|
||||
- Parallel execution works correctly
|
||||
|
||||
---
|
||||
|
||||
## Implementation Order
|
||||
|
||||
1. ✅ **Step 1**: Session-scoped role setup (easiest, low risk)
|
||||
2. ✅ **Step 2**: Session-scoped schema creation (foundation for #3)
|
||||
3. ✅ **Step 3**: Transaction rollback pattern (core optimization)
|
||||
4. ✅ **Step 4**: Update all fixtures (required for #3 to work)
|
||||
5. ✅ **Step 5**: Add pytest-xdist (quick win, independent)
|
||||
6. ✅ **Step 6**: Test and validate
|
||||
|
||||
---
|
||||
|
||||
## Technical Details
|
||||
|
||||
### Transaction Rollback Pattern
|
||||
|
||||
```python
|
||||
# Pseudo-code for transaction pattern
|
||||
@pytest.fixture(scope="function")
|
||||
async def db_transaction(engine):
|
||||
async with engine.connect() as conn:
|
||||
trans = await conn.begin()
|
||||
try:
|
||||
# Create session factory that uses this connection
|
||||
session_factory = async_sessionmaker(bind=conn, ...)
|
||||
yield session_factory
|
||||
finally:
|
||||
await trans.rollback() # Always rollback, never commit
|
||||
```
|
||||
|
||||
### Session-Scoped Engine
|
||||
|
||||
```python
|
||||
@pytest.fixture(scope="session")
|
||||
async def engine():
|
||||
engine = create_async_engine(TEST_DATABASE_URL)
|
||||
# Create schema once
|
||||
async with engine.begin() as conn:
|
||||
await conn.run_sync(Base.metadata.drop_all)
|
||||
await conn.run_sync(Base.metadata.create_all)
|
||||
yield engine
|
||||
await engine.dispose()
|
||||
```
|
||||
|
||||
### Role Setup
|
||||
|
||||
```python
|
||||
@pytest.fixture(scope="session")
|
||||
async def roles(engine):
|
||||
session_factory = async_sessionmaker(engine)
|
||||
async with session_factory() as db:
|
||||
roles = await setup_roles(db)
|
||||
await db.commit() # Commit roles once
|
||||
return roles
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Rollback Plan
|
||||
|
||||
If issues arise:
|
||||
1. Revert `conftest.py` changes
|
||||
2. Remove `pytest-xdist` dependency
|
||||
3. Restore original fixture structure
|
||||
|
||||
All changes are isolated to test files, no production code affected.
|
||||
|
||||
Loading…
Add table
Add a link
Reference in a new issue