AI Agent E2E Testing Protocol

This document defines the standard protocol for AI agents to perform end-to-end testing of Mailpilot using headless browser automation.

Overview

AI agents use the Chrome MCP tools to perform interactive E2E testing during development. This is separate from automated Playwright tests - it's for manual verification of features during implementation.

Test Environment Setup

Step 1: Seed Test Data

Before testing, seed the database with test fixtures:

# Seed all data with defaults
pnpm seed:test

# Or seed specific data with custom counts
pnpm seed:test --audit 50 --dead-letter 10

# Show all options
pnpm seed:test --help

See E2E Testing Reference for all seeding options.

Step 2: Start the Application

Start the development server:

pnpm dev

This starts:

• Backend server on http://localhost:8085
• WebSocket server for real-time updates
• Serves the dashboard from dashboard/dist/

Wait for: Log message Dashboard server started on port 8085

Step 3: Initialize Browser Context

Get the current MCP tab context:

mcp__Claude_in_Chrome__tabs_context_mcp

Create a new tab for testing (each test session should use a fresh tab):

mcp__Claude_in_Chrome__tabs_create_mcp

Step 4: Navigate to Dashboard

mcp__Claude_in_Chrome__navigate with url: "http://localhost:8085"

Verify: Page loads without errors.

Test Session Protocol

Starting a Test Session

1. Document the test objective - State what you're testing
2. Take initial screenshot - Capture the starting state
3. Plan test steps - List the actions you'll take

During Testing

For each test step:

1. State the action - What you're about to do
2. Execute using MCP tools - Perform the action
3. Capture evidence - Take screenshot after significant actions
4. Verify result - Check the outcome matches expectations
5. Document findings - Note any issues or observations

Test Coverage Checklist

For any dashboard feature, ensure you test:

• Happy path - Feature works as expected
• Error handling - Invalid inputs show proper errors
• Loading states - Loading indicators appear during async ops
• Empty states - Proper UI when no data exists
• Responsive behavior - Works at different viewport sizes
• Keyboard navigation - Tab through form fields
• Real-time updates - WebSocket updates reflect in UI

Common Test Scenarios

Dashboard Load Test

1. Navigate to http://localhost:8085
2. Verify page title contains "Mailpilot"
3. Check navigation sidebar is visible
4. Verify stats widgets load (check for loading spinners → data)
5. Take screenshot of loaded dashboard

Settings Page Test

1. Navigate to /settings
2. Verify all setting sections are visible:
   • Email Accounts
   • LLM Configuration
   • Webhooks (if enabled)
3. Test expand/collapse of sections
4. Verify form validation on inputs

Account Management Test

1. Navigate to Settings → Email Accounts
2. Click "Add Account"
3. Fill in test IMAP details:
   • Host: imap.example.com
   • Port: 993
   • Username: test@example.com
   • Password: testpassword
4. Click "Test Connection" (expect failure for fake credentials)
5. Verify error message is displayed
6. Cancel the modal
7. Verify modal closes cleanly

Real-time Updates Test

1. Open dashboard
2. Open Network tab / monitor WebSocket
3. Verify WebSocket connection established
4. Trigger an action (if possible) or wait for periodic update
5. Verify UI updates without page refresh

Error Handling

If a step fails:

1. Take screenshot immediately
2. Check console for errors: read_console_messages
3. Check network for failed requests: read_network_requests
4. Document the failure in the test report
5. Attempt recovery or abort test session

Common Issues

Integration with Development Workflow

When to Perform E2E Testing

1. After implementing UI changes - Verify the change works
2. Before creating PR - Full regression test of affected areas
3. When fixing bugs - Verify the fix and check for regressions
4. After merging dependencies - Smoke test the dashboard

Minimum E2E Verification

Before any PR that touches dashboard code:

1. Dashboard loads without console errors
2. Navigation works (all pages accessible)
3. Changed feature works as expected
4. No visual regressions in affected areas

Next Steps

• Automated E2E Testing →
• Dashboard Features →
• Contributing Guide →