6. MLOps and A/B Testing

Goal

Deploy models safely, monitor runtime behavior, and compare variants with statistical discipline.

Create a deployment

  1. Open MLOps > Deployments.

  2. Select a model version from registry.

  3. Configure deployment settings: - Environment. - Replica/compute profile. - Rollback strategy.

  4. Deploy and wait until status is active.

Monitor deployment

  1. Validate health indicators: - Uptime/health state. - Error rate. - Latency percentile.

  2. Generate test inference call and confirm response.

  3. Review recent logs for runtime exceptions.

Run A/B test

  1. Open MLOps > A/B Tests.

  2. Create a test with: - Baseline model (A). - Candidate model (B). - Traffic split. - Primary success metric. - Minimum sample size/duration.

  3. Start test and monitor allocation.

  4. Evaluate winner decision when threshold is reached.

Functional validation checklist

  1. Active deployment serves predictions without downtime.

  2. Health and metrics update in near real-time.

  3. A/B traffic split is respected by observed request counts.

  4. Reported winner is supported by configured metric.

  5. Rollback can be executed if candidate degrades performance.

Expected result

  1. Production path is stable and observable.

  2. Model promotion decisions are data-driven.

Common errors and recovery

  1. Deployment stuck in pending: - Check environment capacity and model artifact availability.

  2. High error rate after release: - Trigger rollback to previous stable model.

  3. Inconclusive A/B test: - Increase duration/sample size before decision.

Screenshots

Deployment operations and monitoring

MLOps deployment list with runtime status.

AB test setup and tracking

A/B testing configuration and live comparison view.