=========================
6. MLOps and A/B Testing
=========================

Goal
====

Deploy models safely, monitor runtime behavior, and compare variants with statistical discipline.

Create a deployment
===================

1. Open ``MLOps > Deployments``.
2. Select a model version from registry.
3. Configure deployment settings:
   - Environment.
   - Replica/compute profile.
   - Rollback strategy.
4. Deploy and wait until status is ``active``.

Monitor deployment
==================

1. Validate health indicators:
   - Uptime/health state.
   - Error rate.
   - Latency percentile.
2. Generate test inference call and confirm response.
3. Review recent logs for runtime exceptions.

Run A/B test
============

1. Open ``MLOps > A/B Tests``.
2. Create a test with:
   - Baseline model (A).
   - Candidate model (B).
   - Traffic split.
   - Primary success metric.
   - Minimum sample size/duration.
3. Start test and monitor allocation.
4. Evaluate winner decision when threshold is reached.

Functional validation checklist
===============================

1. Active deployment serves predictions without downtime.
2. Health and metrics update in near real-time.
3. A/B traffic split is respected by observed request counts.
4. Reported winner is supported by configured metric.
5. Rollback can be executed if candidate degrades performance.

Expected result
===============

1. Production path is stable and observable.
2. Model promotion decisions are data-driven.

Common errors and recovery
==========================

1. Deployment stuck in ``pending``:
   - Check environment capacity and model artifact availability.
2. High error rate after release:
   - Trigger rollback to previous stable model.
3. Inconclusive A/B test:
   - Increase duration/sample size before decision.

Screenshots
===========

.. figure:: ../../_static/user-manual/common/mlops.png
   :alt: Deployment operations and monitoring
   :width: 90%

   MLOps deployment list with runtime status.

.. figure:: ../../_static/user-manual/common/ab-tests.png
   :alt: AB test setup and tracking
   :width: 90%

   A/B testing configuration and live comparison view.