Safeguards

Write the rules that keep the number honest and keep the test from doing damage: who may touch the metric, what would let someone hit the target without the real win happening, and which side effects must not get worse.

Write the rules that keep the number honest and keep the test from doing damage: who may touch the metric, what would let someone hit the target without the real win happening, and which side effects must not get worse.

Example

In the onboarding pilot: no promo emails or discounts during the four months, since either lifts signups without the new flow doing the work. And 30-day retention and support ticket volume must not drop more than 2 percent, or the pilot stops early.