Prove
Measure
Incremental lift, not clicks. A promotion runs in a week; comparing that week to the last one attributes the weather, the school holiday and the festival two kilometres away to the promotion.
DXB-MAR against its synthetic control
The control is a weighted blend of the other sites, with weights chosen to reproduce this site’s pre-period. Weights are non-negative and sum to one, so the control is always a convex blend of things that actually happened rather than an extrapolation that can go anywhere.
The estimator’s own bias, measured
The estimator returns about -3.5% on a window where nothing happened, and that is not noise. The donor blend is dominated by Al Barsha, which is fully enclosed; Marina Walk is more than half outdoor seating. As the Gulf summer arrives Marina falls away from its own history while the enclosed donor does not, so the counterfactual drifts above the treated site for reasons unrelated to any promotion. With four sites and one of them uniquely weather-elastic, no convex blend of the others can match Marina's weather response: it is inside the donors' hull on level and outside it on elasticity. That is a real limitation of synthetic control on a small donor pool, so it is measured on a placebo window and subtracted, and both numbers are reported.
Does it find a lift it was handed?
The check that makes every number above worth reading. A known lift is injected into the series and the same machinery is asked to find it. An estimator that cannot recover a lift it was given is not measuring anything.
| injected | raw | bias | adjusted | error | result |
|---|---|---|---|---|---|
| +30.0% | +23.9% | -3.5% | +27.4% | -2.61 pts | within 5 |
| +20.0% | +14.4% | -3.5% | +17.9% | -2.14 pts | within 5 |
| +10.0% | +4.9% | -3.5% | +8.3% | -1.67 pts | within 5 |
| +5.0% | +0.1% | -3.5% | +3.6% | -1.44 pts | within 5 |
| +0.0% | -4.7% | -3.5% | -1.2% | -1.20 pts | within 5 |
Totals over the window
Synthetic control with non-negative weights summing to one, fitted on 56 pre-period days; CUPED variance reduction; a permutation placebo across donor sites for the p-value; and a placebo-in-time run for the estimator's own bias.