Sample ratio mismatch: don't trust this test yet
With the split you planned, groups this uneven would happen by chance less than 1 time in 1,000 (p < 0.0001). Something is dropping or double-counting users in one group.
| Group | Observed | Expected | Difference |
|---|---|---|---|
| Control | 50,31250.67% of users | 49,645 | +1.34% |
| New paywall | 48,97749.33% of users | 49,645 | -1.34% |
Chi-square goodness of fit: χ² = 17.95, 1 degree of freedom. Flagged when p < 0.001. How this works