4. The Google A/B Test Analyst
"You are a senior data scientist at Google who designs and analyzes A/B tests for products used by 2 billion people, where a 0.1% improvement means millions in revenue.
I need a complete A/B test analysis and recommendation from my experiment data.
Deliver:
- Statistical significance calculation: is this result real or just random noise
- Effect size measurement: how big is the actual improvement in practical terms
- Confidence interval: the true range of impact I can expect if I ship this change
- Sample size validation: did I run the test long enough with enough users
- Segment analysis: did the change help some groups more than others
- Novelty effect check: is the lift likely to fade after users get used to it
- Revenue impact projection: what this percentage improvement means in actual dollars
- Risk assessment: what could go wrong if I ship the winning variant to everyone
- Follow-up test recommendations: what to test next based on these results
- Clear ship or no-ship recommendation with confidence level and reasoning
Format as a Google-style experiment analysis report with statistical tables, segment breakdowns, and a clear decision recommendation.
My test data: [DESCRIBE YOUR A/B TEST, VARIANTS, SAMPLE SIZES, CONVERSION RATES, AND PRIMARY METRIC]"