Experiments

You made a fix: a new pricing page, an FAQ, a listing on a review site. Did it move anything? An experiment answers that properly, rather than by eyeballing a chart.

Set one up

Describe what changed, such as “Added a pricing FAQ block”.
Give the page address and the date you implemented it.
Choose the affected queries: the questions the change should help.
Choose some holdout queries: similar questions you did not touch.
Set how many days to measure either side.

Reading the result

You get before-and-after figures for both groups, and a net lift: how much the changed questions moved beyond whatever the untouched ones did anyway. The verdict is either Significant or Within noise, the same confidence test used everywhere else. An experiment is only called a win when the before and after ranges are clear of each other.

Why the holdout mattersWithout it you cannot tell your work from the weather. If both groups rose equally, the model changed its behavior and your fix did nothing. Holdout questions are optional, but they are what makes the answer trustworthy.

Next: Growth Playbook →

Updated 23 September 2026. Something unclear? Write to [email protected]. A person reads it.