★KECBKeep ’Em Coming Back
Books · Volume 6: Last Call
← Volume 6: Last Call
Volume 6: Last Call · Contents
Appendix D

The Experiment Planner

One change, one number, one fair comparison, and a decision.

Chapter 31 explains how to run a fair test in a real room. Copy this page for each change you try. Fill in the top half and date it before you start. Fill in the bottom half when the test ends. File the pages together so that a year from now you can see what you learned.

THE RULES IN BRIEF

1. One change at a time. Describe it so exactly that anyone could do it the same way.

2. Predict first. “If this works, [number] will move from [baseline] to at least [target] within [time].”

3. Compare fairly. Before and after is weakest, alternating nights is better, and a hold-out is better still when you have the volume.

4. Run long enough. At least four comparable nights on each side, and longer when counts are small. Don’t stop early because it looks good.

5. Read it without statistics. Is the result outside the normal swing? Did every changed night beat every unchanged night? Did it repeat? Show the counts beside every rate.

6. Decide and write it down. Keep, fix, drop, or label it unproven.

THE EXPERIMENT PLANNER

One page for one change. Fill in the top half before you start and date it. Fill in the bottom half when the test ends.

Fill in: Change being tested, Start date, End date.

  • 1. The change Describe it so exactly that anyone could do it the same way. Who does it, and what are the steps?
  • 2. The number Which one number should move? Write its definition, its baseline, and its normal swing. Name one guard number.
  • 3. The comparison Before and after, alternating nights, or a hold-out? Which nights are in each group, and how many?
  • 4. The prediction “If this works, ____ will move from ____ to at least ____ within ____.”
  • 5. What else changed List holidays, weather, events, and anything else that could explain a result.
  • 6. The result Write the counts and rates for each group. Did the result fall outside the swing? Did the groups overlap? (Result / Unproven / No effect)
  • 7. The decision Keep, fix, or drop? What will we do next, and who will tell the staff?

EXPERIMENT LOG

Columns to fill in: Dates, Change tested, Number, Baseline, Result (counts), Decision. Copy as many rows as you need.