One-sample t, two-sample t, proportion test, one-way ANOVA.
Ref: Cohen (1988) · Lenth (2001) · Abramowitz & Stegun
| Parameter | Value | Unit |
|---|
| Target Power | Required n | Achieved Power |
|---|
Sample size determination answers the question: "How many observations do I need to detect a meaningful difference?" Too few samples and you miss real effects (Type II error). Too many and you waste time, money, and material. Power analysis balances these risks by computing the minimum n that achieves your desired detection probability.
- Effect size — The smallest difference you care about detecting. For t-tests, this is Cohen's d (mean difference divided by standard deviation). For ANOVA, it's Cohen's f. Smaller effects require larger samples.
- Alpha (α) — The probability of a false positive (Type I error). Typically 0.05 (5%). Lower alpha (0.01) means you need more samples but are more confident when you reject the null hypothesis.
- Power (1 − β) — The probability of detecting a real effect. 0.80 (80%) is standard; 0.90 is recommended for critical decisions. Higher power requires larger samples.
- Test type — The statistical test you plan to use. Two-sample t-tests need more observations than one-sample tests. ANOVA sample sizes depend on the number of groups.
Jacob Cohen's conventions provide starting points when domain knowledge is unavailable:
- Cohen's d (t-tests): 0.2 = small, 0.5 = medium, 0.8 = large
- Cohen's f (ANOVA): 0.10 = small, 0.25 = medium, 0.40 = large
- Proportion difference: depends on baseline rate — a 5% improvement from 50% is easier to detect than from 95%
Whenever possible, use your domain knowledge or pilot data to estimate effect size rather than defaulting to conventions. A "medium" effect in psychology may be negligible in manufacturing.
- Running power analysis after the study — Post-hoc power is meaningless. Calculate sample size before collecting data.
- Ignoring practical significance — Statistical significance (p < 0.05) doesn't mean the effect matters. Choose effect sizes that represent meaningful real-world differences.
- Not accounting for attrition — If you expect dropouts, inflate your sample size by 10–20% beyond the calculated minimum.
- One-sided vs two-sided — Use two-sided tests by default. One-sided tests require strong prior justification.
Designed experiments have different sample size requirements. A 2^k factorial with k factors needs 2^k runs per replicate — replication determines power. For process capability context, see the Cpk calculator. For Bayesian alternatives that require no fixed sample size, see the Bayesian Cpk calculator.