Welch's t-test vs Student's t-test: Why Welch Should Be Your Default

Welch's t-test vs Student's t-test explained: what happens with unequal variances and sample sizes, why testing for equal variances first is a mistake, and what to report.

Two groups, one continuous outcome, one question: do the means differ? The classic Student's t-test answers it — but it carries an assumption almost nobody's data satisfies: that both groups have the same variance. Welch's t-test drops that assumption at essentially no cost. Here's the case for making Welch your default, and the evidence for what goes wrong when you don't.

The problem with assuming equal variances

Student's t-test pools both groups' variability into one estimate. When the groups genuinely have equal spread, that's efficient. When they don't — and treated groups, patient subgroups, and real-world segments routinely differ in spread as well as level — the pooled estimate is wrong in a way that depends on your sample sizes.

The dangerous combination is unequal variances with unequal group sizes. If the smaller group has the larger variance, Student's t-test becomes anticonservative: it rejects true null hypotheses far more often than the 5% it promises. Simulation makes it concrete:

False positive rates: Student vs Welch

Four thousand simulated studies with no true difference (n = 12 with SD 4, vs n = 60 with SD 1): Student's t-test cries "significant" in nearly one study in three — a six-fold inflation of false positives — while Welch's holds the advertised 5.0%. Flip the setup (big group, big variance) and Student's becomes overly conservative instead, silently costing you power. Either way, it's answering with the wrong error rates.

Why not test for equal variances first?

The traditional recipe — run Levene's test, then choose Student's or Welch's — sounds rigorous but fails on three counts. It's underpowered exactly when it matters (small samples, where unequal variances do the most damage, are where Levene's is least able to detect them). It distorts your error rates (the two-stage procedure conditions your main test on a preliminary test, invalidating the nominal α). And it's pointless: when variances happen to be equal, Welch's gives virtually identical results to Student's anyway. The methodological literature has been blunt about this for two decades: skip the gatekeeper test and just use Welch. Levene's test still has a legitimate role — when the variance difference is itself the research question — just not as an admission ticket for a t-test.

What Welch actually changes

Welch's test keeps each group's variance separate and adjusts the degrees of freedom (the Welch–Satterthwaite approximation) to account for the imbalance — you'll notice fractional df like 56.5 in your output; that's the adjustment working, not an error. Everything you'd report stays familiar: t, df, p, plus the things that matter more — the mean difference with its 95% CI and a standardized effect size (Hedges' g, the small-sample-corrected cousin of Cohen's d). Same logic extends to 3+ groups: Welch's ANOVA with Games-Howell post-hocs is to one-way ANOVA what Welch's t is to Student's.

What to report

A defensible write-up needs four numbers, not one: the group means, the difference with its 95% CI, the effect size, and the p-value — in roughly that order of importance. "Mean systolic BP was 8.2 mmHg lower in the treatment group (95% CI 3.1–13.3; Hedges' g = 0.62; Welch's t(56.5) = 3.2, p = .002)" tells your reader the size, the precision, and the strength of evidence. A bare "p < .05" tells them almost nothing.

The default is already right in Inference

Inference uses Welch's t-test as the default two-group comparison — no variance-test gatekeeping, no checkbox to remember. Every result card shows the mean difference with CI and Hedges' g inline, and the built-in Statistical Reviewer checks your power, flags interpretation traps, and drafts the APA-style sentence above for you. If your data is skewed rather than merely unequal-variance, the Guided flow will steer you to Mann-Whitney instead — and tell you why.

Compare your groups the defensible way — free →

Related reading

References

The methods on this page are not our inventions — these are the primary sources. Where a claim here is contested in the literature, the reference is the place to check it rather than take our word for it.

  • Welch, B. L. (1947). The generalization of "Student's" problem when several different population variances are involved. Biometrika, 34.
  • Delacre, M., Lakens, D., & Leys, C. (2017). Why psychologists should by default use Welch's t-test instead of Student's t-test. International Review of Social Psychology, 30.
  • Zimmerman, D. W. (2004). A note on preliminary tests of equality of variances. British Journal of Mathematical and Statistical Psychology, 57.
  • Ruxton, G. D. (2006). The unequal variance t-test is an underused alternative to Student's t-test and the Mann–Whitney U test. Behavioral Ecology, 17.

FAQ

Is Welch's test ever worse than Student's? With exactly equal variances and equal n, Student's has a sliver more power — a difference too small to measure in practice. The insurance is essentially free.

Do fractional degrees of freedom mean something's wrong? No — df like 47.3 is the Welch–Satterthwaite adjustment doing its job.

Should I use Welch for paired data? No — paired designs have their own test (the paired t-test on within-pair differences); the equal-variance question doesn't arise.

What if my data is skewed too? Unequal variance is Welch territory; heavy skew or ordinal outcomes point to Mann-Whitney. Skew plus unequal spread in small samples: Mann-Whitney, interpreted as a shift in distributions.

Written by Dr Hoong Sern Lim MB ChB MD FRCP, Consultant Cardiologist, Queen Elizabeth Hospital Birmingham; Honorary Senior Lecturer, University of Birmingham. ORCID 0000-0002-6569-1805

All guides · About the author · How we validate