TOST Equivalence Test calculator

Prove two group means are practically equivalent. Perform a TOST equivalence test online and plot the equivalence bounds.

What it does

Tests whether two groups are practically equivalent, by running two one-sided tests against the bounds of an equivalence margin you set.

When to use it

You want to claim "no meaningful difference" — a non-significant t-test never supports that claim, it only fails to find a difference.

Cautions

• The equivalence margin is a judgement about what difference would matter, and it must be justified before you look at the data. Choosing it afterwards guarantees the answer you want.
• Equivalence and difference are separate questions: a result can be both statistically significant and practically equivalent.
• Underpowered equivalence tests fail to demonstrate equivalence for the same reason underpowered t-tests fail to find differences.

Alternatives

• Welch's t-test — the question is whether a difference exists, not whether it is negligible

How to read the output

The equivalence margin (±)

The largest difference you are willing to call unimportant. It is a judgement about the subject matter, not a statistic, and everything below depends on it. Set it before running the test. A margin chosen after seeing the confidence interval proves nothing — and the widest margin anyone could defend is the one to justify, in writing, in the methods.

The two one-sided p-values and TOST p

One test against the lower edge, one against the upper. Equivalence requires BOTH to be significant, so the TOST p is the larger of the two. The larger one is the whole test. A tiny p against one edge means nothing on its own.

Mean difference and 90% CI

Ninety, not ninety-five, on purpose: a 90% interval falling entirely inside the margin is exactly equivalent to both one-sided tests passing at 5%. Do not compare this interval against zero as if it were a 95% CI. It is the right interval for the equivalence question and the wrong one for the difference question.

The four-way interpretation

Equivalence and difference are separate questions with four possible answers: equivalent, different, both (a real difference that is trivially small), or neither (the data cannot tell — usually too small a sample). 'Neither' is the commonest outcome and is not a null result to be written up as equivalence. It means the study could not answer either question.

How this calculator is validated · Which statistical test should I use?