Sign Test calculator

Perform an online sign test for paired data. Compares only the direction of difference (positive or negative) between pairs.

What it does

Counts how many paired differences go up versus down and tests that against chance. It uses only the direction of each change, nothing else.

When to use it

Paired data where you trust which way each pair moved but not the size — heavily skewed differences, or crude measurement.

Cautions

• Discarding magnitude costs a lot of power. Use it because magnitude is untrustworthy, not to avoid checking assumptions.
• Ties are dropped, which can remove a large share of the data.

Alternatives

• Wilcoxon signed-rank — the magnitudes are meaningful and the differences roughly symmetric — considerably more powerful
• McNemar's test — the paired outcome is binary rather than a change in a measured value

How to read the output

Positive, negative and zero differences

The three counts are the entire dataset as far as this test is concerned. Under the null, positives and negatives are a coin toss. The magnitudes are discarded. A subject who improved enormously and one who improved marginally count the same, which is the price of the test's near-total lack of assumptions.

N (non-zero)

Ties are dropped; the binomial test runs on what is left.

p-value

How surprising a difference at least this large would be IF the groups really were identical on this measure. A small p means the data would be surprising under that assumption. It is not the probability that the result is due to chance, and not the probability that the groups are the same — those read the conditional backwards. It also answers 'would this be surprising?', never 'is this big enough to matter?': with a large sample a trivial difference returns a small p, and the result looks identical to an important one.

Power

This is the least powerful of the paired tests, deliberately. A non-significant sign test on a modest sample is weak evidence of no effect. If the differences are usable, the Wilcoxon answers the same question with more of the data.

How this calculator is validated · Which statistical test should I use?