How We Validate | Inference Stats

Every statistical method in Inference Stats is checked to numerical precision against an independent reference implementation — SciPy, statsmodels, scikit-learn, lifelines, pingouin, or R — with reproducible generator scripts and 2,900+ automated tests on every change.

How it works

Every method below is run against an independent implementation on the same input, and the results must agree to numerical precision. The reference values are produced by generator scripts kept in the repository, so any claim on this page can be reproduced rather than taken on trust. 38 methods are covered across 8 areas, and the full suite of 2,900+ tests runs on every change — a result cannot ship if it disagrees with its reference.

Group comparisons, post-hoc & effect sizes

Method	Checked against	Generator script
Welch's / paired / one-sample t-test	SciPy — scipy.stats.ttest_ind / ttest_rel / ttest_1samp	comparison_ref.py
One-way ANOVA	SciPy — scipy.stats.f_oneway	comparison_ref.py
Welch's ANOVA	pingouin — welch_anova	comparison_ref.py
Kruskal–Wallis · Mann–Whitney U	SciPy — scipy.stats.kruskal / mannwhitneyu (exact)	comparison_ref.py
McNemar · binomial test	statsmodels · SciPy — mcnemar / binomtest	comparison_ref.py
Tukey HSD · Games–Howell · Dunn	SciPy tukey_hsd · pingouin · scikit-posthocs	posthoc_ref.py
p-value adjustment (Holm, BH, …)	statsmodels — multipletests	posthoc_ref.py
TOST equivalence (independent / paired)	statsmodels — ttost_ind / ttost_paired	tost_ref.py
Effect sizes (Cohen's d, Hedges g, η², rank-biserial, ε²)	Derived from the SciPy / pingouin statistic	effectsize_ref.py

ANOVA family

Method	Checked against	Generator script
Two-way / factorial ANOVA + ANCOVA	statsmodels — ols + anova_lm	anova2_ref.py
MANOVA (Wilks, Pillai, Hotelling–Lawley, Roy)	statsmodels — MANOVA.mv_test()	manova_ref.py
Repeated-measures ANOVA · Mauchly + GG/HF	pingouin (sphericity / RM-ANOVA)	rm_sphericity_ref.py

Regression

Method	Checked against	Generator script
OLS linear regression + VIF	statsmodels — OLS + variance_inflation_factor	regression_ref.py
Logistic regression	statsmodels — Logit	regression_ref.py
Poisson · negative binomial · ordinal	statsmodels — GLM(Poisson) / NegativeBinomial / OrderedModel	regression_ext_ref.py
Diagnostics (Breusch–Pagan, Durbin–Watson)	statsmodels — het_breuschpagan / durbin_watson	regression_diagnostics_ref.py
Ridge · Lasso	scikit-learn — Ridge / Lasso	regularized_ref.py
Linear mixed model (random intercept, REML)	statsmodels — MixedLM	mixed_ref.py
Multinomial logistic regression	R — VGAM::vglm(multinomial)	multinom_ref.R
Weighted logistic regression	R — glm(binomial, weights=)	weighted_logreg_ref.R
Firth penalized logistic (profile-likelihood CIs)	R — logistf (Heinze & Schemper)	firth_ref.R

Survival analysis

Method	Checked against	Generator script
Kaplan–Meier · RMST	lifelines — KaplanMeierFitter / restricted_mean_survival_time	survival_ref.py
Log-rank (2 & 3 groups, unstratified)	lifelines — logrank_test / multivariate_logrank_test	survival_ref.py
Cox proportional hazards (Efron) + stratified	lifelines — CoxPHFitter (strata=)	survival_ref.py
Competing risks (cumulative incidence)	lifelines — AalenJohansenFitter	survival_ref.py
Stratified log-rank	statsmodels survdiff (≡ R survival::survdiff)	survival_ref.py
Proportional-hazards test (Schoenfeld)	lifelines — proportional_hazard_test	ph_test_ref.py

Diagnostics (ROC / AUC)

Method	Checked against	Generator script
ROC curve + AUC	scikit-learn — roc_auc_score / roc_curve	diagnostics_ref.py
DeLong paired AUC comparison	R — pROC::roc.test(method="delong")	diagnostics_ref.R

Clustering & machine learning

Method	Checked against	Generator script
K-means · DBSCAN · Gaussian mixture · PCA	scikit-learn — KMeans / DBSCAN / GaussianMixture / PCA	clustering_ref.py
Silhouette score	scikit-learn — silhouette_score	silhouette_ref.py
Local Outlier Factor · Mahalanobis distance	scikit-learn LOF · NumPy + SciPy chi2	anomaly_ref.py
ML models & cross-validation	scikit-learn — linear_model / naive_bayes / KFold / StandardScaler	ml_ref.py

Multiple imputation (MICE)

Method	Checked against	Generator script
Rubin's rules pooling (Barnard–Rubin df)	R — mice::pool.scalar (exact)	rubin_ref.R
Data-augmentation prior (Firth-style)	R — mice:::augment (exact)	mice_augment_ref.R
PMM / logistic / Poisson / NB / polyreg imputation	R — mice + pool (distributional provenance check)	mice_*_ref.R

EDA & power

Method	Checked against	Generator script
Normality (Shapiro–Wilk) & EDA statistics	SciPy — scipy.stats (Royston AS R94)	eda_ref.py
Power / MDE / required-n (t, ANOVA, χ², r)	statsmodels.stats.power + SciPy non-central CDFs	power_ref.py

What this does not claim

Agreeing with SciPy or R means the arithmetic is right, not that the method is right for your data. Choosing an inappropriate test and computing it perfectly still gives a wrong answer, which is why the guidance in the app matters as much as the validation does.

Home · Guides · Open the app