The validation rig that killed my own strategy. Run it on yours.
A self-hosted Python package that pressure-tests your backtest with statistical rigor from published research. Your code stays on your machine.
v1.0 ships shortly. Waitlist gets first access.
Why this exists
Three things no other tool ships
- A 14-item leakage audit. Future-information features, label-overlap CV leakage, embargo enforcement, survivorship bias, eleven more. Each returns PASS / FAIL / ACCEPTED with rationale. Most overfit backtests aren't overfit in the parameters — they're leaking.
- Firm-specific gate evaluation. Eight built-in firm specs at launch — MFFU Pro, MFFU Core, MFFU Rapid and others — plus custom-spec construction for any firm. Intraday vs end-of-day methodology, trailing distance, lock-in mechanics. Your equity curves scored against the rules you actually deploy under.
- Self-hosted. Your code never leaves your machine. No telemetry, no cloud upload, no phone-home. The package installs locally, runs locally, writes the verdict locally. Only network call is an offline license-key signature check at import.
Why we built it
Five times the rig refused us
Eighteen months of work. The rig killed our own model across multiple iterations under proper measurement. It caught our own scoring methodology bug — research math, not deployment math. It found the actual deployable cell with zero maximum-loss-limit breach. We chose not to deploy it because the yield was operationally inadequate at single-account scale. Then we tested a projected calibration fix before recommending it — the fix worked technically but the deployment verdict went the wrong direction, and we marked down our own projections before another dollar was spent.
Most validation tools confirm what their operator wants to believe. Ours refused us five times. That's what gets productized.
Pricing
Three ways to buy
Perpetual license
$1,500
Single user, single machine.
Lifetime updates within v1.x.
Direct founder email access for the first 12 months.
48 business-hour support.
One-time payment.
Monthly
$79 / month
Same single-user entitlement.
All updates including v2.x.
24 business-hour priority support.
Cancel anytime.
Break-even with perpetual at ~19 months.
Annual
$799 / year
Same single-user entitlement.
All updates including v2.x.
24 business-hour priority support.
~16% off monthly.
Renews annually until cancelled.
14-day money-back guarantee on all paid tiers, no questions asked. EU 14-day right of withdrawal honored explicitly. Subscriptions cancel at end of paid period.
Get on the list
Join the waitlist
v1.0 ships shortly. Waitlist gets first access and a heads-up the day the buy link opens. No spam. Easy unsubscribe.
Email + optional firm. No tracking pixels, no third-party scripts, no resale.
Already a customer? Sign in to your account →
Questions
Frequently asked
What's actually in the package?
v1.0 ships seven validation modules behind a single validate() entry point:
- CPCV harness. Combinatorial Purged Cross-Validation, multi-seed by default (fifteen seeds), purging and embargoing, per-path equity curves.
- Deflated Sharpe Ratio. Full formulation — SR0 computed from the observed trial distribution.
- Probability of Backtest Overfitting via CSCV. The Combinatorially Symmetric Cross-Validation method, not a parameter-perturbation approximation.
- Randomization-test baseline. Real-vs-shuffled Sharpe gap.
- Fourteen-item leakage audit. Structured PASS / FAIL / ACCEPTED verdicts.
- Firm-specific gate evaluation. Eight built-in specs + custom-spec construction.
- Structural breaks (SADF). Clean-room implementation per Phillips/Wu/Yu.
Output formats: Markdown report, JSON serialization, plain-text summary, pandas DataFrames.
Code protection: Cython compilation of core IP modules (binaries only, no source), PyArmor obfuscation of the interface layer, offline license-key verification at import time.
How does this compare to vectorbt and other tools?
Honest comparison. We didn't invent the methodology family — we productized the honest application of it in one package with the pieces no one else ships.
| vectorbt free | vectorbt Pro | This package | |
|---|---|---|---|
| CPCV multi-seed | No | Yes | Yes (15-seed default) |
| Deflated Sharpe (full formulation) | No | Yes | Yes |
| PBO via CSCV | No | Yes | Yes |
| Randomization-test baseline | No | No | Yes |
| 14-item leakage audit | No | No | Yes |
| Firm-specific gate evaluation | No | No | 8 specs + custom |
| Unified validation verdict | No | No | Yes |
| Access | Open source | Invite-only paid | Public paid |
The three rows where we're the only "Yes" — randomization-test baseline, 14-item leakage audit, firm-specific gate evaluation — are the durable differentiation.
What if my prop firm isn't in the built-in spec list?
You construct a custom spec via the FirmGateSpec(...) dataclass. Pass in the trailing distance, lock-in floor, lock-in trigger semantics, intraday-vs-EOD methodology, and the validator scores your equity curves against that spec from day one.
The eight built-in specs at v1.0 are MFFU Pro 50K / 100K / 150K, MFFU Core 50K, MFFU Rapid 50K, plus three additional firms. More specs added in v1.1 based on actual user requests.
Can I try before I buy?
Yes — demo mode is free forever. Install the package and you can run validation on the built-in toy fixture without paying, without signing up, without giving us an email. You see exactly what the verdict output format looks like, exactly how the API feels, exactly which fields the methodology returns.
The only thing demo mode won't do is accept your own data. For that you need a paid license — perpetual, monthly, or annual. Every paid tier has a 14-day money-back guarantee, so the buyer's risk is comparable to a trial: pay, test on your own data, refund within 14 days if unhappy. No questions asked.
What's your refund policy?
14-day money-back, no questions asked, on every Perpetual License purchase. Email [email protected], the license key gets revoked, the payment processor refunds you in full within five business days.
Subscription Licenses can be cancelled at any time; the current paid period is non-refundable (cancellation just stops future charges). EU customers get the 14-day right of withdrawal under the Consumer Rights Directive, honored explicitly.
Why don't you cite Lopez de Prado's textbook?
We cite peer-reviewed papers, not textbooks. The clean-room implementation defense against derivative-work claims is stronger when the citation lineage goes back to the underlying academic literature directly.
- Bailey, D. H., & López de Prado, M. (2014). "The Deflated Sharpe Ratio." Journal of Portfolio Management, 40(5), 94-107.
- Bailey, D. H., Borwein, J. M., López de Prado, M., & Zhu, Q. J. (2017). "The Probability of Backtest Overfitting." Journal of Computational Finance, 20(4), 39-69.
- López de Prado, M. (2018). "The 10 Reasons Most Machine Learning Funds Fail." Journal of Portfolio Management, 44(6), 120-133.
- Phillips, P. C. B., Wu, Y., & Yu, J. (2011). "Explosive Behavior in the 1990s Nasdaq." International Economic Review, 52(1), 201-226.
- Phillips, P. C. B., Shi, S., & Yu, J. (2015). "Testing for Multiple Bubbles." International Economic Review, 56(4), 1043-1078.
- Hosking, J. R. M. (1981). "Fractional Differencing." Biometrika, 68(1), 165-176.
What about Topstep? Didn't they ban algos?
Topstep banned ProjectX API automation on Live Funded Accounts in February 2026. Algos remain allowed on Topstep's Trading Combine and Express Funded Account stages — Combine specs are in the v1.0 built-in spec library.
This is exactly why firm-agnostic validation matters. We built our own work against Topstep's gates over eighteen months. Then the rules changed under us. The rig adapts — same equity curves scored against MFFU Pro, MFFU Core, MFFU Rapid, or any custom spec you construct. Rules change. Your strategy doesn't. The rig adapts.
What Python versions are supported?
Python 3.10, 3.11, 3.12, and 3.13 on Linux x86_64, macOS arm64, and Windows x86_64. Distribution is platform-specific wheels delivered via the Merchant of Record payment processor.
Is this investment advice?
No. The Null Hypothesis Labs Validator is a statistical evaluation tool. It produces statistical outputs about historical data. It does not recommend trades, predict future returns, or warrant any strategy's suitability for live trading. The deployment decision is the user's alone and should involve a licensed financial advisor. Full disclaimer in the footer.
What if my license key gets compromised?
Contact [email protected]. We issue a replacement key bound to your verified email. The compromised key is blacklisted in the next patch release. No remote-revocation infrastructure in v1.0 — that's intentional, since remote revocation requires a phone-home telemetry path that contradicts the privacy guarantee.
Where can I read the full license terms?
The End User License Agreement, Website Terms of Use, Refund Policy, and Privacy Policy are linked from the footer of the documentation site (shipped with v1.0). All four are reviewed by a Colorado-licensed attorney before public sale.