**I Tested 1,000,000 Bitcoin Trading Strategies. — Transcript & Summary | SozAI**
Source: https://sozai.app/transcript/i-tested-1-000-000-bitcoin-trading-strategies/

Algovibes tests over 1 million Bitcoin trading strategies with pre-declared rules and finds zero survive a strict multiple-testing correction, highlighting how luck mimics skill.

## Key Takeaways

- Testing a million strategies makes pure luck indistinguishable from skill unless results are corrected for multiple comparisons.
- After correction, none of the over one million Bitcoin strategies cleared a strict significance bar.
- Zero survivors does not necessarily mean no edge exists, but it does show that impressive backtests can easily be artifacts of search.
- Always ask whether a trading strategy was validated against the many alternatives that were tried.

## What the video covers

- The creator tested over 1 million Bitcoin trading strategies across nine coins, six time frames, and 67 strategy types, with every parameter variation tested individually.
- Hypotheses were pre-declared before results were seen, using walk-forward out-of-sample testing only, next-candle-open fills, and realistic trading costs.
- Nonsensical parameter combinations (e.g., a fast moving average slower than the slow one) were excluded before testing to avoid cherry-picking.
- Top raw results showed out-of-sample returns above 100%, one above 3,000%, and Sharpe ratios near three, with a very small raw p-value.
- After correcting for testing over a million strategies at once, zero strategies survived the conservative significance bar.
- The creator notes that zero significant results does not prove zero edge, since many parameter combinations are near-duplicates, making the correction stricter than truly independent tests would require.
- The study is not a strict pre-registration and lacks untouched future data for confirmation, which is acknowledged as a limitation.
- The core lesson is that testing enough strategies makes luck look like skill, so viewers should ask whether any sold strategy was tested against many alternatives.
- The creator promotes a lab platform offering code, data, and results of many studies, including failures, as a more honest alternative to selling only winning strategies.

## Chapters

1. 00:00 Introduction: Testing 1 Million Strategies
2. 00:37 Study Design and Hypothesis Counts
3. 01:18 Pre-Registered Rules and Realistic Costs
4. 02:17 Top Results Before Correction
5. 02:55 Why Testing Many Strategies Creates Luck
6. 03:14 Correcting for Multiple Testing: Zero Survivors
7. 03:51 Limitations and What the Result Means
8. 05:21 Lab Platform and Closing Remarks

Answers

## Questions about this video

How many Bitcoin trading strategies were tested in this study?

The creator tested over 1 million strategies, with 1,051,758 hypotheses declared and 1,024,956 producing valid, scorable results. Each was tested on its own pre-declared setting across nine coins and six time frames.

Did any strategy survive the statistical correction?

No. After correcting for testing over a million strategies at once, zero strategies cleared the deliberately conservative significance bar, even though some raw results looked highly impressive.

Does zero surviving strategies prove Bitcoin trading has no edge?

No. The creator explicitly states that zero significant results is not the same as zero edge, since many parameter combinations are near-duplicates and the correction may be stricter than necessary.

## Full Transcript — Download SRT & Markdown

00:00

Speaker A

So, I tested over 1 million Bitcoin trading strategies. Every strategy, every coin, every time frame. Every meaningfully different setting. Each tested on its own pre-declared before I saw a single result. No cherry-picking, no here is my best one. So, how many of them actually

00:25

Speaker A

survived? Everything here, the code, the data, all of it is in the lab along with every other study I've run. More on that after. So, let's get into it.

00:37

Speaker A

1 million [clears throat] 51,758 hypothesis declared. 1 million 24,956 produced a valid, scorable result.

00:54

Speaker A

67 strategies, nine coins, six time frames, every parameter variation tested on its own. Nine coins split perfectly evenly. Time frames aren't even. Minute-level data obviously cost roughly 100 times more compute to test than daily data. So, density was set by compute cost before

01:18

Speaker A

any result existed not to favor an outcome. The rules fixed before any result existed. Walk forward out-of-sample only. Fills at the next candle's open.

01:33

Speaker A

Real trading costs corrected for auto-correlation, corrected for testing over a million things at once.

01:42

Speaker A

And also nonsensical parameter combinations like a fast moving average that's actually slower than the slow one excluded before testing, not after seeing which ones happened to win.

01:57

Speaker A

This is what working in { quotation marks} looks like. These are the top results before any correction. Real out of sample returns, some over 100% one over 3,000% with Sharpe ratios near three.

02:17

Speaker A

The single best of over a million a raw p value so small that on its own it would normally be called highly significant.

02:29

Speaker A

But, I didn't test just one strategy. I tested 1,024,956. So, if you test enough things, pure luck starts looking exactly like skill. I've explained and covered that a ton of times before, but it is still applying.

02:55

Speaker A

And it is important to understand this. So, here is what happens when you correct for this luck.

03:03

Speaker A

Every strategy from the last chart is in here too. And we got, not surprisingly, zero survivors.

03:14

Speaker A

Not just the top eight, all of them. Every result ranked. Every single bar lands in the same place. The strategies that looked incredible before correction, those tiny yellow numbers, everyone gets crushed by the same bar everyone else has to clear.

03:36

Speaker A

Zero. And think about that. Zero of 1,024,956 tested strategies cleared a deliberately conservative bar.

03:51

Speaker A

So, let me be precise about what that does and doesn't prove. Zero significant isn't the same as zero edge. It means nothing cleared this specific strict bar.

04:06

Speaker A

A lot of these parameter combinations are near duplicates of each other, which makes this correction stricter than if every test were truly [snorts] independent. So, zero survivors could partly reflect that, not just an absence of any real edge anywhere.

04:26

Speaker A

This also isn't a strict pre-registration, and there is no untouched future data yet to confirm it on.

04:34

Speaker A

What it does show is real execution, real cost, no look ahead, and the single best-looking result out of over a million tests or a million backtests still didn't survive being checked against everything else I tried.

04:51

Speaker A

So, the next time someone tries to sell you a strategy, ask them if they tested it against a million others.

05:01

Speaker A

So, this was one of many studies the lab has dozens. Every test I've ever run, the winners and the failure failures, all the code, that's the real value. Years of me finding out what doesn't work, so you don't burn month and money finding out

05:21

Speaker A

yourself. Everyone else basically sells you the one thing that looked good. I show you everything, including the 99% that didn't. That's not a weaker product, it's the only honest one.

05:37

Speaker A

lab.wisequant.app Support my work, and members also vote on what I test next. Thanks a lot for watching, and looking forward to see you in the upcoming videos. Let me know what you're interested in below.

05:50

Speaker A

Cheers. Bye-bye.

Topics: Bitcoin trading backtesting multiple testing correction overfitting quantitative trading out-of-sample testing strategy validation p-hacking algorithmic trading Sharpe ratio

Study this video

- [Flashcards from this video](https://sozai.app/tools/flashcard-maker/#from=transcript&id=96567)
- [Quiz on this video](https://sozai.app/tools/quiz-generator/#from=transcript&id=96567)
- [Crossword from this video](https://sozai.app/tools/crossword-maker/#from=transcript&id=96567)
- [Word search from this video](https://sozai.app/tools/word-search-maker/#from=transcript&id=96567)
- [Study guide from this video](https://sozai.app/tools/study-guide-maker/#from=transcript&id=96567)

Free, made by AI from this transcript. No signup.


---
This is the markdown twin of https://sozai.app/transcript/i-tested-1-000-000-bitcoin-trading-strategies/ — the same content, without the markup.
Published by SozAI (https://sozai.app). Reuse and quotation are allowed with attribution and a link back.
Machine-readable index: https://sozai.app/llms.txt · data API: https://sozai.app/api/
