In the first article of this series, a coin list invented a strategy. This one is about a quieter trap: deciding which way a signal works only after you have seen the data. Few mistakes in backtesting are easier to make, because the data seems to hand you the answer.
Why this is on a funding-arbitrage site. Both strategies below bet on price direction, using changes in funding rates as the signal. That is a different mechanism from collecting funding with a delta-neutral hedge, which is what FundingArb does. We publish failed research because the lessons apply to any signal you are shown, including funding-based ones.
The idea: watch funding move, not its level
Most funding-based signals look at the level: is funding high or low right now? Funding momentum looks at the change. For each coin, we took the average funding rate over the last 30 days and subtracted the same 30-day average as it stood one week earlier. A positive number means funding is getting more expensive for longs; a negative number means it is getting cheaper.
Which way should that predict prices? There are two stories, and both sound right:
That is the first warning sign. When both directions come with a convincing story, the story is not evidence. Only a test can decide, and only if the direction is fixed before the test runs.
Three tests, in the order they happened
-
Exploratory check · not written down in advanceContrarian, 10 large coinsA quick look on ten large perpetuals, three coins long and three short. It looked promising: 13.2% a year, Sharpe 0.57, placebo p = 0.024. But the whole result came from the second half: 2022–2023 lost money (Sharpe −0.80), 2024–2026 made it back and more (Sharpe 1.43).Sharpe 0.57 · p 0.024 · halves −0.80 / 1.43
-
Registered test 1 · written down before the runContinuation, 41 liquid coinsA formal test on a broader universe with fixed pass criteria. The hypothesis we wrote down was continuation, the opposite of the exploratory hint: with both stories sounding plausible, we picked one and committed to it. It failed clearly.Sharpe −0.92 · p 0.79
-
Check · no hypothesis, no verdictIs the change just the level in disguise?Before testing the mirror image, we measured how much it would overlap with plain funding-level signals. More on that below.
-
Registered test 2 · written down before the runContrarian, same 41 coinsThe mirror image of test 1, registered as a new hypothesis with its own pass criteria, and counted as one more test. It also failed.Sharpe 0.21 · p 0.056
Test 1: continuation fails clearly
| Strategy | APR | Max DD | Sharpe | Placebo p |
|---|---|---|---|---|
| Continuation | −24.3% | 76.8% | −0.92 | 0.79 |
41 liquid coins, Jan 2022 – Aug 2026, after costs and funding.
Negative in both halves (−0.22 and −1.68), negative or flat in all four walk-forward windows, worse at double costs (−1.26). In the placebo test, where we assigned coins to the long and short side at random 10,000 times, 79% of the random portfolios did at least as well. The registered direction wasn't just unhelpful; it pointed the wrong way.
The tempting move: just flip the sign
A Sharpe of −0.92 looks like an invitation. If going long rising funding loses this reliably, surely going short it wins? Two things stand in the way.
1. Choosing the direction after the fact is a second test
If you test a signal, look at the result and then keep whichever direction worked, you have run two tests and kept the better one. The reported p-value then pretends only one test happened. Statisticians call this the garden of forking paths: every choice made after seeing the data (direction, lookback, coin list, threshold) quietly multiplies the number of tests, and with enough of them something always looks significant.
So we did not flip the sign. We wrote the contrarian version down as a new hypothesis, fixed its pass criteria before running it, and counted it as one more test against our overall error budget.
2. Costs break the symmetry
The two directions are exact mirror images only before trading costs. Both directions pay costs on every rebalance, so flipping a losing strategy does not turn its loss into a gain of the same size:
Without costs, the two directions are almost perfect mirrors (−0.57 and +0.56). With realistic costs, the loser loses more and the winner keeps little: −0.92 becomes +0.21, not +0.92. Double the costs and both are negative.
Test 2: the contrarian mirror fails too
| Strategy | APR | Max DD | Sharpe | Placebo p |
|---|---|---|---|---|
| Contrarian | 2.1% | 48.9% | 0.21 | 0.056 |
Against the criteria written down before the run, it fails every single one:
| Pass criterion (fixed in advance) | Result |
|---|---|
| Placebo p-value at or below 0.0015 | 0.056 ✗ |
| Positive Sharpe in both halves | −0.51 / 1.00 ✗ |
| Positive Sharpe in all four walk-forward windows | −0.47 / −0.78 / 1.64 / 0.49 ✗ |
| Positive with any single coin removed | −0.09 to 0.33 ✗ |
| Positive at double trading costs | −0.14 ✗ |
"Almost significant" isn't
A p-value of 0.056 looks close to the textbook 0.05. It isn't close to anything that matters here. By the time this test ran, we had logged 33 hypothesis tests in this research programme, not counting about 120 earlier exploratory ideas. If you run 33 tests on pure noise, the chance that at least one of them clears p = 0.05 by luck is over 80%. To keep the overall false-alarm rate at 5%, each single test has to clear roughly 0.05 / 33 ≈ 0.0015. The contrarian result missed that bar by a factor of about 37. Counting the earlier exploratory ideas too, the bar would be closer to 0.0003.
Both tests are compared with the same random portfolios, so one distribution serves both. Random long/short baskets lose money on average here (trading costs and funding), which is why the distribution centres below zero. To pass at p ≤ 0.0015, the contrarian rule would have needed a Sharpe of at least 0.82.
The same pattern, three times
The exploratory check lost money in 2022–2023 and made it in 2024–2026. The registered contrarian test did the same: −0.51 in the first half, 1.00 in the second. Split into four windows, most of its return comes from a single stretch:
A signal that works only in one regime is a bet on that regime returning, not a signal. Continuation: −0.25 / 0.05 / −2.42 / −1.12. Contrarian: −0.47 / −0.78 / 1.64 / 0.49.
The contrarian line sits below its starting value for most of the period and is down more than 40% by April 2024. A single sharp move in the summer of 2024 recovers most of that, and it ends barely above where it started. That is what a sign change between halves looks like on an equity curve.
Here are all three tests side by side:
| Exploratory | Test 1 | Test 2 | |
|---|---|---|---|
| Direction | Contrarian | Momentum | Contrarian |
| Coins | 10 large | 41 liquid | 41 liquid |
| Registered | No | Yes | Yes |
| APR | 13.2% | −24.3% | 2.1% |
| Max drawdown | 52.6% | 76.8% | 48.9% |
| Sharpe | 0.57 | −0.92 | 0.21 |
| Placebo p | 0.024* | 0.79 | 0.056 |
| Halves | −0.80 / 1.43* | −0.22 / −1.68 | −0.51 / 1.00 |
* Exploratory check: 1,000 placebo runs instead of 10,000; halves split at January 2024 instead of July 2024.
Is the change just the level in disguise?
Before running the second test, we checked how much "funding is falling fastest" overlaps with "funding is lowest", the classic level-based signal. On average, the two measures are only loosely related (correlation 0.24). But the overlap is lopsided: the long side of the contrarian test shared about 40% of its coins with a plain "lowest funding" basket, twice what chance would give; the short side shared 21%, about chance level.
So a part of the contrarian long leg was buying the same coins as a funding-level strategy. That matters, because in the survivorship article exactly that leg, the coins with the most negative funding, turned out to be the weak spot once the coin universe was chosen honestly.
A tailwind both tests had. Both registered tests ran on 41 coins from a fixed list of liquid perpetuals that are still trading today: the same kind of hindsight list that inflated results in the survivorship article. That list flatters backtests. Even with that advantage, neither direction passed. On an honest point-in-time universe, we would expect them to look worse, not better.
What we take away
- When both directions have a good story, the story is not evidence. Fix the direction before the test runs, or the data will pick it for you.
- Don't flip the sign of a failed signal. If the mirror image is worth testing, register it as a new hypothesis, fix its criteria in advance and count it as one more test.
- Costs break the symmetry. Reversing a strategy with a Sharpe of −0.92 does not give +0.92. Both directions pay to trade.
- Judge a p-value against the number of tests. After 33 logged tests, 0.056 is not "almost significant"; the bar is closer to 0.0015.
- A sign change between halves is a red flag. A full-period number that averages a losing regime and a winning one describes neither of them.
What this means when you evaluate any signal
When someone shows you a backtest of "long X, short Y", ask whether the direction was fixed before the data was looked at, whether the opposite direction was also tried, and how many variants were run before this one was shown. A single good-looking result is not informative without that context.
FundingArb doesn't try to predict where funding goes next or which way prices move. It collects funding with a hedged position, and only enters when projected funding clears fees by a margin.
Methodology
- Data
- Binance USDⓈ-M perpetuals: daily closes and daily funding.
- Universe
- Registered tests: 41 liquid perpetuals from a fixed list of coins still trading today (see the tailwind note above); minimum age 220 days. Exploratory check: 10 large perpetuals.
- Signal
- 30-day average funding up to yesterday minus the same average as of one week earlier.
- Period
- 2022-01-15 to 2026-08-31 (registered tests).
- Portfolio
- Weekly rebalancing; quintiles (at least 3 coins per side); 50% gross per side; volatility-targeted. Exploratory check: 3 coins per side.
- Costs
- Per side and per change in position, plus daily funding payments.
- Placebo
- 10,000 portfolios with coins assigned to the long and short side at random each week, same random sequence for both directions.
- Multiple testing
- Threshold 0.05 divided by the number of logged tests run so far (33 at registration of the second test, ≈ 0.0015). This is the earliest test in the series, which is why its count is lower than in the other articles. About 120 earlier exploratory hypotheses are not in this count; including them makes the threshold stricter and changes no verdict here.
- Known limits
- Binance data only; fixed coin list with hindsight (flatters results); execution on other exchanges not modelled.