How We Know
What We Know
Most trading claims cannot be checked. This module is a method for checking them, and it is the standard the rest of the course is held to.
Why Most Trading Advice Cannot Be Checked
Trading content is full of claims that sound settled and aren't. A setup "works." A level "holds." A pattern "has an edge." Usually nobody says what the rule was, when it was written down, what it was compared against, or how many times it was tried.
Without those things there is nothing to check. A claim you can check has a rule stated exactly enough that someone else could run it, a record of when the rule was fixed, a comparison that could have made it look bad, and results from conditions that were not used to build it. Most advice offers none of these, so the only thing left to judge it by is how confident the person sounds.
Lock The Rule Before You Look
Write the rule down, in full, before you open the data. Entry, stop, target, the conditions under which it applies and the conditions under which it does not.
The order matters because of what happens when you reverse it. If you look at the data and then write the rule, you have fitted the rule to the data, and a fitted rule always looks good on the data that produced it. You will find a version that would have worked, and it will tell you nothing about the next session. Writing the rule first means the data gets a fair chance to say no.
Test Against A Null
A result needs something to be compared with. The comparison here is a null: take the same number of entries at the same places, but with the timing scrambled so that any real pattern is destroyed while the volatility and the context stay the same. Run the rule on that.
If the rule does no better than random entries at the same places, it is not a rule. Markets drift, ranges expand and contract, and plenty of things look like skill when they are just the market doing what it was going to do. The null is how you find out how much of the result was the rule and how much was the weather.
Split The Sample, And Look At Recent Data Separately
Divide the sample in half and check both halves. A rule that is positive in one half and negative in the other has not shown you an edge. It has shown you one good stretch and one bad one, and you cannot tell which is the exception.
Then look at recent data on its own. Markets change, and a rule that worked in conditions that no longer exist is a historical fact rather than something you can use. Each of these splits is one more chance for the result to fail, and a result that survives all of them is worth more than one that has only been shown in aggregate.
A rule that tests well on market data tells you the market behaved that way. It does not tell you that you can trade it.
Count Your Own Executions Separately
A backtest assumes a perfect trader. It takes every signal, at the stated price, without hesitation. You do not. You skip some signals, hesitate on others, and place orders a little differently each time.
So keep your own executions in their own count, apart from any backtest, and never blend the two. A rule that tests well and that you cannot execute is not yours yet. The market data says the idea has merit. Your own record says whether you can collect on it, and the two answers are independent.
Why Simulated Fills Cannot Measure Edge
The platform's simulator fills a limit order the moment price touches it. Real markets do not. Price can touch your level and turn around without ever filling you, and the trades that get away from you are often the best ones.
Where in the level you get filled is exactly the variable that decides whether a trade works, so a simulator that ignores it is biased on the one thing that matters most. Sim can tell you how often something happened. It cannot tell you whether you would have made money from it. Treat simulated trades as practice, and as a count of occurrences, but never as evidence of edge.
Why Small Samples Mislead
A small run of trades feels like information, and mostly it is noise. A few wins in a row proves little, and so does a few losses.
The count of trades is also not the whole story. Thirty trades across four sessions is one week's conditions, not a sample. If those days were trending, everything looked like a trend setup. If they were quiet, nothing worked. What you need is trades spread across many different days and conditions, so that no single kind of market is doing all the work.
Of any claim, including mine, ask: what would have to be true for this to be wrong, and has that been checked? If nobody can say what would count as wrong, the claim cannot be tested. If they can, and it has not been checked, then it is an idea and not a result.
The Standard This Course Is Held To
Everything in this course is meant to be read through this lens: a rule stated before it was tested, compared with a null, checked in separate halves and in recent data, and kept apart from the author's own execution record. That is the standard, and the course should be held to it as much as anything else you read.
The framework is still being built and tested, in the open, and nothing here is presented as proven. That does not make it worthless. It makes it a set of testable ideas, and it tells you how to use them: test them yourself, and be suspicious of anyone, this site included, who tells you they are settled.