Today's trading software is amazing. It can perform mundane tasks, perform complex analysis and it can even be used to automate your trading strategy. But, it also has its drawbacks. Probably the biggest danger in using trading software is with backtest optimization.
Today's best trading software packages all include the capability to program trading strategies into them, and then test the idea over historical data. This is commonly referred to as a "backtest." The benefit in backtesting is that one can see how a strategy or idea performed historically, which helps gives confidence that it might work in the future.
Since most strategies have variable inputs, for example the number of bars to use in a moving average calculation, or the dollar amount of a stop loss, trading software also includes optimization ability. With optimization, one can run backtest with multiple values for each and every value. This can easily turn one trading idea into thousands, or even millions, of unique trading strategies - one for each different set of variables.
Therein lies the rub. Practically any idea, if tested with enough combinations, will show at least some cases where the strategy has been profitable. Some combinations might even show incredible results, and the strategist will get goose bumps thinking about the money possible from trading this optimized strategy. In fact, many people just choose the set of variables with the highest profit, plug them in and immediately start trading with real money.
This is absolutely the quickest road to ruin. Numerous studies have been conducted that show the best variable combination historically will NOT be the best combination going forward. Another example of this is with financial magazine ratings of the best mutual funds. Typically, the best funds over the last 5 years will not be the best performers over the next 1-5 years in the future.
Here is a test you can run at home to see this phenomenon in action. When you check out at the grocery store, you usually pick the checkout line that you think will be fastest. So, how often are you right? If you are like me, you almost never pick the quickest line, even though you did some analysis and optimization to pick that line. Picking values for trading strategy inputs can behave the same way.
So, how does one get around this optimization issue? One way is to look for consistent performance regions in the results. For example, if a moving average length of 2 to 10 was tested, and the only profitable value was a length of 6, chances are good that is not a good value or strategy going forward. But if all values for 2-10 were profitable, with little variation, then picking any value, not just the best one, might be a good way to go.
Instead of optimizing, one is better off studying the markets and trying to understand the behavior of prices, and then develop a strategy around it. That is a far superior method to optimizing and picking the peak value.
