What to consider when analyzing a system
Someone asked me what I look at when testing and analyzing a system. He asked what was the minimum Profit Factor (Gross Profit / Gross Loss) I considered as a valid for trading, so I’ll start with that.
Minimal profit factor varies depending on the kind of system. Quite honestly I don’t really pay much attention to it. For daytrading, which is all I do, I like to see 1.4-1.8. Anything less than 1.4 probably has too low of an average trade and above 1.8 is probably overfit. These are my opinions of course.
These PF’s include transaction costs (commission and slippage). I always include transaction costs – it makes zero sense to me to not include them. These Profit Factor’s include $35 roundtrip for commission and slippage because the trade-assist brokers that I lease my systems through charge an average of $20 roundtrip for commissions. If you are trading in TradeStation for yourself then $20 roundtrip for commission and slippage should be plenty and the Profit Factor range might be 1.4-2.0.
The main things I DO look at is average trade (absolute minimum $50 after transaction costs and really prefer $60 or higher) and the ratio of annual net profit to drawdown – ALL BASED ON OUT OF SAMPLE DATA. Disregard in-sample data for system validity. Basically if I have a system with out of sample results of an annual profit / drawdown ratio of at least 2 and an average trade of $60 or more after transaction costs then I have a tradable system in my opinion. To confirm I’ll also look at the equity curve, Profit Factor and other things to reinforce the validity – especially the equity curve for consistency and as a check for overfitting (if equity curve chops around and then goes straight up that is not consistent, if the equity curve looks like a straight line then something is wrong somewhere in the testing!).
Make sense? Questions?
gary