Backtesting, Demo, and Live Trading

[trelco 10]

What is the difference between each?

 

We have all heard that 90% of traders lose money, so why are most people profitable on demo's but not in practice?

 

What changes?

What changes if it is an automated system?

 

thank you.

[Tams 20]

  trelco said:

What is the difference between each?

 

We have all heard that 90% of traders lose money, so why are most people profitable on demo's but not in practice?

 

What changes?

What changes if it is an automated system?

 

thank you.

 

the core problem with backtest is, most backtest system do not replay the data as in live stream.

 

ie. to be real, you have to replicate the bid, ask, bid size, ask size, and trade size in the same quantity and timing as the real life.

Edited October 5, 2011 by Tams

[zdo 20]

trelco, (it may seem like this is a smart ass answer but it isn't)

The best way to find out the difference between real and sim trading is to go live and real now and find out the differences that are YOUR differences - before you get a bunch of well intended, but ultimately damaging, feedback from others about their differences!

[trelco 10]

My systems account for 4-12 ticks of slippage (randomly selected) with much more than that (4-36 ticks) during extreme volatility. Those are usually capitulation days.

 

They also don't seriously depend on exact bid/ask size or trades for that 1 bar (assuming there are at least 100 contracts on book during recent years).

 

All of the strategies were developed during the last 3 months based on theories I had. None of them use "fitting."

 

They all outperform the S&P every year and meet my scalability and risk metric requirements (max drawdown, max length of losing streak, max sustainable drawdown, std deviation, Sharpe/Sortino/Beta/R-squared) *Am I missing anything here?*

 

The data used starts from 1981-1998 (depending on security) to the present.

 

 

Basically what I'm asking is what could I be missing that will mess up real world performance?

 

 

zdo: You are right. Knowing the types of problems others ran into or know about might help me and others find what may stand in the way of successful implementation though.

[krohara 10]

Backtesting is not real money. Once emotions are involved, it's a whole different ballgame.

[jdcool10 10]

  trelco said:

What is the difference between each?

 

We have all heard that 90% of traders lose money, so why are most people profitable on demo's but not in practice?

 

What changes?

What changes if it is an automated system?

 

thank you.

 

What really changes is YOU, Trelco, and that is everything. In a simulated environment, you have no emotion at play (that fear and greed thing that causes us not to follow our rules, aka taking profit too early or moving our stop loss in a wrong direction). Therefore whether your systeme looses $1 or a thousannd does not change your mood or thought or feelings. Once real money is involved and your position is in the red, they DO COME INTO PLAY and you throw the rationale behind your system down the drain. And I think that is also true even if you have an automated system because the market itself changes while your system does not. You would have to make adjustement to curve fit it or correct it according to the market and you are in a never ending adjudsting mode ....of an automated system...

 

Just my two cents

 

P.S- About TAMS' comments earlier, if you use Ninja Trader software, it allows you to record and replay the markets as it has happened with ticks and the like, thus allowing you to backtest your strategy and see how YOU would have performed if you would have recognized and taken the signals as they occured.

[juniorh 10]

Back Testing is vanity, Forward Testing is sanity. After reading Mar Douglas' book Trading the in the zone I realised the importance of backtesting but also it's fallacy. Especially when you consider that there are so many variables that makes each point in the past unique and somewhat unrepeatable.

 

Now I appreciate more J Dalton's Mind over Market and itradepod's discretionary approach to trading based on inherent market mechanics rather that a purely mechanical system that heavily relies on backtesting.

[trelco 10]

lol okay but it is basically people who can't manage their emotions that are trading on what they "think" (eg: this will come back to me or I need to take profits now +1tick). Also the fact that this seems to attract gambling types who take illogical risk.

 

My main problem if anything is that I hold winners too long and sell losers too early when I diverge from the system. Divergence generally only happens when I rush through the set up. This accounts for 3% of trades when I replay tested my system by hand for several years. This is unlikely to happen in real time because I won't have to get through 1000's of set ups in a day. The losses because of that are equivalent to taking 1 extra emergency stop per year. Relatively insignificant. The other part about forward testing: because of the way it was built that is already factored in/done.

 

So far today I am a little over +$2100 using 1 ES contract. After a couple of weeks like this I think I will increase size. I skipped NFP.

 

After all the zerohedge propaganda, poor retail performance studies, and general sub 100%+ returns people have posted with under 300k AUM I figured there had to be some mysterious market force that screwed people out of their money...

 

I guess even with all the resources of the internet there is a lot of dumb money floating around...

[charles1 10]

GREED GREED GREED. Do yourself a favor and create rules for your trading. Laminate them and glue to your trading desk. Never sway from your rules. If you do sway from your rules write down why? There is no such thing as greed in paper trading... i.e. lets say you open a position in a paper account, and you set no stop. Than you go to the bathroom, the kitchen, answer the door, etc., etc... Than you come back to your trading desk and see you profited a lot from the position and choose to flat it or cover it. You wold never do such a thing with real money till you got really really good at trusting your trading skills and handling your emotions... A really good trader is emotional sound. And anyone who can learn to trade a real account like a paper account will enjoy immense riches. When real money is on the line "attachment begins", there's no feeling of attachment with a paper account....

[juniorh 10]

  trelco said:

lol okay but it is basically people who can't manage their emotions that are trading on what they "think" (eg: this will come back to me or I need to take profits now +1tick). Also the fact that this seems to attract gambling types who take illogical risk.

 

My main problem if anything is that I hold winners too long and sell losers too early when I diverge from the system. Divergence generally only happens when I rush through the set up. This accounts for 3% of trades when I replay tested my system by hand for several years. This is unlikely to happen in real time because I won't have to get through 1000's of set ups in a day. The losses because of that are equivalent to taking 1 extra emergency stop per year. Relatively insignificant. The other part about forward testing: because of the way it was built that is already factored in/done.

 

So far today I am a little over +$2100 using 1 ES contract. After a couple of weeks like this I think I will increase size. I skipped NFP.

 

After all the zerohedge propaganda, poor retail performance studies, and general sub 100%+ returns people have posted with under 300k AUM I figured there had to be some mysterious market force that screwed people out of their money...

 

I guess even with all the resources of the internet there is a lot of dumb money floating around...

 

 

W.C. Fields famously said, "It is morally wrong for a sucker to keep his money".