How to Compare Properly Two EA/strategy/indicator?

[singularity 10]

I would like to know how should I compare the performance of two different strategy.

 

Let's say I give you two source code. Each of them is a winning strategy, ready to run. However - of course - you won't send them immediately to your live account, but you check their performance in your environment. How would you decide which one is the better? How much data is required to draw some conclusions of the performance?

 

I'm really interested in this guys, because I have seen great looking equity curves but I don't trust them. How do you decide which EA has better performance, and what are your criterions?

 

Now I am fully aware of the fact comparing their performance is a subjective thing since everyone has his own taste. I'm not asking to help what should I use. I'm asking you to show me some examples how you really compare the performance when you analyze them.

 

Also - just theoretically - please assume that you have to choose one strategy from the two winners because you can only run one on the live acccount. Let's just leave portfolios for a moment in this thread...

 

In a geeky way my question is the following:

double GetStrategyScore(string sourceCode)
{
 double score = ..... // <--- what would you put here so you could compare two strategy and decide which one is better?
 return score;
}

 

Thanks for any advice!

[zupcon 10]

double score = MathSrand(); :rofl:

 

 

Well it's as likely to be as good as anything else given your dealing with a non stationary time series, and since you've given no details about what you consider to be "better", how could we even begin to guess ?

 

What's important to you? Returns, drawdown, frequency of trades, capital requirements, correlation with other markets etc etc etc. the list is endless, and my idea of a good system, is unlikely to meet your requirements and vice versa.

 

Most traders using automation tend to continually diversify risk across multiple systems anyway, so when in the situation of having 2 profitable strategies, they'd run both.

 

The key point here is just because a strategy performs in a particular way historically really doesn't mean it will continue to do so in the future, you can almost guarantee that it won't, but you can't predict if it'll get better or worse, but it's unlikely to be the same

[Seeker 11]

I would go with expectancy multiplied by average number of trades per day.

[singularity 10]

  zupcon said:

.., and since you've given no details about what you consider to be "better", how could we even begin to guess ?

 

Zupcon, thanks for feedback, now I can come up with a new better but similar question. Here it goes:

Let's say I have a nice strategy. I even implemented for you. Then I send it to you because I maybe stupid to keep it...

So you have now an implemented strategy out of nowhere. Let's assume that you already running a portfolio on one or more live account and you are doing fine with it. However you check this new strategy and if it's reasonably good, then you put it into your portfolio.

Now here comes my question: what requirements need to be met for you to give this strategy a chance on a live account? What do you want to see to have the conclusion that you should try to run this strategy next to your already running instances?

I hope this is a more understandable question.

 

Now, I want to put emphasis on the part what do YOU check? I am interested in what you, Zupcon or others check before it can go live. I would like to hear examples from you how do do it? What are your priorities?

This is simply because I want to see examples how others do this thing, since I have no real life example yet. I simply learn these things now and I think there are no better knowledge than real life examples that are used by real life people. I'm aware of lot of theoretical, mathematical writings on this topic, but I really would like to hear your routine.

 

Thank you again.

[singularity 10]

  Seeker said:

I would go with expectancy multiplied by average number of trades per day.

 

That's clear and simple but I'm curious here if this is the only thing you need to know before the strategy can go live?

[Seeker 11]

  singularity said:

That's clear and simple but I'm curious here if this is the only thing you need to know before the strategy can go live?

 

There are quite a few metrics of interest, I just suggested a simplistic one, tha captures a lot for comparing two systems.

 

You could look into Sharpe ratio, drawdown etc.

 

It is obviously necessary to forward test the strategy.

[renvik 10]

Have written several strategies which made thousands of $$ in backtest but failed in forward testing, the best some of them did was make 2 ES points per day which was not a worth it for me - that is the bottom line (having examined sharpe ratio, draw down, etc.).

I do look at the largest losing trade (while not using a stop loss) to gauge the efficacy

of the strategy logic.

[Seeker 11]

  renvik said:

Have written several strategies which made thousands of $$ in backtest but failed in forward testing, the best some of them did was make 2 ES points per day which was not a worth it for me - that is the bottom line (having examined sharpe ratio, draw down, etc.).

I do look at the largest losing trade (while not using a stop loss) to gauge the efficacy

of the strategy logic.

 

 

 

2 ES points per day is plenty. If you have something like that, then you should trade it. I would guess that you don't have a strategy that returns 2 ES points a day, otherwise you would be trading it and making a lot of money.

[renvik 10]

  Seeker said:

2 ES points per day is plenty. If you have something like that, then you should trade it. I would guess that you don't have a strategy that returns 2 ES points a day, otherwise you would be trading it and making a lot of money.

 

I just wanted to point out to @singularity that there could be other factors to consider.

Your guess is wrong, but your comment serves to illustrate my point; food for thought for you too.

 

Cheers.

[Seeker 11]

No, my guess is not wrong. Anyone who can develop a strategy that makes 2 points a day net average on ES, knows that is more than enough and would appreiate it and trade it and not look for other things.

 

One can easily get filled 100 contracts on the ES, which corresponds to $5000 per point, which for a 2 point a day strategy corresponds to $10,000 a day or $50,000 a week average.

 

And yet here we have you saying you're not interested in $50,000 a week salary and you'll look for something else.

 

What a load of nonsense.

 

You DON'T have a strategy that makes 2 points a day on the ES. What you perhaps have is some curve fit strategy that doesn't work and loses money per day on average.

[zupcon 10]

  Seeker said:

No, my guess is not wrong. Anyone who can develop a strategy that makes 2 points a day net average on ES, knows that is more than enough and would appreiate it and trade it and not look for other things.

 

One can easily get filled 100 contracts on the ES, which corresponds to $5000 per point, which for a 2 point a day strategy corresponds to $10,000 a day or $50,000 a week average.

 

And yet here we have you saying you're not interested in $50,000 a week salary and you'll look for something else.

 

What a load of nonsense.

 

You DON'T have a strategy that makes 2 points a day on the ES. What you perhaps have is some curve fit strategy that doesn't work and loses money per day on average.

 

Would you trade such a system if it experienced significant drawdowns ?

 

I probably wouldn't , even if it averaged 2 points a day

 

I've got a few profitable ideas that I developed and won't trade based on drawdown, or variance in returns. It's not all about profit at any cost.

[sergso 10]

  Seeker said:

2 ES points per day is plenty. If you have something like that, then you should trade it. I would guess that you don't have a strategy that returns 2 ES points a day, otherwise you would be trading it and making a lot of money.

 

CAGR/Max DD

 

This is how mutual funds are rated.

[Seeker 11]

  zupcon said:

Would you trade such a system if it experienced significant drawdowns ?

 

I probably wouldn't , even if it averaged 2 points a day

 

I've got a few profitable ideas that I developed and won't trade based on drawdown, or variance in returns. It's not all about profit at any cost.

 

 

Depends.

 

Now, if you have 20 strategies which work, then you can compare expectancy and variance, and then you may decide it's not worth trading the 2 point a day ES strategy as you have better.

 

However, if you have no strategies which work, except this 2 point a day strat, then that's all you got that's profitable. So yes I would trade it in those circumstances while I looked for something better, or developed the strat further. So perhaps I misread renvik's post, seemed to me he was saying he just had loads of strategies which didn't work and this one strategy which did but he didn't consider good enough and nothing else. So apologies if I read your post wrong renvik, and hope you're trading an even better strategy.

 

I think I've just been reading too many false claims over the years, with people talking about 70% win rates and winners twice as large as losers, or 80% winners and 1:1 risk reward etc (read both only the other day). A 2 point a day edge on the ES seems a lot in my opinion. In my experience a lot of edges (mechanical ones at least) are small and a lot closer to the 50% for a 1:1 risk reward or closer to 33% for a 1:2 risk reward than most people seem willing to admit on public forums.