Jump to content

Welcome to the new Traders Laboratory! Please bear with us as we finish the migration over the next few days. If you find any issues, want to leave feedback, get in touch with us, or offer suggestions please post to the Support forum here.

  • Welcome Guests

    Welcome. You are currently viewing the forum as a guest which does not give you access to all the great features at Traders Laboratory such as interacting with members, access to all forums, downloading attachments, and eligibility to win free giveaways. Registration is fast, simple and absolutely free. Create a FREE Traders Laboratory account here.

singularity

How to Compare Properly Two EA/strategy/indicator?

Recommended Posts

I would like to know how should I compare the performance of two different strategy.

 

Let's say I give you two source code. Each of them is a winning strategy, ready to run. However - of course - you won't send them immediately to your live account, but you check their performance in your environment. How would you decide which one is the better? How much data is required to draw some conclusions of the performance?

 

I'm really interested in this guys, because I have seen great looking equity curves but I don't trust them. How do you decide which EA has better performance, and what are your criterions?

 

Now I am fully aware of the fact comparing their performance is a subjective thing since everyone has his own taste. I'm not asking to help what should I use. I'm asking you to show me some examples how you really compare the performance when you analyze them.

 

Also - just theoretically - please assume that you have to choose one strategy from the two winners because you can only run one on the live acccount. Let's just leave portfolios for a moment in this thread...

 

In a geeky way my question is the following:

double GetStrategyScore(string sourceCode)
{
 double score = ..... // <--- what would you put here so you could compare two strategy and decide which one is better?
 return score;
}

 

Thanks for any advice!

Share this post


Link to post
Share on other sites

double score = MathSrand(); :rofl:

 

 

Well it's as likely to be as good as anything else given your dealing with a non stationary time series, and since you've given no details about what you consider to be "better", how could we even begin to guess ?

 

What's important to you? Returns, drawdown, frequency of trades, capital requirements, correlation with other markets etc etc etc. the list is endless, and my idea of a good system, is unlikely to meet your requirements and vice versa.

 

Most traders using automation tend to continually diversify risk across multiple systems anyway, so when in the situation of having 2 profitable strategies, they'd run both.

 

The key point here is just because a strategy performs in a particular way historically really doesn't mean it will continue to do so in the future, you can almost guarantee that it won't, but you can't predict if it'll get better or worse, but it's unlikely to be the same

Share this post


Link to post
Share on other sites
  zupcon said:
.., and since you've given no details about what you consider to be "better", how could we even begin to guess ?

 

Zupcon, thanks for feedback, now I can come up with a new better but similar question. Here it goes:

Let's say I have a nice strategy. I even implemented for you. Then I send it to you because I maybe stupid to keep it...

So you have now an implemented strategy out of nowhere. Let's assume that you already running a portfolio on one or more live account and you are doing fine with it. However you check this new strategy and if it's reasonably good, then you put it into your portfolio.

Now here comes my question: what requirements need to be met for you to give this strategy a chance on a live account? What do you want to see to have the conclusion that you should try to run this strategy next to your already running instances?

I hope this is a more understandable question.

 

Now, I want to put emphasis on the part what do YOU check? I am interested in what you, Zupcon or others check before it can go live. I would like to hear examples from you how do do it? What are your priorities?

This is simply because I want to see examples how others do this thing, since I have no real life example yet. I simply learn these things now and I think there are no better knowledge than real life examples that are used by real life people. I'm aware of lot of theoretical, mathematical writings on this topic, but I really would like to hear your routine.

 

Thank you again.

Share this post


Link to post
Share on other sites
  singularity said:
That's clear and simple but I'm curious here if this is the only thing you need to know before the strategy can go live?

 

There are quite a few metrics of interest, I just suggested a simplistic one, tha captures a lot for comparing two systems.

 

You could look into Sharpe ratio, drawdown etc.

 

It is obviously necessary to forward test the strategy.

Share this post


Link to post
Share on other sites

Have written several strategies which made thousands of $$ in backtest but failed in forward testing, the best some of them did was make 2 ES points per day which was not a worth it for me - that is the bottom line (having examined sharpe ratio, draw down, etc.).

I do look at the largest losing trade (while not using a stop loss) to gauge the efficacy

of the strategy logic.

Share this post


Link to post
Share on other sites
  renvik said:
Have written several strategies which made thousands of $$ in backtest but failed in forward testing, the best some of them did was make 2 ES points per day which was not a worth it for me - that is the bottom line (having examined sharpe ratio, draw down, etc.).

I do look at the largest losing trade (while not using a stop loss) to gauge the efficacy

of the strategy logic.

 

 

 

2 ES points per day is plenty. If you have something like that, then you should trade it. I would guess that you don't have a strategy that returns 2 ES points a day, otherwise you would be trading it and making a lot of money.

Share this post


Link to post
Share on other sites
  Seeker said:
2 ES points per day is plenty. If you have something like that, then you should trade it. I would guess that you don't have a strategy that returns 2 ES points a day, otherwise you would be trading it and making a lot of money.

 

I just wanted to point out to @singularity that there could be other factors to consider.

Your guess is wrong, but your comment serves to illustrate my point; food for thought for you too.

 

Cheers. :)

Share this post


Link to post
Share on other sites

No, my guess is not wrong. Anyone who can develop a strategy that makes 2 points a day net average on ES, knows that is more than enough and would appreiate it and trade it and not look for other things.

 

One can easily get filled 100 contracts on the ES, which corresponds to $5000 per point, which for a 2 point a day strategy corresponds to $10,000 a day or $50,000 a week average.

 

And yet here we have you saying you're not interested in $50,000 a week salary and you'll look for something else.

 

What a load of nonsense.

 

You DON'T have a strategy that makes 2 points a day on the ES. What you perhaps have is some curve fit strategy that doesn't work and loses money per day on average.

Share this post


Link to post
Share on other sites
  Seeker said:
No, my guess is not wrong. Anyone who can develop a strategy that makes 2 points a day net average on ES, knows that is more than enough and would appreiate it and trade it and not look for other things.

 

One can easily get filled 100 contracts on the ES, which corresponds to $5000 per point, which for a 2 point a day strategy corresponds to $10,000 a day or $50,000 a week average.

 

And yet here we have you saying you're not interested in $50,000 a week salary and you'll look for something else.

 

What a load of nonsense.

 

You DON'T have a strategy that makes 2 points a day on the ES. What you perhaps have is some curve fit strategy that doesn't work and loses money per day on average.

 

Would you trade such a system if it experienced significant drawdowns ?

 

I probably wouldn't , even if it averaged 2 points a day

 

I've got a few profitable ideas that I developed and won't trade based on drawdown, or variance in returns. It's not all about profit at any cost.

Share this post


Link to post
Share on other sites
  Seeker said:
2 ES points per day is plenty. If you have something like that, then you should trade it. I would guess that you don't have a strategy that returns 2 ES points a day, otherwise you would be trading it and making a lot of money.

 

CAGR/Max DD

 

This is how mutual funds are rated.

Share this post


Link to post
Share on other sites
  zupcon said:
Would you trade such a system if it experienced significant drawdowns ?

 

I probably wouldn't , even if it averaged 2 points a day

 

I've got a few profitable ideas that I developed and won't trade based on drawdown, or variance in returns. It's not all about profit at any cost.

 

 

Depends.

 

Now, if you have 20 strategies which work, then you can compare expectancy and variance, and then you may decide it's not worth trading the 2 point a day ES strategy as you have better.

 

However, if you have no strategies which work, except this 2 point a day strat, then that's all you got that's profitable. So yes I would trade it in those circumstances while I looked for something better, or developed the strat further. So perhaps I misread renvik's post, seemed to me he was saying he just had loads of strategies which didn't work and this one strategy which did but he didn't consider good enough and nothing else. So apologies if I read your post wrong renvik, and hope you're trading an even better strategy.

 

I think I've just been reading too many false claims over the years, with people talking about 70% win rates and winners twice as large as losers, or 80% winners and 1:1 risk reward etc (read both only the other day). A 2 point a day edge on the ES seems a lot in my opinion. In my experience a lot of edges (mechanical ones at least) are small and a lot closer to the 50% for a 1:1 risk reward or closer to 33% for a 1:2 risk reward than most people seem willing to admit on public forums.

Share this post


Link to post
Share on other sites

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.

Guest
Reply to this topic...

×   Pasted as rich text.   Paste as plain text instead

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.


  • Topics

  • Posts

    • LYV Live Nation Entertainment stock watch, attempting to move higher off the 128.81 double support area, target 140 area at https://stockconsultant.com/?LYV
    • VCEL Vericel stock, couple strong days, watch for a range breakout at https://stockconsultant.com/?VCEL
    • WIX stock, watch for a narrow range breakout at https://stockconsultant.com/?WIX
    • Date: 9th January 2025. FOMC Minutes Signal Slower Rate Cuts, UK Borrowing Costs Surge, & Global Market Update.   Asia & European Sessions: The FOMC minutes showed that the Committee expected to be slowing the pace of rate cuts after its decision to trim rates another -25 bps. Following an unexpected emergency rate cut in September, despite there being no immediate crisis, the Fed has since shifted towards a more measured approach, indicating that a slower pace of rate reductions would be “appropriate” by December. The core strategy remains consistent: to bring inflation down. While inflation-related discussions did touch on concerns over US President-elect Trump’s trade taxes and deportation plans, these issues were not the main focus of the inflation debate. The Greenback was firmer overnight on reports Trump would declare a state of emergency to get his tariff plans through. It dipped on the ADP report but bounced on the tight jobless claims data. The index had firmed yesterday after Trump denied reports he would soften his tariff plans, and after the strength in the JOLTS numbers Tuesday. Solid 30-year auction results also supported in the afternoon. China's inflation data for December showed largely stable consumer prices, with food prices stabilizing (a notable factor given food’s significant weight in the consumer basket) and only modest increases in non-food prices, despite efforts to boost domestic consumption. Producer prices, however, continue to struggle with deflation. In the UK, the BRC shop price index fell more sharply than anticipated, with a significant drop in non-food item prices, likely influenced by Black Friday discounts. When combined with sales data, this suggests that UK consumers increased their real-term spending in the fourth quarter, driven by lower prices and promotions. Gilts remain under pressure in early trade, with the UK 10-year rate up 2.1 bp at 4.81%. UK 10-year borrowing costs surged to their highest point since the global financial crisis, while the Pound plummeted, as a deepening bond sell-off raised concerns over the Labour government’s ability to meet its self-imposed budget targets. So far in 2025, borrowing costs in the UK have increased at a faster pace than in other major economies, driven by investor fears over the government’s large borrowing requirements and the mounting risk of stagflation. Eurozone industrial production rose 1.5% m/m in November. Germany's jobless rate still is very low by European standards, but the overall picture remains pretty gloomy, with political uncertainty and the threat of Trump tariffs not helping.   Financial Markets Performance: European stock markets are mixed, with the FTSE100 outperforming and up 0.4%, while the DAX is down -0.2%, after a largely weaker close across Asia. Hang Seng and CSI 300 lost -0.3%, after Chinese inflation numbers. The USDIndex is up 0.2% and at 109.17, while Sterling continues to sell off. GBPUSD slumped below 1.2300 on budget angst and as the 10-year Gilt spiked. EURUSD slumped to 1.0273 after weak Eurozone data. USOIL is slightly down on the day and at USD 73.24 per barrel. Gold is unchanged at USD 2662.44 per ounce. Always trade with strict risk management. Your capital is the single most important aspect of your trading business. Please note that times displayed based on local time zone and are from time of writing this report. Click HERE to access the full HFM Economic calendar. Want to learn to trade and analyse the markets? Join our webinars and get analysis and trading ideas combined with better understanding of how markets work. Click HERE to register for FREE! Click HERE to READ more Market news. Andria Pichidi HFMarkets Disclaimer: This material is provided as a general marketing communication for information purposes only and does not constitute an independent investment research. Nothing in this communication contains, or should be considered as containing, an investment advice or an investment recommendation or a solicitation for the purpose of buying or selling of any financial instrument. All information provided is gathered from reputable sources and any information containing an indication of past performance is not a guarantee or reliable indicator of future performance. Users acknowledge that any investment in Leveraged Products is characterized by a certain degree of uncertainty and that any investment of this nature involves a high level of risk for which the users are solely responsible and liable. We assume no liability for any loss arising from any investment made based on the information provided in this communication. This communication must not be reproduced or further distributed without our prior written permission.
    • NTRA Natera stock, nice rally with a top of range breakout at https://stockconsultant.com/?NTRA
×
×
  • Create New...

Important Information

By using this site, you agree to our Terms of Use.