Jump to content

Welcome to the new Traders Laboratory! Please bear with us as we finish the migration over the next few days. If you find any issues, want to leave feedback, get in touch with us, or offer suggestions please post to the Support forum here.

  • Welcome Guests

    Welcome. You are currently viewing the forum as a guest which does not give you access to all the great features at Traders Laboratory such as interacting with members, access to all forums, downloading attachments, and eligibility to win free giveaways. Registration is fast, simple and absolutely free. Create a FREE Traders Laboratory account here.

cunparis

Avoiding Curve Fitting

Recommended Posts

I have developed 3 indicators that each test profitably. I've determined the optimal parameters by optimization (periods, thresholds, etc.). I do not expect to get the same results in the future, but I prefer to use the optimized values rather than some arbitrary values.

 

My question is this: I'm now working on combining these 3 into one signal (short, flat, long). I've tried two different approaches to do this:

 

1 - I use the optimal parameters that I determined on each indicator individually

 

2 - I re-optimized all parameters together.

 

#1 seems to be more realistic, with the acknowledgment that the performance will not be the same as the backtests, due to the performance of each system not being the same. This I know. So the final results will probably not be as good.

 

#2 - Seems to be more optimal, with an even stronger acknowledgment that the results will not be as good as the backtest. However there is a greater risk of curve fitting due to the increased rules and degrees of freedom. In defense of the optimization I will say that lots of attempts produced unacceptable results so I believe that if optimization finds something good say PF > 3.0 then it's very likely to be positive in forward testing even though the PF will most likely be less.

 

I'm curious what people think about these two approaches. I am currently forward testing both #1 & #2 but since they trade on daily charts and not very often, it will take a while to have something meaningful.

 

I've developed systems that have held up and systems that have fallen apart. I understand the limitations of backtesting and automation. So I prefer not to debate that but focus on which approach would be more optimal (and not necessarily more realistic).

Share this post


Link to post
Share on other sites
I have developed 3 indicators that each test profitably. I've determined the optimal parameters by optimization (periods, thresholds, etc.). I do not expect to get the same results in the future, but I prefer to use the optimized values rather than some arbitrary values.

 

My question is this: I'm now working on combining these 3 into one signal (short, flat, long). I've tried two different approaches to do this:

 

1 - I use the optimal parameters that I determined on each indicator individually

 

2 - I re-optimized all parameters together.

 

#1 seems to be more realistic, with the acknowledgment that the performance will not be the same as the backtests, due to the performance of each system not being the same. This I know. So the final results will probably not be as good.

 

#2 - Seems to be more optimal, with an even stronger acknowledgment that the results will not be as good as the backtest. However there is a greater risk of curve fitting due to the increased rules and degrees of freedom. In defense of the optimization I will say that lots of attempts produced unacceptable results so I believe that if optimization finds something good say PF > 3.0 then it's very likely to be positive in forward testing even though the PF will most likely be less.

 

I'm curious what people think about these two approaches. I am currently forward testing both #1 & #2 but since they trade on daily charts and not very often, it will take a while to have something meaningful.

 

I've developed systems that have held up and systems that have fallen apart. I understand the limitations of backtesting and automation. So I prefer not to debate that but focus on which approach would be more optimal (and not necessarily more realistic).

 

I use step 1. Don't optimize together. I always test different "indicators" or rules in isolation then I bring them together one at time. If one rule does not contribute to making the system better I don't re-optimize it - I dump it.

 

Good systems, in my humble opinion, only need a 2-3 basic rules. Your key trading concept shouldd work well without much, if any optimization.

Share this post


Link to post
Share on other sites
I use step 1. Don't optimize together. I always test different "indicators" or rules in isolation then I bring them together one at time. If one rule does not contribute to making the system better I don't re-optimize it - I dump it.

 

Good systems, in my humble opinion, only need a 2-3 basic rules. Your key trading concept shouldd work well without much, if any optimization.

 

Thanks for the feedback. I did a lot of forward testing this weekend. What I found was that performance going forward was pretty good until the past few years. Then even if I reoptimized it didn't walk forward well. i think it's due to changing from bull to bear and from the increased volatility. At this point I have doubts about the predictive capability. I'm going to give it a few more goes.

 

I'm using a moving average difference for the main signal, so that's 2 rules. Then I added an upper & lower threshold, that's 2 more. I think that's too many. The reason is in some of the optimizations (3-4 years, 100+ trades) I'd have moving averages like 5,6 and other times 7,5. This didn't make sense because having a faster average slower than the slow (inverting them) would effectively inverse all the signals. So I got suspicious.

 

I think I need to find a way to make an indicator without using 2 moving averages. It's too much curve fitting I think.

 

any ideas?

Share this post


Link to post
Share on other sites
Thanks for the feedback. I did a lot of forward testing this weekend. What I found was that performance going forward was pretty good until the past few years. Then even if I reoptimized it didn't walk forward well. i think it's due to changing from bull to bear and from the increased volatility. At this point I have doubts about the predictive capability. I'm going to give it a few more goes.

 

I'm using a moving average difference for the main signal, so that's 2 rules. Then I added an upper & lower threshold, that's 2 more. I think that's too many. The reason is in some of the optimizations (3-4 years, 100+ trades) I'd have moving averages like 5,6 and other times 7,5. This didn't make sense because having a faster average slower than the slow (inverting them) would effectively inverse all the signals. So I got suspicious.

 

I think I need to find a way to make an indicator without using 2 moving averages. It's too much curve fitting I think.

 

any ideas?

 

I do have a lot of ideas. :) I wish I had more time to experiment and build systems. But let me say this…

 

In my limited experience attempting to create a trading system with moving averages is very difficult. You can make strategies from basic indicators, but it's hard to do and can result in curve fitting. Try using common indicators in a different way - ways in which most people don't use them. For example, RSI is often used to highlight overbought and oversold conditions. Try using it as a trend indicator. This is just an example.

 

Price patterns are another way to go. Price breaking out from trading ranges or price behavior around opening day gaps are examples of trading without indicators.

 

In short, to make money in automated systems you are either 1) trend following or 2) trend fading. Decide what you want to do and focus on markets and market sessions that are favorable to those conditions. Your trading system does not need to trade all day or even every day. My best system trades about once a month as it fades extreme moves on a 5-minute chart. So be picky.

 

I think it's interesting to note that you stated " This didn't make sense because having a faster average slower than the slow (inverting them) would effectively inverse all the signals. So I got suspicious. "

 

Sounds like fading your original signal is a better idea. In other words, using your moving averages in a method that is "unusual" may produce better results than your original concept.

Share this post


Link to post
Share on other sites

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.

Guest
Reply to this topic...

×   Pasted as rich text.   Paste as plain text instead

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.


  • Topics

  • Posts

    • Back in the early 2000s, Netflix mailed DVDs to subscribers.   It wasn’t sexy—but it was smart. No late fees. No driving to Blockbuster.   People subscribed because they were lazy. Investors bought the stock because they realized everyone else is lazy too.   Those who saw the future in that red envelope? They could’ve caught a 10,000%+ move.   Another story…   Back in the mid-2000s, Amazon launched Prime.   It wasn’t flashy—but it was fast.   Free two-day shipping. No minimums. No hassle.   People subscribed because they were impatient. Investors bought the stock because they realized everyone hates waiting.   Those who saw the future in that speedy little yellow button? They could’ve caught another 10,000%+ move.   Finally…   Back in 2011, Bitcoin was trading under $10.   It wasn’t regulated—but it worked.   No bank. No middleman. Just wallet to wallet.   People used it to send money. Investors bought it because they saw the potential.   Those who saw something glimmering in that strange orange coin? They could’ve caught a 100,000%+ move.   The people who made those calls weren’t fortune tellers. They just noticed something simple before others did.   A better way. A quiet shift. A small edge. An asymmetric bet.   The red envelope fixed late fees. The yellow button fixed waiting. The orange coin gave billions a choice.   Of course, these types of gains are rare. And they happen only once in a blue moon. That’s exactly why it’s important to notice when the conditions start to look familiar.   Not after the move. Not once it's on CNBC. But in the quiet build-up— before the surface breaks.   Enter the Blue Button Please read more here: https://altucherconfidential.com/posts/netflix-amazon-bitcoin-blue  Profits from free accurate cryptos signals: https://www.predictmag.com/ 
    • What These Attacks Look Like There are several ways you could get hacked. And the threats compound by the day.   Here’s a quick rundown:   Phishing: Fake emails from your “bank.” Click the link, give your password—game over.   Ransomware: Malware that locks your files and demands crypto. Pay up, or it’s gone.   DDoS: Overwhelm a website with traffic until it crashes. Like 10,000 bots blocking the door. Often used by nations.   Man-in-the-Middle: Hackers intercept your messages on public WiFi and read or change them.   Social Engineering: Hackers pose as IT or drop infected USB drives labeled “Payroll.”   You don’t need to be “important” to be a target.   You just need to be online.   What You Can Do (Without Buying a Bunker) You don’t have to be tech-savvy.   You just need to stop being low-hanging fruit.   Here’s how:   Use a YubiKey (physical passkey device) or Authenticator app – Ditch text message 2FA. SIM swaps are real. Hackers often have people on the inside at telecom companies.   Use a password manager (with Yubikey) – One unique password per account. Stop using your dog’s name.   Update your devices – Those annoying updates patch real security holes. Use them.   Back up your files – If ransomware hits, you don’t want your important documents held hostage.   Avoid public WiFi for sensitive stuff – Or use a VPN.   Think before you click – Emails that feel “urgent” are often fake. Go to the websites manually for confirmation.   Consider Starlink in case the internet goes down – I think it’s time for me to make the leap. Don’t Panic. Prepare. (Then Invest.)   I spent an hour in that basement bar reading about cyberattacks—and watching real-world systems fall apart like dominos.   The internet going down used to be an inconvenience. Now, it’s a warning.   Cyberwar isn’t coming. It’s here.   And the next time your internet goes out, it might not just be your router.   Don’t panic. Prepare.   And maybe keep a backup plan in your back pocket. Like a local basement bar with good bourbon—and working WiFi.   As usual, we’re on the lookout for more opportunities in cybersecurity. Stay tuned.   Author: Chris Campbell (AltucherConfidential) Profits from free accurate cryptos signals: https://www.predictmag.com/   
    • DUMBSHELL:  re the automation of corruption ---  200,000 "Science Papers" in academic journal database PubMed may have been AI-generated with errors, hallucinations and false sourcing 
    • Does any crypto exchanges get banned in your country? How's about other as Bybit, Kraken, MEXC, OKX?
    • Does any crypto exchanges get banned in your country? How's about other as Bybit, Kraken, MEXC, OKX?
×
×
  • Create New...

Important Information

By using this site, you agree to our Terms of Use.