The Non-Optimisation Myth

[equtrader 10]

Everything you do can be considered optimization one way or the other. The important question is whether the optimization process results in different system dynamics when it varies the parameters. Different dynamics can be assessed by different trade distributions. In that case the system is data-mining and it is subject to data-mining bias, i.e. the uncovering of spurious hypotheses. So this should be a criterion. If this criterion holds then you should take the final system and see if it performs well in several unrelated markets. If it does not, you have wasted you time and possibly money. The following posts in price action blog provide some good initial background but I think one must go even further than that:

 

Curve-fitting and Optimization | Price Action Lab Blog

 

Fooled by Randomness Through Selection Bias | Price Action Lab Blog

[onesmith 10]

I second your recommendation to study what Michal Harris of Price Action Lab's has disclosed. Everyone (serious) like all of you posting to this thread, should understand the advantages and disadvantes of artificial intelligence. Some of you should specialize in it and take it beyond where it is, to something that can benefit us as a species.

 

Everything you do can be considered optimization one way or the other. The important question is whether the optimization process results in different system dynamics when it varies the parameters. Different dynamics can be assessed by different trade distributions. In that case the system is data-mining and it is subject to data-mining bias, i.e. the uncovering of spurious hypotheses. So this should be a criterion. If this criterion holds then you should take the final system and see if it performs well in several unrelated markets. If it does not, you have wasted you time and possibly money. The following posts in price action blog provide some good initial background but I think one must go even further than that:

 

Curve-fitting and Optimization | Price Action Lab Blog

 

Fooled by Randomness Through Selection Bias | Price Action Lab Blog

[SIUYA 11]

I had read the Price action Blog previously and it is definitely highly recommended.

One of his conclusions sums it up nicely for the Q of this thread.....

 

"As a conclusion we can state that the issue is not whether a system is optimized, because all systems are in one way or another, but to what degree optimization impacts the probability that the system will fail in the future due to its nature."

[zdo 20]

For my own 'optimizationing' of the types of systems being discussed in this thread , I would amend

"As a conclusion we can state that the issue is not whether a system is optimized, because all systems are in one way or another, but to what degree optimization impacts the probability that the system will fail in the future due to its nature."

 

to

 

"As a conclusion ...the issue is not whether a system is optimized, because all systems are in one way or another, but to what degree optimization impacts the probability that the system will degrade in the future due to its nature."

 

ie...to "failure" is too kiss... if the parameters are optimizable for a system, get busy on them to the left or the right as soon as it degrades

[equtrader 10]

For my own 'optimizationing' of the types of systems being discussed in this thread , I would amend

"As a conclusion we can state that the issue is not whether a system is optimized, because all systems are in one way or another, but to what degree optimization impacts the probability that the system will fail in the future due to its nature."

 

to

 

"As a conclusion ...the issue is not whether a system is optimized, because all systems are in one way or another, but to what degree optimization impacts the probability that the system will degrade in the future due to its nature."

 

ie...to "failure" is too kiss... if the parameters are optimizable for a system, get busy on them to the left or the right as soon as it degrades

 

Good suggestion. It is important to always be alert to prevent failure by constantly analyzing and upgrading systems. It is a horrible mistake to assume that a system should not be changed after it is deployed.

 

This is another good one by Harris where he argues that certain types of exits, like trailing stops, may produce curve-fitted systems:

 

Trailing Stops and Curve-Fitting in Trading System Development | Price Action Lab Blog

[zdo 20]

thx Equtrader...

 

One reason that I do not use this type of stop when developing trading systems and in algorithms I have developed for machine design of trading systems is that it renders the entry signals irrelevant in most cases. However, the robustness of the entry signals and their predictive ability under the majority of possible market conditions is what secures longer-term positive performance for trading system. Trailing stops can be used during actual trading to enhance profits and cut losses. One way to check whether signals generated by a trading system have predictive ability is to set a small percent or dollar stop. If the profit factor when the stops are set like that is not sufficiently high, you are probably dealing with a random entry generator.

The risk of curve-fitting is also high when using indicators to exit signals although entry signals may not be rendered irrelevant. But this is an even more interesting and complex case that will be the subject of another post.

from the closing of the article you cited

 

Haven’t posted these particular yada yadas in a while so…

>It is ALWAYS a “3 body problem” (anecdotally borrowing the phrase to illustrate the integral interplay, not same as it’s used in real physics, btw) ---

entry, exit, stop – never isolated

"optimized" close to, but not at, optimal - all at once…

 

>In system and automation design, any tweak is actually not a tweak, it is a whole distinct new system

 

If he persists, I predict before he’s done he will

1) retract that across the board “the robustness of the entry signals and their predictive ability under the majority of possible market conditions is what secures longer-term positive performance for trading system”

You can make such generalities about certain types of systems, but you can’t make such generalities about all systems… ie that is a good generality for systems exploiting limit cycles / oscillations of markets. It is not a good generality for systems exploiting ‘trend’ where “entry signals [can be rendered more] irrelevant” and

2) he will also conclude that “using indicators to exit” really isn’t an “even more and complex case” ...

 

Not a criticism at all… I periodically check in with this blog, learn a lot, and admire his development