Filtering an Indicator

[Shell128 10]

I have developed an indicator that I would like to feed into my system. The indicator usually outputs small values 99.9% of the time. However, it's the large values that I care about. The indicator output looks like a series of spikes surrounded by noise. I would like to process this signal such that the background noise is filtered out and the spikes are less intense, but hold their value longer. One way to do this would be to high-pass filter or threshold the signal then apply an infinite impulse-response filter such as an exponential moving average to "hold" the spikes so that they stay around for longer. But I would like to preserve a bit more information. I would like the intensity of processed signal at some time to be proportionate to:

 

1) Intensity of last spike

2) Time last spike was observed (intensity would decay as time of last observation increases)

3) Spike intensity relative to background noise

 

Does anyone know of a good way (simple is better) to create such filter? I thought about taking the difference of two moving averages...one longer window moving average to capture the background noise and then subtract that from a shorter moving average to capture the spikes. Problem with that approach is that the filter would likely either allow too much of the noise to pass (pass-band too large) or would not hold the spike on the output long enough.

[Tams 20]

can you post some chart examples?

[Iris 10]

You could add another filter such as a DMI with ADX - when the ADX is over (for e.g.) 25 then a buy would be taken if the DMI+ is above the DMI- and a sell if the DMI- is above the DMI+ and your indicator is also signaling a large value. You could change the value of the length of the DMI/ADX to print faster such as 11 instead of the conventional 14. When the ADX is over 40 you might want to start looking to get out of your trade.

 

Another way you could consolidate your indicator is to change the way the bars are structured. For example, minute based charts are time-bound and a lot can happen before the bar is allowed to change by the setting you use so that a lot of action occurs within that bar's time-frame as things be-bop around. Tick charts (e.g. 500, 1000, 2000 ticks) show momentum when it occurs as the bars are filling up faster; and range bar charts are even better from this point of view e.g. setting on a range of 1.5 and an 8 tick interval - much more stable.

[Tradewinds 12]

I know what you are talking about, and I've programed code to deal with situations like this. What trading platform are you using? You could set up a counter to hold you signal as valid for a certain number of bars. So as soon as that big spike happens, it would show a signal for a certain number of bars, according to the counter. Count 5 bars after the signal. I don't know if this is a good solution for you, but I'm just trying to do some "brain storming" here.

If you want the most current spike to be proportionate to the last spike, you could hold the value of that last spike in some variable, and then make a comparison. So it seems like you want to know if the signal had a higher high, or a weaker high, etc. Maybe you need a way to keep track of the current highs and lows.

I'm just guessing here about what your indicator might look like in a lower study/sub graph. Sounds like you need someone who can make the programing happen. Is there a programing group you could join for the language that your platform uses? You could ask the customer service support for the trading platform you are using if there is any programing help "out there" somewhere.

Basically what you need is to find some appropriate ratio that condenses things down to make the data easier to "read". But the ratio may need to be dynamic, as the data is cycling through large ranges. This is really a math problem.

Let's see, I wonder if a fractional exponent would somehow help here. If you could make the small values bigger, and the big values smaller, it could smooth the data. So you are looking at a, sort of inverse relationship.

You could experiment with a spreadsheet, and feed different exponents into the formula to see how the numbers react. Put a bunch of numbers into a column in a spreadsheet that are representative of the data you might have. Then write a formula for the output in the next column and copy it down. Have the formula reference a cell that you could change, that would affect all the formulas.

If you looked at the graph of the curve for something like x^2, (x squared) it would be a curve getting steeper and steeper as "x" gets bigger 2x2, 4x4, 100x100.

But you want the big numbers to get smaller. When you get a real big spike, you want that number to be cut down to a smaller size. I think a fractional exponent will do that x^1/3.

Something to think about.

[Shell128 10]

can you post some chart examples?

 

It looks like this:

 

 

That's zoomed out so you can see the big spikes. When you zoom in, there's noise in-between the big spikes. Most of the chart is noise and the big spikes don't take up as much time as they appear on the plot above.

 

Iris and Tradewinds, I appreciate the comments. My current thoughts on this indicator is to use a modified exponential moving average. Change the weight factor for large changes in value upwards only. For example, if the past 1000 point average is 10 and the next value is 900, instead of using the minimum weight of 0.001, use a weight factor based on the ratio between new value and past average, like 0.4 to allow the indicator to ratchet up quickly. Then then change the weight factor back to some smaller value to allow the indicator to decay off with time. If I knew more about filters, I'd probably know of a standard filter that behaves in this way. Does anyone know?

 

Basically, I'm looking for a filter that reacts quickly to spikes in one direction, but then decays off gradually to the longer-term average value.

Edited December 30, 2010 by Shell128
Added last comment

[Tradewinds 12]

Thanks for the chart example. To me it looks like a volume histogram, with up volume positive and down volume negative. I've attached a .jpg file of a lower study that shows volume in green histogram bars, and then a red line. The red line has much lower highs. It fluctuates much less, but it still maintains it's respective ratios. If a spike is higher, then the red line is higher. All I did was raise that data to a fractional exponential power of 0.75. So instead of the power being squared, 2, or cubed, x^3, the exponent is less than 1. If you look at the attachment you will see that the red line does a very good job of compressing the spikes down and squeezing the data into a much tighter range.

[Iris 10]

his is John Ehler's generic filter for Tradestation - perhaps this might help:

 

{

Here is a generic EasyLanguage version of the nonlinear Ehlers filter, based on John Ehlers's article in this issue,

"Nonlinear Ehlers Filters."

This EasyLanguage filter accepts the coefficients statistic as an input.

In the following EasyLanguage indicator code, the Coef input is set to

AbsValue( MedianPrice - MedianPrice[5] ),

and the price input is set to MedianPrice. MedianPrice is a built-in function in EasyLanguage that

returns (H + L) / 2.

With these values for the inputs, the indicator is equivalent to the momentum-based Ehlers filter,

for which a specific EasyLanguage version has been developed by John Ehlers and is given at the top of this

Traders' Tips section.

 

To derive the distance coefficient Ehlers filter, which is also described in Ehlers's article,

the Coef input value in the following EasyLanguage can be replaced with DistanceSqrd( MedianPrice, 15 ).

DistanceSqrd is a custom function, and the code for that follows the indicator code.

}

 

inputs:

 

Price( MedianPrice ), //MedianPrice = (H+L)/2

Length( 15 ) ,

UseDistSquared ( False )

;

 

variables:

Num( 0 ),

SumCoef( 0 ),

Count( 0 ),

Filt( 0 ) ,

Coef ( 0 )

;

 

If UseDistSquared = False then begin

Coef = ( AbsValue( MedianPrice - MedianPrice[5] ) ) ;

end else

Coef = DistanceSqrd ( Price, Length ) ;//DSqrd + Square( Price - Price[LookBack] ) ;

 

Num = 0 ;

SumCoef = 0 ;

 

for Count = 0 to Length - 1

begin

Num = Num + Coef[Count] * Price[Count] ;

SumCoef = SumCoef + Coef[Count] ;

end ;

 

if SumCoef <> 0 then

Filt = Num / SumCoef ;

 

Plot1( Filt, "Ehlers" ) ;

 

 

 

Condition1 = FALSE ;

if Condition1 then

Alert( "" ) ;

[Tams 20]

It looks like this:

 

 

That's zoomed out so you can see the big spikes. When you zoom in, there's noise in-between the big spikes. Most of the chart is noise and the big spikes don't take up as much time as they appear on the plot above.

 

Iris and Tradewinds, I appreciate the comments. My current thoughts on this indicator is to use a modified exponential moving average. Change the weight factor for large changes in value upwards only. For example, if the past 1000 point average is 10 and the next value is 900, instead of using the minimum weight of 0.001, use a weight factor based on the ratio between new value and past average, like 0.4 to allow the indicator to ratchet up quickly. Then then change the weight factor back to some smaller value to allow the indicator to decay off with time. If I knew more about filters, I'd probably know of a standard filter that behaves in this way. Does anyone know?

 

Basically, I'm looking for a filter that reacts quickly to spikes in one direction, but then decays off gradually to the longer-term average value.

 

I would recommend reading up on Ehler.

He's got lots of filters.