Neural Networks and Genetic Optimizers

[Ranger 10]

I attached a zip file with screenshots showing how I setup a model for CAT using 60 Minute data. I made some notes so that interested forum members can follow along.

 

Note:

 

a) I didn't include any other timeseries in this model. That will be the next step ie a SMA model for S&P; DJI etc.

 

b) I checked with Tradecision and they said that pre-processing on minute data is probably unnecessary. I need to check my data set and make sure that it's statistically sound but I'll leave that as a future excercise following review of model output.

 

c) Tradecision has implemented a filter for outliers that will help normalize data but the beta version has crashed on me a couple of times ... in all fairness to Tradecision, I got this hot off the press. I sent them error reports.

 

d) The screenshots show a henious estimated time for completion. The estimated time at posting is now 1.5 hrs and I'm on stage 2. If it completes soon I'll post the results.

 

e) I'll likely need to improve the directional accuracy .... let's see what happens.

 

Any comments especially on the inputs would be really helpful.

 

RANGER

CAT Model NN.zip

[Ranger 10]

Here are the results! Conclusions and improvements to follow.

 

RANGER

CAT_Model_1_Results.zip

[Ranger 10]

Attached find a predictive MACD developed using Tradecision. For the most part the MACD is triggering approximately 1-3 intervals before the standard MACD(can post the full time series).

 

Directional accuracy and error of model shown.

 

RANGER

MACD.pdf

[waveslider 10]

so the "test data range results" are out of sample?

[Ranger 10]

so the "test data range results" are out of sample?

 

Correct Waveslider!

 

RANGER

[natedredd10 10]

The next step would be to examine the equity curves of these systems as a time series through the lens of a Neural Net. If the neural net indicated that the equity curve would rise for one of the systems, might would take the trade. In a sense it's just a confirmation of your signal. One could take 100 various systems and examine the NN consensus, etc.

 

Make sense? I'd be surprised if this isn't being done somewhere..

 

One thing I would question is if you can look for the most robust system as opposed to the max profit. I don't see how you get around that the system with the max profit is probably the most fragile unless you expect the markets distribution to stop moving.

[BlowFish 10]

Actually there is another intervening (crucial) step and that is to concider how you might take potential trades based on what the forecast MACD is. The output from this would presumably go into your equity net?

[Ranger 10]

Actually there is another intervening (crucial) step and that is to concider how you might take potential trades based on what the forecast MACD is. The output from this would presumably go into your equity net?

 

Thanks BlowFish ...

 

I've heard of MACD strategies but I don't have one that works so I intend to use this as a permissive that I'll poll as part of an overall autoexecution strategy.

 

I started modeling a Stochastic using similar inputs but the results are partly heinous and partly unbelievable. I queried Tradecision on the matter and hope to hear back today.

 

One of things that I'd like to do is filter the noise found in intraday price data and use the output for the neural models. I've been reading some research papers; investigating websites etc and found some resources but I haven't found a starting point for easy language. I seems like I might have to generate my own wavelet filter .... talk about inventing the wheel on some of this stuff .... :crap:

Edited July 22, 2010 by Ranger
correct spelling errors

[Ranger 10]

Urma

 

I think that in one of your posts you mentioned that you use JMA for smoothing. Do you use this MA to filter the data of your indicators before modeling ie volume or others for your regression models?

 

Also, do you have an opinion on the use of wavelet algorithms for smoothing data sets for use in NN?

 

Trust all is well

 

RANGER

[UrmaBlume 10]

UrmaI think that in one of your posts you mentioned that you use JMA for smoothing. Do you use this MA to filter the data of your indicators before modeling ie volume or others for your regression models?Also, do you have an opinion on the use of wavelet algorithms for smoothing data sets for use in NN?Trust all is well RANGER

 

A year or two ago we developed our own smoothing which tests much closer to the mean than JMA. It involves serial exponential smoothing, much like the Hull, and we have not used the JMA since. We also use a regression curve for smoothing.

 

We have fooled around with wave & wavelet utilities in the past and even bought the Gista Caterpillar from the Russians but currently use none of them for anything.

 

 

cheers

[Ranger 10]

Hi Urma

 

Perhaps my questions was unclear. Do you smooth your price and volume data prior to regression modeling?

 

I compared JMA a version of Hull and found them comparable. My partner and I compared JMA; Hull and Haar Wavelet MAs and found Wavelet more responsive. The Wavelet MA seems to make the curves with little under/overshoot.

 

Thanks

 

RANGER

[Shell128 10]

Ranger,

You might be interested in a book called Conceptual Wavelets by Dr. Lee Fugal.

[dbg 10]

There is a free open-source application called RapidMiner that's powerful. It incorporates the WEKA learning library and libSVM. Also has parameter optimization built-in. I actually use this platform for all my trade system development and data mining. The only thing it is missing is R integration which was present in version 4.x and in development for 5.x.

 

Shell128, How do you use Rapidminer to trade....or more specifically how to you get it to do predictions?

 

Thanks in advance.

 

DBG

[Shell128 10]

Shell128, How do you use Rapidminer to trade....or more specifically how to you get it to do predictions?

 

Thanks in advance.

 

DBG

 

Each pre-processing step is an operator. I made my own operators, not too hard to do. I recommend that you buy the RM developer's guide, shows you how to do it. Each operator outputs an ExampleSet which can be input to any of the hundreds (now thousands thanks to R integration) of learning algorithms and statistical processing routines available in RM. However, I you will need to hack some of the classes to make it run real-time prediction. For example, one of my classes loads incoming data an outputs an example set. I then added update listeners to the RM's ExampleSet class as well as all my custom operators. If you're starting from scratch, I don't recommend doing this. RM is a great platform for system development, but when you get to the prediction stage, you're probably better off making a stand-alone system which operates outside of RM. Updating ExampleSets is not most efficient for a real-time application. But I have found RM productive for prototyping.