Neural Networks and Genetic Optimizers

[Ranger 10]

Hello

 

I searched the forum and found limited information on the topics of Neural Networks and Genetic Optimizers. My friend and I have started reviewing this topic with the objective of selecting a software; designing signals for active trading. We both program; we're both engineers. In the past, I studied linear programming and operations research and I also have advanced mathematics at the masters level.

 

If the topic interests you; you're a day trader or very active trader; and you think that you can contribute at a higher level to our team, we'd love the hear from you.

 

Also, any forum members with constructive comments, we'd love to hear from you too.

 

Happy Trading

 

RANGER

[waveslider 10]

I too am looking at NN technology. I trade end of day successfully using proprietary algorithms. My systems are somewhat adaptive, but I am looking for multiple and diverse levels of confirmations. I don't know if NN is a path I should be following.

Can anyone recommend a good starting point?

 

thanks

[natedredd10 10]

This book is the only one I've found that I'm starting to actually make some sense of NN.

Amazon.com: Machine Learning: An Algorithmic Perspective (Chapman & Hall/Crc Machine Learning & Pattern Recognition)…

 

Unfortunately, It doesn't get as deep into other machine learning algorithms as it does NN.

 

Ranger, you should check out hidden markov models. I don't know the math to make heads or tails of them but Rentech hired the Baum in this algo awhile back:

http://en.wikipedia.org/wiki/Baum-Welch_algorithm

 

As far as other books there are a lot of machine learning bioinformatics books that I think are easier to follow than straight ML textbooks. Also has the nice feature that the problem domain is a bit similar as far as large data sets and not totally sure what it is you are looking for.

Edited July 1, 2010 by natedredd10

[MseTrap 10]

IMO the Marsland machine learning book linked above is pretty awful. Just skip the book and download the source code from the website. Although it's fairly obfuscated python.

[waveslider 10]

As far as other books there are a lot of machine learning bioinformatics books that I think are easier to follow than straight ML textbooks.

 

could you recommend some which are easier to follow? I do not have a higher mathematics background and I fell asleep in class most of calculus.

 

thanks

[Ranger 10]

Happy to see this posting getting some interest and not because it's mine.

 

@natedredd10 - thanks for the posting and links. I'll check them.

 

@waveslider - NN are intended to be predictive. So, let's say you build a NN based on a few momentum indicators; and another based on few Osc; and another based on Price Action. You can then create a committee of those nueral networks and base your trade signals on your criteria - short answer YES.

 

I just started playing with Tradecision. Why Tradecision - most of the NN packages are around 2.5K. TD Ameritrade offers Tradecision lease 109/month as an add-on. I thought this was a reasonable price for a training tool and possibly more.

 

If I get additional interest in this, I don't mind posting my progress but it takes a few hours to make screenshots and document, so there needs to be an interest in the topic. Skype screenshare session is also possible.

 

pages 1-2: show performance on TRAINING DATA; TEST RANGE DATA. (Particulars of model are unimportant but for example this could be MA x Over NN.

 

Note the performance between the TRAINING DATA & TEST RANGE DATA.

 

pages 3-4: Shows predictive line (page 4 is close-up scale).

 

pages 5-6: NN Signals applied to chart

 

pages 7 - 8: simulation applied to all data; equity curve and results.

 

Apologize for the quality of the documentation!

 

 

 

RANGER

Tradecision.pdf

Edited July 1, 2010 by Ranger
Where's the Attachment

[BlowFish 10]

I'd say just get some tools and start 'playing' you don't really need to know too much about exactly how neural networks work the real skill (and art) comes in picking what inputs to feed in and what you look for in the output. (predicting price, particularly a long way in to the future is about as tough a task as one could give a net). Genetic algorithms are a much more straight forward beast, they are pretty good at determining suitable inputs or ranking outputs of nets incidently. A lot of the heartache is in pre and post processing the data. In fact as far as I can tell that is the number one issue, normalising the inputs appropriately and trying to predict sensible things. I am not sure it's stuff you can learn from a book (though I would be first in line for a book on practical applications, tips & tricks) you just need to try and see what kinda 'works' and what does not.

 

I have had a copy of NeuroShell Daytrader for years (I used to be a bit of a software junky). If I am honest I never got past the 'dabbling with intent' phase though I learnt much more through that than anything I read. I must say I was impressed by the vast array of tools in a single package (some are pay extra modules but the basic package has loads). It has a couple of types of NN's, genetic optimisation, a vast amount of maths stuff, regular TA stuff, plus various 'exotic' bits and pieces. Edit: unlike Ranger I prefer to buy an out and out license.

 

If you want to place an order you learn how to use your brokers platform rather than learn the FIX protocol. Of course with the brokers platform it has a pretty specific task so it is much easier to see that it 'works'.

 

Does anyone have recommendations for resources that cover the practical application of these sorts of tool? I do have a few older tomes though not sure any are worthy of an out and out recommendation.

[natedredd10 10]

could you recommend some which are easier to follow? I do not have a higher mathematics background and I fell asleep in class most of calculus.

 

thanks

 

That book I posted above is by the far the easiest to follow I've found, its kind of a machine learning for dummies book which may be why MseTrap thinks its crap.

I don't have the background to gain much of anything from the most highly recommended books in the field and I do think you have to be open to the idea that if you don't have the background this may be a huge waste of time.

I also have Bioinformatics: The Machine Learning Approach by Pierre Baldi..Maybe the only thing I find easier about it though is that I find bioinformatics interesting in and of itself so the concepts are not so abstract as with a straight ML textbook.

I think without the background its important to find text that give real world examples on real datasets that interest you since there is no book on ML with financial datasets at its base that I know of. A straight up ML textbook is as much fun to read as your highschool calc book.

[natedredd10 10]

I'd say just get some tools and start 'playing' you don't really need to know too much about exactly how neural networks work the real skill (and art) comes in picking what inputs to feed in and what you look for in the output. (predicting price, particularly a long way in to the future is about as tough a task as one could give a net).

 

The problem to me with retail software in this area is its using some strange assumptions as far as feature extraction. Its easy to argue that for ES, price is a feature, volume is a feature, cash index price is a feature of, the volume of GS is a feature of ES.

I don't see how you can make an argument though that the RSI, MA, slow stochastic of ES price is a feature...Its making a huge assumption there is predictive power in these things to start with and that they are something other than simple price summaries.

To me the whole interesting thing in this area is in ditching these old models of price prediction and going straight to the factors that we know matter and creating new models.

[Ranger 10]

Blowfish - Glad you joined this post.

 

I picked Tradecision soley because it was offered by TD Ameritrade. As far as I can tell, no lead applications offer a 30 trial or month to month option so if you select a software at a price tag of 2K, it makes sense that it meets your needs.

 

Your outline on the NN is 100% accurate and I really don't care too much about the mathematics behind the model. Choosing good inputs and correct interrpretation of results and application of the final version to the trading world are the real hurdles.

 

Right now, I'm just toying around the software. Making a model based on inputs & time series(ie S&P 500 Index); optimizing inputs; creating the NN; applying to simulation.

 

The Improvian Language used by Tradecision is really simple; like easy language so it's simple to create functions; indicators etc.

 

RANGER

[UrmaBlume 10]

The most effective models we have built come from a cascade of technologies. An example of such a cascade would be a model developed by first optimizing input preprocessing and input selection with genetic algorithms.

 

Then input data is used to develop a model using technologies that develop their models through what is called feed back mathematics, machine learning. Neural Networks and certain regression packages are used for this phase. Genetic algorithms can be used in this process to select and optimize such parameters and the number of nodes in middle layers of NNs and the transfer fuctions that weigh and pass the data between them. Our most used inter-nodal transfer function is the long-tailed sigmoid.

 

After the model is developed its output can be further optimized using decision trees or rules generators.

 

This site is a good place to find articles on and vedors of such tools. Every year that have a competition that ranks tools according to class.

 

For genetic optimizations we have written our own.

 

For the modeling process we like MARS® (Multivariate Adaptive Regression Splines) from Salford Systems in San Diego. We have built an application that converts the final MARS function to Trade Station's Easy Language so that we can deploy these models so that they can make their predictions in real time with just a cut and paste.

 

For rules generation we use CART, again from Salford Systems described on their website as "a robust, easy-to-use decision tree that automatically sifts large, complex databases, searching for and isolating significant patterns and relationships." We also use WizWhy from WizSoft. WizWhy cost around $4k and has served us very well.

 

Our process is described here which is an update and rewrite of this thread here on TL.

 

Good Luck with your project

 

cheers

UB

[natedredd10 10]

Does anyone have recommendations for resources that cover the practical application of these sorts of tool?

 

Here are applications I've found that are market related, but nothing in the way of theory/concepts:

SVM on RSI

Support Vector Machine RSI System Quantum Financier

 

Decision tree bagging system on GLD

Max Dama on Automated Trading: Decision Tree Bagging System (R code)

 

this guy has a bunch of different algos on market data, some with code.

Intelligent Trading

[BlowFish 10]

The problem to me with retail software in this area is its using some strange assumptions as far as feature extraction. Its easy to argue that for ES, price is a feature, volume is a feature, cash index price is a feature of, the volume of GS is a feature of ES.

I don't see how you can make an argument though that the RSI, MA, slow stochastic of ES price is a feature...Its making a huge assumption there is predictive power in these things to start with and that they are something other than simple price summaries.

To me the whole interesting thing in this area is in ditching these old models of price prediction and going straight to the factors that we know matter and creating new models.

 

Quite so! Feeding an RSI (of price) a MACD (of price) and a CCI (of price) into a net will not end well! Using a normalised MA to pre process price dat before using it as an input might be a step in the right direction.

 

Ranger keep us posted on how it goes.

 

Incidentally user forums and blogs for some of these packages are likely to be as good a place for practical advice on how to get the best from these sorts of tools. Most of the issues are not unique to financial time series so no reason not to cast your net wider (if you pardon the pun).

 

This is not a recommendation and it is years since I read it but I do recall getting a bit out of Amazon.com: Neural, Novel & Hybrid Algorithms for Time Series Prediction (9780471130413): Timothy Masters: Books (pretty sure I didn't pay 355 bucks for it though!).

[Ranger 10]

Hi Urma

 

I reviewed the resources provided by you in your posting; some are very interesting including the article "Is MARS better than Neural Networks". The article was within my technical range of understanding and I enjoyed the conclusions which are straight foward.

 

Why did you select MARS vs NN?

 

It appears from your work that you use Tradestation. Perhaps the justification for using MARS vs a NN is because the output is a linear approximation that can used in TS signals rather than the blackbox? It appears that model development using MARS plainly requires more work and expertise - advanced users only? Please comment.

 

On your website I read something about normalization of inputs; I rechecked the site and couldn't find the sentence again but anyway, it would be interesting to understand how you normalize inputs such as volume; balance of trade. Can you share this information with us?

 

@BLOWFISH - same comment about normalizing goes to you.

 

Thanks all for your VERY VALUABLE input.

[UrmaBlume 10]

Hi UrmaI reviewed the resources provided by you in your posting; some are very interesting including the article "Is MARS better than Neural Networks". The article was within my technical range of understanding and I enjoyed the conclusions which are straight foward. Why did you select MARS vs NN?

It appears from your work that you use Tradestation. Perhaps the justification for using MARS vs a NN is because the output is a linear approximation that can used in TS signals rather than the blackbox? It appears that model development using MARS plainly requires more work and expertise - advanced users only? Please comment.

On your website I read something about normalization of inputs; I rechecked the site and couldn't find the sentence again but anyway, it would be interesting to understand how you normalize inputs such as volume; balance of trade. Can you share this information with us?@BLOWFISH - same comment about normalizing goes to you.Thanks all for your VERY VALUABLE input.

 

Ranger,

 

Thank you for the kind words.

 

I am constantly amazed at what a small percentage of supposedly educated and power traders have any meaningful experience with these " intelligent power tools" of data processing.

 

As to the selection of MARS vs NN, we are versed in both but with out level of experience find MARS easier to use and the models easier to deploy. Of note is that the selection of NN or MARS is not nearly so important as the engineering of the preprocessing of the inputs and targets.

 

Of note also is that these projects involve several technologies other than just the tools that build the models. We use genetic survival of the fittest code that we have written in house to aid in input selction and preprocessing as well as decision trees and rules generators to help us get more value from the model's output.

 

As to preprocessing and normalization, depending on the project, the input data must be scaled to the targets, outliers in training data considered and sometimes removed, input data must be carefully weighed and selected based on its relevance to the target. As to normalization we have several criteria. For volume and volatility based inputs one our more important considerations is that the data be normalized for time of day.

 

This link describes an application that demonstrates both preprocessing and the ouput from intelligent agents. The price specific Trade Points that you see on the app are the output of such agents while such indicators as the percentage of commercial presence and commercial bias are time of day normalized inputs.

 

cheers

 

UB

[waveslider 10]

Urmablume,

Thank you for weighing in and we appreciate any examples of practical applications of NN with existing "non-forecasting" studies.

Also, are there any retail products that you would prefer for NN processing?

[Ranger 10]

Found this link on pre-processing of data. Not much but it was more than I knew a few minutes ago.

 

Market Technologies - Preprocessing Data For Neural Networks

[Shell128 10]

There is a free open-source application called RapidMiner that's powerful. It incorporates the WEKA learning library and libSVM. Also has parameter optimization built-in. I actually use this platform for all my trade system development and data mining. The only thing it is missing is R integration which was present in version 4.x and in development for 5.x.

[waveslider 10]

I wonder if NN users apply their technology to basic systems performance (equity curves) instead of price time series.

 

The age old challenge is to apply a trending system to a trending market and fading system to a ranging market. The problem is of course when to apply which, and to know in advance that a market is likely to slip into a mean reversion or trending mode. Basic systems using archaic indicators can identify profitable zones in the proper circumstances, so why not apply Neural Nets to the equity curves of some of these systems to point out when it would be advantageous to act on their signals.

 

Anyone doing that?

[natedredd10 10]

I wonder if NN users apply their technology to basic systems performance (equity curves) instead of price time series.

 

Would a moving average be better applied to system performance than price time series?

The perceptron was "invented" in 1957 by Rosenblatt..

The fact most retail traders are ignorant of even the oldest and most basic machine learning algos should really not be a surprise since stander fair is a MA which although I don't know who thought of the idea first, its hard to see it getting much farther past Gauss in the late 18th century since the algo is so obvious and basic and its ubiquity is exactly because its old and it doesn't crush the performance of a 1984 apple 2e...

When you try to use 200 year old algorithms and fail it should come as no surprise...

As if no one between 1810 and now thought of it, but thats the heuristic in retail trading..

Crazy is the man who moves on to 50 year old algorithms like the perceptron and then finds out it doesn't work anymore being 50 years old...who would have thought...Better to go back to the 200yo algo says the retail moron since the 50yo algo didn't work...Even though rationally that makes zero sense...as if you can't figure out maybe both are simply outdated being decades and century old analysis techniques...Imagine as if the field of data analysis has actually PROGRESSED in the last 50 years...who would imagine.....

 

Ditmar, i see you taking that fraud ArturiusX to school on 2+2...

I would jump in but i don't want you searching on my microstakes poker play there i post and laughing at me with my username...

I don't know if you were trying to fish me out there with posting that Max Dama chart I posted here but that was funny how you crushed ArturiusX with it...Latency arb?? what?? hehe.

You should up the aggression on that tool.

I would love to play you 2/5 no limit 10 years from now in vegas, full ring, seated two off from my right since you have so many years on me.

Your boys should think about the poker equity calc software market...its a fucking joke.I had moved from stove to equilab for hand analysis but the moron just reversed his position that equilab was freeware and basically made anyone using equilab crash..Hes moving to commercial but the site is in german, i LOVED it because it would give you a flop equity quiz based off hole cards and a villain range..with this lovely turn/river visualisation..the download site was only in german, i'll buy it still but most won't...

Huge gap in a market that if you have what you say you do team wise can crush all on the cheap.

PM me if interested in my ideas there, I would advise simply for a lifetime license at this point and maybe negotiate for a few custom mods that aren't commercial.

Edited July 17, 2010 by natedredd10

[Ranger 10]

Hi natedredd10:

 

I was considering Waveslider's comments and I was going to ask him to clarify his posting by providing a simple example but the first thing that came to my mind was a MA.

 

@ Waveslider - is it possible to provide a simple example or further clarification?

 

@ Natedredd10 - Appreciate your Ted Kaczynski like posting - we get one a year at TL, right?

[natedredd10 10]

@ Natedredd10 - Appreciate your Ted Kaczynski like posting - we get one a year at TL, right?

 

Yea because what I posted should be compared to a lunatic who murdered people with bombs...

I'm just trying to lead you astray from the very simple to find ATM machine at the end of the rainbow that some person you don't even know is going to magically give you the map for out of the kindness of their heart...

As opposed to the fact that mining for gold is a very complicated, capital and intellectually intense business if you don't want to lose money and actually mine ore at a profit.

[Ranger 10]

Yea because what I posted should be compared to a lunatic who murdered people with bombs...

I'm just trying to lead you astray from the very simple to find ATM machine at the end of the rainbow that some person you don't even know is going to magically give you the map for out of the kindness of their heart...

As opposed to the fact that mining for gold is a very complicated, capital and intellectually intense business if you don't want to lose money and actually mine ore at a profit.

 

Thank you natedredd10 - no friend, the length of your post .... please no offense intended.

 

You have so many useful posts and comments + you have spirit ... so thanks. I appreciate your guidance; read this wonderful poem by Robert Frost - he seems like a breakout trader.

 

 

Road Not Taken

by Robert Frost

 

Two roads diverged in a yellow wood,

And sorry I could not travel both

And be one traveler, long I stood

And looked down one as far as I could

To where it bent in the undergrowth;

Then took the other, as just as fair

And having perhaps the better claim,

Because it was grassy and wanted wear;

Though as for that, the passing there

Had worn them really about the same,

 

And both that morning equally lay

In leaves no step had trodden black

Oh, I kept the first for another day!

Yet knowing how way leads on to way,

I doubted if I should ever come back.

 

I shall be telling this with a sigh

Somewhere ages and ages hence:

two roads diverged in a wood, and I --

I took the one less traveled by,

And that has made all the difference.

[waveslider 10]

example:

 

Ranging system:

 

entry: Fade upper/ lower Bollinger bands

exit: close trade at mean

 

Trending system

 

entry: breakout after retracement to a trending moving average.

exit: trailing "x" avg true ranges

 

These "canned" systems certainly do not outperform on their own due to their under-performance in the unfavorable market mode.

 

The next step would be to examine the equity curves of these systems as a time series through the lens of a Neural Net. If the neural net indicated that the equity curve would rise for one of the systems, might would take the trade. In a sense it's just a confirmation of your signal. One could take 100 various systems and examine the NN consensus, etc.

 

Make sense? I'd be surprised if this isn't being done somewhere..

[dam5h 10]

I trade end of day successfully using proprietary algorithms.

 

could you recommend some which are easier to follow? I do not have a higher mathematics background and I fell asleep in class most of calculus.

 

thanks

 

say wha oh well ???