Neural Networks: Myths and Reality by Dima Vonko

[Tradecision 10]

In this age of previously unheard of technological progress many technology-related things either come unnoticed as they appear or, vice versa, are vastly extolled and turned into totems that inevitably attract a following of ardent worshippers. If such a popular technology-related phenomenon can make a difference to your business, it is, sometimes, vital to learn about this phenomenon as much as possible before you start with it so that you know what to expect from the selected technology and what to beware of while using it. For a modern trader, one of such potentially important phenomena is neural nets.

 

So what is the neural network technology, what should and what shouldn't a trader expect from it if he selects to use it to achieve his trading goals?

 

Dispelling the Myths

 

Myth 1: Supernatural Intelligence

 

One of the commonly held misconceptions about neural networks is that they represent the kind of Artificial Intelligence which is it not only capable of fully replacing the human brain, but which also possesses some nearly supernatural power, thus enhancing the capacity and functions of this brain to the point when any kind of task can be solved almost miraculously without any effort on the part of the proud owner of this Holy Grail. This vulgar and frequently occurring notion of an undoubtedly valuable trading tool is dangerous in several ways. Let us see why.

 

First of all, neural networks are not all things to all traders. To understand what neural networks can and cannot do one should look into what they are. Neural networks are algorithms, loosely based on the nervous systems of humans and animals. Neural networks can detect and use to advantage the numerous interdependencies in data that are hidden from the human eye due to the data's complexity and non-linearity. This has been proven by the broad experience of neural networks' application in a wide array of industries, and trading is no exception. However, on no account should you consider neural networks to be something that will think or decide for you.

 

Myth 2: Magic Software

 

The second as, if not more, dangerous, misconception about neural networks emanates directly from the first one: somewhere out there, there is a heaven-sent trading software that basically works as a minting machine and all you need to do is find it. This misconception is not dangerous only because you will lose time and money while looking for what doesn't simply exist, but also because your delusions are well-known to those who crank out one-magic-button, slipshod software programs and fob them off on the seekers of the neural Holy Grail. Normally, those who try to exploit others' delusions make poor professionals and, thus, poor software too. Remember - neural network software can only do what neural networks themselves can do, and they can do a lot if you know how to apply them and what software to purchase. However, no neural network software can tell you the exact time and the type of action you should take at this particular time to profit.

 

Myth 3: Neural Networks Can Predict Precise Figures

 

The third frequent misconception is that by using a neural network you will be able to predict the future prices. Many traders believe that their networks are capable of telling them when to buy and when to sell. If you understand that those people are wasting their time and money you will probably be a success with neural networks. No neural network, no matter how sophisticated or well-built, will be able to precisely inform you about the future price or, at the push of a button, tell you, and you alone, when it's best to buy or sell (for, otherwise, there would no longer be a market). However, you can, undoubtedly, predict the likelihood of other important things happening, which will help you make better trading decisions. Therefore, even with what neural networks really can do, they remain the most powerful market analysis tool ever in situations, involving noisy data or non-linear dependencies. In other situations, using neural networks may be inexpedient. We will dwell on the predicting ability of neural networks and on what and how they can actually forecast later in this article.

 

Myth 4: Some Nets Are Significantly Better than Others

 

Many traders who want to employ AI for making their trading solutions mistakenly believe that the quality of the neural network capabilities of the different trading applications on the market varies significantly, and there is some special neural network somewhere that will eclipse all the rest in terms of the quality of the forecasting results. However, practice and experience show that the quality of different neural networks, no matter how much touted for, differs within the range of 10%, and even so it varies for different tasks and data sets. Of course, while selecting a trading software program one should look at the AI background of its developers (building a good neural network takes a great deal of skill and experience), but, at the same time, the application must provide the rest of the required functionality (such as, for example, the charting functionality) with excellent quality. In other words, one should look for a successful combination of neural network functionality and other vital functionality.

 

Looking for the only magic net is much like looking for one magic technical indicator. Aside from that, this quest often feeds those who are after a quick buck.

 

Myth 5: The Quality of the Forecasting Result Depends Solely on the Quality of the Network Used

 

The quality of the forecasting results does depend on the quality of the network you apply, but for not more than 10-15 %. The rest depends on how well the trader has prepared the data sets the network works with. The data sets must be sufficiently representative. They must include all the important influencing factors. Besides, the application of a neural network must be combined with Money Management and classical filters.

 

What Neural Networks Can Do for You and What You Need to Know to Make Them Work

 

Neural networks are definitely not a solution to all problems and they shouldn't be regarded so. What they are is a most powerful, technology-based method of technical analysis that can become an inestimable addition to your trading arsenal. Just like any other method, neural networks have their advantages and limitations, but their unique ability to track even the most subtle interdependencies in the available data no other method can establish, as well as build patterns based on this analysis, definitely make neural nets stand out from the rest of the existing methods and tools.

 

You can effectively use neural nets to:

• estimate the likelihood of a trend continuing;
  
• classify market phases;
  
• produce time estimates of highs and lows for various timeframes and combine results into a committee;
  
• predict the probability of a new, strong upswing after an uptrend, followed by a classic correction;
  
• track inter-market dependencies.
  

 

In other words, you receive a TA tool which will be a lot more efficient than classic TA methods anywhere where there is too much noise or where the interdependencies in the data are floating and significantly non-linear. For example, if after analyzing a number of charts you have discovered that the closer an uptrend is to a pivot point, the closer the bar's Close is to the bar's High, and you are planning to create an oscillator to anticipate reversal, you should use classic math as was done by the inventor of the Stochastic oscillator George Lane. But if you are trying to find a formula for the inter-relationship between S&P, InterestRates, $/Euro, Oil prices, and so on, you will make sure that the classical correlation or ratios won't be any use, since although interdependencies do exist, they are not stationary or linear. These interdependencies oscillate, ”float” through time and are influenced by noise. In this case, neural networks can solve the task better than the classical statistics.

 

When used in a combination with other technical analysis methods, and when sufficient attention is paid to the preparation of data sets (this procedure is, actually, central to success with neural networks), neural networks will undoubtedly provide the punch you need to success on the market. After all, this has been proven by both time and experience.

 

Source: Trading Software, Technical Analysis, Neural Networks, Stock, Forex, Futures

[Danaty 10]

I agree with Dima 100% when it comes to this post about Neural Networks software and the search for the Holy Grail. I am guilty of a lot of the myths Dima mentions. Dima is right stop wasting time and money searching for the non existing Holy Grail and trying to make a quick buck. Just do your home work find a method of trading that you are comortable that matches your personality and stick with it. Stop changing software and methods every time you take a loss. Keep a journal which will help you see where you need to improve on your trading style.

 

Dima thank you for giving such a blunt and realistic explaination of Neural Networks.

[darthtrader3.0beta 10]

Dima is right stop wasting time and money searching for the non existing Holy Grail and trying to make a quick buck.

 

Well I both agree and disagree. Personally, I think traders should throw out this bizarre concept of the "Holy Grail"..using modern data mining techniques is not akin to a Monty Python movie and running into the Black Knight.

I do agree with the quick buck part though, there is nothing quick about learning these techniques.

The bigger problem I see is that retail trading software that tries to use data mining, tends to not leverage what the computer is better at doing than the brain.

A week of 5 minute OHLC bar data on YM is only 1560 data points. The computer doesn't have much an advantage data mining that data set vs the brain. That is a very simple summary of the trading data that could be used, most of it has been discarded. Now if you store tick data for all 30 DOW stocks and tick data for YM for that same week, now your on the computers playing field and the brain simply can't compete as far as analysis goes. If there are patterns in 5 minute OHLC data then certainly there are patterns in the underlieing data that make up those 5 minute bars.

Where are retail software fails us is that we can barely store that level of tick data in a useable way, let alone do any kind of analysis on it, let alone do any kind of complex machine learning/data mining analysis on it...

Its just not economic to build and sell that kind of software to the retail crowd, your basically on your own.

[tlfx 10]

There is no holy-grail in trading.

 

The trader himself is the holy-grail IMO

 

 

 

 

 

 

 

[asavage 10]

NN are like any tool they can help but they can also hurt if you don't know what you are doing.

 

I know a lot of guys who overtrained their NN's and when these models tried to trade out of sample they got destroyed.

[darthtrader3.1beta 10]

NN are like any tool they can help but they can also hurt if you don't know what you are doing.

 

I know a lot of guys who overtrained their NN's and when these models tried to trade out of sample they got destroyed.

 

Most traders are just "over smart" when it comes to this area...too "smart" for their own good and miss the obviousness of the context of the data they are trying to analize..."Context" to me is exactly what holds current AI research back, but that is a tangent.

A Boltzmann Machine is much more obvious than any kind of Back-Propagation

training BS...Even more obvious though is ant colony optimization in reverse of a tick database to weight probabilistic paths..hint, hint.

[Indrionas 10]

Neural network is nothing more than just another non-linear modeling tool. When you are training neural network on the data you are making a few very important assumptions:

1) There are patterns in the input data. Neural network cannot answer this to you. If your data contains noise only, prepare for failure.

2) The patterns in the input data has relationship with the target variables. If there is no relationship, the NN will just model noise (aka overfit).

3) You have enough data to find the said relationships. Which is usually not the case with the market data. For instance, having 5 years of daily data makes only about 1250 trading days. When training NN, you have to put some data aside for validation and out of sample testing. You are left with less than 1000 data points, which is a very low number for learning complex multivariate relationships.

[FFTrader 10]

I day trade the ES futures exclusively. Using Traderstation as the main platform. Any form of "point me in the right direction" is greatly appreciated for the following ...

 

Because of my pre-trading understanding of Neural Networks some 20+ years ago, was thinking about adding Neural Networks tools to my arsenal today. Questions:

 

1. Where can I find a good place to read more on programming and applying concepts of Neural Networks in Tradestation? With short examples to use would be nice. :idea:

2. My objective(s) is(are) obviously those suggestions appeared in Dima's starting post. Having said that, my current system lacks a confirmation element / clarity for taking a trade or not taking a trade (with a fixed stop loss range). Is it reasonable to expect I can find something that can deliver some form of answers such that GIVEN a set of price levels (e.g. ESM09 for Easter Monday 2009 ... selling 848.50 and buying 841.50, although my bias is down), it will rate it as least / most risky (to buy/sell), etc.? If so, which ones are the alternatives?

 

Thank you in advance!

[bakrob99 11]

checkout http://www.breakoutfutures.com

[FFTrader 10]

checkout http://www.breakoutfutures.com

 

I took a quick look. Which ones are Neural Network based?

[bakrob99 11]

Michael was selling 100 copies of his neural network base system which I see has been removed as he has sold all of them that he will release. (I think he was selling them for $3000 if memory serves me right).

 

However, he did some coding which you can se at:

http://www.breakoutfutures.com/Newsletters/Newsletter1203.htm

 

This is an EasyLanguage structure for neural network software which you can play around with and gain some insight into its structure.