The Normality Assumption and Fractal Nature of Asset Prices

[Flojomojo 10]

The Normality Assumption and Fractal Nature of Asset Prices

 

Hi everyone, I’ve recently read the book “The (mis-)Behavior of Markets” by Benoit Mandelbrot. I found that some of his ideas matched the experience I have made so far. The key message of the book is that he basically says that all financial theory built so far on the assumption of a normal distribution are for the trash bin and we need a different approach. Especially his idea of “trading time” instead of “normal time” rang a bell in my head. Inspired by his 1963 study on cotton “The variation of certain speculative prices” I decided to test two assumptions myself on EUR/USD Forex rates:

 

1) Are Forex returns normally distributed?

2) Is there a fractal nature in Forex data?

 

In the next couple of paragraphs I want to share my findings and thoughts with you. Comments and corrections are always welcome! As I am a ‘hobby quant’ I do not claim the results to be set in concrete. If you have made a study on your own I’d be happy to hear from you.

 

The data I have used for this study is 47 weeks of IB minute OHLC data from the year 2007. I made a test on the following time intervals: 1, 5, 15, 30 minutes

For that I analyzed the open to close movement of each bar for the respective time intervals. That are 334012 data points for one minute and 11111 for the thirty minute bars…so don’t complain about too little data!

 

Let’s get started:

 

1) Are Forex returns normally distributed?

 

Are absolute changes [P(t+1)-P(t)] normally distributed?

I was always a bit irritated why academic literature always works with returns (absolute or logarithmic) instead of absolute changes. Sure, there are advantages to using them, but when sitting in front of the screen I look for absolute changes. To analyze whether the normality assumption holds I chose to plot the return data in a probability plot. The red line shows the points of the fitted normal distribution. If the return distribution is normal, all points would lie on or close to the red line. As you can see this is clearly not the case for any of the four time horizons! In fact the S shape of the blue data shows that the distribution is a lot more fat tailed than the normal distribution suggests.

 

 

Are logarithmic returns [ln(P(t+1)/P(t))] normally distributed?

When looking at logarithmic returns the same picture arises as seen on the probability plot for logarithmic one minute returns.

 

Therefore, however the returns are calculated, the normal distribution assumption is pretty far off reality. Beware if you are using it!

 

2) Is there a fractal nature in Forex data?

 

So if changes are not normal, do they follow a power law of the form a*x^n? In Mandelbrot’s book he states on page 151: “Such power laws are common in physics and are a form of what I call fractal scaling”.

To check the power law assumption one needs to create a histogram and plot its data in a loglog plot. As the absolute return distribution is of interest, the amount of occurrences in the negative histogram half were added to the positive side giving a distribution of absolute changes. Since the number of observations varies from timeframe to timeframe, the frequency was normalized in each case by dividing it by the total number of observations. If the data in the loglog plot follows a power law, the data points should form a straight line.

 

Before showing any pictures, there is one open question: Histograms count the number of occurrences in specific intervals (bins). How big should these bins be?

 

When trading, the first moves are critical as the position should show a profit as soon as possible. The first move that really matters is the move that overcomes the initial spread plus payable commissions. So assuming a one pip spread and two pip cost for opening and closing, then a short term trader is only interested in moves >3 pips in his direction. Traders using larger timeframes aim to catch larger moves and don’t watch every pip move. To them moves of for e.g. 10, 20, 30,… pips are more important. Due to this reasoning it makes sense to use larger bins for the histogram of larger timeframe data.

 

Here are the loglog plots of the different time frames for the 2007 data:

 

 

To the human eye they all look fairly similar. When fitting a power law into the tails one can see that in the tails they all follow approximately the same power law. This shows by all lines having approximately the same slope n. (I want to point out here that such a visual test is scientifically not sufficient to prove that there truly is a power law at work!)

 

Now you might say “Well, all results were calculated on the same sample data! The picture might look different if the different bar intervals were taken from different data sets!” To test his, I split up the 2007 data set into three parts that had the same length for bar intervals of 1, 5 and 15 minutes. In addition the bin size was chosen so that the number of data points in the histogram is equal. Here is the result with the data points being offset from one another to display them in one graph:

 

 

If I wouldn’t have labeled which data points belong to which timeframe, I bet neither you nor I would have been able to determine the timeframes correctly other than by luck!!!

 

With this finding I conclude that Forex data indeed shows fractal characteristics over different timeframes.

 

Now the really relevant question is: What are the implications?

 

Here are some points that come to my mind:

- Due to the self similarity, a good trading approach should work on multiple timeframes

- If a trend can be pinned down in one timeframe, entries can be timed in a lower timeframe

- If trading speeds up, change to a lower timeframe

- Accumulation, mark up, distribution, mark down cycles are present in all timeframes

 

If you actually managed to read until this line in this monster post…what comes to your mind?

 

Best regards and good trading,

Flojomojo

Edited March 19, 2009 by Flojomojo

[JBWTrader 10]

Awell put together point of View Flowjomojo

 

I cannot fault it. A well put together analysis.

 

Well done. I agree with your conclusions.

 

All the Best

John

[Minetoo 10]

Flo:

You may be interested in Nichols' stuff here, http://www.fractalmarketreport.com/index.php

Your avatar should be a piece of broccoli.

[aediaz1 10]

Thank you for doing this research. I just got confirmed, from an academic point of view, some of my believes regarding the FX market. Great post !

[Yorkid 10]

Flojo -

Here is another implication: social change is fractally ordered.

I think you must be disappointed that you have not more responses to your excellent post.

If I were you I would submit your work to the Socionomics Institute for publication in their online journal. It is exactly the kind of independent corroboration they are looking for.

If you want to get in touch by email I'd be interested. I have ideas that my mathematical ability is not up to pursuing and I'd be grateful for your help.

Y.

[Flojomojo 10]

Flojo -

I think you must be disappointed that you have not more responses to your excellent post.

Thats true. I was hoping for some of the big boys in this forum to comment a bit on my work. But to be honest there already has been research in this field, especially from Fama, a former doctoral student of Mandelbrot. Their work is a lot more detailed than my, in scientific terms "sloppy" analysis. So the study is not that new, but as far as I am aware of, the academic literature does not touch the practical implications for traders.

 

Since this post I have taken the results a lot further with explaining the fat tail problem and implications for risk management. Not sure yet whether I make a post out of it though.

 

Feel free to post your ideas here and I'll see how I can comment on them.

[Yorkid 10]

Hi Flojo -

Referring specifically to the implications of the fractal nature of markets for traders, Elliott wave theory is the largest relevant body of work. Elliott is credited by Prechter with discovering fractals before Mandelbrot. I think the evidence bears out his claim.

I should have given you a link for the Socionomics Institute (although I'm sure you already found it): http://www.socionomics.net

Although I trade on a daily basis, and I take the fractal view of markets as something of a given, I was interested in your post because of its application to wider social theory, which is something that would prabably be of little interest for a traders' discussion site.

I was particularly struck by the way you focus on the change from one time interval to the next, thus laying emphasis on the dynamic aspect of the market rather than the static pattern-related aspect. The patterns are evidence of the fractal nature (Elliott's work) of the way in which they are themselves generated (your work): it is exploring the possibility of the existence of an algorithm which generates this fractal dynamic which interests me. I don't know if this is ground that Fama has covered, but I suspect that if it is then his mathematics would be too difficult for me to follow. It was with a view to getting some digestible mathematical input that I responded to your post.

When you say "sloppy", I think you are being too hard on yourself because there is an inherent barrier in this investigation: it is the irreducibly inductive and reflexive nature of thinking about patterns which we ourselves constitute.

My email address is leslie[at]weydale.freeserve.co.uk

[Logic 10]

I've been researching and studying pure price with a view of their fractal nature for over 15 years and have come up with a few objective conclusions with regards to applying them inside price charting environments.

 

1. Applying any study to variable chart environments (all time, tick or range based charts are inherently variable in nature) will result in inaccurate and inconsistent conclusions.

 

2. Applying any study to the FOREX Markets (pairs) is a futile effort. FOREX is based on separate intra-bank (6) data feeds each of which are producing only time or tick data. None of these feeds release volume data, either in whole or in groups because this is how these individual or groups of banks manipulate the data they release. These banks have repeatedly broken negotiations with GLOBEX when trying to join these feeds into a single stream of information. The only point in time where FOREX will be a reasonable and fair environment to trade will be when all of these feeds are joined and their data and volume related inforamtion is validated.

 

3. Constant (capped) Volume/Share Bar charting is the only way to accurately, objectively and consistently view the effect of fractals on pure price action (movement). This is because this charting environment is based on the fact that each bar is equally weighted and thus contains no variable aspect. Having the bars of any chart we are making are trading or investment decisions from being grounded in equally weighted bars not only levels the risk associated with that chart but tilts it in our favor.

 

Mandelbrot & Elliott were on the right track but neither of them had access to the tools to take the information to the next level and that was to smooth out the information that price gives us in its pure movement. Elliott saw price move in oscillations (waves) but erroneously concluded those waves were predictable. Yes, price moved in oscillations (waves) but there is no consistent number of waves inside the longer term extreme moves in price action. Mandelbrot saw how fractals made up everything on the face of the earth but couldn't accurately define the exact effect they had on price action.

 

A clearer definition to the "fractal effect" on price is "Fractionals" not fractals.

 

I've been lobbying some of the software companies since 1995 to produce Constant (capped) Volume/Share bars but it took Ensign in 2003 to first offer them. Since then a few more charting companies have offered them (I use MultiCharts) but due to their complete difference to the established way charts are currently built a lot of companies refuse to redo their complex code to offer them. The charting companies approach their products with blinders and feel they only need to do the minimum expected and do not see a need to be cutting edge on any of their products and services. This is a real shame.

[Anton 10]

Your graph, 1) "Are Forex returns normally distributed?" has produced a spot on (if not perfect) Sigmoid Transfer Function (also known as an S shaped semi linear function).

 

In looking to answer your question, "Are absolute changes [P(t+1)-P(t)] normally distributed?"; you would / might have anticipated with Normal Distribution a Gaussian Transfer Function (better known as a 'Bell Curve').

 

However, the consequences of your finding and observation given as Sigmoid, has very significant implications.

[Yorkid 10]

Logic -

If I understand you correctly, I think you are setting up a straw man there: we may only see through a glass darkly but that does not mean that we have not discerned the main features of what we are looking at.

I would suggest that more complete information will certainly be of use but it will not of itself overcome the problem of (the probabilistic nature of) the predictive task.

I would also suggest that it is in the nature of fractal patterning that the smallest action of the smallest player has the capacity to resonate in the outcome at the largest level, so I'd conclude that somehow the information one needs for successful prediction is contained in every participant's attitude to every trade and that therefore the progress of the whole can be discerned in the state of the smallest part. We have either got fractals or we have not - more detailed resolution is not the key to either making use of this or indeed to verifying their existence. Perhaps what you are saying is that, given complete data the fractal nature of it would be shown to be an artefact of its previous incompleteness, but I think that there is overwhelming evidence to the contrary.

Where I would agree with your plea to get equivolume data would be this: that it might enable us to dispel some of the myths about the level of participation which is supposed to obtain at various stages of a wave's development.

It fascinates me that we participate in these things we call markets without really knowing what constitutes a measure of expertise. In "the market's eye" we are all of us equally disadvantaged. I suppose I'm talking about the ideal situation there, but in reality it seems that human nature will see to it that even a rigged market is rigged fractally. Even when mere incompetence reigns supreme over the nature of the information that goes around, the nature of that incompetence is ours and therefore just as likely to be fractally patterned as anything else we are responsible for producing as a body of social actors.

[SunTrader 10]

......Constant (capped) Volume/Share Bar charting is the only way to accurately, objectively and consistently view the effect of fractals on pure price action (movement). This is because this charting environment is based on the fact that each bar is equally weighted and thus contains no variable aspect...

 

But the markets are dynamic and not static so why make your chart static?

 

IMO analysis of how the markets are and not how we wish seems the way to go.

 

Trend traders try to avoid the ranges and range traders vice versa. Some attempt to trade both scenerios which I don't believe makes much sense. But price is what it is and will do what it will do.

 

I use EW/fibs/fractals and do see repeatable, predictable patterns. Just not all the time in all markets. I believe Prechter or anyone else who believes you can define 100% of all price action is missing the point. It is not necessary, in order to trade, to know every minute of every day what the market is doing.

 

So we market participants and academics will continue to try to disect what it is price is telling us, but the day we truly understand will be the day the markets cease to exist.

[Logic 10]

Logic -

If I understand you correctly, I think you are setting up a straw man there: we may only see through a glass darkly but that does not mean that we have not discerned the main features of what we are looking at.

I would suggest that more complete information will certainly be of use but it will not of itself overcome the problem of (the probabilistic nature of) the predictive task.

I would also suggest that it is in the nature of fractal patterning that the smallest action of the smallest player has the capacity to resonate in the outcome at the largest level, so I'd conclude that somehow the information one needs for successful prediction is contained in every participant's attitude to every trade and that therefore the progress of the whole can be discerned in the state of the smallest part. We have either got fractals or we have not - more detailed resolution is not the key to either making use of this or indeed to verifying their existence. Perhaps what you are saying is that, given complete data the fractal nature of it would be shown to be an artefact of its previous incompleteness, but I think that there is overwhelming evidence to the contrary.

Where I would agree with your plea to get equivolume data would be this: that it might enable us to dispel some of the myths about the level of participation which is supposed to obtain at various stages of a wave's development.

It fascinates me that we participate in these things we call markets without really knowing what constitutes a measure of expertise. In "the market's eye" we are all of us equally disadvantaged. I suppose I'm talking about the ideal situation there, but in reality it seems that human nature will see to it that even a rigged market is rigged fractally. Even when mere incompetence reigns supreme over the nature of the information that goes around, the nature of that incompetence is ours and therefore just as likely to be fractally patterned as anything else we are responsible for producing as a body of social actors.

 

I don't think you are understanding what I am saying but then again I've spent years practicing my ability to simplify my position so that everyone could grasp the foundation of my points. That process has a tendency to confuse they more complicated minds out there.

 

What is the difference between watching a TV show on a CRT television with an analog signal and a 1080i HD television and a digital signal?

 

Simple answer . . . the picture is simply clearer on the HD television. Same program, same actors, same everything.

 

What is the difference between watching a price chart made up of variable bars and watching a price chart of the same market made up of non-variable equally weighted bars?

 

Simple answer . . . the view of the oscillations, as they relate to that specific chart is perfectly clear. Price always moves in oscillations and those oscillations are clearer when viewed in a non-variable environment.

 

Don't we want to have a clear unobstructed view of the price action regardless of the chart we are making are trading or investment decisions on?

 

Then breaking those charts up into segmented Fractionals (I personally use increments of 7; 7, 49, 343, 2401, 16807, 117649, 823543 etc.). I can clearly and consistently see those waves of price action as they move inside the longer term larger fractional charts.

 

I consider myself advantaged not disadvantaged. I consider myself perfectly and objectively competent, not incompetent due to my view of price action but then I have 15 years of personal research and over 35,000 hours of screen time to validate my position.

 

The markets are random and chaotic only if one views those markets on charts based in a state of randomness and chaos.

[Logic 10]

But the markets are dynamic and not static so why make your chart static?

 

IMO analysis of how the markets are and not how we wish seems the way to go.

 

Trend traders try to avoid the ranges and range traders vice versa. Some attempt to trade both scenerios which I don't believe makes much sense. But price is what it is and will do what it will do.

 

I use EW/fibs/fractals and do see repeatable, predictable patterns. Just not all the time in all markets. I believe Prechter or anyone else who believes you can define 100% of all price action is missing the point. It is not necessary, in order to trade, to know every minute of every day what the market is doing.

 

So we market participants and academics will continue to try to disect what it is price is telling us, but the day we truly understand will be the day the markets cease to exist.

 

Hey, why buy an HD TV. I can see the picture just fine with my analog signal and CRT TV, right? I mean that is how the signal first started out, why improve on the delivery of information, right?

 

I see repeatable patterns too but mine are objective, consistent and readable in real-time. Should I give back all of the profits they have generated because I have missed some point or better yet, found a point others have missed? (that was a funny statement not meant to be argumentative).

 

Einstein once stated, "Everyone knows something cannot be done. Then along comes a man who does not know that it cannot be done and does it."

 

We all trade based on an edge we have found or are searching for. I found one. I've shared what I've discovered with many traders and without exception they have all commented that using Constant Volume/Share bars has enhanced their profitability, regardless of their method. At least those that have applied them correctly. That speaks volumes as far as I'm concerned.

 

I understand how to objectively and consistently read the oscillations price creates continuously but I don't think anyone understands why price HAS to oscillate . . . and it is still moving today and tomorrow and from this point forward. When those oscillations cease, then we something to worry about.

[SunTrader 10]

Hey, why buy an HD TV. I can see the picture just fine with my analog signal and CRT TV, right? I mean that is how...........

 

Hey keep doing what you are doing. No argument here.

 

But 'time' is not a consideration with static bars. I know that is one of the points of using them to begin with and agree fixed tick/volume bars help as far as markets such as the eMini, which has light volume through much of the globex night session, by ignoring the time axis.

 

But to me to truly understand the dynamic nature of the markets on a longerterm such as 1H/4H/D1/W1/M1 it is an absolute necessity to consider 'time' cycle factors. To each his own.

[Flojomojo 10]

Hi Anton,

 

Your graph, 1) "Are Forex returns normally distributed?" has produced a spot on (if not perfect) Sigmoid Transfer Function (also known as an S shaped semi linear function).

 

Now that you mention it you are right about the sigmoid, but I'm not sure whether I'd call it a "transfer function" as I'm not directly sure what it should transfer. I know its application only from neural nets, but I don't see a connection here...enlighten me!

 

In looking to answer your question, "Are absolute changes [P(t+1)-P(t)] normally distributed?"; you would / might have anticipated with Normal Distribution a Gaussian Transfer Function (better known as a 'Bell Curve').

That's right, I was anticipating a normal distribution. Thats why I have tested with a normal probability plot:

 

The normal probability plot (Chambers 1983) is a graphical technique for assessing whether or not a data set is approximately normally distributed.

The data are plotted against a theoretical normal distribution in such a way that the points should form an approximate straight line. Departures from this straight line indicate departures from normality. (found here)

 

But this has nothing to do with a transfer function. A gaussian transfer function still looks like a bell curve, whereas in the normal probability plot it is the straight red line.

 

However, the consequences of your finding and observation given as Sigmoid, has very significant implications.

 

Come on...you can't just make a statement like this and leave us in the dark by not stating any implications!

 

Regards,

Flojo

[Yorkid 10]

Logic -

I would be interested in your methods if you would teach me.

[Logic 10]

Logic -

I would be interested in your methods if you would teach me.

 

There's a thread in here which is a good place to start.

Teach yourself to trust yourself.

[zdo 20]

Flojomojo, a Preface: Even though it will most likely push some head buttons, this is (paradoxically) not a disagreement with your posts

 

But! … after nearly 25 years of “turning charts every way but loose” , I have concluded the following needs to be said plainly.

 

They lied to us. “As above, so below” – NOT!

[BlowFish 10]

Prof, I am a proponent of constant volume bars for certain applications and have followed your arguments at other venues.

 

Might I ask is it your contention that time is an unimportant variable when analysing price series data? (I think I can guess what your answer might be )

 

As an aside constant volume bars with a time histogram below gives an interesting depiction of 'pace' or momentum (if such things are of interest).

 

Sorry FJM strayed a bit off topic.... good work though!

[Logic 10]

Prof, I am a proponent of constant volume bars for certain applications and have followed your arguments at other venues.

 

Might I ask is it your contention that time is an unimportant variable when analysing price series data? (I think I can guess what your answer might be )

 

As an aside constant volume bars with a time histogram below gives an interesting depiction of 'pace' or momentum (if such things are of interest).

 

Sorry FJM strayed a bit off topic.... good work though!

 

I don't want to pull this thread off kelter but yes time is irrelevant to price action. Adding time in a histogram is just adding an unnecessary distraction. Look at a clock if you must.

[Flojomojo 10]

Hey ppl...as there seem to be some controvertial viewpoints here I suggest to open up a new thread to discuss the relevance of the concept of "time" in trading.

Best regards,

Flojo

[Logic 10]

Hey ppl...as there seem to be some controvertial viewpoints here I suggest to open up a new thread to discuss the relevance of the concept of "time" in trading.

Best regards,

Flojo

Agreed and I apologize for any interruption.

[davelansing 10]

Hi,

 

I understand the concept, to some limited degree, of the fractal nature of time-based price data... but wouldn't one expect that? Doesn't the price movement on a 5 min chart manifest itself on a 15 min chart which manifests itself on a 60 min chart which manifests itself on a daily chart, et cetera? How can one "predict" where price is heading by multiple time frame analysis using the trend of a higher time frame (e.g. 60 min) to assess the movement of a shorter time frame (i.e. 5min) when its the shorter time frame that ultimately makes up the higher time frame movement? My only answer that I can come up with is that there are "more" large traders that follow price movement on the higher time frames (i.e. 60 min and above) whose decisions and actions based off higer time frame chart points (S/R levels, pivots, et cetera) manifest themselves in the "immediate sense" at the level of the shorter time frames. I've struggled with this for a long time, so any feedback would be greatly appreciated to help clear up my muddled thinking! Thanks!

Edited April 28, 2009 by davelansing

[Flojomojo 10]

I understand the concept, to some limited degree, of the fractal nature of time-based price data... but wouldn't one expect that?

He there,

maybe the term "fractal nature" was a bit misleading in my post. The final chart in my post can better be describes as the timeframes being "self similar".

 

Doesn't the price movement on a 5 min chart manifest itself on a 15 min chart which manifests itself on a 60 min chart which manifests itself on a daily chart, et cetera?

Yes, but this does not conclude self similarity of the different timeframes.

[davelansing 10]

self similar? please explain.... I mean if the trend on the 60 min chart is "down", obviously its because price on the 5 min chart is moving "down" and not the other way around (since the 60 min chart is made up of sequential 5 min intervals). So how can analysis of a higher time frame aid in the analysis of price movement on the 5 min chart?

 

Thanks!