Frank

I am kicking off the Market Statistics forum with some analysis of S&P futures range in pit session vs 'overnight'.... Now, I want to come at this from a 'traders perspective' and not as an academic might. What I am really getting at is --- what dynamics are at work from a traders perspective? Personally, I am just trying to find some trades to make and then get out of the market and get my exposure back to zero, I am not trying to catch the entire range of every day. It does not matter that X happens or Y happens, it only matters that I find some trades to make and execute those at a decent % and get rewarded for the risk I take -- and then some. So I started by looking at the overnight market --- and would like to use an equal amount of time BEFORE the U.S. open --- as the cash market IS open -- which turns out to be ~6.5 hours. The U.S. market is open 9:30am to 5pm NY time, so I am using that as a reference point. I am looking at futures so I will use Central time as basis. Thus, I want to look at 'range' for the S&P futures from 2am - 8:30am -- which might be thought of as its 'own session' -- approximately consistent with when the European cash market is open. This is the point range for the S&P futures during that timeframe -- going back ~7.5 years: Note that the range for the S&Ps during 2004-2006 was generally ~5 pts during this time of 'night'. Note also that while range is far off its high, its also far above the old 5 point plateau. In fact, the chart looks a lot like VIX -- which is also still far above the '04-06 level but way off the extreme highs. continued next post

CrazyNasdaq, you do not need a 2 or 3d array for this. a programming friend of mine did this in Visual Basic and used a 1-dimensional dynamic array. Here were the steps: 1) set up the basic array structure: if 'this bar' is first bar of a new day (date>date[1]), clear out array and set counter variables to zero if not a new day, then expand the existing array (the one created if the first if statement was true) to add new data, while preserving the existing array I believe you do this using the EL syntax Array_SetMaxIndex(ArrayName, #Elements) 2) populate array. something like this: barrange1 = ((high - Low) / 0.25)+1; x=0 For Counter1 = 0 to barrange1 begin bars1[x] = low +(0.25 * Counter1); x=x+1 end; note that the variable x is going to keep track of total elements in the array the part I don't understand is how often you have to re-set the size of the array and/or preserve the array so it doesn't lose data elements within the array. the code I write invariably is out of bounds of the array or some syntax just isn't right. its damn hard to find supporting documentation on EL.

hi Kiwi, I haven't presented any data on Asia yet -- this was just europe vs US. Yes, in a market that trades 23 hours a day, moves are going to happen outside of precisely 9:30 - 4pm NY time. Let me give an example of how I am calculating this -- I should have done this in the first post, I realize. On Friday (Oct 23, 2009): the 9:30am - 4pm EST (pit session) low and high was 1071.50 to 1093.00 -- for a range of 21.50 pts during that 6.5 hour time period. If I look at the range from 3am to 9:30am EST, the high was 1095.25 and the low was 1089.25, so the range was 6 pts during that particular 6.5 hour period of time. If I think of 1 'session' as 6.5 hours and I sat at my computer for those 13 hours (2 'sessions') --- there was ~27.50 pts to make a few trades over that combined 13 hour period. In this situation, the US session was the one where most of the range occured --- with US hours representing 78% of the combined 2 session today (21.50 / (21.50+6.00)) = 78%. The chart was just a way to view what has happened over the last 7.5 years from this perspective. This isn't data mining with an agenda -- this is just observing 2 equal periods of time and doing subtraction and division. If you want to cut it another way -- let me know the 'rules' you would like to use and I may try it out. (In this calculation, I am just trying to show that if you sat at your computer for 6.5 hours -- and you chose those hours to be 9:30-4pm EST -- how much range did you see? and using 'range' as a proxy for 'opportunity'... I can slice and dice this data in many other ways that are certainly better -- this is just one way to do it). frank

I took a look at quantifying the theory that all the 'action' is occuring on Globex and not during regular pit session hours. Here is a first pass attempt. The futures market is generally open ~23 hours a day. The S&P Cash Index is open for 6.5 hours of the day. Then there is the 15-min stub period from the cash close to the futures close. The European market is the 2nd most active market in the world and this is where I want to focus first. It starts to get complex because world clocks are not aligned -- time changes seasonally -- creating diff't amounts of overlapping sessions. Some assumptions have to be made or else this becomes a major 'get lost in trees' exercise. So here is my assumption: only compare the 6.5 hours of US cash index (9:30am to 4pm NY time) to the 6.5 hours prior to the cash/futures index open (3am to 9:30am NY time). The 'amount of time' in these two time periods is the same and therefore we will not be comparing a 6.5 hour period to a ~17 hour period of time -- as you would if you just did 'pit' vs 'globex'. Here is the chart result, shown as a 20-day rolling moving average of the ratio of pit range vs 'euro session' range. The Red Line is the average for that year. Year to date, pit has represented 63.2% of the overall '13 hour period' measured. note to James (TL Founder): could you start a 'Market Statistics' Forum?? --- unless that is what 'Futures Laboratory' is meant for.....

ongoing volatitlity increase...

some starter code for you -- I plotted this on 30-min chart. note that times are Pacific Standard in this so adjust as needed. -------------------------------------------- //get return of first and last 30 min bars and find the point change from open to close //value1 & value2 serve as 'variables' holding that particular days data // note: California time 700 is first bar if time = 700 then value1=close-open; if time = 1300 then value2=close-open; //calculate the difference value3=value2-value1; //take the calculated values and place them on a chart if time=700 then plot1(value1); if time = 1300 then plot2(value2); if time > 1300 and time <1400 then plot3(value3);

Maybe today jumpstarted volatility --- hard to say but will continue to track it. The real 'issue' with volatility has been death of afternoon volatility. Its not the only time that afternoon volatlility has gone into a funk -- it usually reverses.

here is the hack-code I came up with. it needs some work --- BEWARE to Tradestation users, your system might hang if you try to run this on a LTF chart. (and it won't calculate correctly if you run it on Higher Timeframe chart). vars: firstbar(0), length(0); if date > 1091015 then begin // <== this is in here so doesn't get locked up in loop looking back past 10/15/09 if date > date[1] then firstbar=currentbar; length=currentbar-firstbar; value1 = (O+H+l+C)/4; value2 = round2fraction(value1); value4=mode(value2, length, -1); //the -1 finds the lowest of modes if more than 1 mode price exists if value4>lowd(0) and time>330 then plot1(value4); end;

I am posting this in this forum in hopes someone can make the code more efficient. It runs in Open Ecry/Easylanguage --- but it won't run in Tradestation as it seems to be too "compute-intensive" for Tradestation in its current form. The Idea is to create an estimate of the most heavily traded price of day so that you can chart it historically and see how the volume profile filled out. The concept comes from how VWAP is computed, which is really an estimate of VWAP if done on a chart -- but it is usually within less than 1 tick of actual vwap if you drop down to a 2-min chart or below. here is the snapshot of the calculation -- note how the line that is created by the code closely matches the volume bars on the right and you can visualize how the volume profile would look if it were not included in the chart.

the platform is built on modern C#.net --- I have done EasyLanguage indicators that require significant 'looping' -- I have found some cases where the exact same copy/pasted EL code runs fine on OEC and blows up tradestation. I am not a coding expert so I don't write the best code but I think this is interesting nonetheless. OEC announced with last software upgrade that this was very major upgrade as they are gearing up to do equities (incld internals). I think its great -- though I think Tradesation is a good value too for historical data and internals and watching ETFs & particular stocks -- I just do the minimum trades at Tradestation and do everything else through OEC.

I would like to see this supported with some statistical analysis.

did a shorter term study on that --- working on expanding it. but idea is that market forms 2 extremes during globex and then takes out one or the other relatively early in day. that 'violation' can trigger reversal -- or continuation -- which is critical time to watch.. with 1 idea playing for that extreme as target and exiting --- not waiting around to see if continuation or not.. but at end of day, narrow is narrow and good analysis is still only going to get small wins if range is narrow so that is the dominant thing at moment.

Forgot to mention the main point of last post: 'Regime Change' will show itself when the TPO counts in the morning start getting high. Once again today, very low TPO counts.

one way to get better grip on this is to look at historical statistics comparing THAT DAYS intraday volatility. The concept being that if TPO count is low in morning, the afternoon will rarely show dramatically different TPO counts -- thus, can adjust to that days volatility to some extent --- with statistical caveats of 'outliers' always in back of mind. So I did this a while ago and updated it -- basically it compares the first 7 pit session bar TPO counts to the final 7. And basically, the FINAL tpo count has fallen into a range that is 1.7 -2.1x the first 7 30-min bar TPO count. Thus, a general idea would be that to temper profit objectives on any afternoon trades in low TPO count days. And perhaps look for incrementally more in high TPO count environments.

Let's put some data behind this -- first just update the data table for new month-to-date data. As can see, TPO counts are down again this month. A second look at traditional market profile would be to look at range and its relationship to the first hours range. What is odd about this market is that it has in general been doing first hour range consistent with 2007's type of average -- ~7.75 - 8.00 pts ---- but then there is not much range past the first hour. so while overall range for the day is similar to 2003-2006, ~11-13 pts -- it is skewed more towards the beginning of the day. This isn't meant as something I expect to continue or a forecast in any way -- just pointing out the nature of the profile relative to history.

its its been regime change alright -- but for the worse... TPO counts are down again this month, signalling even narrower conditions.

this is pretty ridiculous gooni. They say 'OEC Trader' in the disclaimer -- not 'OEC Trader Demo Platform'. I think its a reasonable assumption that they will not stream free data to everyone for demo trading. I have never paid a platform fee so there are no hidden costs -- disclaimer is just fine as is....

andypap, can you post the final completed code to fully understand what your indicator was doing? it would likely be useful to others to use this simple array structure used in an example to use in their own indicators (that do other things). thanks in advance

hi dam, I have data back to 2002 -- its 25,000 rows so I will just present the Monthly Average In table form. As you can see, there is a very wide range of values and it is somewhat similar to a VIX chart -- I will overlay them but here is the data table to chew on. I added 1 calculation this time which relates TPO's to the closing value --- while 1 tick is $12.50 no matter what the level of S&Ps, thought I should put it on there anyway. One possible reason for an increase in volatility is not so much the level of TPO's -- but the general seasonal uptick --- the last 7 years have been flat or up from September to October --- and the fact that its basically gone down every month this year.

'Regime Change' is quant terminology for when a market takes on a different environment. Typical 'regimes' are inflationary vs recessionary regimes, mean-reverting vs trending regimes etc.... Basically, the idea is that the market environment changes dramatically from time to time and blows up all the strategies that have been optimized to the 'last regime'. I present data to discuss 'MP based Regime Change: TPO Count as Measure of Volatility in S&P Futures'. That is, wider 30-min bars represent a different regime than a narrow 30-min bar market does. Here is some recent data: note how TPO count (Volatility) has been dropping for 11 straight months (scan column on far left). Despite VIX remaining elevated, the S&P futures went into very narrow action over past few months. The last few days seemed to be start of a new regime. I propose that we get an upcoming 'regime' that shows some better range than past few months. But let's track it here and see how it goes.

I think its a good time --- OEC just did a very major software upgrade and is rapidly improving the platform. OEC still does have some limits in its functionality --- but I believe they are on the right path and are strong in the areas that count most -- latency, outages and total cost of ownership (low commission rates and no hidden fees).

its my impression, and I could be wrong, that he wants to start holding prices in an array (presumably closing prices) of the bars that meet a condition and setting the max elements of the array to 20. I don't know what the counter part is for but I am guessing that he doesn't need the counter -- only to preserve the array from one bar to the next and keep the last 20 elements by adding a new element to array(19) or array(20) and kicking out the first element array(0) or array(1).

I see, what you want to do is preserve the array... at least that is how you do it Visual Basic --- redim preserve array() I am trying to find a resource for this on EL -- haven't found it yet.

what do you mean by 'align all the closing values'? what do you want the indicator to plot?

having trouble processing this. this calculation doesn't store a resulting number to use in the next bar, don't you have to repeatedly look back at all the cumulative array elements and re-count the mode as it may have changed? funny, maybe I am just a visual person -- but this was really easy for me when I originally did it in Excel but now seems quite compute intensive when thinking about the array coding steps.