taotree

I am working with futures using the zen-fire feed. I would like to get access to market internals like $TICK which zen-fire does not provide. Does anyone know of an economical feed just to get market internals? Thanks!

Seems to me highly unlikely that a separate feed is required for that. I have a single feed and I can get all that info. Also, an arbitrage trade could include multiple futures symbols at the same time as the stock market.

Why does it have to be especially "accurate"? S&P 500 is the futures market for a stock market. If the second data feed was something indicating the volume in the underlying stock market (whether aggregated index, premium, whatever) then couldn't it be possible that a trade intensity spike corresponding to a spike in the volume in the stock market would likely be an arbitrage and therefore something we might ignore... if there is no corresponding spike in volume in the stock market, then we can consider it commercial spike and use it. 1-2 second accuracy would be enough, it takes longer than that for the spike to "take affect" anyway. See a spike, check the market internals, no spike there, ok, take the trade. Not sure... but fits the description so far anyway, I think.

It seems to me the information we had before is all there and more. If you want to aggregate trades to subsecond resolution and group them together again, go for it. You couldn't be sure if trades were aggregated or deliberately split up before, and you still can't--so there's no change there. You have more information than you had before. Maybe... check the timestamps--if they're all identical, there's a reasonable chance it came from one source. Also, if you want T&S and tick data aggregated to time periods as it seems you're suggesting--many platforms already do that. That's all the platforms I avoided

I'm a little confused... The peak is obvious on UB's charts: velocity changes sign and he has what I assume is a crossover of 2 differently smoothed values. If what you're saying is that you don't have that on your chart, fair enough. I don't either

By crossover... you don't mean when the plot crosses 0 do you? The trade is at the peaks of the plot. I've seen whipsaw, too, but I haven't applied any sophisticated smoothing to it, so... I haven't really evaluated it properly yet. UB's nicely goes up and down whereas looking at the raw calculation I'm using, it wiggles a lot. Hmm.... which might mean: 1) UB is using some cycle smoothing on it and 2) I already know UB uses a different underlying formula for calculating it

And I can get it in a month at a time: Forex Calendar @ Forex Factory Bingo, I think that'll do for me. I had seen that before, but wasn't really excited about parsing HTML, but reconsidering, that's an easy one, especially full month at a time. One needs to be careful crawling web sites without permission--play nice and all. But... 6 calls (for last 6 months) to their site is much less than I've already done just exploring their site, so... I don't think they would complain about that. Thanks!

Well, here's some RSS feeds that might/might not be useful: FRB: RSS Feeds They're not what I need (since I want historical, and though there is some here, not quite enough for what I want), but they might be useful for some things.

Oh, hi NetTecture, I was just reading some of your very interesting posts over on ET the other day. Right, actually, I don't need a "feed" so much as just the data--especially historical. Well, if I could find the info somewhere, maybe I'll just reformat it myself.

The relevance is that it is my understanding that some platforms include news feeds as part of their service. I may be wrong. All I was indicating is that I do not have a news feed, and I want access to that info. It seems rather absurd that it doesn't exist. And charging an arm and a leg for public information even if conveniently formatted seems rather silly, too.

I'm on NT/Zenfire, so don't have a news feed service with my platform. I want programmatic access to the date and time of Fed events and such--including historically. I don't need to know what was said or anything, I only need to know the date and time of an event. Is this available somewhere? Even somewhere I could download a CSV or some easily parseable format of the events for the past x months? Thanks.

When you say "for each bar", do you mean that you have a + or - bias on an input for each bar given it's context? So in one context, an input might have a negative bias, but in another it might have a positive bias? I created a response function that assigned a value to every tick. It uses future data to determine what level of short/long interest I would want to have at that point in time. The idea is that then I could use that in any sort of supervised learning or fitting algorithm through which I might want to run the data. Basically, if you could have a crystal ball that would tell you one value at any point in time, what would that value be? I took a glance at MARS. If I understand it correctly, one of the rather interesting things it does as a it finds thresholds or turning points in the value of the data. When working with continuous data based on which one must make a discrete decision, that makes a lot of sense to have something that will find those thresholds.

Interesting you mention that. I would like to hear UB's take on today as well. Here is that time frame from my chart (time is pacific). Lots of spikes (down spikes generally signify a buying pressure--its backwards) along that time. Interestingly, the biggest spike of the day so far for me was a sell pressure spike at 9:45 eastern. It then continued up over 5 points after that. Second highest just happened 12:22 eastern suggesting buying pressure. Granted, UB includes other things in his calculation than I do at this point, and I don't think these "false" spikes are an issue for him, if they even show up as proper spikes at all.

use DateTime.Now.Ticks. That gives you to the 100 nanoseconds. Not that it's actually that accurate of course, but... I did some checking at the actual granularity was I think 16 microseconds or something? Can't remember exactly. Not quite. Internet latency is not just a delay, it's a jittery delay. One packet might take 10 ms to get here, the next might take 100 ms to get here. So... Although one might expect some level of average delay to hold true in general (which also will go up and down depending on lots of factors), there will be lots of noise in that delay as well. So... In reality, my guess is the error is pretty high, in the 10's of ms at least, but... I think sequence is pretty reliable since it's using tcp. And if you calculate over a number of ticks coming in, the errors should average out.

I'm fairly sure that UB is doing this calculation as each tick comes in. Given that, the acceleration and such is calculated across ticks, not across bars. The result on the chart is a summary or result of the calculation that occurred during that bar.

I'm interested in brainstorming, sharing ideas about what people have found to be the best programmatic definition of a "pivot" (not like s/r or floor trader pivot, I mean price turned around). Also, by extension, a trend change is just a higher time frame pivot, so discussing definitions for that would be interesting as well. Here are some things I have tried: 1) Min move: specify certain value that it has to move. Once it has moved that far, the previous local extreme is determined to have been a pivot. 2) Min move with min retrace % 3) MA crossover: Pivot is whatever the extreme was between MA crossovers. 4) MACD and bollinger bands, as MACD goes back and forth between/across the bollinger bands, choose extreme based on those "states". 5) There's the n number of bars up/down approach before/after the pivot. Trend change: Typically what I hear is: higher low or lower high or some combination thereof, but that then requires a high confidence in one's choice of pivots. Those are some ideas. One thing that may be lacking in those, though, is a more interpretive approach. How does one determine that the market has given some significance to that candidate pivot? Arbitrarily choosing a move size results in arbitrary pivots. How does one discern that the market has placed significance at a certain turn around?

From what I understand UrmaBlume is saying, it doesn't really matter so much. Whether it hits the bid or ask is not the whole picture to determine buy/sell sentiment. So much is done at the bid and at the ask by replenishing and canceling of orders. I wonder if we need a different term or something. Buy at ask happens: that's "buying", but we recognize that it doesn't necessarily mean overwhelming long sentiment. Because 100 more people might buy at that same ask price but there's a big buy replenishing the ask absorbing it all. So... although, yes there is "buying", the likely future is short because there are some deep pockets who are accumulating short (or exiting long). Umm... I'm going to assume I don't understand what you're saying because it sounds like: if the price goes up then there was net buying. If in any period bid/ask moves up 5 times and down 10 times, that means price has now moved 5 ticks lower (assuming one tick per move).

Interesting. However... After the bid/ask moved up, someone else had to hit the bid in order to fill that large buyer. So... couldn't it still be considered selling volume--but from a different participant? So, you're not measuring naive aggregate buying/selling trade volume, you're measuring buy/sell volume for specific participants or behaviors. I have been watching exactly this recently. It's fun to see huge bid intensity right before the market turns up and vice versa.

This is what I typically use in NT. // In vars section: double lastAsk; double lastBid; // Methods protected override void OnMarketData(MarketDataEventArgs e) { trackBidAsk(e); if (e.MarketDataType == MarketDataType.Last) { double mp = (lastBid + lastAsk) / 2; if (e.Price > mp) { // Transaction was at Ask, do whatever here. } else if (e.Price < mp) { // Transaction was at Bid, do whatever here. } } } private void trackBidAsk(MarketDataEventArgs e) { if (e.MarketDataType == MarketDataType.Bid) { lastBid = e.Price; } else if (e.MarketDataType == MarketDataType.Ask) { lastAsk = e.Price; } }

I know you were directing this at someone else, but your question seems applicable to something I read recently that I found thought provoking: "That is to say, the change of price reflects the average change in traders’ beliefs, while volume reflects the extent of the differences in their beliefs." Some things that come to mind: 1) Overnight there is less efficiency, less people to disagree, more difficult to find an equilibrium. Kind of like in statistics, it's the noise/confusion/inaccuracies that happens when you have insufficient data points. 2) Fewer people involved in the market (and potentially similar types of participants?) thus they tend to agree, so price moves according to how they all agree without much volume 3) Due to the lack of volume, people react more to what little there is. 4) Simply less liquidity, so anyone doing any ordering at all moves the market more. Perhaps 'it is the occurrence of trade that motivates price' might be reworded to: the occurrence of trade provides insight into other agents' motivations. It's interesting that "the market" (meaning bid/ask) could move all over the place without any transaction ever occurring. The large volume in the session is because there are lots of people that "disagree". That disagreement moves the market to a place of temporary and precarious equilibrium: the sideways market. Sorry, slightly incoherent post, it's late, but this just sparked some thought.

Oh, but that old school trader has the fanciest equipment in the known universe: the "little grey cells" (spoken in that delightful Hercule Poirot French accent)

Would you share with me what you mean by "balance of trade"? And I assume you're defining your derivatives with respect to clock time.

So, I was playing with this again. I'm attaching an image. The magenta is price action (each circle is a transaction--FYI: it's 5/28/2009 on the ES, basically random day), the rest is a "trade intensity" calculation I'm trying out (blue is "at the bid", cyan is "at the ask" and green is total). As you can see, it might be interesting. I marked just the 4 biggest spikes, though the others are potentially interesting as well. Anyway... the trade intensity thing intrigues me, and it certainly appears worth further investigation. Thanks again, UrmaBlume.

I don't know if this is why but I think I saw UrmaBlume or someone else mention in another thread that Ninjatrader changes the timestamp that comes in from Zenfire so you no longer get the microsecond granularity. That might be what he's referring to. Not sure why it matters. Zenfire has an API you can interact with directly so any shortcomings of Ninjatrader are irrelevant. Doing the level of analysis that UrmaBlume is doing, I would just go direct with the API. Then you don't have to be stuck in some charting vendor's paradigm of how they think you should be analyzing the market. Paint anything you want (I prototyped a fun zoom panel you can zoom in and out of the market ad infinitum) however you want. You want higher timeframe data together with other data? Trivial in any way, dimension, etc. In fact, you could even define a non-linear time axis. To paraphrase Scotty on Star Trek: "Candlesticks? How quaint".

And Bid/Ask, and T&S, and market depth... they are all just indicators (and writing my own software, that's how I coded it) because they indicate information about something else: the market. "The map is not the territory": the market is not made up of the numbers streaming through the servers--the market is thousands of agents (human or otherwise) that are interacting with each other in a system. The numbers simply "indicate" some information about one or more of those agents at a given moment. Let me use those for a while so I can see if you're right