Volume Splitter

[taotree 10]

From what I understand we only get the update message on each trade.

 

Let's say first trade is at 150.05 on the bid and we've never been higher. Then the bid goes 150.06 but there are no trades. and now you get an update message with bid 150.07 ask 150.08 and a trade at 150.06. we wouldn't have 150.06 in our dictionary?

 

If the bid/ask changes were sent independently of the trades then everything would make a lot more sense. Please forgive me if I'm missing something really simple.

 

It depends on your platform. Ninjatrader for example, the bid/ask changes are sent independently of the trades.

 

In many platforms where your indicator is only updated on trades and you call some API to get the inside bid/ask... in some cases (Tradestation, for example) that means you might want to forget about trying to assign bid/ask to trades because it's using snapshotted bid/ask data which are stale. In some platforms, that can be accurate as it sounds like Agekay mentioned he was using one like that in a previous post.

[cunparis 10]

It depends on your platform. Ninjatrader for example, the bid/ask changes are sent independently of the trades.

 

In many platforms where your indicator is only updated on trades and you call some API to get the inside bid/ask... in some cases (Tradestation, for example) that means you might want to forget about trying to assign bid/ask to trades because it's using snapshotted bid/ask data which are stale. In some platforms, that can be accurate as it sounds like Agekay mentioned he was using one like that in a previous post.

 

I'm using Ninjatrader & Market Delta (and Tradestation but not for anything related to bid/ask). So far I've seen a lot of discrepancies in bid/ask data between ninja with zenfire and Market Delta with IQFeed. In a few cases the ninja volume ladder was missing 6% of the volume, in other cases it has more trades at bid than ask for a particular price level when market delta has the opposite. When that happens I've looked at the tradestation time & sales and determined that Market Delta was more correct. I'm not sure if it's the ninja software that has problems, the zen-fire datafeed, or both.

 

The platform can send the bid/ask changes independently but only if the datafeed supports it. I am under the impression that IQFeed sends them together.

[taotree 10]

I'm not sure if it's the ninja software that has problems, the zen-fire datafeed, or both.

 

Search some posts by FulcrumTrader as he has mentioned that he has analyzed zen-fire data and found it does have issues, and apparently Ninjatrader also has some issues with keeping up with data properly sometimes.

[FulcrumTrader 10]

Cunparis, there was this guy on YouTube who would occasionally record his DAX trading and post it there. He was a phenomenal DAX trader. I remember one time where he had over 20 winning trades in a row and he risked less than he made on each trade so that is pretty insane for a volatile market like DAX. If I remember correctly he looked at the FESX chart and somehow always knew where good entries and exits were. He deleted all of his videos though so the only one that is left is this one: YouTube - The DAX Scalping,Oct 19 USA

Yes, he scalps the DAX in a very very similar fashion to how Mark Oryhon would scalp the DAX (used to be the top volume DAX trader based in the US). They actually watch aspects of the equities market internals (groupings of particular stocks that make up the DAX index) and they watch the DJEuroStoxx50 action (much more liquid instrument.....higher liquidity level players).

 

When I am working the DAX as intraday only type scalp/position trades (during the EU cash session hours) then I ALWAYS keep my eye on the FESX. I found I did not need to watch EU markets cash session issues as long as I watched the FESX. I mainly have a way I track the two instruments against each other from a movement of Price/Delta Volume comparison. I usually counter-trend scalp the instrument that overshoots the movement between the two instruments. At other times, I will trend follow with the instrument that hangs for a brief period of time in a covering order flow pullback while the advancing instrument remains in a continuing trending move.

 

I much prefer to make scalp type intraday entries in the DAX, but on occasion I will work trades in the FESX.

Edited March 31, 2010 by FulcrumTrader

[FulcrumTrader 10]

I'm using Ninjatrader & Market Delta (and Tradestation but not for anything related to bid/ask). So far I've seen a lot of discrepancies in bid/ask data between ninja with zenfire and Market Delta with IQFeed. In a few cases the ninja volume ladder was missing 6% of the volume, in other cases it has more trades at bid than ask for a particular price level when market delta has the opposite. When that happens I've looked at the tradestation time & sales and determined that Market Delta was more correct. I'm not sure if it's the ninja software that has problems, the zen-fire datafeed, or both.

 

The platform can send the bid/ask changes independently but only if the datafeed supports it. I am under the impression that IQFeed sends them together.

Yes sir.....Zenfire/Rithmic/TT Fix Adapter/Transact/OEC or most any of the broker supplied feeds are not usable for BID/ASK differential work (Cumulative Delta). I ONLY use DTN.IQ feed for all my Cumulative Delta BID/ASK differential data needs.

[taotree 10]

TT Fix Adapter

 

I thought you said TT Fix could get an uncoalesced version that was good?

[FulcrumTrader 10]

I thought you said TT Fix could get an uncoalesced version that was good?

Yes traders can get access to TT Fix Adapter feed which is uncoalesced, but can they build a mechanism to catch all the BID/ASK data without data loss.....that is the challenge. So the best way to go without having to deal with numerous potential hassles (for the retail trader) is to just use DTN.IQ feed. DTN.IQ feed has a ticker plant with historical lookback of the BID/ASK data with efficient formating.

 

This is the exact reason why TradeVec is setting up connectivity at this time with DTN.IQ feed for those who need clean BID/ASK data. Handling even uncoalesced broker supplied data feeds has challenges for those who need proper BID/ASK data.

[AgeKay 10]

From what I understand we only get the update message on each trade.

 

Let's say first trade is at 150.05 on the bid and we've never been higher. Then the bid goes 150.06 but there are no trades. and now you get an update message with bid 150.07 ask 150.08 and a trade at 150.06. we wouldn't have 150.06 in our dictionary?

 

If the bid/ask changes were sent independently of the trades then everything would make a lot more sense. Please forgive me if I'm missing something really simple.

 

Sorry for leaving out how IQFeed works. This is an implementation detail and the code I posted was just an example and should obviously be adapted to how your feed works. If your feed sends the inside market independently of the trades then you have to update the _marketSideByPrice whenever the inside market changes and then just check for the side when you get the trade. FYI, IQFeed always sends an update message when either the inside market changes or there is a new trade. The message then has a field that tells you what changed. The data that didn't change is also in the message so that's why the example I posted works. The important thing about my example is that it shows you how you can fix the trade without side issue which should work with any feed if slightly adapted.

 

Iare you programming to their API directly? Is their API C#?

 

They have a COM API and TCP sockets. I use the TCP sockets. I don't like using COM interop with .NET because calling COM methods from .NET is extremely slow.

 

Yes sir.....Zenfire/Rithmic/TT Fix Adapter/Transact/OEC or most any of the broker supplied feeds are not usable for BID/ASK differential work (Cumulative Delta). I ONLY use DTN.IQ feed for all my Cumulative Delta BID/ASK differential data needs.

 

I compared my TT feed to IQFeed and noticed that my TT feed is coalesced while IQFeed is not. It depends on how your broker has set it up though. Even though it doesn't matter to me anymore whether they are coalesced or not, it is good to know that IQFeed is not.

[taotree 10]

This is the exact reason why TradeVec is setting up connectivity at this time with DTN.IQ feed for those who need clean BID/ASK data. Handling even uncoalesced broker supplied data feeds has challenges for those who need proper BID/ASK data.

 

Someone (I think on this thread) said that you could get delayed data from dtn for only $15/month. I haven't been able to find this information on their web site, though. Does anyone know where this is stated? Especially if it's possible to get this data through some type of API or download?

[MidKnight 12]

YouTube - The DAX Scalping,Oct 19 USA

 

I agree that guy is awesome. When I first saw that vid sometime last year, there was another guy doing very very similar stuff on the DAX that I thought was even better. Both these guys were uploading around the same time. Sadly they have stopped it seems....Anyhow, here is the other guys channel.

 

YouTube - rimmamed's Channel

[AgeKay 10]

Someone (I think on this thread) said that you could get delayed data from dtn for only $15/month. I haven't been able to find this information on their web site, though. Does anyone know where this is stated? Especially if it's possible to get this data through some type of API or download?

 

I don't think so. Their basic service fee is already $60/month which gives you access to delayed data. You also need to add exchange fees to that for every exchange you need. Access to the API is another $300 or $600 per year, don't remember the price. It think it was $600 at some point and they lowered it to $300, but I might be wrong.

[cunparis 10]

I don't think so. Their basic service fee is already $60/month which gives you access to delayed data. You also need to add exchange fees to that for every exchange you need. Access to the API is another $300 or $600 per year, don't remember the price. It think it was $600 at some point and they lowered it to $300, but I might be wrong.

 

Investor RT offers "DTN Market Access" which is historical only data from DTN for $15/month. I used it when I was trying out Investor RT and it works great. I was using zen-fire & transact as my primary data feeds and I would periodically reload the data with DTN data. That's when I discovered how different zen-fire & transact data was from DTN. Often I'd see divergences that would disappear when I reloaded with DTN. And often I'd see divergences form too. So that's when I decided it was worth the money to get IQFeed.

 

The same service is available with Market Delta but it's $25. I guess MD takes $10/month and then IRT takes the $15 and splits it with DTN. That's why MD is so expensive btw because you pay for IRT + the MD add on, which I find worth it for the software but for the DTN MA it's a bit extra cause MD adds no value there.

 

IQFeed participates in the CME exchange fee waiver program. I am paying the basic service plus Eurex exchange fees and I get all Globex (Nymex, CME, & Comex) free. That saved me around $180/month.

[taotree 10]

Investor RT offers "DTN Market Access" which is historical only data from DTN for $15/month.

 

So... I wonder if there is a way to get this without paying the fee for Investor/RT?Any chance I could get this for Ninjatrader, I wonder... DTN says $60/month for basic service, how are you avoiding that?

[BlowFish 10]

What must be happening internally with IQFeed is that they update the inside market before the trade so the bid is already 1 tick lower before the trade is reported.

 

This sounds like a similar problem to how Tradestation & Multicharts implement insidebid insideask but in the feed implementation. Last is not properly synchronised with bid / ask and race conditions can occur.

 

I am not sure your solution will get round the issue unless DTN send all order book changes (properly sequenced with BB & BA changes and no aggregation) as well as trades? What if the BB or BA moves due to order cancellation in between 2 trades?

 

At the end of the day most these indicators use delta as a proxy for order flow/trade direction (as per Lee & Ready). Research suggests that these techniques are about 10-15% more efficacious at this task than using upticks/downticks, However if errors in the data are greater than that sort of magnitude one may be better using the latter technique.

 

Interesting stuff AK.

[BlowFish 10]

Deleted......clarified later in the thread

Edited March 31, 2010 by BlowFish

[taotree 10]

This sounds like a similar problem to how Tradestation & Multicharts implement insidebid insideask but in the feed implementation. Last is not properly synchronised with bid / ask and race conditions can occur.

 

I think it's quite different. Explained below.

 

 

I am not sure your solution will get round the issue unless DTN send all order book changes (properly sequenced with BB & BA changes and no aggregation) as well as trades? What if the BB or BA moves due to order cancellation in between 2 trades?

 

Based on FulcrumTrader's statements, DTN.IQ does send all order book changes properly sequenced. Now... given the situation in question let's consider, what IS the proper sequence?

 

A trade comes across that will take out all the rest of the bid. Is it correct to show the trade first, or the bid update first? And this goes for all the time--should the trade show up first, or the book change first? From a programming perspective... I would remove the bid before I processed the trade, otherwise there might be potential synchronization issues. I grab the amount off the bid to "hold" it while I process the order. Kind of like how someone puts an authorization on your credit card pre-sale and then charging it post-sale.

 

Now, in the data feed, that low level of trade stuff does not need to be reflected, but... It seems quite reasonable to show book updates prior to transactions. It could go either way--so long as you are consistent and always do it one way or the other.

 

I would be interested... in the low level data of dtn.iq--are book updates always given before transactions? So a trade of 10 at bid will be always immediately preceded by a reduction in the bid size by 10?

[BlowFish 10]

Search some posts by FulcrumTrader as he has mentioned that he has analyzed zen-fire data and found it does have issues, and apparently Ninjatrader also has some issues with keeping up with data properly sometimes.

 

The issues is that it uses UDP and so there is the potential to loose packets. This does not happen (or happens rarely) with good infrastructure. I am also not convinced by the test methodology and data set used as a benchmark, FT does not seem to want to discuss that.

 

Anyway loosing the odd packet is a far lesser evil than not having things sequenced correctly (wherever that occurs). There's the thing you have 3 asynchronous event streams (bid ask and last) they need to get sequenced.

Edited March 31, 2010 by BlowFish

[BlowFish 10]

I think it's quite different. Explained below.

 

 

 

Thanks for clarifying. Personally I would report the trade first without the trade the inventory level would not decrease. There are arguments for the other way however I wonder how the exchange does it? As they are liquidity motivated maybe they prioritise order book changes?

[taotree 10]

The issues is that it uses UDP and so there is the potential to loose packets. This does not happen (or happens rarely) with good infrastructure. I am also not convinced by the test methodology and data set used as a benchmark, FT does not seem to want to discuss that.

 

Anyway loosing the odd packet is a far lesser evil than not having things sequenced correctly (wherever that occurs). There's the thing you have 3 asynchronous event streams (bid ask and last) they need to get sequenced.

 

I have looked at event data for zen-fire and can say that there are definitely things that don't add up. Something like bid is at 19, there's a trade at same bid price of 25 and next update is bid is at 43 at same price. Obviously, there should have been some updates in the middle in order for that to have happened but they didn't show up in the feed I received.

 

Also, I can attest to data arriving out of sequence.

[BlowFish 10]

I have looked at event data for zen-fire and can say that there are definitely things that don't add up. Something like bid is at 19, there's a trade at same bid price of 25 and next update is bid is at 43 at same price. Obviously, there should have been some updates in the middle in order for that to have happened but they didn't show up in the feed I received.

 

Also, I can attest to data arriving out of sequence.

 

That sounds like a lost packet(s) to me?

[cunparis 10]

At the end of the day most these indicators use delta as a proxy for order flow/trade direction (as per Lee & Ready). Research suggests that these techniques are about 10-15% more efficacious at this task than using upticks/downticks, However if errors in the data are greater than that sort of magnitude one may be better using the latter technique.

 

Fascinating post there BlowFish! As you know, Tradestation doesn't have bid/ask but has uptick/downtick. I do use uptick/downtick on my tradestation charts but since I have been trying Investor RT & now Market Delta I've been putting more confidence in the bid/ask delta.

 

I've done a lot of comparisons by showing just delta on both platforms. In many cases they're very similar and then now and then they'll differ. Sorry but I don't have any screen shots to illustrate this. These small differences really add up when summing the delta as in cumulative delta or the volume breakdown "delta momentum" indicator.

 

You raise a good point about bid/ask in that is it really worth it to have bid/ask instead of the more reliable uptickd/downtick? For now I have found IQFeed very reliable when I compare it to the time & sales of tradestation. I use TS's T&S simply because I can get it historically so it's easy to go back and check.

 

Here is a ladder comparison between Ninjatrader with GOM Ladder & Zen-Fire & Market Delta with IQFeed. Look at the circled area. IQFeed & Ninja have totally different results. Interestingly enough, the price above and below they are almost the same.

 

This kind of error could cause a trader to enter or exit based on incorrect information and just one mistake could cost more than the subscription to Market Delta & IQFeed!

[cunparis 10]

So... I wonder if there is a way to get this without paying the fee for Investor/RT?Any chance I could get this for Ninjatrader, I wonder... DTN says $60/month for basic service, how are you avoiding that?

 

The DTN MA is only available for Investor RT. It requires the IRT software so you can't use it with any other platform. IRT has negotiated a really good partnership with DTN for this service. IRT bills for it and you can add it simply by checking a box. It was very nice.

 

I forget what I pay for DTN IQ but I think it's around $100/month. So with Market Delta the DTN MA is $25 so I figure it's worth the extra $75 to have IQ realtime and avoid the data problems shown in my previous post.

[BlowFish 10]

 

I've done a lot of comparisons by showing just delta on both platforms. In many cases they're very similar and then now and then they'll differ. Sorry but I don't have any screen shots to illustrate this. These small differences really add up when summing the delta as in cumulative delta or the volume breakdown "delta momentum" indicator.

 

 

I would be less worried if using a delta oscillator and divergence (take a look at 5pillars posts from a few years back at ET if that approach appeals). I have to say that does not interest me particularly though it appears to 'work'. For cumulative work the issue is more of a concern as you point out.

 

I wonder if you have compared up/down ticks with delta? In theory they should be similar. Doh forget that ....seems like thats what you are using on TS.

[BlowFish 10]

The DTN MA is only available for Investor RT. It requires the IRT software so you can't use it with any other platform. IRT has negotiated a really good partnership with DTN for this service. IRT bills for it and you can add it simply by checking a box. It was very nice.

 

I forget what I pay for DTN IQ but I think it's around $100/month. So with Market Delta the DTN MA is $25 so I figure it's worth the extra $75 to have IQ realtime and avoid the data problems shown in my previous post.

 

Not to picky but I believe Ensign has a similar deal and that it's included in the lease of the software....in fact they where the first that I was aware of.

[FulcrumTrader 10]

FT does not seem to want to discuss that.

Yes....a $2,000 a month feed.

 

DTN.IQ feed passes the verification test on a week by week basis......none of the broker supplied feeds do from all the various testing done.