Zenfire and DTN Feed Different?

[O66 10]

I’m using zenfire feed with Investor RT (Linnsoft)

(Zenfire via Mirus futures)

Via Irt I also have backfilling possibility via DTN

 

My charts are quite often completely different after backfill

 

See indicator pane (cumulative delta (bidxask)):

 

http://www.charthub.com/images/2009/12/14/Fulcrum_4

 

and after DTN backfill:

http://www.charthub.com/images/2009/12/14/Fulcrum_5

 

I compared my chart with other traders who are using Zenfire with Ninja charts and they have exactly the same chart as my first chart (before backfill)

My conclusion is that it is not Irt or my connection but there is a difference between Zenfire and DTN

 

Both feeds are known as quality feeds.

 

Any ideas?

[Tams 20]

they look the same to me

[O66 10]

really?

 

see this:

http://www.tradersinfo.net/jing/2009-12-18_2106.png

[Head2k 10]

Interesting. Price data seem to be the same but the indicator differs. How is the indicator calculated? Or is it provided directly by the data providers?

Do you backfill tick-by-tick data? Does backfill contain information about bid and ask on tick-by-tick basis? Maybe the indicator can be calculated properly only on real-time data. I am just guessing, of course.

[O66 10]

indicator is based on bidxask, but same issue in Ninja with Zenfire

 

charts in Irt and Ninja feeded with zenfie are equal, when backfilling chart in IRT with DTN feed its changing

 

the backfilled chart is equal as chart with DTN used for livefeed

[FulcrumTrader 10]

I’m using zenfire feed with Investor RT (Linnsoft)

(Zenfire via Mirus futures)

Via Irt I also have backfilling possibility via DTN

 

My charts are quite often completely different after backfill

 

See indicator pane (cumulative delta (bidxask)):

 

Images | ChartHub.com

 

and after DTN backfill:

Images | ChartHub.com

 

I compared my chart with other traders who are using Zenfire with Ninja charts and they have exactly the same chart as my first chart (before backfill)

My conclusion is that it is not Irt or my connection but there is a difference between Zenfire and DTN

 

Both feeds are known as quality feeds.

 

Any ideas?

 

I have been doing some research on this apparent problem now with Zenfire feed linked to Ninjatrader for bid/ask Delta computations, and I think I have found the problem. I should have the verification of what I suspected was the problem on Monday next week....I will pass on the info as soon as I get the answer (and also what the entities involved will be doing to correct their problem).

 

Here is a verified Investor RT with DTN.IQ feed data run of the Delta for that time period.......

 

Images | ChartHub.com

[diablo272 10]

My understanding is that most data feeds will have slightly different data output due to differences in hardware on both your end and their end.

[BlowFish 10]

I have always been a bit suspicious (but everyone thinks I am just a kill joy ). From what you say the client implimentation does not seem to be the issue however I am also a little suspicious of how some of the ninja stuff is written. I do believe that it is neccsaary to use onMarketData to get all the market data to make sure that everything is handled syncronously.

 

Incidentaly Globex alway used to reserve the right to aggregate ticks in fast markets too. I am not sure if that is still the case, the last time I looked on their web site I could not find the bulletin. presumably that would not account for differences between data providers.

 

Be very interested to hear what you find out FT

[FulcrumTrader 10]

I have always been a bit suspicious (but everyone thinks I am just a kill joy ). From what you say the client implimentation does not seem to be the issue however I am also a little suspicious of how some of the ninja stuff is written. I do believe that it is neccsaary to use onMarketData to get all the market data to make sure that everything is handled syncronously.

 

Incidentaly Globex alway used to reserve the right to aggregate ticks in fast markets too. I am not sure if that is still the case, the last time I looked on their web site I could not find the bulletin. presumably that would not account for differences between data providers.

 

Be very interested to hear what you find out FT

 

I do agree that Ninjatrader is not the most robust option for tracking Delta and other more data intensive work (for instance, the TT Fix Adapter feed has to be somewhat coalesced just to work in NT). The potential problem with Rithmic based Zenfire feed imo may stem from the recent CME data outflow enhancements. I have some in depth research into this matter and I should have my answers by Monday.....this is important imo to figure out since imo Zenfire has been a good option for data feed. As Rithmic offers alternative branded versions of their feed to various charting platforms, it would be nice to know that their feed can be used for proper bid/ask data runs.

[FulcrumTrader 10]

My understanding is that most data feeds will have slightly different data output due to differences in hardware on both your end and their end.

 

The data will only be potentially tainted for very detailed work if it is coalesced......if you have access to uncoalesced data from the CME then you have the perfect set up. Now what you do with that data as it is pumped into your charting application is then your responsibilty to make sure your computational factors are verified proper.

[mikka 10]

Can anyone please give me some information on Zenfire datafeed i.e an introduction to it and advantages/disadvantages.

 

Thanks.

[FulcrumTrader 10]

Zenfire brand feed....... Zen-Fire. The Ultimate Connection.

 

Built by Rithmic...... Rithmic - Home

[mikka 10]

Thanks for the links Fulcrum Trader.

 

Having browsed the sites, I'm still a little unsure about the difference between the two. Basically i'm using investor rt and just want a decent broker and feed. Also, if i'm leasing a seat from CME what would be the best options regarding execution. By the way, i'm not leasing a seat just curious to learn more. This area is all a little hazy to me.

 

Sorry if this is the wrong place to ask these questions, but you seemed to know what you're talking about.

 

Thanks so much.

Tony

[FulcrumTrader 10]

OK......feed update;

 

Rithmic and Zenfire feeds MAY have data drop off as it is routed to a Ninjatrader platform or Investor RT platform as a result of the way they put out the data over the internet (unlike a DTN.IQ feed they do not have any "ticker plant" to verify ALL the data ALL the time before it gets to the end user) since the feed is for basic charting and for order entry use.

 

So it looks as if a person will have to "backfill" data with DTN.IQ feed to make sure your bid/ask data runs are proper, this is a result of the recent CME data output increases to all data providers. It looks as if the recent CME data granularity increase has overwhelmed the basic infrastructure of most retail traders set ups (their internet, their data, and their platforms), if they try to use a broker provided feed for any Cumulative Delta bid/ask volume work.

 

Using a DTN.IQ feed to "backfill" the broker provided feed seems to be the only solution for those who do not use a regular feed into whatever trade/charting platform they have.

[BlowFish 10]

OK......feed update;

 

Rithmic and Zenfire feeds MAY have data drop off as it is routed to a Ninjatrader platform or Investor RT platform as a result of the way they put out the data over the internet (unlike a DTN.IQ feed they do not have any "ticker plant" to verify ALL the data ALL the time before it gets to the end user) since the feed is for basic charting and for order entry use.

 

 

Thanks for the update.

 

The priorities for a feed that has an order entry engine are somewhat different than those for a simple data feed. Timeliness is more important than completeness. In the old days (again not verified for years) Globex had a kind of two tier setup where clearing house/brokerage type operations would sometimes get more current pricing information at the expense of not getting every tick.

 

Incidentally do we know for sure that bid ask and last changes are fully syncronised for DTN.IQ ? Can we even be sure that the exchange actually provides that information synchronously? (Rather than giving precedent to actual completed order information over quote changes for example).

 

At the end of the day things might be 'good enough' depending on what your objectives are.

[FulcrumTrader 10]

Thanks for the update.

 

Incidentally do we know for sure that bid ask and last changes are fully syncronised for DTN.IQ ? Can we even be sure that the exchange actually provides that information synchronously? (Rather than giving precedent to actual completed order information over quote changes for example).

 

At the end of the day things might be 'good enough' depending on what your objectives are.

 

Yes.....I have mentioned before that I have verified DTN.IQ feed.....it is the ONLY regular feed that I use and have been able to verify.

 

I do think that CQG probably has exceptional feed too....Esignal does not have clean bid/ask data last I checked and I am not sure why. Tradestations feed is a mess so I will not even go into that one.....LOL!

 

Anyone trying to use OEC or Transact feed will definitely have to "backfill" imo with DTN.IQ to make sure they have clean bid/ask data runs each day. If you use Investor RT or Marketdelta with DTN.IQ feed you are set up perfect for bid/ask data work.

[BlowFish 10]

Yes.....I have mentioned before that I have verified DTN.IQ feed.....it is the ONLY regular feed that I use and have been able to verify.

 

I do think that CQG probably has exceptional feed too....Esignal does not have clean bid/ask data last I checked and I am not sure why. Tradestations feed is a mess so I will not even go into that one.....LOL!

 

Anyone trying to use OEC or Transact feed will definitely have to "backfill" imo with DTN.IQ to make sure they have clean bid/ask data runs each day. If you use Investor RT or Marketdelta with DTN.IQ feed you are set up perfect for bid/ask data work.

 

I was wondering how you can verify it is complete and synchronised? Against what? You would need co-located ticker plant and even then data loss could be encountered at the exchange (do CME guarentee too provide every bid ask change synchronously with each trade)? Just being my usual sceptical/devils advocate self and hopefully encouraging people to think for themselves. (Which O66 the OP is clearly doing!)

 

Tradestation also has some weaknesses in the client (even if the data was up to snuff), I wouldn't go there personally. Also just to add a comment on Multicharts too. There are some issues in the client that have been discussed recently on the TSSupport forum. Possibly fixed in version 6 beta 1 possibly not. A while back I posted some code for doing delta type calculations and various types of block size filtering. A couple of people commented that they where getting inconsistent results on different charts running in the same workspace...... well it turns out that MC can miss ticks as well as not handling bid ask changes synchronously.

 

Anyway sadly it seems that currently we don't have too many options for this type of work though NeoTicker should easily be up to the task as well as IRT/MD at least giving some choice.

[BlowFish 10]

Just remembered....I heard a rumour that zenfire was going to be offering a days history (on the multicharts forums) this would suggest they are introducing some sort of ticker plant.

 

Sounds like its kind of new TS SUPPORT :: View topic - FYI: Rithmic (Zen-fire) provides historical data ...

[FulcrumTrader 10]

I was wondering how you can verify it is complete and synchronised? Against what? You would need co-located ticker plant and even then data loss could be encountered at the exchange (do CME guarentee too provide every bid ask change synchronously with each trade)? Just being my usual sceptical/devils advocate self and hopefully encouraging people to think for themselves. (Which O66 the OP is clearly doing!)

 

Yes I have mentioned this before I do have access to CME data runs and that is what I use to verify......DTN.IQ feed is as solid as you can get.

[FulcrumTrader 10]

Just remembered....I heard a rumour that zenfire was going to be offering a days history (on the multicharts forums) this would suggest they are introducing some sort of ticker plant.

 

Sounds like its kind of new TS SUPPORT :: View topic - FYI: Rithmic (Zen-fire) provides historical data ...

 

10 days minimum would sure be nice!

[BlowFish 10]

Yes I have mentioned this before I do have access to CME data runs and that is what I use to verify......DTN.IQ feed is as solid as you can get.

 

That's interesting, I guess by your turn of phrase (data runs) these are histories rather than real time? I wonder what exactly do you mean by 'CME data runs'.... if you don't mind me being nosey that is. Are full order book changes available or just best bid best ask? Are order book changes/best bid best ask changes time stamped or do you need to rely on sequencing (the order of changes)?

 

Or maybe I have it wrong and you have a leased line connection to the CME and are using a comparison to data you are collecting in in real time directly from Globex?

 

Edit: Incidentally FT, as your testing on Zenfire, IQ.Feed and CME data runs have shown no discrepancies I wonder if the issues is actually at O66's end after all:). Quite understandable that the protocol to deliver real time quotes might drop data between zen servers and the client? I understand its fairly well compressed so loosing a packet might actually loose a chunk of data. I would guess it uses UDP rather than TCP.

Edited December 22, 2009 by BlowFish

[FulcrumTrader 10]

That's interesting, I guess by your turn of phrase (data runs) these are histories rather than real time? I wonder what exactly do you mean by 'CME data runs'.... if you don't mind me being nosey that is. Are full order book changes available or just best bid best ask? Are order book changes/best bid best ask changes time stamped or do you need to rely on sequencing (the order of changes)?

 

Or maybe I have it wrong and you have a leased line connection to the CME and are using a comparison to data you are collecting in in real time directly from Globex?

 

Edit: Incidentally FT, as your testing on Zenfire, IQ.Feed and CME data runs have shown no discrepancies I wonder if the issues is actually at O66's end after all:). Quite understandable that the protocol to deliver real time quotes might drop data between zen servers and the client? I understand its fairly well compressed so loosing a packet might actually loose a chunk of data. I would guess it uses UDP rather than TCP.

 

A trader can get very granular CME data through brokers/FCM (off their servers) or by paying for higher end feeds.....I will leave it at that. I think Urma Blume has added some good examples of this data feed capability in the past and some of the high monthly costs involved.

 

You are correct that data leaving Zenfire/Rithmic CAN drop data as it is delivered to the end user (so the end users PC capabilities CAN be the cause of SOME data drops). Also, Ninjatrader is not robust enough to catch all data and that CAN also be some of the data drops as the market has periods of high trade rate. So this leaves the need to use a data feed from a regular data provider like DTN.IQ to make sure you get clean bid/ask data for Delta volume work.

[MoreYummy 10]

What is DTN.IQ? Data fee only? Which chart software can use their data?

 

And how is CQG compare to zenfire and DTN.IQ's data completeness/accuracy during real time?

[diablo272 10]

What is DTN.IQ? Data fee only? Which chart software can use their data?

 

And how is CQG compare to zenfire and DTN.IQ's data completeness/accuracy during real time?

 

Yes, DTN is a widely used and reasonably priced data feed. It is used by several platforms--most of the popular ones support it. I can't answer your other questions though.

[BlowFish 10]

A trader can get very granular CME data through brokers/FCM (off their servers) or by paying for higher end feeds.....I will leave it at that. I think Urma Blume has added some good examples of this data feed capability in the past and some of the high monthly costs involved.

 

 

Whilst UB contributed some valuable thoughts and insights some of the stuff he wrote, particularly about data feeds, was complete hyperbole. I pulled him on a couple of occasions but did not press the matter as the good stuff far out weighed the more. I always thought it a shame that some one that clearly was well informed with much to contribute on certain topics could talks such nonsense about others.

 

I guess you are talking about things like CQG when you talk of higher end feeds? Probably another good option.

 

From the last few posts it would seem safe to conclude that the problem issue is potentially with any live data feed and infrastructure/capacity limitations. However the original poster suggests that this is not the case and that IQ.Feed matches live and historically. This is odd as Zenfire is fairly highly compressed, it seems to get more ticks in fewer packets so you would expect it to suffer less from data loss. Also your tests would suggest that it matches. Strange.