Jump to content

Welcome to the new Traders Laboratory! Please bear with us as we finish the migration over the next few days. If you find any issues, want to leave feedback, get in touch with us, or offer suggestions please post to the Support forum here.

  • Welcome Guests

    Welcome. You are currently viewing the forum as a guest which does not give you access to all the great features at Traders Laboratory such as interacting with members, access to all forums, downloading attachments, and eligibility to win free giveaways. Registration is fast, simple and absolutely free. Create a FREE Traders Laboratory account here.

O66

Zenfire and DTN Feed Different?

Recommended Posts

I’m using zenfire feed with Investor RT (Linnsoft)

(Zenfire via Mirus futures)

Via Irt I also have backfilling possibility via DTN

 

My charts are quite often completely different after backfill

 

See indicator pane (cumulative delta (bidxask)):

 

http://www.charthub.com/images/2009/12/14/Fulcrum_4

 

and after DTN backfill:

http://www.charthub.com/images/2009/12/14/Fulcrum_5

 

I compared my chart with other traders who are using Zenfire with Ninja charts and they have exactly the same chart as my first chart (before backfill)

My conclusion is that it is not Irt or my connection but there is a difference between Zenfire and DTN

 

Both feeds are known as quality feeds.

 

Any ideas?

Share this post


Link to post
Share on other sites

Interesting. Price data seem to be the same but the indicator differs. How is the indicator calculated? Or is it provided directly by the data providers?

Do you backfill tick-by-tick data? Does backfill contain information about bid and ask on tick-by-tick basis? Maybe the indicator can be calculated properly only on real-time data. I am just guessing, of course.

Share this post


Link to post
Share on other sites

indicator is based on bidxask, but same issue in Ninja with Zenfire

 

charts in Irt and Ninja feeded with zenfie are equal, when backfilling chart in IRT with DTN feed its changing

 

the backfilled chart is equal as chart with DTN used for livefeed

Share this post


Link to post
Share on other sites
I’m using zenfire feed with Investor RT (Linnsoft)

(Zenfire via Mirus futures)

Via Irt I also have backfilling possibility via DTN

 

My charts are quite often completely different after backfill

 

See indicator pane (cumulative delta (bidxask)):

 

Images | ChartHub.com

 

and after DTN backfill:

Images | ChartHub.com

 

I compared my chart with other traders who are using Zenfire with Ninja charts and they have exactly the same chart as my first chart (before backfill)

My conclusion is that it is not Irt or my connection but there is a difference between Zenfire and DTN

 

Both feeds are known as quality feeds.

 

Any ideas?

 

I have been doing some research on this apparent problem now with Zenfire feed linked to Ninjatrader for bid/ask Delta computations, and I think I have found the problem. I should have the verification of what I suspected was the problem on Monday next week....I will pass on the info as soon as I get the answer (and also what the entities involved will be doing to correct their problem).

 

Here is a verified Investor RT with DTN.IQ feed data run of the Delta for that time period.......

 

Images | ChartHub.com

Share this post


Link to post
Share on other sites

I have always been a bit suspicious (but everyone thinks I am just a kill joy :D). From what you say the client implimentation does not seem to be the issue however I am also a little suspicious of how some of the ninja stuff is written. I do believe that it is neccsaary to use onMarketData to get all the market data to make sure that everything is handled syncronously.

 

Incidentaly Globex alway used to reserve the right to aggregate ticks in fast markets too. I am not sure if that is still the case, the last time I looked on their web site I could not find the bulletin. presumably that would not account for differences between data providers.

 

Be very interested to hear what you find out FT ;)

Share this post


Link to post
Share on other sites
I have always been a bit suspicious (but everyone thinks I am just a kill joy :D). From what you say the client implimentation does not seem to be the issue however I am also a little suspicious of how some of the ninja stuff is written. I do believe that it is neccsaary to use onMarketData to get all the market data to make sure that everything is handled syncronously.

 

Incidentaly Globex alway used to reserve the right to aggregate ticks in fast markets too. I am not sure if that is still the case, the last time I looked on their web site I could not find the bulletin. presumably that would not account for differences between data providers.

 

Be very interested to hear what you find out FT ;)

 

I do agree that Ninjatrader is not the most robust option for tracking Delta and other more data intensive work (for instance, the TT Fix Adapter feed has to be somewhat coalesced just to work in NT). The potential problem with Rithmic based Zenfire feed imo may stem from the recent CME data outflow enhancements. I have some in depth research into this matter and I should have my answers by Monday.....this is important imo to figure out since imo Zenfire has been a good option for data feed. As Rithmic offers alternative branded versions of their feed to various charting platforms, it would be nice to know that their feed can be used for proper bid/ask data runs. :)

Share this post


Link to post
Share on other sites
My understanding is that most data feeds will have slightly different data output due to differences in hardware on both your end and their end.

 

The data will only be potentially tainted for very detailed work if it is coalesced......if you have access to uncoalesced data from the CME then you have the perfect set up. Now what you do with that data as it is pumped into your charting application is then your responsibilty to make sure your computational factors are verified proper.

Share this post


Link to post
Share on other sites

Thanks for the links Fulcrum Trader.

 

Having browsed the sites, I'm still a little unsure about the difference between the two. Basically i'm using investor rt and just want a decent broker and feed. Also, if i'm leasing a seat from CME what would be the best options regarding execution. By the way, i'm not leasing a seat just curious to learn more. This area is all a little hazy to me.

 

Sorry if this is the wrong place to ask these questions, but you seemed to know what you're talking about.

 

Thanks so much.

Tony

Share this post


Link to post
Share on other sites

OK......feed update;

 

Rithmic and Zenfire feeds MAY have data drop off as it is routed to a Ninjatrader platform or Investor RT platform as a result of the way they put out the data over the internet (unlike a DTN.IQ feed they do not have any "ticker plant" to verify ALL the data ALL the time before it gets to the end user) since the feed is for basic charting and for order entry use.

 

So it looks as if a person will have to "backfill" data with DTN.IQ feed to make sure your bid/ask data runs are proper, this is a result of the recent CME data output increases to all data providers. It looks as if the recent CME data granularity increase has overwhelmed the basic infrastructure of most retail traders set ups (their internet, their data, and their platforms), if they try to use a broker provided feed for any Cumulative Delta bid/ask volume work.

 

Using a DTN.IQ feed to "backfill" the broker provided feed seems to be the only solution for those who do not use a regular feed into whatever trade/charting platform they have.

Share this post


Link to post
Share on other sites
OK......feed update;

 

Rithmic and Zenfire feeds MAY have data drop off as it is routed to a Ninjatrader platform or Investor RT platform as a result of the way they put out the data over the internet (unlike a DTN.IQ feed they do not have any "ticker plant" to verify ALL the data ALL the time before it gets to the end user) since the feed is for basic charting and for order entry use.

 

 

Thanks for the update.

 

The priorities for a feed that has an order entry engine are somewhat different than those for a simple data feed. Timeliness is more important than completeness. In the old days (again not verified for years) Globex had a kind of two tier setup where clearing house/brokerage type operations would sometimes get more current pricing information at the expense of not getting every tick.

 

Incidentally do we know for sure that bid ask and last changes are fully syncronised for DTN.IQ ? Can we even be sure that the exchange actually provides that information synchronously? (Rather than giving precedent to actual completed order information over quote changes for example).

 

At the end of the day things might be 'good enough' depending on what your objectives are.

Share this post


Link to post
Share on other sites
Thanks for the update.

 

Incidentally do we know for sure that bid ask and last changes are fully syncronised for DTN.IQ ? Can we even be sure that the exchange actually provides that information synchronously? (Rather than giving precedent to actual completed order information over quote changes for example).

 

At the end of the day things might be 'good enough' depending on what your objectives are.

 

Yes.....I have mentioned before that I have verified DTN.IQ feed.....it is the ONLY regular feed that I use and have been able to verify.

 

I do think that CQG probably has exceptional feed too....Esignal does not have clean bid/ask data last I checked and I am not sure why. Tradestations feed is a mess so I will not even go into that one.....LOL!

 

Anyone trying to use OEC or Transact feed will definitely have to "backfill" imo with DTN.IQ to make sure they have clean bid/ask data runs each day. If you use Investor RT or Marketdelta with DTN.IQ feed you are set up perfect for bid/ask data work.

Share this post


Link to post
Share on other sites
Yes.....I have mentioned before that I have verified DTN.IQ feed.....it is the ONLY regular feed that I use and have been able to verify.

 

I do think that CQG probably has exceptional feed too....Esignal does not have clean bid/ask data last I checked and I am not sure why. Tradestations feed is a mess so I will not even go into that one.....LOL!

 

Anyone trying to use OEC or Transact feed will definitely have to "backfill" imo with DTN.IQ to make sure they have clean bid/ask data runs each day. If you use Investor RT or Marketdelta with DTN.IQ feed you are set up perfect for bid/ask data work.

 

I was wondering how you can verify it is complete and synchronised? Against what? You would need co-located ticker plant and even then data loss could be encountered at the exchange (do CME guarentee too provide every bid ask change synchronously with each trade)? Just being my usual sceptical/devils advocate self :) and hopefully encouraging people to think for themselves. (Which O66 the OP is clearly doing!)

 

Tradestation also has some weaknesses in the client (even if the data was up to snuff), I wouldn't go there personally. Also just to add a comment on Multicharts too. There are some issues in the client that have been discussed recently on the TSSupport forum. Possibly fixed in version 6 beta 1 possibly not. A while back I posted some code for doing delta type calculations and various types of block size filtering. A couple of people commented that they where getting inconsistent results on different charts running in the same workspace...... well it turns out that MC can miss ticks as well as not handling bid ask changes synchronously.

 

Anyway sadly it seems that currently we don't have too many options for this type of work though NeoTicker should easily be up to the task as well as IRT/MD at least giving some choice.

Share this post


Link to post
Share on other sites
I was wondering how you can verify it is complete and synchronised? Against what? You would need co-located ticker plant and even then data loss could be encountered at the exchange (do CME guarentee too provide every bid ask change synchronously with each trade)? Just being my usual sceptical/devils advocate self :) and hopefully encouraging people to think for themselves. (Which O66 the OP is clearly doing!)

 

Yes I have mentioned this before I do have access to CME data runs and that is what I use to verify......DTN.IQ feed is as solid as you can get.

Share this post


Link to post
Share on other sites
Yes I have mentioned this before I do have access to CME data runs and that is what I use to verify......DTN.IQ feed is as solid as you can get.

 

That's interesting, I guess by your turn of phrase (data runs) these are histories rather than real time? I wonder what exactly do you mean by 'CME data runs'.... if you don't mind me being nosey that is. Are full order book changes available or just best bid best ask? Are order book changes/best bid best ask changes time stamped or do you need to rely on sequencing (the order of changes)?

 

Or maybe I have it wrong and you have a leased line connection to the CME and are using a comparison to data you are collecting in in real time directly from Globex?

 

Edit: Incidentally FT, as your testing on Zenfire, IQ.Feed and CME data runs have shown no discrepancies I wonder if the issues is actually at O66's end after all:). Quite understandable that the protocol to deliver real time quotes might drop data between zen servers and the client? I understand its fairly well compressed so loosing a packet might actually loose a chunk of data. I would guess it uses UDP rather than TCP.

Edited by BlowFish

Share this post


Link to post
Share on other sites
That's interesting, I guess by your turn of phrase (data runs) these are histories rather than real time? I wonder what exactly do you mean by 'CME data runs'.... if you don't mind me being nosey that is. Are full order book changes available or just best bid best ask? Are order book changes/best bid best ask changes time stamped or do you need to rely on sequencing (the order of changes)?

 

Or maybe I have it wrong and you have a leased line connection to the CME and are using a comparison to data you are collecting in in real time directly from Globex?

 

Edit: Incidentally FT, as your testing on Zenfire, IQ.Feed and CME data runs have shown no discrepancies I wonder if the issues is actually at O66's end after all:). Quite understandable that the protocol to deliver real time quotes might drop data between zen servers and the client? I understand its fairly well compressed so loosing a packet might actually loose a chunk of data. I would guess it uses UDP rather than TCP.

 

A trader can get very granular CME data through brokers/FCM (off their servers) or by paying for higher end feeds.....I will leave it at that. I think Urma Blume has added some good examples of this data feed capability in the past and some of the high monthly costs involved.

 

You are correct that data leaving Zenfire/Rithmic CAN drop data as it is delivered to the end user (so the end users PC capabilities CAN be the cause of SOME data drops). Also, Ninjatrader is not robust enough to catch all data and that CAN also be some of the data drops as the market has periods of high trade rate. So this leaves the need to use a data feed from a regular data provider like DTN.IQ to make sure you get clean bid/ask data for Delta volume work.

Share this post


Link to post
Share on other sites
What is DTN.IQ? Data fee only? Which chart software can use their data?

 

And how is CQG compare to zenfire and DTN.IQ's data completeness/accuracy during real time?

 

Yes, DTN is a widely used and reasonably priced data feed. It is used by several platforms--most of the popular ones support it. I can't answer your other questions though.

Share this post


Link to post
Share on other sites
A trader can get very granular CME data through brokers/FCM (off their servers) or by paying for higher end feeds.....I will leave it at that. I think Urma Blume has added some good examples of this data feed capability in the past and some of the high monthly costs involved.

 

 

Whilst UB contributed some valuable thoughts and insights some of the stuff he wrote, particularly about data feeds, was complete hyperbole. I pulled him on a couple of occasions but did not press the matter as the good stuff far out weighed the more. I always thought it a shame that some one that clearly was well informed with much to contribute on certain topics could talks such nonsense about others.

 

I guess you are talking about things like CQG when you talk of higher end feeds? Probably another good option.

 

From the last few posts it would seem safe to conclude that the problem issue is potentially with any live data feed and infrastructure/capacity limitations. However the original poster suggests that this is not the case and that IQ.Feed matches live and historically. This is odd as Zenfire is fairly highly compressed, it seems to get more ticks in fewer packets so you would expect it to suffer less from data loss. Also your tests would suggest that it matches. Strange.

Share this post


Link to post
Share on other sites

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.

Guest
Reply to this topic...

×   Pasted as rich text.   Paste as plain text instead

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.


  • Topics

  • Posts

    • Thx for reminding us... I don't bang that drum often enough anymore Another part for consideration is who that money initially went to...
    • TDUP ThredUp stock, watch for a top of range breakout above 2.94 at https://stockconsultant.com/?TDUP
    • How long does it take to receive HFM's withdrawal via Skrill? less than 24H?
    • My wife Robin just wanted some groceries.   Simple enough.   She parked the car for fifteen minutes, and returned to find a huge scratch on the side.   Someone keyed her car.   To be clear, this isn’t just any car.   It’s a Cybertruck—Elon Musk's stainless-steel spaceship on wheels. She bought it back in 2021, before Musk became everyone's favorite villain or savior.   Someone saw it parked in a grocery lot and felt compelled to carve their hatred directly into the metal.   That's what happens when you stand out.   Nobody keys a beige minivan.   When you're polarizing, you're impossible to ignore. But the irony is: the more attention something has, the harder it is to find the truth about it.   What’s Elon Musk really thinking? What are his plans? What will happen with DOGE? Is he deserving of all of this adoration and hate? Hard to say.   Ideas work the same way.   Take tariffs, for example.   Tariffs have become the Cybertrucks of economic policy. People either love them or hate them. Even if they don’t understand what they are and how they work. (Most don’t.)   That’s why, in my latest podcast (link below), I wanted to explore the “in-between” truth about tariffs.   And like Cybertrucks, I guess my thoughts on tariffs are polarizing.   Greg Gutfield mentioned me on Fox News. Harvard professors hate me now. (I wonder if they also key Cybertrucks?)   But before I show you what I think about tariffs… I have to mention something.   We’re Headed to Austin, Texas This weekend, my team and I are headed to Austin. By now, you should probably know why.   Yes, SXSW is happening. But my team and I are doing something I think is even better.   We’re putting on a FREE event on “Tech’s Turning Point.”   AI, quantum, biotech, crypto, and more—it’s all on the table.   Just now, we posted a special webpage with the agenda.   Click here to check it out and add it to your calendar.   The Truth About Tariffs People love to panic about tariffs causing inflation.   They wave around the ghost of the Smoot-Hawley Tariff from the Great Depression like it’s Exhibit A proving tariffs equal economic collapse.   But let me pop this myth:   Tariffs don’t cause inflation. And no, I'm not crazy (despite what angry professors from Harvard or Stanford might tweet at me).   Here's the deal.   Inflation isn’t when just a couple of things become pricier. It’s when your entire shopping basket—eggs, shirts, Netflix subscriptions, bananas, everything—starts costing more because your money’s worth less.   Inflation means your dollars aren’t stretching as far as they used to.   Take the 1800s.   For nearly a century, 97% of America’s revenue came from tariffs. Income tax? Didn’t exist. And guess what inflation was? Basically zero. Maybe 1% a year.   The economy was booming, and tariffs funded nearly everything. So, why do people suddenly think tariffs cause inflation today?   Tariffs are taxes on imports, yes, but prices are set by supply and demand—not tariffs.   Let me give you a simple example.   Imagine fancy potato chips from Canada cost $10, and a 20% tariff pushes that to $12. Everyone panics—prices rose! Inflation!   Nope.   If I only have $100 to spend and the price of my favorite chips goes up, I either stop buying chips or I buy, say, fewer newspapers.   If everyone stops buying newspapers because they’re overspending on chips, newspapers lower their prices or go out of business.   Overall spending stays the same, and inflation doesn’t budge.   Three quick scenarios:   We buy pricier chips, but fewer other things: Inflation unchanged. Manufacturers shift to the U.S. to avoid tariffs: Inflation unchanged (and more jobs here). We stop buying fancy chips: Prices drop again. Inflation? Still unchanged. The only thing that actually causes inflation is printing money.   Between 2020 and 2022 alone, 40% of all money ever created in history appeared overnight.   That’s why inflation shot up afterward—not because of tariffs.   Back to tariffs today.   Still No Inflation Unlike the infamous Smoot-Hawley blanket tariff (imagine Oprah handing out tariffs: "You get a tariff, and you get a tariff!"), today's tariffs are strategic.   Trump slapped tariffs on chips from Taiwan because we shouldn’t rely on a single foreign supplier for vital tech components—especially if that supplier might get invaded.   Now Taiwan Semiconductor is investing $100 billion in American manufacturing.   Strategic win, no inflation.   Then there’s Canada and Mexico—our friendly neighbors with weirdly huge tariffs on things like milk and butter (299% tariff on butter—really, Canada?).   Trump’s not blanketing everything with tariffs; he’s pressuring trade partners to lower theirs.   If they do, everybody wins. If they don’t, well, then we have a strategic trade chess game—but still no inflation.   In short, tariffs are about strategy, security, and fairness—not inflation.   Yes, blanket tariffs from the Great Depression era were dumb. Obviously. Today's targeted tariffs? Smart.   Listen to the whole podcast to hear why I think this.   And by the way, if you see a Cybertruck, don’t key it. Robin doesn’t care about your politics; she just likes her weird truck.   Maybe read a good book, relax, and leave cars alone.   (And yes, nobody keys Volkswagens, even though they were basically created by Hitler. Strange world we live in.) Source: https://altucherconfidential.com/posts/the-truth-about-tariffs-busting-the-inflation-myth    Profits from free accurate cryptos signals: https://www.predictmag.com/       
    • No, not if you are comparing apples to apples. What we call “poor” is obviously a pretty high bar but if you’re talking about like a total homeless shambling skexie in like San Fran then, no. The U.S.A. in not particularly kind to you. It is not an abuse so much as it is a sad relatively minor consequence of our optimism and industriousness.   What you consider rich changes with circumstances obviously. If you are genuinely poor in the U.S.A., you experience a quirky hodgepodge of unhelpful and/or abstract extreme lavishnesses while also being alienated from your social support network. It’s about the same as being a refugee. For a fraction of the ‘kindness’ available to you in non bio-available form, you could have simply stayed closer to your people and been MUCH better off.   It’s just a quirk of how we run the place and our values; we are more worried about interfering with people’s liberty and natural inclination to do for themselves than we are about no bums left behind. It is a slightly hurtful position and we know it; we are just scared to death of socialism cancer and we’re willing to put our money where our mouth is.   So, if you’re a bum; you got 5G, the ER will spend like $1,000,000 on you over a hangnail but then kick you out as soon as you’re “stabilized”, the logistics are surpremely efficient, you have total unchecked freedom of speech, real-estate, motels, and jobs are all natural healthy markets in perfect competition, you got compulsory three ‘R’’s, your military owns the sky, sea, space, night, information-space, and has the best hairdos, you can fill out paper and get all the stuff up to and including a Ph.D. Pretty much everything a very generous, eager, flawless go-getter with five minutes to spare would think you might need.   It’s worse. Our whole society is competitive and we do NOT value or make any kumbaya exception. The last kumbaya types we had werr the Shakers and they literally went extinct. Pueblo peoples are still around but they kind of don’t count since they were here before us. So basically, if you’re poor in the U.S.A., you are automatically a loser and a deadbeat too. You will be treated as such by anybody not specifically either paid to deal with you or shysters selling bejesus, Amway, and drugs. Plus, it ain’t safe out there. Not everybody uses muhfreedoms to lift their truck, people be thugging and bums are very vulnerable here. The history of a large mobile workforce means nobody has a village to go home to. Source: https://askdaddy.quora.com/Are-the-poor-people-in-the-United-States-the-richest-poor-people-in-the-world-6   Profits from free accurate cryptos signals: https://www.predictmag.com/ 
×
×
  • Create New...

Important Information

By using this site, you agree to our Terms of Use.