Volume Splitter

[duhhhh 10]

Anyone has tried to save ask bid data into SQL server using API ??

If it could , can we use this information with multicharts or tradestation ??

I have plan to setup DB server for this..

[cunparis 10]

  duhhhh said:

Anyone has tried to save ask bid data into SQL server using API ??

If it could , can we use this information with multicharts or tradestation ??

I have plan to setup DB server for this..

 

One problem with tradestation is it uses minutes for the timestamp. It doesn't even go to second.

 

For bid ask I think Investor RT or Market Delta are really the best.

[AgeKay 10]

  paolfili said:

I'm working with ZenFire API.

You can have the Bid/Ask and Best_Bid/Best_Ask data,but....

But UDP streams can loose data.

If you cannot afford a very High quality connection (as the usual Rythmic customers can probably afford) you have not guaranteed sync (and no value from any CumulativeDelta analysis... if you believe in any value about CumulativeDelta analysis...).

 

Is there even a feed that guarantees receipt of all data? Do you even want that? X_Trader API doesn't, and there is a good reason for it. They told me that most customers prefer to get the most up-to-date data than all data which might be delayed when there is a lot of data at one point in time. It makes sense if you think about it. You don't want to stale information (e.g. old inside market) even if it is complete. I guess one problem to solve this would be to have 2 feeds, one feed that guarantees the most recent data that you can use for quick decisions and execution (e.g. for MD Trader) and another feed that guarantees that you get all data for processing (e.g. for data analysis). TT also has a FIX API and the FIX server can be configured to your needs so that might be possible there, but I haven't had time to closely look at it.

 

Can you enumerate Zen-Fire's advantages/disadvantages. I thought about using them but there aren't any brokers with decent commissions that offer Zen-Fire.

 

  paolfili said:

DTN seems TCP.

 

Yes, DTN is TCP/IP. I still have access to their developer documenation which is not free.

 

  paolfili said:

Someone can add something on this matter about TT' s API?

 

See above.

 

  paolfili said:

Can you explain the latency issue between Ninja/IQFeed?

 

I was comparing Eurex Futures with Zen-Fire in Ninja Trader, IQFeed and TT's feed. Zen-Fire and TT were basically identical in terms of speed while IQFeed lagged by about 1-2 seconds. Unacceptable for any kind of trading really. I generally had the feeling that IQFeed was the discount airline of data feeds. A trader friend of mine who used to use them told me that it even went down for half a trading day while you would never have this problem with TT. There is a reason half of Futures' volume goes through TT and that you see MD Trader (part of X_Trader) on every trading screen on every trading floor.

 

  BlowFish said:

Historical data. They generally pick markets where there is a historical database available where trade direction is recorded.

 

Then why did you bring them up? You can't use their papers to prove that there are "mid-point" trades that don't trade at the bid or ask if they look at trades (spread trades, OTC trades) we don't even see when live trading. So far you have only confirmed what I have said so I don't get why we are still arguing.

[paolfili 10]

Is there even a feed that guarantees receipt of all data? Do you even want that? X_Trader API doesn't, and there is a good reason for it. They told me that most customers prefer to get the most up-to-date data than all data which might be delayed when there is a lot of data at one point in time. It makes sense if you think about it. You don't want to stale information (e.g. old inside market) even if it is complete. I guess one problem to solve this would be to have 2 feeds, one feed that guarantees the most recent data that you can use for quick decisions and execution (e.g. for MD Trader) and another feed that guarantees that you get all data for processing (e.g. for data analysis). TT also has a FIX API and the FIX server can be configured to your needs so that might be possible there, but I haven't had time to closely look at it.

 

As always, It depends.

If your signals/oscillators/edges are based on 1/2 seconds timeframe (and you cannot afford to loose 5 seconds to send the order) you are right.

Anyway having ALL the data you can explore(in the ExploratoryDataAnalisys meaning)/backtest/observe what you cannot see in partial data.

(The Best Bid/Ask - Trade sync problem of Zenfire is an example).

You talk about Book Visualization,but if your Book data are partial the Visualization

is based on partial data....probably the data you get is enough for your needs, but always partial.

 

Can you enumerate Zen-Fire's advantages/disadvantages. I thought about using them but there aren't any brokers with decent commissions that offer Zen-Fire.

The main disadvantage is (for me) the impossibility to have correct Cumulative Delta.

But you have the (partial) data about all levels on the Book if you are interested in book' s tricks.(don' t' know about X_Trader API book).

I think it' s fast enough,doesn' t coalesce data,and offer (from API) a microsecond timestamp.The backoffice "chain" (for me) is short enough(double login account etc).

On the other side is probably used from a very little group of customers (from the small number of forum' s posts) and the "developer" support is close to 0.

On the next weeks probably I 'll can have a deep look to X_Trader API (via TTNET),so any comparision will be more meaningfull.

 

 

 

  AgeKay said:

 

 

Yes, DTN is TCP/IP. I still have access to their developer documenation which is not free.

 

 

 

See above.

 

 

 

I was comparing Eurex Futures with Zen-Fire in Ninja Trader, IQFeed and TT's feed. Zen-Fire and TT were basically identical in terms of speed while IQFeed lagged by about 1-2 seconds. Unacceptable for any kind of trading really. I generally had the feeling that IQFeed was the discount airline of data feeds. A trader friend of mine who used to use them told me that it even went down for half a trading day while you would never have this problem with TT. There is a reason half of Futures' volume goes through TT and that you see MD Trader (part of X_Trader) on every trading screen on every trading floor.

 

 

 

Then why did you bring them up? You can't use their papers to prove that there are "mid-point" trades that don't trade at the bid or ask if they look at trades (spread trades, OTC trades) we don't even see when live trading. So far you have only confirmed what I have said so I don't get why we are still arguing.

[cunparis 10]

  paolfili said:

The main disadvantage is (for me) the impossibility to have correct Cumulative Delta.

 

Can you explain why the cumulative delta is incorrect with zen-fire? I've been comparing CD with Transact, Zen-fire, & DTN. Transact was not very accurate. Zen-fire seems to be pretty close to DTN. I cannot say for sure that DTN is more accurate but I do trust fulcrum trader who has examined the DTN feeds and claims they are most accurate.

 

What I'm interested in is knowing how zen-fire falls short. I'm not sure it's worth it for me to subscribe to IQFeed.

 

AgeKay & Blowfish - your discussions are respectful and very informative. I'm learning a lot. Thank you both for sharing. In fact thanks to everyone for a great thread.

[cunparis 10]

I saw a trade take place between the bid & ask on the Tradestation time & sales.

 

[BlowFish 10]

  AgeKay said:

Then why did you bring them up? You can't use their papers to prove that there are "mid-point" trades that don't trade at the bid or ask if they look at trades (spread trades, OTC trades) we don't even see when live trading. So far you have only confirmed what I have said so I don't get why we are still arguing.

 

Not arguing discussing , I am quite happy to respectfully disagree:D.

 

The reason I bought them up as a) it was Lee & Readys work 20 years ago that gave rise to this whole line of bid/ask analysis. (Arguably Wyckoff sowed the seeds back in 1910 when he penned 'Studies in Tape Reading' under the pseudonym Rollo Tape). Secondly a couple of the reports conclude that the greatest source of inaccuracies in classification occur when a trade is reported between bid and ask.

 

Anyway it's probably better to let that drop at this stage.

 

Getting back on track re splitting volume. There is one paper particularly relevant to this thread as it specifically deals with order size as well as trade direction. Caught On Tape: Predicting Institutional Ownership With Order Flow. To cut to the chase for those that don't read papers they conclude that institutions use both large orders and high frequency small orders to increase or diminish holdings (within the dataset they used of course). The interwebz becomes more and more remarkable almost daily it seems. Some pretty interesting stuff is there if you sniff around a bit.

 

  AgeKay said:

Is there even a feed that guarantees receipt of all data?

 

Is there even an exchange that guarantees the transmission of all data. I was told that CME reserve the right to aggregate data. Haven't verified that for myself but it seems only prudent for exchanges to protect themselves against certain eventualities.

 

As some one else mentioned would you rather have timely data or complete data? Is it unreasonable to expect both with top grade retail infrastructure?

[BlowFish 10]

Just one more thing as Columbo might say.

 

Seems to me a great working solution would be R|thmic / Zenfire or TT for live data and IQFeed for historical. I have not got round to testing the completeness of IQFeeds historical data but there are couple of people here that swear by it.

[paolfili 10]

http://www.traderslaboratory.com/forums/208/zenfire-dtn-feed-different-7301-4.html#post84940

 

  cunparis said:

Can you explain why the cumulative delta is incorrect with zen-fire? I've been comparing CD with Transact, Zen-fire, & DTN. Transact was not very accurate. Zen-fire seems to be pretty close to DTN. I cannot say for sure that DTN is more accurate but I do trust fulcrum trader who has examined the DTN feeds and claims they are most accurate.

 

What I'm interested in is knowing how zen-fire falls short. I'm not sure it's worth it for me to subscribe to IQFeed.

 

AgeKay & Blowfish - your discussions are respectful and very informative. I'm learning a lot. Thank you both for sharing. In fact thanks to everyone for a great thread.

[AgeKay 10]

@cunparis: That just means that the TradeStation feed didn't get the new best bid or ask before it got the trade. My application would crash if I got a trade that did not trade at the best bid or ask. So far it hasn't.

[cunparis 10]

  BlowFish said:

Just one more thing as Columbo might say.

 

Seems to me a great working solution would be R|thmic / Zenfire or TT for live data and IQFeed for historical. I have not got round to testing the completeness of IQFeeds historical data but there are couple of people here that swear by it.

 

That's what I'm doing now. When I reload the data which loads it from DTN, the delta bars change. Sometimes divergences will appear and sometimes divergences that were there will disappear. It's quite frustrating. I'd rather pay for DTN data and not have to do that.

[BlowFish 10]

  AgeKay said:

My application would crash if I got a trade that did not trade at the best bid or ask. So far it hasn't.

 

Leaving aside whether you can legitimately get a trade that does not exactly match BB/BA this strikes me as rather unnecessary. Robust exception handling is pretty important imho. An unfiltered Zenfire feed has the first trade of the day on the FDAX as 0 for example. Why tempt providence.

 

Actually I am struggling to think how you might organise your logic to achieve this? Presumably you test for @BB and if it is not you assume it is @BA, even if you where making assumptions and using these as pointers somehow I still can't see how you could cause a crash as long as 'last' is an actual price the instrument can trade at. (Again why even assume that?)

 

Actually, I am rather fascinated by this, I have seen remarkable code in the past where people have done very clever stuff with bitwise instructions, rotates, etc. to construct pointers, usually the side effect off this 'tight code' is you actually rotate out or mask off bits that would cause addressing issues.

[taotree 10]

  BlowFish said:

Just one more thing as Columbo might say.

 

Seems to me a great working solution would be R|thmic / Zenfire or TT for live data and IQFeed for historical. I have not got round to testing the completeness of IQFeeds historical data but there are couple of people here that swear by it.

 

If you're going to pay for IQFeed, why use a different feed for real-time? Why not use it for both real-time and historical?

[cunparis 10]

  taotree said:

If you're going to pay for IQFeed, why use a different feed for real-time? Why not use it for both real-time and historical?

 

Realtime $100+

Historical with Investor RT $15 (or $25 with Market Delta)

 

that's the only reason I see.

 

the new exchange fee waivers from IQFeed do make it much more affordable than it used to be.

[AgeKay 10]

  BlowFish said:

Actually I am struggling to think how you might organise your logic to achieve this? Presumably you test for @BB and if it is not you assume it is @BA, even if you where making assumptions and using these as pointers somehow I still can't see how you could cause a crash as long as 'last' is an actual price the instrument can trade at. (Again why even assume that?)

 

The X_Trader API tells you which side the trade was on, so there are no trades without BB or BA. In several places, my code would look like this which would cause an exception (error in .NET) if there was no side (i.e. no bid or ask):

 

switch(trade.Side){

case eSide.Bid: ...

case eSide.Ask: ...

case eSide.None: throw new Exception("trade has no side");

default: throw new ArgumentOutOfRangeException("trade.Side");

}

 

Fact is, there are no trades without a side if you have a decent feed, so no assumptions need to be made. And I want my application to wouldn't really crash since my application handles and logs all unhandled exceptions, but it wouldn't if it didn't handle unhandled exception. The point is that an exception will launch the debugger if debugging and you want this when you're developing because if you get a trade with no side then something must have gone wrong in your code.

[BlowFish 10]

Ahh OK thanks for clarifying that.

[taotree 10]

  AgeKay said:

The X_Trader API tells you which side the trade was on, so there are no trades without BB or BA.

 

This statement suffers from a non-sequitur falacy. The first (the API tells you) does not imply the second (no trades without BB/BA). All it means is that X_Trader or the exchange or wherever that data is coming from is using some type of algorithm for you. It would be rather interesting to dig into that and track down exactly what algorithm they're using.

[AgeKay 10]

I think this is getting silly. I think the exchange knows damn well whether a trade was on the bid or ask. And this is the "algorithm" they are using: if trade on best ask, then report trade on ask, if trade on best bid, then report trade on bid.

 

If there really were trades without a bid or ask, then that would mean that you could enter limit orders without specifying whether you want to buy or sell, which is obviously absurd. Whichever limit order your marketable order is matched with determines whether it traded on the bid or ask. If you don't get that, then there is no point discussing this any further as this is how FIFO works, and that is a simple fact.

[BlowFish 10]

FIFO is indeed simple (first in first out). It should probably be pointed out that there are a lot of separate queues, At each price there will be two queues. A 'limit' queue, the sum of which shown, there is also a a stop queue on the other side of the market that is not shown. If you have used a 'DOM' or price ladder this is quite easy to visualise.

 

Matching will precedent price first followed by arrival time of the order.

[paolfili 10]

If you have used a 'DOM' or price ladder this is quite easy to visualise.

 

IYO the trades are from 2 bid/ask queues.Limit (visible from DOM) and stop (unvisible from DOM.)

Can you explain how visualize from bid/ask/trade flow some limit/stop order execution? (or make some examples)

 

 

  BlowFish said:

FIFO is indeed simple (first in first out). It should probably be pointed out that there are a lot of separate queues, At each price there will be two queues. A 'limit' queue, the sum of which shown, there is also a a stop queue on the other side of the market that is not shown. If you have used a 'DOM' or price ladder this is quite easy to visualise.

 

Matching will precedent price first followed by arrival time of the order.

Edited March 18, 2010 by paolfili

[BlowFish 10]

Look at the dom the numbers you see are the cumulative values of limit queues at each level. Each number you see is the cumulative value of orders on the book at that level (that particular queue). On the other side of the order book is another queue at each level where stop orders reside. These are not shown (though in some markets a market maker or specialist can see these).

[paolfili 10]

I see.

But there' s no difference between stop orders and market orders.

How can you select?

 

  BlowFish said:

Look at the dom the numbers you see are the cumulative values of limit queues at each level. Each number you see is the cumulative value of orders on the book at that level (that particular queue). On the other side of the order book is another queue at each level where stop orders reside. These are not shown (though in some markets a market maker or specialist can see these).

[BlowFish 10]

A stop order will become a market order when price trades at the stops price.All the stops at that price (in that queue at that price if you like) will be matched against orders in the queue on the other side of the book (limit). This will be matched in order that the stops where placed against the limit orders in the order they where placed.

 

If there are more resting stop orders than on the limit side they will be matched against the limit queue at the next level (as they are now market orders). That is slippage. This might trigger more stops however they will not be filled until all the stops from the previous level are filled in FiFO order. If the limit side of the book is thin you might get several ticks slippage before your stop is filled.

 

Edit in early days of electronic markets many exchanges did not implement market orders. When you placed a market order your broker would actually convert it to a stop well inside the market so it would trigger immediately. So for example if the market was trading 6523.5 and you wanted to sell market the broker would sell on stop @ 6023.5

Edited March 18, 2010 by BlowFish

[AgeKay 10]

Thanks to BlowFish for the explanations. I am still surprised how people start to trade without even understanding the basics of how orders are matched, which ultimately determines how price changes.

 

This is how I visualize it in my head or how I would even teach it to a beginner (or even child): Imagine a stack of cards for each price. Each card represents a contract. Cards with a red cover represent offers and cards with a blue cover represent bids. Align them on a table so that they look like a DOM / price ladder. If you enter a limit order, then you would place a card on top of the card stack. When a certain amount of contracts trade at that price, then you remove that number of cards from the bottom of the stack (imagine another stack of cards that represent the trade slide across the table to push the cards away from the bottom of the limit order stack). Let's say this price is the best offer (ask) and all cards are gone. Now someone could enter more sell limit orders on that price to keep that price being offered, but if someone instead entered a buy limit order on that price where there are no more cards, then this will become the best bid. In this case, the inside market has moved up 1 tick because the bids are 1 tick higher and the offers are 1 tick higher than they were before.

 

Marketable orders (market orders, stop orders or limit orders placed at the market) always initiate a trade and are matched with the best limit orders (best bid for sell orders or best ask for buy orders). Even if the market was bid 8, offered 10 and two traders entered market orders to buy and sell at the "same" time (technically impossible, but same lets say the same millisecond), then the buy market order would be matched with 10 and the sell order with 8. Notice that both market orders would not be matched at price 9 even though it would be "more fair", but each market order does not know of the other market order because the exchange matches them as they arrive and one of them will arrive a microsecond earlier than the other.

[paolfili 10]

These are not shown (though in some markets a market maker or specialist can see these)

 

Please,can you say which Market(s) are you talking about?

 

 

  BlowFish said:

Look at the dom the numbers you see are the cumulative values of limit queues at each level. Each number you see is the cumulative value of orders on the book at that level (that particular queue). On the other side of the order book is another queue at each level where stop orders reside. These are not shown (though in some markets a market maker or specialist can see these).