AgeKay

I have come to the realization that it might actually be my explanation b) after looking at the DOM for some time. There are certain prices that everyone is watching, e.g. high and low of day. And since S/R are good entries, you usually see a lot of volume transact at these prices. The winner of those battles then immediately post huge bid/offers to prevent the losers from getting out break-even of their positions. Everyone who is on the losing side wishes they were on the other and since breaks of important S/R signify trend changes, no one wants to be on the losing side any more and the previous S/R looks like a good price to go with the new trend.

With regards to data feeds. I talked to the API guy of IQFeed yesterday and he couldn't tell me whether inside market updates come before trades or not. He said it depends on how the exchanges report it because they just forward it. I just called Eurex to find out and they told me they'll get back to me with an answer on how they report trades to the feed providers. I compared IQFeed with TT's feed today and IQFeed serves more trades. There was even a point where MD Trader showed a low of the day without any volume having traded there, which means that they somehow filtered out that trade. I've said before that ideally you want two different feeds. One that reports all information and is possibly a bit slower (used for charting and data processing) and one that reports only the most recent information and is really fast (used for execution/DOM). I do have the impression that IQFeed is the former and TT's feed is the later.

I agree. I saw the same thing with the TT feed. Book updates always came before trades. I assume that this is also the case with IQFeed because if trades came before book updates then you wouldn't have the issue with trades between the inside market. I saw the same thing with TT's feed. Most likely they just drop the book updates in favour of showing you the most recent inside market. I've talked about his earlier. The feed providers have to decide whether speed or complete data is more important and in most cases it's speed. DTN claim that they don't filter anything. Even their API documentation says that you they can't guarantee that all the data you get is valid because they don't filter anything.

I don't think so. Their basic service fee is already $60/month which gives you access to delayed data. You also need to add exchange fees to that for every exchange you need. Access to the API is another $300 or $600 per year, don't remember the price. It think it was $600 at some point and they lowered it to $300, but I might be wrong.

Sorry for leaving out how IQFeed works. This is an implementation detail and the code I posted was just an example and should obviously be adapted to how your feed works. If your feed sends the inside market independently of the trades then you have to update the _marketSideByPrice whenever the inside market changes and then just check for the side when you get the trade. FYI, IQFeed always sends an update message when either the inside market changes or there is a new trade. The message then has a field that tells you what changed. The data that didn't change is also in the message so that's why the example I posted works. The important thing about my example is that it shows you how you can fix the trade without side issue which should work with any feed if slightly adapted. They have a COM API and TCP sockets. I use the TCP sockets. I don't like using COM interop with .NET because calling COM methods from .NET is extremely slow. I compared my TT feed to IQFeed and noticed that my TT feed is coalesced while IQFeed is not. It depends on how your broker has set it up though. Even though it doesn't matter to me anymore whether they are coalesced or not, it is good to know that IQFeed is not.

I started working on IQFeed again. It doesn't explicitly give you the side that the trade traded on. Instead it sends a message (they call it "Update Message") that contains the current trade price, bid price and ask price. I can see how chart software developers that implement this feed made the mistake of comparing the trade price to the inside market of that message. It initially did the same and I noticed trades and the inside market that are contained in the same message are not synchronized which is the reason for "trades between the bid or ask". It would report a trade at 123.22 while it bid was 123.21 and the ask was 123.23. If you had looked at the DOM though you would have seen that 123.22 went bid and as soon as it went bid, it got taken out. So that trade "between the bid or ask" was on the bid. What must be happening internally with IQFeed is that they update the inside market before the trade so the bid is already 1 tick lower before the trade is reported. Instead of comparing against the current inside market, you should keep track of the market side of each price using an associative array (dictionary, map, hash, index or whatever your programming language calls it). This will always correctly determine the side of a trade independent of the how the feed reports the data. Here is what an implementation in C# would look like: //enumeration for market side public enum eSide { None, Bid, Ask, } //static dictionary taht keeps track of market side at price private static readonly Dictionary<decimal, eSide> _marketSideByPrice = new Dictionary<decimal, eSide>>(); private eSide GetTradeSide(decimal bidPrice, decimal askPrice, decimal tradePrice) { //set side at bid if (_marketSideByPrice.ContainsKey(bidPrice)) _marketSideByPrice[bidPrice] = eSide.Bid; else _marketSideByPrice.Add(bidPrice, eSide.Bid); //set side at ask if (_marketSideByPrice.ContainsKey(askPrice)) _marketSideByPrice[askPrice] = eSide.Ask; else _marketSideByPrice.Add(askPrice, eSide.Ask); //get side at trade price eSide tradeSide = marketSideByPrice[tradePrice]; return tradeSide; }

Cunparis, there was this guy on YouTube who would occasionally record his DAX trading and post it there. He was a phenomenal DAX trader. I remember one time where he had over 20 winning trades in a row and he risked less than he made on each trade so that is pretty insane for a volatile market like DAX. If I remember correctly he looked at the FESX chart and somehow always knew where good entries and exits were. He deleted all of his videos though so the only one that is left is this one: [ame=http://www.youtube.com/watch?v=-WEvm9u1MOI]YouTube - The DAX Scalping,Oct 19 USA[/ame]

This is what a good day should look like (this was today from 10:08 - 10:35 CET). Less than 30 mins of trading, no losing trade, 4 winning trades, 1 scratch. With the exception of that scratch, market didn't even go 1 tick against me. Winning streaks don't go on forever and it's Friday, so I just call it a day. Entry and exit on my first trade really sucked though (the one that ended up being a 3 tick winner). I got in 2 ticks too late (cause I just sat down to trade and I don't like trading right away without looking at it for at least 5 mins) and got out 2 ticks too late (hadn't had time to analyze the volume since I had just sat down). My entries generally sucked today, I could have gotten in 1 tick better on each trade if I didn't insist on getting filled on the limit prices.

I've never looked at CL, but I looked quite a bit at DAX a few years ago. I wouldn't trade it without looking at the FESX or ES, like I wouldn't trade Bund without looking at Bobl. I don't know if it's still the case but based on the ever existing 99% correlation of FESX and DAX, when I looked at DAX, it moved 4 ticks for every tick the FESX moved. Back then you could anticipate DAX moves based on how FESX traded or its depth changed. It was similar to the Bund/Bobl relationship. Bobl might go bid 500 from 1000 without any trade and Bund trades 1 tick lower. Same with FESX/DAX. It's great for getting in at the right prices, risking as little as possible. The reason I mentioned ES is that I had the feeling that ES would sometimes lead the FESX the same way FESX would lead the DAX. So you could basically look at ES to trade the DAX. It could be related to what time you trade, I don't remember. In any case, don't look at more than 2 DOM's at the same time, it will be too much information for you to process, especially since FESX/DAX are a lot more volatile than the markets I am used to. Commissions are always a concern for short term traders. Unless you get really bad commission from your broker, they shouldn't eat too much into your profits since you're trading pretty volatile markets. But at the same time, you should always get the best commission deal you can. Maybe, but I am scalping not because Bund doesn't much that much (which is the case) but because my decisions are based on things that require me to get out of a position after just a few ticks a lot of times. If I am long and think that it should go higher, but see that it will go down now, I get out, I might even reverse. I can always get back in long when it looks like it goes up again. No reason to give back profits and possibly let a winning trade turn into a losing trade. Of course, the risk of missing out has a much higher impact in markets that are volatile, so it's always up to your trading style.

It depends a lot on the market. If you're market is volatile, you're stops might need to be larger. What market do you trade? I don't use stop orders anymore, there were way to many times that I got stop out when a lonely 10 lot order traded. I just exit manually. It does happen that I lose more than 2 ticks but it doesn't happen very often and if I don't manage to get out so do others. Bund is not that volatile and I managed to have only 1 tick losers yesterday (I was on fire). I am scalping. I usually go for 3-4 ticks, but I also take only 2 if that is all the market is going to give me. Sometimes I enter and the market just keeps moving my way so I stay with it for as long as it will go. Yesterday I caught the sudden move up at the news but I panicked when it moved back 2 ticks so exited way to early for a 6 tick profit. If I hadn't been so eager to take profit, I could have made 15 ticks on that move.

Well, bids or ask are never aggregated but I guess what you mean is that you don't see all depth changes, which can certainly happen because you only get the current snapshot of the order book when you request it. I explained the reason for this above, you usually don't want stale order book data. The X_Trader API offers a subscription method that guarantees that all trades are received but they might still be aggregated if your broker or exchange chooses to do so, can't do anything about that, but I don't see at as a problem. It doesn't really matter whether someone sells 2 x 100 or 100 x 2, it's still the same amount of trade unless you're looking for a certain pattern. No, as I said, it can certainly happen. You can reduce lost data though with a good connection and offloading all processing to a different thread than the one receiving the events to prevent blocking the thread from receiving updates.I've looked a long time on what's exactly happening in the order book of bund but I found no edge. It might be because it is so strongly correlated to the Bobl. A lot of what you see in the order book in the Bobl is just because something happened in the Bobl. I haven't analyzed the order book of the Bobl though, I might do it in the future, but I am actually working on looking at less, not more. Call it want you want, but this algorithm clearly tries to manipulate the market without risking more than it has to. This is not a method or strategy I am presenting. This is just what happens. I do have very specific rules but it's very discretionary. I just developed them over time. I don't use indicators or set targets so there is no "trading system" to build around it. I try not to risk more than 2 ticks. A lot of times I just get out with a 1 tick loser or break-even if it doesn't do what I expected it would do or if I see something where I would reverse my position. Sometimes I reverse just because I am wrong and I know it is going to move a lot so I make back my loss and more. Sometimes I get out just because momentum slows down or I've reached my daily target. Sometimes I get out because I really have to take a piss In general I like to enter at very low risk entries (i.e. where I can risk only 2 ticks or less and I know I am wrong), get the "edge" (be filled on a limit order or use a market order just before the price leaves) go with the trend/momentum, buy support, sell resistance and follow the largest traders. You can't automate this. All the large short term traders are discretionary so they don't always use the same methods, but they do use some pre-programmed actions. They use what makes sense. Sometimes they sweep the offer only because they know everyone will puke if it moves another tick having entered their targets (offers) in advance and they are happy with just a few ticks on some quick (puking) action.

You're obviously smart and you've put a lot of thought into this so yes, the logic is not flawed if you fully understand what's really going on in the market, which you do. I just know there aren't many people like you. They think "buy if delta positive, sell if delta negative". But of course, it's never as simple as that. I base all my trading decision based on what I see in the DOM and visualizing the individual depth changes had definitely helped and I can see whether a large order is real or just there to push the market. But I don't think they really use them to screw over traders leaning on price, the risk/reward is not worth it. What follows is what I understand from watching the Bund, so other markets might be completely different. Many fake orders are entered using an algorithm that quickly enter a random number of contracts until the desired number of contracts appear. For example, it goes +34, +76, +22, +89, etc. which of which the sum then (surprisingly or not) ends up to be exactly 500, 1000 or 2000. This then usually get pulled very quickly without even trading or upon the first trade. I think they enter it using smaller contracts to avoid the risk of getting completely filled if they just flash 1000. Sometimes I've seen it happen, where some guy flashes 1000 a couple of times to push the market one way and another trader who then just waited for that took at the entire offer as soon as he flashed his 1000 again. He must have used a special software though because it got taken out before I could even see the offer with my eyes. I could only see in my software that it did happen. If you have two large traders like that, it will be the one with the most money who wins. Just last week, I saw the market going down to an important support, as soon as that price went offer, all offers were swooped up. This kept going until 12k contracts traded at that price alone without the inside market even changing which is a lot in the Bund for just one inside market. So this looked very bullish and I went long. It traded down the next 5 ticks for another 12k contracts (I got out after it went 1 tick against me). Then it reversed and went straight up like 30 ticks. You obviously never know these guys risk tolerance or profit targets. The long guy might have been unlucky that momentum was against him and he might have cut his losses which might have the reason you saw another 12k contracts trade on the next 5 ticks. Or, he might have been a massive trader and he might have just bought another 12k making sure that it wouldn't drop much further against him and thinking that the shorts would take profits if they see so much more buying and loss of momentum. If he did that, then he obviously cleaned up that day. Sometimes it's just large spreaders that put of huge bids or offers hoping to get filled while the opportunity is there for them. If it disappears they pull them because it's not favourable for them to get filled any more. I pay a lot of attention to the Bobl because it influences the Bund so much. You might see one guy offering huge in the Bobl just to get his bid filled in the Bund. He knows that momentum is down and that he is unlikely to get filled on his fake order in the Bobl, but he does want to make sure he gets his bids in the Bund filled. Sometimes outright traders just enter huge bids or offers to get a feel for the market or fool others. They offer big to pull it on the first trade. Make it look like a fake offer while in reality they want to get filled. Or they offer just to see if anyone wants to buy. If no one does, then it confirms their position. Sometimes outright traders just enter huge bids or offers push it over the edge. Sometimes a big offer is all a market needs to make the stuck longs puke which then cause a quick long covering frenzy. They don't necessarily want to get filled on that big offer, but the risk/reward might be worth it if the market cracks a few ticks because of that offer. Sometimes a guy is long 1500 in the Bund and it goes a couple ticks against him. He is still bullish but wants to hedge it just in case by selling 500 in the Bobl because there might be a big bid in the Bobl and he doesn't want to drive the Bund against him. If he times it right, he might even push the Bund a few ticks his way when his covers his short position in the Bobl. I could go on for pages, there is so much going on and there could be so many reasons. It's all psychology and who has the most money. I now just pay attention to how the depth changes and how much trades and not necessarily how much is bid or offered.

Thanks to BlowFish for the explanations. I am still surprised how people start to trade without even understanding the basics of how orders are matched, which ultimately determines how price changes. This is how I visualize it in my head or how I would even teach it to a beginner (or even child): Imagine a stack of cards for each price. Each card represents a contract. Cards with a red cover represent offers and cards with a blue cover represent bids. Align them on a table so that they look like a DOM / price ladder. If you enter a limit order, then you would place a card on top of the card stack. When a certain amount of contracts trade at that price, then you remove that number of cards from the bottom of the stack (imagine another stack of cards that represent the trade slide across the table to push the cards away from the bottom of the limit order stack). Let's say this price is the best offer (ask) and all cards are gone. Now someone could enter more sell limit orders on that price to keep that price being offered, but if someone instead entered a buy limit order on that price where there are no more cards, then this will become the best bid. In this case, the inside market has moved up 1 tick because the bids are 1 tick higher and the offers are 1 tick higher than they were before. Marketable orders (market orders, stop orders or limit orders placed at the market) always initiate a trade and are matched with the best limit orders (best bid for sell orders or best ask for buy orders). Even if the market was bid 8, offered 10 and two traders entered market orders to buy and sell at the "same" time (technically impossible, but same lets say the same millisecond), then the buy market order would be matched with 10 and the sell order with 8. Notice that both market orders would not be matched at price 9 even though it would be "more fair", but each market order does not know of the other market order because the exchange matches them as they arrive and one of them will arrive a microsecond earlier than the other.

I think this is getting silly. I think the exchange knows damn well whether a trade was on the bid or ask. And this is the "algorithm" they are using: if trade on best ask, then report trade on ask, if trade on best bid, then report trade on bid. If there really were trades without a bid or ask, then that would mean that you could enter limit orders without specifying whether you want to buy or sell, which is obviously absurd. Whichever limit order your marketable order is matched with determines whether it traded on the bid or ask. If you don't get that, then there is no point discussing this any further as this is how FIFO works, and that is a simple fact.

The X_Trader API tells you which side the trade was on, so there are no trades without BB or BA. In several places, my code would look like this which would cause an exception (error in .NET) if there was no side (i.e. no bid or ask): switch(trade.Side){ case eSide.Bid: ... case eSide.Ask: ... case eSide.None: throw new Exception("trade has no side"); default: throw new ArgumentOutOfRangeException("trade.Side"); } Fact is, there are no trades without a side if you have a decent feed, so no assumptions need to be made. And I want my application to wouldn't really crash since my application handles and logs all unhandled exceptions, but it wouldn't if it didn't handle unhandled exception. The point is that an exception will launch the debugger if debugging and you want this when you're developing because if you get a trade with no side then something must have gone wrong in your code.

@cunparis: That just means that the TradeStation feed didn't get the new best bid or ask before it got the trade. My application would crash if I got a trade that did not trade at the best bid or ask. So far it hasn't.

Is there even a feed that guarantees receipt of all data? Do you even want that? X_Trader API doesn't, and there is a good reason for it. They told me that most customers prefer to get the most up-to-date data than all data which might be delayed when there is a lot of data at one point in time. It makes sense if you think about it. You don't want to stale information (e.g. old inside market) even if it is complete. I guess one problem to solve this would be to have 2 feeds, one feed that guarantees the most recent data that you can use for quick decisions and execution (e.g. for MD Trader) and another feed that guarantees that you get all data for processing (e.g. for data analysis). TT also has a FIX API and the FIX server can be configured to your needs so that might be possible there, but I haven't had time to closely look at it. Can you enumerate Zen-Fire's advantages/disadvantages. I thought about using them but there aren't any brokers with decent commissions that offer Zen-Fire. Yes, DTN is TCP/IP. I still have access to their developer documenation which is not free. See above. I was comparing Eurex Futures with Zen-Fire in Ninja Trader, IQFeed and TT's feed. Zen-Fire and TT were basically identical in terms of speed while IQFeed lagged by about 1-2 seconds. Unacceptable for any kind of trading really. I generally had the feeling that IQFeed was the discount airline of data feeds. A trader friend of mine who used to use them told me that it even went down for half a trading day while you would never have this problem with TT. There is a reason half of Futures' volume goes through TT and that you see MD Trader (part of X_Trader) on every trading screen on every trading floor. Then why did you bring them up? You can't use their papers to prove that there are "mid-point" trades that don't trade at the bid or ask if they look at trades (spread trades, OTC trades) we don't even see when live trading. So far you have only confirmed what I have said so I don't get why we are still arguing.

So are they using live data in their analysis or historical data?

You're right taotree. I need to take back my statement. I was wrong. I've worked with Ninja Trader, IQFeed's API (DTN), X_Trader API and another one that was tied to a specific broker. I've worked so long with the X_Trader API that I thought all of them didn't give me sequential order book updates while I just checked the API documentation of each data feed again and it looks like X_Trader API is the only one that doesn't. I stopped working with Ninja Trader and IQFeed pretty quickly because Ninja Trader didn't allow access to its data from external applications and I had latency issues with the IQFeed. A limit order that improves the best bid or ask will by definition be the new best bid or ask. Do you agree? If so, then you have to agree that a trade cannot occur between the best bid or ask, right? I don't understand "if there are enough stop orders currently on the book to fill it". Stop orders are triggered by trades not limit orders improving market prices. And even if they were, they would still trade on that bid or ask and not some magical "mid-point". All I have been saying is that a trade always occurs at the best bid or ask.

I know how to interpret the delta indicator. What you don't see though is that after taking out the bid, 800 were left on the offer. He might even offer an additional 1000 and some long traders will be hitting the bid to get out of their losing position since they won't get filled on the offer with a huge offer like that. You always need to interpret whatever you are looking at. My point is that indicators like this leave out a lot of information. I'd never trade a market with specialists. You have an information disadvantage. In Futures everyone sees the same information. You are comparing apples with oranges. These papers use historical data from the exchanges that includes spread trades and OTC trades that are not reported in real-time so you do not see them in a live-feed any way. So yes they do happen, but you won't see that live-trading. I was on the phone with TT and Eurex yesterday with regards to another issue and they confirmed that you won't see spread trades and OTC trades taking place in the data feed. I've actually gotten filled several times without seeing a trade being reported at that price, which must have been a spread trade. Of course, it does. You see it in Time & Sales, don't you? Well if it's there, then it was in the order book, you just couldn't see it with your eyes because it happened to quickly for your eyes to see. I can actually prove this because I have developed a way of visualizing all changes in the order book. And I do look at it during news. The exchange still executes these market orders sequentially (it's software after all) and if it's sequential then there is a small time difference. If you don't understand this, then I don't see the point of continuing this discussion. You just say stuff that isn't true and makes no sense. Well, they don't. At least in none of the data feeds I've worked with can you get sequential order book updates. Instead you get the state of entire order book at one point in time. You won't find it because it's not true. I think before believe anything on an anonymous forum you should use common sense and/or call the exchange and just ask them if it is important to you to know how that works.

You need more accurate time stamps to determine the order of events. It's annoying to have events occur at the "same time" even though you know they didn't. This might not bother most people, but I have programmed some pretty amazing stuff that would improve with more accurate time stamps. So this is not theoretical stuff, I actually need to know the sequence of events. It doesn't matter how long it takes to get processed as long as it is reasonable. Assuming humans can see 25 frames per second, you wouldn't even be able to see the difference with anything more accurate than 40 milliseconds. Sorry, I mainly look at the most liquid Futures contracts which almost all use FIFO. The only one of those that I know that doesn't strictly use FIFO is the 2 year Treasury Note which uses 40% FIFO/ 60% Pro-Rata. Stop orders don't get matched any different than manual orders. They might get there quicker but they still go through the same matching algorithm. @Paolo: Sorry, I don't understand what you are trying to say with your formulas. You bet that the largest traders win. I wouldn't worry about these guys. They don't go through the book so they don't affect the market. At least not in the short-term. Yes, my front end uses TT's API. Sorry, I don't understand. Can you maybe rephrase your question or elaborate?

The size of the bid/ask spread does not matter. You just confirmed what I said. If you bid 10315, then that will be the best bid at that point in time. I can't enter an order at the exact "same" time, it will either be a microsecond before you or after you. So if I offer the same price just a microsecond after you then that will obviously appear as a trade on the bid since my order will be matched with your order on the bid which was the best bid at that time. If I had entered my order a microsecond before you then the trade would have been reported on the ask since I offered first at that price. Your limit order acts like a market order in that case since it hits the market price. So to repeat it again: A trade always happens at the best bid or ask at that point in time (even if it happens so quickly that you can't see it) whether using limit or market orders. Go call up your exchange, they will tell you the same. Regarding time stamps, the most accurate I have seen is TT's API which gives you 100 nanoseconds, it's still not accurate enough and they don't even come from the exchange. Trades will still appear to have occurred at the same time. Even Eurex sells tick data stamped only down to seconds. What papers are you talking about? See above how that is impossible. These indicators are extremely inaccurate even if you have the best data feed in the world based on the fact that its logic is flawed. First see example above how the same trade by two traders can give you opposite results and I give you another example: 200 bid by a smaller trader, 800 offered by a larger trader. Smaller trader buys 600 at the offer, then the larger trader enters a limit order for 1000 to sell on the bid, thereby taking out the bid, so 800 remain on the offer. Another 200 are bid below that by the smaller trader. Now assume the same thing happens again. So now according to the delta indicator 1200 were bought and only 400 sold even though the smaller trader bought 1200 and is willing to buy only another 200 while the larger trader trader sold 1200 and is willing to sell another 800. If you really want to know what's going on then you should watch the order book. I don't know whether it's only Eurex. All I know that it's definitely possible on Eurex.

BlowFish, I would also like to know how that works. None of the order matching rules of exchanges that I have read mention that. All I know is that block trades can be executed on Eurex without going through the book. They are basically just reported for clearing without affecting the market in the order book.

I don't want to beat a dead horse, but even if the spread is 2 ticks, the "mid-point" of a 2 tick spread is still either bid or ask. If it looks as if it traded at that "mid-point" without being bid or offered, it just means that as soon as someone bid or offered, someone sweeped the market so quickly that you couldn't see it. I just want people to understand how orders are matched. There are no magic trades at mid-points on exchanges.

Not really. You are talking about two different things here. Stale data happens. But how is a trade at the mid-point supposed to happen? You can't enter limit orders at half a tick on an exchange, can you?