Daytraders - Do You Know Your Enemy?

[BlueHorseshoe 10]

They maybe classed by the exchange as limit or market only but I what I mean is that just because you crossed the spread does not mean you have decided to have a market order.

eg; Lets say the market is 55-56, 10 volume quoted each side, and there is also 10 bid for at 54, 10 at 53.

I sell 200 at a limit order of 53. I will cross the spread and hit the market for 10 at 55, 10, 54, 10 at 53....I will then be in the first in the que for 170 on offer at 53.

 

So in your counting - are the first 30 market orders, then the next 170 limit orders if they get hit? How does the exchange see it?

From the person who hit the button they were aggressive and crossed the spread for the first 30 with a market order, and passive and used a limit order for the rest......

 

If that makes sense......mind you it might not make any difference if you are only counting those that cross the spread and hit the bid or the offer, regardless of passive or aggressive and what type of order they used.

(I saw some great uses of manual order placements in a previous life and watching the brains tick away about what was happening - one example a massive sell order sat at a level and was hit and hit and hot. Finally it seemed complete the stock ticked up about 1% - the order came back at the original level about half an hour later taking out all the bids down to that level- this occurred 2-3 times over about 6 days. They sold pretty much near the top for that recent move.....and the volume was huge)

 

Re Paul - I just read his feedback as dont make it more complex than it needs to be...he can answer that. Often a well researched thought out complex plan will be just as good as a guess if there are too many fudges in there not revealing enough data to be accurate.

 

I see your point now. In terms of my description they fit the term I (incorrectly) used of market orders, in that they cross the spread to produce volume at the other side. But they're obviously limits.

 

My head hurts!

 

BlueHorseshoe

[Predictor 11]

The algorithm for tracking position in queue is not very complicated. First, you record the queue position the moment before your order is placed. Let's say 500 are offer and you add 1. From here out.. you don't even need to monitor the book (for the most part)

 

There are 3 cases:

 

1. Adds -> These will be above your position. We can ignore.

2. Subtracts -> These could be above or below your position. We can only know if the total drops below our count.

3. Transactions -> These can always be subtracted from our recording.

 

Now, you track the market orders that execute at that level and subtract them from the first value that you recorded. You, also, monitor the total depth at the level and if at any time the value is below your running total then you reset it to new value (it means that orders were pulled underneath your own). It is shown that as more orders transact then the more accurate your estimate of your position will be.

 

Of course, this won't give a perfect estimate. I estimate about 50% of the orders in the ES book are fleeting.

[BlueHorseshoe 10]

The algorithm for tracking position in queue is not very complicated. First, you record the queue position the moment before your order is placed. Let's say 500 are offer and you add 1. From here out.. you don't even need to monitor the book (for the most part)

 

There are 3 cases:

 

1. Adds -> These will be above your position. We can ignore.

2. Subtracts -> These could be above or below your position. We can only know if the total drops below our count.

3. Transactions -> These can always be subtracted from our recording.

 

Now, you track the market orders that execute at that level and subtract them from the first value that you recorded. You, also, monitor the total depth at the level and if at any time the value is below your running total then you reset it to new value (it means that orders were pulled underneath your own). It is shown that as more orders transact then the more accurate your estimate of your position will be.

 

Of course, this won't give a perfect estimate. I estimate about 50% of the orders in the ES book are fleeting.

 

Hi Predictor,

 

Thanks for posting.

 

I think the description above describes pretty much the same conclusion that I had arrived at, which is encouraging. What I am now doing is considering ways to go further than this, which obviously means working with estimations rather than certainties. I was wondering if you had any insights about how to do this? For instance, ways to estimate what proportion of cancelled orders are behind or ahead in the queue?

 

Cheers,

 

BlueHorseshoe

[SIUYA 11]

I see your point now. In terms of my description they fit the term I (incorrectly) used of market orders, in that they cross the spread to produce volume at the other side. But they're obviously limits.

 

My head hurts!

 

BlueHorseshoe

 

dont whipsaw yourself....they are not necessarily limits either - this is a case of they may be a market order or a limit order - if the exchange classifies them as such then that may be so.

Point is - what are you trying to get out of it....if you want to know and count everytime someone hits the bid or the offer then it maybe irrelevant how that person entered the order....plus when people really get aggressive or passive may have little to do with hitting bids and offers and more about volume. You are counting the times people cross the spread....That was all I thought (???)

 

Here is another thought experiment .... market is 55-60. Someone ticks up and down to close the spread..... 56-59, then 57-58....what changed?.....or

55-60.....then 55-59, 55-58, 55-57, 55-56, then someone buys at 56 and takes it bid 56-57

What was registered the down ticks of the quotes, or the first trade as someone being aggressive and buying at the offer?

[horace 10]

dont whipsaw yourself....they are not necessarily limits either - this is a case of they may be a market order or a limit order - if the exchange classifies them as such then that may be so.

Point is - what are you trying to get out of it....if you want to know and count everytime someone hits the bid or the offer then it maybe irrelevant how that person entered the order....plus when people really get aggressive or passive may have little to do with hitting bids and offers and more about volume. You are counting the times people cross the spread....That was all I thought (???)

 

Here is another thought experiment .... market is 55-60. Someone ticks up and down to close the spread..... 56-59, then 57-58....what changed?.....or

55-60.....then 55-59, 55-58, 55-57, 55-56, then someone buys at 56 and takes it bid 56-57

What was registered the down ticks of the quotes, or the first trade as someone being aggressive and buying at the offer?

 

Thanks guys.

I have followed this conversation today plus a couple of other threads and I am with you BHS, my head hurts.

On a brighter note it convinces me that price only trading suits me better since we are looking at locked in events in the sense that the price has printed and that is the end of the matter.

How we string the prints together is our challenge.

Over the years I have visited and revisited the DOM and each time I come away thinking that this is a job for specialists.

[BlueHorseshoe 10]

Thanks guys.

I have followed this conversation today plus a couple of other threads and I am with you BHS, my head hurts.

On a brighter note it convinces me that price only trading suits me better since we are looking at locked in events in the sense that the price has printed and that is the end of the matter.

How we string the prints together is our challenge.

Over the years I have visited and revisited the DOM and each time I come away thinking that this is a job for specialists.

 

There are plenty of traders making plenty of money who never look at any of this stuff. If the average profit per trade from your strategy is large enough that paying the spread doesn't significantly impact upon it, then all of this probably isn't worth bothering too much about.

 

If you look back to post 50 you'll see that my starting point was a strategy where the average profit was about $10 per trade. Pay the spread in and out on this in the ES and you're out by $15 per trade. Use limits only and you might miss too many winners and also lose money. At this point execution becomes the difference between a great strategy and a terrible one.

 

If you were swing trading from a daily charge and averaging a profit of $500 per trade, would you be too bothered about paying the spread? I wouldn't.

 

BlueHorseshoe

[BlueHorseshoe 10]

dont whipsaw yourself....they are not necessarily limits either - this is a case of they may be a market order or a limit order - if the exchange classifies them as such then that may be so.

Point is - what are you trying to get out of it....if you want to know and count everytime someone hits the bid or the offer then it maybe irrelevant how that person entered the order....plus when people really get aggressive or passive may have little to do with hitting bids and offers and more about volume. You are counting the times people cross the spread....That was all I thought (???)

 

Here is another thought experiment .... market is 55-60. Someone ticks up and down to close the spread..... 56-59, then 57-58....what changed?.....or

55-60.....then 55-59, 55-58, 55-57, 55-56, then someone buys at 56 and takes it bid 56-57

What was registered the down ticks of the quotes, or the first trade as someone being aggressive and buying at the offer?

 

I've only given any consideration to markets that maintain a single tick spread, so what you describe is a useful thing to think about.

 

BlueHorseshoe

[Paul-TC 10]

So beyond what I described in that second post it would indeed be a guess/estimate. I'm not sure why you think that the formula need be arbitrary

 

I only meant that, as there is no way to determine your PIQ exactly, any formula you do use to estimate will be based on a personal judgement or discretion rather than having a probability based formula giving much beyond 50/50 accuracy. I didn't mean a 'random' formula if that's what you're asking. e.g. E=MC2 won't be much use. .

OTOH, if you use a feed that provides an API or other program access to more detail about the composition of the queue, you'll be able to improve on that ratio.

 

If a method of estimating PIQ is only sufficiently accurate to improve a strategy by a small amount, but is unique in its ability to improve this aspect, then I would consider it worthwhile.But just to be absolutely clear: I'm not suggesting that it is possible to arrive at a highly accurate estimate of PIQ, and certainly not one that is known with certainty.

 

I agree, Anything that improves a strategy, by even a small amount, is worthwhile. Originally, it wasn't clear to me whether accuracy in determining your PIQ was important to you. From this reply, I see that it isn't

 

I'm not sure that was what Paul was saying though - I understood him to be saying that a guestimate was pretty much useless. I certainly agree that a guestimate will often be useless, but I think what is fascinating is that it is possible to know whether or not the guestimate is useless. In his example a guestimate could be made, but clearly it was of little use - such a guesstimate could be thrown out the door (read: order cancelled), and the process repeated until a guestimate in which one had greater confidence was achieved.

 

As I don't know how you intend to use this information within your strategy, I can't comment on whether knowing your PIQ, even as a rough guess, will really be of any use at all. I can't see how it would be but if it works for you, fill yer boots and good luck.

 

Glad to hear working for uk banks improved your spelling, even if it tarnished your soul - maybe you should check your contract with the devil - it mightn't be valid in your jurisdiction if the terms are poorly spelt - or should that be spelled?

 

The terms are usually in such a small font that even a Spell Checker can't find errors.

 

P.S. FOK orders are held at the CME. However, not all brokers/platforms offer every order type.

Edited August 16, 2012 by Paul-TC

[BlueHorseshoe 10]

P.S. FOK orders are held at the CME. However, not all brokers/platforms offer every order type.

 

I couldn't find them listed in the globex manual - can you give me a pointer to more information on this?

 

Cheers,

 

BlueHorseshoe

[Paul-TC 10]

I couldn't find them listed in the globex manual - can you give me a pointer to more information on this?

 

Cheers,

 

BlueHorseshoe

 

Try this:

 

Order Types

 

I just noticed that the list on that website while quite thorough, isn't entirely complete - e.g. 'FaK' is also held at the CME

Edited August 16, 2012 by Paul-TC

[Paul-TC 10]

Paul - if it may still exist? is a question. It may be cynical or rhetorical, or just a question.

"even if it may still exist" could never be considered 'a question'.

It's a statement of having no certainty about whether it still exists.

 

Even if it was a question, there is enough information readily available on the web for it NOT to be a question nor a subject about whose existence you're unsure before you start discussing it.

 

I am going to run with this Paul as it clearly seems something that you have definitely got issues with.....

lol - Yeah, well - nice try - but I really don't have any issues with anything much beyond your ignorance of what's commonly known and your laziness in not researching a subject before wasting my time and presuming to debate it with uninformed statements - that's all.

 

As I said before, it does become tiresome trying to have a discussion with someone that intellectually dishonest and lazy. To that burden, I now have to add: 'Having to repeatedly spell this out as it obviously didn't register the first time'.

 

As much as I'm sure you'd like to believe otherwise, it really isn't that much of an issue to cause me any problem not discussing this with you any further - OK?

 

Let's just leave it at that (you probably won't - but I will).

Edited August 16, 2012 by Paul-TC

[BlueHorseshoe 10]

Last night, to help me think about this more clearly, I created some games. I'll post them on here, and if you have five minutes to kill as you wait for something to happen on a chart, then you're very welcome to play them and post your answers. They're all very easy.

 

In the game below you just have to use the magenta sequence to complete the last four steps of the cyan sequence below it.

 

BlueHorseshoe

[BlueHorseshoe 10]

In the second game the aim is to complete the last four steps of the yellow sequence using the magenta sequence above.

 

BlueHorseshoe

[BlueHorseshoe 10]

An interesting link here, not that they're giving much away, of course:

 

System and Method for Estimating Order Position - Patent application

 

BlueHorseshoe

[BlueHorseshoe 10]

Yesterday I posted two games.

 

In the first of these it is easy to see that each of the magenta symbols corresponds with a particular cyan symbol, so that the final four missing symbols in the cyan sequence should be: cannonball, skull, hourglass, cannonball.

 

If I could identify a static and certain relationship between what I am trying to model (PIQ), and another discrete time series (eg Price), then it would be very easy to use the one to 'fill in the gaps' in the other. When I became uncertain of the PIQ value, I could simply look at price and this would tell me.

 

Unfortunately such static relationships in market data are incredibly rare. A little more common, however, are relationships that hold for most of the time, where two or more time series are correlated to some degree.

 

In the second game we can guess that this is the sort of relationship that exists between the magenta and the yellow sequences. We can't know for sure what the final four symbols in the yellow sequence will be, but we can make a pretty good guess.

 

Two out of three times, omega corresponds to square; three out of five times, paper corresponds to circle; three out of four times yin-yang corresponds to diamond.

 

P(if omega then square)=0.66

P(if paper then circle)=0.60

P(if yin-yang then diamond)=0.75

 

The final four symbols in the sequence will most probably be square, circle, diamond, square.

 

How useful is any of this in solving the PIQ problem? I reckon probably not very. . . . If there were sections of the PIQ series that we knew for certain, and we were able to find time series with which they were reasonably correlated, then we might be able to model the missing sections. Unfortunately though, pretty much all sections of the PIQ series are hidden.

 

There are only two times that I can think of when we can be remotely confident about the actual PIQ. The first is when we place the order, and the second is when the market ticks, the depth remains unchanged, and the depth is fairly significant. Suppose that there are 99 bid and we join the queue, making it 100 bid. The market ticks and there are still 100 bid. It is entirely possible that though the depth remains unchanged at 100, 73, say, of the original orders were cancelled and 73 new orders added, so that our PIQ is now 27.

 

Assuming the probability of a group of orders of any particular size being cancelled is identical, and that the maximum number of orders that can be cancelled ahead of us 99, then the probability of N orders being cancelled is N/99. The probability of N orders being added is a bit strange because the only obvious ceiling to the number of members of the set of all N is X, where X is the maximum number of contracts that can be traded. For the ES, X is 2000 (Page 15); for other exchanges the number may theoretically be infinite.

 

So, the probability of any particular order size being cancelled, and an identical order size being added at the same time, in our example, would be (1/99)*(1/2000). That's not even worth calculating - let's just say that we can be pretty much certain that when the depth remains unchanged it is because orders have remained unchanged.

 

Although the depth regularly does remain unchanged, this still doesn't help too much with our problem. If you return to the first game, it's a bit like having all the cyan skulls marked in to correspond with the magenta hands, but none of the other symbols provided. You'd have no idea what a crucifix corresponded with because you'd never even have seen an hourglass.

 

The next thing I will try and think about is situations where relationships exist between events in the time series. In the context of the games this means that a symbol is always dependent upon those that preceded it in the sequence.

 

BlueHorseshoe

Edited August 18, 2012 by BlueHorseshoe

[BlueHorseshoe 10]

One thing to remember is that we’re only really interested in reductions in the orders ahead of us in the queue. This means that we could potentially treat all other changes as ‘noise’, and then try and filter the noise out to reveal an estimate of the underlying time series. Something like a Kalman Filter could be used, or some sort of Neural Network.

 

The problem with these methods is that they both require the back-propagation of error. This essentially means that a ‘prediction’ is made about the subsequent state of the order queue ahead of us, discounting for an historically optimal degree of noise, and then the prediction is compared against the actual output at the future time, and is used to correct the weighting variables within the algorithm.

 

Here we face exactly the same issue as was described in my previous post, namely that there is no reliable description of the underlying state of the order queue. Not only can we not derive it ‘live’, we can’t even get at it with the benefit of hindsight!

 

This is very frustrating! Imagine you were allowed to send orders back in time to trade last week’s markets, but you don’t have a chart of last week’s markets, or any information about last week’s price movements. What do you do?

 

The only other time that we can be sure of our error is at the time an order is executed. Theoretically, data could be collected in this way and then used to train the algorithm. One problem with this is that it would still only allow us to map the state at two points - entry and exit. Another is that it would be expensive. A round trip in the ES would be required just to provide two data points. Not much use . . .

 

BlueHorseshoe

[Predictor 11]

One thing that most people assume is that the markets aren't controlled or manipulated. I am not so sure of that: although I've always felt that feeling they were fair helped my performance.

 

One hint may be that many proprietary firms are often the largest firms in the markets they make. This may allow them to push the market around within constraints contrary to what most believe. If true then it basically means the markets are rigged but may be somewhat predictable for those who can read what they do and may be even more predictable for the firm that controls the orders. An example of one advantage, if a firm holds most the orders in the book then they should be able to estimate the remaining orders at a higher probability then a firm that only held fewer orders in the book.

 

It also worth questioning if any firm is solely trying to profit from the spread (in futures). I am not so sure they are. I mean that most liquidity providers are going to be selective to a degree. They could be selective on price and hold many orders above and below the market. They could be selective on time and move their orders often. I suspect there are two types of these providers operating in the market.

Edited August 19, 2012 by Predictor

[BlueHorseshoe 10]

One thing that most people assume is that the markets aren't controlled or manipulated. I am not so sure of that: although I've always felt that feeling they were fair helped my performance.

 

One hint may be that many proprietary firms are often the largest firms in the markets they make. This may allow them to push the market around within constraints contrary to what most believe. If true then it basically means the markets are rigged but may be somewhat predictable for those who can read what they do and may be even more predictable for the firm that controls the orders. An example of one advantage, if a firm holds most the orders in the book then they should be able to estimate the remaining orders at a higher probability then a firm that only held fewer orders in the book.

 

It also worth questioning if any firm is solely trying to profit from the spread (in futures). I am not so sure they are. I mean that most liquidity providers are going to be selective to a degree. They could be selective on price and hold many orders above and below the market. They could be selective on time and move their orders often. I suspect there are two types of these providers operating in the market.

 

Hi Predictor,

 

Thanks - some really useful ideas there.

 

I think I would agree that it's possible for very well capitalised participants to push the market around by a few ticks or so, although probably not usually be more significant amounts.

 

There are no designated MMs in the ES, so it's always hard to guess whether anyone unofficially assumes this role. It's always easy to say that there is no need for MMs because the market is always so liquid, but is it only so liquid because of those who discreetly act like MMs? Personally, I would guess that there probably are HFT firms trying to occupy this space, although how successfully it's impossible to know.

 

Your point that if a firm holds most of the book they can more accurately estimate position is a fascinating one. It means that if such a participant is already queued at the time one wishes to join the queue, then one may be at a disadvantage in terms of estimating one's own PIQ thereafter when compared to them. It would also provide an additional motive for things like trying to re-construct icebergs.

 

Thanks.

 

BlueHorseshoe

[RichardHK 10]

Here's a useful presentation from premarketinfo.com that addresses how to trade vs the HFT firms/algos.

 

Trading in an HFT World/

 

I visit this site every day before the open to listen to their free review. And lots more HFT info and videos here to learn from.

 

Richard

Hong Kong

[SIUYA 11]

Blue - stumbled across this - it might be interesting though I could not really be bothered to read it as it a one sided defence of HFT, but I figured you and others might find it interesting.

 

http://www.directedge.com/Portals/0/docs/Direct%20Edge%20Flash%20Trading%20Ban%20Comment%20Compressed%2011%2020%202009.pdf

[johnnydaymon 11]

The first chart attached below shows a stream of orders sitting overhead. These orders are placed above the market at price levels where they will never be likely to interact with the bid. They don't serve any obvious function in the market.

 

What is significant is that the orders streams manifest as geometric patterns that become apparent in sub-second timeframes. Sometimes these patterns evolve on a price axis, sometimes on an ordersize axis (ie at a constant price), and sometimes combine both (order size diminishing as the last traded price is approached, say). See the second attachment.

 

What possible purpose could such orders serve?

 

One possibility is that they allow the producer to introduce apparently random noise into the market with the intention of confusing other market participants. But because the producer knows the underlying algorithm generating the orders, this firm can then cancel out the noise from their own data.

 

Many HFTs are, we're told, well versed in procedures for cancelling out noise to produce an accurate estimate of a system's underlying state (many of their developers coming from backgrounds in speech recognition and encryption) - but what could be easier than cancelling out noise when one is adding that noise in the first place, and it is governed by a simple formula?

 

Maybe you never look at the Matrix or DOM more than three or so tiers away from the last traded price, so how could this affect you?

 

Because other market participants do, and then they act by executing orders, and also by cancelling them. So the next time you're about to act on any kind of information derived from the order flow of other traders, you might wish to consider whether these traders are themselves acting under a misapprehension induced by the pretty patterns an HFT is drawing 30 points overhead . . .

 

BlueHorseshoe

 

Hello BlueHorseshoe!,can you post those indicators on post 1 & 3? Tryen to track the enemy.

[krasaves1275 10]

i know this one - I am :crap:

[petertrading 10]

The first shot shows a pointy come by the worth of associate degree FX combine over the course of 200ms. The sub-pane supposedly shows the order streams of variety of HFTs, measured on the hand axis by distance from this best bid.

 

The pack up shows the decline once more. Notice that four of the order streams move their bid deeper into the order book 60ms before the drop, and another 2 do thus 2ms before, thus on avoid a fill.

 

Quite except the formidably spectacular incontrovertible fact that it's attainable for associate degree algorithmic program to supply associate degree on-line estimation of the standing of another HFT’s orders, there's the apparent ability of the tracked algorithms to spot a value decline before it happens, and to implement a defensive response in milliseconds. What did they see that you simply don’t? may you sight the activity of one participant in a very multitude of orders on a tick by tick basis? If you don’t grasp the mathematics, it'd also be magic.

 

If you’re day-trading, this can be the type of opponent you’re up against on a commonplace. These square measure the capabilities of these WHO might take the opposite aspect of your position. in person i might realize that thought deeply redoubtable . . .

[BlueHorseshoe 10]

High Frequency Trading Hails Its First Billionaire | Michael Shedlock | FINANCIAL SENSE

 

BlueHorseshoe

[Patuca 13]

Not sure why so much fussing about front running...etc. It was happening pre-livermore days and still happens everyday..legal or illegal. We have always lived with it and always will. Ways will be found to circumvent it. Remember the soes bandits? They used to front run mm's any time they could catch the spread in such a way to do so. So...that got them labeled with a dirty name...BANDITS. I am sure they were quite proud of their name...like jesse james and his gang..So..things evolved into tighter spreads...wonder who wanted that?.....now who is bellyaching about the hft front running on tighter spreads? Could it be .......? No way they wouldn't do that would they? Then there are laws about insider trading....etc... Did the specialist have inside info about the market? Was he allowed to trade his own account? Mmm...

 

Point...market has always had manipulation. It never has been fair nor will it ever be fair. The only thing fair about it is that all have the same environment to trade in i.e. an unfair environment LOL! It has always been a game of wits. Does anyone know that sometimes the mm's got screwed? does anyone realize that htf's are screwing each other every day all day long? I used to frown when i would see a specialist screw someone over. Then i used to laugh when i saw someone screw the specialist over. In the end neither thing did me any good or bad. It is what it is. If i am going to trade the markets i have to find a way to outwit someone else. Forget the fair senario. Never has been. Never will be. Laws won't make it so. There are laws against murder. People still murder. There are laws against stealing...people still steal...there are laws against stealing in wall street...hmm..hmm well there are supposed to be...laws never stopped the unsavory player. They may make some think twice about becoming an unsavory player and for that purpose they are good to have but we must realize there are TOO MANY people who simply don't give a sh$t about ANY laws. So the gov cannot and will not protect you from these unsavory lots of human flesh and brains so guess who has to protect who?

 

Its a case of if it is too hot in the kitchen then GET OUT FOR CRYING OUT LOUD!

 

Screw em....htfs or no htfs..algos..bots..super computers...young wippersnappers with their baseball cap on backwards and baggy pants... ...screw...em ..... I trade the markets..and will continue to do so..sometimes i win .... sometimes i lose..

 

People should study george douglas taylor and a mr brooks imo, which generally, is not heeded to but rather mocked. But i don't give a rats a$$ what you think of my opinions so just be cool and have fun and take the bends as the river gives them to you...catch a few fish along the way...enjoy life...

 

Patuca

 

Time for some fishing. Liked my trip to Honduras. Getting a hankering for Belize. Ole WHY? is still alive and cantankerous as ever...

Edited April 6, 2014 by Patuca