Daytraders - Do You Know Your Enemy?

[BlueHorseshoe 10]

Take a look at the two attached images.

 

The first shot shows a sharp drop in the price of an FX pair over the course of 200ms. The sub-pane purportedly shows the order streams of a number of HFTs, measured on the left hand axis by distance from the current best bid.

 

The close up shows the decline again. Notice that four of the order streams move their bid deeper into the order book 60ms before the drop, and another two do so 2ms before, so as to avoid a fill.

 

Quite apart from the formidably impressive fact that it is possible for an algorithm to produce an on-line estimation of the status of another HFT’s orders, there is the apparent ability of the tracked algorithms to identify a price decline before it occurs, and to implement a defensive response in milliseconds. What did they see that you don’t? Could you detect the activity of a single participant in a multitude of orders on a tick by tick basis? If you don’t know the math, it might as well be magic.

 

If you’re day-trading, this is the sort of opponent you’re up against on a daily basis. These are the capabilities of those who may take the other side of your position. Personally I would find that thought deeply unnerving . . .

 

BlueHorseshoe

Edited August 5, 2012 by BlueHorseshoe

[SIUYA 11]

my time frame is vastly different and so it is largely irrelevant.

As I understand it - what they are largely doing is market making (clipping) more than position taking (definitions dependent maybe) and reacting to other changes in other markets, information and order flows as well.

They have as much predictive power as the stars - they do however have the ability to make judgement calls in very short term time frames based on the information available and know that they have a back stop if they are wrong (speed and the ability to interpret and read order flow) (Simplistically - To a large extent it is no different to what floor traders always did - only it is super fast, in super size, with less margin and more interconnectedness between markets and instruments (scary contagion effects)

 

More worrying is the fragmented nature of the various market places, lack of obligations to keep fair and orderly markets and the 'fact' that certain firms have different market order types and different access to information (front running and manipulative abilities- ie;; its mot a level playing field even for the HFT.)

 

My concern is still what do i do once I have my trade on.....so the enemy is still me

[BlueHorseshoe 10]

The first chart attached below shows a stream of orders sitting overhead. These orders are placed above the market at price levels where they will never be likely to interact with the bid. They don't serve any obvious function in the market.

 

What is significant is that the orders streams manifest as geometric patterns that become apparent in sub-second timeframes. Sometimes these patterns evolve on a price axis, sometimes on an ordersize axis (ie at a constant price), and sometimes combine both (order size diminishing as the last traded price is approached, say). See the second attachment.

 

What possible purpose could such orders serve?

 

One possibility is that they allow the producer to introduce apparently random noise into the market with the intention of confusing other market participants. But because the producer knows the underlying algorithm generating the orders, this firm can then cancel out the noise from their own data.

 

Many HFTs are, we're told, well versed in procedures for cancelling out noise to produce an accurate estimate of a system's underlying state (many of their developers coming from backgrounds in speech recognition and encryption) - but what could be easier than cancelling out noise when one is adding that noise in the first place, and it is governed by a simple formula?

 

Maybe you never look at the Matrix or DOM more than three or so tiers away from the last traded price, so how could this affect you?

 

Because other market participants do, and then they act by executing orders, and also by cancelling them. So the next time you're about to act on any kind of information derived from the order flow of other traders, you might wish to consider whether these traders are themselves acting under a misapprehension induced by the pretty patterns an HFT is drawing 30 points overhead . . .

 

BlueHorseshoe

[Blaiserboy 10]

I have only one mirror in my house now as I do not like to look at the enemy, he is a scary lookin critter. :rofl:

[Predictor 11]

Some of these are believed to be quote stuffing patterns.. Some HFTS arbitrage across the fragmented markets. One might even speculate that the markets were purposefully fragmented to allow these risk free profits.

 

I am interested in these sorts of things. They can have an impact. But there is still opportunity for the skilled day trader that is willing to assume risk. The HFT's in general seem to want to operate in a lower risk space then the traditional speculator.

[BlueHorseshoe 10]

I have only one mirror in my house now as I do not like to look at the enemy, he is a scary lookin critter. :rofl:

 

Ha! Yes, apologies for the slightly melodramatic title - it's just an easy way to get people to view the thread.

 

BlueHorseshoe

[BlueHorseshoe 10]

Some of these are believed to be quote stuffing patterns.. Some HFTS arbitrage across the fragmented markets. One might even speculate that the markets were purposefully fragmented to allow these risk free profits.

 

I am interested in these sorts of things. They can have an impact. But there is still opportunity for the skilled day trader that is willing to assume risk. The HFT's in general seem to want to operate in a lower risk space then the traditional speculator.

 

Hi Predictor,

 

I'm no expert in these things, so the description that I gave is just one possible explanation.

 

I think quote-stuffing patterns are slightly different, however. They tend to involve order bursts at rates in excess of 8000 quotes per second, whereas the patterns here are continuous cycles, typically operating throughout an entire session. Also, with quote stuffing the aim is basically to try and scupper the servers of one or more exchanges to enable the stuffer to benefit from the resultant latency arbitrage. Hence there would be no need to produce the fancy geometric patterns we see here - rather, the aim is to get as many orders in as quickly as possible.

 

Nevertheless, quote stuffing is a phenomenon that would be perfectly suited to this thread, so I'll try and find something interesting on it to post in the next couple of days!

 

Cheers,

 

BlueHorseshoe

[zdo 20]

related ?

Chart of the day, HFT edition | Felix Salmon

 

In index futures over the past 7 or 8 years, I have only experienced a couple of moments where my suspicion ‘hairs on the back of my head’ got bristled up that maybe HFT just somehow got me… ie in my own life it has had no measurable (or at least, identified ) effects

… the ppl and bots are all really in the same boat – for all, the only successful line of attack is long when price is moving up and short when price is moving down

… ie 'they' can jitter around with bid and ask till they run out of bandwidth but ultimately price must move up or down for ‘anyone’ to make any money - excepting superior premium writers of course …

... ie their timeframes have no impact on my time frames... btw, this final statement has grounding in my studies of 'cycles' and is pervasive across all timeframes.. non continuous 'times' do not interact, etc etc

Edited August 8, 2012 by zdo

[Predictor 11]

For sure not an expert on HFT here or market micro structure. Only know what I've read at NANEX etc.. Not a stock trader either so I don't much experience on that.

 

With equities futures.. you might think a move like that was maybe driven by changes in underlying..perhaps the quotes in the baskets the underlying were changed. See arbitrage.. triangular arbitrage etc.

 

I also observed what appeared to be 2 distinct ways of executing market orders in futures. Dropping sell bombs on upticks and driving in small orders on up ticks at a fast clip..

 

One hypothesis is that the stream of orders is generated by equities bots whereas the larger sell orders are generated by institutions or the futures LQ providers. Suspect most institutions sit limit and refill... or define a range they sell into.. LQ providers tend to sit at extents of that range.. from my tape reading skills

 

I notice that the order flow tends to often be highly serially correlated while the price tends to be the opposite.. not sure what that means either.

[BlueHorseshoe 10]

For sure not an expert on HFT here or market micro structure. Only know what I've read at NANEX etc.. Not a stock trader either so I don't much experience on that.

 

Likewise - the 'Nanex Daily Crop Circles', a handful of other blogs and sites, and three or four books. The charts on this second post came from an article about Nanex, incidentally.

 

The idea behind the thread was to post things I felt might be of interest as I came across them myself, and hopefully help others who are trying to think about how other market participants may impact upon their own daytrading.

 

With equities futures.. you might think a move like that was maybe driven by changes in underlying..perhaps the quotes in the baskets the underlying were changed. See arbitrage.. triangular arbitrage etc.

 

There's definitely a dearth of information about HFT in futures, and the only info I have found has related to the role it plays in arbitrage against baskets/ETFs, exactly as you suggest.

 

I also observed what appeared to be 2 distinct ways of executing market orders in futures. Dropping sell bombs on upticks and driving in small orders on up ticks at a fast clip..

 

One hypothesis is that the stream of orders is generated by equities bots whereas the larger sell orders are generated by institutions or the futures LQ providers. Suspect most institutions sit limit and refill... or define a range they sell into.. LQ providers tend to sit at extents of that range.. from my tape reading skills

 

I notice that the order flow tends to often be highly serially correlated while the price tends to be the opposite.. not sure what that means either.

 

Participants with longer term outlooks sitting at limit as you describe is certainly something that would fit with my expectations.

 

One thing that I find interesting is that although such firms can complete a round trip in the time it would take you or I to move our hand to the mouse, a trader can still potentially recognise their activity in the order book and make use of this information. Although, I hasten to add, I couldn't read the tape to save my life . . . !

 

BlueHorseshoe

[BlueHorseshoe 10]

related ?

Chart of the day, HFT edition | Felix Salmon

 

Yep - the chart on the second post came from an article about Nanex at TheAtlantic.com. The Nanex site itself is great if you get kicks from looking at endless examples of this sort of stuff.

 

By the way, I'm not bothering listing sources as I assume it's pretty much obvious that none of this is my own research (would that I were that smart!!!).

 

… the ppl and bots are all really in the same boat – for all, the only successful line of attack is long when price is moving up and short when price is moving down

 

That's not technically correct - breakeven trading for rebates, latency arbitrage etc - but I do take your point.

 

… ie 'they' can jitter around with bid and ask till they run out of bandwidth but ultimately price must move up or down for ‘anyone’ to make any money - excepting superior premium writers of course …

... ie their timeframes have no impact on my time frames...

 

SIUYA made a similar remark, and I think it's perfectly valid. I think that it is good to have a general knwledge about all aspects of the markets you operate in, but I would also agree that if you're making money without worrying about this stuff then there's no need to start worrying about it. Despite the sensationalist title, this thread is mostly for curiosity.

 

BlueHorseshoe

[Predictor 11]

Tape reading goes way beyond the order book... it doesn't require lightning fast reflexes, either. I wouldn't claim myself an expert at much but tape reading is one of my strongest abilities.

 

As others stated, as a short term trader I am not interested in the arbitrage anomalies that are gone before I can even trade them. But, I don't want to be the most ignorant trader of the bunch... as i wrote in my newsletter, the goal isn't to be the most informed (as a trader) but I don't want to be the least informed.

 

 

As for HFT-type solutions for retailers.. I'm interested in such capabilities and am surprised there aren't more offerings in that space. On the other hand, I see many types of hurdles for anyone trying that style of trading... I don't see true HFT coming down to retail levels but "retail" algorithmic trading in the sub second space should be possible.

 

 

 

One thing that I find interesting is that although such firms can complete a round trip in the time it would take you or I to move our hand to the mouse, a trader can still potentially recognise their activity in the order book and make use of this information. Although, I hasten to add, I couldn't read the tape to save my life . . . !

BlueHorseshoe

Edited August 8, 2012 by Predictor

[MightyMouse 12]

 

Although, I hasten to add, I couldn't read the tape to save my life . . . !

 

BlueHorseshoe

 

Sure you could.

 

Put up 2 T&S, one unfiltered and another one that is filtered. Also, put up a DOM, a tick chart and a longer term chart.

 

Stare at it and It will click. Once it does, you'll know what is going on by glancing at it. It might take you a day, 2 days, maybe 3 days. I highly doubt it will take you longer than that.

[Ingot54 13]

Might be a tad off-topic, but there is a certain schadenfreude in this article

 

Why Knight lost $440 million in 45 minutes - The Term Sheet: Fortune's deals blog Term Sheet

 

Those with the fastest algo don't always win ... :missy:

[Predictor 11]

lol.. this is funny for me to read as a tape reader. I read tape for over several years and it was always something I felt I was good at before I developed the skills that I have today. If you are gifted tape reader then I think it will take 6 months to 2 years to develop the basic skills (without training and 6 months minimum regardless). But in order to advance to the higher levels, it requires a lot more then just staring at quotes. My 2 cents..

 

 

Sure you could.

 

Put up 2 T&S, one unfiltered and another one that is filtered. Also, put up a DOM, a tick chart and a longer term chart.

 

Stare at it and It will click. Once it does, you'll know what is going on by glancing at it. It might take you a day, 2 days, maybe 3 days. I highly doubt it will take you longer than that.

[SIUYA 11]

sticking with my theme - for all those that complain about the bots scalping you/loading up orders/market making with ultra tight spreads and clipping you......

think about what it was like before the HFT. (no offense Blue as its an interesting subject, but 'enemy' is a bit harsh.)

 

HFT have reduced spreads and increased liquidity - whether they have produced other problems as a result of this might be another discussion, but I dont think its fair to call them the enemy of the retail trader.

I mean ....previously when you had to phone your order to the broker, he phone his clerk, they went across the floor, the market maker gave you a spread....blah blah blah.....I find it funny that people complain about the service they get now days as a retail trader.

 

Even with all the latency trading, and rebate fluff - these guys are market making and arbitraging - its only us poor schmoes who actually try and make money out of speculation, and that IMHO will never change. Find value entry, cut if wrong, run if right, rinse repeat. the one thing we all have in our favour is that we dont need to trade......until it suits us.

anyways... i still enjoy the threads and ideas, and do think there are other issues as a result of HFT (they may have more to do with multitudes of exchanges, poor incentives, non level playing field, lack of real control, correlation of markets and other issues). thanks

[MightyMouse 12]

lol.. this is funny for me to read as a tape reader. I read tape for over several years and it was always something I felt I was good at before I developed the skills that I have today. If you are gifted tape reader then I think it will take 6 months to 2 years to develop the basic skills (without training and 6 months minimum regardless). But in order to advance to the higher levels, it requires a lot more then just staring at quotes. My 2 cents..

 

I am glad I can amuse you.

 

What is also amusing is that you make this seem so much more difficult than it is.

 

If bluehorseshoe puts up the indicators as I suggested and watches for a short period of time, he will begin to understand what is going on in just a few days. I am not stating that he will be as proficient as someone who has done it for 1, 2, or 5 years, But, he will be able to see the entered orders and executed orders, how the sum of traders position themselves at various points in the market and at various points during the day, develop a read for the direction of order flow, and he will learn to tell when support or resistance may or may not hold. Once he gets these basics, he can then begin to implement them into his discretionary trading. Without tape reading, the best one can do is be a victim of target and stop.

 

I am assuming, too, that bluehorseshoe knows what trading is. If he has to learn to trade at the same time that he is learning to read the market, then it may take him as long or longer than you are suggesting or he may never get it.

 

It takes a lot of time for traders to learn how to trade because they first try to wade through all the bullshit only to find that most everything they read or learn is bullshit. So, a lot of time gets wasted. Most traders drop out, IMO, because they run out of time which I think is very different from failing. An instructor like you should greatly shorten the time cycle for a new guy. What else are you offering if you are not offering a short cut in time?

[BlueHorseshoe 10]

I highly doubt it will take you longer than that.

 

I'll happily take the other side of that bet!

 

However, I have been spending a lot of time recently exploring mechanical approaches to analysing order flow. A number of the inputs are known, as are the outcomes at a time t[-1], so assuming that some kind of non-random relationship exists then it should be possible to model it. As with all such problems, it's a lot easier said than done.

 

BlueHorseshoe

[BlueHorseshoe 10]

Might be a tad off-topic, but there is a certain schadenfreude in this article

 

Why Knight lost $440 million in 45 minutes - The Term Sheet: Fortune's deals blog Term Sheet

 

Those with the fastest algo don't always win ... :missy:

 

Not even slightly off topic, and an interesting article - thanks!

 

The piece mentions the NYSE's 'Retail Liquidity Program', introduced on the day of Knight's misfortune. For anyone interested, then Nanex uncovered a similar algorithm operating on the Nasdaq which transpired to be that exchange's equivalent liquidity program.

 

BlueHorseshoe

[BlueHorseshoe 10]

It takes a lot of time for traders to learn how to trade because they first try to wade through all the bullshit only to find that most everything they read or learn is bullshit. So, a lot of time gets wasted. Most traders drop out, IMO, because they run out of time which I think is very different from failing.

 

Well, time or money . . .

 

I have said this many times elsewhere on TL, but I think one of the most important things someone can do to accelerate this process in terms of time, and decelerate it in terms of financial loss, is to learn to test ideas properly across large data sets. In other words - learn to program. This does not neccessarily imply an automated trading approach, it doesn't mean trying to devise 'holy grail' indicators, and it certainly doesn't suggest optimising the hell out of a set of parameters until they're curve fit to historical data.

 

I'm getting a bit off-topic, but there we go!

 

BlueHorseshoe

[BlueHorseshoe 10]

(no offense Blue as its an interesting subject, but 'enemy' is a bit harsh.)

 

None taken. The sensationalist title is just a way to get people reading . . .

 

I think one vital aspect of the comparison with traditional human market makers is the capacity for error.

 

A human can make errors in the sense that they can make an invalid assessment of various inputs. A simple algorithm cannot; it will always process the information it is presented with according to the parameters of its functions. Whether or not the parameters are appropriate is, of course, dependent on the human who creates the algorithm.

 

So far so simple. But the algorithms that are supposedly so rampant in modern markets (in all timeframes) are not so simple. They have the capacity to adjust their own parameters, and in many cases decide their own inputs and functions. As they crawl the terrain of the error space they may find themselves trapped in ravines from which they are not able to escape to areas of lower error, unable to make the leap over surrounding 'high ground'. They can be empowered to do this by giving them sufficient 'momentum', but then the algorithm that controls this is subject to the same caveats of local minima in its own error space . . .

 

In other words, an algorithm can be 'correctly' programmed with appropriate parameters and still get it wrong, exactly like a human market maker who makes poor decisions based on the information he has available.

 

BlueHorseshoe

[Paul-TC 10]

think about what it was like before the HFT. (no offense Blue as its an interesting subject, but 'enemy' is a bit harsh.) ...........

 

HFT have reduced spreads and increased liquidity -

 

A common misconception - probably initiated and perpetuated by the HFT crowd which attributes the reduced spreads to HFT rather than the real cause - 'decimalization'.

 

It was decimilization that was responsible for the decreased spreads in Stocks by 2002 when HFT still only acccounted for about 10% of the market.

 

http://www.gao.gov/new.items/d05535.pdf

 

FWIW, HFT doesn't increase liquidity either. It increases volume - not the same thing - especially in melt-ups/downs.

[zdo 20]

 

That's not technically correct - breakeven trading for rebates, latency arbitrage etc - but I do take your point...

 

...

 

SIUYA made a similar remark, and I think it's perfectly valid. I think that it is good to have a general knwledge about all aspects of the markets you operate in, but I would also agree that if you're making money without worrying about this stuff then there's no need to start worrying about it. Despite the sensationalist title, this thread is mostly for curiosity.

 

BlueHorseshoe

 

 

...while reporting that in my own experiences I've been apparently uneffected, I should have also taken the time to acknowledge more of the host of obstacles, costs, and (systemic,etc) risks we, the individual traders, must literally overcome or at least brush off ... the corruptions accompanying fiat, in my thinking, is more of a threat than the houses gaming the system with HFT, etc... see charles hugh smith-A Common-Sense View of the Stock Market , etc

 

From recent incidents, it appears that most often the damaging effects of HFT would fall on the HFTer's themselves... In any event, while the media fearmongers wring word hands ad nauseum over HFT'S threats to the whole 'system' (...what they're really worry about is price levels of course...) - individual traders need to be remain prepared to utilitize any HFT generated "flash crashes", etc. as opportunities... For example, resting orders placed a while ago that were set to expire early next week made me a few bucks off the recent "HFT" flame up in EURCHF see "RBS Algo Went Berserk" | ZeroHedge... this is not a very 'active' / 'instrumental' example ... but still is in keeping with the clustering an opportunity orientation with "anything can happen" / occasionally things are really going to illustrate/manifest the underlying 'non parametrics'...

Edited August 9, 2012 by zdo

[SIUYA 11]

individual traders need to be remain prepared to utilitize any HFT generated "flash crashes", etc. as opportunities... For example, resting orders placed a while ago that were set to expire early next week made me a few bucks off the recent "HFT" flame up in EURCHF see "RBS

 

a friend of mine who is a deep value investor does just this - he has resting buys and sells at levels he wishes to purchase and sell and figures he might just get lucky on a few occasions more than previously.

[MightyMouse 12]

I'll happily take the other side of that bet!

 

However, I have been spending a lot of time recently exploring mechanical approaches to analysing order flow. A number of the inputs are known, as are the outcomes at a time t[-1], so assuming that some kind of non-random relationship exists then it should be possible to model it. As with all such problems, it's a lot easier said than done.

 

BlueHorseshoe

 

Ok, if you are going to do it that way then it will probably take you 2 years. :haha: