BlueHorseshoe

That makes sense, although if you look at the origins of Stat Arb and other mean reversion strategies, they generally didn't give a damn what caused the price movement. Having said which, they were also delta-neutral and heavily capitalised . . . But what you're effectively saying is that one event is 'some guy' who sold 1000 contracts, whereas the other event is the result of an informed participant. If this is the case, then maybe you could look at ways to distinguish the two (MightyMouse might have some suggestions on this?) - certain types of market participants traditionally are supposed to react at different levels. On the other hand, if you believe Hunsader, 99.9% of all posted quotes are by the HFTs, who might count as 'informed', but certainly don't care about anything that happens outside of the next couple of seconds, and won't therefore be moving markets with the longer term outlook that you're fearing. I think that what you're trying to do may be harder to implement than you think, but not impossible. Yep, I'm in the same boat, so I can empathise Have a good weekend! BlueHorseshoe

Scott Hunsader of NANEX, quoted from an alphaville article: BlueHorseshoe

On what basis do you assign fills to limit orders in the SIM environment? BlueHorseshoe

If the basic premise of your strategy is correct, then surely getting orders filled under such circumstances is the whole point? You'll just end up with a couple of dozen positions that mean revert rather than one. Or have I misunderstood? Furthermore, when do I seed the orderbook? I don't know, it's not something I've ever done (I don't have the capital )Given available capital, then as far out as you can? I imagine if you place a buy order in the ES down at 1239 today, then by the time price gets there (if price ever sees that level again) you'll probably be pretty near the front of the queue. Do I pull everything when German ZEW data is due out? I don't know - do you have any interest in German ZEW data? Does it have any relevance to your strategy? I would assume that if a big buyer/seller wants to come in off the back of such a data release then making a market for them is exactly what you want to be doing, not what you want to be avoiding? Maybe I'm confused by your goals . . . - Again I really do like your thinking, although I am still waiting for a flash of inspiration that can allow me to overcome my ridiculously high commission costs with a strategy that fills 5% of the time. Remember though, within reason you can control which orders you allow to be filled. Suppose your commission is $4 per round trip, which should be possible with enough volume (and you'll be generating plenty if you can get to breakeven on your 'compliance trades') and exchange membership through a leased seat. That means every 3 roundtrips cost you $12. So, if you can take your expected single-tick win-rate from 50% (coin toss) to 66%, you'll be $12.5 + $12.5 - $12.5 = $12.5 trading profit before a deduction of $12 commission takes you back to breakeven. Ultimately, whichever way you want to slice it, what you're wanting to do is a "big" idea, so you'll need to think big, not think like a retail trader. Things like exchange membership shouldn't be an issue. Yes, very interesting thoughts that would not have occurred to me, thanks for taking the time to write such a lengthy post - I sensed something big was coming after your initial shot-across-the-bow to Predictor The point Predictor was trying to make is actually reasonably valid - most people don't realise the impact that unfilled limit orders will have on their strategy. I'm assuming that you do. For general backtesting purposes it is best to grant that limit orders would only have filled if price traded through that level. When we're talking about single-tick market making though, the differences become too significant to ignore, and you either need an intelligent way to estimate this or enough realtime trading ruselts to know the outcome. Predictor gave you a glimpse of the latter when he mentioned a limit order placed twelve hours in advance that didn't fill. Predictor probably has many years of real time experience to draw these conclusions from. I'd prefer something a little less anecdotal before I put any money at stake, however. Given all of the above, I think you need to do a bit more research. You know that the further out from the market your order is placed the less chance there is that you will need to adjust it to avoid being filled by casual trading. And you know that the larger an elephant order the more price tiers it will likely take out in one hit, so the further you are from the market (up to a point) the more chance you have of getting filled only by a larger elephant, while the closer to the market the more chance you have of being filled by a smaller elephant. And you should know (I don't) the probabilities of a profitable outcome from different sizes of elephant order. Combining all three pieces of information, is there a sweet spot somewhere in there that will allow you to profit by market-making for elephants without too many cancelations to attract fines, and too few cancellations to get filled by casual trading too often? BlueHorseshoe

That's real swell, Alan, but I think your threads might be a bit more interesting if you got Lee in to do them - or aren't you two pals anymore? BlueHorseshoe

That's easily solved by using the principle of 'hysteresis'. Think of your thermostat. If you set it to 15, it doesn't switch the heating on every time the temperature falls below fifteen and off every time it rises above fifteen. Instead, it places a 'tolerance' band around your desired temperature. It may switch on at 13 and switch off at 17, for example. This reduces the number of on/off signals and minimizes wear and tear on components. In your scenario above, of course, you'll need boundaries around your boundaries! BlueHorseshoe

BEFORE my software has a chance to re-position my orders Hi AJ, This sounds interesting - so in a world where the HFT firms scramble for nanoseconds of improvement in execution, your algorithmic edge depends on the inherent latency of your platform? That's sufficiently far through the looking glass that it might actually work. I have absolutely no hands-on experience of any of this stuff, so my thoughts are purely theoretical . . . As has already been mentioned by The Dude, many large institutional orders will be algorithmically executed as iceberg orders. It is supposedly possible to reconstruct icebergs using quant models, but I have no idea how. An iceberg nowadays does not mean executing a 1000 lot as two 500 hundred lots. It means executing a 1000 lot as a random number of randomly sized blocks executed at random time intervals and with random cancellations. Unless you're the sort of person who thinks String Theory is a bit simple, you'll have your work cut out modelling that. I've also heard it said that when a 1000 lot type order hits in an obvious way, this is usually the tail end of such a buy/sell program - they've already filled 95% of their order, so they're not too worried about showing their hand with the last 1000 contracts. I have no idea how true that is. Given that you’re fading such orders, if they’re the tail end of a program then that’s probably a good thing. As Predictor points out, the CME execution to cancellation ratio is 1:20. There are two obvious ways to operate within that constraint. Firstly, don't perpetually 'float' orders at a specified distance from the market; just seed the entire order book above and below the bands. That way, when your algorithm calls for a 100 bid to be pulled and a new bid submitted at 99, you'll just need to pull the 100 bid as the 99 bid will already be sitting there. Secondly, play the game the way the CME want you to; devise a method for order placement that can pretty much breakeven (disregarding your primary strategy trades), and then allow 1 order in 40 to fill in the knowledge that these will net out to breakeven whilst keeping you compliant. It's 1 in 40 rather than 1 in 20 because you know that you'll need another filled order to exit that position. And remember that you don't strictly need to fill every 21st order, you just need to fill 5% of your posted orders by the end of the day. This means that you can be a little picky about how these are distributed through time, based on your expectation of a breakeven get-out. So, simplistically, you’ve cancelled 20 sell orders and you need the 21st to fill, so you leave it in place, it gets filled, and you immediately post bids a single tick above and below. That’s pretty much a coin toss for a 1-tick loss or a 1-tick profit, and could net out over a large enough sample size (although you’ve got your commission costs to factor in, so you’ll find you need to be a bit cleverer than that!). Another thing that may be useful to you, referring again to a point that Predictor makes, is a basic ability to estimate your position in the queue. This is another very difficult quantitative hurdle. I ran a thread on this subject here: http://www.traderslaboratory.com/forums/day-trading-scalping/13811-daytraders-do-you-know-your-enemy.html, although my posts are rather diluted by other’s contributions. I posted a model for a 'worst-case PIQ estimation'. I eventually ran into limitations in the data that I had available, and was unable to continue to pursue my ideas for modelling PIQ, but if you want to go down this avenue I will update that thread with my suggestions. Having an estimation of your PIQ would enable you to make an informed decision about the likelihood of getting filled with a limit order. Your algorithm will then be able to monitor decay in the rate at which the inside prices are being matched with market orders relative to your anticipated PIQ on either side of the book; when market interest wanes and you’re too far from the front of the queue then it may make sense to fire in a market order and guarantee a fill. I hope some of that is helpful, and let me know if I haven’t explained anything clearly. Regards, BlueHorseshoe

That's not at all true, Predictor. BlueHorsehose

All derivatives trading is gambling. Spread-betting differs only in the details, in exactly the same way as the details differ between options and futures, cmos and cdos, bonds and swaps, etc etc. With all of the above, it's crucial to understand the details and how they impact upon your chances of success. Sometimes the impact can be minimized. For instance, which would you prefer - 60k profit in futures, or 40k profit in a spreadbetting account? Unless you know a good tax lawyer, it's a no-brainer. So the only question is, can you make 40k as easily spread-betting as you can make 60k trading futures? To answer that you need to know the details of how each one operates. BlueHorseshoe

I think this depends on whether we're talking purely about the reliability of the signal, or whether we're talking about the capacity of a strategy based on that signal to make money in the real world. It wouldn't be hard to give you examples of signals from very short term charts which are highly reliable, but then once you've paid the spread and the commission you're left with (less than) nothing. Assuming that the size of the move you wish to exploit, the trading frequency, and the time frame are all directly proportional, and that we only consider performance metrics before deduction of costs, then the shorter term signal will tend to be more reliable. Factor in costs, and you're probably more likely to end up with a profit trading higher timeframes. BlueHorseshoe

You find out the formula that is used to calculate the $TICK, and then you apply it to data for the individual constituents of the index you want to trade. But if you're that hung up on the $TICK as a trading tool, why not just trade one of the US markets? BlueHorseshoe

Perhaps you should consider trading in the underlying futures market rather than using spreadbetting - the tax relief might appeal, but in the long run the spread and "execution irregularities" will kill you. If you traded futures and made money then you could get a tax expert to investigate ways of protecting your profits. BlueHorseshoe

I know - I sound like a stuck record! BlueHorseshoe ***MMS - you should have someone design a 'stuck record' emoticon for the site.

Hi Tams, I think that's a very useful distinction, and would ideally have been placed much earlier in the thread. With what you're calling 'raw' indicators, assuming we exclude price itself, then we are obviously chosing which data points we pay attention to (eg today's high). Is our selection method a form of processing? Also, a category three indicator which uses category one inputs is arguably less abstracted than one using category two inputs. For example, a Donchian mid line (considered as a 1-period average of upper and lower channels) is rather different to the momentum type oscillator obtained by subtracting one moving average from another. Is the Donchian mid-line, for exampe, even processing two other indicators, or is it a one-time process enacted upon selective data? BlueHorseshoe

Hi OptionTimer, Thanks for your reply. In my experience, what I was describing would greatly increase the percentage win-rate, and also reduce the payoff, assuming a dollar optimal exit or similar was used. As far as a mechanical system goes, I am almost certain that gunning for a 125 point excursion in the ES is neither dollar nor win-rate optimal - what is 125 points as a percentage of yearly range in that instrument? If you exited all positions with a profit target at the high/low prior to the correction, what effect would this have on performance? I would guess more trades, shorter holding time, higher percentage win-rate, and lower average profit per trade. As I said before, if we were talking about countless other markets then this would be madness, but for the ES, historically, I feel it's an unavoidable reality. BlueHorseshoe

Hi SIUYA, Can you explain what you meant by the above in a bit more detail? For instance, I wasn't suggesting waiting for confirmation that the short term counter trend move had finished (this was my point) - I was suggesting buying/selling into the counter trend move rather than awaiting confirmation. As for matching tools to the job, I would like to point out the following in relation to 'confirmation turns' in banded oscillators . . . If you study the calculation involved you will see that something like the RSI compresses extreme values in a similar fashion to a sigmoid function. This means that as the indicator value approaches 100 or 0 the price counter-movement that is required to cause the indicator to 'turn' becomes infinitely small. As such, the information content of indicator value changes at such levels becomes minimal. The fact that an RSI was at 100 yesterday but has now turned downwards is telling you virtually nothing at all (although the fact that it reached 100 is arguably very useful information). Finally, my earlier comment was not a criticism of OptionTimer, who is clearly a far more experienced and capable trader than I am; I just wanted to highlight that indicator use should be market specific as well as system specific, and that understanding the actual historical behaviour of a market, rather than its idealised or perceived behaviour, is vital. BlueHorseshoe

Hi OptionTimer, I'm currently enjoying reading through your excellent thread from last year. I'd like to question the paragraph above, however, as I think this is very much dependent upon the market being traded. For instance, taking the approach described in your thread, waiting for an impulse move back in the direction of the longer term trend following a pullback in a market like the ES would not be a wise approach; this market does not exhibit sufficient follow through to justify awaiting this 'confirmation move'. In the ES, it would be more profitable to buy into the pullback (ie buy the oversold reading as the market is falling) rather than to wait for the indicator to turn up again, or for price to close higher. Indices in particular seem to exhibit this type of behaviour. However, as a generalisation across diverse asset classes (which the notional portfolio in your thread is presumably designed to illustrate with various commodity and currency futures), then you may well be correct. BlueHorseshoe

Are you fishing for tips? A serious (if somewhat vague) response to the OP, is that I am interested in anything that achieves the following: Any totally recursive method of long term trend definition. Any approach for the short term normalisation of price. Any statistically appropriate and robust data-mining technique capable of measuring auto-correlation and mean reversion tendencies along with historically optimal thresholds in number 2 above. Ironically, although that was an attempt at a meaningful response, it's probably even greater gibberish than my earlier sarcasm! BlueHorseshoe

Very sorry for not contributing in the exact way that you would like me to, SMMatrix. Here, is this better? Simple Moving Average Donchian Channel GRAB Index* That's obviously a much more informative and on-topic post, and I'm sure everyone will have learnt a lot from it. Regards, BlueHorseshoe * The GRAB (Goldfish Revolutions Around Bowl) Index was the result of my attempts to incorporate aquatically-backpropogated non-artificial neural networks into my trading. Basically, I generated a Pavlovian response by only feeding the goldfish following a significant price change. Their tank now sits opposite my screens, and every time a big move is about to unfold the fish sense this and increase the rate at which they swim around their bowl. This has worked very well for me, except during the flash-crash, when the fish swam so fast that they churned their water to boiling point. Apparently RenTech were developing a similar idea but with wolf-cubs, but abandoned the experiment because the combined costs of commission, slippage, and wolf-feed were too high . . .

Hi Vince, Sorry - I only just stumbled across your reply! The book (Dark Pools) goes into quite a bit of detail about the SOS Bandits, yes. Basically, the original bandits operated out of the same office as the programmer who put the ISLAND exchange together, so they arguably had an unfair advantage in exploiting it, and probably about the best collocation in history (on the shelf above the exchange server!). They were a shady bunch by all accounts, and I think lots of them found the way to jail about the same time that ISLAND went legit. In terms of your later point, it's exactly the sort of 'loophole' that I continue looking for - I'm not fussed about the ego satisfaction of watching a chart pattern unfold having 'predicted' the market ket correctly - give me bucket loads of cash and I'll be happy What did Sekoyta say - "everyone gets what they want from the markets". BlueHorseshoe

Well, sort of . . . The close of day is at the end of 5 minutes though, isn't it? But I do see that you're trying to respond to what someone else asked. BlueHorseshoe

Hi Tams, In every single instance on this chart the stock had gapped to give you a worse price buying on the next day's open. If you're using EOD closing price, would it not be better to enter on the close? And if you don't want the overnight exposure, why not just trade with opening price data? BlueHorseshoe

I think that you've probably answered your own question there - either lack of liquidity or a large buyer who'll happily chase the market up to 341 but won't pay higher. Try looking at a single tick chart to see exactly how the stock ticked up to that price. As for the reason why - who knows? Stick with Morrissons, that's my advice! BlueHorseshoe

Hi Tams, Do you use R? BlueHorseshoe

For a non-intraday view, take a look at Open Interest with the COT Report here: Commitments of Traders - CFTC Be careful about reading too much into the breakdowns though - commercials get it wrong sometimes (and have deep pockets to ride it out as they generally hedge physical stock in the underlying), and speculators often get it wrong as well, depending on what type of speculator is making up the volume in any given week. BlueHorseshoe