Best Strategy Backtesting Software

[BlowFish 10]

If Ninja 7.0 delivers what is being claimed on the recently published feature list then it goes a long way to fixing some of it's shortcomings. Lets face trade station has major shortcomings of pretty fundamental things. Still it is somewhat imature.

 

I don't know much about IRT I hate the lease model to be honest, I'd rather buy an outright license. I did try it once (probably over a decade ago) I had a few stability issues guess they have had plenty of time to fix those

 

People might want to concider NeoTicker (though it has a steep learning curve) it has had some remarkable features for 'hardcore' development for some years. Things like there tickprecise technology and grid optimisation. Quite a remarkable product. Mind you any backtesting needs to offer tick by tick processing 'inside' the bar.

[ZOSO 10]

ZOSO,

 

Don't most of the customers at EVERY Brokerage lose money? Didn't they risk capital building Trade Station and their other ventures?

 

I don't understand your complaint/praise. Is it becasue they build good software, because some of their customers make money, because you can't make money using their software or just because you are feeling negative and not sure what about?

 

The Cruz brothers have built a substantial company with a very competitive platform that I, thousands of other customers, Barron's and "The Technical Analysis of Stocks and Commodities" find to be the best around. So what is it you are mad about?

 

 

:spam:

 

This is your third post touting TradeStation that includes a copy of their ad. Obviously, you have a vested interest in TradeStation other than being a satisfied client.

 

Stop promoting a vendor and offering testimonials here--it's inappropriate. If TradeStation wishes to become a sponsor, that's fine; have your Cruz buddies contact James.

[UrmaBlume 10]

:spam:

 

This is your third post touting TradeStation that includes a copy of their ad. Obviously, you have a vested interest in TradeStation other than being a satisfied client.

Stop promoting a vendor and offering testimonials here--it's inappropriate. If TradeStation wishes to become a sponsor, that's fine; have your Cruz buddies contact James.

 

No vested interest, but don't worry, if James thinks my comments about Trade Sation are spam he will not hesitate to act. He hasn't and you're wrong - again.

 

I believe that TradeStation is the best and if not the best at least among the best packages around.

 

Backtesting is one of the package's strong points. is the topic of this thread and as a long time user - I will indeed continue to make positive comments about the package whenever I think it is appropriate.

 

Seems like your posts to this particular thread have nothing to do with the topic - backtesting software. But more than that you seem to be mad at me, mad at TradeStation or just plain mad.

 

As to what you think is appropriate - please try and measure/imagine the degree to which that opinion will influence my future thoughts and actions.

 

 

cheers

[ZOSO 10]

Thanks for not posting the TradeStation ads for the 4th time.

 

We all know how you feel about TS. You've made your point ad nauseam.

 

cheers

Edited June 30, 2009 by ZOSO

[bgtrader 10]

Hi everyone,

I wanted to chime in with a few thoughts:

1. My trading buddy loves Tradestation for backtesting and developing strategies and has been doing so with nothing but praise for around 2 years. He recently told me how psyched he was that TS's backtesting feature allowed you to tweak the order fill parameters to make getting your fill a bit more realistic. I, currently, use Thinkorswim as my trading and charting platform, and they plan to offer backtesting on their updated platform in the near future. Though my hunch is it won't be as robust as TS or Neoticker, but still worth visiting their cool free archives about.

2. In addition to Urma's awesome and professional trading group liking Tradestation, another serious trading group that really likes tradestation's testing features are the good folks at Hawkeye Traders (you can look up their website). They have developed innovative (not as innovative as Urma's software imho) ways of strategy testing in TS that they call "Hawkeye Flex Strategy". And, in webinars when asked about various software packages, their founder Nigel Hawkes prefers tradestation too. I don't think I have to mention this, but I wrote about Hawkeye here not as an ad for them but rather as a "hey, go read these guy's free material, it might inspire some new backtesting techniques/ways of using backtesting software,etc...ideas"-kind of thing.

3. I wish folks would reserve strong accusations towards posters when there is really something blatant that's not appropriate. I know it's only text, but I can feel the anger and the mean energy coming through the words in some of the posts towards Urma. As far as offering testomonials and stopping that, while I understand the need to protect the site from unsponsored vendors, we are all offering the "testomonial" of our experience in this thread; the danger I see is that the more folks are trigger happy with accusations, the more restricted one might feel in expressing themselves, and then things just get awkward. I understand that folks have to express themselves when an insight or opinion arises; also, I understand that at this site it's important to maintain an integrity regarding no vendor promo and all that . I am just suggesting that it could be done in a friendly manner in keeping with building friendships and community. Such as asking a question rather than accusing with sarcasm, and/or attempting to first assume that everyone's here to learn and share and realize that due to the inherent problems in language in not being able to fully communicate reality--and the compounding of us all using computer text here and how that limits our communicative abilities...perhaps considering how limited our language tools are here in cyberspace can humble us a little and make for more gentle and friendly posts towards one another. -best wishes to all, B:cool:

Edited July 2, 2009 by bgtrader

[Blaiserboy 10]

Has someone developed a method of comparing the various platforms based on certain aspects of backtesting and optimization...... If so.. can you point me to it.

 

I have another question..perhaps this is the wrong thread......

 

'What would be the target areas to aim for when doing an optimization'......... perhaps someone has some research that they will share.

 

Thanks so much

 

Have a great day, everyone...!!!!!

[Tams 20]

new features just released on MultiCharts v5.5 beta 1

 

 

Backtesting and Optimization

 

• Walk-forward optimization has been added.

The walk-forward optimization can be used with both the Exhaustive and Genetic optimization types.

The walk-forward optimization makes it possible to run optimization on the in-sample data and then backtest the strategy on the out-sample data with the best parameters found during the previous step.

 

• High-precision backtesting has been added.

The new Bar Magnifier feature allows for backtesting with higher precision to simulate the real-time price movement within a bar of a selected data resolution. The precision levels range from tick-by-tick through minute to daily. The desired precision level can be selected both for time-based and count-based bars.

[Blaiserboy 10]

BGtrader

 

Please elaborate on Urma's software and group

 

 

thanks

 

 

dave

[edeeb 10]

the answer really depends on needs, abilities, and what the software is going to be used for. none are necessarily 'better' just different with areas of strengths, and weaknesses.

 

tradestation's strengths: tons of data, a large user group with add on products and programmers, lots of built in strategies/indicators to use as a base, good charting, relatively easy to program, etc

 

weaknesses: doesn't time stamp tick data, expensive if you're not a client, and a very big one, lack of portfolio testing.

 

neoticker can do things that tradestation simply can't do, however, it takes much more time to program things, has bugs here and there, is difficult to learn etc,

 

have not used ninjatrader, but it has some strengths the others don't, free to use until you are automated, lots of broker connections, decent user base, etc,

 

i use neoticker and tradestation at the moment, but must employ a programmer for the former. hope this helps.

[trade-samarai 10]

Hi UrmaBlume & Tams:

 

Urma, what TS ad-on Genetic Optimizer do you recommend and why? I wish to speed up TS's backtesting. It is soooo slow. From my immature understanding of back-testing, TS uses brute force rather than smart GO and thus wastes time number crunching on outlier tests that are away from the sweet spot. Is this correct or just propaganda from add-on venders?

 

Tams, how would compare MC to TS for backtesting? I know that you're a user of TS.

 

Thanks in advance.

[Tams 20]

...

Tams, how would compare MC to TS for backtesting? I know that you're a user of TS.

 

Thanks in advance.

 

 

I have both TS and MC.

 

TS uses only one CPU core in its operation, even if you have a multicore computer.

while MC utilizes all available CPUs.

 

here's a screen shot of the Task Manager during one of my MultiCharts backtest.

you can see all the CPUs are at max'ed !

 

[UrmaBlume 10]

Hi UrmaBlume & Tams:Urma, what TS ad-on Genetic Optimizer do you recommend and why? I wish to speed up TS's backtesting. It is soooo slow. From my immature understanding of back-testing, TS uses brute force rather than smart GO and thus wastes time number crunching on outlier tests that are away from the sweet spot. Is this correct or just propaganda from add-on venders?Tams, how would compare MC to TS for backtesting? I know that you're a user of TS. Thanks in advance.

 

Samarai,

 

Trade Station's most recent upgrades include a genetic, survival of the fittest, optimization routine that cuts optimization runs to a fraction of the time required by brute force.

 

I have been using Trade Station and other products from Bill Cruz and his brother longer than anyone and the only 2 issues I have with Trade Station are 1) the lack of granularity of their time stamp and 2) No optimization for multi-core technology. I have information from inside Trade Station that they are working on both of those issues.

 

We have a shift to Multicharts under consideration especially since MC uses Easy Language and we will be able to port any of the several thousand indicators, strategies and functions that we have written over the years. The issue for us is that we are very familiar with TradeStation and would hate to change and start almost from scratch if it is just a matter of a few months until Trade Station implements these changes.

 

We have been able to get around the time stamp issue with certain dll's and for some or our optimizations we have built our own engine that runs outside TS. Our most recent work is about intra-bar (inside the bar) trading.

 

BTW San Antonio can be a very fun town. I was born there and spent my summers on nearby Lake McQueeney.

[Tams 20]

MultiCharts v5.0 Gold

 

Backtesting and Optimization

 

• Backtesting can now be based on bids and asks.

The new backtesting has two modes: classical and extended.

In the classical mode, backtesting will be based on the underlying data series (trades, bids, OR asks).

In the extended mode, backtesting will be based on bids AND asks.

 

• Improved Genetic Optimization parameters.

Backtesting-Chart Synchronization has been added. It is now possible to click on a point on one of the available Equity Curve lines or on a trade in the List of Trades and have the chart scrolled to the respective trade and have the entry arrow highlighted.

 

 

the following features have been added to the MultiCharts 5.5 Beta

 

Backtesting and Optimization

 

• Walk-forward optimization has been added.

The walk-forward optimization can be used with both the Exhaustive and Genetic optimization types. The walk-forward optimization makes it possible to run optimization on the in-sample data and then backtest the strategy on the out-sample data with the best parameters found during the previous step.

 

• High-precision backtesting has been added.

The new Bar Magnifier feature allows for backtesting with higher precision to simulate the real-time price movement within a bar of a selected data resolution. The precision levels range from tick-by-tick through minute to daily. The desired precision level can be selected both for time-based and count-based bars.

[UrmaBlume 10]

And for those that really want to get a bit deeper into optimization theory check out the attached article on "Ant Algorithms for Discrete Optimization."

 

From the Introduction - "Ant algorithms were first proposed by Dorigo and Colleges as a multi-agent approach to difficult combinatorial optimization problems...."

IJ.23-alife99.pdf

[lordbinder 11]

first i must state that i consider optimization as the mother of all f**kups.

 

Having said that, the best tools are both Ninjatrader and matlab

 

Matlab, when one has the ability to program at a certain level, goes beyond anything.

 

For quick and dirty (and for people not familiar with programming) NinjaTrader is excellent. I have the opportunity to use NT7 and indeed solves a lot of problems regarding resources use.

 

Two additional interesting programs for system design are quant developer and rightedge

 

But I repeat the best actually is matlab

[Tams 20]

curve fitting uses optimization.

 

not all optimizations are curve fitting.

 

there is a difference.

 

 

.

Edited August 3, 2009 by Tams

[trade-samarai 10]

Thanks for responding Urma and for the AntOpt paper. Nature is a great teacher.

 

I'm not planning on dumping TS, just maybe adding on. I've had problems with 8.5 back-testing giving multiple spurious orders (at least with LIBB on MACD). Two different TS employees admitted the spurious order problem (one indirectly) to me but said that it had been addressed in the upcoming 8.6 (still in beta testing). The tech had the exact same problem on her chart. I came up with a system to quasi-GenOpt myself (just today) and I did find the GenOpt on 8.5 today also. Thanks for that tip.

 

I'm trying to improve and anything that is relatively repetitive should be automated IMHO. Who knows how long it will take TS to get all of the bugs worked out of 8.5/8.6 (like MS Vista)? And, like they say: "time is money." And trading untested discretionary systems is most certainly a great way to lose money. So if an add-on costs some money that's OK if I get a good ROI on it.

 

BTW, I've read that MC has (or had) a digital rights verification problem.

 

I found your 24-point post indicator posted here on TL most interesting. It's far more complex than my piddly little 6 to 10 part strategies. But, I'm just going through my EL home study courses now and am by no means an EL expert. I couldn't help but notice that your indicator seems to be leading the price. Of course, it begs the question: What's the performance?

 

P.S. I know Lake McQueeny. As for Vegas, I gambled there professionally for a short while in the 90's (blackjack). I've been following the real estate market there (58% underwater).

[trade-samarai 10]

Thanks for the additional tips lordbinder and Tams.

[JKLM 10]

can anyone recommend good data source for MC backtestiong and trading

I use IB as broker , and they dont have contionous data contracts and tick data ...

[alex_laxya 10]

Genetic Optimization is available in Investor/RT (in the form of "express optimization") and auto-execution is available as well. The execution aspect is not a strength currently, but something that Linn Software is presently devoting a great deal of time to. Look for auto-execution to become a strength in the very near future (within a month or two).

 

In response to the OP, I guess it depends on what your needs are in regards to which is best. If you need execution, today, then I/RT probably is not the best for you. I think you'll find that it's much quicker/easier to code your system in Investor/RT using RTL than with "easy" language. For a taste of how easy RTL is, you can read through a tutorial I've been working on recently: RTL 101.

 

Just providing an alternative for the OP to look at.

 

Cant agree more than this, very very easy to code in Investor/RT.Especially one is discretionary trader and just want to back test few concepts real quick without much of a programming knowledge then Investor/RT is helpful. Gone through the thread from start to end but i haven't found any comments on coding market profile related indicators/signals.I used trade station as charting software but market profile and auction market related codes are tough to make.

One problem i am facing in Investor/RT while back testing is TPO related codes give error signals when backtest on more than 3 months data, it give accurate signals ( say exit at POC) from current to last 90 days but afterwords it start giving error signals( data is esig)

[LS_Chad 10]

alex,

 

I assume you are using the TPO indicator in your backtesting to get MP related prices like historical developing VAH, VAL, etc. The TPO indicator actually uses the same code as the MP charts which are restricted to displaying/computing a max of 120 days at a time. Theoretically, you should be able to backtest about 6 months at a time. The Profile Indicator has no such limitations, and might be a good alternative. It will give you the VAH, VAL, etc, but it will not give you the developing VAH, VAL, etc. If you'll email your trading system definition to support I'll take a look and see what we can do.

[alex_laxya 10]

alex,

 

I assume you are using the TPO indicator in your backtesting to get MP related prices like historical developing VAH, VAL, etc. The TPO indicator actually uses the same code as the MP charts which are restricted to displaying/computing a max of 120 days at a time. Theoretically, you should be able to backtest about 6 months at a time. The Profile Indicator has no such limitations, and might be a good alternative. It will give you the VAH, VAL, etc, but it will not give you the developing VAH, VAL, etc. If you'll email your trading system definition to support I'll take a look and see what we can do.

 

Yes I am talking about dev POC, way to tackle this problem is divide back testing frequency to 6 month each ?

Volume breakdown indicators also giving the same error .

Add/edit - problem with using profile token instead of TPO is value areas wont follow along with intraday price and i am backtesting day trading system, profile tokens are handy for reference indicators or confirm rule.

[Tams 20]

latest MultiCharts backtesting enhancements

 

Power Language

 

 Information on the market position can now be received directly from a broker.

 

Four keywords to request and receive the market position from a broker have been added.

 

a) Market_Position_at_Broker_for_the_Strategy returns the market position at the broker for a given symbol and a particular strategy. Changes in the values returned by the word are based on the events at the broker. The word can only be used in signals and functions.

 

b) Market_Position_at_Broker returns the market position at the broker for a given symbol (irrespective of whether the position has resulted from orders submitted through a strategy or manually). Changes in the values returned by the word are based on the events at the broker. The word can only be used in signals and functions.

 

c) I_Market_Position_at_Broker_for_the_Strategy is similar to Market_Position_at_Broker_for_the_Strategy, the difference being that the former can be used in all types of studies (including indicators).

 

d) I_Market_Position_at_Broker is similar to Market_Position_at_Broker, the difference being that the former can be used in all types of studies (including indicators).

 

 The type of the AverageFC function has been changed to the ‘Series Function’.

 

 

Backtesting and Optimization

 

 Walk-forward optimization has been added.

 

The walk-forward optimization can be used with both the Exhaustive and Genetic optimization types. The walk-forward optimization makes it possible to run optimization on the in-sample data and then backtest the strategy on the out-sample data with the best parameters found during the previous step.

 

 High-precision backtesting has been added.

 

The new Bar Magnifier feature allows for backtesting with higher precision to simulate the real-time price movement within a bar of a selected data resolution. The precision levels range from tick-by-tick through minute to daily. The desired precision level can be selected both for time-based and count-based bars.

 

 

Auto Trading

 

 Trading Technologies is now supported as a broker.

 Patsystems is now supported as a broker.

[snowblind 10]

To the O.P....

From my experience, (and I have used all software mentioned in this thread), Neoticker is by far the best for backtesting and optimization(see grid optimizer). However, it's brokers and data choices are limited. For testing, it's good to go with until you are ready to take it live.

I would say that Sierra is the best for stability and real-time speed, and not to mention C# coding. You don't want to run a live auto strategy on a buggy platform (ninja) or one with bad data (TS).

[Traders37 10]

It`s good nobody knows the backtesting speed and strength of AMIBROKER. Gives me an edge.

 

Try to backtest a portfolio of 10 symbols (1min-1 year), repeat it for 10 runs, compare results......

 

With AMibroker it wil take a couple of minutes and results will always be identical. Same with optimisation runs.

 

How long did it take your software ? So stick to your software, leave the edge to me.

 

Regards

 

Traders37