Extensive Historical Data

[Dinerotrader 10]

Okay. I have been looking into getting historical data in varying amounts but now I am thinking, "doesn't someone just offer all available historical data for each given future?" I just want all the data down to the tick level for a few different futures.

 

I trade oil a lot and say I want to go back to see how price reacted to weekly inventory numbers each week for the past 2 years. I don't want a day, hour, or a 5 minute chart. I want to be able to look at what price did sometimes at the tick level.

 

So my question is who offers this and how much should it cost? I guess it is just a bunch of data with a price and then from there you can just use any ol' charting software to show you charts of whatever time frame you want in whatever scale you want.

 

I get ideas for different trading strategies all the time that I simply cannot even start to evaluate because I don't have nearly enough data.

 

Also, is there a good way to start saving the data I do receive (I use OEC) so that I can keep tick level data even when OEC stops making it available on their service?

[Tams 20]

usually the respective exchange offers the data for sale.

 

A friend recently bought 20 yrs of ag future data,

it set him back $900.

 

(I don't know how many symbols or exact # of contract months)

[Dinerotrader 10]

usually the respective exchange offers the data for sale.

 

A friend recently bought 20 yrs of ag future data,

it set him back $900.

 

(I don't know how many symbols or exact # of contract months)

 

So I just call them up and say, "hey there, I am looking for a boat load of oil data, who do I need to haggle with to get a good price on 5 to 10 years of CL data?" or something to that effect.

 

Maybe I just need to find out if some other trader already bought this data and is willing to sell it to me. Thanks for the ideas.

 

Is there a specific charting service that is cheap that I can plug a bunch of expensive data into it and do my analysis?

 

Edited December 16, 2009 by Dinerotrader

[Tams 20]

So I just call them up and say, "hey there, I am looking for a boat load of oil data, who do I need to haggle with to get a good price on 5 to 10 years of CL data?" or something to that effect.

...

 

 

.........check their website under Market Data Service

data is intellectual property...selling data is a business

 

http://www.cmegroup.com/market-data/licensed-quote-vendors/index.html

 

 

you are not the first one, nor the last one, to look for historical data.

Edited December 17, 2009 by Tams

[Head2k 10]

Out of curiosity I googled "historical tick data" and several companies popped up. The first one in the list offers CL since 1987.

[Dinerotrader 10]

you are not the first one, nor the last one, to look for historical data.

 

Your kidding me. I thought for sure I was the first and probably the last.

Your a funny guy Tams.

 

I'll look into this.

[brownsfan019 17]

Quite awhile ago I purchased some data and it wasn't that costly. For the life of me I can't recall the name of the site or company but somewhere (buried deeply) on TL is the link. I'll see if I can find it somewhere around here.

[Dinerotrader 10]

Quite awhile ago I purchased some data and it wasn't that costly. For the life of me I can't recall the name of the site or company but somewhere (buried deeply) on TL is the link. I'll see if I can find it somewhere around here.

 

I figured someone would have already gone through this process and evaluated what is the easiest cheapest path.

 

CQP.com looks like the main google search result so far.

 

I need the CL globex, correct (noted below)?

 

[sevensa 10]

I've heard good things about Tickdata. Never used them and I think it is on the expensive side (which is probably relative).

[brownsfan019 17]

Dinero - another option would be to use TradeStation for this kind of stuff. I was on their site and here's what data is available: http://www.tradestation.com/strategy_testing/hist_market_data.shtm

 

Over 27 Years of Intraday Futures Data

TradeStation also provides the historical data you need to thoroughly test your futures trading strategies. You'll have access to 6 months of tick-by-tick data, plus over 27 years of intraday (minute and above) data for both continuous and individual contracts for the real-time exchanges you have chosen to receive. Click here for a list of available intraday data. Additionally, you'll have up to 48 years of daily data.

 

Custom Continuous Futures Contracts

 

You can now customize the rules that determine when and how your continuous futures contracts roll from one month to the next. This includes the ability to choose the specific months to include in the continuous contract, the data series on which to base the continuous data, the rollover trigger event, and the back adjustment method. You also have the option to "use this symbol for trading", which means that you will be able to automate a strategy that is applied to a chart with this symbol.

 

To create a custom continuous contract from a Chart Analysis or RadarScreen window simply open the Symbol Lookup dialog and click on the "Custom Futures" tab and specify the settings that you would like to use. Once you click on the OK button the platform will automatically create a custom continuous contract symbol based on the settings that you specified.

 

Tick data is very intensive, so I'm not surprised they only offer 6 months of it. You do get a ton of minute data (27 years). Perhaps you could just drill it down to a 1 minute chart and get the same effect you are after. If nothing else, you could see the price action.

 

It's times like this when having TradeStation is very nice. I would not use them as my primary broker by any stretch, but you can get the platform for free if you do 10 round turns per month. Depending on how much trading you are doing, that could literally be knocked out in 1 day. There will be data fees though, regardless of how many trades are being done.

 

So if you used TS and traded 10 round turns, you'd be looking at just data fees. Which isn't terrible if this is what you need/want for your trading. And there's nothing saying you have to stay w/ them forever. You might stay a month or two, get what you need and then close the account.

[brownsfan019 17]

Along the same lines, you might want to check out OEC's eSignal plug-in. That will probably be more expensive than TS, but then you get to put it all in OEC trader which you are already familiar with. It's never fun having to learn a brand new platform.

[TrueBalance 10]

Another option is to collect the data yourself and/or use a downloader. Years ago I used QCollector for that purpose with mixed results. It did permit you to download a lot of tick data unattended. Beware though, the data from your charting platform provider is not always what you would expect.

 

I've heard good things about Tickdata. Never used them and I think it is on the expensive side (which is probably relative).

 

If you choose tickdata.com, they offer something (that used to be) called "pick 15" and "pick 30". You get to pick 15 or 30 years of tick data in any combination of symbols/years, which is a less expensive entry point. Still pricey though.

 

I have used them for several years and never had any problems. Service and support is ok. They used to deliver tick data once a month (if you have a subscription), now you get it every day after market close.

 

There are cheaper (and more expensive options) out there. It all comes down to how important quality is to you, and what timeframe you are testing on. As always you get what you pay for. And to have any use of high quality data you probably need a platform that can use it in an efficient manner.

 

Best wishes

TB

Edited December 17, 2009 by TrueBalance

[SangreDeToro 10]

I'm going to buy data from these guys:

 

FOREX Historical Data, Major Indices and Futures Historical Intraday and Daily Market Data - Through Download and on DVDs/CDs.

 

Will post what I think of it.

 

Would be good if anyone has any random months of tick data for quality comparison.

[TrueBalance 10]

I'm going to buy data from these guys:

 

FOREX Historical Data, Major Indices and Futures Historical Intraday and Daily Market Data - Through Download and on DVDs/CDs.

 

Will post what I think of it.

 

Would be good if anyone has any random months of tick data for quality comparison.

 

If you are looking for FX data you can find it for free at Oanda/FXTrade. The data is bid/ask, going back to Jan 1, 2004 and is what was actually used for trading. You need an FXTrade account with some minimum balance to be able to get it. They offer five pairs (and of course you can create synthetic pairs from the ones supplied).

 

To get some ideas on what constitutes quality data, check out High Frequency Data Filtering from tickdata.com. It is worth reading even though it was written by a vendor.

Edited December 17, 2009 by TrueBalance

[clmacdougall 11]

For the longest time I got all the info I wanted for free from finam. I downloaded years of back data in multiple time frames, right down to the minute time frame for data concerning all the emini futures products, no delay as well, real time. It was some Russian source I stumbled across, but I just today noticed they are asking for a subscription. I guess the free ride is over with them. If you're not too picky get the free stuff from CRB, it gives you about 6 months of free info which you can build upon. Pretty good source, as Tick Data provides for them Also give TimingCharts.com a try, not bad. Nowadays I just use Esignal at $25.00 per month plus an extra $10.00 for intraday back to Mar 2007. Anybody found any better?

[ektrader 10]

Hi dinotrader,

 

I'm curious to know what you are going to do with all that past data and how it's going to help you:roll eyes:. Oil is not something thats' heavily weighted like the S&P or NASDAQ. Maybe I'm wrong and I'll except the fact that it's possible. I just can't think of one reason why 27 years worth of news related data will help you trade todays markets. It's really not that serious. JMO

 

Happy Trading

 

Ektrader

[Dinerotrader 10]

Hi dinerotrader,

 

I'm curious to know what you are going to do with all that past data and how it's going to help you:roll eyes:. Oil is not something thats' heavily weighted like the S&P or NASDAQ. Maybe I'm wrong and I'll except the fact that it's possible. I just can't think of one reason why 27 years worth of news related data will help you trade todays markets. It's really not that serious. JMO

 

Happy Trading

 

Ektrader

 

I wouldn't be looking for 27 years. More like 5-10 years. I have a fundamental belief that markets do some similar things over time and that you can learn from the past in order to determine what future movements have higher probabilities. I don't have the ability to trade using history but I would like to test some ideas that I have. For example, I would like to look at how oil prices reacted to the weekly oil inventory report going back over the past couple years. To do this I would need tick data since the moves are so fast when inventory comes out.

[ektrader 10]

I wouldn't be looking for 27 years. More like 5-10 years. I have a fundamental belief that markets do some similar things over time and that you can learn from the past in order to determine what future movements have higher probabilities. I don't have the ability to trade using history but I would like to test some ideas that I have. For example, I would like to look at how oil prices reacted to the weekly oil inventory report going back over the past couple years. To do this I would need tick data since the moves are so fast when inventory comes out.

 

I understand your rationale. Thing is, the markets, the economy, and volume levels have changed. Whos' to say that the same speculators and operators from 10 years ago still have interest in oil. Could be a whole new set of players. Anyway, not to discourage you. If that's what you want to do. Good luck with your analysis and your trading.

 

Happy Trading

 

Ektrader

[DugDug 10]

If you buy data from anyone, ensure that they have gone through and made sure the data is clean. There is a lot of stuff that is full of repeated days, gaps, etc;

the good stuff is not cheap. Make sure the data you receive is able to be changed because I guarantee there will be modifications needed.

eg; when is the close. As the exchange says, or the last recorded trade.

eg; when to the futures contract roll over months, to which month.

There will be big gaps.

 

I would suggest that if you want to look at announcements etc then tick data is overkill, its just noise. a 5 min chart would be plenty....but thats up to you.

[Dinerotrader 10]

If you buy data from anyone, ensure that they have gone through and made sure the data is clean. There is a lot of stuff that is full of repeated days, gaps, etc;

the good stuff is not cheap. Make sure the data you receive is able to be changed because I guarantee there will be modifications needed.

eg; when is the close. As the exchange says, or the last recorded trade.

eg; when to the futures contract roll over months, to which month.

There will be big gaps.

 

I would suggest that if you want to look at announcements etc then tick data is overkill, its just noise. a 5 min chart would be plenty....but thats up to you.

 

Watch oil next time inventory comes out on Wednesday at 10:30 am ET and see how well a 5 min charts works for you.

[shrike 10]

Hi Dinerotrade

i've just bought 2 years of data(tick) from HISTORICAL TICK DATA ? Historical tick data for stocks, equities, futures, options, indexes and indicators!

ES and XX , i'm satisfied about the data quality , i've checked with another data and it's all ok .

i sent email, several time,to the support center and always they replied quickly.

 

Tick Data - Available Futures Data

[Dinerotrader 10]

Hi Dinerotrade

i've just bought 2 years of data(tick) from HISTORICAL TICK DATA ? Historical tick data for stocks, equities, futures, options, indexes and indicators!

ES and XX , i'm satisfied about the data quality , i've checked with another data and it's all ok .

i sent email, several time,to the support center and always they replied quickly.

 

Tick Data - Available Futures Data

 

Thanks for the post. Nice to here from a happy customer before I go spend some money.