Check Missing Data from Strategy

[skan 10]

Hello

 

How can I check from a easylanguage strategy or indicataor if some data is missing?

 

I'm trying to code a pair trading strategy. Sometimes the data from one of the instruments has gaps but the other one is still continuous.

Some of these gaps are right because of a weekend or holiday but sometimes is because of data feed problems.

Either way, I need to detect it from the code.

 

If my strategy is running on the the "continuous" instrument and waiting for data from the one with gaps in order to exit I could have a serious problem.

 

What easylanguage instruction can check if data is missing?

 

thanks

[Tams 20]

  skan said:

Hello

 

How can I check from a easylanguage strategy or indicataor if some data is missing?

 

I'm trying to code a pair trading strategy. Sometimes the data from one of the instruments has gaps but the other one is still continuous.

Some of these gaps are right because of a weekend or holiday but sometimes is because of data feed problems.

Either way, I need to detect it from the code.

 

If my strategy is running on the the "continuous" instrument and waiting for data from the one with gaps in order to exit I could have a serious problem.

 

What easylanguage instruction can check if data is missing?

 

thanks

 

instead of asking "What easylanguage instruction can check if data is missing?"

try to think, what would you be looking for if you were looking at the data manually?

what would constitute a "bad" data?

what would raise a red flag in your head?

what would sound an alarm in your ears?

[skan 10]

Hi

 

Maybe something as simple as

If Close > 0 then ....

 

But Maybe there is something more correct.

Some other languages has some instruction to detect nonnumeric numbers or missing data.

[Tams 20]

  skan said:

Hi

 

Maybe something as simple as

If Close > 0 then ....

 

But Maybe there is something more correct.

Some other languages has some instruction to detect nonnumeric numbers or missing data.

 

do you need something more "sophisticated" than what you have written above?

 

note: I am not saying you do, or you don't. This is just the continuation of the thinking process in programming. Remember, you are teaching the computer to think like you.

Edited December 2, 2010 by Tams

[skan 10]

If the chart were supposed to be continuous I could check that the current bar's datetime is exactly equal to the previous one plus the bar interval. But this is not as simple because there are weekends, holidays, trading hours,...

[Tams 20]

  skan said:

If the chart were supposed to be continuous I could check that the current bar's datetime is exactly equal to the previous one plus the bar interval. But this is not as simple because there are weekends, holidays, trading hours,...

 

ok, you are bringing more clarity to the table.

what now do you think you need to do? or what EasyLanguage "should" do?

can you articulate it in a step by step process?

[skan 10]

Hi

I don't know it, that's why I'm asking.

I need to know what to write within an IF expression in order to check if a price doesn't exist on the current bar.

Something like is.na() on R.

Edited December 2, 2010 by skan

[suckassjuice 10]

Ah, data verification, the bane of any attempt to back test. Not only are you probably missing data, how exactly do you know that the data you do have is correct. You don't. So, now your faced with the nasty task of cross-referencing your data against a different data source. And then that leads to having to further cross-reference to yet another source since you'll inevitably find differences. And then you realize, there's enough differences in all the data such that it's going to materially impact the results of any back test. Now whatcha gonna do? Probably what most do, ignore it...pretend it doesn't matter or that your data is the "real" data.

[skan 10]

but that's another history, now I just want to solve one problem, another day I'll solve a different one.

[suckassjuice 10]

Yes, I quickly moved on from your initial concern, but the answer was implied. It sounds like your relying on one data source. You need another source (i.e., you need at a minimum 2 sources from different providers and preferably 3-4 total different sources if you're serious about the ultimate results) so as to do the comparison and flag the differences. Missing data is easy to isolate. Then just cut and paste until you build your own "reliable" data set. But it won't be quite that simple. You'll likely see many other differences when you start your comparisons (and it's then that you'll realize you need more than 2 sources). From there you'll begin to better understand my initial post.

[skan 10]

Hello.

I think we are not speaking about the same thing.

 

Checking data quality is very important but is not what I need to do now.

I just need to check if a bar has a price associated or not.

 

Imagine I have a chart with data1= 15min EURUSD forex

and data2=15min TF.

Even If all my data were correct I'll have many bars were data1 exist and data2 doesn't, because data2 has shorter opening sessions and TS or MC fills data2 chart with gaps.

 

If a try to calculate something, such as exit stop, from data2 there will be moments in which my system won't be able to do anything.

I need to detect this gaps on data2 in order to exit from data1, just in case. Or in order to make different calculations.

[yambo188 10]

I will look into the on-line broker http://www.thinkorswim.com that has a strategy builder called prodigio. There a cci cross over strategy setup where you can modify to suite your needs, No programming language is required. All done through drag-and-drop system in prodigio.

 

if you open a funded account with thinkorswim, you can use the prodigio strategy builder for free. it can autotrade foe you.

[Iris 10]

back checking in TS forums - there were wireless connectivity issues especially with EUREX at times (i.e. it is on your side not the transmission side), and also Norton Antivirus causes some issues.

[jdevron 10]

No algorythm can be programmed to fix bad data which wont introduce a new adverse side effect somewhere. There is no substitute for good information. Redundant data sources are essential to correct bad data. You need to merge them by taking the best inside quotes that don't cross each other.

Last trade info can be filtered by comparing it to the best inside quotes. This will clean up spike and gaps in poor quality data.