Lets Roll Our Own Software!@

[AgeKay 10]

True, but isn't that the exact reason you would want your data to be indexed?

 

As BlowFish has correctly pointed out a few times, your file system would basically be your index. You just construct the file path from the market and date, which you use to directly access file you want to read.

[darthtrader3.0beta 10]

No to be I haven't for quite a while, but it really is pretty powerful. It supports a variety of languages and frameworks including c sharp and .net only problem is is its a steep learning curve.

 

Where is the library to write a trading system in Fortran though? My problem with neoticker is to me they have taken a stance that is just as absurd going forward as supporting Fortran. Supporting Fortran would not be a feature here, it may as well be a "bug" in forward thinking.

 

Did you ever get into the mind set of lisp or scheme during your software career grandpa?

 

Sometimes I contemplate that the recursive nature of those languages probly "talk" in the markets language much more fluent than some bizarre hack of OOP as a scripting language. I got obsessed with "Structure and Interpretation of Computer Programs" to the detriment of a woman who should be now my first divorce a decade ago. I'm really contemplating retracing that path to see what I left behind there...knowledge wise, not ***** wise...

[darthtrader3.0beta 10]

As BlowFish has correctly pointed out a few times, your file system would basically be your index. You just construct the file path from the market and date, which you use to directly access file you want to read.

 

Meh, like I said...I think if we take this aruement to the level of Big O notation..your absurdly wrong.

[AgeKay 10]

Sometimes I contemplate that the recursive nature of those languages probly "talk" in the markets language much more fluent than some bizarre hack of OOP as a scripting language. I got obsessed with "Structure and Interpretation of Computer Programs" to the detriment of a woman who should be now my first divorce a decade ago. I'm really contemplating retracing that path to see what I left behind there...knowledge wise, not ***** wise...

 

That's really funny because I just listened to a .NET Rocks interview today where the interviewee talked about that book and said it was wonderful. I also think that functional languages are the future since individual chips are not getting any faster but going multicore. Parallelism is a no brainer with functional languages.

 

Meh, like I said...I think if we take this aruement to the level of Big O notation..your absurdly wrong.

 

I read my data at O(1) [constant time for those who do know the Big O notation] no matter how big the file. You can't say that about databases and most other data storage solutions.

[darthtrader3.0beta 10]

I read my data at O(1) [constant time for those who do know the Big O notation] no matter how big the file. You can't say that about databases and most other data storage solutions.

 

Well like i said I'm talking out my ass here..

why does anyone use a database though? What you say conforms exactly to all my research that flat files kill a db...it just strikes me though there is an obvious missing variable in the analysis that we are all missing. Humans in general are dumb but not this dumb to create the entire concept of a DB for no reason.

[BlowFish 10]

Where is the library to write a trading system in Fortran though? My problem with neoticker is to me they have taken a stance that is just as absurd going forward as supporting Fortran. Supporting Fortran would not be a feature here, it may as well be a "bug" in forward thinking.

 

 

So you would rather a platform that 'locks you in' to .net & c sharp? You clearly don't understand click quest's stance if you think it is absurd. It's an open architecture that is capable of supporting pretty much anything you can imagine. Actually its a shame as it is probably the quickest route to where you want to get to by a metric crap tonne. You do yourself a mis service by mis understanding and latching on to some old rhetoric about fortran. There is an expression in the UK "cutting off your nose to spite your face".

[taotree 10]

I have been writing specs for developing some new charting software and just ran into this thread.

 

To some points on this thread:

 

Not Matlab: It's expensive. "If you have to ask, you cannot afford it". I don't see a price on their site, therefore it's too expensive Unless you're talking about the free clones of it?

 

Time Series Database: interesting, but we don't need to manipulate the data--only read/write it. So probably overkill.

 

Relational db: yeah, don't even think about it.

 

Files in folders in filesystem: qmail does this to handle millions of emails. It does put you somewhat at the mercy of the filesystem's limitations. However, indexing by folder by symbol and one file per day, no folder will have more than 31 entries. Path would be short enough: db/symbol/2009/12/31.dat

 

Custom indexed file: it would be easy to run into file system limit on file size with this.

 

Multiple big files with separate index: this might be best bet. Because data is never modified, you don't have to worry about fragmentation--you can just keep adding onto the end of the file. You would have some sort of indexing scheme in a separate file that could be kept in memory so you can skip to the beginning of a certain day or something like that. The nice thing about that is once you have found the right location to start in the data file, you can just move forward straight from there without ever having to jump somewhere else, with the sole exception if you run to the end of that data file and have to go to the next one (which is trivial to deal with). The index file could be a custom job, or use a simple db for it. This might even open up possibility of keeping index completely in memory and memmap the data file for quick access wherever... I might have to consider that some more.

 

Custom bar types: My concept at the moment is that bars are just an indicator--and so are handled just like an indicator. Someone could create their own bar type, or chart type, and have it generated any way they want from the ticks and have it paint any way they want. Strategies would be able to use their custom bar types just like any other.

[BlowFish 10]

 

db/symbol/2009/12/31.dat

 

 

if you go this route I would break the symbol directory down further so maybe have a symbol/a/ symbol/b/ etc. directory. Other wise (under NTFS) you might easily have enough symbols to become an issue. Another consideration is that you sometimes get the same symbols on different exchanges so you might want to consider db/exchange/symbol.

[darthtrader3.1beta 10]

So you would rather a platform that 'locks you in' to .net & c sharp? You clearly don't understand click quest's stance if you think it is absurd. It's an open architecture that is capable of supporting pretty much anything you can imagine. Actually its a shame as it is probably the quickest route to where you want to get to by a metric crap tonne. You do yourself a mis service by mis understanding and latching on to some old rhetoric about fortran. There is an expression in the UK "cutting off your nose to spite your face".

 

No you missed my point completely...Fortran was a metaphor for the fact that tickquest still supports delphi...I would speculate that the number of speculators trading systems in delphi is 5:1 with number of speculators trading in Fortran, and basically no one trades in Fortran. Its a waste of resources..bad management..exactly the way I start with valuation as far as any company goes. I don't invest in incompetant management ever.

Like you said, the biggest problem is getting the architecture right from the start. That is the make or break decision..

There is no reason to not go with matlab. You can't economicaly make trading softare to that degree, its impossible economics wise because trading software will never have the market that matlab does.. Its simply a datafeed problem if viewed as straight up trading software. I took this entire week off from the markets to wrestle with the macro economics and if I can accept 25ms snapshot data from IB...there is no choice.

[darthtrader3.1beta 10]

I have been writing specs for developing some new charting software and just ran into this thread.

 

To some points on this thread:

 

Not Matlab: It's expensive. "If you have to ask, you cannot afford it". I don't see a price on their site, therefore it's too expensive

 

Custom bar types: My concept at the moment is that bars are just an indicator--and so are handled just like an indicator. Someone could create their own bar type, or chart type, and have it generated any way they want from the ticks and have it paint any way they want. Strategies would be able to use their custom bar types just like any other.

 

Matlab with the the toolboxes you would want runs about $2k...

There is no other choice. You can easily spend $1k on trading software..

What you get for that extra $1k is decades of software engineering into making things simple for the highest common denominator, instead of a few years of software engineering aimed at the lowest common denominator. Thats exactly why you can post matlab code form market profile in this thread while posting ninja code for the same thing would not fit...

The entire idea of making "advanced" retail software is flawed...

There is no market. The only way to have that is for some software to be designed for much more complex analysis and visualisation of data, then piggy back that engineering down to retail trading..

its already been done..

[OAC 10]

I think we need to put the breaks on "secondary agendas" here. It clearly drove away that poker guy.

y.

The poker guy left because he can't make a dime here.

[darthtrader3.1beta 10]

The poker guy left because he can't make a dime here.

 

The "poker guy" was Pat Dittmar..

http://www.amazon.com/Practical-Poker-Math-Probabilities-HoldEm/dp/1550228331/ref=pd_bbs_sr_1?ie=UTF8&s=books&qid=1235784284&sr=8-1

 

Who left because your an insecure dipshit who single handedly ruined some great exachanges of ideas that will now never happen. If I was James I would ban you.

 

Or maybe I'm wrong and you are right..I mean who wants to talk to a published poker author on here who runs his own private trading firm and employes a C++ engineer to write financial software...(fish his linkd profile)

 

good job "moderator"!!!!

Edited February 28, 2009 by darthtrader3.1beta

[BlowFish 10]

No you missed my point completely...Fortran was a metaphor for the fact that tickquest still supports delphi...

 

No I did not, but you have missed mine. Re read carefully. It is an open platform you can use anything within reason to program it. You have the blinders on again actually its more a blindfold. You aren't going to get anywhere until you actually start trying to understand rather than assuming mate. Its really hard to have any sort of meaningful discussion with someone who is so woefully ill informed on the topic in hand. Sorry it has to be said.

 

BTW Delphi is an excellent language. (though the whole Delphi thing is kind of irrelevant) C# has taken much more from it than it has from C and C++. It is actually superior in many ways, inferior in others. Have you ever used it? Nope didn't thing so. Have you even examined it in any sort of depth? Nope didn't thing so.

[OAC 10]

The "poker guy" was Pat Dittmar..

http://www.amazon.com/Practical-Poker-Math-Probabilities-HoldEm/dp/1550228331/ref=pd_bbs_sr_1?ie=UTF8&s=books&qid=1235784284&sr=8-1

 

Who left because your an insecure dipshit who single handedly ruined some great exachanges of ideas that will now never happen. If I was James I would ban you.

 

Or maybe I'm wrong and you are right..I mean who wants to talk to a published poker author on here who runs his own private trading firm and employes a C++ engineer to write financial software...(fish his linkd profile)

 

good job "moderator"!!!!

 

All I did was asking him what data feed he uses.

You are mad because you are a sucker, they are borned every minute.

[darthtrader3.1beta 10]

All I did was asking him what data feed he uses.

You are mad because you are a sucker, they are borned every minute.

 

lol..no im mad because there is alot of interesting discussion that you killed off for no valid reason other than your own insecurity that there actually may be things that are interesting to talk about that you have not a clue about.

Its pathetic you can't admit you were wrong. Its not everyday that you get a published author on a board like this. Maybe he eventually would try to sell something? Who cares..Just his posting on high frequency data spurred some of the most interesting discussion for awhile on here..And you couldn't take it so you killed him off. I've lost all respect for you simply because you can't admit you were wrong. In the long run, everyone makes mistakes, who cares. Mods who can't admit they are wrong certainly have no business moderating something as uncertain as a trading forum.

[OAC 10]

Buddy, Dream On........

[darthtrader3.1beta 10]

You have the blinders on again actually its more a blindfold. You aren't going to get anywhere until you actually start trying to understand rather than assuming mate. Its really hard to have any sort of meaningful discussion with someone who is so woefully ill informed on the topic in hand. Sorry it has to be said.

 

BTW Delphi is an excellent language. (though the whole Delphi thing is kind of irrelevant) C# has taken much more from it than it has from C and C++. It is actually superior in many ways, inferior in others. Have you ever used it? Nope didn't thing so. Have you even examined it in any sort of depth? Nope didn't thing so.

 

Well of couse I'm going to "assume" things in this area. I am not a software engineer, I am a speculator interested in software engineering. Of course I'm going to "speculate"...I've spent 5 years now training my brain to speculate against uncertain information, its not possible for me to no do so.. Maybe its you who mistakes something you can't see for a blindfold. Just because you can't see something doesn't mean your blind..The eyes/brain take in a massive amount of data in continuous time, the fact I don't remember 99% of the data that has come across my brain via my eyes today certainly does not mean I'm blind..

Basically, I don't get your point..

The ironic thing to me is that there is a title to this thread and I've asked

for you geeks to shootdown matlab, yet no one even wants to touch the subect...why? Because you know you cant Its EXACTLY designed for the purpose we are talking about...Is that an "informed opinon"? Of course not, its a speculation..."informed opinion" is vastly over priced though...its exactly why the financial system sits at the cliff right now. I don't "buy" "informed opinions". I'm just a speculator.

[darthtrader3.1beta 10]

Buddy, Dream On........

 

Thats it? I would really like to hear you explain yourself as far as why you banned a published poker author who was clearly not selling anything.

You can't and you know you can't. Its so pathetic you can't admit you were wrong. I've always thought you were a pretty cool guy until this but you have to check power once its given out. You clearly don't know how to manage power.

To me I view your actitions valuation wise that you just cost James a good deal of cash flow from the poker guys Pat would have brought to this board. That is exactly why I said If I was James I would ban you.

[OAC 10]

Thats it? I would really like to hear you explain yourself as far as why you banned a published poker author who was clearly not selling anything

 

Darth , come back to earth. He is NOT banned. I still see him snooping around here reading various post from time to time.

I challenged him to make a fool out of me by proving me wrong. He could'nt because probably because it won't make him a dime.

But he sure abandoned you.

[darthtrader3.1beta 10]

Darth , come back to earth. He is NOT banned. I still see him snooping around here reading various post from time to time.

 

I fail to see the difference between banned or "annoyed" enough to stop posting...although I called him out at first, I certainly did not contribute to him not posting here...

whatever though, I'm out...next post is as darthtrader4.0

[OAC 10]

whatever though, I'm out...next post is as darthtrader4.0

 

If James tells me this site is not for serious traders but for science fiction buffs. Then I am out too and I WON'T be back.

[diablo272 10]

Traders don't like drama. If you can control your emotions in the market, you can control your emotions on a message board.

Edited February 28, 2009 by diablo272

[zdo 20]

Did anyone 'trial' TickZoom and if so any feedback? Many thanks. zdo

[BlowFish 10]

All I did was asking him what data feed he uses.

You are mad because you are a sucker, they are borned every minute.

 

No that is a flat out lie. It was not all you did. You alleged he was a vendor and hounded him to some how prove he was not. That he has done (in spades) by never trying to sell anytrhing, and by simply withdrawing. Now you try to cover your tracks by saying 'all I ever wanted to know was what data feed he used'.

 

To be fair it the hounding was not as bad as ET but you expect a moderator to be dousing the flames not pouring on gasoline. The appropriate course of action would be to a) warn him by PM of any infraction of forum rules he had made. b) Ban him.

 

Instead you chose the ET method of attack the poster. I notice you have done the same right above, rather than deal with the point Darth raises you resort to the ad hominem school of argument that is so prevalent at ET.

 

So OAC ...what was he selling again ? How is darth a sucker? Just for listening what the guy had to say?

Edited March 5, 2009 by BlowFish

[Hlm 10]

All I did was asking him what data feed he uses.

Curious though, was this question ever answered? If no, why not?

 

------------------------

 

As for this thread, I would be extremely cautious at rolling your own software without first being a successful trader. In most cases (imo) programming to become successful is putting a dozen carts in front of the horse . Designing and programming a platform completely from scratch can be a tedious and painful process. This is why many people who want a solid base to start out with but still want absolute control end up using Matlab or similar matrix manipulation software. Yes, it's expensive, but you get what you pay for. Though most student versions with these types of programs are much less.

 

As for the original posters initial comment...

"The popularity of this board though shows there is an untapped market for software that has no concern for that kind of uselessness."

 

...what type of tools/analysis does this board specifically have a concern for?