darthtrader3.0beta

Well like i said I'm talking out my ass here.. why does anyone use a database though? What you say conforms exactly to all my research that flat files kill a db...it just strikes me though there is an obvious missing variable in the analysis that we are all missing. Humans in general are dumb but not this dumb to create the entire concept of a DB for no reason.

Meh, like I said...I think if we take this aruement to the level of Big O notation..your absurdly wrong.

Where is the library to write a trading system in Fortran though? My problem with neoticker is to me they have taken a stance that is just as absurd going forward as supporting Fortran. Supporting Fortran would not be a feature here, it may as well be a "bug" in forward thinking. Did you ever get into the mind set of lisp or scheme during your software career grandpa? Sometimes I contemplate that the recursive nature of those languages probly "talk" in the markets language much more fluent than some bizarre hack of OOP as a scripting language. I got obsessed with "Structure and Interpretation of Computer Programs" to the detriment of a woman who should be now my first divorce a decade ago. I'm really contemplating retracing that path to see what I left behind there...knowledge wise, not ***** wise...

True, but isn't that the exact reason you would want your data to be indexed?

comeon Grandpa...do you own neoticker? Of course not...its absurd they are still supporting delphi. Maybe if they started over and converted everyting to C# it would be interesting but in its current incarnation neoticker is a scatterbrained joke.

To your first point, without question....Thats why I'm waiting for you geeks to knock down the idea I have that there is no reason not to use matlab and HDF5 for this..Whatever can be done on a retail windows machine with that setup can transparently move to a massively parralell unix environment 20 years from.. To your second point, if you have alot of tick data building bars in matlab goes a lil something like: % numberOfBarsPerDay: 24 = 1 hour bars (60 minutes) % 48 = 30 minute bars (24*60)/30 % 72 = 20 minute bars (24*60)/20 % 96 = 15 minute bars (24*60)/15 % 144 = 10 minute bars (24*60)/10 % 288 = 5 minute bars (24*60)/5 % 1440 = 1 minute bars (24*60)/1 %%-------------------------------------------------------------------------- numberOfBarsPerDay=24; x=Bid; datetimeGrid=(floor(datetime.*numberOfBarsPerDay))./numberOfBarsPerDay; timeChgPointIndex=find(diff(datetimeGrid)~=0); intervalDatetimeStart=datetimeGrid(timeChgPointIndex); intervalDatetimeEnd=datetimeGrid(timeChgPointIndex+1); intervalDatetimeActual=datetime(timeChgPointIndex); intervalData=x(timeChgPointIndex); I don't currently understand that code but its so simple, elegant and deadly....

Or the issues with NTFS can be alleviated by moving the kitchen sink to a Unix environment...

Well first though your thinking of a database as a relational database..Its obvious if you investigate this area that a relational database is just a bad idea when it comes to market data exactly because the tables become absurdly large. I'm a dotcom CS casualty so I'm talking out my ass here but I would think the reason to use a database instead of flat files is a simple arguement if you get into algorithm analysis as far as sorting and searching with an index vs not having an index. From what I understand HDF5 is basically an indexed version of having flat binary files in a hierarchical tree. "H"ierarchical "D"ata "F"ormat.. Also from what I understand to say HDF5 won't be around in 10 years doesn't make any sense. At this point it seems to be a defacto standard in the scientific computing of multiple terrabyte data. Even if a technology breakthrough occurs in the next 10 years it will surely support an easy move from HDF5 because thats what the guys at that level use. Thats also why I like matlab..if someone comes up with the ultimate AI/machine learning algorithm in the next 10 years, its an easy bet that matlab will support it.

haha, comeon old man...you need one last interesting project to work on before the grave.. As far as data storage its not that hard to figure out that a relational database is a joke when what we are really talking about is financial TIME SERIES..why TIME SERIES DBs are not a big topic on message boards is why i made this thread to start with, it just doesn't make sense.. http://en.wikipedia.org/wiki/Time_series_database The king there bang for the buck is berkley DB.. http://en.wikipedia.org/wiki/Berkeley_DB If your talking massive data storage with a time series component though your talking HDF5... http://en.wikipedia.org/wiki/Hierarchical_Data_Format Obviously your not going to beat the performance of accessing flat binary files of tick data directly with an engine for backtesting..My problem with that though is I'm looking out 10 years from now. 10 years of tick data as flat binary files is just not logical..There is a reason to have a database... Matlab is already integrated with HDF5 directly.. http://www.mathworks.com/access/helpdesk/help/techdoc/ref/hdf5.html Like I said, there is almost nothing you can think of that has not already been thrown at matlab...BESIDES retail market datafeeds..which the only current real world solution seems to be IB data...my only hang up there is if IB data is "good" enough because i would be storing timestamps with "only" 20ms of granularity. Its absurd...I'm not looking to do high freq algo trading...Its just the stupid bias that I'm missing out on something, even though ninja was "filtering" the tape because I didn't understand how it handled my "true tick datafeed" for basically the past year. .

Well first I respect that Wayne is done with this thread. I just disagree that its a big deal that they have pricing for alpha stage software on the site. I mean while I may very well end up getting a license, I'm certainly not going to pay for alpha stage softare. Just like I wouldn't pay up as a sponsor when I'm only offering alpha stage software but I do see how that is a bit unfair here to other sponsors. To me the real problem with a great open source C# trading platform is all you will really end up doing is alot of grunt work for anyone that has programmers on staff. Its like how I've read David Shaw's last published paper was on how to connect Unix machines to market data...Then he realized how stupid it was to just be doing alot of work for free for other people and went into business with his ideas. I would actually like to discuss with you real programmers, why not just use matlab? It seems like the only stumbling block is getting data into matlab...Once you have that then most the other grunt work is done for you.. The grunt work for almost any database you would want to use is done, plotting/visualation wise I don't see how anyone can compete with the work that has been put into matlab, you can probly find almost any algorithm already done, all wrapped in a deadly programming language. Like I said in that post earlier, being able to build a market profile from tick data with this few lines of code is just off the map mind blowing considering what else it can do. MATLAB CODE: % clear all % clc; % %assuming tick by tick data % data = load('may01d.mat'); % ESdata = data.ESlast; ESdata=close; ESprices=unique(ESdata); bars=20; timertick=1; periods=floor(size(ESdata,1)/(timertick*bars)); MP=zeros(size(ESprices,1),periods); for i=1:periods g=ESdata(timertick*bars*(i-1)+1:timertick*bars*i,1); for m=1:size(g,1) s=g(m); s1=find(ESprices==s); MP(s1,i)=i; end end B = MP;%A(:,2:end) ; B(B==0) = inf ; B = sort(B,2) ; % sort each row B(isinf(B)) = 0 ; TPO=zeros(size(B)); for l=1:numel(B) if B(l)~=0 TPO(l)=B(l)+64; end end Also matlab has an engine now that supports Nvidia's CUDA directly for doing computation on the GPU.. http://www.accelereyes.com/

I think we need to put the breaks on "secondary agendas" here. It clearly drove away that poker guy. I'm very glad to see tickzoom here...Keep in mind I didn't invite him here like I posted earlier in this thread, I guess someone else pointed him here. If you followed the elitetrader threads at all what you quickly see is he just really likes to talk If you didn't follow those threads it will look like aggressive marketting. I would like to ask James to consider that this is early alpha stage software that is clearly trying to do something different. I can't see how you can really be a sponsor until you have a first release...In the mean time he seems extremely open to ideas, so bomb away.

I just don't buy that our representatives really care what the individual person has to think..Money talks, so if your not giving these guys campaign money you have no voice... The bright side of that though is we are all against this just as much as the heaviest hitters money wise in the world. Really though, this has to just be political posturing...I'm sure this guy knows this bill doesn't have a chance in hell of passing. However it makes him look good to the average person next election.

Just wanted to add I found this amazing blog from the guy who made MATLAB2IB. http://tradingwithmatlab.blogspot.com/ If you check out the code for creating market profile from tick data in matlab its pretty stunning how simple it is. He also has code for replicating the VIX...considering matlab can do that, it would imagine it blows away any retail software as far as breadth goes. Seems like if you really want to roll your own software its pretty sensless to not go with matlab/IB..the real grunt work is already done. $300 bucks a year.. They also have software for connecting matlab to TT and are working on a generic hookup to FIX.

Fish oil is just something everyone should probly take in general. It just doesn't seem realistic to get enough omega 3 from diet alone. Choline and stuff like that I imagine only works if your deficient. What your probly looking for is straight up nootropics.. I take piracetam daily and like what it does. http://www.bulknutrition.com/?products_id=912 For trading though its hard to beat coffee and cigerettes

I thought you died forever!!!! **** what anyone says here...a candle pattern is a simplistic 4 point representation of tick data...if your not front running the otherside of the breakout of the triangle then you have already given up optimal trade position...who cares after that.

That does look pretty sick. This kind of thing does seem to be the future. I seen some videos of something like this for matlab in 3d doing stuff gpu vs cpu and it was off the map. I guess it makes sense when you consider the complexity of what is going on in a modern computer game vs how simple what we are trying to do.

Very true...I guess the more and more I've thought about this, the biggest reason this is impossible is the same reason retail software will always fall short for yourself. Since everyone does something different trading wise, most the resources are going to be wasted on things that have nothing to do with how YOU trade. Even if you go a non commercial route where you could strip away having to cater to the lowest common denominator, there would still be massive wastes from every person's individual perspective and no one would be happy.

Meh, its not like this area of AI is not well studied. It just is an area that does not lend itself well to commercial sofware. I would never consider a software that pretended to intigrate trading and AI... Just look into SVM http://en.wikipedia.org/wiki/Support_vector_machine Why is there no commercial software that "yells" about SVM? Obviously, because its much easier to go yell about things you dont understand, like neural networks, whatever.." while you will keep the real "AI" to yourself., most importantly "if you have it all"?" then market it as something advanced. That is the trading software "market" to me in a nutshell....

I have no idea. It seems to me like how Jim Cramer can give an hour of investment advice, then run the blurb...You don't see every trading board though append the blurb to each post though. Obviously that kind of thing as far as software glitches is covered in the license agreement..it doesn't make much sense to be worried about that without appending "this does not constitute investment advice" to every post on this board.

Dcraig on elite...If you search for him and delta stuff you should find his charts. They are pretty badass. As far as .net charting, you ever check out the commercial chart packages? Seems like that would be the path to take, I'm sure this is where the rubber meets the road for guys like ninja as far as headaches. An interesting excersise here would be descibe your ultimate trading software, forgetting about any constraints... mine is: datafeed -> into an HDF5 database, capturing everything i'm paying for as far as data goes -> a flexible charting package at the other end a backtester that can access the database Somewhere in there is execution but that has to be after the fact. The obvious issue in that equation for something like this getting off the ground is the datafeed. That datafeed is what puts us on different pages. For that to me you have to go with DTN starting out capturing SPY tick data...only real dirt cheap non shuttled data that is feasible for the price as far as adding to minimal overhead that would not mess with anyones trading if this project takes 5 years.

Well I don't disagree with you at all. From thinking about it more, its better to forget the TickZoom guy. From what I've read he seems like a great software engineer, his charting stuff is a secondary to producing a monsterly fast tick backtesting engine. My open source suggestion was just because thats what tickzoom aims to be, but my experience is open source is really only good at mostly producing half finished software and sometimes you get an outlier. Forgetting opensource, I think a more interesting idea is how feasible would it be to make a traderslab "in house" software? That way there is no delusions of marketting to a losest common demoninator like commercial software but it still has the "feel" of an open source project as far as the "swarm intelligence" here. The basic guts of that software I would view as you have a datefeed that datefeed writes to a database then you pull a chart from that database My biggest problem with current software is it seems to be datafeed -> chart -> then maybe think about storing data because most people don't care about storing data. Even though its obviously much faster to pull from a database than to go out across the internet to retrive data.

No, No, that is missing the "context" of what I am proposing...The "context" of what I'm going for has to do with that I'm completely obsessed with ants. No metaphor, the little black creatures. Everynight I read some of Holldobler and Wilson masterpiece on ants..What you quickly get from this is a feel for how much better "swarm intelligence" is than a single human brain. http://en.wikipedia.org/wiki/Swarm_intelligence If you realize it or not, you have built a swarm intelligence trading network here that crushes anything else done previously. In that context to say you can't support such software is obviously wrong. Its like saying that this place would be better if you could reply to every thread, but that is obviously false..the "swarm" here knows far far more than both you or I taken singlely.. To me that right there is the flaw/opportunity with trading software. Me "pulling this off" in that context makes no sense...you would obviously need to be the Queen Bee though. I'm only interested in being a worker bee. The Queen bee though doesn't plan anything like a human project manager..it just gets laid all day.. I just think that a decentralized, open source trading software can not be beat because of the inputs, especially if the inputs come from this board.

There are obviously flaws in the business models of comercial trading software. Namely for the people who post on this site its that you can't stay in business without catering to the newbs who it would be alien to not have MACD or RSI as a base...The popularity of this board though shows there is an untapped market for software that has no concern for that kind of uselessness. There are alot of smart people here who I assume probly know other smart people who don't post here. I can think of 3 mad "scientists" I'm friends with who don't post here but a software project would get them to. James has already locked in probly the best marketting name for such software that at least I can imagine..."Traders Laboratory". I assume some other people here have seen the TickZoom guys open source C# software ideas on elite...the code has been released but seems to have died on the vine. He should have come here instead of elite It would probly be quite an interesting "experiment" if we invite the TickZoom guy to here to work on "Traders Laboratory"...he is clearly a monster software engineer but doesn't appear to be a monster as far as a project lead. The project lead should be our collective "swarm intelligence" here.

Well I do want to apologize for what I wrote yesterday. I was going to post some functions I had wrote for chartcontrol yesterday then the ninja guys post pissed me off to no end. I do understand the business model constraints with trading software... Once you get past the mean, your customer base is just looking to proxy "rolling their own" through your software. Thats always going to lead to conflict. Beggars and can't be choosers when it comes to this area.

Hmm...are you sure that limit orders don't appear on the tape? That doesn't really make sense from what I understand unless you ment to say they don't apprear on the book. I think this thread is getting quite off track. To me the two most important things here is first Urma mentioned that to trade this you need to have auto execution. I'm just really not sure what good these indicators are that have been posted unless you are looking to do algo high frequency stuff. By the time you move you mouse this stuff is already done. Secondly, I think what we should be talking about is what level of premium triggers this. That kind of thing you might be able to gain some information on with a mouse.