darthtrader3.0beta

Well like i said I'm talking out my ass here.. why does anyone use a database though? What you say conforms exactly to all my research that flat files kill a db...it just strikes me though there is an obvious missing variable in the analysis that we are all missing. Humans in general are dumb but not this dumb to create the entire concept of a DB for no reason.

Meh, like I said...I think if we take this aruement to the level of Big O notation..your absurdly wrong.

Where is the library to write a trading system in Fortran though? My problem with neoticker is to me they have taken a stance that is just as absurd going forward as supporting Fortran. Supporting Fortran would not be a feature here, it may as well be a "bug" in forward thinking. Did you ever get into the mind set of lisp or scheme during your software career grandpa? Sometimes I contemplate that the recursive nature of those languages probly "talk" in the markets language much more fluent than some bizarre hack of OOP as a scripting language. I got obsessed with "Structure and Interpretation of Computer Programs" to the detriment of a woman who should be now my first divorce a decade ago. I'm really contemplating retracing that path to see what I left behind there...knowledge wise, not ***** wise...

True, but isn't that the exact reason you would want your data to be indexed?

comeon Grandpa...do you own neoticker? Of course not...its absurd they are still supporting delphi. Maybe if they started over and converted everyting to C# it would be interesting but in its current incarnation neoticker is a scatterbrained joke.

To your first point, without question....Thats why I'm waiting for you geeks to knock down the idea I have that there is no reason not to use matlab and HDF5 for this..Whatever can be done on a retail windows machine with that setup can transparently move to a massively parralell unix environment 20 years from.. To your second point, if you have alot of tick data building bars in matlab goes a lil something like: % numberOfBarsPerDay: 24 = 1 hour bars (60 minutes) % 48 = 30 minute bars (24*60)/30 % 72 = 20 minute bars (24*60)/20 % 96 = 15 minute bars (24*60)/15 % 144 = 10 minute bars (24*60)/10 % 288 = 5 minute bars (24*60)/5 % 1440 = 1 minute bars (24*60)/1 %%-------------------------------------------------------------------------- numberOfBarsPerDay=24; x=Bid; datetimeGrid=(floor(datetime.*numberOfBarsPerDay))./numberOfBarsPerDay; timeChgPointIndex=find(diff(datetimeGrid)~=0); intervalDatetimeStart=datetimeGrid(timeChgPointIndex); intervalDatetimeEnd=datetimeGrid(timeChgPointIndex+1); intervalDatetimeActual=datetime(timeChgPointIndex); intervalData=x(timeChgPointIndex); I don't currently understand that code but its so simple, elegant and deadly....

Or the issues with NTFS can be alleviated by moving the kitchen sink to a Unix environment...

Well first though your thinking of a database as a relational database..Its obvious if you investigate this area that a relational database is just a bad idea when it comes to market data exactly because the tables become absurdly large. I'm a dotcom CS casualty so I'm talking out my ass here but I would think the reason to use a database instead of flat files is a simple arguement if you get into algorithm analysis as far as sorting and searching with an index vs not having an index. From what I understand HDF5 is basically an indexed version of having flat binary files in a hierarchical tree. "H"ierarchical "D"ata "F"ormat.. Also from what I understand to say HDF5 won't be around in 10 years doesn't make any sense. At this point it seems to be a defacto standard in the scientific computing of multiple terrabyte data. Even if a technology breakthrough occurs in the next 10 years it will surely support an easy move from HDF5 because thats what the guys at that level use. Thats also why I like matlab..if someone comes up with the ultimate AI/machine learning algorithm in the next 10 years, its an easy bet that matlab will support it.

haha, comeon old man...you need one last interesting project to work on before the grave.. As far as data storage its not that hard to figure out that a relational database is a joke when what we are really talking about is financial TIME SERIES..why TIME SERIES DBs are not a big topic on message boards is why i made this thread to start with, it just doesn't make sense.. http://en.wikipedia.org/wiki/Time_series_database The king there bang for the buck is berkley DB.. http://en.wikipedia.org/wiki/Berkeley_DB If your talking massive data storage with a time series component though your talking HDF5... http://en.wikipedia.org/wiki/Hierarchical_Data_Format Obviously your not going to beat the performance of accessing flat binary files of tick data directly with an engine for backtesting..My problem with that though is I'm looking out 10 years from now. 10 years of tick data as flat binary files is just not logical..There is a reason to have a database... Matlab is already integrated with HDF5 directly.. http://www.mathworks.com/access/helpdesk/help/techdoc/ref/hdf5.html Like I said, there is almost nothing you can think of that has not already been thrown at matlab...BESIDES retail market datafeeds..which the only current real world solution seems to be IB data...my only hang up there is if IB data is "good" enough because i would be storing timestamps with "only" 20ms of granularity. Its absurd...I'm not looking to do high freq algo trading...Its just the stupid bias that I'm missing out on something, even though ninja was "filtering" the tape because I didn't understand how it handled my "true tick datafeed" for basically the past year. .

Well first I respect that Wayne is done with this thread. I just disagree that its a big deal that they have pricing for alpha stage software on the site. I mean while I may very well end up getting a license, I'm certainly not going to pay for alpha stage softare. Just like I wouldn't pay up as a sponsor when I'm only offering alpha stage software but I do see how that is a bit unfair here to other sponsors. To me the real problem with a great open source C# trading platform is all you will really end up doing is alot of grunt work for anyone that has programmers on staff. Its like how I've read David Shaw's last published paper was on how to connect Unix machines to market data...Then he realized how stupid it was to just be doing alot of work for free for other people and went into business with his ideas. I would actually like to discuss with you real programmers, why not just use matlab? It seems like the only stumbling block is getting data into matlab...Once you have that then most the other grunt work is done for you.. The grunt work for almost any database you would want to use is done, plotting/visualation wise I don't see how anyone can compete with the work that has been put into matlab, you can probly find almost any algorithm already done, all wrapped in a deadly programming language. Like I said in that post earlier, being able to build a market profile from tick data with this few lines of code is just off the map mind blowing considering what else it can do. MATLAB CODE: % clear all % clc; % %assuming tick by tick data % data = load('may01d.mat'); % ESdata = data.ESlast; ESdata=close; ESprices=unique(ESdata); bars=20; timertick=1; periods=floor(size(ESdata,1)/(timertick*bars)); MP=zeros(size(ESprices,1),periods); for i=1:periods g=ESdata(timertick*bars*(i-1)+1:timertick*bars*i,1); for m=1:size(g,1) s=g(m); s1=find(ESprices==s); MP(s1,i)=i; end end B = MP;%A(:,2:end) ; B(B==0) = inf ; B = sort(B,2) ; % sort each row B(isinf(B)) = 0 ; TPO=zeros(size(B)); for l=1:numel(B) if B(l)~=0 TPO(l)=B(l)+64; end end Also matlab has an engine now that supports Nvidia's CUDA directly for doing computation on the GPU.. http://www.accelereyes.com/

I think we need to put the breaks on "secondary agendas" here. It clearly drove away that poker guy. I'm very glad to see tickzoom here...Keep in mind I didn't invite him here like I posted earlier in this thread, I guess someone else pointed him here. If you followed the elitetrader threads at all what you quickly see is he just really likes to talk If you didn't follow those threads it will look like aggressive marketting. I would like to ask James to consider that this is early alpha stage software that is clearly trying to do something different. I can't see how you can really be a sponsor until you have a first release...In the mean time he seems extremely open to ideas, so bomb away.

I just don't buy that our representatives really care what the individual person has to think..Money talks, so if your not giving these guys campaign money you have no voice... The bright side of that though is we are all against this just as much as the heaviest hitters money wise in the world. Really though, this has to just be political posturing...I'm sure this guy knows this bill doesn't have a chance in hell of passing. However it makes him look good to the average person next election.

Just wanted to add I found this amazing blog from the guy who made MATLAB2IB. http://tradingwithmatlab.blogspot.com/ If you check out the code for creating market profile from tick data in matlab its pretty stunning how simple it is. He also has code for replicating the VIX...considering matlab can do that, it would imagine it blows away any retail software as far as breadth goes. Seems like if you really want to roll your own software its pretty sensless to not go with matlab/IB..the real grunt work is already done. $300 bucks a year.. They also have software for connecting matlab to TT and are working on a generic hookup to FIX.

Fish oil is just something everyone should probly take in general. It just doesn't seem realistic to get enough omega 3 from diet alone. Choline and stuff like that I imagine only works if your deficient. What your probly looking for is straight up nootropics.. I take piracetam daily and like what it does. http://www.bulknutrition.com/?products_id=912 For trading though its hard to beat coffee and cigerettes

I thought you died forever!!!! **** what anyone says here...a candle pattern is a simplistic 4 point representation of tick data...if your not front running the otherside of the breakout of the triangle then you have already given up optimal trade position...who cares after that.