Jump to content

Welcome to the new Traders Laboratory! Please bear with us as we finish the migration over the next few days. If you find any issues, want to leave feedback, get in touch with us, or offer suggestions please post to the Support forum here.

  • Welcome Guests

    Welcome. You are currently viewing the forum as a guest which does not give you access to all the great features at Traders Laboratory such as interacting with members, access to all forums, downloading attachments, and eligibility to win free giveaways. Registration is fast, simple and absolutely free. Create a FREE Traders Laboratory account here.

darthtrader

Data Mining with R and Rapidminer

Recommended Posts

From the thread that brownsfan and I posted in about Steenbarger and his volume/days range stuff until now I've stumbled across some AMAZING stuff in this area. To program alot of this stuff in stock software is to reinvent the wheel.

 

This interesting thread came up on Elite that mentions Rapidminer.

http://elitetrader.com/vb/showthread.php?s=&threadid=117361

 

In trying to find information about it I finally installed and actually loaded some YM data into the heavy weight open source statistical computing environment, R. Its amazing how simple it makes tasks that are far beyond most stock software analysis wise.

 

Here is an entire course on data mining with R

http://www.stats202.com/

The videos for the class are all up on google video if you search for data mining + long format.

 

This is an entire free book on data mining with R, 2 different hands on projects and one is a forecast for IBM stock prices using nets.

http://www.liaad.up.pt/~ltorgo/DataMiningWithR/

 

this has a 5 part video tutorial on using Rapidminer:

http://www.neuralmarkettrends.com/tutorials/

 

Jerry in that elite thread had a project idea for rapidminer and I had just sent him a message inviting him to bring it to TL. The premium section here would be quite an ideal place for such results.

 

is anyone else interested in this stuff? While I'm not interested in "prediction" it just seems like a huge waste of time to not use these tools to find relationships that would take years of experience to uncover(if ever).

Share this post


Link to post
Share on other sites

Well I have already given up on Rapidminer. While extremely interesting there simply is no way documentation wise that someone with no background in stats/data mining is going to be able to get up to speed with it.

R is a different story having entire series of textbooks written on its use from introductory stats to extremely complex stuff.

 

I've just ordered this textbook which sounded like a good way to get up to speed on the various mining algorithms

Data Mining: Concepts and Techniques, 2nd ed

http://www.amazon.com/Data-Mining-Second-Techniques-Management/dp/1558609016/ref=pd_sim_b_3

 

I'm still trying to figure out what R book to go with but this one sounds pretty nice, focusing on working code and available data sets to learn R with.

http://www.amazon.com/Statistics-Introduction-Michael-J-Crawley/dp/0470022981/ref=pd_bbs_sr_1?ie=UTF8&s=books&qid=1209041520&sr=1-1

Share this post


Link to post
Share on other sites

Just some general R resources before I lose them.

 

RExcel is probly the way to go at this level. You can basically either use R within excel or read and write excel files directly in R. Should be able to get R doing real time market stuff with a data providers DDE link with this.

http://sunsite.univie.ac.at/rcom/server/doc/RExcel.html

 

Good R learning resource

http://www.mayin.org/ajayshah/KB/R/index.html

http://www.agr.kuleuven.ac.be/vakken/statisticsbyR/

 

good financial stuff in R

http://www.burns-stat.com/

 

charts for R

http://addictedtor.free.fr/graphiques/RGraphGallery.php?graph=65

 

financial engineering stuff

http://www.rmetrics.org/

Share this post


Link to post
Share on other sites

ok - new to data mining (although not new to the concept, or to the markets). Is there anything online you can reccomend for me if I'm lookign to really start from scratch. Can program in VBA so I'm sure I could get my head round the code if I try.

 

Any / all answers appreciated. Have a tactical relative value trading product here which I am spearheading (in between doing a grillion other things as per usual), so might be getting involved in this a bit. Been looking at all sorts of options. FX engines etc etc. But while there are plenty of pruducts out there to backtest stuff for you if you are specific enough, ultimately what I really think I have to do is do the data mining myself.

 

And for both cost and time reasons I'm thinking it might be easiest to actually do it myself (or at least, supervise while ony of our jnr traders does the grunt work) than pay for an engineer to come in and do it (especially as I'm not 100% sure what I'm looking for, just have a few ideas I want to look at right now and thats never an efficient use of outside IT type people in my experience).

 

Sorry this post is a bit rambling - busy afternoon on the desk so I keep coming back and adding a bit

 

GJ

Share this post


Link to post
Share on other sites

I'm actually looking to do some data mining myself soon. I have JMP 7 by SAS. It's an extrodaniry piece of software to mine large amounts of data:

 

http://www.jmp.com/software/jmp7/

 

I use Investor RT and the only problem I'm having at the moment is figuring out how to extract 5 years worth of Market Profile pivot data for the E-Mini S&P. If anyone can help do this I can hopefully post some results for people in the premium forum.

 

Cheers.

 

:)

Share this post


Link to post
Share on other sites

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.

Guest
Reply to this topic...

×   Pasted as rich text.   Paste as plain text instead

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.


  • Topics

  • Posts

    • AXON Enterprise stock, top of range breakout at https://stockconsultant.com/?AXON
    • FOUR Shift4 Payments stock with a solid breakout above 122.9, from Stocks to Watch at https://stockconsultant.com/?FOUR
    • EBS Emergent Biosolutions stock watch, pullback to 9.9 triple+ support area with bullish indicators at https://stockconsultant.com/?EBS
    • POAI Predictive Oncology stock with a bull flag breakout, target 2 area at https://stockconsultant.com/?POAI
    • Date: 18th February 2025.   UK Unemployment Rate Falls and The Pound Spikes Upwards.     The British Pound spikes upwards against all currencies as the UK releases its employment data. However, the latest employment data release does not give long-term confidence as the UK continues to see a higher possibility of economic stagnation in 2025. Can the GBP maintain momentum?   UK Releases Latest Employment Data! The UK employment data had its positive and negative points. The Monthly Unemployment Claims rose 22,000 which is at a 3 month high, and higher than analysts’ previous expectations. This is known to be negative for the British Pound. However, the UK also saw some positive data which investors are clinging onto. The UK Unemployment Rate fell for the first time since October 2024. The UK Unemployment Rate, to the surprise of analysts, fell from 4.5% to 4.4%. Lastly, the Average UK Salaries Index rose to 6.00%, the highest in 13 months and higher than previous expectations. This is the main reason why the GBP is increasing in value. That said, the Bank of England and economists continue to expect the UK to witness stagnation in 2025.     The British Pound The British Pound is now one of the best-performing currencies of the day so far. The US Dollar and Japanese Yen are also strongly increasing in value. The Governor of the Bank of England, Mr Bailey, is due to speak at 09:30 GMT and is likely to comment on the latest employment data.   Previously, Bailey described the UK’s economic growth as “static,” despite stronger-than-expected Q4 2024 data—0.1% growth instead of the forecasted –0.1% quarterly and 1.4% annually versus the expected 1.1%. Meanwhile, the BoE revised its 2025 GDP growth forecast down to 0.75% from 1.0% in November. Traders are also hoping Governor Bailey will comment on the possible future rate cuts.   Tomorrow at 09:00 (GMT+2), the UK will release January inflation data. Analysts expect the annual CPI to rise from 2.5% to 2.8%, while monthly prices may drop by 0.3% after a similar increase in December. The Core CPI is projected to climb from 3.2% to 3.6%.   When evaluating the GBP Index, the GBP is currently trading 0.95% higher in 2025. However, the upward price movement is largely due to last week’s Gross Domestic Product which beat expectations. The performance of the GBP will also depend on whether the US imposes tariffs. Additionally, pressure on the UK to increase defence spending could further strain the country's already scrutinized budget.   GBPUSD - Technical Analysis and Price Condition The GBPUSD is trading above the main moving averages on the 2-hour timeframe and is trading high on most oscillators. These factors indicate that the buyers are currently controlling momentum, but traders are concerned about two factors. The first is that the GBPUSD is struggling to break above the 1.26300 level and the fact that both the USD and GBP is simultaneously increasing in value. As both currencies are increasing in value, technical analysts view the price action as conflicting. On the 5-minute chart, the GBPUSD is trading at the 200-bar average price movement indicating a neutral signal. This also follows the concerns of traders that the price action is conflicting.     If the price breaks above 1.25918, the GBPUSD may witness sell signals materialize. However, if the price breaks above 1.26200, buy signals may arise which will also be in line with the indications on the 2-hour timeframe.   Key Takeaway Points: GBP rises as the UK employment data lifts GBP, but stagnation concerns remain. UK Salaries hit a 13-month high, boosting the Pound. The Bank of England Governor, Mr Bailey may hint at future rate cuts and advises the UK will witness economic stagnation. The key risks for the GBP remain inflation data, US tariffs, and UK defence spending pressure. Always trade with strict risk management. Your capital is the single most important aspect of your trading business.   Please note that times displayed based on local time zone and are from time of writing this report.   Click HERE to access the full HFM Economic calendar.   Want to learn to trade and analyse the markets? Join our webinars and get analysis and trading ideas combined with better understanding of how markets work. Click HERE to register for FREE!   Click HERE to READ more Market news.   Michalis Efthymiou HFMarkets   Disclaimer: This material is provided as a general marketing communication for information purposes only and does not constitute an independent investment research. Nothing in this communication contains, or should be considered as containing, an investment advice or an investment recommendation or a solicitation for the purpose of buying or selling of any financial instrument. All information provided is gathered from reputable sources and any information containing an indication of past performance is not a guarantee or reliable indicator of future performance. Users acknowledge that any investment in Leveraged Products is characterized by a certain degree of uncertainty and that any investment of this nature involves a high level of risk for which the users are solely responsible and liable. We assume no liability for any loss arising from any investment made based on the information provided in this communication. This communication must not be reproduced or further distributed without our prior written permission.
×
×
  • Create New...

Important Information

By using this site, you agree to our Terms of Use.