Systematic Trading research and development, with a flavour of Trend Following
Au.Tra.Sy blog – Automated trading System header image 2

How-To: CSI Data into TradersStudio

January 12th, 2010 · 4 Comments · Data

Dear readers, I hope you enjoyed the holiday break. I wish you a happy New Year – may it bring you progress and success in your automated trading endeavours!

I wanted to start this year with an idea I left off last year, when investigating robustness: Using the median as a more robust tool than the ubiquitous mean (or average). This will be covered in the next post via a TradersStudio simulation results. To do that, I had to re-import some data from CSI Unfair Advantage to TradersStudio. I thought a “how-to” post on this would be useful – if only as an aide-memoire for next time (note-taking and documentation are important after all…)

Extract CSI Data

The first step is to extract the data from the CSI database into text files
(by the way, CSI have a 20% discount running until the end of January – just check their homepage).
This can be done with code using the Unfair Advantage API.

Pick all the instruments to be extracted and create a “contracts.param” file containing the CSI number for those instruments in the first column (comma-separated file with other columns being ignored). The CSI numbers can be found on their website (like here for the CME Futures) or in their Unfair Advantage software UI. Running the EXE code will generate one file for each instrument (proportionally back-adjusted contract) in the following format:

19900403,199006,0,240,110,918,503,2,0,1.59,1.655,1.58,1.635,1.635,1.635,1.635
19900404,199006,0,84,127,656,721,3,0,1.6,1.65,1.599,1.62,1.62,1.62,1.62
19900405,199006,0,57,112,554,885,4,0,1.615,1.635,1.61,1.615,1.615,1.615,1.615
19900406,199006,0,62,136,436,1092,5,0,1.615,1.635,1.61,1.625,1.625,1.625,1.625

Load in TradersStudio

The second step is to load the files in TradersStudio. Copy all generated text files in a dedicated data directory (you might have to rename them as TradersStudio allowed filename length is fairly short, i.e. 10-15 characters).
Create the relevant mapping file (txtinfo.txt) in the data directory. The mapping file should read:

delimit=,
skip=0
dt=1
ti=0
op=10
hi=11
lo=12
cl=13
vol=6
oi=7
dtformat=CCYYMMDD
tiformat=

In TradersStudio, menu Data Manager/Add Data Links, point to the relevant data directory with the following options:

DataManager

InstrumentType

If all goes well, this will load the data in the system (errors I encountered were due to file names being too long).

Data Universe: Contract meta-data

The third step is to populate the meta-data (margin, min tick move, etc.) for each instrument in TradersStudio. This is done via the menu Data Manager/Data Universe.

DataUniverse-MetaData

Once done, the instruments can be used just by adding them to any session.

As mentioned earlier the next post will be a test comparing moving average and moving median indicators.

Note: You can receive a 10% discount on CSI data subscriptions (use coupon code LIBERTY)

Related Posts with Thumbnails

Tags: ····

4 Comments so far ↓

  • Milktrader

    Good post on the mechanics of importing data. I use Pinnacle Data. Do you have a strong preference for CSI or are they about the same in your view?

    One extra thing I’ve done is add a category to certain markets. This option is available on the meta-data dialog box. For my purposes, I’ve designated a market as 70 if its data goes back to the 1970 decade. And 80, 90, 97. But you can also add type of markets such as metals, softs, etc.

  • Jez

    Thanks milk! I see that you have been busy yourself with R over the break. Another thing I need to get into ;)

    With regards to CSI vs Pinnacle, I have to admit that I did a bit of “web research” and it seemed like they were 2 of the best (in that range). But I did not make any direct comparison analysis myself.

    Not wanting to engage too much in analysis paralysis, I just read about CSI, saw that they provided data to Yahoo and Google, that they had a thorough back-adustment software with API, a very wide instrument coverage, and was endorsed by serious trend followers (so was Pinnacle). And I just went for them!

    You might be interested in this post including a link to a Futures magazine article covering data and vendor comparison (note: it is part of CSI marketing collateral)

  • jeff

    i just bought the CSI deluxe equity package for US equities along with data on delisted stocks (i saved $400 as they currently have a promotion running – thanks for pointing that out). i like the API too, which i use from Excel. i also bought similar data from Norgate (an australian company) but their data is not dividend adjusted. i’m trying to develop a 1 week horizon stat arb strategy, so survivorship free data is important for me. CSI and Norgate were the most affordable i came across. with 2 vendors, i can cross-check some aspects of my data. i used an Excel macro to pull data from CSI and Norgate api and save data to MS Access database which i will later call from matlab or excel. do you think traderstudio can efficiently handle a strategy that is dependent on runing regressions on 1,500 stocks?

  • Jez

    Jeff,
    I have not looked at stat arb (I am more geared toward trend following on futures). So I cannot tell you precisely ho well TradersStudio would handle running regression on 1500 stocks but I know that it offers inter-market analysis and I am sure you could develop some sort of regression calculation. Your best bet might be to get in touch with Murray Ruggiero, their CEO – who is pretty active on the elitetrader boards and the tradersstudio yahoo group.

    Note that TradersStudio’s performance is not great, as you might have seen in my speed comparison post with AmiBroker

Leave a Comment