In order to avoid bad surprises in live trading, the maxim forming this post title should apply to as much of a trading system as possible. The goal is to ensure back-testing results are as realistic as they can.

Input data is one component of a trading system and the area this post focuses on, more specifically roll-over methodologies for futures contracts.

Real Monthly Trading Also Means Monthly Roll-Over

One of the triggers for this post was a comment on the Monthly Trading post with regards to the futures data used for the test. In order to have comparable results between the daily and monthly instances of the systems tested, I simply and mechanically reused the same portfolio of instruments…

However, futures contracts in the portfolio roll on Open Interest. What this practically means, is that a trader using this methodology would have to monitor OI every day to decide whether to roll to another contract. Not much of a monthly process.

I still think that test had some value, but it is important to realise that the monthly instance in that test was not representative of a pure monthly process. Instead we were actually simulating a trading strategy with monthly signal taking but daily roll-over monitoring process.

It highlights the fact that it is important to understand what the data used for testing actually represents, and to roll futures contract data in the same way as they are rolled in real-life.

This post is a good opportunity to revisit the concepts of generating continuous futures data and some of the older posts I had written on the subject using CSI (here and here).

ETF: Outsource your Contract Rolling Process

Another reason why I think that the monthly trading test has some value is because the individual monthly trader might decide to trade ETFs instead of futures.

ETFs can be seen as simple wrappers of futures contracts: they trade continuously and their managers take care of rolling their underlying positions with a predefined methodology and timing, charging you a fee in the process.

Of course, this is not futures Trend Following any more, and using ETFs lacks the inherent leverage embedded in futures instruments. But it is possible to obtain decent performance with Trend Following on ETFs (check this ETF Trading book by Anthony Garner for example). An assumption in that book (which I will attempt to verify in a next post) is that futures can act as proxies for ETFs/ETCs.

Contract Rolling and Testing

Rolling contracts is a process necessary to generate a continuous time-series for back-testing futures instruments. There are many different ways to perform this operation (some of which are covered just below). What is important is that the continuous contract should be a realistic representation of one’s trading.

Proportional vs Point-Based Adjustments

At the time of writing these posts on continuous contracts, I was leaning towards the idea that proportional adjustment was the way to go:

“This ensures a constant relative (percentage-wise) relationship between any prices across the trading history“, as opposed to the point-based adjustment, with which “you lose relative price difference (shifting all prices by an absolute amount has that effect on percentage changes: 10 to 11 is +10% but add 100 to both for an extreme case and 110 to 111 becomes less than +1%)“.

When back-testing with Trading Blox, the standard functionality expects the prices to be point-based adjusted, to calculate trade PnL:
(ExitPrice – EntryPrice) * BigPointValue * NumberContracts
With a point-based adjustment, the number of points between entry and exit will represent the actual points the market has moved during the life of the trade, as opposed to proportional adjustment (which would give wrong PnL calculations).

But point-based adjustments prevents us from using relative price difference, such as “check if close today is 5% higher than close yesterday” as an example. This might be an issue for some systems, but there are workarounds for this in Trading Blox though.

We do not have to limit ourselves to one or the other methodology, but we have to understand what the rolling algorithm represents and which data best suit each test.

Of course, you can even go with a more fancy approach to rolling (if this represents what your trading is), like one we had covered here last year with a rolling methodology attempting to optimize roll yield.

Back-Adjusted or Forward-Adjusted?

Another switch on the rolling methodology dashboard (in CSI) is the possibility to perform back-adjustments or forward adjustments. This only determines which price is fixed and in which direction the “adjustment drift” will propagate through the time-series. Forward-adjustment picks the earliest price as the reference price and moves each following contract by the required adjustment amount/ratio to fill the “rolling gap”. Backward-adjustment is the opposite: reference price is last price and adjustment propagates backward.

In order to get recent prices in line with real price levels, most traders prefer back-adjustment. From a back-testing point of view, it should not have much impact. However, the chart time-series will look quite different.

To illustrate this, here is an example chart plotting the same underlying (Crude Oil) since 2007, with four different standard CSI rolling methodologies:

• Proportional, Forward Adjustment
• Proportional, Backward Adjustment
• Point-based, Forward Adjustment
• Point-based, Backward Adjustment

Mix of Point and Proportional, Back and Forward Adjustments

Going back to the monthly trading test, I did take my data “for granted” as it works for most cases… but not really for this one. It does emphasise the point that “Trade what you Test and Test what you Trade” should also apply to data and rolling methodologies, which need to be checked/reviewed for each new system.
 
 

I use CSI as my main source of data, including for all the research published on the blog and I have no problems recommending them.

So, when they approached me with a proposition for a “partnership”, which could also benefit the blog readers, I did not really hesitate.

I hope some of you can benefit from this offer – and unlike the January sale, it will be there all year.

Disclaimer (in the interest of transparency): any reader signing up to CSI using the coupon code would result in an affiliate commission – on top of your discount. A win-win proposition in a way. Note that I am a paying CSI customer and have been enjoying their data services for 2 years now (I use the Gold World Futures package and am about to get the Gold Stocks package).

For more info on CSI, please find below is a list of posts I ran a while ago on CSI data, their API, etc.:

• I just got myself an Unfair Advantage…
• Continuous Contract options
• Unfair Advantage API: Retrieve Back-Adjusted Contracts function
• Unfair Advantage API Code (C#): Extract Futures Continuous data
• CSI Raw Contract Data Extractor

CSI Data Extractor: Utility tool to extract CSI data in plain text files

I have mentioned before that I use CSI as my main provider for Futures data. Overall, I judge CSI’s service as good quality – however one drawback is the closed aspect of the data access. And I decided to write a utility to make it more “open”.

To access the historical data, you have to go through their software (Unfair Advantage), which extracts and decrypts it from their proprietary binary format. You can always “build” historical files to disk, but these are usually pre-processed with a special algorithm (ie. back-adjusted, etc.).

What more, if you decide to stop your subscription to CSI’s data services, you will not be able to access any data any more, even historical data downloaded under the subscription contract.

For some of the reasons above, it might be interesting to be able to extract/store CSI’s raw data in plain text files. This would allow for:

• Browsing the data outside of the CSI’s Unfair Advantage software
• Applying back-adjustment or other manipulation algorithms not supported by CSI’s UA (such as calculating the yield curve)
• Keep the historical data independently of your CSI subscription

Note: what I consider CSI raw data is the actual daily information for each contract traded on any day. For example, in the Corn market, there are currently 15 different contracts trading (with different maturities), each with their own Open, High, Low, Close, Volume and Open Interest.

CSI API to the rescue

Now, you could easily go to Unfair Advantage and pick every single historical contract that you want to extract… but this would be a fairly tedious job

Luckily for us, the API allows for programmatic access to CSI’s data to automate the process of extracting all contracts that we want – so I decided to write a little utility to do just that. And now my data is really mine!

Below is a zip file containing the files to copy to “install” the utility:

CSI-DataExtractor.zip

Note that you need to have Unfair Advantage installed on the machine running this exe (which does not come with any guarantee by the way… but I have been using it fine so far).

The operation is very simple: just enter CSI’s number for the market you want to extract and choose which path the files need to be extracted to.

Hope you’re finding this tool useful…

Note: You can receive a 10% discount on CSI data subscriptions (use coupon code LIBERTY)

Data - image by Pixelsior@flickr

In my other job, at a big investment bank, one of our main focus on a daily basis is DATA. Making sure that the hundreds of feeds and millions of records get uploaded correctly and contain the right information is key to a smooth, successful day.

There is surely not as much data to interact with in automatic/systematic trading but getting data right is equally vital to ensure quality trading or backtesting results.

Even with the best data package, there can be many pitfalls to look out for when trading or backtesting a systematic trading strategy. Below is a non-exhaustive list – please let me know in the comments if you think of anything to add.

Data Errors

Despite CSI being deemed one of the most reliable End-Of-Day Data providers, they still send you bogus data (albeit rarely) and you might want to run additional checks to see if the data is valid. Some ideas of checks to perform:

• Open, Low or Close above the High
• Prices = 0
• Price/Volume/Open Interest changes above tolerance
• Gap in the dates
• etc.

Of course, this will not catch all errors, but will give a pointer on large data anomalies.

Static Data

Back in the bank job, part of the system data which changes infrequently (hence static) is subject to 4-eyes approval (basically enforcing that another pair of eyes double-checks the correctness of the data). The idea is that, as this data drives the calculations in the system, any error would give flawed calculation results, and needs double-checking.

The equivalent in the automatic trading system world is the futures contract specifications and formats information (price quote type, decimal places, native currency, point value), contained in the FuturesInfo file in TradingBlox. Imagine an error of a factor 10 on the contract point value – this could have you get in a trade at 10 times the intended position size!

CSI appears to contain several of these errors (they seem to have less focus in getting this type of data right) and even official exchange websites have been seen to quote incorrect information. Reconciling both would be a starting point to getting the right information.

A possible solution to mitigate this problem (when starting trading a new, exotic instrument) is a quick in-and-out trade to double-check the actual information on your broker statement.

Data History and Overrides

The CSI data downloading process firstly retrieves all new prices to be stored in the Unfair Advantage database. It then applies potential corrections to past prices, in case errors were present. Rollover logic is applied and the full data history file is generated (to be used in the trading/backtesting software).

The change in data resulting from error corrections might change the outcome of past trade results. For example, a data correction in the recent past history might trigger a breakout and therefore a trade, now open. How would you handle this?

Additonally, if the historical data contains errors not corrected by your data provider, you might want to apply overrides to correct them yourself. However, since the price history gets re-written everyday, you need to have some form of automated overrides upload.

You might also want to make daily backups of price files to be able to compare when price corrections have taken place (and run your system with the exact data that was available on that day).

Other Data

Data such as historical FX/interest rates or Margin data are necessary to calculate some of the system statistics (including overall performance).

The FX rates are mostly used to convert the P&L from the native contract currency to the main account currency.

Interest rates would be used to calculate the interest earned on the account, which impact on the overall performance is not negligible for a long-term Trend Following system.

The Margin data is useful to determine the total amount of margin your system requires at any one time, with statistics such as the Margin-to-Equity ratio. Unfortunately, it is fairly hard to get hold of this type of data, which changes fairly often, at the discretion of the exchanges. Most backtesting platforms I have come across use a static margin number, which is fairly inaccurate.

To get more accurate results, you could run your backtesting simulations using historical (real or simulated) margin requirements. A method suggested on various forums to estimate historical margin requirements is to run a regression analysis between margin requirements (known history) and underlying instrument volatility (using different potential measurements such ATR, High-Low, Bollinger Bands, etc.). Finding a good-fit model would allow you to extrapolate past margin requirements based on volatility.

Holidays are also useful data to collect, to be able to know which exchanges are open when (note that exchanges can have different holidays even when they are in the same country).

Backup Provider

Your main data provider might be down or unavailable (hopefully on rare occasisons only). A backup, alternative source of market data might come in very handy for those special days.

Back-Adjustments and Rollovers

There is unfortunately no ideal or universal solution to roll and back-adjust contracts. As discussed earlier, a proportional back-adjustment allows for better representations of prices, keeping historical ratio values between all prices. However, backtesting systems usually require point-based back-adjustements in order to calculate the P&L of each trade correctly. You might need to maintain several sets of back-adjusted prices to be used in your simulations.

Furthermore, different instruments might require different rolling logic. Different delivery months in agricultural futures, for example, might relate to different crops. It might not make as much sense to roll from one ot the other in the same way that could be done with equity index futures.

Do you get the rollover information on rollover day?
Let’s assume that, on Thursday, you receive the data indicating that a roll should have occurred on Tuesday (ie. roll based on Open Interest or Volume shifts). Let’s also assume that an entry signal was triggered on the Wednesday and that the high volatility of the front contract would have prevented the system from entering the trade (risk management threshold breached) – whereas the next contract’s volatility, being much lower, would have allowed the trade. In real life, Wednesday’s trade would not take place (as the system does not know yet that the contract should have rolled on Tuesday). In backtested life however, Wednesday’s trade would take place, as the system would assume the roll has taken place on Tuesday. These things do actually happen!

You might want to test using the data as you would have received it, instead of a constructed back-adjusted series, potentially containing hindsight information.

LME Data

Some instruments deal in “strange fashion” and the London Metal Exchange is a perfect example of this. Trading on the LME is done by purchasing a forward contract with a given maturity (or prompt date). For instance, you might purchase a 3-month forward contract, which will run for the next 3 months gradually maturing to converge to the Cash price (at expiration), including the effect of contango or backwardation. However, the data quoted from CSI (and probably any other data provider) is a new daily quote for a new 3-month forward, which does not reflect real trading.

You would probably need to construct a custom price series with a bespoke algorithm to be able to reflect and backtest real-life trading. This topic was further discussed on a tradingblox forum post

Need for an ETL layer?

Drawing the parallel to the bank job again, where we use specialised tools/platforms to deal with data (Extract-Transform-Load = ETL); I believe that a similar layer between your data provider(s) and your trading/backtesting system, used to perform data checks, archive, roll and back-adjustments, etc. (all poinst discussed above) is necessary. It could also be used to add extra functionality such as creating spread time series to be traded/backtested as new instruments.

The data issue might not be as much of a trading problem as an IT one, nonetheless it is an important one that needs to be addressed – or it could impact your trading and backtesting results.

I just have to get something out on this 1st beginning of March. I could keep polishing this report until Christmas… But instead here is a first draft. The State of Trend Following report will be published at the beginning of each month (expect it to be evolving for the first few months).

The Concept

The principle was inspired to me by Michael Stokes from MarketSci with his very smart State of the Market report describing how the markets are behaving.

The idea is to track several classic Trend Following systems over a wide range of instruments and different timeframes. This should, in theory be a good complement to tracking the Trend Following Wizards to get an idea of how well Trend Following is doing as a strategy.

For this first report, I picked (arbitrarily) 3 standard Trend Following systems that ship with Trading Blox:

• Moving Average Crossover system with ATR-based stops
• Bollinger Band Breakout
• Triple Exponential Moving Average

Each system is tested over a short, medium and longer timeframe. Details of the system rules, parameters used and instruments traded can be found at the end of the post.

The Background Picture

As an initial run – to give a long-term idea of how each of the 9 systems performed (3 systems x 3 timeframes) – the whole futures data available in Trading Blox (provided free by CSI) was used for the backtest simulation.

One of the main interest of the study is how trends evolve over time and different systems. Therefore, the simulation parameters include no trade friction (ie no commissions and slippage at all). The equity curves below are therefore over-optimistic but they should really be used for relative comparisons.

Chart showing Systems Equity curves and Drawdowns. Each column represents a system (MA=MA Crossover; TMA=Triple Moving Average; BBBO=Bollinger Band Breakout) , with each row representing a timeframe (short, medium, long from top to bottom) - Click to zoom in

If there is one trend that seems to be shared across all 9 systems, it is the increased volatility – expressed by the larger drawdowns having appeared in the last 4-5 years in all systems.

Imagine starting trading the Golden Crossover system (50-200 MA Crossover) on January 1st 2003 – after good backtesting results (CAGR>20% and worst drawdown around 10% – a MAR of 2), only to see drawdown figures quadruple in the next 5 years!

This could be down to the fact that most instruments have become more correlated in the last few years, in effect negating some benefits provided by diversification.

Zoom in to 2009 and 2010

We are also interested in the most recent picture. Below are 2 charts comparing our 9 systems on the 2009-2010 and 2010-only time horizons (each system adjusted back to 100 at the start of the period). Each type of system is represented in one colour and each timeframe in one line style:

Nothing very exciting in 2009, and with addition of trade friction, it seems to correlate with the (negative) results witnessed on the Trend Following Wizards. Also note that most systems follow the same path (apart from the short-term Bollinger system, all over the place with extreme volatility – the chart does not show it, but it closed February 2010 at around 60).

2010 seems to be a continuation of 2009 so far: with a slight downward trend emerging across all systems.

Improvements to the report

There are quite a few things that could be added to this report, starting with integrating more systems and parameters (probably aggregated as an average per timeframe to avoid the data deluge syndrome). It might also be interesting to be able to break down the performance by type of markets/sectors.

But I am also open to your suggestions and requests. So please let me know in the comments below and I will integrate the best ideas.

Until next month…

Appendix: System Details

System Rules and Parameters

All the systems were tested with the same simplistic position sizing rules of 1% per new trade. No other Money/Risk Management rules were used.

The system rules are detailed on the Trading Blox online documentation.
The MA Crossover system was used with moving average pairs of 10-20, 20-50 and 50-200 days. The stops are set at 5 x ATR(39).
The Bollinger Band system is the classic use of the Bollinger Bands with entries taking place at Breakouts. The parameters used were 20, 50 and 200 days with 2 standard deviations.
The Triple moving Average system was used with moving average triplets of 10-20-50, 20-50-200 and 50-200-800 days. The stops are set at 5 x ATR(39).

Portfolio Instruments

The instruments included in the simulation are a semi-diversified list of futures, contained in the All Futures Trading Blox set:

I wanted to start this year with an idea I left off last year, when investigating robustness: Using the median as a more robust tool than the ubiquitous mean (or average). This will be covered in the next post via a TradersStudio simulation results. To do that, I had to re-import some data from CSI Unfair Advantage to TradersStudio. I thought a “how-to” post on this would be useful – if only as an aide-memoire for next time (note-taking and documentation are important after all…)

Extract CSI Data

The first step is to extract the data from the CSI database into text files
(by the way, CSI have a 20% discount running until the end of January – just check their homepage).
This can be done with code using the Unfair Advantage API.

Pick all the instruments to be extracted and create a “contracts.param” file containing the CSI number for those instruments in the first column (comma-separated file with other columns being ignored). The CSI numbers can be found on their website (like here for the CME Futures) or in their Unfair Advantage software UI. Running the EXE code will generate one file for each instrument (proportionally back-adjusted contract) in the following format:

19900403,199006,0,240,110,918,503,2,0,1.59,1.655,1.58,1.635,1.635,1.635,1.635
19900404,199006,0,84,127,656,721,3,0,1.6,1.65,1.599,1.62,1.62,1.62,1.62
19900405,199006,0,57,112,554,885,4,0,1.615,1.635,1.61,1.615,1.615,1.615,1.615
19900406,199006,0,62,136,436,1092,5,0,1.615,1.635,1.61,1.625,1.625,1.625,1.625

Load in TradersStudio

The second step is to load the files in TradersStudio. Copy all generated text files in a dedicated data directory (you might have to rename them as TradersStudio allowed filename length is fairly short, i.e. 10-15 characters).
Create the relevant mapping file (txtinfo.txt) in the data directory. The mapping file should read:

delimit=,
skip=0
dt=1
ti=0
op=10
hi=11
lo=12
cl=13
vol=6
oi=7
dtformat=CCYYMMDD
tiformat=

In TradersStudio, menu Data Manager/Add Data Links, point to the relevant data directory with the following options:

If all goes well, this will load the data in the system (errors I encountered were due to file names being too long).

Data Universe: Contract meta-data

The third step is to populate the meta-data (margin, min tick move, etc.) for each instrument in TradersStudio. This is done via the menu Data Manager/Data Universe.

Once done, the instruments can be used just by adding them to any session.

As mentioned earlier the next post will be a test comparing moving average and moving median indicators.

Note: You can receive a 10% discount on CSI data subscriptions (use coupon code LIBERTY)

• Broker Account – The starting point to trade in the markets
• Live Market Data – To feed to your trading robot so that it can generate trading signals. Most brokers provide market data with proprietary or third-party technology- although market data can also be obtained from an independent vendor (e.g. eSignal).
• Order Routing – To transmit your orders to the exchange. These can be proprietary to the executing broker or third-party based (e.g. Zen Fire).
• Broker Trading Platform – To access your account information, place orders, etc.
• Charting Platform – Sometimes the broker platform charting capabilities are mediocre and an independent charting platform is required (e.g. eSignal)
• Signal Generation Platform – The actual trading robot: it analyses market data, account and positions information and implements the automated trading strategies to generate trade management signals (i.e. Buy / Sell / Exit, etc.).
• Excecution Platform – Receives trade signals and forwards them to the order routing.
• Platform Hosting – You can host the trading robot on your local machine, host it in a dedicated rented server or entrust a specialised broker to host and run your trading robot.
• Historical Market Data – Used as source for back-testing trading strategies
• Strategy Development Application – Allows for coding, optimisation, back-test and result analysis of trading strategies.
• Statistical Analysis package – for in-depth statistical analysis of back-testing results.

Some of these distinctions are physical, some are logical. For example, TradeStation packs many of the points above into one system: a broker account and a trading platform that allows you to:
Access live market data + Visualise price charts + Automate trading strategies that generate signals + Route the trading signals as orders on the exchange + Access account information + Store historical data + Back-test strategies

Sounds like it fits the bill perfectly, doesn’t it?
Well let’s look at the areas you need to consider when deciding on your automated platform.

Type of broker

Different issues to consider: is the broker a clearing broker, a non-clearing FCM or an introducing broker? If the latter is it an independent introducing broker or a guaranteed introducing broker (backed by a clearer)? It would seem that the closer to the exchange you broker is (in terms of relationship), the better/safer it is. See Anatomy of a Futures transaction for more detail on the different types of brokers.

Markets supported

Every broker offers access to different products and different markets. For example Interactive Brokers offers one of most complete coverage (worldwide Futures, Options, Equites and Forex) all “housed” under the same Universal Account. Other brokers only offer access to specific products/markets.

Latency/Speed

For some type of trading strategy, this can have a large impact and various platforms/feeds offer different levels of performance. This has led to specialist companies such as Zen Fire or Trading Technologies – which locate their servers very close to the exchange – achieving speed and reliability for data feed and order routing. They offer their services to brokers who can in turn offer it to their clients.

Development Language

This is mostly a matter of personal preference, experience and expertise. TradeStation has been around for longer and a big collection of Easy Language scripts are available for re-use while NinjaTrader, for example, uses C# and CQG Trader their own language.

Strategy Complexity vs. Platform Functionality

Most of the automated trading platforms that offer back-testing functionality lack sophistication of dedicated applications such as Mechanica, TraderStudio or Trading Blox. For example portfolio testing is usually not available in the live trading platforms. This might work for some simple strategies but not for more complex Money Management ones.

Hosting

You can run the trading robot on your machine, which makes you subject to power and connection failures or you can choose to rent a server to host your robot. Some brokers also offer trading strategy hosting and execution on their own servers.

Charting

Some specialised platforms offer better charting capabilities than the standard trading platforms and can be used as a standalone charting application. Some, such as eSignal, also offer a data feed.

Costs

An obvious one but between platform costs, execution costs, hardware, etc. there exists a wide range between bottom-end and top-end offerings.

What/how to choose?

While researching for this article I have come across many different packages offering different solutions. One of the best ways to help you decide on a solution that makes sense to you is to list your goals and cross-check each platform against them in this fashion (PDF doc).

There are endless possibilities for combining all the differenty packages. Just for fun I tried to “virtually build” the opposite of an all-in-one package such as TradeStation (which will probably not address most automated trader goals) – here is one “wacky” implementation.

A trader decides to use TradeStation as a signal generating platform. However she does not want to use TradeStation data feed and instead decides to go for the eSignal package integrating the broker Dorman Trading and the Zen-Fire connection for market data and a charting platform. In order to route the market data to TradeStation, she needs a bridge application - OwnData from TS Support for example. The backup feed solution would be TradeStation data feed. For the execution side of things, our trader decides to send the trading signals generated by TradeStation to NinjaTrader, which can route the orders to the exchange through a broker. The primary route is through Velocity Futures broker using Trading Technologies order routing, while the secondary route is through Interactive Brokers via their TWS trading platform and API! Additionally, the strategy research and testing is performed with Mechanica using CQG tick data while the results are analysed using R.

Conclusion

Sorry to end up on a cliche but there are no silver bullet to making the choice. If you take away one point is that you need to list your own requirements clearly and make the choice for yourself based on them.

Check that 60-page thread at EliteTrader if you want an endless discussion as to which is the best platform/architecture…

In the last few posts I have been exploring the e-ratio and wrote the code in TradersStudio. I also tried to implement it in AmiBroker AFL language mostly to check how it performed speed-wise – as I found TradersStudio quite slow.

While testing AmiBorker, it actually felt so fast that I decided to perform a more formal speed comparison test.

Conditions of the test

I exported the CSI data via the API for a proportionally back-adjusted Corn contract going back to 1949. The data was imported in both AmiBroker and TradersStudio via their ASCII Import.

The system tested is a simple 20-day Donchian Channel Breakout (Buy-only) and the ATR used to normalise the MAE/MFE is 20-day also.
The optimiser in both systems were used to generate the e-ratio for 50 51 different trade durations (from 10 to 50 days).

One of the condition to validate the results of the test was that the trades generated by both systems are similar (to double-check I did not make a coding mistake resulting in simpler/faster process for one of the platforms).

The computer I was running the test on is a quad-core CPU (2.4 GHz) with 3.25 GB of addressable RAM. Only one platform was running at the time it was tested.

Results are in!

And they look astoninglishly good for AmiBroker:

• The TradersStudio test was run first and took 4 mins 15 sec to complete and produce the custom report (which needs to be manipulated in Excel to calculate the e-ratio). Furthermore, it appeared that every incremental run in the optimisation process took slightly longer than the previous one.
• The AmiBroker test ran in (…drumrolls…) 10 sec! And the e-ratio was readily available from the results grid:
  
  

  AmiBroker optimisation results showing the e-ratio. Click to expand

Both apps maxed out their allocated CPU (i.e. overall CPU usage of 25% = a quarter of the quad-core CPU available).

I then decided to run the AmiBroker code over 5 markets to get a feel of how long it would take. I added Crude Oil (going back to 1985), Cotton (1968), Gold (1975) and Yen (1972).
AmiBroker ran the same code + optimisation in 2 mins 20 sec, twice as fast as one market in TradersStudio!

Applying a simple proportional calculation to derive the time it would take in TradersStudio for the same dataset would give us a completion time of 1 hour (59 min 30 sec exactly…). Since the performance seems to degrade over the course of the optimisation in TradersStudio (i.e. the first back-test is quicker than the last one of the optimisation run) we could assume that it would actually take longer, which is what I experienced when I was testing on similar data in the last few weeks.

So here you have it: AmiBroker is 25 times faster than TradersStudio and it gives you the results in a much more friendly format.

Trade Reconciliation

As mentioned earlier, to ensure that the test was valid, the set of trades generated by both systems should be compared. I ran one back-test of the system over the Corn data for a single optimisation step (e-ratio for 20 days duration) in both systems.

I could find that 296 trades out of 340 presented a very good match (some matches had one day difference on the Entry date and rounding differences on the prices). But overall the trade comparison was good enough to give assurance that the same systems and results were being tested.

I’ll keep investigating for myself where these errors are coming from (and post a further code update if warranted) but we can assume that the differences are trivial with regards to performance comparison.

Did I mention that AmiBroker is 3 times as cheap with a much wider following (many more code samples, better docs, more forums, etc.)?…

Why I chose TradersStudio

I did consider a few options before buying TradersStudio (Amibroker, TradeStation, Trading Blox, WealthLab, NinjaTrader) and based on feature analysis, recommendations and actual testing, I settled on TradersStudio.
My requirements were for a standalone platform that would allow me to test any system, with money management and multiple system interaction.
I did hesitate with Trading Blox, which I enjoyed testing (demo version), but in the end the price sorted the argument (Trading Blox is $3,000 for similar functionality).
For more info on what each product offers, I would definitely recommend heading to EliteTrader where many platform comparison posts can be found. Please also feel free to ask me questions in the Comments section below.

First impressions

Not so good… There is no demo available on the website, you can not download the app from the website once purchased and the manuals you receive after waiting a few days seem quite unprofessional (despite being over 300 pages).
However, the install is quite easy and you can be up and running following examples from the manual in a few minutes.

Main Features

The app offers some interesting features which help me take the decision:

Integrated Systems testing

TradersStudio uses different hierarchical levels: Instrument, System, Session, Trade Plan. This allows you to test a complete system (i.e. more than simple entry and exit testing). A Trade Plan can contain several sessions, which in turn can contain several systems and instruments with possible interaction between them. Additionally, Money Management options can be tested as part of the Trading Plan. This makes it much more realistic than a collection of independent strategy tests.
There is also to the concept of virtual systems that allow you to run a system in “monitor” mode and only use it for decision making on other “live” systems (i.e. enter a trade in real system only after virtual system posts 3 losing trades).

Customisation

TradersStudio implements its own Macro language, which appears to be a cross between TradeStation EasyLanguage and Visual Basic. Theoritically you can implement whatever extra functionality you require (indicator, software add-in, etc.). Moreover there is an import tool that automatically converts TradeStation EasyLanguage code, which should allow to reuse available code.

Data-independent

You can basically load data from whatever source you wish in any supported format (any standard text file format will do). I personally use Unfair Advantage from CSI and getting it loaded to TradersStudio was a breeze (see Unfair Advantage posts for more information on CSI UA -> TradersStudio)

Walk Forward

Walk forward is a useful concept to use in back-testing to avoid over-optimization and curve-fitting. TradersStudio implements this functionality and runs it automatically which means less manual workaround for you to deal with.

Documentation issues

My main grudge is about the manuals. Despite being over 300 pages it is still missing an essential reference guide for programming using their language (there are a few examples but they do not cover the whole range of functionality and much is left to be guessed by the user). It is a “Learning by Example” type of manual which can sound a bit patronising at times.
For that reason I have found it hard to seriously get into TradersStudio… but I have just signed up for the yahoo group dedicated to it and will be spending many hours in front of it very soon!

I will explore TradersStudio in more details as I start using it and share in a later post if you are interested.

Getting started with the API

Fire up your favourite IDE/language that supports COM Interoperability (I am using Visual C# 2008 Express Edition), start Console Application project and add a reference to the Unfair Advantage API. For this you just need to “Add reference” and browse to the Unfair Advantage installation folder and select the main EXE (uad.exe). This gives access to the CSI UA API.

Add UA into your project and "Bob's your Uncle!"

Simple extraction code

The rest is fairly simple as long as you understand the RetrieveBackAdjustedContract2 parameters.
The code below takes 2 parameters: an input file name containing the list of markets to extract and a folder name to extract each continuous contract to. It sets specific UA session parameters and calls the RetrieveBackAdjustedContract2 with specific variables to obtain the concatenation desired. Each continuous contract is output to a folder in a .txt file.
The file containing the list of markets is of the following format (CSI number, Symbol, Market name):
…
8,CT,Cotton
…

The code

Please find the code there (very nicely formatted thanks to this great tool by manoli). You can also download the Program.cs file:

Note: You can receive a 10% discount on CSI data subscriptions (use coupon code LIBERTY)