In order to avoid bad surprises in live trading, the maxim forming this post title should apply to as much of a trading system as possible. The goal is to ensure back-testing results are as realistic as they can.

Input data is one component of a trading system and the area this post focuses on, more specifically roll-over methodologies for futures contracts.

Real Monthly Trading Also Means Monthly Roll-Over

One of the triggers for this post was a comment on the Monthly Trading post with regards to the futures data used for the test. In order to have comparable results between the daily and monthly instances of the systems tested, I simply and mechanically reused the same portfolio of instruments…

However, futures contracts in the portfolio roll on Open Interest. What this practically means, is that a trader using this methodology would have to monitor OI every day to decide whether to roll to another contract. Not much of a monthly process.

I still think that test had some value, but it is important to realise that the monthly instance in that test was not representative of a pure monthly process. Instead we were actually simulating a trading strategy with monthly signal taking but daily roll-over monitoring process.

It highlights the fact that it is important to understand what the data used for testing actually represents, and to roll futures contract data in the same way as they are rolled in real-life.

This post is a good opportunity to revisit the concepts of generating continuous futures data and some of the older posts I had written on the subject using CSI (here and here).

ETF: Outsource your Contract Rolling Process

Another reason why I think that the monthly trading test has some value is because the individual monthly trader might decide to trade ETFs instead of futures.

ETFs can be seen as simple wrappers of futures contracts: they trade continuously and their managers take care of rolling their underlying positions with a predefined methodology and timing, charging you a fee in the process.

Of course, this is not futures Trend Following any more, and using ETFs lacks the inherent leverage embedded in futures instruments. But it is possible to obtain decent performance with Trend Following on ETFs (check this ETF Trading book by Anthony Garner for example). An assumption in that book (which I will attempt to verify in a next post) is that futures can act as proxies for ETFs/ETCs.

Contract Rolling and Testing

Rolling contracts is a process necessary to generate a continuous time-series for back-testing futures instruments. There are many different ways to perform this operation (some of which are covered just below). What is important is that the continuous contract should be a realistic representation of one’s trading.

Proportional vs Point-Based Adjustments

At the time of writing these posts on continuous contracts, I was leaning towards the idea that proportional adjustment was the way to go:

“This ensures a constant relative (percentage-wise) relationship between any prices across the trading history“, as opposed to the point-based adjustment, with which “you lose relative price difference (shifting all prices by an absolute amount has that effect on percentage changes: 10 to 11 is +10% but add 100 to both for an extreme case and 110 to 111 becomes less than +1%)“.

When back-testing with Trading Blox, the standard functionality expects the prices to be point-based adjusted, to calculate trade PnL:
(ExitPrice – EntryPrice) * BigPointValue * NumberContracts
With a point-based adjustment, the number of points between entry and exit will represent the actual points the market has moved during the life of the trade, as opposed to proportional adjustment (which would give wrong PnL calculations).

But point-based adjustments prevents us from using relative price difference, such as “check if close today is 5% higher than close yesterday” as an example. This might be an issue for some systems, but there are workarounds for this in Trading Blox though.

We do not have to limit ourselves to one or the other methodology, but we have to understand what the rolling algorithm represents and which data best suit each test.

Of course, you can even go with a more fancy approach to rolling (if this represents what your trading is), like one we had covered here last year with a rolling methodology attempting to optimize roll yield.

Back-Adjusted or Forward-Adjusted?

Another switch on the rolling methodology dashboard (in CSI) is the possibility to perform back-adjustments or forward adjustments. This only determines which price is fixed and in which direction the “adjustment drift” will propagate through the time-series. Forward-adjustment picks the earliest price as the reference price and moves each following contract by the required adjustment amount/ratio to fill the “rolling gap”. Backward-adjustment is the opposite: reference price is last price and adjustment propagates backward.

In order to get recent prices in line with real price levels, most traders prefer back-adjustment. From a back-testing point of view, it should not have much impact. However, the chart time-series will look quite different.

To illustrate this, here is an example chart plotting the same underlying (Crude Oil) since 2007, with four different standard CSI rolling methodologies:

• Proportional, Forward Adjustment
• Proportional, Backward Adjustment
• Point-based, Forward Adjustment
• Point-based, Backward Adjustment

Mix of Point and Proportional, Back and Forward Adjustments

Going back to the monthly trading test, I did take my data “for granted” as it works for most cases… but not really for this one. It does emphasise the point that “Trade what you Test and Test what you Trade” should also apply to data and rolling methodologies, which need to be checked/reviewed for each new system.
 
 

Following the previous post on backwardation, contango and crude oil, let’s look at how several factors can play a part in the overall trend following trading system performance.

I have previously referenced the study by EDHEC Risk, which shows the different sources of return of a Trend Following strategy.

The authors build the Mt. Lucas Management (MLM) index, which applies a 200-day moving average Trend Following strategy to a diversified range of 25 futures markets, rebalanced monthly. The performance of the index is separated in three periods and further broken down by the three individual sources of return identified by the authors:

1. T-Bill returns on marginable assets
2. Static returns from trend-following futures markets
3. Rebalancing gains

The chart below shows the representation of each component’s participation to the overall return:

You might find it surprising how relatively small the impact of the actual Trend Following strategy return is (I know I did…)

Further breakdown

The previous post highlighted that the performance of any Crude Oil futures contract was unable to match the performance of the spot price (despite a fairly high beta of 0.83). This is mostly because holding commodities via futures contracts expose you to the spot price beta move (as mostly anticipated) but also to the yield curve of that futures contract.

Over time, the impact of the backwardation or contango in the yield curve (roll yield) can create a large drift between the actual spot price performance and the equivalent futures contract.

This is very apparent in a simple “buy-and-hold” example (as in the Crude Oil case) but the same would apply to any trading strategy using futures instruments. Any futures trade could be broken down into two “components”:

• spot price beta move
• roll yield

And therefore the sum of all the trades resulting from a trading strategy (such as Trend Following) could fall into these two same categories, giving us now four overall components.

1. T-Bill returns on marginable assets
2. Returns from trend-following spot price beta moves
3. Roll yield
4. Rebalancing gains

Roll Yield: a misnomer?

Note that Roll yield is not aptly named; it implies that the act of rolling contracts yield a specific return. However, the return resulting from contango or backwardation is merely “crystallized” at the time of rolling contracts: the contango premium (backwardation discount) actually deflates over time gradually.

For example, think of buying a Crude Oil contract at the beginning of the month with a premium over spot of $1 per barrel (at $76). If the spot price does not change during the whole month, the price of the future contract simply converges to the spot price ($75) – the premium simply deflates. When selling the contract at the end of the month, you will have lost $1 without doing a contract roll and with the price having stayed stable.
 
 
Breaking down the actual components of a global strategy might be helpful in order to understanding the sources of returns and focus on a specific one to possibly improve it and enhance the overall system. For example, a more sophisticated approach to rolling contracts might enhance the overall roll yield produced by the trading strategy.

Petroleum has unfortunately been in the news lately. Nevertheless, Crude Oil performance last year gave us a good illustration of the impact that contango/backwardation can have.

Crude Oil – 2009

Crude Oil’s had a fantastic year in 2009. The spot price bottomed around 35 and topped 80 to finish on a near +100% performance

Many would assume that quick and easy way to double their money was to invest in Crude Oil in 2009 (assuming you could time the top and bottom perfectly). This is without counting the strong effect of contango that would have eaten into the return.

This can be illustrated by the fact that the USO ETF – supposed to reflect the performance, less expenses, of the spot price of West Texas Intermediate (WTI) light, sweet crude oil – did not manage to emulate the levels of performance seen in the Crude Oil spot price in 2009:

A mere +34% performance over 2009 pales in comparison with spot price performance

This is, of course, because the ETF managers invest in Crude Oil futures and are subject to the same contango, which eats into their returns.

Below is a chart of several prices for Crude Oil in 2009:

• The spot price is the headline price.
• The rolled contract price represents a back-adjusted contract, trading in the front-month contract and rolling over to the next contract before expiry.
• Two other single contracts, with different maturities are also plotted.

All prices are rebased to start 2009 at the same level as the spot price.

Whereas you cannot trade the spot price, you can trade using the 3 other contracts. Note how they all underperform the spot price performance: this is contango in action.

Most people call this a negative roll yield (long positions in a market in contango or short positions in a market in backwardation) because the drift is more apparent at time of rolling over to the new contract (which is priced dearer than the current contract), however the decay induced by the contango is gradual and erodes the price regularly – as can be seen in the further-dated contracts (the premium priced in the future-dated contract deflates gradually to zero at time of expiry).

The commodity yield curve is clearly an incidental impact on an overall trading strategy results, but it can also be used to form the basis of the strategy itself

Term Structure Trading Strategy

One such example of a strategy using term structure (aka. yield curve) as a trading signal is described in the paper
Tactical Allocation in Commodity Futures Markets: Combining Momentum and Term Structure Signals.

ABSTRACT: This paper examines the combined role of momentum and term structure signals for the design of profitable trading strategies in commodity futures markets. With significant annualized alphas of 10.14% and 12.66% respectively, the momentum and term structure strategies appear profitable when implemented individually. With an abnormal return of 21.02%, a novel double-sort strategy that exploits both momentum and term structure signals clearly outperforms the single-sort strategies.

The authors calculate the roll return for various instruments and select the most backwardated and contangoed markets. Only backwardated markets are allowed to be long and contangoed markets to be short. The increase in performance compared to a standard momentum strategy appears interesting.

Term Structure as a Strategy Filter

Another idea to explore is using term structure as a filter to a Trend Following strategy. Similarly to the concept explained above, one could prevent going short strongly backwardated contracts or long strongly contangoed contracts – basically avoid the markets where the odds are stacked against them.

I have not come across any such published test but found this paper:

Separating the Wheat from the Chaff: Backwardation as the Long-Term Driver of Commodity Futures Performance; Evidence from Soy, Corn and Wheat Futures from 1950 to 2004

It gives interesting fundamental insights as to why markets might be in contango or backwardation and studies the impact or prediction power of the roll yield on a passive long investment strategy. One of their conclusions is that yield return rates start having predictive power when considered on a long-term basis (multi-year) as opposed to monthly measurements.

Data - image by Pixelsior@flickr

In my other job, at a big investment bank, one of our main focus on a daily basis is DATA. Making sure that the hundreds of feeds and millions of records get uploaded correctly and contain the right information is key to a smooth, successful day.

There is surely not as much data to interact with in automatic/systematic trading but getting data right is equally vital to ensure quality trading or backtesting results.

Even with the best data package, there can be many pitfalls to look out for when trading or backtesting a systematic trading strategy. Below is a non-exhaustive list – please let me know in the comments if you think of anything to add.

Data Errors

Despite CSI being deemed one of the most reliable End-Of-Day Data providers, they still send you bogus data (albeit rarely) and you might want to run additional checks to see if the data is valid. Some ideas of checks to perform:

• Open, Low or Close above the High
• Prices = 0
• Price/Volume/Open Interest changes above tolerance
• Gap in the dates
• etc.

Of course, this will not catch all errors, but will give a pointer on large data anomalies.

Static Data

Back in the bank job, part of the system data which changes infrequently (hence static) is subject to 4-eyes approval (basically enforcing that another pair of eyes double-checks the correctness of the data). The idea is that, as this data drives the calculations in the system, any error would give flawed calculation results, and needs double-checking.

The equivalent in the automatic trading system world is the futures contract specifications and formats information (price quote type, decimal places, native currency, point value), contained in the FuturesInfo file in TradingBlox. Imagine an error of a factor 10 on the contract point value – this could have you get in a trade at 10 times the intended position size!

CSI appears to contain several of these errors (they seem to have less focus in getting this type of data right) and even official exchange websites have been seen to quote incorrect information. Reconciling both would be a starting point to getting the right information.

A possible solution to mitigate this problem (when starting trading a new, exotic instrument) is a quick in-and-out trade to double-check the actual information on your broker statement.

Data History and Overrides

The CSI data downloading process firstly retrieves all new prices to be stored in the Unfair Advantage database. It then applies potential corrections to past prices, in case errors were present. Rollover logic is applied and the full data history file is generated (to be used in the trading/backtesting software).

The change in data resulting from error corrections might change the outcome of past trade results. For example, a data correction in the recent past history might trigger a breakout and therefore a trade, now open. How would you handle this?

Additonally, if the historical data contains errors not corrected by your data provider, you might want to apply overrides to correct them yourself. However, since the price history gets re-written everyday, you need to have some form of automated overrides upload.

You might also want to make daily backups of price files to be able to compare when price corrections have taken place (and run your system with the exact data that was available on that day).

Other Data

Data such as historical FX/interest rates or Margin data are necessary to calculate some of the system statistics (including overall performance).

The FX rates are mostly used to convert the P&L from the native contract currency to the main account currency.

Interest rates would be used to calculate the interest earned on the account, which impact on the overall performance is not negligible for a long-term Trend Following system.

The Margin data is useful to determine the total amount of margin your system requires at any one time, with statistics such as the Margin-to-Equity ratio. Unfortunately, it is fairly hard to get hold of this type of data, which changes fairly often, at the discretion of the exchanges. Most backtesting platforms I have come across use a static margin number, which is fairly inaccurate.

To get more accurate results, you could run your backtesting simulations using historical (real or simulated) margin requirements. A method suggested on various forums to estimate historical margin requirements is to run a regression analysis between margin requirements (known history) and underlying instrument volatility (using different potential measurements such ATR, High-Low, Bollinger Bands, etc.). Finding a good-fit model would allow you to extrapolate past margin requirements based on volatility.

Holidays are also useful data to collect, to be able to know which exchanges are open when (note that exchanges can have different holidays even when they are in the same country).

Backup Provider

Your main data provider might be down or unavailable (hopefully on rare occasisons only). A backup, alternative source of market data might come in very handy for those special days.

Back-Adjustments and Rollovers

There is unfortunately no ideal or universal solution to roll and back-adjust contracts. As discussed earlier, a proportional back-adjustment allows for better representations of prices, keeping historical ratio values between all prices. However, backtesting systems usually require point-based back-adjustements in order to calculate the P&L of each trade correctly. You might need to maintain several sets of back-adjusted prices to be used in your simulations.

Furthermore, different instruments might require different rolling logic. Different delivery months in agricultural futures, for example, might relate to different crops. It might not make as much sense to roll from one ot the other in the same way that could be done with equity index futures.

Do you get the rollover information on rollover day?
Let’s assume that, on Thursday, you receive the data indicating that a roll should have occurred on Tuesday (ie. roll based on Open Interest or Volume shifts). Let’s also assume that an entry signal was triggered on the Wednesday and that the high volatility of the front contract would have prevented the system from entering the trade (risk management threshold breached) – whereas the next contract’s volatility, being much lower, would have allowed the trade. In real life, Wednesday’s trade would not take place (as the system does not know yet that the contract should have rolled on Tuesday). In backtested life however, Wednesday’s trade would take place, as the system would assume the roll has taken place on Tuesday. These things do actually happen!

You might want to test using the data as you would have received it, instead of a constructed back-adjusted series, potentially containing hindsight information.

LME Data

Some instruments deal in “strange fashion” and the London Metal Exchange is a perfect example of this. Trading on the LME is done by purchasing a forward contract with a given maturity (or prompt date). For instance, you might purchase a 3-month forward contract, which will run for the next 3 months gradually maturing to converge to the Cash price (at expiration), including the effect of contango or backwardation. However, the data quoted from CSI (and probably any other data provider) is a new daily quote for a new 3-month forward, which does not reflect real trading.

You would probably need to construct a custom price series with a bespoke algorithm to be able to reflect and backtest real-life trading. This topic was further discussed on a tradingblox forum post

Need for an ETL layer?

Drawing the parallel to the bank job again, where we use specialised tools/platforms to deal with data (Extract-Transform-Load = ETL); I believe that a similar layer between your data provider(s) and your trading/backtesting system, used to perform data checks, archive, roll and back-adjustments, etc. (all poinst discussed above) is necessary. It could also be used to add extra functionality such as creating spread time series to be traded/backtested as new instruments.

The data issue might not be as much of a trading problem as an IT one, nonetheless it is an important one that needs to be addressed – or it could impact your trading and backtesting results.

Regular readers might think that I suffer from backtesting-software-indecision-itis. Having first settled for TradersStudio, I then evaluated (and purchased) AmiBroker and found that it was 25 times faster than TradersStudio (at least for the calculation of the e-ratio). However, AmiBroker is not really geared towards true portfolio allocation testing with Futures and could not just replace TradersStudio – so I purchased it as a (cheap: $199) complement to it.

Why TradersStudio in the first place?

During my first round of evaluation, I did hesitate between TradersStudio and Trading Blox. At the time, it appeared that there was not a huge difference in functionality but a substantial price difference ($499 for TradersStudio, $3,000 for the full version of Trading Blox Builder). And this is how the choice was made.

However, I never really got past my TraderStudio’s first (not so good) impressions. Ultimately, I find the platform awkward to work with, documentation rather poor and the user community is very small.

So I decided to give Trading Blox another go and test their latest trial version (v3.3), and give you a teaser review of it.

Trading Blox: friendly, efficient, fast, professional

Trading Blox is vastly superior in terms of user interface “friendliness” and efficiency (love that script management/edit screen!). This makes the documentation lookup less necessary (and in any case, it is rather better).

The simulation runs are also very fast (300 stepped parameter tests under 7 minutes) and the software comes pre-packed with about a dozen ready-coded systems including the famous Turtle Trading system.

Finally, the numerous backtest options (slippage, commission, rollover slippage, volume, interest, etc.) should simulate trading reality much more closely.

Overall, it feels that the difference between Trading Blox and TradersStudio can be summed up by Pro vs. Amateur: both good at what they do, but playing in different leagues.

User community

Trading Blox forum is very good, with top-notch contributors. I joined it a few months ago and the discussion there is top-level, be it on all aspects of trading in general, backtesting, Data problems, Trading Blox questions, etc. It also has a marketplace where users can exchange code/systems, etc. This forum played a big part in my decision to re-consider Trading Blox (TradersStudio’s own forum and yahoo user group are barely ticking…).

In closing: screenshots

I feel I am about to give in and “cut my losses short” with TradersStudio and redeploy my resources towards Trading Blox. In the end, it is a “time vs dollar trade” and I feel that the initial outlay will give a great payback (in time savings and progress made). After all, automated trading is a business and one should not shy away from the essential investments.

Tomorrow, I should post a detailed system test using Trading Blox; but for now please find below some screenshots of the software (click to zoom in):

Result Reports

rich choice in general options

System Editor

Blox Editor

Heat map for stepped parameter

Getting started with the API

Fire up your favourite IDE/language that supports COM Interoperability (I am using Visual C# 2008 Express Edition), start Console Application project and add a reference to the Unfair Advantage API. For this you just need to “Add reference” and browse to the Unfair Advantage installation folder and select the main EXE (uad.exe). This gives access to the CSI UA API.

Add UA into your project and "Bob's your Uncle!"

Simple extraction code

The rest is fairly simple as long as you understand the RetrieveBackAdjustedContract2 parameters.
The code below takes 2 parameters: an input file name containing the list of markets to extract and a folder name to extract each continuous contract to. It sets specific UA session parameters and calls the RetrieveBackAdjustedContract2 with specific variables to obtain the concatenation desired. Each continuous contract is output to a folder in a .txt file.
The file containing the list of markets is of the following format (CSI number, Symbol, Market name):
…
8,CT,Cotton
…

The code

Please find the code there (very nicely formatted thanks to this great tool by manoli). You can also download the Program.cs file:

Note: You can receive a 10% discount on CSI data subscriptions (use coupon code LIBERTY)

No long-term continuity

Futures are specific in the way they trade in series of short-lived contracts that are only active for a few months (the most active contract for Silver at the moment is SIZ09 i.e. Dec 09 Silver 5,000 oz – see this post for Futures Symbology. Around mid-December most activity will roll-over to the March contract SIH10).
This is different to stocks which offer a continuous stream of prices (generally since start of trading). In order to back-test a system on historical futures data, you need to “stitch” the futures contract prices to generate a similar continuous stream of prices.

The $64,000 Question: How do you stitch the data?

Easy, you might say: just chain all the contracts one after the other. But there are always multiple contracts trading concurrently with different expiration dates (for Silver currently December 09, March 10, May 10, etc.). One accepted rule (in testing, but also in trading) is to always consider the front-month contract (nearest expiration date contract) and roll the price series into the next front-month contract around expiration time. That selection process determines the actual contract to consider for any given date. Now, it is “just” a matter of stitching them up together.

But…

As you might know, contango, backwardation and other factors (crop seasons, etc.) generate a difference in the price of different expiration date contracts. In effect, when comes the time to move from one contract to the next, there will most likely be a gap between the old contract price and the new contract price. These gaps can be substantial and make your data appear disjointed.

Cocoa March 2005 (yellow) and June 2005 (Green) contracts - a big price gap exists at the time of transition between the 2 contracts

The distorting “Panama canal” method

One of the workarounds (described by Ed Seykota in his Panama article) is to raise or lower each of the series one after the other so that each contract joins without a gap (similar to the boats going through the Panama canal). There are several problems with this “primitive” method:

• You introduce a trend bias (by always lowering or raising prices at rollover time, the impact over time will introduce a large drift. Far past data could even become negative)
• You are losing relative price difference (shifting all prices by an absolute amount has that effect: 10 to 11 is +10% but add 100 to both for an extreme case and 110 to 111 becomes less than +1%).

The March contract (in yellow) has been lowered to eliminate the gap with the the June contract. This distorts the relative price difference: the 25% move from 1480 to 1850 becomes a 27% move from 1380 to 1750.

Taking a leaf from the Equities book

The right approach seems to consider each contract roll-over in a similar fashion to a classic “stock-split” on Equities. When a stock splits, you effectively get X shares for every Y shares that you hold and the price is adjusted accordingly to reflect market capitalization. If a company performs a 10-for-1 split (meaning you get 10 new shares for every share that you hold), the price will effectively be divided by 10 after the split is effected – and historical data will normally be divided by 10 to represent the prices in the same “terms” as today (and therefore avoid showing a big price gap during the split). The historical data is “back-adjusted” by applying a proportional ratio related to the stock split (ratio=1/10 here).

Proportional back-adjustment splicing for Futures contracts

For Futures contract we can apply the same process of proportional back-adjustment at every contract roll-over. You would determine the adjustment ratio by dividing the price of the new contract by the price of the old contract. This ensures a constant relative (percentage-wise) relationship between any prices across the trading history.

The March contract has been lowered to join the newer contract with no gap. This has been done by multiplying the whole price series for that contract by 6% = 1850/1750 (prices before/after roll-over). This keeps the relative price difference: The 25% move highlighted stays a 25% move. You can see that the junction price is identical to the previous chart but the intermediate low circled in red has been raised (along with the whole series) to maintain the existing price ratio.

There might still be some concerns related to actual tick size vs. calculated tick size compared to the minimum tick size (a parameter you should probably take into account in your strategy) but proportional adjustments seems to be the best compromise because of the points highlighted above

There are many other ways to aggregate historical Futures data as well as other parameters to consider (nothing is ever so simple!) which I will address in a later post.

Credits: This issue first came to my attention and described in detail by Thomas Stridsman in his two interesting Trading Systems books:
Trading Systems that work and Trading Systems and Money Management
  

Note: You can receive a 10% discount on CSI data subscriptions (use coupon code LIBERTY)