Welcome to my online repository of research and insights on automated trading system developmentAnd how to avoid easy common mistakes when choosing which data to use to back-test a trading strategy on Futures… This is a “long-ish” post but I believe essential for good-practice back-testing.
No long-term continuity
Futures are specific in the way they trade in series of short-lived contracts that are only active for a few months (the most active contract for Silver at the moment is SIZ09 i.e. Dec 09 Silver 5,000 oz – see this post for Futures Symbology. Around mid-December most activity will roll-over to the March contract SIH10).
This is different to stocks which offer a continuous stream of prices (generally since start of trading). In order to back-test a system on historical futures data, you need to “stitch” the futures contract prices to generate a similar continuous stream of prices.
The $64,000 Question: How do you stitch the data?
Easy, you might say: just chain all the contracts one after the other. But there are always multiple contracts trading concurrently with different expiration dates (for Silver currently December 09, March 10, May 10, etc.). One accepted rule (in testing, but also in trading) is to always consider the front-month contract (nearest expiration date contract) and roll the price series into the next front-month contract around expiration time. That selection process determines the actual contract to consider for any given date. Now, it is “just” a matter of stitching them up together.
But…
As you might know, contango, backwardation and other factors (crop seasons, etc.) generate a difference in the price of different expiration date contracts. In effect, when comes the time to move from one contract to the next, there will most likely be a gap between the old contract price and the new contract price. These gaps can be substantial and make your data appear disjointed.
Cocoa March 2005 (yellow) and June 2005 (Green) contracts - a big price gap exists at the time of transition between the 2 contracts
The distorting “Panama canal” method
One of the workarounds (described by Ed Seykota in his Panama article) is to raise or lower each of the series one after the other so that each contract joins without a gap (similar to the boats going through the Panama canal). There are several problems with this “primitive” method:
- You introduce a trend bias (by always lowering or raising prices at rollover time, the impact over time will introduce a large drift. Far past data could even become negative)
- You are losing relative price difference (shifting all prices by an absolute amount has that effect: 10 to 11 is +10% but add 100 to both for an extreme case and 110 to 111 becomes less than +1%).
The March contract (in yellow) has been lowered to eliminate the gap with the the June contract. This distorts the relative price difference: the 25% move from 1480 to 1850 becomes a 27% move from 1380 to 1750.
Taking a leaf from the Equities book
The right approach seems to consider each contract roll-over in a similar fashion to a classic “stock-split” on Equities. When a stock splits, you effectively get X shares for every Y shares that you hold and the price is adjusted accordingly to reflect market capitalization. If a company performs a 10-for-1 split (meaning you get 10 new shares for every share that you hold), the price will effectively be divided by 10 after the split is effected – and historical data will normally be divided by 10 to represent the prices in the same “terms” as today (and therefore avoid showing a big price gap during the split). The historical data is “back-adjusted” by applying a proportional ratio related to the stock split (ratio=1/10 here).
Proportional back-adjustment splicing for Futures contracts
For Futures contract we can apply the same process of proportional back-adjustment at every contract roll-over. You would determine the adjustment ratio by dividing the price of the new contract by the price of the old contract. This ensures a constant relative (percentage-wise) relationship between any prices across the trading history.
The March contract has been lowered to join the newer contract with no gap. This has been done by multiplying the whole price series for that contract by 6% = 1850/1750 (prices before/after roll-over). This keeps the relative price difference: The 25% move highlighted stays a 25% move. You can see that the junction price is identical to the previous chart but the intermediate low circled in red has been raised (along with the whole series) to maintain the existing price ratio.
There might still be some concerns related to actual tick size vs. calculated tick size compared to the minimum tick size (a parameter you should probably take into account in your strategy) but proportional adjustments seems to be the best compromise because of the points highlighted above
There are many other ways to aggregate historical Futures data as well as other parameters to consider (nothing is ever so simple!) which I will address in a later post.
Credits: This issue first came to my attention and described in detail by Thomas Stridsman in his two interesting Trading Systems books:
Trading Systems that work and Trading Systems and Money Management
 |
Like this post? You may want to read these:
Jez,
Great blog so far! I’m in about the same development phase as you are with my trend following system. I have UA and am trying to find the best contract backwardization method for system testing.
Thanks for the site!
-Mike
Mike – Thanks I’ll try to keep it rolling.
What back-testing software do you use? (I have Traders Studio but still need to warm up to it…)
I’ll probably make a later post about the Unfair Advantage settings and what my set up/options would be for continuous contracts. Would be good to compare notes.
-Jez
Jez–
Good to see your blog and the posts are excellent.
Keep up the great work!
–Matt
Thanks Matt, I enjoy your blog too!
This is a good post that illustrates a variety of clever mechanisms for massaging data for easy backtesting of futures strats.
That said, for my money a much better approach is to leave the futures data alone and smarten up your strategies so they don’t look to trade a specific contract but instead trade the current most active contract. This avoids all the data mangling issues you mention as well as providing a strategy that can be deployed directly into the market. It also enables you to backtest strategies which are trading the curve instead of just the most active or front contract.
For me, any approach that involves mangling your source data should be looked at with skepticism as it will inevitably introduce distortions to your backtested results.
Tito,
I agree with what you are saying.
This is only a compromise and will not beat any “straight clean data” back-testing. Its disadvantage is potential distortions (although – as per my examples – the idea is to minimise them) but its advantage is that it produces a format much easier to process. For example I don’t know of any back-testing systems that handles individual contract files for strategy testing. So you have to do something similar if you want to use these testing platforms – which speed up the testing process from my point of view.
The way I envisage the testing process is a 3-step one:
- research, test, optimise and choose strats in a rapid manner in a back-testing platform (like in TradersStudio or TradeStation)
- Validate the elected system(s) by doing a “more serious” testing on raw contracts data (i.e. applying skepticism to results found above) by coding up the strategy from scratch and not using the back-testing platform.
- Implement/deploy the code from step 2 in live if the results are satisfactory.
That way, the distortions would only affect initial systems testing/selection but the decision of putting a system “live” is still based on raw data back-tests.
Excellent blog! I’m working on these same issues. The problem with proportional ratio adjusting is that the resulting P&L will not be correct. When you apply percent based adjustments the point moves are corrupted. You would have to keep track of all the adjustments you made and apply those to the “point value” of the contract expressed as a time series. CSI does not do this.
Thanks Eric!
I agree, I dont think it is easy to find an “ideal” transformation – however it makes back-testing in most software packages much easier – and I am not convinced the actual results are so different to using the raw data. Obviously you have the issues that you highlight in your comment but it shoud still allow to test some characteristics of a system – what back-testing platform do you use by the way?
Ultimately I would like to implement my own Futures contract concatenation program so that I would have control over all the variables I need (including adjusting the point value, also storing actual rollover dates to take rollover costs+slippage into consideration, etc.).
One thing that CSI does not seem to provide either is historical “market characteristics” (ie meta-data such as min Tick move, margin requirements, etc.). I have been unlucky so far to even locate a source for this data. I’d be glad to hear if you have? They would obviously need to be adjusted similarly to the point value.
I would be interested to hear on what method you have settled to concatenate your Futures data (if you have elected one)?
As of today I only use a set of continuous Futures data as it is more practical. But in the future – and before I go “live” with a trading strategy I fully intend to back-test the version on pure raw data (se my comment just above in response to tito’s comment). When I come to that stage I’ll post an update on how the results compare between raw vs concatenated data (actually that could be an idea for a future post – before I find that “killer” trading strategy ;-)
I suspect there is much valuable data lost and unnecessary risk taken by transforming the data. In many commodity markets the liquidity is spread out across contracts and different contracts represent different supply/demand realities. For example, rolling out of the fall crop into the spring crop due to higher open interest and/or volume makes no logical sense to me. They are two different products that can be moving in different directions and/or can have different degrees of hedger (premium payer) participation. The fact that the spring contract is trading more volume than the fall contract doesn’t convince me that I’m now best served abandoning the fall contract and NOW starting to trade the spring. All it means is that there is plenty of liquidity in the spring contract, implying that there has probably been sufficient liquidity to trade it for some time. There is no rule that says you can’t trade both at the same time. Imagine such logic applied to stocks in the software sector. I’ve been trading Oracle and ignoring Microsoft, but now that Microsoft has more volume I’m done looking at Oracle and will start trading only Microsoft? Maybe as a market maker, but as a trend follower? Same with natural gas; front month contracts can skyrocket while back months decline. The very act of rolling locks you into a mutually exclusive decision that is a function of liquidity AFTER it has transacted. People rarely question whether the decision SHOULD be mutually exclusive. A roll methodology requires one selection from the following two options (re: frontMonth/nextMonth): Yes/No; No/Yes. Not using a roll methodology offers the following four options: Yes/No; No/Yes; Yes/Yes; No/No. I understand that simple is preferable, but most people do not consider the risks and opportunity costs resulting from assumptions.
The logic supporting abnormal returns in the futures markets argues against rolling on volume or open interest. Trend followers are supposed to be providing insurance to hedgers by purchasing rising markets and selling falling markets (trading against hedgers). But if you wait until AFTER the open interest or volume is so high in a contract that it exceeds that of any other contract you are choosing to avoid most of the hedging activity (since that’s what produced the open interest and volume). If you wait until the hedgers are already in (already hedged) to participate, all that’s left is the beta of the market. That’s close to just trading the spot market. Not saying that can’t work…but it should be considered for what it is.
I’ve tried manually concatenating contracts using various different methods (OI, Vol, time, delta of each, etc.) As expected, the results are different.
At this point I’m inclined to use un-adjusted data, if only to be intellectually honest. This is a problem for most backtesting software programs. But most backtesting software programs don’t produce realistic results anyway. TradingBlox, Mechanica, TradersStudio, PowerST are a few that are worthy. It is inconvenient to use individual contracts. But in this business you do not want to be a payer of the “convenience premium”…because it tends to be very high.
I use PowerST http://www.powerst.com
Thanks for a great and insightful comment Eric!
I have to agree that I tend to think of concatenating contracts for markets where it seems to make more sense (ie where there is no/less seasonality – like gold for example – although it could well exhibit serious backwardation in the future, similarly to how oil was showing serious contango at the beginning of the year). As you highlight, agriculturals with different seasonal crops is even worse (I like your comparison with trading different stocks).
I see what you are saying with trading something close to the spot market – which is basically what I have in mind when generating a continuous contract – but as you rightly point out, it is true that we might be missing some opportunities in the other contracts.
In any case thanks again for taking the time to post your views – which will definitely benefit my reflections on that matter (and hopefully so for other readers)
I’ll definitely check PowerST – thanks.