Stepping back from the story of the cockroach and the cheetah, it seems that robustness is all about survival. For an automated trading system, this means the system “survives and thrives” after you decide to put it “live”.
However there are different ways to look at robustness in a mechanical trading system.
Different definitions for robustness
Trading system robustness usually implies that the system exhibits similar performance when subject to slight variations. However thinking about it, you realise that this can mean several things:
- Robustness to future prices (the survival aspect)
- Robustness to internal changes (i.e. variation in system parameters)
- Robustness to external changes (i.e. variation in price data)
- Robustness in system design
- Robustness in system testing
Also check that Trading Blox forum post for more discussion on robustness (some of which directly inspired ideas in this post).
There are 2 assumptions that can be made:
The first: a system robust to changes (internal and external) will likely be robust to future prices: this is because markets always evolve and changes – and your system should be able to cope with these changes.
The second one is that creating a system using robust concepts and procedures, both in design and testing, should ensure that the system will show robustness.
Robust system design
This generally starts with a trading strategy that makes sense, with few parameters, not many bells and whistles (we do not want the Ferrari that looks good and go fast round the track but which can not go over speed bumps or underground car parks… Well, I do really…but you get the point! ;-)
Another aspect to consider when aiming to build a robust trading system is illustrated by this quote from Bill Eckhardt who aims to use robust tools and components in his systems:
We developed all our systems exclusively in-house so there aren’t any sort of publicly recognizable indicators I could mention. We definitely use non-linear systems and non-linear indicators. Linear indicators, such as filters with moving averages, have been mined dry.
In statistics, the median is a more robust tool than the mean/average (it is les subject to underlying data changes such as outliers). Possibly, using median instead of average in a trading strategy is more robust. A moving “median” crossover system would show more robustness than using a classic moving average crossover system – this is something I am planning to test out.
Robust system testing
The main aspect of robust system testing is to ensure that the back-test is realistic and that no over-fitting takes place. We will not go too much in detail as these have been well documented (I recommend Perry Kaufman’s book which has a whole chapter on System Testing and robustness).
Important points are good quality data, in-sample vs. out-of-sample data, realistic assumptions (costs, slippage, etc.), logic in the strategy.
Robustness to internal changes
This is when you change the parameters of the trading system. It is probably fairly easy to test and measure. Assume you have a Donchian Channel Breakout system with Channel length at 20 days and ATR-based stop at 30 days with a multiplier of 2.
A robust system would exhibit very similar performance with slightly different parameters: the robustness could be quantified by measuring the overall difference/standard deviation in system’s performance when varying parameters (for example in a range of +/- 10%), e.g. Donchian Channel Breakout system with Channel length at 22 days and ATR-based stop at 33 days with a multiplier of 1.8, etc.
Robustness to external changes
Here, this is mainly the “parameters” of the price data that can be amended to test for robustness. There are quite a few things that can be changed such as the set of instruments traded (small permutations should not affect performance drastically), testing period (system performs similarly over all time periods), or even random slight amendments in actual prices.
Robustness to future prices
Unfortunately future price data is pretty hard to come by (CSI are pretty good but they only provide historical data! Feel free to email me if you have found a good provider ;-), so you obviously cannot test your system for future price robustness before putting it live.
Arguably, the aim of focusing on robustness with the four previous points is to ensure that the system performs equally well on future prices. However these four robustness aspects in a trading system can only act as a proxy for future prices robustness.
Although we, as trading system developers, like to gain some sort of “comforting” certainty, we have to keep in mind that the future will be unpredictable, markets will change, and therefore designing a system to trade future prices will always contain some degree of uncertainty. It is up to us to decide (and predict) what parameters and proxies are important to ensure robustness in the future.
This might sound like a paradox, but designing an automated trading system involves more discretion and prediction than one would think.