To explore this further, let’s study the animal world with the Cheetah and the Cockroach – bearing in mind how they would compare to trading systems.

The Animal Metaphor

A Cheetahs is a highly specialised and elegant predator. It is a fascinating animal, the fastest on this Earth. However, despite this status, its survival is in jeopardy…
A cheetah mostly hunts gazelles (ideal size/speed ratio) in vast expanses of land (so as to use their speed). Changes in their environment (types of prey, habitat, terrain, competition from other predators) makes them lose their advantage (edge) and they struggle to hunt anything and survive.

Whereas cockroaches suffer from a vile and dirty reputation, they are very tough little critters, being able to survive in a vast array of unhospitable conditions with barely any food (they are rumoured to be able to survive a nuclear war – although that seems to be an urban myth).

Back to Trading Systems

You will have guessed it: the Cockroach “trading system” is more likely to be robust than the Cheetah “trading system”. Today it might not be pretty compared to the Cheetah, but the Cockroach can adpat to changing conditions. In 100,000 years, it will still probably be around unlike Cheetahs.

What this means is that you might back-test a system that looks very exciting (a Cheetah – maybe a curve-fitted backtested system) but chances are: it will not survive very long, unlike a trading system that might look a “bit dull” in comparison but offers great robustness (the Cockroach).

How do you know if your trading system is robust

As we could see robustness in a trading system means that changing the conditions under which the system runs should not have significant impact on its performance. In practice this means that by changing either of:

• Date range: all the returns should not come from a specific period
• Type of markets: if it works on Grains and Metals, it should also work on Interest Rates and Currencies
• Parameters of the trading decision tools (i.e. indicators): if a MA of 20 is chosen, a MA of 18, 19, 21 or 22 for example should not alter system performance significantly

the characteristics of the trading system should stay similar

To illustrate this point in the real world, listen to Michael Clarke’s (a Trend Following Wizard) description of his trading programs:

CCM’s philosophy when developing a model is to test it using a large pool of commodity interests, approximately 105, with data as far back as 1945. In order for the model to be accepted into CCM’s portfolio of available models, it must trade all 105 markets using the same rules and parameters and the results should indicate excellent performance characteristics for the vast majority (at least 90%) of the markets and for the group of markets as a whole. Also no model is accepted unless it shows stability of performance during tests involved with shifting parameters and altering rules. Much effort has been expended in developing tools to assist in this effort to assure robustness of the models. I consider the software we have developed in this area to be one of our edges in the markets.

In closing

I hope this metaphor helps you thrive for robustness: Don’t try to build a Cheetah, better build a Cockroach! It will have better chances of survival – even though it does not look too pretty.

I believe a trading system robustness is pivotal to its success; and will carry on with a series of posts covering this topic: How to measure robustness? How to make your system robust? Can walk-forward be robust?

PS: Just as I was getting ready to publish this post, I saw this post from Mebane Faber relaying a study from BlackStar funds showing that long-time survivor CTAs all exhibit a Sharpe ratio below 1 (not so attractive) whereas all Sharpe ratios above 1 “disappear” after time. Quite a useful chart proving the point! All the cockroaches seem to out-live the cheetahs…