Robustness in trading is a tough beast to tame and understand. The more “robust” the research and development process, the better (read: robust) the results ought to be, right? With this in mind, I decided to test robust “tools” within the actual mechanical trading strategy itself.

The moving average indicator is so ubiquitous in trading that most folks (me included) use it without second thoughts. Its legacy probably dates from the era of expensive and complicated computing (it is relatively inexpensive to compute), so I wanted to revisit its hegemony – and give it a run for its money: by pitching it against a moving median indicator (on the basis of better statistical robustness for the latter).

Could it be that a moving median is actually a better indicator than the moving average?…

The experiment

To find out I used a basic and simple mechanical trading strategy: the Moving Average Crossover. This trading systems is always in the market, buys when the fast moving average crosses over the slow moving average and sells short when the fast average crosses under the slow average.

The second system would be a Moving Median Crossover. You guessed it: the same system, but replacing the average by the median.

The markets tested were a random collection of 17 Futures daily prices (proportionally back-adjusted contracts) – all going back as far as CSI history goes (1920′s for Wheat!):

 
The money management for both systems is to trade each instrument in a separate independent sub-account, fully funded (i.e. no leverage used) with profit re-invested. All commissions or slippage are ignored.

The main interest of the experiment is the robustness of each indicator. To quantify this, each system is run over 9 combinations of parameters for the Golden Cross (fast indicator values: 45, 50 and 55 days; slow indicator values: 180, 200 and 220 days). A measure of the robustness of the indicator is the uniformity of the results over the 9 combinations.

The results

Below are the total returns for both systems over each parameter set:

 
At first glance, it appears that the Moving Median indicator significantly under-performs the standard Moving Average indicator for these crossover systems.

Let’s look a bit deeper at the results with some basic statistical analysis:

 
The calculation confirms the under-performance of the Moving Median Crossover system. What about robustness you ask? Well, the Moving Median still scores worse than the Moving Average on both measures of uniformity/dispersion: the standard Coefficient of Variation (0.64 v 0.68) and its alternative cousin based on Median and Median Absolute Deviation (0.36 v 0.45): The Moving Average System produces more uniform (robust?) results!

Below is also a histogram of all 288 individual returns (per market per parameter combination, i.e. Wheat 180/45, Wheat 200/50, Silver 200/50, etc.):

There is clearly more blue presence on the left side of the chart and more red one on the right side…

A potential explanation

Using some inductive logic (warning: this might be dangerous when dealing with data from Extremistan^*), I started eye-balling the charts in search for some clues as to why the Median under-performs the Average. Below is an example of what I found:

Crossovers for Cocoa 2004-2010

The chart above only shows Moving Averages and Moving Medians (the prices have been removed to make the picture clearer). Average and Median seem to closely follow each other both on slow and fast sides. Indeed the bulk of the trades take place at roughly the same time (i.e. they both detect large trends fairly similarly).

However, if we zoom in over that red-circled congested area:

Zoomed-in portion of Cocoa chart

We can see that the Median Crossover system generates more signals than the Moving Average one (9 v 5). Trend following systems notoriously make big bucks in large moves but lose money in trend-less, range-bound markets – like the one being zoomed into. If the Moving Median Crossover system is more active in these sort of markets it will generate more losing trades while capturing similar big winners to the Moving Average Crossover system.

Intuitively, it could be hypothesized that the Moving Average evolves in a smoother manner and will generate smoother curves with less erratic moves and consequently less losing trades during trend-less markets – while the Moving Median does not generate a significant edge in detecting large trends.

One test is hardly enough to provide siginificant evidence, however this should give us some insights in the nature of the Moving Median indicator. The first insights being no increase in robustness and a drop in performance (when comparing total returns).

 
 
^*Extremistan: concept popularised by Nassim Taleb to describe the “province” where the total can be conceivably impacted by a single observation (e.g. financial data, wealth distribution). The opposite is Mediocristan: the province dominated by the mediocre, with few extreme successes or failures. No single observation can meaningfully affect the aggregate (e.g. human height and weight distribution). The bell curve is grounded in Mediocristan. There is a qualitative difference between Gaussians and scalable laws, much like gas and water.

For more info on Taleb’s terms, check his Fooled by Randomness glossary (PDF).

In the last few posts I have been exploring the e-ratio and wrote the code in TradersStudio. I also tried to implement it in AmiBroker AFL language mostly to check how it performed speed-wise – as I found TradersStudio quite slow.

While testing AmiBorker, it actually felt so fast that I decided to perform a more formal speed comparison test.

Conditions of the test

I exported the CSI data via the API for a proportionally back-adjusted Corn contract going back to 1949. The data was imported in both AmiBroker and TradersStudio via their ASCII Import.

The system tested is a simple 20-day Donchian Channel Breakout (Buy-only) and the ATR used to normalise the MAE/MFE is 20-day also.
The optimiser in both systems were used to generate the e-ratio for 50 51 different trade durations (from 10 to 50 days).

One of the condition to validate the results of the test was that the trades generated by both systems are similar (to double-check I did not make a coding mistake resulting in simpler/faster process for one of the platforms).

The computer I was running the test on is a quad-core CPU (2.4 GHz) with 3.25 GB of addressable RAM. Only one platform was running at the time it was tested.

Results are in!

And they look astoninglishly good for AmiBroker:

• The TradersStudio test was run first and took 4 mins 15 sec to complete and produce the custom report (which needs to be manipulated in Excel to calculate the e-ratio). Furthermore, it appeared that every incremental run in the optimisation process took slightly longer than the previous one.
• The AmiBroker test ran in (…drumrolls…) 10 sec! And the e-ratio was readily available from the results grid:
  
  

  AmiBroker optimisation results showing the e-ratio. Click to expand

Both apps maxed out their allocated CPU (i.e. overall CPU usage of 25% = a quarter of the quad-core CPU available).

I then decided to run the AmiBroker code over 5 markets to get a feel of how long it would take. I added Crude Oil (going back to 1985), Cotton (1968), Gold (1975) and Yen (1972).
AmiBroker ran the same code + optimisation in 2 mins 20 sec, twice as fast as one market in TradersStudio!

Applying a simple proportional calculation to derive the time it would take in TradersStudio for the same dataset would give us a completion time of 1 hour (59 min 30 sec exactly…). Since the performance seems to degrade over the course of the optimisation in TradersStudio (i.e. the first back-test is quicker than the last one of the optimisation run) we could assume that it would actually take longer, which is what I experienced when I was testing on similar data in the last few weeks.

So here you have it: AmiBroker is 25 times faster than TradersStudio and it gives you the results in a much more friendly format.

Trade Reconciliation

As mentioned earlier, to ensure that the test was valid, the set of trades generated by both systems should be compared. I ran one back-test of the system over the Corn data for a single optimisation step (e-ratio for 20 days duration) in both systems.

I could find that 296 trades out of 340 presented a very good match (some matches had one day difference on the Entry date and rounding differences on the prices). But overall the trade comparison was good enough to give assurance that the same systems and results were being tested.

I’ll keep investigating for myself where these errors are coming from (and post a further code update if warranted) but we can assume that the differences are trivial with regards to performance comparison.

Did I mention that AmiBroker is 3 times as cheap with a much wider following (many more code samples, better docs, more forums, etc.)?…