While searching for robustness, you might come across the term of robust statistical estimator: the median, for instance, is a robust measure of central tendency, while the mean (average) is not (the latter is much more sensitive to outliers).
Robustness in trading is a tough beast to tame and understand. The more “robust” the research and development process, the better (read: robust) the results ought to be, right? With this in mind, I decided to test robust “tools” within the actual mechanical trading strategy itself.
The moving average indicator is so ubiquitous in trading that most folks (me included) use it without second thoughts. Its legacy probably dates from the era of expensive and complicated computing (it is relatively inexpensive to compute), so I wanted to revisit its hegemony – and give it a run for its money: by pitching it against a moving median indicator (on the basis of better statistical robustness for the latter).
Could it be that a moving median is actually a better indicator than the moving average?…
The experiment
To find out I used a basic and simple mechanical trading strategy: the Moving Average Crossover. This trading systems is always in the market, buys when the fast moving average crosses over the slow moving average and sells short when the fast average crosses under the slow average.
The second system would be a Moving Median Crossover. You guessed it: the same system, but replacing the average by the median.
The markets tested were a random collection of 17 Futures daily prices (proportionally backadjusted contracts) – all going back as far as CSI history goes (1920’s for Wheat!):
CSI Num.  Futures Contract  1st Date 

5  Pork Bellies (Floor+Electronic Combined)CME  20/09/1963 
25  Swiss FrancCME(Floor+Electronic Combined)  16/05/1972 
26  British PoundCME(Floor+Electronic Combined)  01/01/1970 
41  TBillU.S. 3 MthCME(Floor+Electronic Combined)  01/06/1976 
64  Canadian DollarCME(Floor+Electronic Combined)  16/05/1972 
65  Japanese YenCME(Floor+Electronic Combined)  16/05/1972 
150  TNoteU.S. 10 Yr w/Prj ACBT(Floor+Electronic Combined)  05/03/1982 
290  S&P 500CME(Floor+Electronic Combined)  01/03/1950 
412  CornCBT (Floor+Electronic Combined)  01/03/1949 
413  WheatCBT (Floor+Electronic Combined)  01/03/1922 
856  Crude OilLightNYMEX(Floor+Electronic Combined)  30/03/1983 
859  PlatinumNYMEX(Floor+Electronic Combined)  14/01/1964 
868  SilverCOMEX(Floor+Electronic Combined)  21/06/1963 
869  Natural GasHenry HubNYMEX(Floor+Electronic Combined)  03/04/1990 
1148  CocoaNYCE(Floor+Electronic Combined)  30/12/1965 
1150  Orange JuiceFrozenNYCE (Floor+Electronic Combined)  26/10/1966 
The money management for both systems is to trade each instrument in a separate independent subaccount, fully funded (i.e. no leverage used) with profit reinvested. All commissions or slippage are ignored.
The main interest of the experiment is the robustness of each indicator. To quantify this, each system is run over 9 combinations of parameters for the Golden Cross (fast indicator values: 45, 50 and 55 days; slow indicator values: 180, 200 and 220 days). A measure of the robustness of the indicator is the uniformity of the results over the 9 combinations.
The results
Below are the total returns for both systems over each parameter set:
Params (Slow/Fast) 
Average Sys.  Median Sys. 

180 / 45 
621.35%

391.33%

180 / 50 
682.41%

71.48%

180 / 55 
805.35%

304.81%

200 / 45 
849.72%

327.43%

200 / 50 
968.72%

216.40%

200 / 55 
1,506.61%

773.04%

220 / 45 
2,506.34%

1,156.11%

220 / 50 
3,207.70%

535.93%

220 / 55 
2,486.55%

577.39%

At first glance, it appears that the Moving Median indicator significantly underperforms the standard Moving Average indicator for these crossover systems.
Let’s look a bit deeper at the results with some basic statistical analysis:
Stats  Average Sys.  Median Sys. 

Mean Return 
1514.97%

483.77%

Std. Dev. 
970.08%

326.67%

Coefficient of Variation  0.64  0.68 
Median Return 
968.72%

391.33%

Median Absolute Deviation 
347.37%

174.93%

Coefficient of Variation  0.36  0.45 
The calculation confirms the underperformance of the Moving Median Crossover system. What about robustness you ask? Well, the Moving Median still scores worse than the Moving Average on both measures of uniformity/dispersion: the standard Coefficient of Variation (0.64 v 0.68) and its alternative cousin based on Median and Median Absolute Deviation (0.36 v 0.45): The Moving Average System produces more uniform (robust?) results!
Below is also a histogram of all 288 individual returns (per market per parameter combination, i.e. Wheat 180/45, Wheat 200/50, Silver 200/50, etc.):
There is clearly more blue presence on the left side of the chart and more red one on the right side…
A potential explanation
Using some inductive logic (warning: this might be dangerous when dealing with data from Extremistan^{*}), I started eyeballing the charts in search for some clues as to why the Median underperforms the Average. Below is an example of what I found:
The chart above only shows Moving Averages and Moving Medians (the prices have been removed to make the picture clearer). Average and Median seem to closely follow each other both on slow and fast sides. Indeed the bulk of the trades take place at roughly the same time (i.e. they both detect large trends fairly similarly).
However, if we zoom in over that redcircled congested area:
We can see that the Median Crossover system generates more signals than the Moving Average one (9 v 5). Trend following systems notoriously make big bucks in large moves but lose money in trendless, rangebound markets – like the one being zoomed into. If the Moving Median Crossover system is more active in these sort of markets it will generate more losing trades while capturing similar big winners to the Moving Average Crossover system.
Intuitively, it could be hypothesized that the Moving Average evolves in a smoother manner and will generate smoother curves with less erratic moves and consequently less losing trades during trendless markets – while the Moving Median does not generate a significant edge in detecting large trends.
One test is hardly enough to provide siginificant evidence, however this should give us some insights in the nature of the Moving Median indicator. The first insights being no increase in robustness and a drop in performance (when comparing total returns).
^{*}Extremistan: concept popularised by Nassim Taleb to describe the “province” where the total can be conceivably impacted by a single observation (e.g. financial data, wealth distribution). The opposite is Mediocristan: the province dominated by the mediocre, with few extreme successes or failures. No single observation can meaningfully affect the aggregate (e.g. human height and weight distribution). The bell curve is grounded in Mediocristan. There is a qualitative difference between Gaussians and scalable laws, much like gas and water.
For more info on Taleb’s terms, check his Fooled by Randomness glossary (PDF).
George // Jan 15, 2010 at 8:43 am
Hi Jez,
Great post! It’s neat to read about how you are dissecting the complex art of trading and figuring out what is really happening. I have never heard of the Moving Median before!
Here’s some random ideas.
1. Why use the same values for the averages and the means? If a 220/50 combination works best for the Moving Averages, then why couldn’t a different combination work best for the Moving Medians? Why couldn’t another set of 9 combinations be more robust than the ones that you chose for the Moving Averages?
2. I think the Moving Average is the oldest technical indicator, and it is easy to use and understand. That’s why it is so common. But whether it is the best indicator for our systems remains to be seen…I suspect it is more of a relic…
3. Your results are going to be skewed by high performing systems that make tons of money because they reinvest their profits. Even if you do this with actual trading, I think during the development it is better to use a fixed dollar position size. This allows us to understand what is happening without the distorting effect of exponential growth.
For example, the standard deviation of a set of systems that reinvest profits will always be much greater than systems that use a fixed position size. I think that it is easier to understand what is happening without this exaggerated effect.
4. I agree that the reason that the Moving Median systems did not perform as well is because of the signals generated that did not precede a movement in price.
Thanks for sharing.
Jez // Jan 15, 2010 at 10:47 am
@George
The main goal of this exercise was to “check my premises” that using robust statistical tool in the trading system rules would result in a more robust trading system in general – hence the idea of replacing the moving average by the moving median…
I take the points you made in your comments (and thanks for adding to the discussion)… However I wanted to compare likesforlikes with the only variable being the actual statistical measure of tendency calculation method (ie median vs mean) – hence the same parameters, and identical money management (without worrying whether it is the most optimal one for testing…).
Basically, the question I wanted to answer was: “Can we take a trading system (MA crossover), replace its indicator by a more statistical robust one (median) and obtain a more robust system?”
david varadi // Jan 15, 2010 at 12:20 pm
hi Jez, great post, and the median is a superior short term filter for spiky/noisy data. As a related test I demonstrated the superiority of the MMDI vs the MACD by using a median for the MMDI. However the concept there was to use the median for the short term and the moving average for the long term. In this case the crossover wasn’t significant but rather the net difference between the two—ie the zero line. You may wish to try this on the futures markets.
Keep up the great work.
best
david
Jez // Jan 15, 2010 at 3:52 pm
Thanks David!
Great idea regarding the MMDI. Especially since a concept i am considering is using a highertimeframe MACD filter to enter trend following trades – ie go with the major trend only. If MMDI can be better at filtering noise out, that sounds like a perfect improvement!
I’ll definitely give that a try. Do you have a link to a blog post of yours covering that by any chance?
david varadi // Jan 15, 2010 at 4:10 pm
hi jez the link is http://cssanalytics.wordpress.com/2009/08/06/meanmediandivergenceagreattrendindicatorpart1/ and it contains the formula and the indicator itself is available for free for tradestation on dvindicators.com
as a second note a multi time frame macd/mmdi may be useful to create a long/cash/short strategy whereby you enter in the direction of the long term using the short term indicator and you exit to cash with the short term indicator etc
keep up the good work…..the trend side of things is certainly underresearched
best
dv
Jez // Jan 15, 2010 at 4:16 pm
Great, Thanks! Will check that tomorrow
The MACD/MMDI filter on multiplee timeeframes is exactly the sort of things I have in mind…
Milktrader // Jan 18, 2010 at 9:35 pm
The most intriguing element for me is looking at things a little differently. The basic logic behind MA crossovers remains intact, but you’ve chosen to look at it differently. This time it didn’t work, but I’m convinced sooner or later it will. The trick is to balance imagination with craziness – I’m definitely working on that part.
Michael Fox // Jul 26, 2011 at 9:53 pm
Late to the party, but I thought I’d weigh in..
I see that you were a little uncertain what people mean by “robustness” in this context. Let me help.
Folks in signal processing like to use median filters. Consider for instance processing an image from a digital camera. We can take the data and use an MA filter, but this will just smooth out the image, making it blurry. Since the camera is digital the errors are pretty all or nothing. Dead pixels, as an example, can give black or white spots in the image. The image looks a lot better if you apply a median filter because this “salt and pepper” noise is replaced with median values from nearby pixels. This empirically looks good.
Robustness must always be in reference to some disturbance or uncertainty. One should not generalize to think of this as good for return or variance. Median is considered a robust estimator because it downplays the role of any specific data point, so a small set of erroneous or nonrepresentative data won’t skew results. Since you are using daily settle prices, these are already typically the average of the last few trades of the day. These “outliers” are thus very real and discarding them is essentially throwing away data.
Jez Liberty // Jul 27, 2011 at 2:57 am
Michael,
Thanks for dropping by and weighing in.
This is an old article indeed. I just reread it and I agree with your point that robustness does not mean low variance. I actually think David Druz said that robust systems tend to be volatile.
Properly testing for robustness would be evaluating the system under slight variations as you mention:
– Robustness to future prices (the survival aspect)
– Robustness to internal changes (i.e. variation in system parameters)
– Robustness to external changes (i.e. variation in price data)
(see article on types of robustness)
Ed // Aug 22, 2011 at 11:34 am
Anyone have results orn on the KST (Know Sure Thing) oscillator?
Jez Liberty // Aug 22, 2011 at 3:25 pm
Interesting – never heard of KST before and just googled it – although I dont think there is any “sure thing” in trading ;)
It is just an entry/exit signal though and it might be a good starting point, but only a small part of a complete system (I’m of the opinion that the entry/exit signal is not the most important part of a system).
Thanks for the suggestion though. I might decided to check it out in a backtest..
Andrew // Dec 18, 2011 at 4:02 am
Have you done any studies on linear regression vs. moving averages? – Thanks.
Jez Liberty // Dec 20, 2011 at 1:57 pm
Andrew – short answer: no.. But thanks for your contribution to the “idea box” ;)
Marvin // Jan 9, 2013 at 8:44 pm
It’s not astonishing that the average works better. Assume the market is going sideways and an upward trend is starting. it will take the half of the rolling window period that the median will change since the new values are less than 50% of the sample and therfore the median is chosen from the sideways market. Even the first values could get very large, the median doesn’t care about it.
Compare it to the mean. There each value has an influence. First it’s small but with the trend developing a mean system will react quite faster.
In short: with a median system the lagging of the trading system will be increased which explains to me the underperformance in the long run.
Jez Liberty // Jan 13, 2013 at 8:45 pm
Good point Marvin.