Systematic Trading research and development, with a flavour of Trend Following
Au.Tra.Sy blog – Automated trading System header image 2

The Good, the Bad and the Ugly Portfolios

November 18th, 2010 · 13 Comments · Backtest

Many combinations...

Many combinations...

Following the last study on diversification using a “portfolio randomizer”, I wanted to further explore further the concept of hindsight bias in portfolio selection and how it can impact performance. This short post is an illustration of that concept.

51 Instruments, 40 Picks, How Many Combinations?

The answer is… a lot: there are 47,626,016,970 ways of picking a portfolio of 40 instruments from a total set of 51 (formula is: 51! / [40! x 11!]). Running so many simulations was not practical (!) so I decided to run a fraction of the possible combination (5,000 random iterations). We get a similar set of results as the last study:


This is still for a run of the 20/50 Moving Average cross-over system running from 1990.
Note that the results are slightly different from the run with 40 instruments from the last post, as the position size was greater here (and therefore the “cloud” of CAGR/MaxDD results has moved towards the top-right).

The average performance figures for the systems are as follows:

Performance Stats
Max DD
MAR 0.68
Sharpe Ratio 0.59
Trade Number 3629

Isolating portfolios

The next step was to isolate the worst and best cases (circled on the scatter plot) and identify the difference in portfolio composition between the two. Since this is a combination of 40 elements from a set of 51, there can only be 11 differences at a maximum between each portfolio.

Indeed, the “worst” and “best”portfolios share 32 of their 40 components but still exhibit a wide difference in their performance figures (the MAR ratio for example is 3.5 times better for the best case: 1.02 v. 0.29). For reference, the “good part” of the best portfolio (ie its instruments that do not overlap with the other portfolio) are:

  • Cattle-Feeder-CME(Floor+Electronic Combined)
  • CopperHG-COMEX(Floor Trading Only)
  • Euro(Floor+Electronic Combined)-CME
  • Gas Oil(Combined)-ICE(IPE)
  • IBEX 35 Index-MEFF
  • Natural Gas-Henry Hub-NYMEX(Floor+Electronic Combined)
  • OMX Helsinki 25-EUREX
  • Palm Oil-Crude-MDEX

On the other hand, the “bad part” of the portfolio is:

  • Azuki Beans-Red-TGE
  • Cocoa-CSCE
  • DJ Euro STOXX 50 Index-EUREX
  • Gold-COMEX(Floor Trading Only)
  • MSCI Taiwan Index-SGX(SIMEX)
  • Silver-COMEX(Floor Trading Only)
  • Soybeans-CBT(Floor Trading Only)

To take the comparison to a further extreme, I ran the exact same system over both bad and good portfolios by themselves.

The difference in results is fairly striking:

Good portfolio

Good portfolio

Bad portfolio

Bad portfolio

Going from a CAGR / MaxDD combination of 26.73% / 34.9% to -3.20% / -63.3% is fairly drastic.
This is not a big surprise, knowing that 20/20 hindsight was used in picking both sets of instruments. But bearing in mind that the results above come from the exact same system and parameters, using the same number of instruments, it does illustrate the point even more clearly on portfolio selection and how they can impact performance results.

Robustness Testing

If anything, I believe this illustrates that testing over several portfolio sets might be a good way to identify robustness in system results. If the system only shows good results on specific portfolios, it might simply be a “lucky” outlier.

On Diversification

Finally, there were a few comments on the last post with regards to how to implement diversification. I had only focused on diversification from a portfolio point of view. However I believe that ideally one would diversify with a large set of instruments to trade, different systems covering different timeframes.

One problem is that diversification on all these levels bring on an increase in required starting capital (one likely reason why most Trend Following Wizards have a minimum managed account size in the millions).

So you might have to make a choice in how to apply diversification.

As diversification is really beneficial thanks to the non-correlation it brings, diversifying across different systems could also be a good idea, as systems can be more or less “engineered” to be un-correlated to each other (ie a Trend Following system with a Mean Reversion system).
It is also possible to “trade” a large portfolio without taking all the signals (by using filters or an overall risk/size limiter).
After all, this is something that the Turtles used to do (when they were at full position size they had to skip signals)
Some ideas to think about…
Credits: Thanks to Trading Blox forum member sluggo for the reminder of how the Turtles used to skip trades, and how filtering trades could enable small accounts to trade a large portfolio – on this thread.

Related Posts with Thumbnails


13 Comments so far ↓

  • Jez Liberty

    Hey Josh,

    I have had a quick look at the links and they look very interesting – and definitely more sophisticated than my tests above. I haven’t had the chance to check them in detail as I am travelling (again!) but will do when I come back
    I remember reading on your blog that Patrick Burns had started blogging. I didn’t know of him and did not check his blog out but will definitely do. 

    Thanks for these links!

  • Lawrence

    Cool, How hard would be to implement a picker?
    Rather than running a raw random selection, pick the instrument that is most likely to succeed based on the current information gathered so far.

    Does the simulation picking an instrument because it was just signaled?

    Does the simulation actually reflect how one would trade?

  • Michael Harris

    Hi Jez,

    Another interesting post!

    Can you share a few details about the system you used? Was that a stop and reverse 20/50 SMA crossover?

    Also, where you wrote that “… I ran the exact same system over both bad and good portfolios by themselves.”

    did you mean the whole portfolios or just the good and bad parts?

  • Dave Redpath

    Hi Jez, have you had a look at any criteria other than the end profit to optimise the porfolio selection? I did some work on ensemble classifiers years back and there are a whole bunch or criteria for measuring diversity. I was using the kappa-error plot to pick a pareto optimal set.

  • Donald

    Your math is off for your total number of possible baskets.

    51 (51! / [40! * 11!])=2428926865470


  • Paro

    Interesting! About the data:

    # Cattle-Feeder-CME(Floor+Electronic Combined)
    # CopperHG-COMEX(Floor Trading Only)

    Shouldn’t the settle prices of “Floor” and “Combined” be the same?

    And you equally weighted the 40 picks?

    With respect to the system: do you have an explanation on why does the system perform well in 2008 but not in May/June 2010?

  • Jez Liberty

    I am not sure the approach of walk-forward testing on portfolio selection can add much value as I believe different markets will evolve in random cycles of over/under-performance but this would probably be worth testing.

    I always remember that interview from Bill Dunn where he mentioned that he had dropped the whole grains sector from the portfolio just before they took off…

    Trying to change the portfolio kind of assumes that you can predict what is going to perform better.

    Not sure about your 2 other questions but the system(s) each had a static list of instruments and each signal was taken as they arose. But because this is a 2-phase system all instruments had a position on. The simulation should be close to how one would trade (ie signal generated off close and order executed at next open).

  • Jez Liberty

    Hi Michael,
    You can check the State of Trend Following on this blog report where there is more info on the system but it basically is a simple MA cross-over system (reversal: ie exit and sell short when short MA crosses long MA).

    I meant only the good and bad parts (the whole portfolios were run in the first simulation – you can see them circled in the scatter plot.

  • Jez Liberty

    @Dave – so far I havent looked at anything to optimise the portfolio selection – I am not sure how robust this would be but need to give it a try (see other comment on Dunn’s portfolio change). Never heard of kappa-error plot and pareto optimal set I am fraid…

  • Joshua Ulrich

    @Donald: you misread. Jez has calculated the number of combinations correctly (you’re not supposed to multiply the quantity by 51).

  • Jez Liberty

    @Donald – sorry the text was not really clear:
    “there are 47,626,016,970 ways of picking a portfolio of 40 instruments from a total set of 51 (51! / [40! x 11!])”
    The bracket is not a maths bracket and just meant to contain the formula – the first 51 is part of the sentence…
    Anyway I updated it for sake of clarity, but the same number still stands.

  • Jez Liberty

    Hi Paro,
    To be honest, I am not sure it makes a lot of difference to the system results (although something else I should add to my list of things to test…). I actually picked different types for different instruments to add a bit of random diversification – but there’s no real strong reason behind it.
    Re: your specific question, not really, usually the electronic market closes later at which point the price might have moved substantially.

    All instruments are equally weighted as x% of total equity when opening a new position.

    I believe these types of systems go through phases of under/over performance (which is usually a sign of robustness) and this is just such a case. Note that this was similar for the Trend Following Wizards…

Leave a Comment