Systematic Trading research and development, with a flavour of Trend Following
Au.Tra.Sy blog – Automated trading System header image 2

Your worst Drawdown is yet to come

May 27th, 2010 · 4 Comments · Money Management

Looking over the drawdown cliffs

Looking over the drawdown cliffs

in this world nothing can be said to be certain, except death and taxes.

Arguing against Ben Franklin, I would say it is certain your (and my) Max Drawdown so far – whether in real trading or backtest – will be surpassed in the future. The maths even say that the probability of any drawdown in the future is 100% (although this might require a very long future, in a similar logic to the Infinite Monkey Theorem).

Dave Harding on Drawdown

I came across this paper (PDF download) from Winton Capital, co-authored by David Harding, which discusses the pros and cons of drawdown as a statistical measure, whether used to evaluate managers or trading systems.

The main points of the paper are:

  • Unlike volatility, drawdown represents a physical reality: the magnitude of loss that an investor could have suffered – this is probably why it is a popular statistic to evaluate systems and funds performance/risk.
  • Drawdown is not such a good indicator of quality for a system or manager – at least not as straightforward as usually assumed.

  • A couple of charts illustrate how expected maximum drawdown increases with volatility, track record and reporting frequency. On the other hand, drawdown decreases as mean return increases. Nothing too surprising.
  • Maximum drawdown is a a single number derived from a single string of data: it is going to have a large error associated with it. Any extrapolation of future performance will therefore be highly error-prone. Even with adjustments to equalise the volatility of track records, maximum drawdown is a poor statistic for making inferences about future reward/risk ratio or even future drawdown. Averaging worst drawdowns would be less error-prone, statistically speaking.

The paper concludes with:

Drawdown may have a role in manager risk control, but it should be used with caution, and should be calculated with reference to probability (95%, 99% confidence level) from the characteristics of the underlying process rather than purely from the historical track record.

Risk of Drawdown

A couple of volumes on my bookshelf discuss drawdowns and how to calculate their probability.

Balsara, in Money Management Strategies for Futures Traders, publishes tables of calculated risk of ruins based on different parameters.

However risk of ruin is different from risk of drawdown. Ruin is usually defined as a fixed capital level, representing a large percentage loss on initial capital. For example, a risk of ruin at 60% is the probability that your equity falls to 40% of your startuing capital. As the equity grows, the risk of hitting that “ruin threshold” decreases.

Risk of drawdown, on the other hand, stays constant regardless of how high the equity grows, because the drawdown “capital barrier” keeps moving up in line with the equity. Vince expands the concept of risk of ruin by modifying the calculation to derive the risk of drawdown (in The Handbook of Portfolio Mathematics)

Both risk of drawdown and risk of ruin increase as the track period or backtest length increases. However, the risk of drawdown tends to 100% as track period length increases, whereas the risk of ruin is bounded at a value determined by the characteristic of the trading system results (probability of win, payoff, trade risk, etc.).

Monte-Carlo simulation allows for estimating the risks of drawdown and ruin by iterating a random process governed by characteristics such as probability of win, payoff ratio, percentage of capital risked on each trade.

Risk of Ruin formula

Perry Kaufman, in New Trading Systems and Methods, presents a formula to calculate the risk of ruin. This is more convenient than having to run a Monte-Carlo simulation but it does not allow for calculating a risk of drawdown.

The formula is as follows:

RiskRuinFormula

Calculate your risk of Drawdown/Ruin

Below is a calculator that implements risk of ruin or risk of drawdown calculations based on the two methods described above (the risk of ruin is calculated from both a Monte-Carlo simulation and from the formula).

Prob. Win:
Win/Loss ratio:
Risk Amt (%):
Number of periods:    
Risk of:
Loss level (%):
iteration:    0/1000
Risk (MC):
Risk of Ruin (formula):

Just fill in the stats of the trading system, the test length and the level of drawdown/ruin to be tested and hit the Calculate button. Note that both calculated values can diverge significantly (as in the pre-populated example) if the number of periods is relatively low.

In case you find this tool useful, it has been added under the resources page. Disclaimer: I have not done full-proof 100% testing on it but playing with it seems to give good results.

Note that this method is probably not ideal as it only considers the average trade statistics and simulates sequential trading (whereas real-life systems will have multiple trades on the go at the same time). Also importantly, it completely ignores any time-dependence in the stream of results (such as auto-correlation, etc.).

The Trading Blox approach

Trading Blox runs Monte-Carlo simulations for any system backtested and produces some useful charts and information from it.

The Monte-Carlo simulation is different as it is applied to the daily equity curve returns – which is probably more realistic (although it still removes the time-dependence of the return stream).

One of the standard charts produced analyses drawdowns and attempts to give confidence levels as discussed in the Harding paper:

MonteCarloDrawdownGraph_P1

Related Posts with Thumbnails

Tags: ····

4 Comments so far ↓

  • cordura21

    Hey Jez. Trading Blox has a parameter called “Sample Grouping Days” that does the resampling in group of days instead of single periods. This allows you to keep somehow the time dependence of the return stream. It is a fixed number of days, so if you’re doing short period examinations, it can give odd results.
    Cheers, Cord
    PS: I am considering getting Amibroker, how are you finding it? Still use it?

  • Jez

    Thanks Cordura – I did not know about that Sample Grouping Days parameter. I was thinking of that concept though but how would you know what grouping you need to apply? Maybe returns are correlated in some way on a yearly basis (ie 2009 was bad for trend followers after a good 2008) or monthly basis or even weekly basis.
    I’ll try it out though…

    Another metrics mentioned in the Harding paper and that I have seen employed at other CTAs is measuring things such as average of 10 worst 5-day losses. You could probably calculate this for different durations or also calculate probability of 10%, 20% monthly losses.

    On AmiBroker, I still think its a good software and very good value, however now that I am up-to-speed with Trading Blox I have not used it for a while.

  • popon

    Hi;

    what is the fill logic in Trading Blox? Possibility of partial fill in backtesting. Say using 1-minute data, the fill chance depends on order size and bar volume.

    Thank you.

  • Jez Liberty

    The standard fill logic is all or nothing with added slippage configuration (x% of high-order price). There is also an option to decide what minimum volume level is necessary for trading.

Leave a Comment