Arguing against Ben Franklin, I would say it is certain your (and my) Max Drawdown so far – whether in real trading or backtest – will be surpassed in the future. The maths even say that the probability of any drawdown in the future is 100% (although this might require a very long future, in a similar logic to the Infinite Monkey Theorem).
Dave Harding on Drawdown
I came across this paper (PDF download) from Winton Capital, co-authored by David Harding, which discusses the pros and cons of drawdown as a statistical measure, whether used to evaluate managers or trading systems.
The main points of the paper are:
- Unlike volatility, drawdown represents a physical reality: the magnitude of loss that an investor could have suffered – this is probably why it is a popular statistic to evaluate systems and funds performance/risk.
- Drawdown is not such a good indicator of quality for a system or manager – at least not as straightforward as usually assumed.
- A couple of charts illustrate how expected maximum drawdown increases with volatility, track record and reporting frequency. On the other hand, drawdown decreases as mean return increases. Nothing too surprising.
- Maximum drawdown is a a single number derived from a single string of data: it is going to have a large error associated with it. Any extrapolation of future performance will therefore be highly error-prone. Even with adjustments to equalise the volatility of track records, maximum drawdown is a poor statistic for making inferences about future reward/risk ratio or even future drawdown. Averaging worst drawdowns would be less error-prone, statistically speaking.
The paper concludes with:
Drawdown may have a role in manager risk control, but it should be used with caution, and should be calculated with reference to probability (95%, 99% confidence level) from the characteristics of the underlying process rather than purely from the historical track record.
Risk of Drawdown
A couple of volumes on my bookshelf discuss drawdowns and how to calculate their probability.
Balsara, in Money Management Strategies for Futures Traders, publishes tables of calculated risk of ruins based on different parameters.
However risk of ruin is different from risk of drawdown. Ruin is usually defined as a fixed capital level, representing a large percentage loss on initial capital. For example, a risk of ruin at 60% is the probability that your equity falls to 40% of your startuing capital. As the equity grows, the risk of hitting that “ruin threshold” decreases.
Risk of drawdown, on the other hand, stays constant regardless of how high the equity grows, because the drawdown “capital barrier” keeps moving up in line with the equity. Vince expands the concept of risk of ruin by modifying the calculation to derive the risk of drawdown (in The Handbook of Portfolio Mathematics)
Both risk of drawdown and risk of ruin increase as the track period or backtest length increases. However, the risk of drawdown tends to 100% as track period length increases, whereas the risk of ruin is bounded at a value determined by the characteristic of the trading system results (probability of win, payoff, trade risk, etc.).
Monte-Carlo simulation allows for estimating the risks of drawdown and ruin by iterating a random process governed by characteristics such as probability of win, payoff ratio, percentage of capital risked on each trade.
Risk of Ruin formula
Perry Kaufman, in New Trading Systems and Methods, presents a formula to calculate the risk of ruin. This is more convenient than having to run a Monte-Carlo simulation but it does not allow for calculating a risk of drawdown.
The formula is as follows:
Calculate your risk of Drawdown/Ruin
Below is a calculator that implements risk of ruin or risk of drawdown calculations based on the two methods described above (the risk of ruin is calculated from both a Monte-Carlo simulation and from the formula).
Just fill in the stats of the trading system, the test length and the level of drawdown/ruin to be tested and hit the Calculate button. Note that both calculated values can diverge significantly (as in the pre-populated example) if the number of periods is relatively low.
In case you find this tool useful, it has been added under the resources page. Disclaimer: I have not done full-proof 100% testing on it but playing with it seems to give good results.
Note that this method is probably not ideal as it only considers the average trade statistics and simulates sequential trading (whereas real-life systems will have multiple trades on the go at the same time). Also importantly, it completely ignores any time-dependence in the stream of results (such as auto-correlation, etc.).
The Trading Blox approach
Trading Blox runs Monte-Carlo simulations for any system backtested and produces some useful charts and information from it.
The Monte-Carlo simulation is different as it is applied to the daily equity curve returns – which is probably more realistic (although it still removes the time-dependence of the return stream).
One of the standard charts produced analyses drawdowns and attempts to give confidence levels as discussed in the Harding paper: