

Arguing against Ben Franklin, I would say it is certain your (and my) Max Drawdown so far – whether in real trading or backtest – will be surpassed in the future. The maths even say that the probability of any drawdown in the future is 100% (although this might require a very long future, in a similar logic to the Infinite Monkey Theorem).
Dave Harding on Drawdown
I came across this paper (PDF download) from Winton Capital, coauthored by David Harding, which discusses the pros and cons of drawdown as a statistical measure, whether used to evaluate managers or trading systems.
The main points of the paper are:
 Unlike volatility, drawdown represents a physical reality: the magnitude of loss that an investor could have suffered – this is probably why it is a popular statistic to evaluate systems and funds performance/risk.
 Drawdown is not such a good indicator of quality for a system or manager – at least not as straightforward as usually assumed.
 A couple of charts illustrate how expected maximum drawdown increases with volatility, track record and reporting frequency. On the other hand, drawdown decreases as mean return increases. Nothing too surprising.
 Maximum drawdown is a a single number derived from a single string of data: it is going to have a large error associated with it. Any extrapolation of future performance will therefore be highly errorprone. Even with adjustments to equalise the volatility of track records, maximum drawdown is a poor statistic for making inferences about future reward/risk ratio or even future drawdown. Averaging worst drawdowns would be less errorprone, statistically speaking.
The paper concludes with:
Drawdown may have a role in manager risk control, but it should be used with caution, and should be calculated with reference to probability (95%, 99% confidence level) from the characteristics of the underlying process rather than purely from the historical track record.
Risk of Drawdown
A couple of volumes on my bookshelf discuss drawdowns and how to calculate their probability.
Balsara, in Money Management Strategies for Futures Traders, publishes tables of calculated risk of ruins based on different parameters.
However risk of ruin is different from risk of drawdown. Ruin is usually defined as a fixed capital level, representing a large percentage loss on initial capital. For example, a risk of ruin at 60% is the probability that your equity falls to 40% of your startuing capital. As the equity grows, the risk of hitting that “ruin threshold” decreases.
Risk of drawdown, on the other hand, stays constant regardless of how high the equity grows, because the drawdown “capital barrier” keeps moving up in line with the equity. Vince expands the concept of risk of ruin by modifying the calculation to derive the risk of drawdown (in The Handbook of Portfolio Mathematics)
Both risk of drawdown and risk of ruin increase as the track period or backtest length increases. However, the risk of drawdown tends to 100% as track period length increases, whereas the risk of ruin is bounded at a value determined by the characteristic of the trading system results (probability of win, payoff, trade risk, etc.).
MonteCarlo simulation allows for estimating the risks of drawdown and ruin by iterating a random process governed by characteristics such as probability of win, payoff ratio, percentage of capital risked on each trade.
Risk of Ruin formula
Perry Kaufman, in New Trading Systems and Methods, presents a formula to calculate the risk of ruin. This is more convenient than having to run a MonteCarlo simulation but it does not allow for calculating a risk of drawdown.
The formula is as follows:
Calculate your risk of Drawdown/Ruin
Below is a calculator that implements risk of ruin or risk of drawdown calculations based on the two methods described above (the risk of ruin is calculated from both a MonteCarlo simulation and from the formula).
Just fill in the stats of the trading system, the test length and the level of drawdown/ruin to be tested and hit the Calculate button. Note that both calculated values can diverge significantly (as in the prepopulated example) if the number of periods is relatively low.
In case you find this tool useful, it has been added under the resources page. Disclaimer: I have not done fullproof 100% testing on it but playing with it seems to give good results.
Note that this method is probably not ideal as it only considers the average trade statistics and simulates sequential trading (whereas reallife systems will have multiple trades on the go at the same time). Also importantly, it completely ignores any timedependence in the stream of results (such as autocorrelation, etc.).
The Trading Blox approach
Trading Blox runs MonteCarlo simulations for any system backtested and produces some useful charts and information from it.
The MonteCarlo simulation is different as it is applied to the daily equity curve returns – which is probably more realistic (although it still removes the timedependence of the return stream).
One of the standard charts produced analyses drawdowns and attempts to give confidence levels as discussed in the Harding paper:
cordura21 // May 27, 2010 at 6:26 am
Hey Jez. Trading Blox has a parameter called “Sample Grouping Days” that does the resampling in group of days instead of single periods. This allows you to keep somehow the time dependence of the return stream. It is a fixed number of days, so if you’re doing short period examinations, it can give odd results.
Cheers, Cord
PS: I am considering getting Amibroker, how are you finding it? Still use it?
Jez // May 27, 2010 at 9:16 am
Thanks Cordura – I did not know about that Sample Grouping Days parameter. I was thinking of that concept though but how would you know what grouping you need to apply? Maybe returns are correlated in some way on a yearly basis (ie 2009 was bad for trend followers after a good 2008) or monthly basis or even weekly basis.
I’ll try it out though…
Another metrics mentioned in the Harding paper and that I have seen employed at other CTAs is measuring things such as average of 10 worst 5day losses. You could probably calculate this for different durations or also calculate probability of 10%, 20% monthly losses.
On AmiBroker, I still think its a good software and very good value, however now that I am uptospeed with Trading Blox I have not used it for a while.
popon // Nov 20, 2010 at 9:53 am
Hi;
what is the fill logic in Trading Blox? Possibility of partial fill in backtesting. Say using 1minute data, the fill chance depends on order size and bar volume.
Thank you.
Jez Liberty // Nov 21, 2010 at 7:48 pm
The standard fill logic is all or nothing with added slippage configuration (x% of highorder price). There is also an option to decide what minimum volume level is necessary for trading.