Systematic Trading research and development, with a flavour of Trend Following
Au.Tra.Sy blog – Automated trading System header image 2

Further Musings on Randomness

March 23rd, 2011 · 31 Comments · Backtest

The last post on “Monkey-Style” trading with random entries and trailing stop exits generated a fair bit of interest, comments and questions. In this follow-up post, I want to explore the concept further and address some of the points raised in the discussions resulting from that study.

Before I get started, I want to clarify that I am not advocating the use of random trading in real-life system design. I simply used random logic, as per David Harding’s quote, to remove any impact from the entry signals and solely focus on exit and money management.

The Other Side of The Coin

David Harding’s quote (“If you put in stops and run your profits and trade randomly you make money; and if you put in targets and no stops, and you trade randomly you lose money. So the old saw about cutting losses and running profits has some truth to it.”) was really two-sided. The previous post looked into the first part of the quote with random entries and trailing stops. Let’s now look at the other part of the quote with random entries and profit targets.

The overall concept of the system is identical to the one tested in the last post: entries in random directions every time a position is closed, with a fixed fractional money management system over a portfolio of diversified futures. The only difference has to do with exits, which are now triggered by a volatility-adjusted take-profit order. At the time of trade entry, the ATR is calculated as a measure of volatility and a limit order is entered at a distance equal to a multiple of the ATR value.

As the test contains a random element, I ran several hundreds of them in order to average them to detect a “central tendency”. The ATR-multiple also stepped through different values from 2 to 10, each of these being run 200 times to generate 1,800 tests in total.

Well, David Harding seems to be proven right again as the average performance of these 1,800 runs generated a near 85% loss over the 20 years of the back-test (and this is excluding commissions – more on that later).

I have run other tests and generated one chart comparing all outcomes. The “Random Entries with Target Profits” equity curve is plotted on that chart further below.

Note that the Win ratio for this system with Profit Targets comes close to 95%. This is logical: all winning trades are exited when they hit their target profit but losing trades are never exited (except at the end of the test); this shows that focusing on single stats like this one can be misleading (how many times do you hear market gurus claiming 90% accuracy rate?).

More Randomness

I decided to “hire another monkey” to help with another instance of a random system: random entries and random exits.

My first monkey is still in charge of “tossing a coin” to decide of the direction of each new trade, but instead of having a deterministic exit, the second monkey decides randomly of the duration of each trade: monkey picks 69 and the system holds the position for 69 bars. I did assist the monkey a bit, by giving him some “hindsight bias”, so that the average trade duration was roughly equal to the trade duration of the systems in the “Random Entries, Trailing Stops” scenario (for example 13 days for 2xATR, 63 days for 5xATR or 220 days for 10xATR).

This is a very random system, which can go either short or long, and we would therefore expect for it to perform neutrally when tested without any trade frictions. And this is exactly what happens. Check the chart further below: the equity curve is a near flat-line. This goes some way to prove that averaging the results of 1,800 runs with random trading logic does generate a result close to the theoretical/expected result. And we can clearly see the impact of a trailing stop exit (positive) and a profit target exit (negative) compared to a full random system.

Retiring the Entries Monkey

Some readers did comment on the last randomness post, pointing out that entries were as important as exits and such good results could also be obtained with random exits and “good entries”. This was my next test: using a standard Moving Average cross-over for entry generation, I left my “exit monkey” in charge of exiting the trades after a random period of time (again, with the help of hindsight, so that the average trade duration for this random system was similar to the equivalent MA cross-over system). Note that the system only entered on the actual cross-over of moving averages (i.e. when a position exits, the system waits for the next cross-over instead of re-entering in the direction of the moving averages).

The system only returned 64% over 20 years, with relatively high volatility, not really displaying more than what could be described as “random oscillations”. Replacing random entries with MA cross-over entries can not conclusively be said to have improved the system, as the signal-to-noise ratio is too high to assert whether the positive performance of MA entries is significant or not.

MA Entries with Profit Targets

Another way to compare random and MA cross-over entries would be to take the “random entries with profit targets” system and replace the entry generation by a moving average cross-over. This is the next (and final) test in this post.

The MA Entries with Profit Target Exits system variation performs very similarly to the one with random entries, even performing slightly worse, by losing more than 90% over the time span of the back-test (20 years). Again not a strong evidence for the superiority of MA entries over random entries.

Putting it All Together

Below is the chart of each of the 5 scenarios tested, exhibiting a wide variation in results obtained.


 

System CAGR Max DD
Random Entries / Trailing Stops
16.14%
23.12%
Random Entries / Random Exits
-0.26%
10.49%
Random Entries / Profit Targets
-7.94%
89.03%
MA Entries / Random Exits
2.36%
59.76%
MA Entries / Profit Targets
-10.46%
93.05%

 
Some pretty clear trends in there. Note that the Drawdown figures are calculated on a monthly basis. Drawdowns based on daily performance would therefore be larger, as mentioned in this post, discussing a short Drawdown paper by David Harding.

Of course, it is not possible to generalise the results of these findings. The entry logic chosen for these tests for example is specific and some other entry logic, with a different average trade duration might have yielded different results.

This was indeed the case in some previous tests discussed on the Trading Blox forum (thanks to Pumpernickel for pointing these threads out in the last post comment), where similar discussions/tests took place. The results found there seemed to indicate profitability for Trend Following entries and random exits (with a slightly different logic) and even some random entry/exit combination (different logic from above again: direction of positions were not random). Check these two threads for more info and results.

Note on averaging and result normalization

Because most of the tests involved some random logic, I deemed it necessary to run a fairly large sample for each case in order to average the results and obtain an overall tendency.
Some readers have pointed out that averaging the results hides the distribution of the outcomes and might hide the fact some the average performance is skewed by some outliers.

These are valid points to raise, which I want to address by displaying the shape of the distribution of CAGR for all instances of the “Random Entries with Trailing Stops” system (with a 2-ATR stop).

There is of course some variation around the average (average CAGR = 17.03% with a standard deviation of 4.96%), but I feel the average is still a good indicator for the central tendency of the system concepts being tested here. I actually decided to check this by estimating the statistical significance of this result by using the bootstrap method. With the null hypothesis that the average return is zero, the p-value for obtaining an average of 17.03% is actually 0.00 (100,000 resamples).

The other aspect of this comparison test is the normalization of results: every family of tests was normalized so that the standard deviation of the monthly returns was identical for all scenarios, allowing for a better, apples-for-apples comparison.

Notes on Slippage and Commissions

I usually do not include trade friction on the tests published on the blog, one of the reasons being to avoid making assumptions on commissions and slippage amounts, which are variable for every trader based on size, broker, market, etc. (and which can definitely have a non-negligible impact as previously discussed in here and here).
So, I usually prefer to leave this out of the equation (at the risk of making the “raw” results less realistic) and let the readers interpret the results based on their friction costs assumptions. Moreover, we are not really testing a system here, we are using a system back-test to check the impact of entries and exits.

Nevertheless, I have re-run the “Random Entries with Trailing Stops” system including commissions ($10 per contract) and slippage (5%). Of course, these extra costs have an impact: the CAGR drops from 16.14% to 10.16%, as can be seen in the chart below:

And here are some additional figures that should help estimate trading costs:

System Round-Turns per Million Average Trade Duration
Random Entries / Trailing Stops
877
138

 
I hope this (long) post has clarified some points/questions raised from the last “randomness” post. If not, we can always continue the discussion in the comments section…

 

Picture credits: stuckincustoms via flickr (CC)
Related Posts with Thumbnails

Tags:

31 Comments so far ↓

  • oliver

    I get a feeling that your personal biases play a roll in the systems you have developed and the results that you have precented? I do not know if this is the case or not, but we tend to see what we want to see. Other than that good research.

  • Trader

    What about MA Entries, Trailing Stops?

  • Jez Liberty

    @Oliver: to some extent this is true, as I am mostly focusing on Trend Following systems. Also, as mentioned the entry criteria selected might influence the results. Possibly some other entry criteria (shorter-term Trend Following or even other strategy type might have produced different results.. But I cannot test it all! ;-)
    However, I believe the tests simply try to verify David Harding quote and extend on the concept by making other areas of the system random. I usually try to avoid pre-conceived ideas before going into a test, but to some extent it might be impossible to completely block out all cognitive (often unconscious) biases: for example, our thinking is probably affected in some way by all the history of market technical studies published for the last 100 years. On the other hand, it helps avoid “re-inventing the wheel”… A fine balance to find. For this post, I had decided of the methodology and tests to run before running/analysing the tests, so I feel the results are fairly impartial.
    What sort of biases do you think might affect the systems/results presented here?

    @Trader: “MA Entries, Trailing Stops” would probably have a profile very similar to a standard Trend Following system. In a way, the classic MA cross-over system can be thought of as a “cousin” to “MA entries, trailing stops” as the MAs will follow the price up or down during the position and exit the trade when prices turn back (this is not a hard stop though, so big gaps against the trends would take longer to get stopped out).

  • Nicolas

    Very, very interesting research. Thanks.
    After reading your post, I feel that “random entries / profit targets” system is so bad because you don’t cut your loses at all, never. But realistically, a trader would go for a target profit always protecting with a trailing stop. What would be the result in this case? Would the profit target decrease the ending results or increase it?
    I see so many buy/sell recommendations with profit targets that go against my “running profits” philosophy.
    Then one question is left for me: what about “random entries / profit target / trailing stops”?
    It would be great to see the result in a third post ;-)

  • Jing

    Hi, Jez
    I think the random entry/trailing stop result looks a little too good. It has a MAR 0.71 in 20 years. Could you show some traditional trend systems (MA-cross over, channel breakout, volatility breakout) results and compare them with random entry/trailing stop? I guess these standard trend systems (MA-crossover, channel breakout) may not work much better than random entry/trailing stop. Even some parameter set of MA-crossover and channel breakout may have a worse MAR. If that is the case, I think random entries are not bad at all, in some cases/periods, random entries may be better than some specific entry rules.

  • Baltazar

    Hi Jez,

    I think the random entry has a drawback, it makes the outcome of the stratefy random even given an history.
    May I suggest using a deterministic entry system instead: enter long and when you exit reenter with opposite direction.

    Sure it is not a smart system given that it doesn’t depend on the market conditions to make it’s decision.
    But it takes teh randomness out and you won’t face the critics that “in order to acheive the result you present (average return) you would need to run simultinously a large number of random systems”

    If such system worked it would certainly validate harding comment: smart exit are more important than entry (stupid or random)

  • Baltazar

    Someone pointed out to me something quite interesting:

    Assume you trade this strategy with 1k monkeys and bet 1 every time.
    -at the beginning you will have 500 worth of bet up and 500 of down. So total exposure is 0

    -if it goes in one direction, say up, half the monkey are going to be stoped and 50% of these will bet up. This happens after you break your clever stop loss point.
    your exposure is now 500 (the other half cancels each other).

    -as it continues going up you will be eventually betting 1k to the upside. and as your stop trails, they will indeed merge into one big stop.

    You though you were using smart stops/random entre but in fact you are using a “smart” enter point to bet half your wealth and then scale up.

  • Rick

    Jez,

    What is the account required as per your simulation? Maybe the random entry/ATR exit system is just an indication that commodity markets trend but in reality trading any of these commodities separately may not work in general.

    For example, would it be easy for you to say for which commodities this system worked and for which it didn’t?

    Also, from the distribution I assume there were no negative trials. Is that right? What would happen in the case of just one commodity? Can you show results for just one commodity contract?

    Thanks-

  • Jez Liberty

    @Nicolas: this is a valid point. I think the test shows that random entries + trailing stops has a similar profile to a “classic” Trend Following system, so your question is probably equivalent to asking what happens when adding a profit target to a Trend Following system. I have done a similar study before, which you might want to check out:
    http://www.automated-trading-system.com/profit-targets-trend-following/

    @Jing: Something I forgot to mention (I updated the post now…) is that the drawdown figures are calculated on monthly performance reporting, which understates the “real” figures that would be based on daily numbers. So the MAR would normally be smaller. For comparison with other Trend Following systems (bearing that DD difference in mind), I would suggest that you browse the archives as I have covered quite a few of these systems in the past. Nevertheless, I believe the study shows that random entries and trailing stops give you a Trend Following profile. Not sure if these entries are better than classic Trend Following entries though. Check this old post on the e-ratio: the TF entry actually had an overall positive expectancy (which we would not expect from random entries):
    http://www.automated-trading-system.com/e-ratio-trading-edge/

    @Baltazaar: The outcome is random and this is why a couple of hundred tests were run: to detect the overall tendency, which was the goal of the post. The goal of the post was not to design a system that was reproducible (I think most traders would not want to trade random systems!…). I see what you mean with the 1000 monkeys example and maybe this “synthetic scaling in” helps smoothen the equity curve. However, consider this: all of the 1000 monkeys taken individually would still have positive results (as per the distribution plot of the individual tests in the post), showing that there is still a strong impact from random entries and “smart stops” despite the averaging/scaling in.

    @Rick: As mentioned above, the goal of the post was not to test a practical a system so I set up the test so that the actual account equity/volume of futures traded were not an issue: once again purely theoretical, just to be able to “check the odds” of trading that way. Very good point about individual/portfolio of markets. I believe diversification is key for a Trend Following system to work: some markets will have lacklustre performance for years, while other ones will perform very well, with the overall result positive. I have run some tests in the past with individual markets and results were not great at all for some markets (depending on periods tested).
    Actually, I wrote a post showing the impact of diversification by adding more in more markets. It does not cover single-market tests but should give you a good idea of the impact of adding more/less markets to a portfolio:
    http://www.automated-trading-system.com/trading-diversification-free-lunch/
    For one market, I would expect the results to be much lower,with some being positive (unless you happen to pick a great trending market for the period tested). You’re right no negative trials: the lower cagr was 6.44%

  • Michael Harris

    Great work!

    My comment is that I would not go as far as declaring exits more important than entries from these results. Actually, your results show in my opinion that in the test period considered diversification in commodity markets outperformed S&P 500 returns significantly but on a non-risk adjusted basis.

    The reason these results do not prove that exits are more important than entries is because they represent, as you also pointed out, a central tendency of a market using past data and random entries. If you apply the same model to a market with shorter trends you will probably get dismal results.

    It would be very interesting to see how this method applies to other groups, like the Dow-30 stocks. I have a feeling your results are good only for commodities only.

    One question: since your model traded commodities only did you account for locked limit up/down markets? I have an example in one of my books with the Lumber futures back in 1993, if I recall correctly. Back then, if someone placed a short trade just before a long period of locked limit up movements it would result in a huge loss that would not show in backtesting results if these conditions were not properly accounted for.

  • Baltazar

    I get that the system is random and each monkey you show made money, hence the distribution.

    What I say is the limit of such system is in fact not random anymore and you don’t need to really trade thousands of such systems to reach that limit.

    Your system converges toward a deterministic system with scaling in.
    Such deterministic system doesn’t have random entry but increases the position every time you break your 2ATR limit. So you just show that simple breakout systems worked on equities.

    You also show that suboptimal systems (with random entries) also produce positive results.

    As MH points out this is because the underlying market is trending (maybe even artificially as the per the lock market comment).

  • Jez Liberty

    @Michael: Thanks. I suppose everybody can read into the results differently.
    With regards to markets traded, the portfolio did not only contain commodities, but also currencies and rates (check here for the full list: http://autrasy.wpengine.netdna-cdn.com/wp-content/uploads/2011/03/Instruments.html ).
    I only included markets from my State of TF report which had history going back to 1990, and this did not contain equity indices. However, we could speculate that since most Trend Following Wizards trade equity indexes, adding these could produce similar results.
    With regards to individual stocks, not many funds I know of apply Trend Following directly on individual stocks. Blackstar do though, and their paper is worth a read:

    re: lock limit markets, the back-testing software I use (Trading Blox) does handle them by preventing trading on lock-limit days (and hence entering short before a series of locked-limit up would create a big loss indeed).
    Although, from what I understand (never done it myself) you can “hedge yourself out” of a locked-limit situation with options:
    http://www.automated-trading-system.com/get-out-locked-limit-situation/

  • Michael Harris

    Jez, Thanks for the link to the paper. Very interesting work.

    It would be very interesting if you could list the average return for each of the futures contracts in the list. Also, it would be even more interesting to me, if you could apply the system to just one contract, let us say to BP, and display a return distribution for a number of trials.

    Is it possible that the returns can be attributed only to a few specific contracts, which happened to exhibit very long trends? In that case, this would be a result of survivorship bias.

    As you can see, I am very skeptical about this method. I believe there is a catch some place and it is hidden in the averaging of many contract and many random trials.

  • RiskCog

    Interesting posts. One positive thing about trading systems with some randomness is that this would seem to make it less likely to see prolonged draw downs. It would just be improbable to randomly loose very many trades in a row.

    I think many traders do actually take random entries on their trades (and not just because their predictive models don’t work well). For example if you are trading in an options pit and trying to earn a spread on each trade then you are basically buying and selling randomly based on incoming orders. As long as you don’t have an unlucky streak of taking the “other side” for a bunch of folks who ARE able to predict the market, then your stops will tend to keep you solvent.

  • Alvantage

    Hi Jez,

    Another good post. I think your conclusion is still valid, i.e. the limiting loss and let profit run is a robust concept, especially in futures markets (which are known to be trending).

    However, I still have issues with your drawdown statistics as I wrote here http://www.automated-trading-system.com/trend-following-monkey-style/#comment-1133

    Since your drawdown number is biased downward (more biased with more sampling), your MAR statistic is also biased upward, as pointed out by Jing.

  • Jez Liberty

    @Alvantage, thanks for the comments and code on the other post… yes I also believe that the averaging does reduce the volatility as mentioned in other comments (ie check the volatility of the completely random system for example), so in a sense the drawdown/MAR stats are not so relevant – however the main goal of the study was to check the impact on raw performance (ie return) of different strategies (ie positive for random entries/trailing stops and negative for smart entries and random exits) rather than considering them as real systems and considering their risk measures.

  • Jing

    @Alvantage, thanks for your observation. It solves my doubt why random entry/trailing stop has so good MAR. If look at trend following wizards’ performance, most of their MAR are just around 0.7 -0.8. Now I know it is the process of averaging monthly returns reduce the drawdown. So I think good exit alone can contribute a lot to overall profit. To improve risk-adjusted performance, good entries and diversification are critical. For long-term survival, risk management and money management is the key.

  • Rick

    Jez wrote: “however the main goal of the study was to check the impact on raw performance (ie return) of different strategies (ie positive for random entries/trailing stops and negative for smart entries and random exits) rather than considering them as real systems and considering their risk measures.”

    Maybe this needs a lot more clarification. I don’t see the point of studying systems that have no practical use. This statement of yours assumes that conclusions about the validity of various methods can be drawn from systems with no practical use. Ii is in my humble opinion a contradictory statement. How can value be derived from useless results?

  • Jez Liberty

    I suppose everybody will find different “levels of value” from the results above. It is unfortunate you did not seem to find any.

  • Rick

    Jez,

    I had no intention of undermining your work but I don’t see any practical use for it ( I refer to to this particular study of course). If you see any, please let me know because I may be missing something and I admit that is highly possible too. I do not think that your work proves or even remotely indicates timing entries is not as important as timing exits. Nothing of that sort can be deduced from your studies because as others pointed out already your mechanism of averaging a large number of random trials across a very large number of future contracts makes any conclusion along those lines not possible. This reminds me of “the average family has 1.5 kids” conclusion of statistics.

    Maybe it would be interesting if you could summarize your own conclusions based on the results of this study of yours and try to justify them. What are those different “levels of value” you mentioned? Why is it unfortunate that I do not seem to find any? I am asking because I might have missed something here, not with some other intention in mind.

  • RiskCog

    The best advice I have to give is to be more conversational and less pointed in the comments! Jez is doing a great job on this blog.

    I think this study has a lot of value in creating some data around the question of how much money management generally contributes to system performance. The expected return for a trading account is the combination of return from the system’s edge, and the return or cost of the money management. If you let the expected return of the edge be zero with a random system, then you can disambiguate the return of the money management quite smartly.

  • Jez Liberty

    Thanks RiskCog.
    I think you summarized it very well.

  • Amit

    Fantastic series of posts. Curious why you didn’t run the test with a MA entry and trailing stop loss..? Does it perform superior to the random entry with a trailing stop loss?

  • Jez Liberty

    Thanks Amit!

    To be honest, I can not remember why I did not run a “MA entry / trailing stop exit” test then. I would expect it to perform slightly better as you suggest.
    Jez

  • James

    Great study and very interesting findings. Just curious as to why you did not try random entries with a fixed stop and fixed profit target? For example, random entry with a 1R stop and a 1R profit target. Then step the profit target R up from 1,2,3,4,5 etc and see what the results would be like.

    I think that could be quite interesting also.

  • Jez Liberty

    James – the main aim was not to test all combinations (although in itself could be a useful exercise) but rather to compare stop-losses and profit-target (both sides of “let your profits run” and “cut your losses short”)

  • James

    Thanks for your reply. I see what you mean.

    If you do ever happen to do it I, for one, would be very interested in the results!

  • boyPlunger

    Interesting how 90% of traders spend 90% of their time on the entry and 90% of all traders lose money.

  • Peter

    Hi!
    I have also been testing random entries in combination with profit targets and stops but my testing results are the exact opposite to those you have posted. Stops impair trading results and profit targets enhance results.
    Can you post the actual formulas you have used for testing? I think the answer to the discrepancy must lie there.
    My random signal formula was very simple:
    Buy = Cover = random() > 0.8;
    Sell = Short = random() < 0.2;
    This produces an equal number of long and short signals; 1 signal on each side every 5 days; average trade duration 5 days.
    I tested it on 50 years of SP500 index data. For profit targets I used 3% profit on intraday basis. For stops I used 3% both for MM and Trailing, also on intraday basis. Testing was done with Amibroker. You can easily duplicate it on other platforms.
    The results strongly favour the use of profit targets, not stops. Can you find error in my method or execution?

  • Jez Liberty

    Peter,
    One of the differences might lie in the average trade duration – note my comment in the post: “I did assist the monkey a bit, by giving him some “hindsight bias”, so that the average trade duration was roughly equal to the trade duration of the systems in the “Random Entries, Trailing Stops” scenario (for example 13 days for 2xATR, 63 days for 5xATR or 220 days for 10xATR).”

    Stop-losses are good when they let the time for a trend to form/develop but I suspect after 5 days only there would be a stronger mean-reversion component, especially in the S&P500 (I tested on a variety of markets)

  • prem nath

    Dear Sir,
    I have a different suggestion.Assume a stock is $100 & one month option costs $1 & 4 month out option costs $2 based on option formulas & also real life.The reason is that to travel $2,it takes this stock 4 times longer time span compared to distance of $1.IF entry was random & take profit was $1 & stop loss was $2,then out of 4 runs you would make $1 times3 & lose $2 one time,with net gain of $1–only thing one has to do is to cancel stop loss automatically as soon as take profit is HIT.REPEAT THIS PROCESS OVER & OVER & pretty soon you would make more money than the KING ON WALL STREET.Option formulas may not be perfect but trillions of dollars of derivatives are priced based on THAT & market makers stay in business & do not blow up.Where is the problem with this thinking?

Leave a Comment