Comments on: Intricacies of Market and Trend Following Changes

By: Michael Covel

Michael Covel — Tue, 23 Mar 2010 03:45:45 +0000

Jez, not to belabor this post direction, but there is no radical shift with this one player. His bio is an open book now and ‘system trader’ was never an accurate description. Discretion all over — even going back to Turtles days. That’s not even getting into a myriad of other inconsistencies and controversies that cast doubt.

By: Jez

Jez — Mon, 22 Mar 2010 22:21:46 +0000

very true and I agree: nothing’s easy… I was just saying that I wished your Trend Following book covered more of the not-so-simple nitty-gritty details.

Ahaha, re: Trading from Your Gut: not much chance of a review here as I have no interest in this book (except to see how he can justify this radical shift from being a system trader)

By: Michael Covel

Michael Covel — Mon, 22 Mar 2010 21:06:14 +0000

Jez, I am not sure what worthwhile endeavor in life is easy! That said, I have thought about giving up my trend following preaching now that I have read “Trading from Your Gut!” That might make a good book review here?

By: Jez

Jez — Mon, 22 Mar 2010 19:33:29 +0000

@Mike – Thanks for dropping by.
Credit where Credit is due: I have updated the post to reflect this – Apologies, as I did get my quotes mixed up between yours and Faith’s (in his Way of the Turtle book) which are very similar.

Regarding the debate, and I did enjoy reading your books, I still came away with the impression that your writing made Trend Following sound easier/simpler than it actually is when digging in the details – although I cannot comment on the material in your courses (which might include some adaptability rules/systems similar to what Dunn might be doing?)

By: Michael Covel

Michael Covel — Mon, 22 Mar 2010 16:52:19 +0000

Quick clarification. The “every five years trend following is declared dead” line is from page 20 of my book “Trend Following”. The source is not Faith. In fact, Faith’s only money management firm since the Turtle days did go bust and ended in a CFTC permanent ban.

Also, in terms of the “are markets changing debate?” … I hold my (2) books out as my view on the subject, not one blog post alone (even though I support the post).

Keep up the good debate guys! :-)

By: david varadi

david varadi — Sat, 13 Mar 2010 02:14:57 +0000

nice work jez and a good article. i would comment that there doesn’t need to be dividing camps between walk-forward testing and the robust long-term careful testing camp. effectively as humans we are evaluating systems in a way that can be automated, thus ideally the compromise is to simply create a walk-forward test that mimics what we as system developers would be doing at a given point in time.

best
david

By: Jez

Jez — Fri, 12 Mar 2010 21:52:04 +0000

Great comments Troy and Sam, thanks!
Completely agree on the skepticism in general and the possibility of the system breaking down after successful out-of-sample testing. If you think about it this can also be curve-fit: if you test/optimise 100 systems you will probably find one which performance will be very good (by effect of random luck). This is “1st degree curve-fit” if you like… If you now take 10,000 systems, you will probably find 100 very good ones in your optimisation. If you now test these 100 systems on out-of-sample data, you will probably find 1 very good one (random luck again). This is second degree curve-fit. The system is likely to break down in the future because randomness and survivorship bias only made it rise to the top… The same concept applies to any backtesting process. The trick is to limit the odds of picking a “random winner” instead of a true one..

That being said, an appealing aspect of walk-forward is that it is a sort of dynamic backtesting process that should self-adapt. Fwiw I have no strong opinion for/against walk-forward.

It is also true that Garner probably had the benefit of hindsight in tuning his turtle system although I have not found published the method he used to tweak the system (pretty sure it was not walk-forward). It could well be backtesting 100s of params and picking the best ones (although I know that he has generally been leaning towards the long-end of the trading timeframe).

What excites me is the challenge of formalising/automating the optimisation/verification process so that it can be objective and devoid of as much curve-fitting as possible. Some challenge!

By: Troy S.

Troy S. — Fri, 12 Mar 2010 17:18:39 +0000

A very interesting a relevant post (not that your other posts aren’t interesting and relevant too!).

I especially like your alternative walk-forward idea. Determining a relationship b/w returns and fat-tailed distributions/autocorrelation, and its predictive value, would be another way of building a more adaptive system (in addition to your walk-forward method in #1, and the one you mention in the post above).

Garner’s gorgeous equity curve almost made my brain explode. However… I may be dense and completely missing something, but it seems to me that Garner just tweaked the Turtle system with the benefit of hindsight, and not as a result of walk-forward testing?

By: Sam

Sam — Fri, 12 Mar 2010 17:05:01 +0000

Guys it’s important to remember that even a system built on in sample data and successfully tested on out of sample data has a certain probability of falling apart at any given point in the future. This is due to the well known fact that a historical pattern has no obligation to remain intact for any given period of time.

It is wise to always be skeptical of any kind of back tested results regardless of whether or not it was run on out of sample data or walked forward. Just because it has performed on these particular kinds of tests does not mean it is not curve fit.

Things exist in the market until they don’t.

By: Jez

Jez — Thu, 11 Mar 2010 15:20:28 +0000

Prashant,
Agree with you, building a Trend Following system is not as easy as it can sound (in some popular books) but I think you missed a very important point: Money and Risk management. I believe that it can have more impact to the overall performance than the system signals.

With regards to Winton, I am not sure he just runs a 100% Trend Following fund. He definitely wants to build a firm to compete with Renaissance (with other strategies than TF) and that might be a reason for so many PhDs (as well as the possible marketing weight of it).

And for walk-forward testing, please read the link to my older post explaining how it works (http://www.automated-trading-system.com/walk-forward-testing/ ). There is a cool animation illustrating it. What you are describing is simple out-of-sample testing whereas Walk-Forward is an adaptive process which re-optimises the system on a continuous basis.