In a previous post, I mentioned and linked to a presentation by Transtrend: “Potential Profitability by Trend Following Systems”, where the Trendpot Indicator is introduced.

Trendpot is a backward looking measure that indicates for an individual market whether a medium to long-term trend following system could potentially have been profitable when applied on that particular market during the preceding month.

Trendpot is a number between 0 and 1:

• 0 indicates a very low chance for a trend following system to have been profitable.
• 1 indicates a very high chance for a trend following system to have been profitable.

The concept is an interesting way of identifying how much a market/portfolio has been in trending vs. non-trending mode and I decided to implement the indicator in Trading Blox – with a twist:

The original Trendpot calculation uses a fixed one-month lookback period, to evaluate the trend potential for that month (and therefore only considers potential performance for a “medium to long-term trend following system”). I parameterized the formula to look at different periods and be able to evaluate the trend potential on different timeframes on a daily basis.

For those interested in the code only, please see the end of this post for details. The formula code is Trading Blox specific, but simple enough to be derived easily for any back-testing language.

Trendpot Usage

Transtrend describes the indicator’s goal as a help to explain the performance of a trend following system. In effect, Trend Followers performance is subject to the behaviour of the market: ranging markets are not nearly as conducive to good Trend Following performance as trending ones.

Being able to back this argument with a quantitative measure of the trendiness of markets is probably a good way to reassure investors during expected periods of system under-performance. In 2009 for example, markets might have seemed to offer trends (with “headlines” markets such as Gold or S&P 500 “trending” up by more than 50% after their respective lows at beginning of the year), yet Trend Following Wizards performance was mostly negative in 2009.

In the presentation, Transtrend provides some explanation in the form of the Trendpot calculation for a diversified portfolio: 2009 displayed the lowest Trendpot value (i.e. lowest trend potential) of the last 20 years.

I used the indicator to perform a near-identical study on the portfolio used in the State of Trend Following report and measured the Trendpot values for years 2008, 2009 and 2010 (which were respectively, very good, negative and rather positive for the Trend Following Wizards – or to avoid subjective adjectives: +29.9%, -7.3% and +19.1% respectively).

The average values for the 30-day Trendpot were:

• 2008: 0.331
• 2009: 0.203
• 2010: 0.245

Note: to put these numbers in perspective, the average monthly Trendpot values per year calculated by Transtrend for the last 20 years varies between 0.24 (obtained in 2009) and 0.39 (obtained in 2008).
It seems that the value calculated by their version generates slightly higher numbers, which could be for a number of reasons such as:

• Different portfolio (Transtrend portfolio contains more markets, including spreads)
• Different price representation (Transtrend use one price series for every market representing a liquidity-weighted combination of tradable active months, whereas I used a simple back-adjusted contract)
• Type of price used (I considered High, Low and Close whereas Transtrend only considers daily closing prices).

Nevertheless, the pattern in calculated indicator values is identical: 2008 displays a much higher value than 2009.
Graphically, this is also illustrated by the frequency histogram of the Trendpot values for the whole portfolio for these three different years:

2008 shows a larger tendency of markets to be in trending mode, with the frequency of the Trendpot value maxing out at 1 being nearly three times as large as that of 2009, and the histogram being generally “heavier” on the right-hand side (which is expected, as the overall average value is higher).

Trendiness of Different Timeframes

By updating the value of the period used in the indicator calculation, we can look at different timeframes, going from shorter-term trend following (i.e. 5 to 10 days) to much longer periods.

Below are two histograms, for 2008 and 2009, showing the distribution of Trendpot values for 4 period values, ranging from 5 days to 50 days:

And the average values:

• 5-day Trendpot Average Value: 2008= 0.247, 2009= 0.2226
• 10-day Trendpot Average Value: 2008= 0.266, 2009= 0.203
• 30-day Trendpot Average Value: 2008= 0.331, 2009= 0.203
• 50-day Trendpot Average Value: 2008= 0.352, 2009= 0.242

It might be interesting to look at a longer set of historical values and compare the standard deviation of Trendpot values. A lower standard deviation number might indicate a better robustness for a specific timeframe (robust meaning here that the potential – good or bad – of a specific timeframe is less affected by market conditions).

Trendiness of Different Instruments

Another potential use for the indicator is to look at isolated markets and their average Trendpot values to derive their trending potential.

I have run such a calculation for the instruments in the State of Trend Following portfolio over the period 2008-2010. The average value for an instrument is 0.267, with a standard deviation of 0.083.

Here are the top and bottom values:

Top Trendpot values:

 
Bottom Trendpot values:

 
An interesting next step – possibly in a follow-up post – would be to calculate the correlation between the Trendpot value for an instrument and its contribution to the system performance.

Postdictive or Predictive Indicator?

Transtrend specify that “Trendpot is a measure, not an indicator: No predictive value!“.

As a monitoring tool, Trendpot looks like a useful addition to the box. However I wonder whether it could not be used as an input to over/under-weigh timeframes or instruments based on their past Trendpot values.
This would rely on the assumption that futures values are somehow function of past values.
We know that some market characteristics are not independent of past values (serial correlation for example), perhaps the trendiness of markets is one such characteristic…

Code

Trading Blox owners should be able to download the blox code, which I uploaded on the Blox Marketplace section of the Trading Blox formula. This is a simple implementation, which simply plots on the chart and outputs in a file the value of the Trendpot indicator for each instrument in the portfolio.

For those readers not having access to this section of the Trading Blox forum, I include below the main part of the code, which merely implements Transtrend formula, with an additional period parameter:

Ht = Highest(instrument.High, Period)
Ht_1 = Highest(instrument.High, Period, Period)
Ht_2 = Highest(instrument.High, Period, 2*Period)
Lt = Lowest(instrument.Low, Period)
Lt_1 = Lowest(instrument.Low, Period, Period)
Lt_2 = Lowest(instrument.Low, Period, 2*Period)
Ct = instrument.Close
Ct_1 = instrument.Close[Period]
Ct_2 = instrument.Close[2*Period]
 
NewHigh = Ht - max(Ht_1,Ht_2)
HighEnd = Ct - Ht_1
Upmove = min(Newhigh, HighEnd)
SmoothUp = min(Ht - Ht_1, Lt - Lt_1, Ct - Ct_1)
TrendUp = max(0, (Upmove+Smoothup)/2)
Decline = max(Ht_2 - Lt, Ht_2 - Lt_1, Ht_1 - Lt, Ht - Ct)
ScaleUp = max(Decline, trendUp)
TrendpotUp = TrendUp / ScaleUp
 
NewLow = min(Lt_1, Lt_2) - Lt
LowEnd = Lt_1 - Ct
Downmove = min(NewLow, LowEnd)
SmoothDown = min(Ht_1 - Ht, Lt_1 - Lt, Ct_1 - Ct)
TrendDown = max(0, (Downmove+SmoothDown)/2)
Rise = max(Ht - Lt_2, Ht_1 - Lt_2, Ht - Lt_1, Ct - Lt)
ScaleDown = max(Rise, trendDown)
TrendpotDown = TrendDown / ScaleDown
 
Trendpot = IfThenElse(Ct - Ct_1 > 0, TrendpotUp, IfThenElse(Ct - Ct_1 < 0, TrendpotDown, 0))

Note/Update: The functions Highest/Lowest use a Trading Blox-specific notation. The second argument is the number of bars over which to find the value while third argument is the “offset” (ie number of recent bars to skip).

Syntax:
Highest( series, bars, [offset] )

Parameters:
series: the name of the series
bars: the number of bars over which to find the value
offset: the number of bars to offset before finding the value

returns:
the highest value

For example:
Ht = Highest(instrument.High, Period) is the highest High over the last 30 bars (if Period = 30), i.e. roughly last month’s high
Ht_1 = Highest(instrument.High, Period, Period) is the highest High over the 30 bars, starting 30 bars ago (if Period = 30), basically between bar 31 and 60 in the past, i.e. roughly the previous month’s high.
 
 

I have already covered the idea of using ETFs in place of Futures. Today, I wanted to run a quantitative comparison between the two instrument types

The ETF sector has been growing at an impressive rate, with new offerings popping up every month and providing a wider choice in selecting a portfolio to trade. One of the drawbacks of using ETFs for mechanical trading strategies is the relative short history of these instruments, making it hard or impossible to run to back-tests looking back far in the past – which I believe is critical to understand how a strategy works over different types of market conditions.

In A Practical Guide to ETF Trading Systems, author Anthony Garner uses futures contracts as a proxy for testing strategies on ETFs. If futures are good proxies for ETFs, the back-test results from a long run with futures can be extrapolated to the ETF world.

This post is attempting to look at one way of quantifying how well futures can act as a proxy.

Quantification by Correlation

The main “tool” used in this post, to test whether futures are good proxies to ETFs, is that of correlation between prices as well as returns. The assumption being that correlations between an ETF and a good candidate futures proxy must be as close as possible to 1.0. The closer to 1, the better the proxy relationship.

I picked a few ETFs and associated futures and calculated the correlation (using Pearson product-moment) between several of the data attributes:

• Open
• High
• Low
• Close
• t-1 Return
• t-5 Return
• t-30 Return

The data considered in each case was the list of all dates where both futures and ETF had data. The futures contract were proportionally back-adjusted (to keep price ratio at correct values, as discussed in this post).

The table below shows the number of records for each comparison, the start date and the various correlation coefficients.

Note: please make sure you view this post with your “browser zoom” at 100% for the numbers to be readable.

Most of the numbers are pretty close to 1, which seems to validate the idea of using the future contract as a proxy for ETF.

Some underlyings can be traded through different ETFs (or ETNs). Crude Oil, for example, has several ETFs aiming to capture its performance (I picked DBO, OIL and USO for this test). We can see that there are some differences between the correlation figures of these three ETFs. Not all ETFs seem to track the underlying as well….

A more obvious case of divergence is between the two Cotton ETFs: BAL and COTN (traded in London). The COTN ETF has a low correlation reading on previous-day returns (0.319). Obviously, the fact that the two instruments are trading in different timezones (London vs. NY) will cause their daily performance returns to differ as the market still moves after the London close. This would be something to also keep in mind when choosing an ETF/future pair.

Note on Correlation: Pearson vs. Spearman

I have chosen the Pearson correlation method for these calculations, as the linear relationship between the variables under comparison was what I felt needed to be looked at. However there are several “limitations” of using this correlation calculation. One of them is the assumption of the data being normally distributed, resulting in the calculation being very sensitive to outliers.

This is best illustrated with the correlation of “t-1 Returns” between the COTN ETF and its corresponding Cotton future contract. Back in September 08, the ETF price dropped by 60% before gaining 75% the next day. I seem to remember this was related to a scare around AIG-backed ETFs/ETNs, when the insurance giant was rumoured to go bankrupt.

For the purpose of a test, I simply removed these 2 extreme returns from the file and re-ran the correlation calculation. The figure came in at 0.518. A large increase, obtained by simply removing two outliers. This highlights the point about Pearson correlation not being the most robust with regards to outlier data points.

There are other methods for calculating correlation, and Spearman’s rank method does not make any assumptions on the distribution of the data. It can also detect some non-linear relationship (which we were not interested here). Nevertheless, I re-ran the whole set using this alternative method. The figures were approximately identical – except for the case of COTN.

The Spearman correlation coefficient for COTN v CT using both original and modified files were near-identical at around 0.61. No big jump by removing two outliers.

Note on ETFs

ETPs (Exchange Traded Products), be they called ETF, ETN or ETC all come in different flavours. They can be structured in different ways and track different benchmarks, even when they are associated to the same market. They also add an extra layer and therefore additional counterparty risk on the intermediaries, costs/fees and tracking error.

A good example of the differences between some products can be found in this ETFdb article: USO vs. OIL: A Better Crude Oil ETF?

Finally, it is not (yet?) possible to trade the equivalent of a fully diversified futures portfolio with ETFs, as all markets are not represented in the ETF universe.

Appendix: R Code

Once again, I used R to generate the various correlation calculations. The idea being that one it is coded up it can be applied very quickly to any data.

The code takes a parameter file containing the files to be compared (1st column = description, 2nd and 3rd columns = file names) and the correlation calculation type. See sample here.

Here is the code below. Hopefully the comments are clear enough to help you understand the logic. This might not be the most perfect way to implement this but it does the job:

First, it defines a function to perform the calculations on two files (ie ETF vs. future):

# Define correlETFFut function
correlETFFut <- function(params, i, cor.method,...) {
 
  colsFut <- c("date", "O", "H", "L", "C", "Vol", "OI", "Con", "UC")
 
  # Adding headers
  cols1 <- colsFut
  cols2 <- colsFut
 
  # Read in data sets
  data.1 <- read.csv(toString(params[i,2]), col.names=cols1)
  data.2 <- read.csv(toString(params[i,3]), col.names=cols2)
 
  # Join both data sets
  data.merged <- merge(data.1, data.2, by.x="date", by.y="date")
 
  #Calculate correlation on Open, High, Low and Close
  cor.O <- cor.test(data.merged$O.x, data.merged$O.y,method=cor.method)
  cor.H <- cor.test(data.merged$H.x, data.merged$H.y,method=cor.method)
  cor.L <- cor.test(data.merged$L.x, data.merged$L.y,method=cor.method)
  cor.C <- cor.test(data.merged$C.x, data.merged$C.y,method=cor.method)
 
  #Calculate correlation on t-1 Rtn
  log.diffs1 <- log( data.merged$C.x[2:sum ( !is.na ( data.merged$C.x ) )]/data.merged$C.x[1:(sum ( !is.na ( data.merged$C.x ) ) - 1)])
  log.diffs2 <- log( data.merged$C.y[2:sum ( !is.na ( data.merged$C.y ) )]/data.merged$C.y[1:(sum ( !is.na ( data.merged$C.y ) ) - 1)])
 
  cor.Rtn <- cor.test(log.diffs1, log.diffs2,method=cor.method)
 
  #Calculate correlation on t-5 Rtn
  log.diffs1W <- log( data.merged$C.x[6:sum ( !is.na ( data.merged$C.x ) )]/data.merged$C.x[1:(sum ( !is.na ( data.merged$C.x ) ) - 5)])
  log.diffs2W <- log( data.merged$C.y[6:sum ( !is.na ( data.merged$C.y ) )]/data.merged$C.y[1:(sum ( !is.na ( data.merged$C.y ) ) - 5)])
 
  cor.RtnW <- cor.test(log.diffs1W, log.diffs2W,method=cor.method)
 
  #Calculate correlation on t-30 Rtn
  log.diffs1M <- log( data.merged$C.x[31:sum ( !is.na ( data.merged$C.x ) )]/data.merged$C.x[1:(sum ( !is.na ( data.merged$C.x ) ) - 30)])
  log.diffs2M <- log( data.merged$C.y[31:sum ( !is.na ( data.merged$C.y ) )]/data.merged$C.y[1:(sum ( !is.na ( data.merged$C.y ) ) - 30)])
 
  cor.RtnM <- cor.test(log.diffs1M, log.diffs2M,method=cor.method)
 
  #Put it all into cor.all
  cor.all <- c(sum ( !is.na ( data.merged$C.x )), data.merged$date[1], signif(cor.O$estimate,5), signif(cor.H$estimate,5), signif(cor.L$estimate,5), signif(cor.C$estimate,5), signif(cor.Rtn$estimate,5), signif(cor.RtnW$estimate,5), signif(cor.RtnM$estimate,5))
  return(cor.all)
 
}

Second, a function reads the parameter file of all files to be compared, iterates through it and call the first function for each pair of files, building an array of all results:

# Define fullCorrel function
fullCorrel <- function(file, cor.method,...) {
 
  # read input file
  comps <- read.csv(file, header=F)
  for (i in 1:dim(comps[1])[1]){
    cor.tmp <- correlETFFut(comps,i,cor.method)
    if (i==1) cor.full <- cor.tmp else cor.full <- cbind(cor.full, cor.tmp)
  }
 
  rownames(cor.full) <- c("Record Count", " Start Date", " cor Open", " cor High", " cor Low", " cor Close", " cor Rtn", " cor RtnW", " cor RtnM")
  colnames(cor.full) <- t(comps[1])
 return(t(cor.full))
}

Finally, set your working directory (where all the files are) and call the second function with the relevant parameters (here the function is called twice with the same parameter file and two different correlation methods (make sure to update the working dir to your own path):

# Set Working Dir - Replace with your own path!
setwd("D:/ATS/Blog/Posts/ETF v Futures/Data")
 
# Run Pearson correlations
fullCorrel("ETFvFut.txt", "pearson")
 
# Run Spearman correlations
fullCorrel("ETFvFut.txt", "spearman")

Below are included some sample files to run the code above:
Sample_Data

At the same time as Josh Ulrich was making some very good points on why to use R, over on his FOSS Trading blog, I was coming to the same realisation that it is a very neat, useful tool indeed. In this post, I’ll present the code I used for the last post on running a Walk-Forward test of the LSPM. Hopefully this can serve as an introduction to R to readers who do not use it yet.

Getting into R

The LSPM java app distributed by Ralph Vince during his seminar did not seem very robust – and in any case, I could not manage to get it to generate the Joint-Probability Table (JPT) for 6 market-systems over a 20-year history of monthly returns. It would simply hang after 4 of them.

I then decided to write my own JPT tool. Did not take me too long, an hour or two and about 200 lines of C# code. It did the job ok and I could run it in an automated/batch mode. But I had forgotten that Josh had posted some R code to do exactly that (Rats! A couple of hours wasted…). On top of that the code was about 30 lines long. Impressed by this, I decided that the study on the last post was a good opportunity to try and learn R and I attempted to automate the test using R only.

What the Code Does

At a high level, the code reads several files each containing the history of monthly returns for a market-system. The format of the files is as follows:

19900201,-0.5
19900301,-1.16
19900401,2.24
19900501,3.3

First column = date, second column = percentage return

This is a Walk-Forward test, so the optimization runs on a sliding window of past returns, and results are applied to the walk-forward window of returns. The cycle is repeated as many times as necessary.

For each cycle, the return data from the optimization window is extracted from the full set of returns, passed to the LSPM optimization, and calculated leveraged returns (using f amounts from the optimization) are stored for the corresponding walk-forward window. The full results of the test consists of a concatenation of all leveraged returns in each walk-forward window.

The code below is my first forray in R and its robustness could be “enhanced” (it contains several assumptions such as file formats, number of market-systems) but it should be fairly easy to adapt to run different scenarios or make it more generic.
R is quite different from the programming languages I am used to, so some of the syntax might seem a bit hacked – but it does the job (I appreciate any corrections/pointers on better practices from experienced R readers…).

Hopefully, it is a useful example for readers wanting to get started with R. Readers might want to check my initial post on using R to run the LSPM optimization as this code expands on some aspects of it.

The Code

The first part of the code sets the walk-forward parameters, here: 10 years for the optimization window and one year for the walk-forward window:

# Set Walk-Forward parameters (number of periods)
optim<-120 #10 years = 120 monthly returns
wf<-12 #1 year = 12 monthly returns

Open the market-system files:

# Open files
setwd("C:/MyPath")
a <- read.csv("TMA-20-50-200.csv", header = FALSE, as.is = TRUE, sep = ",", dec=".");
b <- read.csv("MA-50-200.csv", header = FALSE, as.is = TRUE, sep = ",", dec=".")
c <- read.csv("BBO-20.csv", header = FALSE, as.is = TRUE, sep = ",", dec=".")
d <- read.csv("Donchian-200.csv", header = FALSE, as.is = TRUE, sep = ",", dec=".")
e <- read.csv("Donchian-20.csv", header = FALSE, as.is = TRUE, sep = ",", dec=".")
f <- read.csv("BBO-50.csv", header = FALSE, as.is = TRUE, sep = ",", dec=".")

Calculate the number of Walk-Forward cycles:

# Calculate number of WF cycles
numCycles = ceiling((nrow(a)-optim)/wf)

Below is the code mentioned earlier, which can be found on Josh Ulrich’s blog – used to create the LSPM JPT:

# Define JPT function
jointProbTable <- function(x, n=3, FUN=median, ...) {
 
  # Load LSPM
  if(!require(LSPM,quietly=TRUE)) stop(warnings())
 
  # Function to bin data
  quantize <- function(x, n, FUN=median, ...) {
    if(is.character(FUN)) FUN <- get(FUN)
    bins <- cut(x, n, labels=FALSE)
    res <- sapply(1:NROW(x), function(i) FUN(x[bins==bins[i]], ...))
  }
 
  # Allow for different values of 'n' for each system in 'x'
  if(NROW(n)==1) {
    n <- rep(n,NCOL(x))
  } else
  if(NROW(n)!=NCOL(x)) stop("invalid 'n'")
 
  # Bin data in 'x'
  qd <- sapply(1:NCOL(x), function(i) quantize(x[,i],n=n[i],FUN=FUN,...))
 
  # Aggregate probabilities
  probs <- rep(1/NROW(x),NROW(x))
  res <- aggregate(probs, by=lapply(1:NCOL(qd), function(i) qd[,i]), sum)
 
  # Clean up output, return lsp object
  colnames(res) <- colnames(x)
  res <- lsp(res[,1:NCOL(x)],res[,NCOL(res)])
  return(res)
}

Loop through each Walk-Forward cycle:

for (i in 1:numCycles) {

Extract the relevant set of past returns for the optimization:

	# Define cycle boundaries
	start<-1+(i*wf)
	end<-optim+(i*wf)
 
	# Get returns for optimization cycle and create the JPT
	rtn <- cbind(a$V2[start:end],b$V2[start:end],c$V2[start:end],d$V2[start:end],e$V2[start:end],f$V2[start:end])

Create the JPT and run the actual LSPM optimization (note that the case above is the “simple” optimization, only taking optimal geometric growth rate into account – no consideration for probability of drawdown. This was mostly to ensure the run did not take hours and hours):

	jpt <- jointProbTable(rtn,n=c(20,20,20,20,20,20))
	outcomes<-jpt[[1]]
	probs<-jpt[[2]]
	port<-lsp(outcomes,probs)
 
	# DEoptim parameters (see ?DEoptim)
	np=60       # 10 * number of mktsys
	imax=1000       #maximum number of iterations
	crossover=0.6       #probability of crossover
 
	NR <- NROW(port$f)
	DEctrl <- list(NP=np,itermax=imax, CR=crossover, trace=TRUE)
 
	# Optimize
	res <- optimalf(port,control=DEctrl)

Retrieve the leverage amounts from the optimization as well as the set of returns from the Walk-Forward window. Apply the leverage amounts to these returns and add the data to the full set of leveraged returns:

	# Save leverage amounts
	lev<-c(100/(-jpt$maxLoss[1]/res$f[1]),  100/(-jpt$maxLoss[2]/res$f[2]),  100/(-jpt$maxLoss[3]/res$f[3]),  100/(-jpt$maxLoss[4]/res$f[4]),  100/(-jpt$maxLoss[5]/res$f[5]),  100/(-jpt$maxLoss[6]/res$f[6]))
	levmat<-c(rep(1,wf)) %o% lev #so that we can multiply with the wfrtn
 
	# Get the returns for the next Walk-Forward period
	wfrtn <- cbind(a$V2[(end+1):(end+wf)],b$V2[(end+1):(end+wf)],c$V2[(end+1):(end+wf)],d$V2[(end+1):(end+wf)],e$V2[(end+1):(end+wf)],f$V2[(end+1):(end+wf)])
	wflevrtn <- wfrtn*levmat #apply leverage to the returns
 
	if (i==0) fullrtns<-cbind(t(wflevrtn)) else fullrtns<-cbind(fullrtns,t(wflevrtn))
}

After the loop is finished, print the full set of leveraged returns

t(fullrtns)

In Closing

Despite my very limited experience, I can really see some of the potential in using R for any sort of data manipulation.

If you want to get started in R and test this code, the following links will be useful:

R homepage, which contains link to a download page. The installation is very simple and quick. Make sure you also check the docs as they are good.

The R LSPM package maintained by Josh Ulrich, the real meat of the code above – credits to him for making this!

The R code included in the post above

And a batch file to run the above code from a command prompt.

Finally, if you want to re-run the same test, here are the return files that were used in the code above:
Return Data Files (zipped)

In part 1 of this bootstrap post, we looked at how to apply the method to establish the statistical significance of a single trading rule. In Part 2, we’ll look at how to deal with the data mining bias, the impact of geometric vs. arithmetic mean return. The code implementing the bootstrap test is available for download at the bottom of this post.

Dealing with Data Mining Bias

The approach described in the single rule test is not valid when performing data mining (whether testing different rules or different parameter values of the same rule). As per the data mining bias (explained previously), the (best) rule selected from the data mining process will invariably owe a large part of its over-performance to random (good) luck.

The way the bootstrap test deals with the data mining bias is by implementing a concept introduced in White’s Reality Check. The Reality Check derives the sampling distribution appropriate to test the statistical significance of the best rule found by data mining.

In effect, the concept is fairly simple – and similar to the single-rule bootstrap: assuming N rules have been tested in the data mining process, each resample iteration will perform a resample with replacement for each rule and the best mean return will be kept as this resample iteration’s test statistic:

1. N back-tests are run on detrended data. The mean daily return, based on x observations, is calculated for each back-tested rule.
2. Each rule’s mean daily return is substracted from the rule’s set of daily returns (zero-centering), This gives a set of adjusted returns for each rule.
3. For each “higher-level” resample (to form the sampling distribution of the best-performing rule in a universe of N rules), perform a “lower-level” resample with replacement on every rule. For each rule select x instances of adjusted returns at random and calculate their mean daily return (rule bootstrapped mean). Compare each rule bootstrapped mean and select the highest one: this is the test statistic of this “higher level” resample (bootstrapped best mean).
4. Perform a large number of “higher level” resamples to generate a large number of bootstrapped best means.
5. Form the sampling distribution of the best means generated in the step above.
6. Derive the p-value of the best back-test mean return (non zero-centered) based on the sampling distribution derived above

In effect: for each iteration, resample each rule, take the best return, keep it as this iteration test statistic and move on to the next iteration. The sampling distribution is formed of each iteration’s best return.

White Reality Check Related Papers

I have not yet found searched hard for White’s paper on the Reality Check but I did find the two following papers which seem to be worth a read:
Stepwise Multiple Testing as Formalized Data Snooping – Romano & Wolf

Abstract:

It is common in econometric applications that several hypothesis tests are carried out at the same time. The problem then becomes how to decide which hypotheses to reject, accounting for the multitude of tests. In this paper, we suggest a stepwise multiple testing procedure which asymptotically controls the familywise error rate at a desired level. Compared to related single-step methods, our procedure is more powerful in the sense that it often will reject more false hypotheses.

Unlike some stepwise methods, our method implicitly captures the joint dependence structure of the test statistics, which results in increased ability to detect alternative hypotheses. We prove our method asymptotically controls the familywise error rate under minimal assumptions. Some simulation studies show the improvements of our methods over previous proposals. We also provide an application to a set of real data.

Re-Examining the Profitability of Technical Analysis with White’s Reality Check and Hansen’s SPA Test
Abstract:

In this paper, we re-examine the profitability of technical analysis using White’s Reality Check and Hansen’s SPA test that correct the data snooping bias. Comparing to previous studies, we study a more complete universe of trading techniques, including not only simple rules but also investor’s strategies, and we test the profitability of these rules and strategies with four main indices. It is found that significantly profitable simple rules and investor’s strategies do exist in the data from relatively young markets (NASDAQ Composite and Russell 2000) but not in the data from relatively mature markets (DJIA and S&P 500). Moreover, after taking transaction costs into account, we find that the best rules for NASDAQ Composite and Russell 2000 outperform the buy-and-hold strategy in most in- and out-of-sample periods. Our results thus suggest that the degree of market efficiency may be related to market maturity. It is also found that investor’s strategies are able to improve on the profits of simple rules and may even generate significant profits from unprofitable simple rules.

Geometric or Arithmetic Mean?

In part 1, I introduced the idea that the mean arithmetic return being positive is not equivalent to the strategy being profitable (ie. this is not a sufficient condition). On the other hand, the mean geometric return being positive is a necessary and sufficient condition to the strategy being profitable (ie. both conditions are equivalent).

Therefore bootstrapping using the mean geometric return as the test statistic should provide a better evaluation of the system’s profitability statistical siginificance.

I will not go into detail of how the calculation is done as it is very similar to the arithmetic mean return, but using log of returns instead. Note that the geometric mean will be a stricter test than the arithmetic mean (a rules can have a significantly positive arithmetic return but a negative geometric return).

To illustrate the multiple applications of the bootstrapping methodology, I decided to run the test on one of the Trend Following Wizards track record (set of monthly returns). I picked Chesapeake and ran the monthly returns (from 1988 to 2009) through the bootstrap test.

The p-value calculated using the arithmetic mean is 0.000098 (less than 1 chance in 10,000 that this kind of results are due to random luck). Using the geometric mean, the p-value is 0.00022. The values are extremely low, which is not surprising given Jerry Parker’s 20-year track record with only one losing year and a monthly average return of 1.7%.

Many people would point out that survivorship bias should be considered, and obviously it depends on how you look at it. The main point of this dual test is that the geometric p-value is higher than the arithmetic p-value, verifying that it is a stricter test of statistical significance.

Bootstrap Code

Finally, here is a tool coded to implement the bootstrap test for a single strategy – available for download. Note that this is distributed “as is”, with no guarantee (but that’s the one I have been using so I still think it does the job…). It should run on any Windows machine with the .Net framework installed (XP or higher should do fine).

It simply takes three parameters (separated by space):

1. Returns file path and name
2. Number of resamples
3. Flag for Arithmetic (A) or Geometric (G) mean calculation

It also generates a file in the same directory with all of the resamples test-statistic values (to draw the histogram).

Simply place the bootstrap.exe in your directory of choice and run it from the command prompt as below:

Run the bootstrap.exe from the command line

Download here:
bootstrap.exe

CSI Data Extractor: Utility tool to extract CSI data in plain text files

I have mentioned before that I use CSI as my main provider for Futures data. Overall, I judge CSI’s service as good quality – however one drawback is the closed aspect of the data access. And I decided to write a utility to make it more “open”.

To access the historical data, you have to go through their software (Unfair Advantage), which extracts and decrypts it from their proprietary binary format. You can always “build” historical files to disk, but these are usually pre-processed with a special algorithm (ie. back-adjusted, etc.).

What more, if you decide to stop your subscription to CSI’s data services, you will not be able to access any data any more, even historical data downloaded under the subscription contract.

For some of the reasons above, it might be interesting to be able to extract/store CSI’s raw data in plain text files. This would allow for:

• Browsing the data outside of the CSI’s Unfair Advantage software
• Applying back-adjustment or other manipulation algorithms not supported by CSI’s UA (such as calculating the yield curve)
• Keep the historical data independently of your CSI subscription

Note: what I consider CSI raw data is the actual daily information for each contract traded on any day. For example, in the Corn market, there are currently 15 different contracts trading (with different maturities), each with their own Open, High, Low, Close, Volume and Open Interest.

CSI API to the rescue

Now, you could easily go to Unfair Advantage and pick every single historical contract that you want to extract… but this would be a fairly tedious job

Luckily for us, the API allows for programmatic access to CSI’s data to automate the process of extracting all contracts that we want – so I decided to write a little utility to do just that. And now my data is really mine!

Below is a zip file containing the files to copy to “install” the utility:

CSI-DataExtractor.zip

Note that you need to have Unfair Advantage installed on the machine running this exe (which does not come with any guarantee by the way… but I have been using it fine so far).

The operation is very simple: just enter CSI’s number for the market you want to extract and choose which path the files need to be extracted to.

Hope you’re finding this tool useful…

Note: You can receive a 10% discount on CSI data subscriptions (use coupon code LIBERTY)

CC picture by almccon@flickr

Improved Vortex indicator

As mentioned in the original Vortex indicator post, I thought there were some flaws that needed to be corrected in the original version published by the authors (Etienne Botes & Douglas Siepman). I have since heard from them .

They gave me a detailed explanation regarding the logic of their design. The most important consideration is that they are using their indicator on Forex – which is a 24-hour market and therefore exhibits virtually no gap. Moreover, I believe they favoured a simple moving average (to Wilder’s exponential one) to add extra lag / beneficial noise filtering.

Anyhow, I still thought that the indicator could be improved for non-Forex markets. I also thought that the process could be taken further by creating the AVX: the Average Vortex Index (similarly to Welles Wilder’s ADX, based on the DMI). Well, I coded that up as well.

Trading Blox blocks for all this code can be downloaded at the end of the article (or in the new free code section).

Updated calculation logic

I started prototyping the calculations in Excel for the original Wilder DMI and ADX, the original Vortex indicator and my improved version. Please find the Excel spreadsheet for your perusal.

As mentioned in the initial Vortex indicator post, the original calculation did not handle gap days appropriately (in my opinion) and was not replicating the smoothing implemented in Wilder’s DMI by the use of an exponential Moving Average (Wilder’s to be precise) for the VI/TR calculation.

The logic implemented is as follows:

VM+ = High - Low[1]

(VM+ is today’s High minus yesterdays’ Low, can be negative, in case of down gaps)

VM- = High[1] - Low

(VM- is yesterday’s High minus today’ Low, can be negative, in case of up gaps)

True Range (TR) is calculated as per the standard formula.

These 3 daily indicators are then smoothed using the “Wilder” exponential moving average (alpha = 1 / N ). These give us AVM+, AVM- and ATR.

Normalise AVM+ and AVM- by ATR to obtain the improved VI+ and VI-.

VI+ =AVM+/ATR
VI- =AVM-/ATR

Building upon these calculations, we can just replicate the ADX logic to create the AVX indicator:

Calculate the VX:

VX = (VI+ - VI-) / (VI+ + VI-) x 100

VX is then smoothed (using Wilder moving average) to produce the AVX.

Now we’re talking!

Note that because VM and VI both have higher values than the original DM and DMI, VX and AVX,which express the relative difference of VI+ and VI-) will not register values as high as the ADX. But that’s ok – it just means we work on a different scale (where AVX>15 might be similar to ADX>60).

Vortex to the test

The vortex indicator can be used as an entry signal (go long when VMI+ crosses above VMI- and go short with the opposite cross). In addition, the AVX can be thought of as a measure of the strength of a directional vortex movement. The entry signal could be enhanced by only taking the signals where the AVX breaches a specific strength threshold (note: this is a very similar logic to the ADX system that ships with Trading Blox).

Calculating the e-ratio for the VMI/AVX entry is a great way to measure the edge of that specific entry.

Another great thing is the Blox marketplace where Trading Blox users share their custom code. And guess what? There was a block available for e-ratio calculation. Great! I love it when I can leverage existing code…

With a bit of updating of the system I managed to run the e-ratio calc on 3 different entry signals:

• Simple VMI cross
• VMI cross with AVX threshold filter (threshold = 10)
• VMI cross with AVX threshold filter (threshold = 15)

Below are the charts for each of the entries:

e-ratio plotted for length of 1 to 100 days

Notice how the AVX threshold does not seem to help and is completely counter-productive in the “AVX>15″ case. In a classic example of curve-fitting / data snooping, we could decide to turn around the AVX threshold logic and only take signals for which the AVX is below 15. But we will not do that. Bad practice ;-)

Conclusion

I was not expecting much out of this study. I was mostly using it as material for getting to use Trading Blox. I was not surprised by either: I found Trading Blox easy to get a handle on. Definitely a productivity boost compared to TradersStudio, both in terms of development and simulation running time.

With regards to the vortex indicator, it does seem to provide a (small) edge when used as the authors intended to (albeit in a modified form). The AVX filtering, on the other hand, did not seem to improve anything at all. Maybe the edge would have been better, had I not “messed up” with the indicator…

Code download

Please find below the individual components (blocks) for download:
improved Vortex Indicator & AVX auxiliary block file (tbx)
AVX Entry Exit block (tbx)
AVX System (tbs)
The lot in a zip file

Vortex is present in chaotic movements such as fluid mechanics - picture courtesy of NASA

The indicator logic is described in the January issue of TASC (Technical Analysis of Stocks and Commodities) and sounds intriguing (link to PDF article). Without rehashing the details of the indicator calculation, it is very similar to Welles Wilder’s Directional Movement Index Calculation (DMI) except that Wilder’s Directional Movement (largest part of today’s range outside of yesterday’s range) is replaced by calculating the differences between today’s High and yesterday’s Low (positive Vortex Movement) and today’s Low and yesterday’s High (negative Vortex Movement).

TASC provides the Vortex indicator implementations in a wide collection of backtesting platforms (here). Unfortunately the Trading Blox implementation is not provided, which provided me a perfect opportunity to test and learn my new “backtesting toy” out.

System Logic Flaw?

The authors of the indicator provide a sample Excel spreadsheet to illustrate their calculation. By replicating the logic in Excel to get a handle on it (link to Excel spreadsheet), I uncovered what I think is a flaw in the logic of the indicator of the calculation.

The VM calculations take the absolute value of the difference of opposite price extremes (High and Low) between today and yesterday. The screenshot of this Excel spreadsheet below clearly illustrates that this is a problem when dealing with gaps.

Additionally, one could argue that the simple summation/averaging of the VMs is different from the original DMI calculations which use Wilder Moving Averages.

I have emailed the authors of the indicator with a suggestion for a fix. I’ll relay their answer here.

Trading Blox implementation

First and foremost, this is a Trading Blox coding exercise so I was not too concerned about these flaws (although I might be tempted to implement my own version addessing both flaws later).

I suggest you follow the calculation logic on the PDF article or the Excel spreadsheet. Here is how I implemented the Vortex Indicator in Trading Blox:

1: Create Auxiliary Block

This will hold all indicators and implement their calculations. Call it Vortex Indicator.

2: Create parameters

There is one parameter: the period for the Vortex indicator (VI) calculation:

3: Create Indicators

These are the first-level calculations: the daily True Range (TR), and VM+ (plusVM) and VM- (minusVM).

VM+ indicator using a calculated formula

The code for each of these indicators is:

plusVM (VM+ is the difference between today’s High and yesterday’s Low):

Abs(instrument.High - instrument.Low[1])

minusVM (VM- is the difference between today’s Low and yesterday’s High):

Abs(instrument.High[1]-instrument.Low)

TR (standard True Range calculation):

Max(instrument.High-instrument.Low,Abs(instrument.High-instrument.Close[1]), Abs(instrument.Low-instrument.Close[1]))

4: Create Instrument Permanent Variables (IPV)

This is for next-level calculations (Sum of TR, Sum of VM+ and VM-, VI+ and VI-). Setting these up as IPV allow for their values to be made available to other blocks (such the Entry block of a system).

Notice how the variable scope has been updated to System

The values for these IPVs are updated in the block Update Indicators script:

plusVMSum=Sum(plusVM,viDays,0)
minusVMSum=Sum(minusVM,viDays,0)
TRSum=Sum(TR,viDays,0)
plusVI=plusVMSum/TRSum
minusVI=minusVMSum/TRSum

Use in Trading Blox

Adding the auxiliary block to any System will enable it to access the values of the vortex indicators.

Here is how it could be done for a simple reversal system switching from Short to Long when VI+ crosses with VI- (and vice-versa).

Create an Entry Exit block

The system will have the Vortex Indicator Auxiliary block and our new Entry Exit block.

Add VI+ and VI- as IPV to the Entry Exit block

This is just a declaration to indicate that the values of these indicators are calculated in another block (our Vortex Indicator Auxiliary block).

The IPV is defined externally in another block (Vortex indicator)

The other parameters and indicators in this block are standard ATR Stop elements.

Code up the entry formula

Simple logic to enter long when VI+ is above VI- and no long position are in and opposite for short entry:

VARIABLES: pvi TYPE: Floating
pvi = plusVI
VARIABLES: nvi TYPE: Floating
nvi = minusVI
VARIABLES: pos TYPE: String
pos = instrument.position
 
IF 	plusVI > minusVI AND
	instrument.position <> LONG THEN
 
	IF useATRStops THEN
		broker.EnterLongOnOpen( instrument.close - averageTrueRange * atrStop )
	ELSE
		broker.EnterLongOnOpen
	ENDIF
 
ENDIF
 
IF	plusVI < minusVI AND
	instrument.position <> SHORT THEN
 
	IF useATRStops THEN
		broker.EnterShortOnOpen( instrument.close + averageTrueRange * atrStop )
	ELSE
		broker.EnterShortOnOpen
	ENDIF
 
ENDIF

Trading Blox test

Although I would like to do a more complete test to compare the Vortex Indicator to the DMI, all I have time left for today is a simple run of the system described above. I arbitrarily chose 39 for the VI period and here is the result:

Logarithmic Equity curve from Vortex indicator system

And as a side note of caution when dealing with backtest results, I’ll show you a quick trick that will allow me to nearly double the CAGR (Compounded Annual Growth Rate) by not touching the system at all:

Logarithmic Equity curve from Vortex indicator system (truncated period)

Notice what happened there: I just dropped years 1996 to 1999 from the backtests. Et voila! Surely a trick employed by more than one dubious trading systems salesmen…

Conclusion

The authors claim that the Vortex indicator improve on the DMI invented by Welles Wilder Update: the authors have sent me an email to clarify that they have not made such claim (I was merely interpolating, given the ressemblance of the Vortex indicator with the DMI). It would be interesting to test this as the idea of mixing chaos/fluid mechanics concepts to price movements is appealing. Fixing the indicator flaws would be good too. More to come… (update: on the next post, I also explain the authors response to my claim of the indicator having flaws).

Code download

Please find below the individual components (blocks) for download:
link to Vortex indicator block file (tbx)
link to VI sample Entry Exit block (tbx)

So I decided to give the “legendary fast” Amibroker a test to see if it could better TradersStudio’s performance. After some “playing and learning”, I have finalised the code to compute the e-ratio. My big thanks go to the ASX gorilla whose own version forms a large part of my code.

Below is the code explanation and downloadable afl file.

Directly from the ASX Gorilla’s website as a prelude to the code:

My implementation of the Edge Ratio involves two profound Amibroker fudges. The first is the use of the AddToComposite function to create a composite ticker symbol in which to hold the ATR array of a given stock for later retrieval within the Custom Back Tester via the Foreign function.

The second fudge is the use of the VarSet/VarGet function to create a quasi array. This was necessary to overcome the limitation where array elements cannot exceed in number the value of barcount-1.

The first part is to actually code up your Buy signal (in our case a Donchian Channel Breakout). The Sell signals are tested separately:

//BUY RULES: implemented with Buy Stop on Upper Donchian Channel(17)
BuyStop = Ref(HHV(High, 17),-1);
Buy = Cross( High, BuyStop );
BuyPrice = Max( BuyStop, Low ); // make sure buy price >= Low

Exiting positions is done on a fixed duration basis:

//Never Sell so that the position is stopped out after N bar instead
Sell = 3 > 5;
//Stop the positon and close it after N bars
//(eratio = N that we step from 1 to 100 in optimisation)
ApplyStop( stopTypeNBar, stopModeBars, eratio );

The first fudge mentioned above to store the ATR:

Normaliser = ATR(17);
AddToComposite(Normaliser, "~atr_"+Name(), "C", 1+2+8);

And the “meat” of the code: the chunk that implements the custom back-testing to:

1. Loop through the signals and store the Entry ATR value.
2. Loop through all trades and retrieve MFE, MAE and ATR.
3. Compute the e-ratio based on values from all trades.

SetCustomBacktestProc(""); //activate the custom backtester
if(Status("action") == actionPortfolio) //called when backtesting/optimising
{
	bo = GetBacktesterObject();
	bo.PreProcess(); // run default backtest procedure
	TradeATR = NumTrades = ATRArr = 0; //init variables
	for( bar=0; bar < BarCount-1; bar++)
	{
		bo.ProcessTradeSignals(bar);
 
		for ( sig=bo.GetFirstSignal(bar); sig; sig=bo.GetNextSignal(bar) )
		{
			if (sig.isEntry())
			{
				NumTrades++;
				ATRArr = Foreign("~atr_"+sig.Symbol, "C");
				VarSet("TradeATR" + NumTrades, ATRArr[bar]);
 
				_TRACE("Symbol " + sig.Symbol + " ATR: " + VarGet("TradeATR" + NumTrades));
			}
		}
	}
 
	AvgMAE = AccumMAE = AvgMFE = AccumMFE = NumTrades = 0;
 
	// iterate through closed trades
	for( trade = bo.GetFirstTrade(); trade; trade = bo.GetNextTrade() )
	{
		NumTrades++;
		EntryATR = VarGet ("TradeATR" + NumTrades);
		if ( EntryATR != 0 )
		{
			_TRACE("EntryATR: " + WriteVal(EntryATR));
			_TRACE("AccumMAE : " + WriteVal(AccumMAE));
			AccumMAE = AccumMAE + (trade.GetMAE()*trade.EntryPrice/(100*EntryATR));
			AccumMFE = AccumMFE + (trade.GetMFE()*trade.EntryPrice/(100*EntryATR));
		}
 
		trade.AddCustomMetric("My MAE", trade.GetMAE()*trade.EntryPrice/100);
		trade.AddCustomMetric("My MFE", trade.GetMFE()*trade.EntryPrice/100);
		trade.AddCustomMetric("Entry ATR", EntryATR*10000);
	}
 
	AvgMAE = AccumMAE / NumTrades;
	AvgMFE = AccumMFE / NumTrades;
 
	_TRACE(WriteVal(AccumMAE ));
	_TRACE(WriteVal(NumTrades));
	_TRACE(WriteVal(AvgMAE));
 
	Eratio = abs(AvgMFE/AvgMAE);
 
	_TRACE(WriteVal(Eratio));
 
	bo.AddCustomMetric( "Avg MAE", AvgMAE );
	bo.AddCustomMetric( "Avg MFE", AvgMFE );
	bo.AddCustomMetric( "Eratio", Eratio);
 
	bo.PostProcess();
}

If you want to run this code, you can download the e-ratio “gorilla” afl file and simply update the BUY signals to whatever you fancy testing.

The next post will be a direct speed comparison between TradersStudio and Amibroker for computing the e-ratio on the same underlying data and with the same signal. I expect Amibroker to win the fight hands-down as it appeared “way” faster!

I will show you how I calculated the e-ratio for a 17-day Donchian Channel breakout coupled with a 108-day moving average filter (the red curve on this chart). All the code referenced is provided at the end of this post.

As we saw in the last post about the e-ratio, you need to run the same signal over multiple fixed trade durations and record trade data. For this the TradersStudio optimisation and custom reports functionalities fit the bill.

System Code

Step 1: code up the system to test in TraderStudio

As you can not have simultaneous open Buy and Sell positions, there are actually 2 systems (long-only and short-only). The below will be based on the Buy-only system.
The entry signal is a breakout of the Donchian channel if the MA filter conditions are met (price > MA and MA rising).
In TradersStudio this translates to:

MA = ScriptAve(Close, MALength,0)
If MA > MA[1] And Close > MA Then
    Buy(EntryName,1,donchianHigh+MinMove ,Stop,Day)
End If

The exit is simple: close the trade after n days (tradeLength):

If BarsSinceEntry=tradeLength And MarketPosition=1 Then
    ExitLong("ExitLong",EntryName,1,0,CloseExit,Day)
End If

Step 2: retrieve trade data in custom report

As discussed in the e-ratio post we need to get MAE, MFE and ATR for each trade. For this we use a TradersStudio custom report. The report code is executed at the end of the system test and retrieve trade info with the following command:

GetInfoAboutTrades(Info, i + 1)

we are interested in the MAE and MFE which can be derived off the Maximum Profit and Maximum Loss of each trade:

SetCell(row, 13, 2, 10, Info[16]) 'Max Pos Profit
SetCell(row, 14, 2, 10, Info[17]) 'Max Pos Loss

For the ATR value, we need to use a little trick as it is not directly available. We record it as part of the entry name which can be output to the report.
In the system code we have:

EntryName = "DonchianBreakoutBuy" + "|" + indLength + "|" + tradeLength  + "|" + avgtruerange(indLength)

which concatenates the entry name with the ATR value as well as the length of the indicator (Donchian Channel) and trade duration – this will come in handy when we do the e-ratio calculation.

Step 3: run the system through the optimizer

The system is called DonchianChanBrkoutFiltBuy and has 3 parameters:

Sub DonchianChanBrkoutFiltBuy(indLength, tradeLength, MALength)

In the optimization process, step through all values of trade duration required (e.g from 1 to 100 in increments of 1) – warning TradersStudio will take a few hours to do this (sic!).
This will generate a custom trade report for all trades containing all the data required.

Step 4: paste data in Excel

Once you run the long-only and short-only systems, we are done with TradersStudio. We need to move the data over to Excel for analysis and the e-ratio computation.
Copy the custom trade report from the short-only system run to one sheet and from the long-only run to another sheet (watch out for the number of rows: if you have more than 65,536 and a version of Excel earlier than 2007 you will need to do that in several chunks).

Step 5: Run e-ratio macro

Copy the macro code from the text file, create a new VBA module in Excel and paste the code in it.
Go back to both sheets and run the macro. This should manipulate the data and create an aggregation pivot table on a separate sheet.

Step 6: Compute the e-ratio

On a brand new sheet, copy both pivot tables (I recommend a paste/special – values only) side by side so that each trade length values corespond with each other. Sum each Total value (column D and J) together for the same Buy and Sell parameters (in column N in example file). Divide each Sum of MFE by Sum of MAE: this gives you the e-ratio for the given trade length.

THE CODE

Please find below code files to support the example below (all txt files except the Excel workbook). This should be easily portable to any system (i.e. just change entry criteria in the system).
Donchian Channel Indicator
Custom trade Report
Buy System
Sell System
Excel e-ratio macro (text file)
Excel example workbook (does not contain underlying raw data as it is 60MB!)

Or if you prefer a zipped version of all files – here it is.

Any questions, clarifications, etc. please let me know how I can help.

This is the list of my real-life library dedicated to building automated trading systems. This is not the exhaustive list: I have more specialised books which I will add at a later stage (some I have not read yet…). But the collection below is a decent starting list:

Please feel free to suggest your book recommendations or any additional info using the comments section below.

Please find below the Library organised in several sections:

• TRADING SYSTEMS
• MONEY MANAGEMENT
• TREND FOLLOWING
• TRADERS INSIGHTS
• MARKETS, RISK AND CHANCE
• OTHER

– TRADING SYSTEMS –
	New Trading Systems and Methods – Perry Kaufman An encyclopedic (1,000+ pages) review of Systems Ideas and Methods –No review/summary yet– Look up on Amazon US | CA | UK
	Quality Money Management – Benjamin Van Vliet & Andrew Kumiega Best practices in Systematic Trading Technology. Build it with Quality! –No review/summary yet– Look up on Amazon US | CA | UK
	Trading Systems and Money Management – Thomas Stridsman Presents a strong approach to developing Trading Systems integrating Money Management concepts –No review/summary yet– Look up on Amazon US | CA | UK
– MONEY MANAGEMENT –
	Money Management Strategies for Futures Traders – Nauzer Balsara Complete introduction to the essential concept of Money Management. Check my review/summary Look up on Amazon US | CA | UK
	The Handbook of Portfolio Mathematics – Ralph Vince Essential reading for Optimal Allocations and Leverage! –No review/summary yet– Look up on Amazon US | CA | UK
– TREND FOLLOWING –
	Trend Following – Michael Covel See Why and How Trend Following works. Over a dozen of top CTAs covered. –No review/summary yet– Look up on Amazon US | CA | UK
	The Complete Turtle Trader – Michael Covel The Turtle story and their complete rules –No review/summary yet– Look up on Amazon US | CA | UK
	Way of the Turtle – Curtis Faith The Turtle experiment from the inside – told by a Turtle! –No review/summary yet– Look up on Amazon US | CA | UK
– TRADERS INSIGHTS –
	Market Wizards – Jack Schwager Legendary interviews of top traders. Inspirational and “motivation-kicker”! –No review/summary yet– Look up on Amazon US | CA | UK
	The New Market Wizards – Jack Schwager Schwager strikes back! You’ll re-read him often. –No review/summary yet– Look up on Amazon US | CA | UK
– MARKETS, RISK AND CHANCE –
	The (mis)behavior of Markets – Benoit Mandelbrot A fractal view of Risk, Ruin and Reward –No review/summary yet– Look up on Amazon US | CA | UK
	The Black Swan – Nasim Taleb Taleb re-writes the rules of risk. An excellent perspective on conventionally accepted concepts –No review/summary yet– Look up on Amazon US | CA | UK
	Fooled by Randomness – Nasim Taleb The hidden role of Chance in Life and the Markets –No review/summary yet– Look up on Amazon US | CA | UK
	The Drunkard’s Walk – Leonard Mlodinow The hidden role of Chance in Life and the Markets –No review/summary yet– Look up on Amazon US | CA | UK
	A Random Walk Down Wall Street – Burton Malkiel The original classic claiming Efficiency in the Markets –No review/summary yet– Look up on Amazon US | CA | UK
	A Non-Random Walk Down Wall Street – Andrew lo & Craig MacKinlay Collection of academic papers on Efficiency Market Theory –No review/summary yet– Look up on Amazon US | CA | UK
	When Genius Failed – Roger Lowenstein How mis-understanding of Risk and Money Management drove LTCM to their graves. Fascinating! –No review/summary yet– Look up on Amazon US | CA | UK
– OTHER –
	Schwager on Futures Managed Trading – Jack Schwager In-depth investigation into the Managed Futures/CTA industry –No review/summary yet– Look up on Amazon US | CA | UK
	Starting Out in Futures Trading – Mark Powers Learn the ropes of Futures Trading: a Beginner’s reference. –No review/summary yet– Look up on Amazon US | CA | UK
	DeMark Indicators Jason Perl The clearest presentation of all TD indicators –No review/summary yet– Look up on Amazon US | CA | UK