It might not capture the imagination as much as the recent Haye v. Valuev WBA World Heavyweight Championship fight (it probably might for some of you… ;-) but I decided to organise my own “fight”: AmiBroker V. TradersStudio!
And similarly to the boxing, speed was of the essence – with one platform completely out-performing the other one. Let’s find out which one…
In the last few posts I have been exploring the e-ratio and wrote the code in TradersStudio. I also tried to implement it in AmiBroker AFL language mostly to check how it performed speed-wise – as I found TradersStudio quite slow.
While testing AmiBorker, it actually felt so fast that I decided to perform a more formal speed comparison test.
Conditions of the test
I exported the CSI data via the API for a proportionally back-adjusted Corn contract going back to 1949. The data was imported in both AmiBroker and TradersStudio via their ASCII Import.
The system tested is a simple 20-day Donchian Channel Breakout (Buy-only) and the ATR used to normalise the MAE/MFE is 20-day also.
The optimiser in both systems were used to generate the e-ratio for
50 51 different trade durations (from 10 to 50 days).
One of the condition to validate the results of the test was that the trades generated by both systems are similar (to double-check I did not make a coding mistake resulting in simpler/faster process for one of the platforms).
The computer I was running the test on is a quad-core CPU (2.4 GHz) with 3.25 GB of addressable RAM. Only one platform was running at the time it was tested.
Results are in!
And they look astoninglishly good for AmiBroker:
- The TradersStudio test was run first and took 4 mins 15 sec to complete and produce the custom report (which needs to be manipulated in Excel to calculate the e-ratio). Furthermore, it appeared that every incremental run in the optimisation process took slightly longer than the previous one.
- The AmiBroker test ran in (…drumrolls…) 10 sec! And the e-ratio was readily available from the results grid:
Both apps maxed out their allocated CPU (i.e. overall CPU usage of 25% = a quarter of the quad-core CPU available).
I then decided to run the AmiBroker code over 5 markets to get a feel of how long it would take. I added Crude Oil (going back to 1985), Cotton (1968), Gold (1975) and Yen (1972).
AmiBroker ran the same code + optimisation in 2 mins 20 sec, twice as fast as one market in TradersStudio!
Applying a simple proportional calculation to derive the time it would take in TradersStudio for the same dataset would give us a completion time of 1 hour (59 min 30 sec exactly…). Since the performance seems to degrade over the course of the optimisation in TradersStudio (i.e. the first back-test is quicker than the last one of the optimisation run) we could assume that it would actually take longer, which is what I experienced when I was testing on similar data in the last few weeks.
So here you have it: AmiBroker is 25 times faster than TradersStudio and it gives you the results in a much more friendly format.
As mentioned earlier, to ensure that the test was valid, the set of trades generated by both systems should be compared. I ran one back-test of the system over the Corn data for a single optimisation step (e-ratio for 20 days duration) in both systems.
I could find that 296 trades out of 340 presented a very good match (some matches had one day difference on the Entry date and rounding differences on the prices). But overall the trade comparison was good enough to give assurance that the same systems and results were being tested.
I’ll keep investigating for myself where these errors are coming from (and post a further code update if warranted) but we can assume that the differences are trivial with regards to performance comparison.
Did I mention that AmiBroker is 3 times as cheap with a much wider following (many more code samples, better docs, more forums, etc.)?…