Back Testing in Forex: Why It Doesn’t Work

Trading forex online is a risky endeavor and one of the main tasks of any trader is reducing the risk involved in trading decisions, says Vincenzo Desroches of ForexCharts.net

This is especially the case if we desire to test a strategy and not trade it. Nobody wants to take risks in a case where the most favorable outcome is breakeven. In this article, we’ll discuss the popular, but misguided back testing solution to the problem of risk-control in the testing of forex strategies.
 
Back testing is the application of some technical strategy to historical data, and the analysis of the resulting profit/loss patterns. Although it is done using computers for the most part, you can perform it manually on a sequence of monthly or yearly data. It is an easy and straightforward approach, which makes it very popular in the trader community as an exciting and safe tool in the everlasting quest for the perfect forex strategy. Traders who apply this method for testing their strategies subscribe to the belief that what works in the past will also work in the future. Historical performance is a guide to future results, they apparently believe, and can be used to build valid solutions to the perennial problems of trading.

Yet, back testing is almost completely useless as a guide to the underlying value or profit potential of a strategy. It provides no benefit or insight whatsoever to the trader with respect to the validity of his trading approach, and may often lead to some disastrous misconceptions about the correct practices in trading.

The basic reason behind the uselessness of back testing is very simple. It is not possible to devise a forex strategy on the basis of a piece of market data and then apply it blindly to another time period with the unjustified hope that it will function there equally well. Scientists assume that the market price action has the Markov property, which means that to guess the price of the next five minutes from now on, all you need to know is the price of now (since we need to know if the asset is priced at $50 or $5000 right now). Nothing else is relevant. Back testing, on the other hand, is built on the notion that prices in general influence each other everywhere and create patterns that are constantly repeated, contradicting this intuitive assumption. 

But let’s take a deeper look at back testing. 

What Is a Technical Strategy?

A technical strategy is a combination of analytical tools and aims to smooth out the volatility in the raw data, transforming it into a pattern that is easier to break down into basic formations such as triangles, ranges, or trends. In a typical piece of forex price data, such as the formations observed in the below chart, it is trivial to define a large number of formations, but such a large number of options does not make trading decisions easier. Instead, we have to choose a particular triangle, macro- or micro-trend, a long- or short-term support or resistance line in order to formulate our approach. This choice is more or less arbitrary. It is common knowledge that two experienced analysts can look at the same chart and reach conclusions that oppose each other.

To deal with problem, we choose a time frame for the overall price formation analyzed, and state the minimum price quantum, which will be the size of the smallest unit on the chart. Recall, however, that this choice is also arbitrary. We then use technical indicators to express an opinion on this formation; that is, the technical strategy that we apply to the developing price pattern creates a case scenario that is reflective of our opinion, not that of the market.

Click to Enlarge

The bearing that this discussion has on the subject of back testing should be obvious. Technical strategies are not like the theorems of mathematics where the results are independent of the assumptions of the person performing the calculation, since a technical trader is uniquely responsible for creating the scenario observed. It is often said that technical analysis is an art, or in other words, that the rationale that lead to the creation of a technical strategy by one trader cannot be duplicated by another trader merely by observation of the underlying data. The absurdity of testing an idea that lacks even a rigorous and commonly agreed definition of what it is, is self-evident. 

Although it is possible to test a strategy on the basis of the results it generates, assuming the causality of the correlation between trade returns and the observable match between the technical strategy and the price action is not meaningful. And if there’s no causality, it makes no sense to back test a random correlation because by definition, no repeatable patterns may result from the strategy subject to back testing.

NEXT: The Problem with Back Testing

The Problem with Back Testing

Yet quite apart from the flawed basis of the assumptions that justify back testing, there are practical and simple reasons that make the method flawed and inefficient. Even if it did work, back testing would require perfect, ideal conditions in order to yield results that would be valuable in real life. There must be no manual intervention in the execution of the trading plan, no connectivity issues, no software crashes, no panic or euphoria on the part of the trader to interfe with trading decisions before the conditions assumed by back testing materialize.

When one back tests a strategy, he needs to use the data stream supplied by a forex currency broker. Since there is no central forex exchange, there is no consensus price on a currency pair, such as the values that are available for stocks at DJIA, or other exchanges. As a result, the trader is in fact testing his strategy on the basis of the unique conditions that exist for the clients of one broker only. With any other firm, the experience would be very different, because of a large number of broker-specific issues. Competence, technical sophistication, suitability to various trading strategies, stability of the software, and reliability of liquidity providers are some of the myriad of factors that would easily differentiate the service and quotes of one firm from another. One cannot back test a strategy on one source of forex quotes and generalize from there to assume an intrinsic value for all brokers and markets. Back testing fails to function on this principle as well.

How Do I Test My Forex Strategy?

The best way of testing a forex strategy is trading it. But before that, one must do away with the notion that there are some perfect strategies somewhere awaiting discovery. Forex price action is generally regarded to be random; it doesn’t submit to any detailed, large-scale analytical framework that exists in a trader’s mind. Trying to find such a formula that can then be used to trade prices over months, years, or decades is a futile endeavor due to the chaotic nature of the market. The prices fluctuates, price trends change, but the rules that generate the trends change as well, and one can imagine a situation where one keeps discovering quasi-fundamental rules only to discover them to be a part of a larger, still changing framework of rules which itself mutates all the time. A forex trading rule, pattern, or strategy that governs the price action is always limited by its own time and place. And as such, the back testing approach, which aims to generalize these rules, is fundamentally flawed and must be abandoned rather than improved.

What the trader can do improve to his results in trading is focusing on understanding the mechanics of probability. Millions of people like to take part in lotteries, but very few know that the expected value of the outcome of a lottery is almost always negative (that is, by purchasing the ticket, you are entering a deal where you expect to lose money). Probability is mostly studied under the heading of money management in forex, and if you like to improve the returns on your trading strategies, the best way of achieving your aim is concentrating on this one topic strongly.

Conclusion

One of the most important rules of trading is that while the trader has no control at all over the market’s behavior, he has complete power over his own decisions. The novice tries to make a strong assumption about where the market is going, and makes a bet on the strength of his hypothesis, which is determined on the basis of criteria that are hypothetical themselves. A professional trader makes assumptions about the market action, but remains well aware of the ultimate value of technical prognostications. A very well defined, plausible trading scenario places the market under no obligation at all to obey its speculations. After all, one can always build a beautiful, coherent theorem about any subject on the basis of some given assumption (termed an axiom, in math). Yet natural phenomena have no duty to obey our conjectures; certainly we are the losers if we attempt to compel nature to submission.  

By Vincenzo Desroches of ForexCharts.net