In portfolio theory, the value of a security is related to the way it interacts with other securities. For example, a security that offers a positive pay-off in a state of the world in which everything else is losing money could be more valuable, even if its expected return is modest. Correlation, the essence of this interaction, is thought of as something structural that we can estimate to create optimal portfolios in the real world. However, there is a strong case to suggest that estimates behave very differently from how we might expect and, as a result, the efficacy of portfolio optimisation and, indeed, risk management, can be severely diluted.

The nature of many variables in financial markets is constantly changing, so true correlation coefficients may be ephemeral and are almost certainly elusive. A paper by Giacomo Livan, Jun-Ichi Inoue and Enrico Scalas (available here) offers an insightful empirical examination of this. The authors calculate the correlation coefficient for 4 securities (6 distinct pairs) over an expanding window of time and compare the results with simulated data.

The first graph below shows the correlation coefficients calculated from the actual time series of the securities’ returns, starting with 200 days and increasing in 10 day increments to a total of 1758 days. There is a lot of variability and, importantly, the variability is not centered on the full-sample coefficient (the horizontal lines).

If the securities had a true correlation coefficient and a large sample gave a better estimate of the truth the graph should look like the graph below.

There is more variability at the start but a clear tendency around the truth (the horizontal lines). The variability reduces as the sample size gets bigger.

Portfolio theory assumes the world looks like graph 2. Giacomo et al. conclude that these graphs “drastically defy the common knowledge that longer time series should eventually produce better correlation estimates”.

So we do not know the true correlation coefficient and cannot reliably estimate it empirically. Therefore portfolio weights calculated based on these estimates may only be optimal during the specific sample period and will almost surely not be outside of that. This result also has implications for risk management tools, such as Value-at-Risk, which rely to a similar degree on having estimates of correlation that accurately describe relationships on a prospective basis.

This point needs further examination by both practitioners and academics, as it is crucial to the foundations of finance. Users of quantitative methodologies in finance need to first examine the stability of relationships before assuming that any implicit theoretical hypotheses hold true.