Some people take risks. Investors and traders take uncertainties. This awkward turn of phrase draws on the distinction between risk and uncertainty made clear by Frank Knight over 90 years ago. Only when the precise probability of a random event occurring is known can we call something a risk. Otherwise it is an uncertainty.

The strap line for the expectingreturns.com website mentions uncertainty for exactly this reason. If pricing capital in the face of risk was all that there was to finance, this website would be as useful as running a regression on a bus timetable.

In one of the best papers I have read in recent years, Professor of Finance Andrew Lo and Physicist Mark Mueller, explore different levels of uncertainty. While they acknowledge that certainty and uncertainty lie along a spectrum, the categories they propose are useful in framing discussions about what we can and cannot know and predict.

Level 1 is Complete Certainty. In the context of this discussion it ‘s quite uninteresting – but it is useful to keep such things as the effects of gravity in mind when going about your daily life.

Level 2 is Risk, and is consistent with how Knight thought about it. Probability distributions are known (with certainty). It is debatable whether this exists in reality, but it is most useful in serving as a base for Level 3.

Level 3 is the first of the “uncertainties”: Fully Reducible Uncertainty. With enough data and knowledge of statistical methods, events in this category can be “rendered arbitrarily close to Level 2”. This makes working with them quite straightforward. Examples include games of chance, heights of people and train arrival times. For practical purposes, events in this category can be called risks.

Level 4 is Partially Reducible Uncertainty. Here the data generating processes can, for example, have stochastic parameters, complex non-linear functional forms or other properties that greatly reduce the utility of statistical methods. This is the level with the most relevance for financial analysis. We can understand processes up to a point, beyond that we use a incomplete mix of tools and techniques to figure out what’s going on. But this opens up an intricate web of signals and noises. We can back-test hypotheses that work well in sample but don’t out of sample. We can convince ourselves and others of crazy rules governing the cost of capital. Ultimately, though, being aware of the nature of Partially Reducible Uncertainty and formulating and sticking with a good strategy is the best way of dealing with it.

Level 5 is Irreducible Uncertainty. This has relevance for financial analysis too but it is much more difficult to work with. If you can develop an investment strategy that is not built on trying to guess the answer to unknowable states you’ll be better off. Lo and Mueller describe Irreducible Uncertainty as a “polite term for a state of total ignorance”. But that doesn’t stop us from getting having strong views on topics in the fields of philosophy, religion and politics (in fact Darwin argued that ignorance is a good starting point for having immovable opinions). Beware of the investor who only talks in unfalsifiable terms.

A lot of people working in finance seem to behave as if the world stops at Level 3. Risk managers spin elaborate models which portfolio managers latch on to in an effort to reduce cognitive dissonance. Some investors look for patterns and trading rules, others buy into the latest fad from the brains of rocket scientists. Lo and Mueller have set out a lexicon and framework that (1) can be used to make people aware of the shortcomings in a variety of financial tools and (2) lays the foundation for a more informed discussion of risk and uncertainty in financial markets.

Correlated Errors

In portfolio theory, the value of a security is related to the way it interacts with other securities. For example, a security that offers a positive pay-off in a state of the world in which everything else is losing money could be more valuable, even if its expected return is modest. Correlation, the essence of this interaction, is thought of as something structural that we can estimate to create optimal portfolios in the real world. However, there is a strong case to suggest that estimates behave very differently from how we might expect and, as a result, the efficacy of portfolio optimisation and, indeed, risk management, can be severely diluted.

The nature of many variables in financial markets is constantly changing, so true correlation coefficients may be ephemeral and are almost certainly elusive. A paper by Giacomo Livan, Jun-Ichi Inoue and Enrico Scalas (available here) offers an insightful empirical examination of this. The authors calculate the correlation coefficient for 4 securities (6 distinct pairs) over an expanding window of time and compare the results with simulated data.

The first graph below shows the correlation coefficients calculated from the actual time series of the securities’ returns, starting with 200 days and increasing in 10 day increments to a total of 1758 days. There is a lot of variability and, importantly, the variability is not centered on the full-sample coefficient (the horizontal lines).


If the securities had a true correlation coefficient and a large sample gave a better estimate of the truth the graph should look like the graph below.


There is more variability at the start but a clear tendency around the truth (the horizontal lines). The variability reduces as the sample size gets bigger.

Portfolio theory assumes the world looks like graph 2. Giacomo et al. conclude that these graphs “drastically defy the common knowledge that longer time series should eventually produce better correlation estimates”.

So we do not know the true correlation coefficient and cannot reliably estimate it empirically. Therefore portfolio weights calculated based on these estimates may only be optimal during the specific sample period and will almost surely not be outside of that. This result also has implications for risk management tools, such as Value-at-Risk, which rely to a similar degree on having estimates of correlation that accurately describe relationships on a prospective basis.

This point needs further examination by both practitioners and academics, as it is crucial to the foundations of finance. Users of quantitative methodologies in finance need to first examine the stability of relationships before assuming that any implicit theoretical hypotheses hold true.