The Risk Numbers on a Fact Sheet, and What Each One Misses

Five numbers, one price series

Kabir is looking at a fact sheet with a row of risk statistics on it. Every one of them was computed from the same thing: a series of past prices. None of them contains any information that is not in that series.

That is worth holding on to as you read what follows. These are different summaries of one history, and the reason they disagree is that they summarise different features of it. This page is a reference for what each is counting. The prior question — whether movement is the same thing as risk at all — is risk against volatility, and it should be read first.

Standard deviation

What it counts: how far returns typically sat from their own average.

It is the default meaning of "volatility". A large figure means the results were spread out; a small one means they clustered.

What it misses: direction. A holding that regularly jumps upward is scored as risky by exactly the same amount as one that regularly falls, because the measure squares the distance from average and stops caring which way it went. That is often not what a reader means by risk.

It also assumes the spread is a stable property. Periods of calm and periods of turmoil are averaged together into one figure that describes neither, and the figure computed over a calm stretch will understate what happens next time.

Beta

What it counts: how much the holding has moved when the market moved, on average, expressed as a multiple.

Roughly: above one, it has amplified the market; below one, it has dampened it.

What it misses: everything that has nothing to do with the market. A holding can have a low beta and be extremely risky in ways the market never caused — a concentrated position in one company has plenty of risk that is specific to that company, and beta by construction ignores it.

It also depends completely on which market you measure against. Beta against a broad index and beta against a sector index are different numbers describing the same holding, and the fact sheet may not say which was used.

Maximum drawdown

What it counts: the worst peak-to-trough fall in the period, as a percentage.

Of the five, this is the one closest to what a person means by risk, because it is expressed in the units of the actual experience: how much did it fall, from the best it had been to the worst.

What it misses: it is a single observation. One number from one episode, and specifically the worst thing that happened to have occurred within the window chosen. A longer window will usually produce a worse figure, so comparing two holdings over different periods is meaningless. It also says nothing about how long the fall lasted or how long recovery took, which is the part people actually live through — drawdown and recovery time separates those.

Downside deviation

What it counts: the same thing as standard deviation, but only for the periods that fell below some threshold.

It exists precisely because of standard deviation's blindness to direction. Only the disappointing outcomes are counted, so a holding that rises sharply is not penalised for it.

What it misses: the threshold is a choice, and it changes the answer. Zero, the deposit rate, or the average return will each produce a different figure and there is no universally correct one. Restricting attention to bad periods also means fewer observations, so the estimate is noisier than the standard deviation computed from the same data.

Tracking error

What it counts: how much the holding's return has differed from a benchmark's, period by period.

It is not a measure of risk at all in the ordinary sense — it is a measure of difference. A fund that tracks its index almost exactly has a low tracking error and may still be a very risky holding, because the index itself is risky.

What it misses: whether the difference was good or bad. It treats beating the benchmark and lagging it as the same thing. And, confusingly, it is not the measure of what an index fund cost you — that is tracking difference, a separate quantity with a nearly identical name, distinguished in tracking error against tracking difference.

The two questions to ask about any of them

Over what period was it computed? Every one of these figures depends entirely on its window. A window that excludes a crash produces reassuring numbers for a holding that would have been savaged by one. Two funds with different windows cannot be compared at all, and fact sheets do not always make the window prominent.

Against what was it measured? Beta, tracking error and downside deviation all require a reference — an index, a benchmark, a threshold — and the reference was chosen by whoever produced the figure. Change it and the number changes without the holding changing.

Those two questions dispose of most misuse of these statistics, and neither requires any technical knowledge.

What none of them capture

Three risks are invisible to all five, because none of them appears in a price series.

Illiquidity. Something that is rarely priced shows low movement on every one of these measures. The stillness is an absence of observations, not an absence of risk.

Rare and permanent losses. A risk that has not yet occurred in the measurement window has a measured value of zero on every metric here.

Anything about you. None of these knows when you need the money, what else you own, or whether you would sell during a fall. Since risk depends on all three, no number on a fact sheet can be your risk.

What to take away

Treat these as five different summaries of one price history, useful for comparing holdings measured the same way over the same window, and useless for telling you whether something is suitable for you.

If you read only one of them, read maximum drawdown, because it is in the units of the experience. Then find out what window it came from — and remember that the worst thing that happened in the window is not the worst thing that can happen.