You are at the doctor, and you have just been told the disturbing news that you tested positive in a routine blood test for a disease you know is rare but typically fatal. But surprisingly the doctor has suggested that you should not worry – the test is ‘only’ 95% accurate. 

Seeing your horrified expression, the doctor clarifies: ‘the disease is very rare so you still only have a very small chance of having it. The test is most likely a false positive. We’ll do some more checks but you’re probably fine.’ You leave the surgery sort of relieved but also perturbed, unsure of the doctor’s logic. It’s hard to get over the fact that 95% seems pretty damn high.

The ability to identify important sources of information and correctly combine them in a probabilistic framework is an important skill for an investor, yet it’s a skill that we’re not naturally much good at. While our ability to quickly recognise patterns or signs of danger has been honed through thousands of years of evolution, probabilistic thinking is a relatively newer cognitive innovation. Some important aspects of it can be unintuitive or oblique, with the result that our instincts lead us to systematically make errors.

Base rate neglect is an important and common type of these errors, where people tend to unduly place excessive weight on specific (and often newer) information sources at the expense of broader, more general information. In the example above, the natural instinct is to treat the positive test result as superseding prior knowledge that the disease is rare. In fact, since the test is not 100% accurate, both sources of information are relevant. Since the test only has 95% accuracy, unless more than one in 20 people taking the test actually have the disease, more positive test results will come from false positives rather than true positives.

Base rate neglect is pervasive. Various studies have shown that when people are asked to make simple predictions based on general and specific information presented to them, many (often the majority) make the error of neglecting the general information. These studies often involve the people whose job it is to make exactly the kind of prediction in question (e.g. doctors in the example above). But they only really pick up the tip of the iceberg. The more serious impact of base rate neglect is that we don’t even look for the right general information in the first place, let alone account for it correctly when it is put in front of us.

This links closely to a previous post I wrote on the book ‘Superforecasting’, looking at what makes some forecasters better than others. One of the key points picked up was that better forecasters would always start by considering base rates. To predict the likelihood of an event occurring, better forecasters would always first take the ‘outside view’ to determine the base rate of that type of event happening, before then taking the ‘inside view’, making adjustments to the base rate for specific factors. For example, to work out the probability of a certain conflict occurring (within a particular time frame), better forecasters would first look at how often that sort of conflict had occurred in the past, resisting the temptation to leap into the facts of the specific political situation at the time.

Base rate neglect and investing

It’s not immediately obvious how base rate neglect applies to investing in practice. In principle, base rates in investing should be factors that predict success (outperformance) for some types of stocks while underperformance for others. In practice, the relevant base rates are less obvious and more nuanced than in the simple example above, where the base rate is more obviously how often the disease occurs in the general population.

Despite the fact that the relevant base rates are not obvious, the same broad principle applies. We are biased to focus much more on specific information and to neglect looking or accounting for more general evidence about what sort of investment tends to work out better on average. This is evident in the sort of information available to a private investor. There is an overload of readily available information about individual stocks: financial reports and statistics, presentations and opportunities to talk to company management, tips and commentary by analysts and the media on the prospects of specific businesses. There is much less evidence readily available on base rates, e.g. statistical studies on what sort of characteristics tend to lead certain businesses to do better than others. One issue is that this kind of evidence requires time, effort and careful analysis to produce and those that have bothered to do it may be keen to keep their valuable intellectual property to themselves. More importantly, most investors just aren’t really looking for it. The good news about this lack of attention is that it means investment opportunities with very high base rate probabilities of success can persist, rather than getting arbitraged away.

Base rate neglect implies that for most of us there should be relatively more to be gained from focusing on general information that puts the odds in our favour, and relatively less to be gained from more detailed research that attempts to predict the future of a specific business with high precision. However, taking advantage of base rates is not straightforward. It requires first identifying what the relevant base rates are and then combining them with more specific information in a sensible probabilistic framework.

What are the right base rates?

Identifying the right base rates is in my view one of most important things an investor can do to be successful. The challenge is to identify categories of opportunities that are likely to outperform on average.

Sometimes what the relevant base rates can be fairly obvious. This is the case particularly in situations where the success of the opportunity is dependent on a discrete outcome. For example, this could be whether an oil explorer successfully digs a new well or whether a pharmaceutical company is successful in gaining approval for a new drug. The relevant base rate here is how often these events have a successful outcome in general for that category of opportunity. For the examples I mention, the base rates are very low – the chances of a successful oil well or the chances of a new molecular entity (NME) drug being approved through all phases are both less than 10% on average. Investors tend to pay less attention to these base rates and much more on the story of why the particular opportunity they are interested in is likely to be successful. As a result, in general these types of business are systematically overvalued and make very poor investments. Most of us are best off employing a bargepole and avoiding them altogether.

More broadly, the study of factor investing has identified several characteristics of shares that have consistently led to outperformance over time: momentum (e.g. the price change over the previous year), value (e.g. the price to sales ratio), quality (e.g. ROCE), small caps etc. There is also research that businesses from certain sectors of the economy tend to perform better than others over the long term: investments in defensive growing sectors like consumer staples and healthcare tend to do better. A good starting point for any investor is to employ an approach that gets these base rates onside.

The research and evidence supporting the outperformance of factors is largely limited to looking at characteristics that are measurable. However, in practice the characteristics that identify the best types of business can often be less measurable, requiring an element of qualitative assessment. One of these is whether the business has a competitive advantage (the so-called ‘moat’). Many successful investors have argued that the competitive advantage is an important predictor of a business’s success (and consequently the success of its share price). Assessing a competitive advantage can involve multiple information sources and require an element of judgment. This more qualitative type of assessment starts to stray away from the general and more into the company-specific. As a result, it becomes harder to think of it as relating to a base rate. This distinction is somewhat artificial – there is not a clear dividing line between what is ‘specific’ and what is ‘general’. The more fundamental lesson from base rate neglect is that it is important to treat each information source appropriately in a way that correctly reflects its value as a predictor and the underlying uncertainty. Evidence of a strong competitive advantage should indicate a type of company for which you think there is a higher base rate of success, rather than necessarily indicating success is especially likely in that particular case.

Combining information

A probabilistic framework is especially important as investing typically doesn’t involve one source of information or factor, but many that can be combined when making an overall judgment. This includes both general information relating to base rates for the type of business involved and information only relevant in the context of the specific opportunity. Combining different sources of information well to make this judgment is  very difficult. There are several reasons for this:

• Base rates in investing are extremely difficult to quantify: while you may have a good insight that shares with positive price momentum are likely to outperform on average, it is very difficult to quantify by exactly how much. Not being able to measure base rates means you can’t just do the maths to work out the probability of success, but need to apply a degree of judgment.
• Information sources may be correlated: the additional predictive power that comes from a second source of information (or base rate) falls the more correlated it is with the first. The more correlated it is, the less of the content of the information is new. For example, a measure of momentum (e.g. 1 yr price change) has less incremental value when combined with another measure of momentum (e.g. 6 month price change) than when combined with a measure of valuation. The relationship between different sources of information can be complex and opaque so correlations are often hard to observe. To get a better sense of how much additional value derives from different sources of information, it helps to group information sources into less-correlated dimensions (e.g. value, quality, momentum).
• Information sources may be interdependent: some factors may apply differently to different groups of businesses (or in different time periods). For example, some valuation measures may work much better for some types of business than others. Because of this, the fact that a characteristic predicts outperformance on average does not necessarily imply it does so for a specific subset of investment opportunities.

Luckily, to invest successfully you don’t need to make very specific predictions about the share price, but rather just a judgment of whether it is likely to outperform over time. The issues above make this judgment more difficult, but not impossible. They suggest there is likely to be quite a large benefit to specialising. This allows you to focus on identifying situations where you are more confident that the underlying base rates are in your favour and develop a better grasp of how additional information adds value. The approaches of very successful investors more often than not revolve around identifying a particular type of opportunity with high odds of success, rather than applying judgment to all sorts of different opportunity.