Insight hidden in plain sight

This is a piece about things getting missed. It’s about what happens when organisations don’t get the insight about their world they need or they don’t see what’s staring them in the face. On the face of it the first problem might look like one that is well within their control and something they can fix. The second presents as more of a cultural issue – a form of organisational myopia. The truth, lies somewhere in-between. The data an organisation rely on, and its ability to make sense of them, are both deeply cultural.

This matters. Organisations exist based on a particular understanding of how things work and what their role in the order of things is. If, over time, the world changes but the organisation doesn’t it becomes misaligned. When a business or entire industry gets ‘disrupted’ it is because it is acting according to already-existing understandings of the world.

Confronting disruption requires organisations to disrupt their conventional understandings of the world. To assist organisations with that requires helping them grapple with how they think.

First, let’s set the scene with a little background on two companies. Both have misread or failed to act on changes in people’s behaviour.

Every little (bit of big data) helps

British supermarket Tesco’s problems are well documented (I especially like this account which fits the thrust of this post). There was a time when every article about the supermarket chain referred to the fact that £1 out of every £7 spent in the UK went through its tills. Tesco’s fall from grace has been as swift as it has been significant. It went from unstoppable giant to victim in the space of a couple of years (though it’s still by some distance the UK’s larger retailer).

According to most commentators Tesco failed to spot and respond to two changes in its world. First, Aldi and Lidl, discount retailers started to undercut it. Second, it’s customers shifted from doing weekly shops – for which its out of town stores are designed – and started doing more frequent, smaller top-up shops.

Other factors are at play: Tesco had become over-bearing and slightly disliked. It also missed the fact, despite the food chain scandals, that people were taking more interest in the quality and origins of their food. The stores are large refrigerated hangars when people pined for something slightly more intimate.

What’s fascinating about Tesco’s decline is it’s a company credited with pioneering data analytics on a colossal scale through its ‘Clubcard’ loyalty scheme. Tesco worked with, and then bought, DunnHumby, a company that gave them a powerful way of understanding their customers. This scheme, at least in theory, gave them the power to tease insight out of 1 out of every 7 pounds spent in the UK. So how is it that they have fallen foul of  shifts in behaviour that must have been at least partially evident?

We’re loving it (less)

Meanwhile, McDonald’s is facing strong headwinds in the US, as this FT piece – Eating their Lunch – piece revealed (and further grim trading news recently). In summary, they have “been caught flat-footed by a dramatic shift in consumer tastes”. Consumers are becoming more interested in the origin of their food, continue to worry about obesity and have consequently shifted to healthier ‘fast casual’ restaurants. Chipotle is on a tear as McDonald’s numbers decline, especially in the US, its biggest market.

The iconic burger chain has been here before, and course corrected. But again, the question emerges: how is that a business which likely spends tens of millions of dollars a year on research miss these shifts?

The choice of Tesco and McDonald’s is arbitrary. There are examples aplenty of companies missing big things. Intel and the mobile revolution comes to mind. And that example helps makes an important point:

There’s a difference between missing a shift / trend and not being willing to hear it and then deal with the cognitive dissonance that accepting it might throw up.

Data, Data Everywhere

These examples pose two intriguing questions about these organisations:

1. Have they been reliant on the wrong sort of data, or information that is at best misleading?

2. Do they have good data but an inability to see or hear what it is really telling them?

Given my precise knowledge of what research and data inputs these businesses have is limited it’s not appropriate to judge what they know. Big data certainly has its critics. One could argue that disembodied quantitive data is not conducive to an empathetic understanding of the world. It may – or may not be the case – that much of the research they procure is not up to scratch. But to my mind that’s a different discussion.

I’m more interested in the idea that the problem may be a more cultural one. By this I mean I’m interested in an organisation’s ability to respond to ideas that threaten or run counter to its worldview. Secondly, I think it’s important to look at the relationship between individuals and organisations and how both come to understand the world.

See no insight, hear no insight

To return to the beleaguered grocer Tesco. It’s likely that both its big data and its qualitative research was throwing up shifts in people’s behaviour. It’s also likely that people in the business had spotted that shopping trips were becoming smaller but more frequent. The internal teams might have also triangulated various data that pointed to a rising concern with food origins or quality.

But what’s now becoming clear about Tesco is that it was committed to a very specific way of doing business. Its land banks were an obvious sign that it was committed, long term, to a big box, out of town model of shopping. Its somewhat notorious relationships with suppliers also signalled a different set of values in terms of the value-quality equation.

In other words, in thought, word and deed ‘Tesco as organisation’ was committed to a retail model that was starting to look at odds with what the data was saying. What’s the half life of insights that challenge that organisation’s way of seeing the world? What of the career prospects of those pushing those challenging views?

In what is allegedly a dominating culture it may have been difficult for the data – and its representatives – to speak truth to power. There may not have been a culture that supported the sharing of uncomfortable truths. Even if challenging views were shared, it’s likely that diagnoses of the issues, and possible solutions, were being laid down according to the pre-existing cultural tracks of the organisation: they were trapped in already-existing views of Tesco’s world. The commitments to large stores, and conventional ways of framing supplier relationships etc became the frame for thinking about the problem and the solution. If you have a large land bank from which to develop out of town superstores it’s difficult to escape the conclusion that these land banks and new superstores are part of the solution to your problems.

How do individuals and organisations think?

Anthropologist Mary Douglas – who spent many years pondering ‘how institutions think’ – would suggest that thinking in terms of individuals and organisations as separate entities might be wrong. In her view:

“it is not our individual minds that are making decisions at all, but our institutions that are doing most of the thinking” (How Institutions Think pg. 111).

Another way of putting this is that big commercial (and non profit) institutions are defined by conventions that limit the range of options that their individuals can come up with to solve our difficult problems and inform our decision making. There’s a host of more sociological observations one can make about large organisations and individuals – namely the distance of executives from the real world of their consumers, the ‘power distance’(Hofstede) and the sheer size and complexity of the organisations – but those are topics for another day.

What to do?

There are several implications for this way of looking at how organisations confront disruption, understand the world and make decisions:

1. The quality of the research going into a business matters. Good thinking is a necessary but on its own not sufficient condition for help organisations address disruption.

2. Attention to how an organisation thinks needs to be wedded to good research and insight work.

3. This means acknowledging that individuals in a business ‘see’ a problem and decide how to solve it in the light of where they are metaphorically ‘coming from’.

4. Well crafted ‘insights’, neatly arranged ‘smoking guns’ of evidence and water tight, logical arguments are all well and good. But if where an individual and their organisation are ‘coming from’ mean they are inhibited from taking the meaning and importance of an insight on board the most conclusive of cases will fail to have impact,


Coda – as always writing something like this in the midst of other pressures makes you realise how big and hairy the issues you’re raising are. And how many other writers have touched on them before. Beyond Douglas’ work, I think Only the Paranoid Survive by Andy Grove (Intel Founder) is worth reading. It’s great on idea killing. Also, Tim Harford’s discussion of Ford, McNamara and the Vietnam war in Adapt is also worth reviewing in terms of truth making itself known to power, or not.