How many things can you name whose price has dropped from $2.7 bn to $300? If you didn’t guess that, the answer is DNA sequencing. Welcome to the world of data abundance. 

If you’re like me, you probably go blue in the face if somebody says to you one more time that data is the new oil (or is it sunlight?) —and big data sounds terribly hollow when you know organisations still don’t get little data right. Like my bank which just a couple of years ago sent my new credit card to my old address along with the PIN number, despite sending the statements to the right address. Nonetheless, it is a fact that data is changing the world, and organisations, in a number of deeply transformational ways. Data is giving us answers to questions we never thought could be answered. Like ‘when did Robert De Niro stop giving a ****?‘

First, we’re seeing the ongoing datafication of businesses. This is akin to electrification of the second industrial revolution. Every mechanical and steam-powered device was being electrified then, from pumps to railway engines. Today’s equivalent of that is we’re datafying every process and activity. Be it a hiring decision or maintaining a complex machine – we are converting a lot of implicit decisions to explicit ones. To use a hypothetical and trivial example, a screen on your wardrobe cupboard which always showed you the weather and your meetings for the day, and gave you recommendations on possible work-wear options. As  I said, a very trivial example, but you could apply the same logic for how you find a date, or whether there is institutional bias against certain types of workers. This, in turn, is creating an entirely new data value chain and the shift of power from the old to the new. But this new data world poses entirely new challenges. Here are some. 

I met with a client recently and they collect huge amounts of data about every part of their business. Yet, when I asked about their data culture, it drew a wry smile from my counterpart. In most organistions, this requires one big behavioural change. Moving to a data culture is very hard for people who have trusted their gut and experience for much of their lives. Trusting the data when it’s saying something different from what you think is right takes a big mind shift. 

Many years back I heard a talk about a project where a highway maintenance team was given devices to monitor the vibration levels on their drills and alert them when this level was too high (health hazard). And now the foreman of the team had a problem he didn’t have before. He had to make a decision when the device alerted him, potentially creating a cost, wasted effort, and a liability. In the work we do in healthcare, this has come up as well. Additional data creates a decision imperative where it didn’t exist earlier, and we need to make sure people are equipped to make those decisions. Here’s how it plays out in the healthcare space in the US. 

Move one step forward from there and we get to decision automation, where we train machines to make these decisions. AI is increasingly becoming factory-fied —yet another reference to the industrial revolution. It’s even been called an assembly line or an ‘AI-ssembly’ line. Expect to see more and more of these decision systems popping up. Although there’s been a lot of talk about biases in AI systems, the good news is that unlike humans, these biases can be identified and fixed. Humans have plenty of biases too, but try fixing them! 

Looking ahead, one of the challenges for businesses and institutions will be to re-organise themselves around the data, and data processes. We’ve seen the introduction of Chief Data Officers. I expect to see increasing numbers of data specialists in every department, division, and team. And then to move to a completely new structure built around data processes rather than customers, or finances, or products. After all, businesses are fundamentally mechanism for returning on investments, allocating resources, and connecting products and services to demand. All of which are data-driven decision domains. The corollary to this is the rise of data nationalism. Are you ready for data politics? 

A word of caution —it’s not always what it looks like. My favourite story comes from the second world war where decisions were being made on how to add armour to planes. The data was straight forward. There were many more bullet holes in the wings than in the engine area, from the planes that came back from sorties. That was until Abraham Wald pointed out that the planes with bullet holes in the engines weren’t coming back. Data tells us many stories, but some of the narratives aren’t the obvious ones.