How to Manage the Data Deluge

Data is more like water, not oil — too much of it can waterlog a business. How can an organization gain control of the volume? Here’s a five-step process that can help.

It is good to start with an admission of guilt. I’m guilty. I accuse myself and convict myself. I’ve done it repeatedly.

Yes, I’ve used the “Data is the New Oil” analogy many times.  And I finally have realized the folly of my ways. Sure, the analogy made for great speeches, superb power-points, and even better images. Sure, on the surface, the analogy is superb. Oil is a source of tremendous wealth and so is data; oil has to be extracted, refined, and marketed and so does data. I even remember using the pictorial concept of an oil rig gushing zeroes and ones to make my point. 

So why have I moved on? Well, I got to thinking one day about data and oil and realized that the analogy isn’t a good one because, simply, no country or organization ever said, “I have too much oil!”  But with data, the deluge can be negative, even paralyzing. With too much data and no infrastructure, process, and ability to quickly convert that data into wisdom and action, businesses can suffer. There is such a thing as too much data! 

A better way to go is to analogize data to water. That comparison stands the test. Water is, clearly, a source of real value (life after all depends on it). Water is necessary, but too much can kill. Too much water can render lush fields fallow and healthy bodies functionless. 

All analogies are fraught with imperfections and all have edge cases where they are inaccurate. But translating from the world of bits and bytes to “business-speak” requires analogies; so, let’s go with “Data is the New Water”.

Now, this begs several questions. How should the organization protect itself from waterlogging?  How should data be converted — quickly and at scale — into actionable wisdom? What are the necessary steps required to get to that level of maturity?

There are five factors that have to be invoked simultaneously in order to embark successfully on this journey. 

1. Decide. The executives of the organization have to make a clear decision, focused on becoming a data-mature and data-forward organization. It cannot happen willy-nilly.
2. Dedicate. Resources — money and personnel — must be dedicated to the upgradation of data infrastructure and culture.
3. Develop. The data-muscle must be developed.  Decision-making has to change based on the data now available — so a wholesale maturation has to be undertaken.
4. Dialogue. Being data-driven requires active, open, and honest dialogue within the organization. Data comes from everywhere and can be consumed everywhere, but only if both hands are coordinated.
5. Demonstrate. The transformation of infrastructure, culture, and process has to be openly demonstrated via clear test cases. Success or failure, notwithstanding, to move from rhetoric to reality requires demonstration.

Organizations are deluged by data today. What they do with it,