Do you have a data rubbish dump, or a treasure trove?

Back in 2018, IDC predicted that by 2025, the Global Datasphere would have grown from 33 zettabytes to 175 zettabytes. Arcserve predicted 200 zettabytes, and Statista 180 zettabytes. Now, with 328.77 million terabytes of data being created daily in 2024, Statista’s prediction looks to be on the money.

While that’s all very impressive, what’s probably of more interest to most of us is what form that data will take. Why? Because data falls into two camps. 1. Structured and immediately useful, and 2. Unstructured, and due to its raw, unprocessed, and often chaotic nature, challenging to utilise.

According to IDC, 90% of business data is unstructured – and consists of customer contracts, employee handbooks, product specs, video, imagery, IoT sensor data – and more. Only 46% of companies report that they analyse their unstructured data to extract value from it – and less than half of it at that.

The problem with unstructured data

Structured data, with its standardised format, is a low-hanging fruit, ripe for transformation into business insights. So, its value is readily appreciated.

Whereas, by its very nature, unstructured data isn’t easy to search or sort – and, more often than not, is sprawled across an organisation in siloes. But given the wealth (and breadth) of information it represents, it’s also immensely valuable – just harder to access.

Stockpiling unstructured data, with its sensitive customer, company, and employee information, has inherent dangers, starting with security. In its report on unstructured data, CIO Dive says, ‘idle data’ brings higher costs: “Costs associated with security breaches double for companies with more unstructured data.”

Then, there is the cost of storing unstructured data. Logic dictates that as storage costs grow, so must budgets for storage and management. With 38% of businesses already saying that data costs are too high or unpredictable, allocating a hard-won budget to data you will potentially never use can be a bitter pill.

Data sprawl is also the enemy of efficiency as employees manually input data from multiple, decentralised sources. Time and time again.

Given all the challenges, it’s no surprise that 88% of organisations agree that data sprawl makes data management hard and complicates implementing an end-to-end data strategy.

So, why hang on to it?

Despite the cost, few are willing to discard that precious (if underutilised) unstructured data – with most organisations saying that if the cost weren’t a factor, they’d like to keep their data longer.

But why?

• Never say no to an opportunity. You can count on the data you never got around to analysing costing you that edge you so desperately wanted. Or significantly improve the customer experience and, as a result, cement their lifetime loyalty. No one wants to be the person responsible for deciding to bin it.  
• Compliance caution. Unstructured data poses a massive compliance problem for many. Perhaps a big part of the problem is that 96% of organisations with mostly siloed unstructured data don’t know what information lies hidden in that sprawl (whereas of those who have centralised their unstructured data, 98% know exactly what lies beneath the chaos). The crux of it is that if you don’t know what’s in your unstructured data or where it is, you can’t be sure you’re effectively complying with the regulatory standards that govern your business. So, unless you have centralised your unstructured data and got to grips with what you have, it’s safer to hang on to all your data.
• AI (artificial intelligence). Harnessing the power of AI is an opportunity that forward-thinking organisations should ignore at their peril. However, if you’re already a convert, you need to get your unstructured data firmly under control with a centralised approach to content. As observed by IDC, while 84% of businesses are already using or exploring AI, “given that LLMs (large language models) are trained on unstructured data, IT leaders can only leverage the power of AI once they have a strategy to manage and secure their data on a single platform.”

Who in your organisation ‘owns’ all this unstructured data anyway?

Answer: Your CDO (chief data officer), aka chief data and analytics officer or just chief analytics officer. Whereas your CIO is typically more focused on technology, your CDO is charged with developing and implementing your data strategy.

While an evolving and relatively new C-suite role, the CDO mantra is ‘data-driven success.’ Part of the CDO’s role, says CIO.com, is to “‘break down silos and change the practice of data hoarding in individual company units.”

With IDC reporting that companies that used their unstructured data in the past 12 months experienced “improved customer satisfaction, data governance and regulatory compliance, among other positive outcomes,” the CDO role is a big step in the right direction.  

With most (93%) CDOs agreeing that AI success is a high priority, it’s no surprise that adding analytics and AI to their portfolio is regarded as a key step to success. As is driving value by transforming and curating data (both structured and unstructured), to make it easier to succeed with generative AI.

And as LLMs are powered by unstructured data, it’s clear that one person’s data rubbish dump is a CDO’s carefully curated treasure trove.

Contact us to have an obligation-free chat about our data management services.