Extra tags: data analytics
Authored by: CK Tan, Senior Director, Qlik
How’s your plumbing? Arguably, that’s never been a great conversation starter, but, if any time is appropriate, it’s now. Last year, IDC together with Qlik, did a sizable survey with decision-makers to find out where organisations are in their usage of data for insights, and how that ties to business outcomes. The lessons from that survey are still widely applicable today.
As part of Qlik’s 2022 Trends, we predicted that the traditional dashboard with static visualisations was over. Instead, dashboards will become highly contextual and highly collaborative, able to provide sophisticated alerts for instant insights and weave in stakeholders into an analytic hub fed by real-time data.
The analogy that IDC uses is that data is actually not like oil but rather like water. If you can capture, cleanse and then irrigate it, it will lead to a significant change in your productivity and harvest. Staying with the water analogy, the outcomes are only as good as the plumbing and the pipeline. You want to channel it, prevent leakage, avoid corrosion and eventually bring it to the right place at the right time- such as that dynamic dashboard.
The executive summary from IDC establishes strongly that organisations which think holistically around data and analytics as a pipeline get better outcomes. Leaders (those with the highest data-to-insight score) perform better on a number of dimensions.
So, what in the data surprised me, and what is the low-hanging fruit? Outside of IDC’s executive summary, these are some of the nuggets that we consider notable.
Survey respondents say they capture a big proportion of all possible data. Most people think they can catch 70-90% of all relevant data. But, overwhelmingly, organisations are trying to make sense of their own data, created by their own business systems.
My POV: The perceived high proportion of data captured shows, on one hand, that “big data” is just, from respondents’ perspectives, today’s data, i.e., size isn’t a constraint. On the other hand, I suspect many respondents are setting their expectations too low (because you don’t know what you don’t know), which can lead to underestimating the potential to capture other data sets, such as external and partner-generated data.
What’s the ROI?
Interestingly, the top 20% of leaders (as identified in the IDC survey sample as those with the highest data-to-insight scores) say they have more challenges than average when it comes to finding the relevant data. Organisations should map out their internal and external data topology that can potentially bring value to the organisation. An information catalogue can serve as connective tissue.
It was apparent from the survey that the main barriers to success are NOT in finding the relevant data, but rather investing in the relevant technology and assessing the ROI to justify budget. It shouldn’t be.
To justify ROI is always hard, but, in these COVID times, to succeed you need to start small and ramp up usage and spend with the value you see from your project. An agile project methodology combined with a SaaS business model helps to keep projects ongoing, even if they are smaller and more tactical. In some cases, water can now often be drawn straight from the tap.
Data quality, a persistent issue?
Almost half of respondents said that they have data quality issues. On the surface of it, this doesn't look too good, but my point of view is that it isn’t necessarily bad news. The only way to improve data quality is to actually identify the problems through shining a light on the data. Through frequent usage, iterations and then applying the right filter, it will gradually improve. Locking data down until you get to 100% quality is not the right answer, because it will become stale, and you’ll never get there, anyway – data is too fluid and changes all the time. We know the business collects an enormous amount of data, from many different sources, in many different formats. That raw data, sitting in various silos, represents potential business value.
Two-thirds of respondents have less than 80% of data automated. This means lots of manual processes. Do you have people passing buckets of water to each other still? This is the low-hanging fruit part, because there are already tools (data movement, change data capture, data warehouse automation, catalogues, etc.) and methodologies (DataOps) that can bring agility and automation to data management, just like self-service has for analytics. The issue that most organisations face is that the “relevant” data, that could be used and applied to better inform decisions and actions and drive value, is not analytics-ready, is not findable, is not accessible, leaving only a small portion of the data “actionable”.
Prevent leaky pipelines
When assessing the success of a project, “improving operational efficiency” came out as a top outcome. I suspect that success metric may grow further in importance and priority. Make sure to choose a tool that can be embedded into your operations, moments and processes. Carefully consider the user personas of your analytics users: the data scientist, the dashboard junkies and the ‘just give me my data’ group. Many organisations have simply put data in the hands of employees and expected them to make a success of it. This can affect their understanding of the potential of data in supporting their work, how comfortable they feel using that data, and, of course, their appetite to use it.
In normal times, during rainy season, when order books were full, organisations may have been excused for having leakage in their pipes. But, that is no longer a luxury that can be afforded. The demand backdrop is getting tougher, and so is the supply side.
Organisations in Asia Pacific need to make it imperative to maximise their data-to-insight score. Synthesising data, cataloguing it and channeling it out for analysis and insight is plumbing for the intelligent enterprise.
