Unstructured data created inadvertently in business, and by consumers, is one of the most undervalued arrows in an organization’s quiver. According to Gartner, dark data is “The ‘information asset’ organizations collect, process, and store during regular business activities, but generally fail to use for any purpose.” Per IBM, 80% of all data is ‘dark’, of which 35% data is useful, of which only 0.5% data is analyzed!
A huge wealth of data goes unnoticed or untapped because it is not actively stored. Since it is scattered, unstructured, and incomplete, processing it becomes overwhelming. Getting insights from dark data is like finding ‘a needle in the haystack’, but the pertinent question is - can this needle be ignored? Especially, if it stiches intelligent insights into business plans with threads of data?
The advancement of Artificial Intelligence driven Business Intelligence platforms have now made it possible to harness ‘Dark Data’. The Extract, Transform, and Load process (ETL) when efficiently applied to dark data and raw information sources such as Contact Centre Transcripts, Customer Reviews, Videos, Images, Geographic Locations, Survey Verbatim, Network Transactions, Industrial And Distributed Databases, Wi-Fi Motions, IoT etc. can help marketing organizations develop insights for revenue maximization through:
Revenue Protection:
- Monitor the impact of marketing initiatives and optimize marketing strategies
- Monitor the customer journey and plug in the deflectors thereby enhancing customer experience
- Deeper understanding of the consumers’ drivers, needs, wants, and values to build brand equity and manage reputation.
- Optimize pricing strategies and avoid revenue loss due to over- or under-pricing.
Revenue Enhancement:
- Using micro-targeting to customize the consumer experience and engage with current and potential customers plus facilitate hyper-personalization, which has become the order of the day for retail and CPG brands
- Understand consumer profiles to leverage opportunities of up-sell and cross-sell
- Create an effective communications mix to break the clutter and grab the much-needed attention of consumers by ensuring the right content is presented, to the right consumer, at the right time, through the right medium
- Enable marketing departments to give inputs based on consumer insights for new product development as well as for new business models, thus reducing the go-to-market time, giving organizations first-mover advantages by understanding trends much before competitors catch up
The trifecta of Access to Cloud, AI-based platforms for Big Data Analytics, and Machine-Learning capabilities are the big bets Retail and CPG brands are taking, in order to be above the curve of digitalization and innovation.
Some of the reasons Retail and CPG organizations miss out on leveraging this wealth of dark data:
- Priorities: Sales and marketing departments have a myopic outlook toward data that only delivers immediate gratification such as customer purchases instead of data associated with what the customer tried and not convert into a purchase and why
- Disconnect among departments: Often data is possessed by one department, which can be actively used by another, but due to lack of coordination this wealth of data goes unutilized
- Technology constraints: Organizations that do not have mature platforms to capture as well as analyze the data miss out on opportunities to develop insights
- Legal constraints: Some of the data has legal constraints for usage and a high propensity to be leaked, which may damage a brand’s reputation
The five S’s to consider in order to make the most of dark data analytics:
- Strategic Goals: View analytics as a business-driven effort to create actionable insights and not a department working in a silo. Work with the business teams to align objectives of the organization across functions. Understand what you’re analyzing and why, and ensure this is line with the overall objectives of the organization.
- Skill Set: Create an effective mix of technical and business skills in the talent pool to ensure availability of data and conversion of data into insights
- Software: Alignment of required technology platforms and software is critical to the objectives of the organization. For example, if an organization wants to analyze customer feedback, it should invest in platforms that have text-to-speech capabilities in their contact centers.
- Scale of Data: Ensure appropriate servers, technologies, tools, and platforms to store large volumes of data as well the computing capability in order to analyze such high volumes.
- Security: Risk management and compliance should be given top priority and effective precautionary steps such as data encryption on local as well as cloud servers should be undertaken.
Retail and CPG brands should consider the following steps while embarking on their data-driven journey for insights:
- Identify data sources: It is important to identify the sources of data as most companies have a wealth of data stored in some form or the other. (Structured, Semi-structured or Unstructured.) The foundation of dark analytics is identifying data repositories and creating a systematic digitization plan of this raw information for easier analysis so that it can be used in the present and the future.
- Prioritize data sources and sets: It is tempting to gather all the information available and analyze it together but this is restrictive in terms of cost and effort. Hence, it is important to identify the most important potential sources and sets of data.
- Plan for new data: Dark data analytics is a continual process with data being generated ceaselessly. Therefore, it is vital to implement a process to capture and analyze the data by leveraging digitization, operational alignment, and appropriate platforms.
- Being responsible: Being responsible for the data -- both in terms of integrity to maintain quality and security to avoid compromising sensitive information — is paramount
- Check with external data: External data can be crucial for decision-making. Not only does it bring in new reference points but also can brings out the differences in the past and present
- Result-oriented approach: The value of data lies in what can be achieved with it. It is important to stay focused and aligned to the overall objectives that are being examined via analytics in order to maximize Return on Investment (ROI).
An integrated technology-based solution that captures dark data, combined with enriched structured data, allows experienced data scientists and analysts to provide valuable business insights. These insights can be used for revenue maximization, cost optimization, efficient performance management, and enhanced consumer experience. Dark analytics is the dark horse of analytics that Retail and CPG brands should bet on to gain the lead in an ultra-competitive consumer landscape.
