Optimize your “Small Data” before moving to Big Data + AI

When it comes to unleashing the full business impact of market intelligence data, many companies are trying to run before they have learned how to crawl. Before you take the plunge into Big Data and / or artificial intelligence (AI), ensure that you are capable of ‘connecting the dots’ between – and release the full value of insight of – your existing “small data” (existing market intelligence data sources). Otherwise your efforts are likely to be wasted. The key is access, adoption and integration.Big Data and AI. If you work in marketing, it is hard to pass a day without someone slipping these terms into a conversation, rightly or wrongly. As a technology company with strong roots in market intelligence and data science, we are the first to recognize the potential of Big Data and AI for marketing.The problem is that many companies are launching Big Data and AI (Watson anyone?) initiatives, when they are still struggling to unleash the full value of their existing “small data” (basic market intelligence data) and “human intelligence” (i.e., the skills and brainpower of their own people).In other words, these companies have yet to build the basic prerequisites for benefiting from these highly complex, expensive and sophisticated instruments. They are trying to run before they have learned how to crawl.

The 3 A’s of maximizing the business impact of your market intelligence data

What do we mean by that? In a few words, it means that they have not yet achieved the 3 “A’s” for optimal insight creation:

• Association (or integration). Most companies already have access to a wide variety of market and customer intelligence data coming from multiple sources. While many are talking about Big Data, very few have managed to unlock the full insight contained in these data sources by “connecting the dots” and integrating these different data sources into a comprehensive view of the customer journey. For example, many researchers harmonize disparate data sources around common denominators such as product categories or customer segmentation. In the world of data, the combined value of insight is many times greater than the sum of its parts. The obstacles to achieving such integrated insight are usually of a cultural-organizational nature (e.g., departmental / team silos each looking only at “their own data”). The greatest challenge and greatest driver of value of Big Data is integrating, harmonizing and matching highly heterogeneous data sources to extract meaningful correlations and patterns for insight and decision making. If you have not managed to achieve this (including in terms of establishing a corporate insight culture) with your Small Data, then you are unlikely to be successful in Big Data and AI.
• Access. In many companies, market intelligence data remains confined to a relatively small group of market research experts, well versed in statistical techniques and able to “safely” use this data (in the sense of applying proper statistical methods and interpretations). The data essentially remains a black box to the more casual business users who would benefit most from the insight contained therein. The introduction of concepts such as Big Data and AI, which even fewer people will have the know how and time to understand, will only make this worse. The key to releasing the full business impact of this data is to emancipate, not to isolate. Ensuring this data can be safely, easily and efficiently used directly by every end user. This requires a new type of "emancipatory" digital reporting system that breaks the traditional divide between analytical (e.g., cross tabulation) and charting modules, with a concept we call “self-service analytical visualizations”.
• Adoption. Simply facilitating access to a more integrated picture, as described above, is not enough. The business return on market intelligence information will still be limited if such a reporting system – and as a consequence the diffusion of the insight it contains and stimulates – is not broadly adopted by the organization. This means that the reporting system needs to be extremely intuitive (zero training) and flexible enough to seamlessly fit the way different people like to work and give them a real sense of control. It is not enough to compel people to work with a system. They need to want to do it. At one of our customers, adoption went from 200 active users with a previous system to 2,000 users with our system. Imagine the exponential effect this has on deep business insight and results.

The Big Data and AI “wishful thinking” trap

Within the world of AI there is a famous joke that “AI is whatever hasn’t been done yet.” This refers to the fact that as machines become more capable, more and more tasks previously considered as requiring intelligence are often removed from the definition of what constitutes AI. This phenomenon called the “AI effect”.To some extent this is the same psychological trap that many companies fall into when thinking about AI and Big Data.Marketing faces one of the most complex challenges around: how to draw conclusions and patterns from the gigantic, chaotic and largely uncontrolled experiment of the marketplace. While the shift to online and the enhanced ability to run A/B testing is helpful, we are still only looking at the tip of the iceberg.

Clearly, we need a miracle. Something that can, somehow, capture this infinite variety and amount of data and learn from it to find hard patterns and suggest optimal decisions. Enter Big Data and AI.And clearly these two concepts are very promising in the long run. No matter how promising, however, it is important to remember that both of these concepts are still in their relative infancy and largely unproven in the most interesting (and complex) scenarios.In the meantime, Big Data and AI initiatives should not be used as an excuse for tackling the current obstacles extracting optimal value of market intelligence data, nor for ignoring high value (albeit less sexy) “low hanging fruit”.

You need to learn how to crawl (Small Data) before you try to run (Big Data + AI)

And this is where we return to our earlier statement: you need to learn how to crawl before you try to run. Many of the companies launching Big Data and AI initiatives have not even come close to exploiting the full potential of their existing “Small Data” (basic market intelligence data) and skills, experience and brainpower of their people. Perhaps more surprisingly, even the specialized agencies producing multiple studies for the same customer often have not integrated these studies to provide a comprehensive, seamless and easily comparable analytical experience.

‍If your organization is incapable of achieving this modest “Small Data” insight optimization objective, it is very unlikely that you will be successful with the much greater challenges presented by Big Data and AI. Invest first in integration, access and adoption.