The say-do gap is inevitable - here is how to trust your survey data

The say-do gap is the difference between what we say and what we do. We all do this and it is simply part of being human: being affected by things we do not realise and sometimes acting differently than intended because of it. That means all market research data - regardless of the methodology - is biased. So how do you trust your data? We will tell you.

It is brisk, but true: consumers are full of sh**. They say things that are not true and they say they intend this and that but they do not follow through. So why bother to ask them? Why is quantitative market research still a billion-dollar business?  Is it not clear that it is flawed? Should you not jump right ahead to behavioral data instead? Probably not and the reason is simply that that it is never advisable to put all of a brands’ consumer insights eggs into one basket; there is no truth about consumers. No methodology that once and for all will uncover exactly what makes them buy. No golden shortcut for success. That is a fact to trust, no matter how you try to uncover trends, opinions, behaviors, or sentiments.

Instead, what what you need to direct your brand position, marketing communication, and your products and services to match your market, is understanding consumers both on the conscious and unconscious level. That means you need consumers’ conscious reflections and it also means that you need to take the say-do gap into account in your survey-based market research.

‍

The problem with consumer insights is reality

Yes, there is a difference between asking a person in a survey and observing their actual behavior afterwards. But that does not make neither the survey nor the observation study obsolete. Instead, it may be in discovering the say-do gap that your opportunities for optimised marketing lie: 

One brand might be making the oh-no-price-is-not-what’s-most-important-to-me consumer walking down the grocery store aisle completely change their mind and throw three of your competitor’s items down their cart. And - to the horror of your consumers-choose-us-for-being-organic-insights - discover the competing brand’s products are actually tasty. That does not make the brand’s organic values irrelevant but it may emphasise a need for better taste, pricing, or packaging.

Or a digital sponsored ad might outperform your carefully orchestrated branding campaign because your competitor went in with a higher bid, a catchier message, or better visuals. And thus, slip the purchase.

That - reality - does not make consumer insights irrelevant, because optimising your likelihood to do good still requires consumer understanding before the situation where the consumer choice is made. And you want to set yourself up for success, do you not?

‍

The solution to the say-do gap is quantitative data. Yes, really

Understanding the real path-to-purchase is understanding not only consumers’ values and attitudes, but also the role of the unconscious: values and attitudes can be overruled by sudden context changes, status, societal convention, etc. 

The bad news: you will never get the perfect understanding of this, regardless of how many resources you invest.

The good news: less can set you up for success! 

It is about generating the consumer insights to understand the context consumers will evaluate your brand/product/service in and what is most likely to hit home with them. And that is where quantitative consumer insights, because big numbers will - in the end - be victorious. It is simple: insights based on a few consumers might give you interesting insights, but also possess a huge risk that results are biased. 

Of course, quantity does not automatically equals quality and there are three things that determine successful handling of the say-do gap in your survey-based research. Mastering those will produce quantitative data with the smallest possible bias and that will in turn lay the foundation for actionable consumer insights and your success.

‍

Checklist for handling the say-do gap in survey data

Handling the say-do gap in survey data is done in the research design phase. That is yet another reason why this is such a crucial part of market research and why you should never neglect developing your research design competencies!

So without further ado, here is your checklist for handling the say-do gap in survey data:

1. Make sure your questionnaire does not produce a bias. This is way more common than you might expect, but it is also one of the key reasons results do not correspond with reality. If you are in doubt, your market research partner should be competent to consult on your questionnaire development. Actually, your research partner should proactively raise a flag if your questionnaire will produce biased data. Listen to them and edit your questions/answer options.

2. Include the right dimensions. Excluding an important dimension could miss out that 25% of your audience are brand shoppers based on promos or personality, so you might want to go about your research a bit more explorative than you initially think. Consider for instance to uncover reasons consumers would not choose your brand, product, service, etc.

3. Make sure you research the right people. When you set up screening criteria, you should consider whether both existing and potential target audiences might have valuable inputs for you. Too often screening is not explorative enough to uncover previously unknown potential.

‍

Work with the say-do gap rather than trying to avoid it

When it comes to the say-do gap, it is all in your hands to handle. And handling it you should, if your consumer insights are to be the competitive advantage you tell your bosses they are investing in. Yes, humans are basically unreliable and will deviate from what they say to some extent, but no, that does not make market research obsolete as the alternative is to operate in the dark with your hands tied to your back. 

So it is upon you to mind your research design and handle the uncertainty that any data collection method has by understanding that the best possible data comes from the best possible research design. The old saying “garbage in - garbage out” also works the other way around: “great in, great out”. 

In short, the most important takeaway for you: understand the uncertainty dimensions of consumers’ path-to-purchase and account for it in your research research design and data processing.