Opinion for Marketing Week, Feb 6 2003
Market Research is not a science. This is not so much because we lack methods, but because most numbers have to be INTERPRETED. So there are never "answers" which have no doubts attached to them. The skill of the researcher is to minimise those doubts, and this comes through knowledge of the market and knowledge gained by doing research generally. Anything else is comparable to reading tealeaves or a MR horoscope.
Statistical methods introduced some consistency of approach, but the results have to be interpreted to make sense — and often this can go badly wrong because, inexperienced hands inadequately scope the project, and have not stated clear hypotheses or framed a cause and effect model against which to test the findings. In addition, they often ask the wrong questions, or phrase questions ambiguously so that different participants interpret them differently.
In many instances, MR around brands asks participants questions which have an emotional content. These are investigated as if they were "facts" — which they are not. Our moodswings and changing priorities are reflected in our answers making them not statements of fact but of opinion loaded with emotional baggage. If you're asked a question one day the answer may be one thing. If asked the next day, the answer may be different, and it depends on the experience and knowledge of the researcher who is analysing the data to be able to make sense of what is, quite often, seriously flawed data.
MR is really trying to unearth the motivation behind people's preferences. Motivation in itself is an inexact concept, personal to each individual. Any data collected is bound to be hugely messy and there is no system invented which could package it into nice, neat certainties. The role of the researcher in many cases is to find the "best fit" interpretation and again, we're back to the experience and the intuition of the researcher themselves.
Working with fe3 consulting, my method of analysing data is to deal with the quantitative data ONLY, without reference at all to anything qualitative, although this is vital for the final analysis and recommendations to the client. I come up with indications of what this quantitative data tells me and pass it to fe3 consulting, who have looked independently at the qualitative data. We then examine any discrepancies and have a conversation to try and identify why this might be. This makes us effectively examine the data three times before it goes to the client, and enriches the final outcome. The questionnaire is designed against a theoretical model of best practise and the questions used are validated and tested for clarity. The method of statistical analysis that is used is predetermined and the questions checked to satisfy the stringent needs of the methodology. Debate and deliberation are used to blend the quantitative and qualitative results. Finally, the 'findings' are subject to peer review of experienced consultants and researchers