Market research: error, error, bias, bias

Error, error, bias,  bias ... and warning.  How we teach market research to undergraduates.

Helping my daughter

Last night I helped my Uni student daughter revise for her market research exam. She had taken the key words out of each chapter of her market research text book, and I tested her to see if she could remember them. By key words, I mean the ones the book defined as key words.

Now, I think I work in a fascinating profession, where I can be involved in helping solve management problems for businesses, all the while talking to real people about their lives and what makes them tick. I was devastated to see that this is not the way market research was depicted in her text book.

It's all about what can go wrong

Of the key words, out of 15 chapters, the most repeated word was 'error', with 'bias' lagging not far behind it. Of course, the text mentioned non-response error, which is fine.  But is it really necessary to list 'administrative error' as a key word?  I do not believe that any of the other subjects studied by undergraduates teaches them that people can make data-entry errors - yes, another key word. The section on survey research dedicates almost all of its first 9 pages to errors and all the other things that can go wrong.

Then it's about bias

You have to have lived under a log not to know that human thinking can be biased in many ways. This text book goes beyond the usual explanation of cognitive biases to list the many many ways that interviewers and  respondents can be biased. The number of ways it seems possible to bias an online survey is simply alarming. They even seem to have made some up, as if we didn't have enough already.

But then it gets vague

Like all good MR text books, this one spends some time on the key concepts of validity and reliability. Unfortunately, clarity of expression is not one of their strong points. Their definition of construct validity did my head in.

In contrast, my daughter has also taken a course in PR this year, which included a week or so on market research. She went back to her PR text book for clear definitions (with examples) of reliability and validity, because the MR text was unclear.

Let's come back to biases ...

 

The text book itself is very biased in the views it condones.

  • The section on qualitative research is mind-boggling in its biases. Unlike any other section, the one on qual has the word 'warning' in large font.
  • It then goes on to critique some oddball examples of qual they claim to have seen, though I have to say I have never come across anything like these.  Classic straw man argument stuff.
  • Then it devotes over 4 pages to Hugh Mackay's qualitative technique, without pointing out that some other professionals do not share Hugh's views.
  • The book also include some industry case studies taken from conference papers and the like. It's really quite alarming that a text book can name individual researchers and agencies, praising some and criticising others (indirectly of course).
  • As a final point, the authors' statements about the reliability of sensory testing are simply laughable.

     

So that is how we encourage today's undergraduates to join our profession, or use our services.  A biased summary of errors and biases. Great.

Tags: Market Research


For a quote or a confidential discussion about your research needs