She’s wearing high heels but manages to glide her way down the street as she chases the man. As she gets closer to him, the camera zooms in and her questions start to fly. She almost knocks his nose with her microphone. He tries to get into his car but she’s faster than him.
She pushes the door closed and repeats her jabbing questions such as…
“Do you agree that you were WRONG to steal their money?” and
“Don’t you think these people DESERVE their money back?”
This is a common scenario on television and one that I’m sure you’ve witnessed.
Although at the time you probably thought “Yeah, you tell him, Miss!” these questions are considered very, very bias… especially when you use them as survey questions.
Why? Because questions like these could influence the answer you receive.
Well, in the first case…
“Do you agree that you were WRONG to steal their money?”
…you could easily hope for a “Yes, OF COURSE! I’m sorry!”
Otherwise, what’s the point asking that question? It might be hard for the guy to say “No!” as he knows it could make him look bad. He’d probably keep quiet, though.
With the next question,
“Don’t you think these people DESERVE their money back?” …you could probably hope for the guy to do one of the following:
- Remain silent and drive off
- Pull out a mobile phone and call his lawyer
- Shout “No comment!” and drive off
- Put on a sad face and respond with “I haven’t done anything wrong!”
- Use violence and push her aside… jump in his car and drive off
But why would anyone ask these questions? Is it to further prove a point that this man is a bad person?
But let’s go back to survey questions.
Why… or maybe ‘how’ do questions like these sneak into surveys?
I haven’t met anyone who has deliberately added bias questions in their surveys but I have been handed questions like these to add to surveys.
They were ‘accidentally’ worded in a way that they easily confuse or lead the respondent. The questions would have forced you (the respondent) to believe that we’re expecting you to answer in a particular way. With questions like these, I would have ended up with bias responses.
It’s like if people were to ask you: “Don’t you think you should find a job?”
You feel that you’re expected to agree with them.
We caught these questions well ahead of the surveys going out and reworded them.
It’s possible that some survey designers don’t know how to properly design survey questions.
Others probably don’t understand the topic well enough. Or they didn’t test the questions with a focus group.
Not testing a survey is considered a ‘sin’ in marketing research. ALWAYS test your questions with a focus group so that you know which of the questions should be reworded or ditched.
Another reason that bias questions sneak into surveys is the writer didn’t consider other people’s perspective on the topic of the survey. Had they tested their questionnaire to see how well it works, they would have known that they would need to reword.
What’s the problem with using misleading survey questions?
Adding bias questions to surveys is rarely intentional but is always a big problem:
- They mislead the respondent – we’ve already established that.
- They mislead your data – so it would appear that people agree to your line of thought.
- The misled data will be used to make important decisions for your business. Yikes!
- The ‘evidence’ can easily be disproven by competitors if they were to conduct a ‘proper’ survey.
- The ‘evidence’ leads customers and prospective customers to believe in you… and they could end up very disappointed later on.
Now… whether or not the survey analyst realises this after collecting data is a whole different story and maybe one for another day.
If you would like to see some examples of leading survey questions used in market research and marketing research, join the mailing list below to receive notification of the next post that offers a FREE list to check your questions against.
If you’re developing a survey for your brand and are not quite sure whether you’ve got things right, I’m able to assist with both your marketing research and market research activities. Get in touch today and we can discuss your requirements.