Over at NBRI, Dr. Terrie Nolinske has begun a new series of articles aimed at explaining the benefits and pitfalls of conducting a survey.  I enjoy Dr. Nolinske’s writings so much that I decided to mirror each her articles with one of my own to bring some of her discussions over to SSO.

Dr. Nolinske’s inaugural article this week is entitled “Minimizing Errors in Survey Research”.  She discusses the potential errors in the layout and design of a survey instrument.  I decided to add to this by giving examples of errors in the wording or phrasing of survey questions.

Common Errors in Creating Survey Questions

Oftentimes those uninitiated in the formulation of survey questions want to go to a website and find questions or a survey template that fit their needs; cookie cutter questions that appear to ask what it is they want to know from customers, employees, voters, or consumers.  This is good if the original author of those questions is knowledgeable and the questions ask exactly what you are attempting to find out from your survey audience.

Some may even create their own questions which can sometimes be dangerous.  Without some insight into how to write survey questions they may potentially do more harm than good.

The questions that are asked, and just as importantly how they are asked, can have an amazing impact on the answers you receive and the data that is collected; data that will be used to make important business decisions.

Take these questions for example:

I can always talk to my superiors about work related problems.

The problem with this question is the word “always”.  Can anyone ever “always” talk to their superior?  In this example the question is too definitive and therefore some will rate this low when in fact they feel they can talk to there superiors - when needed.

Training and career planning are available.

The issue here is that it is a double barreled question.  If the respondent rates this question low, are they having an issue with training AND career planning or one or the other?

Communication at work is good.

This question does not produce actionable data and is too vague.  How is communication good?  The communication between whom?  Is it too fact, too slow, not enough?

The pace at our Company is hectic.

The problem with this question is that it is negative.  Survey questions often convey information before actually asking a question.  In this case, the question broadcasts a bad message about the company.

Information is widely shared at [Company], so that those who make decisions have access to all available data.

This particular question has a list of issues:

• It‘s not a simple sentence making it difficult to understand.
• Is it asking the respondent about information being widely shared?
• Is it asking the respondent about data availability?
• Is it asking about the access that those who make decisions have to data?

Instead, try these questions:

There is appropriate information exchange within the [Company].
The information I need to make good decisions is available.

As you can see, the questions used in a survey can be tricky.  Examples here are from an employee survey, but the same holds true for a customer survey, market research, or a political survey.  Many surveys have been skewed to one side or the other by the simple phrasing, or mis-phrasing of the questions.

Consider that the questions you use can dramatically affect the data that you collect before you use “example survey questions” or “survey templates” found on the net.  And if you deploy the same survey year after year (which you should, how else will you know if you’re improving?) the negative effect of poorly written survey questions will be multiplied.

Now for a shameless plug, the Survey Software Online system includes proven survey questions formulated from years of experience.  Questions are grouped by the type of survey you are conducting and then by topic.  You can pick from 1000’s of questions that have been scientifically designed and used in surveys for decades.  Of course, you can also create questions from scratch if that is what is needed.

Don’t leave your survey to chance and don’t take question formulation lightly.

David Harrison
Survey Software Online