In designing a research instrument for data collection, what are the considerations prior to writing the questions?
What will be an ideal response?
Before putting together the research instrument, investigators should have already identified the study objectives, the major concepts to be investigated, the major dimensions of these concepts, and their representative categories. In the process of designing the research instrument, researchers make decisions about frame of reference, time, response format, composition of questions, instrument assembly, and pre-testing.
Frame of reference is related to respondents’ particular perspectives. Before writing up the questions, investigators must find out about the general characteristics of their respondents in order to help choose the right words and ask relevant questions. If investigators do not have prior knowledge of the frame of reference of the research population, they have to conduct a pilot test of their instrument before finalizing it.
In designing an instrument, researchers can ask questions about the past, present, or future. Current questions generally produce more accurate answers than past or future questions. When asked questions about the past, respondents may recall information inaccurately, incompletely, or not at all. When asked about the future, respondents must make guesses. To improve the accuracy of responses related to memory problems, respondents should be encouraged to check their relevant record where possible. Another helpful strategy is to provide a list of response categories so that subjects have some references to think about. Paying attention to question formulation, including wording questions specifically and including specific contexts, can also reduce memory problems. Visual aids may be used where possible, and recall periods should be reduced to a reasonable level, usually within two weeks.
In terms of response format, the researcher must make the choice between open-ended and close-ended questions. Open-ended questions require respondents to provide answers using their own words, and can have quantitative as well as qualitative answers. Close-ended questions provide answer categories for the respondents to choose. Qualitative open-ended questions should be used wherever possible, but the choice between qualitative open-ended and close-ended questions is based on many factors, including the knowledge of the subject matter, depth of information required, sample size, desired response and completion rates, desired level of standardization, length of questionnaire, data analysis techniques, and the amount of time to complete the research. Other factors taken into consideration when choosing between qualitative open-ended and close-ended questions include the depth of information required, expected sample size, response and completion rates, interviewer training, interview duration, level of standardization in responses, length of the questionnaire, methods of data analysis, and amount of research time.
If researchers decide to use close-ended questions, they need to decide which specific close-ended formats are most suitable for the research. These formats include contingency questions, two-way questions, multiple-choice questions, ranking scale questions, fixed-sum scale questions, agreement scale questions, Likert-type format questions, semantic differential scale questions, and adjective checklist questions.
A research instrument or questionnaire may contain a variety of questions, such as background, knowledge, experience, behavior, feeling, opinion, attitude, values, intentions, or plans. In formulating questions for the instrument, researchers should pay particular attention to content validity, avoid biases wherever feasible, and pay special attention to certain types of questions. They should avoid loaded questions, leading questions, and double-barreled questions. Researchers should also use appropriate words when forming questions, keep questions short, and limit the number of complex concepts in a single question. They should also use item analysis to ensure internal consistency when several statements are used to produce one scale.
When assembling the questionnaire, researchers should take into consideration the quality of the paper, the appearance of the pages, and the sequence of the questions and writing instructions. If email or web-based surveys will be used, they should be presented in a format compatible with multiple types of servers, browsers, and operating systems; use welcoming, motivational colors and fonts; avoid using too many graphics or varying the graphics from screen to screen; and resist the urge to create long, burdensome online surveys. It is also important to assess the appropriateness of survey questions for populations that speak a different language. Additionally, researchers must consider whether respondents will print out their answers or send them electronically, and what steps they will take to ensure privacy and confidentiality in either situation.
After the questionnaire has been assembled satisfactorily, the next step is to conduct a pre-test. Pre-testing is usually conducted with a relatively small number of respondents who closely resemble the research population. With pre-testing, researchers are interested in knowing the actual time it takes to complete the questionnaire; whether the level of language matches respondents’ knowledge; which words, terms, phrases, or sentences are still confusing, ambiguous, or misunderstood; whether response categories are truly mutually exclusive and all inclusive; which questions generate a relatively high nonresponse rate; whether current instructions are understood or additional instructions are needed; and whether responses to open-ended questions are given in the way intended.