• Home
• |
• Contact Us
• |
• User Guide

Interviewer Recruitment, Selection, and Training Guidelines

X. Interviewer Recruitment, Selection, and Training

Webpage last modified: 2008-Sep-10

Introduction

Interviewers play a critical role in surveys, as they implement the survey design and are often required to perform multiple tasks at a high level of accuracy. In a face-to-face survey, the interviewer may be required to physically locate the household and to update the sample frame. In both telephone and face-to-face surveys, the interviewer has to contact the household, explain the purpose of the study, gain cooperation, enumerate household members, select the respondent, ask questions in the required manner, put the respondent at ease, and accurately record the respondent's answers and any other required information. Depending upon the survey topic and survey context, the interviewer may be required to perform additional tasks, such as bio-measure collection or oral translation.

Interviewers can influence responses through their personal attributes and their behavior ("interviewer effects"). These guidelines present strategies to minimize the effect interviewer attributes have on the data through appropriate recruitment, selection, and case assignment; they also present strategies to minimize the effect that interviewer behavior has on the data through formal training.

Guidelines

Goal: To improve the overall quality of the survey data by minimizing interviewer effects.

1. Determine the structure and composition of the interviewing staff.
   Rationale

   The structure and composition of the interviewing staff must be established during the design and planning phases of the project because these decisions will dictate the number of interviewers required, the training protocol, sample assignment, and optimal methods of supervision.

   Procedural steps

   • Consider such parameters as sample size and, for face-to-face studies, geographic distribution; the timing and duration of the data collection period; budget constraints; and the language(s) in which interviewing will occur [26].
   • For face-to-face studies, decide whether interviewers will travel, either individually or in teams, or be locally assigned
     • Factors favoring the use of traveling individual interviewers include:
       • Lower training costs compared to using local interviewers, as there are fewer interviewers to train and trainers do not have to travel to as many different locations.
       • Breech of confidentiality is less of an issue than with local interviewers because interviewers are unlikely to know the respondent personally.
       • Respondents may be more willing to participate in sensitive-topic surveys if the interviewers are strangers.
     • Factors favoring the use of traveling teams include:
       • Lower training costs compared to using local interviewers, as there are fewer interviewers to train and trainers do not have to travel to as many different locations.
       • Traveling as a group may be safer than traveling individually.
       • Monitoring and supervision are easier, since the supervisor is part of the group and is in close daily contact with the interviewers.
       • If multiple household members need to be surveyed, different interviewers can speak to them concurrently.
       • Similarly, if privacy is difficult to achieve, one interviewer can speak to the respondent while another engages other household members.
       • Breech of confidentiality is less of an issue than with local interviewers because interviewers are unlikely to know the respondent personally.
       • Respondents may be more willing to participate in sensitive-topic surveys if the interviewers are strangers.
     • Factors favoring the use of local interviewers include:
       • Employing a larger number of interviewers, each with a smaller workload, reduces the interviewer design effect [19] [23] (see Appendix A).
       • With a larger field staff, data collection can be completed within a shorter period of time.
       • More call attempts can be made per case, since the interviewer remains in the area throughout the data collection period.
       • Local interviewer assignment reduces the need for interviewers to travel large distances, thereby reducing travel costs and time expended.
       • Local interviewers are familiar with the area and are more likely to share the language and customs of respondents; they may achieve higher response rates than would an "outsider."
   • For telephone studies, decide whether interviewers will conduct the survey from a central telephone facility or from their homes (that is, decentralized telephone interviewing).
     • Factors favoring the use of centralized telephone interviewing include:
       • Monitoring and supervision can be easier, since the supervisor is in close daily contact with the interviewers and may, as a result, have access to more information of relevance.
     • Factors favoring the use of decentralized telephone interviewing include:
       • A dedicated telephone facility is not required.
   • Estimate the Hours Per Interview (HPI). The HPI includes time spent traveling to all sample cases, attempting to contact them, documenting contact attempts, and working on project-related administrative duties, as well as conducting the interview with those respondents who agree to participate. The HPI, combined with the hours per week that each interviewer is expected to work on the project and the total number of weeks planned for data collection, helps determine the number of interviewers required (see Appendix B for an example).
   • Utilizing the results of feasibility studies (see Data Collection), consider any special requirements of the study, such as:
     • Would interviewer familiarity with the topic introduce bias or enhance an interviewer's ability to collect data?
     • Does cultural protocol or the nature of the topic necessitate matching interviewers and respondents by gender, dialect, religion, race, ethnicity, caste, or age?
     • Will interviewers be required to walk or bicycle long distances, making physical stamina a consideration [21]?
   Lessons learned

   • If traveling teams of interviewers are used, the interviewer may not always be conversant in the respondent's language, and local interpreters may be needed to facilitate data collection. For example, the French Institut National d'Etudes Démographiques has collected data in several Bwa villages in Mali for over fifteen years. Although French is the official language of Mali, most villagers speak only Boma, so interpreters were essential for collecting data. The interviewer was responsible for administering the questionnaire, while the interpreter's job was to act as a neutral intermediary between the interviewer and respondent, conveying the words and the concepts attached to them to the two speakers [28] (see Translation guidelines).
   • Matching interviewer and respondent characteristics may improve cooperation but only appears to impact survey data quality if the topic of the survey is related to an identifiable and stable interviewer attribute.
     • Indonesian researchers felt that matching interviewers with respondents in terms of age, marital status, and child-rearing experience improved rapport and willingness to participate during in-depth interviews [24].
     • Several studies indicate that when the topic of the survey (e.g., racial attitudes or women's rights) is related to a fixed interviewer attribute (e.g., race or gender), the interviewer attribute can affect respondents' answers [14] [18] [29] [31].
     • If the topic of the survey is not related to a fixed interviewer attribute, matching the interviewer and respondent on the attribute does not appear to affect data quality. Axinn et al. [2] found that matching Nepalese interviewers and respondents by gender and ethnicity for a health survey did not improve the number of technical errors and "don't know" responses or reduce incorrect information gathered during the interview.
   • Attempting to match interviewer and respondent characteristics may strain the project's resources, particularly if this is not an established practice in the locale.
2. Determine the pay structure for the field staff.
   Rationale

   Interviewer pay is one of the largest expenses in interviewer-administered surveys; at the same time, since field staff quality has a major impact on the quality of the data collected, it is important to attract and retain the most qualified interviewers possible.

   Procedural steps

   • Depending on local labor laws, set interviewer pay comparable to the pay for similar jobs in other survey organizations, ideally adjusted for regional cost of living standards.
   • When determining pay, consider the length and complexity of the interview, the expected difficulties of obtaining co-operation, and the amount of record keeping demanded of the interviewer [10].
   • Pay interviewers for time spent in training.
   • Keep in mind local research traditions, the mode of the survey, and local labor laws. The two standard policies are to pay interviewers an hourly rate or to pay per completed interview [10] [26].
     • Factors favoring payment per interview:
       • It is most feasible if each completed interview takes approximately the same amount of interviewer effort, as is more likely in a telephone survey [26].
       • It is easier to monitor and control interviewer costs than when paying by the hour [26] [32].
     • Factors favoring an hourly rate:
       • It is most feasible if the effort to complete an interview varies widely, as is common in face-to-face surveys [20] [26].
       • Interviewers have less incentive to perform hurried, sloppy work or even to fabricate interviews than when paid per interview [26] [32].
       • Interviewers are less likely to focus on easy cases while neglecting those who are hard to reach or hard to persuade to participate than when paid by the completed interview [10] [26].
     • Adjust the pay rate based on interviewer experience and any special skills they may possess and require (e.g., bilingual interviewers).
     • Consider offering incentives for completed interviews achieved above a certain response rate target as a way to keep interviewers motivated [10] [36].
       • Incentives can be extra pay, prizes, or special rewards.
       • Overreliance on interviewer incentives for completed interviews may give interviewers a reason to fabricate interviews [36].
       • Any bonus system must be perceived by the interviewers as being fair. For example, different sample assignments can vary considerably in the challenges they pose for interviewers [10].
   Lessons learned

   • Most survey organizations have a standard policy concerning pay arrangements (either paying per interview or paying by the hour) which they may be unwilling to change [10].
   • If interviewers are paid by the interview instead of by the hour, they may rush the critical respondent-interviewer rapport-building process. It is important for interviewers to spend the time necessary to develop this rapport so that respondents feel comfortable reporting honestly, as this leads to higher-quality responses. For example, to establish a good relationship with non-Western respondents, interviewers must adopt the culture's introductory customs, such as drinking tea or meeting elders [15].
   • To discourage hurried, sloppy work when paying per interview, some organizations set a cap on the number of interviews that each interviewer is allowed to conduct in a day. Another strategy is to offer bonuses for high quality work. For example, set a basic pay per interview plus an additional 10% if the interviewer makes fewer than some predetermined number of errors.
3. Recruit and select an appropriate number of qualified interviewers.
   Rationale

   The quality of an interviewer-administered survey depends, to a large extent, on the quality of the field staff. It is important, therefore, to recruit and select the best possible people for the job.

   Procedural steps

   • Recruit applicants.
     • Sometimes the interviewing component of the study can be subcontracted to an external survey organization with an existing pool of interviewers. At other times, the research team will have to implement outreach measures to find applicants, such as asking local contacts for suggestions, placing flyers in strategic locations, or putting ads in local papers. In the latter case, recruitment and training will take longer.
     • Keeping in mind any special considerations, as described in the previous step, target sources of potential interviewer candidates. Some studies have found college students to be particularly good interviewer candidates; traveling nurses can be a good source of interviewers for health studies; and teachers, or others with substantive knowledge of the study topic, may also be good candidates.
     • Keep cultural norms and logistical factors in mind when recruiting interviewers. For example, it may not be acceptable in some cultures for young people (e.g., students) to interview older persons. Similarly, persons with other job obligations (e.g., nurses and teachers) may not be available to work on the study at the times when respondents are most likely to be at home.
   • Recruit more than the number of interviewers needed for fieldwork to allow for attrition and the dismissal of candidates who prove to be unsuitable.
   • As appropriate, prepare an application form to use in prescreening interviewer candidates before they are invited to an in-person or telephone job interview.
   • Interview applicants in the mode of the study. That is, hold telephone screening interviews for a telephone survey and face-to-face screening interviews for a face-to-face study.
   • Evaluate each candidate.
     • Criteria for employment commonly include interviewing experience, language skills, computer or technical skills, organizational skills, education, availability, location, the ability to meet production (i.e., data collection) goals, and the capability to handle potentially emotional or stressful interactions with respondents [26].
     • When possible, select interviewers who have previously worked on similar studies and have good recommendations based on their performance. Experienced interviewers require less training and are likely to achieve higher response rates [10].
     • Evaluate the accuracy and clarity with which each potential candidate can read and process the survey questions in the language(s) of the interview and make sure that he or she is comfortable reading out loud. Ideally, language proficiency should be formally assessed by an outside expert or language assessment firm and should include evaluation of [25]:
       • Conversational skills (e.g., comprehension level, comprehension speed, speech level, speech speed, and accent)
       • Writing skills (e.g., grammar, spelling, and the ability to enter responses)
       • Reading skills (e.g., reading aloud)
     • Ideally, test applicants' computer skills for studies using a computerized questionnaire and/or sample management system.
     • Select interviewers who are punctual and have good organizational skills (e.g., are able to handle forms and keep track of paperwork).
     • Select interviewers who have completed the full period of required schooling within their country.
     • For face-to-face studies, assess applicants' ability to read maps.
   • Clearly present the candidates with study expectations for workload (weekly, monthly, including evening work and possibly weekend work).
   • Obtain the candidates' commitment to work at the expected level of effort for the duration of the field period.
   • Describe any non-traditional interviewing tasks (e.g., taking physical measures) in the recruitment description and the screening interview.
   • Base selection on an objective evaluation of the candidate's abilities rather than his or her relationship to survey staff or favoritism [16] [21] [35].
   Lessons learned

   • Vaessen et al. [35] suggest that study managers recruit at least 10-15 per cent more than the number ultimately needed for fieldwork.
   • As noted in Guideline 1, it is not always possible to recruit interviewers who are fluent in the language(s) preferred or needed by respondents. In this case, other arrangements must be made. Options may include working with interpreters, data collection by proxy, using a bridge language if available, or using self-completion modes if literacy levels permit.
   • If the topic is sensitive (e.g., domestic violence), empathy and strong interpersonal skills may be more important than high levels of education or previous interviewing experience [17].
   • If the project's interviewing protocol differs significantly from previous studies, experienced interviewers may find it difficult to change their habits. In this case, it may be preferable to recruit and train new interviewers.
4. Provide general interviewer training.
   Rationale

   Newly hired interviewers and supervisors require general training in techniques for successful interviewing before they receive specific training on the study on which they will be working. Research indicates that interviewer training helps improve the quality of survey data by: (1) reducing item nonresponse [3], (2) increasing the amount of information obtained [3], and (3) increasing survey participation by teaching interviewers how to identify and respond to respondents' concerns [22].

   Procedural steps

   • Be sensitive to the local culture.
     • Educate trainers in cultural sensitivity.
     • Take religious holidays into consideration when scheduling training sessions.
     • Make every effort to accommodate dietary restrictions when planning meals or snacks for the training.
   • Allow a minimum of two days to adequately cover general interviewing techniques material.
   • Include the following prescribed procedures:
     • Standardized question-asking. Train interviewers to read each question exactly as written and to read the questions slowly. They should ask all questions exactly in the order in which they are presented in the questionnaire [9] (see Guideline 5 for exceptions).
     • Questionnaire format and conventions. Teach interviewers how to enter the answers to both open- and closed-ended questions. Train them to follow interviewing conventions such as emphasizing words in the questionnaire which appear in bold or are underlined, recognizing and not reading aloud interviewer instructions, reading or not reading optional words as appropriate, and selecting correct fill choices.
     • Clarification. If the study staff has not prepared a stock definition, train interviewers to repeat all or a specified part of the question verbatim when respondents ask for clarification. Interviewers should not make up their own definitions to any word, phrase, or question in the questionnaire.
     • Probing. If a respondent's answer is inadequate and it is culturally permissible to probe, train interviewers to employ unbiased techniques to encourage answers that are more complete, appropriate, and thoughtful.
       • Such strategies of probing for more information may include a pause to encourage the person to fill the silence or a direct request for further information.
       • Verbal probes should be chosen from a stock list of phrases such as "Could you explain what you mean by that?" or "Can you tell me anything else about ___________ ?"
       • Stock phrases must be neutral; that is, they must avoid "sending a message" about what is a good or a bad response.
     • Feedback. Train interviewers to provide their respondents with culturally appropriate feedback when they are doing well in order to encourage them to listen carefully and to give thoughtful answers.
       • This feedback may be in the form of a nonverbal smile or nod or a short encouraging phrase.
       • Verbal feedback should be selected from a prepared list of stock phrases such as "That's useful information" or "Thank you, that's helpful" to ensure that the feedback is not evaluative of the content of the answer. For example, in English the word "okay" is discouraged for use in feedback because it could be construed as agreement with or approval of the respondent's answer.
       • As a general rule, give nonverbal or short feedback to short answers and longer feedback phrases to longer answers.
     • Confidentiality. Train interviewers to keep confidential all identifying respondent contact information as well as respondents' answers to survey questions.
     • Completing call records. Teach interviewers to record when each contact was attempted, pertinent respondent comments (e.g., the best time to reach him or her or reasons for reluctance to participate), and the result of each contact attempt, using disposition codes (further information on call records and disposition codes can be found in Tenders, Bids, and Contracts and Data Processing and Statistical Adjustment; examples of call records can be found in Data Collection).
     • Recording time and meeting production goals. Teach interviewers how to record the time they spend on all aspects of their work for the study, both for their remuneration and to allow supervisors to monitor their progress and efficiency during data collection.
   • Include an overview of different sampling designs and the tasks each poses for the interviewer.
   • If culturally permissible, teach interviewers non-coercive persuasion techniques and practice counter replies to common statements of reluctance.
     • Discuss optimal times and modes for contacting target persons.
     • Train interviewers to tailor their initial interactions with respondents by developing the following skills [13] [22]:
       • Learning the classes of concerns ("themes") that respondents might have.
       • Classifying the respondent's wording into the appropriate theme.
       • Addressing the concern, using their own words.
     • Employ hands-on practice exercises so that the trainees become proficient in quickly identifying respondent concerns and quickly responding to them.
     • For best overall results, employ a training format that combines lecture with visuals and small-group practice sessions.
       • Mixing the format keeps the trainees engaged and acknowledges that different people learn in different ways.
       • Through practice, trainees move from procedural knowledge (knowledge of how to perform a task) to skill acquisition (the ability to perform the task almost automatically) [22].
       • Although the class can be large for lecture sessions, trainees should break up into smaller groups for hands-on practice.
   • At the end of general interviewer training, evaluate the knowledge of the interviewer candidates. This can be done by written test, conducting a scripted certification interview with a supervisor, audio taping, or observing the interviewer conduct an actual interview.
   Lessons learned

   • If the interviewer candidates have access to the necessary equipment, some general interview training material can be presented in the form of audio- or video-tapes for home study [6] [11].
   • Interviewer training and the interviewer manual may need to be adjusted to be culturally sensitive to the population under study:
     • Researchers conducting a women's health study on the Apsáalooke reservation in southeastern Montana felt that standard tactics for handling reluctance would be offensive in that culture. They therefore did not attempt to persuade reluctant respondents to participate. In addition, interviewers were encouraged to display a compassionate attitude and interest in the women, rather than the standard recommended neutral voice tone and lack of responsiveness to respondent answers, to minimize eye contact, and to accept offers of food and drink — all to be more consonant with the Apsáalooke culture [7].
     • In majority countries, a Western trainer may be respected but resented. Researchers in Puerto Rico found allowing interviewer trainees to provide input about the local culture and supplementing trainer criticism with peer criticism helpful [33].
     • The World Mental Health study added country-specific topics to their general interviewer training sessions. In New Zealand, they included cultural empathy to Maori and Pacific Islander households; in Colombia, they provided special training on interacting with governmental authorities and armed guerrilla and paramilitary groups [26].
   • If interviewers whose first language is different from the primary language of the country are to be trained in the primary language of the country, they may need additional training time. The 2001 California Health Interview Study found that it took fifteen percent longer, on average, to train Asian language speaking interviewers in English. Because some of the interviewers had weak English language skills, trainers spent the additional time defining terms, explaining concepts, etc. [5].
5. Provide study specific training.
   Rationale

   Interviewers and supervisors need to be familiar with the study's protocols in order to carry out their tasks. Depending upon the survey, they may need to learn the instrument's branching structure, the study's requirement for field coding, or the use of a respondent booklet, show cards, or other visual materials. There may be special instructions for implementing all or part of the survey that deviate from the standardized interviewing covered in general interviewer training. Interviewers should also be knowledgeable about the project objectives so that their actions help, not hinder, the overall goals. Both newly hired and experienced interviewers require training specific to the study at hand.

   Procedural steps

   • Allow sufficient time for study-specific training, depending upon the complexity of the study.
   • When possible, have the same team from the coordinating center train all interviewers to ensure standardization of study-specific protocols [34]. The team may provide regional trainings, traveling to where interviewers are located.
   • Include a large amount of practice and role playing using the questionnaire [34].
     • Consider having the interviewers complete a self-interview to become familiar with the survey instrument.
     • For role playing to be effective, prepare different scripts in advance so that the different branching structures of the interview, the nature of explanations that are permitted, and anticipated problems can be illustrated.
   • Provide interviewers with an Interviewer Project Manual/Study Guide that has been prepared by the coordinating center, with input from local collaborators. The manual is an important part of training and will serve as reference material while the survey is underway [12].
     • Complete and review the manual before training begins [12].
     • When appropriate, translate the manual into the languages used in the geographical areas encompassed by the study.
     • Include the following content in both the training agenda and the project manual:
       • General information about the project (e.g., the study's background and goals, funding sources, and principal investigators).
       • How to introduce the survey to respondents.
       • Eligibility and respondent selection procedures, if applicable.
       • Review of the survey instrument.
       • Data entry procedures (paper and/or computer-assisted).
       • Computer hardware and software usage, if appropriate (e.g., use of the laptop computer, email, and any other software packages).
       • Use of the sample management system.
       • Review of interview procedures and materials (e.g., informed consent materials and respondent incentive payments).
       • Review of study-specific probing conventions (e.g., when to probe a "don't know" response and/or an open-ended response).
       • Techniques for handling reluctance that are specific to the study (e.g., recommended responses to frequently asked questions) and are approved in advance by an ethics review committee (see Ethical Considerations in Surveys).
       • Nonstandardized interviewing, if appropriate for the study (e.g., event history calendars, time diaries, or conversational interviewing [8]).
       • Any specialized training for the study (e.g., procedures for taking physical measurements, instruction on interviewing minors or interviewing on sensitive topics, proxy interview protocol, protocol for handling respondent or interviewer distress, and recording observational data).
       • Procedures to be used for unusual cases, including general principles to be applied in dealing with unforeseen problems (e.g., how to report mistreatment of children or others that is observed while conducting an interview in the respondent's home).
       • Production goals and maintaining productivity.
       • Safety precautions and traveling issues.
       • The structure of the survey team and the role of all members of the team.
       • Procedures for editing and transmitting data.
       • Any other required administrative tasks.
     • The Project Manual/Study Guide is especially important if it is impossible to personally train interviewers (e.g., if interviewers must be trained via conference call or video).
   • Evaluate the training through an interviewer-debriefing session (i.e., provide the opportunity for trainees to give feedback on trainer performance, the sufficiency of time allocated to different topics, and the adequacy of practice exercises).
   • Certify the interviewers. Certification for study-specific tasks should include:
     • A complete role-play interview with a supervisor.
     • Certification by an appropriate trainer of any physical measurements that are included in the study.
     • Language certification, as appropriate (see Translation).
   • Supplement the initial training with periodic in-person seminars, telephone conference calls, and/or periodic bulletins or newsletters [26].
   • If data collection will extend for a long period of time, hold a brief refresher training course towards the middle of the field period [23].
     • This refresher training session is an opportunity to review various aspects of data collection, focusing on difficult procedures or on guidelines that are not being adhered to sufficiently by interviewers.
     • The session can also be used to provide feedback on what has been achieved to date.
     • Require even experienced interviewers to attend refresher training sessions, including sessions on standardized techniques.
   Lessons learned

   • If interviewers are being hired for one study only, general interviewer training techniques can be incorporated into study-specific training.
   • Glewwe [12] recommends up to a month of intense interviewer training (general and study specific) for inexperienced interviewers. The Survey of Health, Ageing, and Retirement in Europe (SHARE) requires a minimum of two full days in addition to the basic interviewer techniques training for new interviewers [1]. Similarly, the World Health Survey (WHS) recommends three full days of study-specific training [34].
   • Sometimes it is not feasible for the same team to train all interviewers, particularly in very large cross-cultural studies. If this is the case, other steps must be taken to ensure the standardization of study specific protocols:
     • One approach is the "train-the-trainer" model.
       • Training is generally done in one common language.
       • Each country or cultural group sends one or more individuals, who can understand and work in the language of the trainers, to the central training.
       • These representatives return to their own country or cultural group, adapt and translate the training materials as needed, and train the interviewers.
       • This model allows for tailoring at the country or cultural group level.
       • SHARE's train-the-trainer (TTT) program is one example of this approach [1] [4]. The University of Michigan's Survey Research Center, under contract to SHARE, created the TTT program. Each participating country sent a Country Team Leader, his or her Operator, and 2-3 trainers to the TTT sessions. Once the trainers had completed the TTT program, they used the training materials provided, translated if necessary, to conduct country-level interviewer training.
     • Another approach is the training center model [27].
       • A centralized training course is held, but language "regions" are represented.
       • This model is effective when it is not possible for every country to send trainers who are functional in the central trainer's language.
       • The training center model was used in the World Health Organization's Composite International Diagnostic Interview training sessions. For example, trainers from Lebanon were trained in the United States and subsequently trained the trainers in Lebanon, Oman, and Iraq.
     • Organizing training in steps (first training the trainers and then having them train the interviewers) increases the overall time needed for training and should be factored into the project timeline.
   • If the topic is extremely sensitive, additional specialized training may improve response rates and data quality. The WHO Multi-Country Study on Women's Health and Domestic Violence, fielded in multiple culturally diverse countries, found that previously inexperienced interviewers who had received specialized training obtained a significantly higher response rate and significantly higher disclosure rate of incidences of domestic violence than did experienced interviewers who had not received the additional training [17].
   • Training interviewers in adaptive behavior, such as tailoring responses to respondent concerns or nonstandardized conversational interviewing, can be time-consuming and could increase training costs [22].
   • Field interviewers often work some distance away from their trainers and supervisors. Before sending the interviewers to their assigned areas, some organizations have found it useful to have them conduct a few interviews close to the training locale. Afterward, they meet with the trainer, discuss their experiences, and check their questionnaires. Any problems or misunderstandings can be identified and rectified more easily than if they had occurred in a more remote area.
6. Institute and follow appropriate quality control measures.
   Rationale

   Quality control (QC) is a procedure or set of procedures intended to ensure that a product or service adheres to a defined set of quality criteria or meets the requirements of the client. The implementation of quality control measures enhances the reliability and validity of the survey data and maximizes comparability of these data across cultures. To implement an effective QC program in a cross-cultural survey context, the coordinating center must first decide which specific standards must be met. Then real-world data must be collected and the results reported back to the coordinating center. After this, corrective action must be decided upon and taken. Finally, the QC process must be ongoing to ensure that remedial efforts, if required, have produced satisfactory results.

   Procedural steps

   • Considering the factors enumerated in Guideline 3, establish a checklist of minimum interviewer candidate requirements (e.g., educational level, reading/writing fluency, language skills, and computer skills).
     • Require recruiters to complete the checklist as they screen each interviewer candidate. If specific assessment tests are used (e.g., to evaluate language skills), record each candidate's performance on the test.
     • Accept only those candidates who meet the predetermined minimum requirements.
     • To ensure accountability, require the recruiter to sign or initial checklists and assessment tests.
   • Take attendance at general interviewing techniques and study-specific training sessions.
     • Dismiss candidates who fail to attend a predetermined minimum number of training sessions, or make arrangements to train them individually on the missed material.
     • Keep a signed written record of the training completed by each candidate.
   • At the end of general interviewer training, evaluate the knowledge of the interviewer candidates, as described in Guideline 4.
     • Require all trainers to use the same evaluation criteria.
     • Dismiss or retrain those candidates who fail to attain predetermined minimum standards.
     • Keep a signed written record of each candidate's performance on the evaluation measures.
   • At the end of study-specific training, certify the interviewer candidates, as described in Guideline 5.
     • Require all trainers to use the same evaluation criteria for certification.
     • Dismiss or retrain those candidates who fail to attain predetermined minimum standards.
     • Keep a signed written record of each candidate's performance on the certification tests.
   Lessons learned

   Including quality control protocols as part of the overall survey design, and implementing them from the start, permits the survey organization and the coordinating center to monitor performance and to take immediate corrective action when required. For example, if many interviewer candidates fail to pass the study-specific certification test, additional training could be provided. Afterward, the candidates would be tested again. Those passing the certification test could then be sent out into the field.

7. Document interviewer recruitment and training.
   Rationale

   Comprehensive documentation helps analysts correctly interpret the data and serves as a resource for later studies.

   Procedural steps

   • Document the recruitment effort for enrolling field staff on the project, including:
     • Any special criteria used in reviewing field staff employment applications (e.g., language proficiency and/or special knowledge and skills, such as taking physical/biological measurements).
     • The way in which language fluency was assessed, as appropriate for the study.
     • Recruitment scripts and sources used to recruit field staff, as well as an evaluation of the success of the recruitment strategies.
     • Interviewer characteristics (e.g., gender, age, race, length of tenure as interviewer).
     • Characteristics of the multilingual interviewing staff in terms of the percent certified to interview by language.
     • Interviewer pay structure (e.g., hourly or per completed interview) and any bonus program (e.g., amount and when or under what circumstances these bonuses were offered).
   • Document the general and study-specific training, including:
     • Number of training sessions conducted.
     • Number of training days and dates.
     • Background of the trainers, including expertise in training and in any substantive areas as applicable to the survey.
     • Copy of the training agenda(s) (i.e., list of topics covered).
     • All written materials that were used (e.g., the interviewer manual/study guide, trainer/facilitator guide and supplemental training materials).
     • Certification procedures (e.g., scripted certification interview with a supervisor or other staff, written or online test on general interviewing procedures, live practice interviewing with potential respondents).
   • Document any issues encountered (e.g., if the recruitment plan failed to produce a sufficient number of qualified interviewers or interviewer attrition was unexpectedly high, necessitating a second round of recruitment and training; the training agenda did not provide adequate time for hands-on practice; or the ratio of trainers to trainees was inadequate) and suggestions for future studies.
   • Document all indicators of quality control (QC) and any decisions made to change the protocol in order to maintain high levels of quality.
   Lessons learned

   • Documenting the recruitment effort, including method(s) of recruiting, number of candidates recruited, and number of candidates screened, as well as post-study documentation of interviewer retention, is also useful for other projects. This information can guide future recruitment strategies and help estimate the number of recruits needed to provide sufficient interviewers for data collection in similar studies.
   • Documentation of general and study-specific training can pinpoint areas needing improvement in future training efforts.

Return to top

Previous chapter | Next chapter | Home

© 2008 The authors of the Guidelines hold the copyright. Please contact us if you wish to\n publish any of this material in any form.

View Chapters

• Introduction
• Study Structure
• Tenders, Bids, & Contracts
• Ethical Considerations
• Sample Design
• Questionnaire Design
• Translation
•      Translation Assessment
•      Language Harmonization
•      Translators
•      Translation Budgeting
•      Translation Scheduling
•      Translation Tools
• Adaptation
• Survey Instrument Design
• Pretesting
• Interviewer Recruitment & Training
• Data Collection
• Harmonization of Data
• Data Processing
• Dissemination
• Assessing Quality
• Glossary
• Bibliography