Cross-Cultural Survey Guidelines

Assessing the Quality of Cross-Cultural Surveys Guidelines PDF

XV. Assessing the Quality of Cross-Cultural Surveys

Webpage last modified: 2009-Mar-03

Introduction

In mono-cultural surveys, assessing the quality of survey data requires adequate documentation of the entire survey lifecycle and an understanding of protocols used to assure quality. The assessment procedures and criteria become more complicated in cross-national research, which — in addition to methodological, organizational and operational barriers to the implementation of quality monitoring and producing documentation — may often include additional production processes, such as adaptation and translation of questions and pretesting in diverse contexts.

The framework adopted by these guidelines for assessing quality is informed by research on quality management, which defines quality in terms of three main criteria: fitness for use [8], total survey error [6], and survey process quality [9].

Discussions of quality often focus on fitness for use, survey error, or both. Fitness for use is the extent to which statistics meet the requirements of users of the data. Statistical agencies commonly apply this criterion when defining quality. The dimensions of quality that national statistical agencies use to define quality vary to some extent, but there is agreement on several key dimensions. For example: Are concepts, constructs, indicators, and measures relevant and appropriate for addressing research questions of interest? Are the data accurate? Accuracy is generally what is addressed in discussion of total survey error, that is, accuracy of a survey statistic in terms of construct validity, measurement, and representativeness.

Figure 1 shows seven dimensions of quality that are often used to assess the quality of official statistics (see for example, [2] [4]):

Relevance — do the data meet the needs of the client or users? In a multi-cultural context, where there will be many clients or users with possibly competing goals, the dimension of relevance becomes more challenging to fulfill.
Accuracy — are the data describing the phenomena that they were designed to measure? Accuracy refers to the sources of survey error [6] (e.g., sampling error, measurement error, nonresponse error, etc.; see Figure 2).
Timeliness — how much time has elapsed between the end of the data collection and when the data are available for analysis? One example of the challenge of providing timely data in cross-national research would be a political election study in nations with elections occurring in different time periods.
Accessibility — can data be easily obtained by users? In the cross-national context, data access can mean more than simply making data publicly available. Access, particularly in majority countries, may also need to include capacity building or training activities to make the data truly accessible to local populations. Country-level data access laws and regulations will also come into play.
Interpretability — are supplementary data available to analysts that describe the major characteristics and structure of the data (metadata) as well as data about the survey processes (paradata)?
Coherence — are the data available for further recombination with other statistical information for various secondary purposes, and
Comparability — to what extent are observed data differences due to genuine variation as opposed to other factors? The quality dimensions of coherence and comparability are the raison d'être for cross-national and cross-cultural survey research.

Appendix A provides a list of guidelines from specific modules, highlighting recommendations in relation to these dimensions of quality. Fitness for use on these dimensions may be affected further by high or low cost, burden, design constraints, and professionalism:

Cost — to what extent did cost play a factor in implementation decisions?
Burden — were the concerns of burden to respondents adequately considered?
Design Constraints — were there context-specific constraints on study design that may have had an impact on quality (for example, using a different mode of interview in one survey implementation than in others)?
Professionalism — are staff provided with clear behavioral guidelines and professional training, are there adequate provisions to ensure compliance with relevant laws, and is there demonstration that analyses and reporting have been impartial?

In order to provide documentation that allows users to determine the fitness of data for use, and enough information to assess the accuracy of data and their comparability across cultures, data producers need to provide documentation of process quality management efforts. There has been a move from postsurvey evaluation of quality to the use of monitoring and control during the survey process to ensure data quality [1]. Use of a process quality approach requires the use of quality standards, management for quality, and collection of standardized study metadata, question metadata, and process paradata [3]. Figure 3 shows the elements of process quality management that allow users to assess process quality, which are:

Quality Planning and Assurance — Developing and applying planned systematic activities to ensure that the project meets or exceeds expected goals. The outcome of these efforts may or may not be measurable.
Quality Monitoring and Control — Monitoring specific project results against a predetermined baseline to ensure that standards are met or exceeded. The results should be quantifiable, but may not cover all aspects of the project.
Quality Profile — Publishing a document summarizing the quality assurance plan and including the indicators collected during the quality control effort. Such a document may accompany the data and allows analysts to make an informed judgment about the overall quality and usability of the data [1].

Figure 1: Fitness for Use — Dimensions of Quality

Figure 2. Sources of Error That Affect Accuracy as a Dimension of Quality

A quality profile synthesizes information from other sources, documenting all aspects of the survey, providing indicators of process quality, sources of sampling and nonsampling error, and recommendations for improvement and further research. It provides the user all information available to help assess data quality in terms of fitness of use, survey error, and other factors. See [5] for one set of guidelines for such reports, and [7] and [10] for examples of actual quality profiles.

Figure 3. The Elements of Process Quality Management

Organizations and projects will vary in cost-quality tradeoffs that are made, as well as items that will be monitored for quality purposes. However, if each organization in a cross-cultural study provides standardized quality profiles with adequate information, users will be able to assess quality and comparability across cultures.

Except for Guidelines modules still under construction, Appendix B summarizes for each module recommended elements of quality planning and assurance, quality monitoring and control, and a quality profile.

Appendix A

The following table indicates specific recommendations in individual Guidelines modules related to dimensions of quality.

Dimension of Quality	Guidelines
Relevance To ensure that the data meet the needs of the client or users.	Clearly state the study's goals and objectives (see Study, Organizational, and Operational Structure). Conduct a competitive bidding process to select the most qualified survey organization within each country or location (see Tenders, Bids, and Contracts). While designing the questionnaire, ensure all survey questions are relevant to the study objectives (see Study, Organizational, and Operational Structure).
Accuracy To ensure that the data describe the phenomena they were designed to measure.	Pretest all the versions of the survey instrument to ensure that they adequately convey the intended research questions and measure the intended attitudes, values, reported facts and/or behaviors (see Pretesting). In order to reliably project from the sample to the larger population with known levels of certainty/precision, use probability sampling (see Sample Design). If possible, assess accuracy by looking at the differences between the study estimates and any available "true" or gold standard values (see Data Collection).
Timeliness To ensure that the data are available for analysis when they are needed.	Time data collection activities appropriately (see Data Collection). Create a study timeline, production milestones, and deliverables with due dates (see Study, Organizational, and Operational Structure).
Accessibility To ensure that the data can be easily obtained by users.	Establish procedures early in the survey life cycle to insure that all important files are preserved (see Dissemination of Survey and Statistical Data). Test archived files periodically to verify user accessibility (see Dissemination of Survey and Statistical Data). Create electronic versions of all project materials whenever feasible (see Dissemination of Survey and Statistical Data). Produce and implement procedures to distribute restricted-use files, if applicable (see Dissemination of Survey and Statistical Data).
Interpretability To ensure that supplementary meta- and paradata are available to analysts.	At the data processing stage of the study, create a codebook which provides question-level metadata that are matched to variables in the dataset. Metadata include variable names, labels, and data types, as well as basic study documentation, question text, universes (the characteristics of respondents who were asked the question), the number of respondents who answered the question, and response frequencies or statistics (see Data Processing and Statistical Adjustment). Collect and make available process data collected during data collection, such as timestamps, keystrokes, and mouse actions ("paradata") (see Survey Instrument Design).
Coherence To ensure that the data can be combined with other statistical information for various, secondary purposes.	Create a clear, concise description of all survey implementation procedures to assist secondary users. The Study, Organizational, and Operational Structure chapter lists topics which should be included in the study documentation; there are also documentation guidelines within each chapter. Provide data files in all the major statistical software packages and test all thoroughly before they are made available for dissemination (see Dissemination of Survey and Statistical Data). Designate resources to provide user support and training for secondary researchers (see Dissemination of Survey and Statistical Data). See Harmonization of Survey and Statistical Data for a discussion of the creation of common measures of key economic, political, social, and health indicators.
Comparability To ensure, as much as possible, that observed data differences are due to genuine variation rather than other factors.	Establish minimum criteria for inclusion in a cross national survey dataset, if applicable, as follows: Minimize the amount of undue intrusion by ensuring comparable standards for informed consent and resistance aversion effort, as well as other potentially coercive measures such as lage respondent incentives (see Ethical Considerations in Surveys). Define comparable target populations and verify that the sampling frames provide adequate coverage to enable the desired level of generalization (see Sample Design). Minimize the amount of measurement error attributable to survey instrument design, including error resulting from context effects, as much as possible (see Survey Instrument Design). Minimize or account for the impact of language differences resulting from potential translations (see Translation and Adaptation). Minimize the effect interviewer attributes have on the data through appropriate recruitment, selection, and case assignment; minimize the effect that interviewer behavior has on the data through formal training (see Interviewer Recruitment, Selection, and Training). Identify potential sources of unexpected error by implementing pretests of translated instruments or instruments fielded in different cultural contexts (see Pretesting). Reduce the error associated with nonresponse as much as possible (see Data Collection for a discussion of nonresponse bias and methods for increasing response rates). Minimize the effect that coder error has on the data through appropriate coder training (see Data Processing and Statistical Adjustment). If possible, provide a crosswalk between survey instruments fielded at different times or for different purposes, but using the same questions, to facilitate analysis and post-survey quality review (see Dissemination of Survey and Statistical Data).
Professionalism To ensure that staff are provided with clear behavioral guidelines and that there are there adequate provisions to ensure compliance with relevant laws.	Train staff members to keep confidential both identifying material and all information given by respondents, to the extent allowed by law; require them to sign a pledge of confidentiality or to provide assurance in some form that they will maintain confidentiality (see Ethical Considerations in Surveys) . Train staff to comply with government laws and regulations on the storage and retention of survey data (see Ethical Considerations in Surveys). Equip staff involved in design, data collection, and analysis with appropriate skills to perform scientifically rigorous research (see Ethical Considerations in Surveys).
Cost and Burden To ensure that cost and respondent burden are considered when making implementation decisions.	Monitor costs at all stages of the survey (see Study, Organizational, and Operational Structure). Keep respondent burden as low as possible, ensuring that each question in the survey addresses a specific measurement goal, balancing the need for information against the effort required to complete additional questions, and asking questions in a way that is easy for respondents to answer (see Ethical Considerations in Surveys) .

Appendix B

The following table summarizes recommended elements of process quality management relevant to each module in these guidelines (except those still under construction). These are meant to reflect quality management at two levels: (1) at the overall study level; and (2) at the national organization level.

At the study level, the quality profile includes a summary of each organization’s performance; at both the study and individual organization levels, the quality profile includes a set of recommendations for improvement. The quality profile also summarizes any methodological studies in an area (for example, methodological studies relevant to ethics standards). Not all modules have specific measures for monitoring and controlling quality. Even without clear individual rates or measures of quality, there often may be reports on quality assurance activities that facilitate assessing quality.

CCSG Module	Quality Planning and Assurance	Quality Monitoring and Control	Quality Profile
Study, Organizational, and Operational Structure	Study goals and objectives Leadership, roles, and responsibilities Timeline Deliverables Quality standards Budget Create framework and structure of responsibilities and tasks Arrange regular meetings of working group and team leaders Develop communication flowchart Develop quality management plan	Cost and timeline monitoring reports Recommended corrective actions	Study goals and objectives All study implementation procedures Any modifications to study protocol Summary of each organization’s performance Summary of planned methodological studies Recommendations for improvement
Tenders, Bids, and Contracts	Type of contract offered Study specifications Evaluation criteria for bids Prepare tender based on study specifications Conduct competitive bidding process within each country Evaluate bids and select a survey organization in each country	Reports on bidding organization scores	Study specifications Evaluation criteria for bids Recommendations for improvement
Ethical Considerations	Standards for ethical and scientific conduct Local and national human subject regulations and legislation Ethical guidelines in project management and human resource management Voluntary informed consent protocol and procedures Procedures for ethics training of project staff Comprehensive plan for protection of confidentiality Review and apply ethical standards, best practices and relevant regulations and legislation in designing study and collecting and disseminating survey data Develop and apply knowledge of local customs and norms relevant for designing culturally-sensitive survey protocols Pretest consent protocol and forms to ensure comprehension Assess respondent burden (overall and by subgroup, if appropriate) Train project staff on ethics Have project staff sign pledge of confidentiality Complete ethics review submission and maintain documentation of submission materials Review recorded interviews and monitoring to assure adherence to informed consent procedures Monitor implementation of confidentiality protocols and procedures Perform audits to determine adherence to confidentiality protocols and procedures Securely store signed pledges of confidentiality and consent forms Maintain records of all ethics review committee correspondence Conduct verification to detect possible interview falsification Conduct disclosure analysis	Reports on pretesting of consent protocol Report on assessment of respondent burden Reports on staff completion of ethics training Reports on results of review of implementation of informed consent procedures (percent of cases reviewed, percent of cases failing to follow procedures, actions taken, etc.) Reports on interview falsification (percent of cases reviewed, percent of reviewed cases falsified, subsequent actions taken, etc.) Reports on any actual or potential breaches of confidentiality, security, or other adverse event , including any resulting changes to study protocol Reports on disclosure analysis Reports of any failures of statistical disclosure control Recommended corrective actions Recommended preventive actions Updates to standards, best practices, and quality management plan	Description of voluntary consent and confidentiality procedures Copies of materials provided to respondents as part of informed consent process Summary of respondent burden assessment Description of ethics training for project staff Summary of ethics committee review Summary of review of recorded interviews regarding the implementation of informed consent procedures (percent of cases reviewed, percent of cases failing to follow procedures, actions taken, etc.) Summary of falsification findings (percent of cases reviewed, percent of reviewed cases falsified, subsequent actions taken, etc.) Summary of any reported actual or potential breaches of confidentiality (including failures of statistical disclosure control) and actions taken to increase security and confidentiality of data Description of disclosure analysis methods and summary of findings Summary of each organization's performance Recommendations for improvement
Sample Design	Target and survey population definitions Descriptions of sampling frame(s), including definitions of strata and clustering units Selection procedure(s) and estimates of probability of selection at each stage Desired level of precision Quality control procedures for frame construction and sample selection processes Produce, update and/or clean sample frame(s), as needed Calculate sample size Implement selection procedure(s)	Coverage error estimates Reports on percentage of duplicate and ineligible units in frame(s) Recommended corrective actions Recommended preventive actions Updates to standards, best practices, and quality management plan	Target and survey population definitions, including inclusion/ exclusion criteria Sampling frame(s) Description of all stages of selection procedure(s), including any stratification and/or clustering Sample size(s) Time dimension of design (e.g., one time cross sectional, fixed or rotating panel) Documentation of probabilities of selection and weights calculations If applicable Any oversampling Number of replicates fielded Substitution procedures Summary of each organization's performance Recommendations for improvement
Questionnaire Design	Under construction
Translation	Under construction
Adaptation	Under construction
Survey Instrument Design	Instrument specification guidelines Comprehensive design evaluation plan, including goals, evaluation techniques, and timeline Quality assurance metrics (e.g., questionnaire and item timings, review of computer-assisted application audit trails, behavior/event codes) Provide clear instrument specifications and/or data dictionary Establish cross-cultural team with skills appropriate for assessment of design Perform design and report on design assessments Review quality assurance metrics reports Make recommendations for improvement	Quality assurance metrics reports Design evaluation reports Inter-coder reliability reports Audit trail reports Recommended corrective actions Recommended preventive actions Updates to standards, best practices, and quality management plan	Instrument specification guidelines Procedures for design evaluation Results of design evaluations Documentation and results of quality monitoring and control Summary of each organization's performance Recommendations for improvement
Pretesting	Pretesting plan, including pretest goals, evaluation techniques, and timeline, and budget Standard procedures for staff training Provide staff training and certification Review recordings of focus groups and cognitive interviews for staff errors Provide retraining as necessary Test for inter-coder reliability if appropriate	Staff error rates Inter-coder reliability reports Recommended corrective actions Recommended preventive actions Updates to standards, best practices, and quality management plan	Pretest procedures documentation Pretest training documentation Pretest findings, change recommendations, and changes made Staff error rates Summary of each organization's performance Recommendations for improvement
Interviewer Recruitment and Training	Minimum standards for employment Study-specific requirements (e.g., gender, language, etc.) Assessment tests Minimum interviewer requirements checklist Criteria for dismissal or follow-up training Standard certification procedures Complete checklist during candidate screening Take attendance during training Certify candidates Dismiss or retrain candidates who fail certification Maintain written records of results of candidates' certification tests	Training attendance reports Candidate certification reports Certification rates Recommended corrective actions Recommended preventive actions Updates to standards, best practices, and quality management plan	Employment criteria General and study-specific training documentation Certification procedures Certification rates for training and follow-up training Summary of each organization's performance Recommendations for improvement
Data Collection	Target response, contact, and completion rates Target hours per interview Percentage of interviewer cases to be verified Verification questions Interviewer performance checklist Criteria for interviewer dismissal or supplementary training Establish a sample management system Monitor interviewers Review paper coversheets and/or questionnaires Verify case disposition codes and selected responses Dismiss or retrain interviewers with substandard performance	Overall, by key respondent groups, and by interviewer: Screening rates Eligibility rates Response rates Refusal rates Contact rates Completion rates Interview length Hours per interview Number of completed interviews Interviewer performance outcomes Recommended corrective actions Recommended preventive actions Updates to standards, best practices, and quality management plan	Mode(s) of data collection and protocol for determining mode(s) to use Sample management system Study materials Screening/respondent selection procedures Number of completed interviews, overall and by mode Proxy interview protocol Respondent incentives Interviewer incentive protocol Techniques to maximize response (e.g., prenotification and recontact protocol) Outcome rates (e.g., response, refusal, noncontact), overall and by key respondent groups Dates of data collection Interviewer monitoring procedures and outcomes Verification form(s) and outcomes Any validation study descriptions and outcomes Summary of each organization's performance Recommendations for improvement
Harmonization of Survey Data	Standard codebook specifications Standard procedures for collecting and producing national data files Comprehensive plan for harmonization of cross-cultural data files Procedures for testing harmonized files with knowledgeable end-users Create cross-cultural monitoring team Periodically review analytic results to allow for changes in harmonization rules Review end-user test results Make recommendations for harmonization process improvement	Reports on analytic results End-user test reports Recommended corrective actions Recommended preventive actions Updates to standards, best practices, and quality management plan	Documentation of specification and procedures standards Documentation of conversion and harmonization decisions Results of end-user tests Recommendations for improvement
Data Processing and Statistical Adjustment	Percent of manually entered questionnaires to be verified Criteria for data entry staff dismissal or supplementary training Items to be coded Coding protocol (manual or automatic) Percent of manually coded cases to be check coded Minimum acceptable inter-coder reliability Data cleaning protocol Appropriate statistical software Appropriate statistical adjustments (e.g., imputation, weights) Appropriate standard error estimation Quality control procedures for calculation of statistical adjustments and variance estimation Train data entry and data coding staff Verify data accuracy Develop coding scheme(s) Assess inter-coder reliability Check outliers Clean data Calculate statistics	Data entry accuracy rate Inter-coder reliability Number of responses that were coded automatically; were changed after dictionary updates; and/or were coded in error Key process statistics for editing: Edit failure Recontact rate Correction rate Recommended corrective actions Recommended preventive actions Updates to standards, best practices, and quality management plan	Data processing Data coding and data entry training documentation Evaluation protocol for data coding and data entry staff and outcomes Items that were coded or re-coded Coding reliability Data entry verification protocol and outcomes Data cleaning protocol Statistical adjustment Rationale for assigning sample identification numbers Calculation of outcome rates (e.g., response, refusal, noncontact), weighted and unweighted Standard error estimates Percent item missing data Where applicable: Imputation method(s) Generation of weight(s) Trimming of weight(s) Scaling of weight(s) Adjustment(s) for differential nonresponse Poststratification adjustment(s) Summary of each organization's performance Recommendations for improvement
Dissemination of Survey and Statistical Data	Procedures for testing accessibility of archives with knowledgeable end-users Procedures for electronic preservation of files Procedures for testing files with major statistical packages Create electronic versions of all files Provide data files in all major statistical software packages Designate resources to provide user support and training for secondary researchers Review results of end-user tests	Archive test reports Recommended corrective actions Recommended preventive actions Updates to standards, best practices, and quality management plan	Description and classification of target users and their needs Results of user satisfaction assessments Summary of conditions of access to data, accompanying documentation, and user feedback Distribution reports (dataset requests, Web hits, downloads, etc.) Recommendations for improvement

Glossary

Bias: A systematic difference between the survey estimate of the population parameter and the true value in the population.
Coding: Translating nonnumeric data into numeric fields.
Contact rate: The proportion of all cases in which some responsible member of the housing unit was reached by the survey.
Context effects: The impact of question context, such as the order or layout of questions, on survey responses.
Coversheet: Electronic or printed materials associated with each case that identify information about the case, e.g., the sample address, the unique identification number associated with a case, and the interviewer to whom a case is assigned. The coversheet often also contains an introduction to the study, instructions on how to screen sample members and randomly select the respondent, and space to record the date, time, outcome, and notes for every attempt.
Coverage Error: Survey error (variance or bias) that is introduced when some units in the target population are not included on the sampling frame.
Disposition code: A code that indicates the result of a specific call attempt or the outcome assigned to a sample element at the end of data collection (e.g., noncontact, refusal, ineligible, complete interview).
Element: A single unit of the sampling frame.
Imputation: Computational methods that assign one or more estimated answers for each item that previously had missing, incomplete or implausible data.
Hours Per Interview (HPI): A measure of study efficiency, calculated as the total number of interviewer hours spent during production (including travel, reluctance handling, listing, completing an interview, and other administrative tasks) divided by the total number of interviews.
Majority country: A country with low per capita income (the majority of countries).
Measurement error: Survey error (variance or bias) due to the measurement process; that is, error introduced by the survey instrument, the interviewer, or the respondent.
Metadata: Data that describes other data. The term encompasses a broad spectrum of information about the survey, from study title to sample design, details such as interviewer briefing notes, contextual data and/or information such as legal regulations, customs, and economic indicators.
Mode: Method of data collection.
Nonresponse error: Error (variance or bias) that is introduced when not all sample members participate in the survey (unit nonresponse) or not all survey items are answered (item nonreponse) by a sample member.
Nonresponse bias: Bias that is introduced when not all sample members participate in the survey or answer a survey item and those that do not (the nonrespondents) differ from the respondents on the measure of interest.
Paradata: Process data collected during data collection, such as timestamps, keystrokes, interviewer observations, etc.
Poststratification (adjustment): A statistical adjustment that assures that sample estimates of totals or percentages (e.g. the estimate of the percentage of men living in Mexico based on the sample) equal population totals or percentages (e.g. the estimate of the percentage of men living in Mexico based on Census data). The adjustment cells for poststratification are formed in a similar way as strata in sample selection, but variables can be used that were not on the original sampling frame at the time of selection.
Probability sampling: A sampling method where each element on the sampling frame has a known, non-zero chance of selection.
Proxy interview: An interview with anyone other than the person about whom information is being sought (e.g., parent, spouse).
Quality: Achieving excellence for all components related to the data.
Quality assurance: Statement of confidence that quality requirements will be fulfilled.
Quality control: Process focused on fulfilling quality requirements.
Replicates: Probability subsamples of the full sample design
Response rate: The number of completed interviews divided by the total estimated number of eligible sample persons.
Sample management system: A computerized and/or paper-based system used to assign and monitor sample cases and record documentation for sample records (e.g., time and outcome of each contact attempt).
Sampling error: Survey error (variance or bias) due to observing a sample of the population rather than the entire population.
Sampling frame: Lists or materials used to identify all elements (e.g., persons, households, establishments) of a survey population from which the sample will be selected. These lists or materials can include maps of areas in which the elements can be found, lists of members of a professional association, and registries of addresses or persons.
Survey estimate: The value yielded by a survey.
Survey population: The actual population from which the survey data are collected, given the restrictions from data collection operations.
Target population: The finite population for which the survey sponsor wants to make inferences using the sample statistics.
(Sampling) Units: Elements or clusters of elements considered for selection in some stage of sampling. For a sample with only one stage of selection, the sampling units are the same as the elements. In multi-stage samples (e.g., enumeration areas, then households within selected enumeration areas, and finally adults within selected households), different sampling units exist, while only the last is an element. The term primary sampling units (PSUs) refers to the sampling units chosen in the first stage of selection. The term secondary sampling units (SSUs) refers to sampling units within the PSUs that are chosen in the second stage of selection.
Weight(ing): A post-survey adjustment that may account for differential coverage, sampling, and/or nonresponse processes.

References

[1] Biemer, P.P, & Lyberg, L.E. (2003). Introduction to Survey Quality. Hoboken, NJ: Wiley.

[2] Brackstone, G. (1999). Managing Data Quality in a Statistical Agency. Statistics Canada, Survey Methodology, Catalogue No. 12-001-XPB, 25(2), 1-23;

[3] Couper, M.P., & Lyberg, L. (2005). "The Use of Paradata in Survey Research." Paper presented at the International Statistical Institute Meetings, Sydney, April.

[4] Eurostat (2003). Methodological Documents — Definition of Quality in Statistics. Report of the Working Group Assessment of Quality in Statistics, item 4.2. Luxembourg, October.

[5] Eurostat. (2003). Methodological Documents — Standard Report. Report of the Working Group Assessment of Quality in Statistics, item 4.2B. Luxembourg, October.

[6] Groves, R.M., Couper, M.P., Lepkowski, J.M., Singer, E., & Tourangeau, R. (2004). Survey Methodology. Hoboken, NJ: Wiley.

[7] Institute for Social and Economic Research [ISER] (2006). Quality Profile: British Household Panel Survey (Version 2.0). University of Essex. http://www.iser.essex.ac.uk/ulsc/bhps/quality-profiles/BHPS-QP-01-03-06-v2.pdf

[8] Juran , J.M., and Gryna, Jr., F.M. (1980). Quality Planning and Analysis, 2nd ed., McGraw-Hill, New York.

[9] Lyberg, L.E., Biemer, P., Collins, M., DeLeeuw, E.D., Dippo, C., Schwarz, N., & Trewin, D (eds.). 1997. Survey Measurement and Process Quality. New York: Wiley.

[10] U.S. Bureau of the Census (1998). SIPP Quality Profile. SIPP Working Paper No. 230 (3rd Edition). http://www.census.gov/sipp/workpapr/wp230.pdf

Return to top

Previous chapter | Next chapter | Home

XV. Assessing the Quality of Cross-Cultural Surveys

Introduction

Appendix A

Relevance

Accuracy

Timeliness

Accessibility

Interpretability

Coherence

Comparability

Professionalism

Cost and Burden

Appendix B

Glossary

References

View Chapters