Skip to navigation
Webpage last modified: 2010-Sept-29
The following guidelines outline a number of study, organizational, and operational considerations which arise when structuring a cross-cultural survey or any survey involving multiple countries, regions, or languages. Several factors will influence how the overall study is designed and later implemented, including the source(s) and flow of funding, the availability of human and technical resources, the best way of contacting and collecting data from respondents, and the research infrastructure. All of these will vary from country to country and culture to culture. Yet, before much time is spent determining the study structure, it is critical to clearly define a study purpose because it drives all subsequent decisions, especially if conflicts between cross-cultural and local interests arise.
Cross-cultural surveys are organized in many different ways, and each has its advantages and disadvantages. These guidelines predominately address a structure with a coordinating center that designs the overall study and assumes the central organizational responsibility to the contracted survey organizations in each country where the study will be carried out. This type of organizing structure is often used in large-scale, cross-cultural surveys. Although not described here, there are situations where the coordinating center is also responsible for data collection in some or all countries. A coordinating center should include people from different countries, institutions, and affiliations. Given this focus, this chapter's primary audience is members of a coordinating center.
With this organizational structure, the coordinating center will specify the operational structure of the survey for each country to follow. It should determine what elements will be standardized across countries and what elements will be localized; there is a balance between standardization of implementation and adaptation to the cultural context. The coordinating center should inform the survey organizations of the quality standards necessary to execute the study.
Figure 1 shows study, organizational, and operational structure within the survey production process lifecycle (survey lifecycle) as represented in these guidelines. The lifecycle begins with establishing study structure (Study, Organizational, and Operational Structure) and ends with data dissemination (Data Dissemination). In some study designs, the lifecycle may be completely or partially repeated. There might also be iteration within a production process. The order in which survey production processes are shown in the lifecycle does not represent a strict order to their actual implementation, and some processes may be simultaneous and interlocked (e.g., sample design and contractual work). Quality and ethical considerations are relevant to all processes throughout the survey production lifecycle. Survey quality can be assessed in terms of fitness for intended use (also known as fitness for purpose), total survey error, and the monitoring of survey production process quality, which may be affected by survey infrastructure, costs, respondent and interviewer burden, and study design specifications (see Survey Quality).
Studies involving multiple cultures, countries, regions, or languages may benefit from the use of mixed methods. A mixed methods study "involves the collection or analysis of both quantitative and/or qualitative data in a single study in which the data are collected concurrently or sequentially, are given a priority, and involve an integration of the data at one or more stages in the process of research" .The different toolkits of qualitative and quantitative data collection methods can be complementary for studies of cross-cultural similarities and differences in attitudes and behaviors that often require different kinds of methods and evidence . Van de Vijver and Chasiotis  also provide an in-depth discussion and a conceptual framework for mixed methods studies. Researchers wanting to undertake a mixed methods design or to incorporate mixed methods approaches at different stages of the survey lifecycle may include these considerations when designing the study. Examples and references for mixed methods approaches are provided in the pretesting, questionnaire design and data collection chapters.
Goal: To establish the study's overall structure and locus of control at all levels and across all aspects of the study's design and implementation, and to communicate this structure to each participating country's survey organization.
Before work is done to organize or operationalize a study, the empirical aims of the research should be understood by all involved. There should be a clear direction and purpose of the research. In order to move the study goals from ideas to a concrete design, a structure of survey tasks should be clearly defined by the coordinating center. This task framework should take into account the cross-cultural nature of the survey.
The coordinating center should first determine its own organizational structure and then set the organizational standards for participating survey organizations. In order to manage a cross-cultural survey efficiently and effectively, roles and responsibilities must be clearly delineated and communicated throughout all levels. This can be accomplished when the central coordinating center works together with local expertise in each participating country.
Operational specifications ensure that critical aspects of the survey process are defined and then can be controlled. They simultaneously identify required or expected quality standards and help ensure comparability across countries. The specifications should, therefore, be detailed (and measurable, when possible) with clearly delineated deliverables from the participating survey organizations at each task of the survey. In addition, each specification should be justified with a rationale. The specifications form the basis of the country-level tenders and subsequent contracts between the coordinating center and survey organizations (see Tenders, Bids, and Contracts).
The goal of quality standards is to achieve excellence for all components related to the data  . Setting quality standards is critical to ensuring the same level of methodological rigor across countries . Local adaptations will be necessary and appropriate for some aspects of implementation of the study, but any adaptation in the procedure or instrument should be thoroughly discussed, evaluated, and documented beforehand . Frequent measurement and reporting to the coordinating center, along with sufficient methodological support, should allow for timely intervention if problems do arise.
Documentation of procedures ensures that the survey is transparent and allows for replication. Documentation should be detailed and occur throughout the survey lifecycle. If documentation does not occur until the end of the survey or even the end of the survey task, details will likely be lost or forgotten. Therefore, it is important to determine documentation requirements before the study is initiated. The coordinating center should first establish its own documentation procedures and then set documentation procedures for participating survey organizations.
When determining the study structure of a cross-cultural survey, it is important that all necessary survey tasks are identified. Below are examples of survey tasks that correspond with each chapter of the Cross-Cultural Survey Guidelines. This appendix provides example considerations for the completion of each task; please see the subsequent chapters for more detailed guidance. By creating a detailed list of survey tasks, the coordinating center can become assured that no aspect of the study structure has been overlooked and can then use this list to assign organizational responsibilities.
The source and flow of funding impact the structure of a cross-cultural survey. Below are examples of how five large-scale, cross-cultural survey programs have been funded. Please see the websites of these programs for further information.
Below are descriptions of the organizational structures that have been used on three large-scale, cross-cultural survey programs. These examples are only illustrative. Please visit the survey programs' websites for more information about their structure.
 Aitken, A., Hörngren, J., Jones, N., Lewis, D., & Zilhäo, M. J. (2003). Handbook on improving quality by analysis of process variables. Luxembourg: Eurostat. Retrieved March 27, 2010, from http://epp.eurostat.ec.europa.eu/portal/page/portal/quality/documents/HANDBOOK%20ON%20IMPROVING%20QUALITY.pdf
 Biemer, P., & Lyberg, L. (2003). Introduction to survey quality. Hoboken, NJ: John Wiley & Sons.
 Börsch-Supan, A., Jürges, H., & Lipps, O. (2003). SHARE: Building a panel survey on health, aging and retirement in Europe. Mannheim, Germany: Mannheim Research Institute for the Economics of Aging. Retrieved March 26, 2010, from http://www.mea.uni-mannheim.de/mea_neu/pages/files/ nopage_pubs/k3pzhnwk1t1zjymk_dp32.pdf
 Cleland, D.I. & Gareis, R. (Eds.). (2006). Global project management handbook (2nd ed.). New York, NY.: McGraw-Hill.
 Creswell, J.W., Plano Clark, V.L., Gutmann, M.L. & Hanson, W.E. (2003). Advanced mixed methods research designs. In A. Tashakkori and C. Teddlie (Eds), Handbook on Mixed Methods in the Behavioral and Social Sciences. (pp. 209-240). Thousand Oaks, CA: Sage Publications.
 Deming, E. (1986). Out of the crisis. Cambridge, MA : Cambridge University Press.
 Federal Committee on Statistical Methodology. (1983). Statistical policy working paper 9: Contracting for surveys. Washington, DC: Office of Management and Budget. From http://www.fcsm.gov/working-papers/sw9.html
 Fitzgerald, R., & Jowell, R. Measurement equivalence in comparative surveys: The European Social Survey (ESS) - from design to implementation and beyond. Unpublished manuscript.
 Griffin, R.W. & Moorehead, G. (2009). Organizational behavior - Managing people and organizations (8th ed.). Cincinnati, OH: South-Western Educational Publishing.
 Groves, R. M., Fowler, F. J., Jr., Couper, M. P., Lepkowski, J. M., Singer, E., & Tourangeau, R. (2009). Survey Methodology (2nd ed.). Hoboken, NJ: John Wiley & Sons.
 International Organization for Standardization. (2005). Market, opinion and social research-terms, definitions and service requirements. (ISO 20252). Geneva. Retrieved April 30, 2010, from https://www.astandis.at/shopV5/Preview.action%3Bjsessionid=0D8F503B3BD48375403D96257760FABB?preview=&dokkey=342276&selectedLocale=en/
 Jowell, R. (1998). How comparative is comparative research? American Behavioral Scientist, 42(2), 168-177. Retrieved March 26, 2010, from http://www.soc.aau.dk/fileadmin/user_upload/kbm/VoF/Kurser/2010/Research-design/litterature/Jowell-Roger-Comparative-research.pdf
 Juran, J.M. & De Deo, J.A. (2010). Juran's quality handbook: The complete guide to performance excellence (6th ed.). New York: McGraw-Hill.
 Lyberg, L. E., & Biemer, P. P. (2008). Quality assurance and quality control in surveys. In E. D. de Leeuw, J. J. Hox, & D. A. Dillman (Eds.), International handbook of survey methodology. New York/London: Lawrence Erlbaum Associates/Taylor & Francis Group.
 Mohler, P. Ph., Pennell, B-E., & Hubbard, F. (2008). Survey documentation: Toward professional knowledge management in sample surveys. In E. D. de Leeuw, J. J. Hox, & D. A. Dillman (Eds.), International handbook of survey methodology (pp. 403-420). New York/London: Lawrence Erlbaum Associates/Taylor & Francis Group.
 Montgomery, D.C. (2005). Introduction to statistical quality control (5th ed.). Hoboken, NJ: John Wiley & Sons.
 Office of Management and Budget. (2006). Standards and guidelines for statistical surveys. Washington, DC: Office of Information and Regulatory Affairs, OMB. Retrieved January 10, 2010, from http://www.whitehouse.gov/omb/inforeg/statpolicy/standards_stat_surveys.pdf
 Pennell, B-E., Mneimneh, Z., Bowers, A., Chardoul, S., Wells, J. E., Viana, M. C., et al. (2009). Implementation of the World Mental Health survey initiative. In R. C. Kessler & T. B. Üstün (Eds.). Volume 1: Patterns of mental illness in the WMH surveys. Cambridge, MA: Cambridge University Press.
 Ryan, T.P. (2000). Statistical methods for quality improvement (2nd ed.). Hoboken, NJ: John Wiley & Sons.
 United Nations. (2005). Household surveys in developing and transition countries. New York: United Nations, Department of Economic and Social Affairs. Retrieved March 26, 2010, from http://unstats.un.org/unsd/HHsurveys/pdf/Household_surveys.pdf
 van de Vijver, F. J. R. & Chasiotis, A. (2010). Making methods meet: Mixed design in cross-national research. In J. Harkness, B. Edwards, M. Braun, T. Johnson, L. E. Lyberg, P. Mohler, B-E. Pennell, & T. Smith. (Eds.), Survey methods in multicultural, multinational, and multiregional contexts. Hoboken, NJ: John Wiley & Sons.
 Worcester, R., Lagos, M., & Basañez, M. (2000). Cross-national studies: Lessons learned the hard way. Paper presented at the 2000 WAPOR/AAPOR Conference, Portland, OR.
 European Social Survey European Social Survey (ESS). (2010). Retrieved May 03, 2010, http://www.europeansocialsurvey.org
 International Social Survey Programme (ISSP) Retrieved May 04, 2010, from http://www.issp.org
 Latinobarómetro. (2010). Retrieved May 04, 2010, from http://www.latinobarometro.org
 Survey of Health, Ageing, and Retirement in Europe Retrieved May 04, 2020, from http://www.share-project.org
 World Mental Health Surveys (2010) Retrieved May 04, 2010, http://www.hcp.med.harvard.edu/wmh
Heath, A., Fisher, S., & Smith, S. (2005). The globalization of public opinion research. Annual Review of Political Science, 8, 297-333. From http://www.annualreviews.org/doi/pdf/10.1146/annurev.polisci.8.090203.103000
Lynn, P. (2003) Developing quality standards for cross-national survey research: five approaches. International Journal of Social Research Methodology, 6(4), 323-336. Retrieved October 8, 2009, from http://www.soc.uoc.gr/socmedia/papageo/developing%20quality%20standards%20for%20cross-national%20surveys.pdf
@copy; 2008 The authors of the Guidelines hold the copyright. Please contact us if you wish to publish any of this material in any form.