• Home
• |
• Contact Us
• |
• User Guide

Tenders, Bids, and Contracts Guidelines

II. Tenders, Bids, and Contracts

Webpage last modified: 2008-Oct-28

Introduction

The following describes the process for preparing a tender, soliciting bids, and drawing up and executing a contract. The tender should be based on the specifications outlined in the Study, Organizational, and Operational Structure chapter. Bids should be obtained from as many qualified organizations as possible to ensure a high quality survey for the budget available. The aim of signing a contract is to create a legally-binding agreement between the organization coordinating the study and the data collection organization(s).

In cross-cultural studies, the contract is sometimes between a central coordinating center and a data collection organization in each country where the study will be carried out. In other instances, no central coordinating body is involved in contracting; contracts are arranged and signed at the national level. And there are other situations where contracts are signed with an international organization that is responsible for data collection in several countries. The guidelines presented here deal with the first case—outlining a competitive bidding process. However, many of the points made also apply to the other contracting arrangements.

These guidelines refer to a cross-cultural survey that is implemented only once, or to the first wave of a longitudinal, cross-cultural study. There will be differences in the contracting process of additional waves of a cross-cultural study. A more elaborate bidding process will occur when selecting the survey organizations participating in a longitudinal survey that serves as an infrastructure study.

Complications will inevitably arise over the course of the study, so it is important that the final, signed contract defines, in as much detail as possible, the specifications and expectations regarding procedures, players, and outcomes for all parts of the study across all participating organizations. Towards this end, the tender, bids, and resulting contract should all be as specific and detailed as possible [11].

Guidelines

Goal: To create and implement a contract, with agreed upon specifications, and within defined budget parameters and a legal framework in which leverage towards compliance is established.

1. Prepare a tender and conduct a competitive bidding process to select the most qualified survey organization within each country or location.
   Rationale

   The central coordinating center prepares a tender based on the study specifications and engages in a bidding process with several survey organizations. If the research capacity of a country is not already apparent, the bidding process is one way to illuminate this and to determine if any methodological or substantive expertise may be needed to supplement local resources. A competitive bidding process is not always possible; sometimes, there are only one or two competent survey organizations within the location being studied.

   Procedural steps

   • Prepare a tender based on the study specifications while adapting the tender for each individual country.
   • Decide between open tendering and restricted tendering.
     • Open tendering allows any survey organization to provide a bid. Open tendering may be necessary if the coordinating center is not familiar with the availability of qualified survey organizations in the location.
     • Restricted tendering limits the bidding process to a few survey organizations pre-selected by the coordinating center. Restricted tendering is often used when the coordinating center has prior knowledge of survey organizations that are capable of implementing their countries' portion of a cross-cultural study.
   • State in the tender which type of contract will be offered: fixed-price, cost-reimbursable, or time and material. The decision on which type of contract will be offered depends on the level of risk the coordinating center (or funding source) and survey organizations are willing to take [6].
     • A fixed-price (or lump-sum) contract requires stating upfront a fixed total price for the clearly-defined scope of work and deliverable(s). Fixed-price contracts also allow for bonuses if expectations are exceeded. The coordinating center incurs little risk while the survey organizations incur much risk.
     • A cost-reimbursable contract requires paying the survey organizations for the actual costs necessary to complete the agreed-upon scope of work and production of the deliverable(s); it may include paying them a fee — typically received as profit. Cost-reimbursable contracts also allow for bonuses if expectations are exceeded. These guidelines assume cost-reimbursable contracts. This type of agreement is riskier for the coordinating center than for the survey organizations. Thus, it is important for the coordinating center to carefully evaluate survey organizations during the bidding process. See Guideline 2 for a description for how survey organizations may separate out direct costs and indirect costs, which is the basic structure of these contracts.
     • A time and material (T&M) contract has elements of both the fixed-price and the cost-reimbursable contract. Time and material contracts may require a fixed level of effort by a specific class(es) of resources at the survey organizations or may have a variable level of effort by a specific class(es) of resources at an agreed-upon rate of pay for the specific class(es). These contracts may be open-ended, such that the exact price for the scope of work and/or deliverable(s) may not be determined when signing the contract.
   • Request bids in a language understood by the reviewers from the central coordinating center, or arrange for language resources for the reviewing team to enable them to evaluate the bids.
   • Provide bidders with the evaluation criteria [3]. Bidders should know what is expected at each stage of the survey lifecycle, as well as explicitly what deliverables are required at each stage.
   • Seek multiple bids.
   • Encourage consortium bids as seems relevant because, in contexts with sparse resources, partnerships may enable survey organizations to make stronger bids if together they have a broader set of proficiencies [10].
   • Set a timeline for the bidding process that still allows time for clarification/discussion between the contracting parties and for several iterations of bids.
   • Keep the bidding process transparent, open, and fair:
     • Provide the same level of help or assistance to every survey organization [3].
     • If new information becomes available that would be useful in preparing a bid, take care to distribute this information to all bidders.
   • Encourage bidding organizations to identify any elements required in the tender specifications that they are unable or unwilling to meet [1]. Doing so helps avoid bids which the bidding organizations will not fulfill.
   • Check bids for potential problems, such as the following:
     • Can a proper sampling frame be obtained (see Sample Design or Data Collection)?
     • Does the bidding survey organization have access to the sample elements on the frame (e.g., will political conflicts and/or travel restrictions limit the areas in which the survey organization can contact individuals) (see Sample Design or Data Collection)?
     • Is the concept of probability sampling understood and its implementation assured (see Sample Design)?
     • Are suitable protocols and trainers available for interviewer training and interviewer motivation (see Interviewer Recruitment, Selection, and Training)?
     • Are essential nonresponse bias reduction techniques realized (see Data Collection)?
     • Are adequate quality control procedures in place (see Assessing Quality)?
     • Are necessary facilities, such as hardware, software, and internet access available?
     • Is the specification of budget details adequate?
     • Are there local research "traditions," such as quota sampling or undocumented substitution, that may conflict with study specifications?
   • Form a bid evaluation team comprised of a substantive expert, a statistical advisor, a methodological advisor, and, as relevant, legal and local expertise; possibly include a pre-identified consultant [3].
     • Determine in advance the process for final decisions on survey organization selection, in case disagreements among the review team should arise.
     • Have each member evaluate the survey organizations individually and make written notes.
     • Organize among the team a group discussion of the strengths and weaknesses of various bids.
     • Even if there is only one bid, conduct evaluation as described above with notes and a group discussion.
     • If the final required work scope and budget cannot be met by the bidding organization(s), decide whether a new round of bids is necessary or if some other alternative is available.
   Lessons learned

   • Survey organizations may hesitate to mention any obstacles to conducting the study as outlined in the tender specifications for various reasons. Organizations should be encouraged in a culturally appropriate fashion to be open and explicit about legal requirements, restrictions, or any other constraints, such as cultural norms, that would conflict with the study specifications. Some obstacles may be quickly remedied if identified in advance. For example, it may be necessary to appoint male interviewers to some locations (such as lumber camps or mines) [8] or to notify gatekeepers of the study and explain the need to contact given respondents. Strategies and schedules should be developed to accommodate this.
2. Require the survey organizations to provide both detailed technical and business information when preparing their bids.
   Rationale

   Detailed information on technical and business aspects reduces the opportunity for misunderstanding to go unnoticed, and helps ensure that the study specifications have been fully understood and adequately accounted for in the budget. This guideline assumes the survey organizations are bidding for a cost-reimbursable contract (see Guideline 1).

   Procedural steps

   • Ask bidders to detail technical aspects of the study, such as:
     • Confidentiality and data protection practices (see Ethical Considerations in Surveys).
     • Sampling methods (see Sample Design guidelines). [1]
     • Pretesting methods (making sure, in addition to pretesting the questionnaire, that there is a pretest of the field procedure) (see Pretesting).
     • Interviewer characteristics (e.g., age, education, gender, and experience) (see Interviewer Recruitment, Selection, and Training).
     • Unique identification of the interviewers (especially when recording which interviewers contacted which sample elements in the sample management system).
     • Interviewer payment plan (typically by the hour or by completed interview) (see Interviewer Recruitment, Selection, and Training).
     • Interviewer employment conditions (i.e., employees of the survey organization or contract workers).
     • Ratio of interviewers to supervisors (see Data Collection).
     • Mode of data collection (if using a mixed mode design, whether multiple modes will occur concurrently or sequentially) (see Data Collection).
     • Production requirements (e.g., minimum number of contacts to obtain a complete interview).
     • Procedures for local data return (e.g., mail or electronic transfer of completed interviews and/or other materials).
     • Local quality monitoring.
     • Plans in place to address nonresponse bias (see Data Collection).
     • Procedures for processing, managing, and storing data (see Data Processing and Statistical Adjustment).
     • Facilities and equipment (e.g., computers, software, internet access, and e-mail).
     • Procedures developed to handle unexpected problems (i.e., risk management). [1]
   • Ask bidders to provide specific technical information about their survey organization's and staff's experience, such as:
     • Examples of similar studies the bidder has conducted (describing the size, complexity, topic, etc.).
     • Examples of the bidder's training and supervisory materials, details of procedures used, and example reports from studies previously conducted.
     • References or contact names regarding previously completed work.
     • Number and relevant qualifications of all levels of staff assigned to the study.
   • Ask bidders to organize the business information (i.e., components of the budget) by each major study task [2]. Try to think of each set of guidelines as a study task. Suggested components are the following:
     • Project management
     • Liaison work with the national project representative, as appropriate
     • Liaison work with the central coordinating center, including progress communications and debriefings
     • Creation of the sampling frame and preparation of the study sample (see Sample Design)
     • Preparation of the questionnaire and other field documents (e.g., layout, printing, and programming) (see Questionnaire Design and Survey Instrument Design)
     • Translation, as appropriate (this may not be handled by bidders) (see Translation guidelines)
     • Questionnaire developmental work, as appropriate (see Pretesting, Questionnaire Design, and Survey Instrument Design)
     • Interviewer training and debriefing sessions (see Interviewer Recruitment, Selection, and Training guidelines)
     • Postage and telephone costs
     • Fieldwork, including travel, incentives, etc. (see Data Collection guidelines),
     • Quality control, including the cost of the survey organization's routine controls as well as the specific requirements of the project upon which they are bidding, as appropriate
     • Entering/capturing, editing, and coding data (see Data Processing and Statistical Adjustment)
     • Producing required deliverables and documentation
   • Within each organized study task, ask bidders to prepare a detailed budget by labor categories, other direct costs, and indirect costs (which are typically calculated as a percentage of direct costs) [6]. For studies lasting longer than one year, include a cost-of-living increase [4]. For areas suffering rampant inflation, shorter frames may be more appropriate. The following provides only illustrative budget categories:
     • Base Salaries:
       • Project managers
       • Data managers
       • Fieldwork managers
       • Accountants
       • Assistants
       • Supervisors
       • Interviewers
       • Data entry operators
       • Drivers
       • Translators
       • Computer programmers
     • Remuneration:
       • Overtime
       • Incentive payments
     • Travel Allowances:
       • Project managers
       • Data managers
       • Fieldwork managers
       • Assistants
       • Supervisors
       • Interviewers
       • Drivers
     • Materials:
       • Vehicle rental
       • Fuel and maintenance
       • Computers
       • Printers
       • Computer/printer supplies
       • Photocopiers/fax machines
       • Office supplies
       • Communications (e.g., telephone, facsimile, and postage)
       • Publicity
       • Equipment maintenance
     • Printing Costs:
       • Questionnaire
       • Training manuals
       • Reports
       • Miscellaneous (e.g., maps, listings, and other manuals)
     • Consultant Costs:
       • International consultants (e.g., stipend, per diem, and travel)
       • Local consultants (e.g., stipend, per diem, and travel)
     • Overhead (i.e., indirect cost) and labor overhead (i.e., benefits) [4]
     • Contingency (possibly 10%) [5]
   Lessons learned

   • Clarifying aspects of the tenders, bids, and contracts can avoid possible complications later in the implementation of the survey. For example, in many countries the research tradition is to pay interviewers by the completed interview and not by hours worked. This may work well if all interviewer assignments are of the same difficulty and if the length of the interview administration is within well-defined limits. However, if assignments vary in difficulty (longer travel times, for example) or the length of the interview can vary widely, this will not work as well. If interviewers are paid by the number of interviews they complete, they might be tempted to optimize. They might, for example, use strategies to keep interviews as short as possible in order to be able to complete more cases. In the worst scenario, interviewers might be tempted to falsify the interview (i.e., interviewer falsification) (see Interviewer Recruitment, Selection, and Training).
   • Budgets are often underestimated, leading to potential overruns. It is critical to monitor the budget throughout the entire survey lifecycle (see Study, Organizational, and Operational Structure). If the budget is defined by components within each study task (especially to the level of detail of labor categories, other direct costs, and indirect costs), then less effort is required to systematically assess the state of budget.
3. Select the survey organization best suited to carry out the survey in each country or location within the constraints.
   Rationale

   The decision about selecting which survey organization or collaboration of organizations will carry out the study, based on pre-specified and agreed-upon evaluation criteria, is a critical one. A poor choice of an organization will divert attention and resources away from other aspects of the study and may have a lasting impact on the entire endeavor.

   Procedural steps

   • Use the following indicators as the basis of evaluation criteria for choosing an organization:
     • Local knowledge of the population of interest [11]
     • Organizational and staff expertise in the subject area and survey methods envisioned [3]
     • Knowledge of and experience with conducting similar types of survey (both the organization as a whole and the management/personnel assigned to the project) [1] [10]
     • Ability to estimate the costs to complete the entire work scope
     • Transparency of procedures
     • Organization of field staff, including the planned supervisory structure and implementation strategy (e.g., whether interviewers are stationed throughout study areas or travel extensively in teams to different sampled locations)
     • Demonstrated or projected ability to meet the timeline and various specified outcomes [10]
     • Demonstrated or projected availability of management staff and statistical support
     • References from and affiliation with other professional organizations
   • If working in unfamiliar contexts, find out as much as possible beforehand in order to negotiate strategies with the survey organizations. Do not simply rely on someone to both translate and negotiate. In particular:
     • Make use of local or regional feedback about the survey organizations. It can be very useful to ask local contacts (these may not be directly local but at least in the region) to provide information about the organizations.
     • Try to become aware of any local tendencies in terms of management and likelihood of acknowledging obstacles. Encourage people to point out difficulties in terms of the knowledge of local tendencies. If you lack knowledge of what could be involved and do not have someone suitable to act as an informant, introduce the topics you need to know about (for example, "We have sometimes found organizations fear to lose a contract if they admit they have trouble meeting requirements. We have learned to recognize information about local constraints as very important to our task. Is there anything you would like to raise with us?").
     • Learn to wait longer than you may be accustomed for a response/answer, and listen attentively for indirect mention of a constraint.
     • Try to become aware of local survey traditions or their absence. If through preparation for local negotiations the study specifications run counter to local traditions, use the local framework as a way to ask for information about how local protagonists intend to tackle the different approach [7].
     • Try to become adept at recognizing and addressing hesitancy, as people or organizations may be reluctant to engage in unfamiliar procedures.
     • If something is known or found to be unusual in a given context, prepare demonstrations of its usefulness.
     • Aim to persuade those involved to try it out or help adapt it for local conditions before deciding on its use. In other words, avoid anyone having to take a stand on feasibility before trying things out.
   • Negotiate work scope and costs with the most promising organization.
     • If the specifications change significantly, then reopen the bidding process to all competitive organizations [3].
     • Agree upon alternative designs prior to signing the contract, since change is more difficult once a study has started [11].
   • Notify unsuccessful bidders of your selection once the contract has been awarded.
     • Supply them with your reasoning for selection, and provide feedback as to how they could be more successful in future bidding processes [3].
     • Keep records of all such notification.
   Lessons learned

   • When evaluating survey organizations, one of the most difficult decisions made is determining whether a survey organization is truly capable of implementing what has been promised in its bid. If two competing survey organizations propose similar technical bids, it is not always prudent to select the organization with the less expensive bid (even though not doing so might conflict with pre-determined bidding evaluation criteria). It is important to balance the proposed technical aspects (and timeline) and budget with the survey organization's (and staff's) experience and references. Prior work is often very foretelling of future work.
4. Address the rights and obligations of all parties in the contract, and reference local legal requirements, if applicable.
   Rationale

   The final contract is legally binding and thus must fall under the auspices of a recognized legal authority with the power to sanction contract breaches. The sanctions should be explicit, up to and including nullifying the contract. The contract needs to be properly signed and dated by authorized representatives. Local, independent legal advice is critical to this process.

   Procedural steps

   • Write the contract based upon the study design and specifications.
   • Define progress approval points throughout the research process (e.g., sample selection, questionnaire design, interviewer training, and data collection milestones) to ensure each party involved achieves the study's objectives.
     • Require certification from the coordinating center at these formal points before a survey organization can proceed with the study.
     • Sanctions for unnecessary delays or specification deviations should be specified before the study begins.
   • Set reasonable production benchmarks (e.g., on targeted response rates and cut-off requirements), where possible [3]:
     • For the purpose of response rate calculation, provide the survey organizations with a defined list of the disposition codes to be used uniformly (see Appendices A - E for a description of disposition codes and templates for calculating response rates).
     • Go through the list of disposition codes, checking applicability of each for the local situation and define the need for additional codes to account for local conditions.
     • Require field monitoring progress reports (possibly at the individual interviewer, interviewing team, or region level) to ensure benchmarks are met (see Data Collection).
   • Tailor contracts to the funding source, contracting organizations, and countries, as necessary. Each may carry additional requirements, such as stipulated delivery of reports, a cross-national ethics board review, and so forth.
   • Require official pre-approval of any subcontracting. Any known need for subcontracting in any form should be disclosed in advance by the survey organization(s) [3].
   • Establish and specify in writing ownership of the data and respondents' sample/contact information within the limits of any confidentiality restrictions.
   • Specify requirements for how the local survey organization will execute the data delivery and frequency of updates on data collection progress (see Data Collection, Harmonization of Survey and Statistical Data, and Dissemination of Survey and Statistical Data).
   • Specify any deliverables (such as sample specifications, instrument specifications, and/or source questionnaires) and commitments from other parties involved, including any central organization to local organizations (e.g., advisory boards and help lines).
   • Incorporate bonus schemes in the contract and cost estimates as appropriate. Examples may include:
     • Interviewer bonuses
     • Organizational bonuses, such as a payment for completing interviews beyond the expected total
   • Establish responsibility for documenting all aspects of the study.
   • Identify and specify all documents required:
     • Agree on format for this and on who develops the format.
     • Include provisions for training for those required to provide documentation.
   • Identify and specify the coordinating center's right to observe aspects of data collection (e.g., live interviews, call-backs to selected households for verification, spot checks of original questionnaires, and electronic control files) [9].
   • Define any restrictions on the survey organization's ability to present and/or publish any of the substantive or methodological results (with or without review).
   • Specify copyrights for data and documents, including stipulations for data release (by when and by whom) and plans for data access rights (taking into account any legal restrictions).
   • Define the necessary security level of respondent data (e.g., contact information and survey responses) for both physical and electronic storage and transfer.
   • For longitudinal studies, indicate, as appropriate, decisions about the protocol for possible respondent recontact. If potential for future follow-up exists, consider introducing this possibility at the time of initial contact with the respondents and ask the survey organization to budget for this activity:
     • Mention the possibility of recontact in the informed consent procedure.
     • Obtain consent for the transfer of respondent contact information, if applicable.
   • Instruct the survey organization to notify the coordinating center of any change to the contract [11].
   Lessons learned

   • Although it is important to enforce adherence to specifications, a measure of flexibility is also needed. Natural disasters, unexpected political events, and outbreaks of disease can interrupt data collection and make agreed-upon deadlines impossible (see Data Collection). In more predictable fashion, seasonal effects must also be taken into account: one country's harvest time may be another's winter months; access to areas may be restricted or facilitated by one and the same season. In certain times of year, large parts of the population may be on vacation or working away and difficult to reach at their usual residence [11].
5. Define upfront the quality standards that lay the quality framework for rest of the survey lifecycle.
   Rationale

   The bidding process may be the first interaction the survey organizations have with the coordinating center. Hence, it is essential for the coordinating center, from the conception of the survey, to demonstrate and emphasize the importance of quality.

   Procedural steps

   • In specifying the study requirements in the tender (and later contract), carefully weigh how cost and schedule may trade off with a quality survey.
   • Ask bidding survey organizations to detail their quality control and quality assurance procedures, and include minimum quality requirements in the criteria used for evaluating the bidders.
   • Consider re-releasing the tender if no bidding survey organization can meet the requested quality standards.
6. Document the steps taken while preparing a tender, soliciting bids, and drawing up and executing a contract.
   Rationale

   The coordinating center can use the contract resulting from the bidding process to enforce its expectations of the survey organizations. Thus, it is very important that steps taken throughout the process be clearly noted and transparent to those involved. No one involved should be surprised at how the study is to be structured, what production actions are required, and when the final deliverables are to be completed.

   Procedural steps

   • Clearly state specifications in the tender.
   • In advance of releasing the tender, document the evaluation criteria to be used when assessing bids.
   • Keep a record of the information exchanged with each survey organization to make sure no one organization receives differential treatment during the bidding process.
   • Collect notes from each member of the bid evaluation team as to how they arrived at their selection decision.
   • Make sure each survey organization formally details all aspects of their anticipated scope of work in their bid. Information beyond what is written in the bid (e.g., from other forms of correspondence) should not be considered when evaluating the survey organization — so as not to give differential treatment.
   • Write contracts that are tailored to the involved parties (e.g., funding source, coordinating center, survey organization, etc.). When writing the contract, include all specifications of the scope of work, budget, and timeline for which each survey organization should commit.
   • Request documentation of any subcontracts from the survey organizations.
   • Have a signed agreement regarding the ownership of the data and respondent information, within the limits of confidentiality regulations (see Ethical Considerations in Surveys).

Return to top

Previous chapter | Next chapter | Home

Last modified: 2008-Jun-20

© 2008 The authors of the Guidelines hold the copyright. Please contact us if you wish to\n publish any of this material in any form.

View Chapters

• Introduction
• Study Structure
• Tenders, Bids, & Contracts
• Ethical Considerations
• Sample Design
• Questionnaire Design
• Translation
•      Translation Assessment
•      Language Harmonization
•      Translators
•      Translation Budgeting
•      Translation Scheduling
•      Translation Tools
• Adaptation
• Survey Instrument Design
• Pretesting
• Interviewer Recruitment & Training
• Data Collection
• Harmonization of Data
• Data Processing
• Dissemination
• Assessing Quality
• Glossary
• Bibliography