PSI R ESEARCH & METRICS T OOLKIT B UILDING R ESEARCH C APACITY Scales Series: Chapter 8 Expert Review and Forward- Backward Translation PSI s Core Values Bottom Line Health Impact * Private Sector Speed and Efficiency * Decentralization, Innovation, and Entrepreneurship * Long-term Commitment to the People We Serve
Research & Metrics Population Services International 1120 Nineteenth Street, NW, Suite 600 Washington, D.C. 20036 PSI Research & Metrics 2010 Population Services International, 2008 Contact Information Kate O Connell 1, Linda Kaljee 2, and Hibist Astatke 3 1. Deputy Director, PSI Research & Metrics 2. Associate Professor, Wayne State University, Pediatric Prevention Research Center 3. Senior Researcher, PSI/Washington For more information, please contact: Hibist Astatke 1120 19 th St. NW. Washington, DC 20036 Telephone 202-572-4623 Email 202-785-0120
LEARNING OBJECTIVES By the end of this chapter, the reader will be able to: 1. Know how experts can be used to review multi-item scales 2. Know procedures to translate and back-translate multi-item scales BACKGROUND Expert reviews and pre-tests are two additional steps for refining scaled items prior to a pilot study. Each of these activities is needed for optimal scale development/adaptation. Two types of reviews are described in this chapter. The first and recommended expert review includes outside researchers, health administrators, and/or academicians in additional to regional and program staff The second type of review includes regional researchers or a Washington D.C. based researcher and staff and can be used in cases where time and resources are limited. 1) EXPERT REVIEWS OVERVIEW The purpose of an expert review is to assess the content validity of a scale in comparison to the definition of the determinant that it is intended to measure. An expert review can also result in additional items based on personal knowledge and experience. GUIDELINES FOR AN OUTSIDE EXPERT PANEL REVIEW What is an expert panel? An expert panel is a group of individuals from several different backgrounds convened to review the final language and structure of multi-item scales. At PSI scale expert panelists are typically invited to participate in a one day workshop to review scales. 1
Who is included on an expert panel? An expert panel will include the following individuals: 1) a content expert (e.g., HIV / malaria). This will be a technical administrator such as an in-country HIV advisor; 2) an expert on the target population. This can be a gatekeeper, e.g., health staff at an HIV clinic, or a member of the target population; 3) social science survey expert. This can be a regional researcher, or a faculty member from a local university. In addition, two members of the in-country research staff familiar with the survey instrument and multi-item scale development will act as moderator and note-taker during the meeting. What is the primary purpose of the expert panel? The primary purpose of the expert panel is to review multi-item scales to assess and finalize the determinant definitions, items, and response options. In addition, the expert panel should review scale instructions. How to run an expert review panel. 1. The moderator will introduce the purpose of the panel. 2. The moderator will present the determinants of that specific study within the context of the PSI Behavioral Change Framework. 3. For each determinant, the moderator will present the definition and the scale associated with that determinant. 4. For each determinant and associated scale, the expert panel will review the determinant definition, discuss the appropriateness of the items and make suggested changes to the items and/or response options. Suggested changes can include the addition of new items. A majority of the panel members will need to agree on any suggested changes. 5. The expert panel will also review the instructions for the scale to ensure clarity and language appropriateness for the target population. 6. Steps 4 and 5 will be followed until all of the determinants and scales have been reviewed. 7. Throughout the process, a note-taker will document the suggested changes. Ideally this will be accomplished on an electronic copy of the scales using track changes (Microsoft Word). If a projector is available, the scales can be viewed on a screen as these changes are incorporated. What is the product of the expert panel? The final product immediately after the expert panel has convened is a document which includes all agreed upon changes to all multi-item scales included within a single questionnaire. The research team will then incorporate these changes into the master copy of the questionnaire prior to the pre-test (see Chapter 9) 2
GUIDELINES FOR REGIONAL RESEARCHER REVIEWS Scale reviews conducted by regional researchers have the same purpose and follow a similar process as reviews conducted by expert panels. How to conduct researcher reviews. 1. For each determinant and associated scale, the regional researchers are provided with the determinant definitions, corresponding items, response options and scale instructions. 2. Regional researchers assess the appropriateness of the items and make suggested changes to the items and/or response options. Suggested changes can include the addition of new items and changes to instruction to ensure clarity and language appropriateness for the target population. 3. In country researchers incorporate changes that they agree with and discuss any changes that they disagree with a regional researcher until agreement is reached. 2) FORWARD AND BACKWARDS TRANSLATION OVERVIEW Once items have been generated and reviewed, it is important to ensure that the meaning and intent of the items is maintained across translation. The best process is forward and backwards blind translation. This process will allow for individuals to conduct translations independently between English (or other languages such French, Spanish or Portuguese) and the local language in order to correct errors. The local translation is used to prepare a draft of the questionnaire that can be used to conduct the pre-test (see Chapter 9 on Pre-testing Scales). For example, if items to be used in a multi-scale questionnaire in Viet Nam were developed in English, they will be translated into Vietnamese by one individual and then back translated from Vietnamese into English by another translator. This dual step process decreases risks of errors and improves the accuracy of the translations. How to conduct forwards and backwards translation? The following steps should be followed for the translation procedure. 1. An individual who is bilingual in the original language of the questionnaire and the local language translates the questionnaire into the local language(s); 3
2. A second bilingual translator who is not familiar with the questionnaire items (blinded) will then back translates the questionnaire into the original language. The blinding of this process decreases risk of bias in the translation; 3. The back-translation is reviewed by the researcher. The researcher will work with the translator to eliminate any differences in meaning between the original version and the back-translated version. 4. Any errors or translations that do not reflect the original intent are re-translated into the local language(s). 5. The revised tool is then used for the pre-test, and any remaining translation related problems identified during the pre-testing will be resolved using steps 3 and 4. Rules for Translations 1. Translators must be fluent (written, verbal, and comprehension) in both the original language and English; 2. Translators need to be familiar with any dialects or specific lexicons for the study population. This might include knowledge of slang or specific terms for activities, e.g., drug use, that might be included in the scales; 3. Avoid literal translations. All translations are done so as to keep the intent of the original statement, while writing correct grammar in the local language (translations) and then correct grammar in English, e.g., adjective before nouns (back-translations); 4. Pay attention to translation of negatively worded items; 5. If more than one translator is working on the texts, consistency must be maintained between translators in terms of the English word or phrase chosen to best represent the intent of the original language; 6. Translate the questionnaire into the local language and backtranslate into English. Check for consistency in language and intent; 7. Use a non-researcher (an individual blinded or not familiar with the questionnaire) to do the back translation. 4