Reliability and Validity of a Questionnaire to assess Oral Health Literacy among College Students in Bangalore City.

Transcription

1 Reliability and Validity of a Questionnaire to assess Oral Health Literacy among College Students in Bangalore City. M. Aruna Devi, 1 Sugandhi Soni, 2 Radha, 3 Sushi Kadanakuppe, 4 Nagashree, 5 Pallavi 6 ABSTRACT Objective: The objectives of the present study are to develop a new questionnaire and assess its test-retest reliability, internal reliability, convergent validity and predictive validity. Methods: various patient education material were collected, the items of the questionnaire were framed based on the information provided through these materials. The questionnaire was pilot tested among 75 college students and later was administered to 500 college students. The test retest reliability was assessed by administering the questionnaire twice to 20 participants, internal consistency reliability by Cronbach s alpha, convergent validity by Pearson s correlation with REALD-30, predictive validity by Pearson s correlation with OHIP-14. Results: 28% were females, mean age of sample was (S.D.= 1.27). Mean score for the oral health literacy questionnaire was 6.7 (S.D.= 1.27), test-retest reliability was 0.69, internal consistency reliability was 0.408, correlation with REALD-30 was 0.32, with OHIP-14 was Conclusion: The reliability and validity of the questionnaire could not be established in the present study. Key words: health literacy, oral health literacy, oral health literacy questionnaire, reliability, validity. Introduction There are many reasons why preventable diseases remain so common and why people often do not adopt practices that have been scientifically shown to be effective in maintaining health. These range from financial issues, lack of access to providers and adequate preventive care to a spectrum of biological, behavioral, community and cultural factors. Low health literacy is also one among these factors. Since the modern health care system makes complex demands on the health consumers, they are faced with many challenges to seek health care and health care information and those with low health literacy are left unable to reap the benefits of available information. An example of this is mentioned by Bohlman et al - A two-year-old was diagnosed with an inner ear infection and was prescribed an antibiotic. Her mother understood that her daughter should take the prescribed medication twice a day. After carefully studying the label on the bottle and deciding that it didn t tell how to take the medicine, she filled a teaspoon and poured the antibiotic into her daughter s painful ear. 1 The example reflects how patients with low health literacy are vulnerable to misunderstandings and misinterpretations which may result in devastating results at times. Health literacy is increasingly described as the currency for improving the quality of health and health care. It s the basic reading and numerical skills that allow a person to function in the health care environment. Healthy People 2010 defined health literacy as The degree to which individuals have the capacity to obtain, process, and understand basic health information and services needed to make appropriate health decisions. 2 The American Medical Association (AMA) defines health literacy as A constellation of skills, including the ability to perform basic reading and numerical tasks required to function in the health care environment 3. World Health Organization describes it as Health Literacy represents the cognitive and social skills which determine the motivation and ability of individuals and communities to gain access to, understand, and use information in ways which promote and maintain good health. 4 The importance of oral health literacy and its role in oral health promotion was highlighted in the Surgeon General s report on oral health. In January 2004, National Institute of Dental and Craniofacial Research (NIDCR) convened a working group on Functional Health Literacy, the group adapted the health literacy definition given by Healthy People 2010 to the context of oral health as "The degree to which individuals have the capacity to obtain, process, and understand basic oral health information and services needed to make appropriate health decisions" 5 It is important to detect patients with inadequate oral health literacy and to improve the level of communication between the provider and the patient. This is important in clinical setting as well as community level. At the community level it is important in order to appropriately design educational materials and community intervention programs that coincide with the literacy level of the target population. Concerns about low levels of health literacy led 43

2 investigators to search for methods and instruments to screen a patient s health literacy skills. Traditionally, researchers and clinicians used the patient s level of education as an indicator of health literacy. Although education is highly correlated with reading levels, educational level alone cannot predict functional health literacy. This informal method has been augmented by other, more formal methods of screening for Reading and comprehension. The formal assessment of literacy skills have been divided into two categories- word recognition tests and comprehension tests 6 In word recognition tests, the patient read aloud from a list of common medical words. These tests do not measure reading comprehension or interpretation per se, but only the ability to recognize, or read and pronounce individual words. In comprehension tests, patient s ability to understand written text of varying difficulty is assessed. In context of health literacy, two literacy assessments are widely used. 3 - the Rapid Estimate of Adult Literacy in Medicine (REALM), a word recognition test and the Test of Functional Health Literacy in Adults (TOFHLA), a comprehension test. Regarding oral health literacy, the instruments published are- The Rapid Estimate of Adult Literacy in Dentistry-30 (REALD-30) 7, The Rapid Estimate of Adult Literacy in Dentistry-99 (REALD-99) 8, Test of Functional Health Literacy in Dentistry (TOFHLiD) 9, Oral Health Literacy Instrument (OHLI) 6. The first two are word recognition test modeled on REALM. REALD-30 consists of 30 dental words taken from the American Dental Association Glossary of Common Dental Terminology and patient education materials 7. REALD-99 was developed by the same authors by adding 69 more words to the previous one and made the list longer, from 30 to The words are arranged in the order of difficulty, based on the average word length, number of syllables, and difficult sound combinations. The lists of the words are designed to be read aloud by the participant to the interviewer. In scoring one point is assigned for each word pronounced correctly and summed to get the overall score 7,8. The test is rapid, easy and was found to be reliable and valid. But the words presented are in singular fashion and cannot determine comprehension. It is not assured if the person knows the meaning of the particular word or rather is simply able to pronounce it without having the knowledge of the word. The test does not assess for functional literacy, defined as ability to interpret instructions or complete forms. TOFHLiD and OHLI are modeled on TOFHLA. Both instruments consist of reading comprehension section and numeracy section. TOFHLiD contains passages regarding set of instructions about fluoride varnish, consent form for dental treatment, Medicaid rights and responsibilities, instructions for toothpaste, pediatric dental appointment and prescription labels for fluoride drops and fluoride drops 9. OHLI contains passages on dental caries, periodontal diseases, prescriptions associated with dental treatment, post-extraction instructions and dental appointments. 7 It is not feasible to use TOFHLiD in our health care setting, as the contents of the instrument are not fully applicable to our health care settings, like Medicaid rights. Considering the importance of assessing oral health literacy, there is a need for an instrument to assess it. But the present available word recognition instruments lack the ability to assess functional health literacy and the available functional health oral health literacy assessment instruments are not fully applicable in our population, hence this study is taken. The objectives of the present study are to develop a new questionnaire and assess its test-retest reliability, internal reliability, convergent validity and predictive validity. Materials and Method Instrument development The questions to assess oral health literacy were framed on the various aspects like morphology of teeth, dentition, plaque and its effect, dental decay progression, gum diseases, wisdom teeth, toothpaste use instructions, instructions in a consent form and medical prescription. Various patient education materials like pamphlets, brochures, on-line education material, labels and instructions for use of different oral health related products; prescription for medications and consent form were collected. The information and instructions provided through these materials was condensed in a booklet, which was given to each participant to read through before administration of the questionnaire. The oral health literacy questionnaire was self-administered and closed ended. Initially there were 18 questions and for each question there were four options provided out of which one was correct. Other variables Socio-demographic information about the participants was obtained. Information regarding dental visit was recorded. REALD-30, the word recognition test to assess oral health literacy was used to assess the convergent validity of the oral health literacy questionnaire. The participants were asked to read aloud the list of the thirty words in REALD-30 by the investigator and scored based on the pronunciation. The participants were instructed to read those words for which they believed that they know the correct pronunciation. For each correct pronunciation one score was awarded, and for wrong zero. The scores may range from 0 to 30. OHIP- 14 questionnaire 10, which assesses the Oral Health Related Quality of Life (OHRQoL), was also completed by participants to assess the predictive validity. It measures the individual s perception of the social impact of oral disorders on their well being and consists of fourteen items organized in seven subscales that address aspects of oral health that may compromise a 44

3 person s physical, psychological and social well being. OHIP-14 questionnaire asked about the problems they might have encountered with their teeth, mouth during the previous year. The answers were marked on a likert scale- Never, hardly ever, occasionally, fairly often and very often. The scores may range from 0 to 14 when total number of problems reported i.e. occasionally, often or very often are used. A higher score indicates worse OHRQoL. Higher oral health literacy of an individual should be associated with better OHRQoL and hence lower score on OHIP-14. Sample was selected from two government degree colleges in Bangalore. A convenience sample of 500 students was selected with an inclusion criteria that they should had studied English as a language till 12 th standard, without any visual acuity. After brief information about the study, those who agreed to participate in the study were included. An informed consent was obtained from the participants and they were informed regarding maintenance of the confidentiality of their records. For the pilot testing of the questionnaire 75 participants were included in the study. The booklet was given to each participant to read for minutes, after which it was taken back. Thereafter the questionnaire was given to be filled. The reason for not giving any gap between the administration of booklet and the questionnaire was that the participants should not have access to any external source of information which might influence their understanding of the material provided in the booklet. Based on the results of internal consistency, three of the questions were deleted and modifications were done in the construct of other questions. The final oral health literacy questionnaire had 15 questions. REALD-30 and OHIP-14 were also administered to them. For the final study, the three questionnaires- oral Health literacy questionnaire, REALD-30 score and OHIP-14 were administered to 500 participants. Their scores were obtained. To assess the test-retest reliability 20 participants were administered the questionnaire twice. Data Analysis For the statistical analysis of the data Statistical Package for the Social Sciences (SPSS), version 13 software package was used. Pearson product moment correlation test was used. To assess the internal consistency reliability Cronbach s alpha was used. To assess the test-retest reliability of the questionnaire Pearson s correlation was done for the score at the first attempt and the second attempt. p Values 0.05 were accepted as statistically significant Results Out of the 500 participants, 28% (n=140) were females. The mean age of the participants was years. 24.2% (n=121) of the participants reported to visit the dentist every 3-6 months. 18.6% (n=93) visited every year, 5.4% (n=27) visited every 2-3 years. Maximum of 51.8% (n=259) visited the dentist when there was pain only. The mean score for the oral health literacy questionnaire was 6.7 with a standard deviation of 2.32 and minimum and maximum of 0 and 13 respectively. The mean score for REALD-30 was 16.3 and standard deviation of and minimum and maximum of 2 and 29 respectively. The mean OHIP-14 score was 1.23 and standard deviation of 1.89 and minimum and maximum of 0 and 8 respectively. The test-retest reliability score obtained by finding the correlation between the scores obtained at the first attempt and the second attempt was 0.69 (p<0.05). The internal consistency reliability of the questionnaire was Correlation between the two was 0.32 (p<0.05). The correlation was (p>0.05) Parameters Oral health literacy questionnaire and REALD- 30 Oral health literacy questionnaire and OHIP-14 Pearson s correlation (p) p- value 0.32* Table 1. Correlation between oral health literacy questionnaire and REALD-30 and OHIP-14 score * Statistically significant at p 0.05 Discussion This study was undertaken to develop a questionnaire to assess oral health literacy and assess its test retest reliability, internal consistency reliability and convergent validity and predictive validity. The testretest reliability of the questionnaire showed strong positive correlation and was statistically significant (P= 0.69), and was almost similar to the test-retest reliability of Oral Health Literacy Instrument (OHLI) which was Cronbach s alpha values of 0.5 to 0.7 are generally considered to indicate sufficient reliability for an instrument or scale to be used to make group comparisons, and that above 0.85 are reliable enough for individual patient comparison. The internal consistency reliability of the questionnaire was unacceptable with Cronbach s alpha of The internal consistency reliability of TOFHLiD was , OHLI was , REALD-30 was and REALD-99 was To assess the convergent validity, the scores obtained for oral health literacy questionnaire were correlated with another oral health literacy assessment instrument REALD-30. The correlation with REALD-30 was weak positive (p = 0.32) but was statistically significant. The 45

4 reason for low correlation between the two might be because, REALD-30 is a word recognition test whereas Oral health literacy questionnaire is a comprehension test, the two instruments have a different construct, this might may be a reason for low correlation between the two instruments. The convergent validity of TOFHLiD was recorded by finding the correlation with REALD-30 and was That for OHLI was where the correlation was found with TOFHLA, a comprehension test for medical health literacy. The convergent validity of REALD-30 was assessed by finding the correlation with REALM and TOFHLA, which are the word recognition instrument to assess medical health literacy and comprehension test to assess medical health literacy and was 0.86 and 0.64 respectively. To assess the predictive validity of the questionnaire, the scores obtained for the oral health literacy questionnaire were correlated with OHIP-14 scores. The correlation with OHIP-14 was negative, supporting the idea that oral health literacy and OHIP- 14 are inversely related, which suggests that higher the oral health literacy, better the OHRQoL. But the correlation was weak (p= ) and not statistically significant. A reason for low correlation between the two instruments might be that since the study sample was young age group, it is possible that their OHRQoL is not being affected by oral health literacy. The predictive validity of TOFHLiD, REALD-30 and REALD-99 were , , respectively. Limitations of the study Since the data was collected from a convenience sample, it gives rise to sampling bias in the study. The reason for adopting this sampling was that the study required three questionnaires to be filled for which cooperation was required from the participants. Another drawback is that oral health literacy is culture sensitive, and an important component of culture is language. The present study was conducted in English language, which is most of the times not the primary language spoken at the homes of our population. The study lacked that linguistic competence. But since most of the health information provided in our health care setting is in English, the questionnaire was framed in this language, also the sample selected fulfilled the inclusion criteria of heaving studied English till twelfth standard, language was not a barrier for this study sample. Future work should draw sample with probability sampling and representative of a more diverse background characteristics such as age, education, occupation, income, health status. Also to overcome the language barrier, the questions can be translated in the local language and tested again. The predictive validity of this instrument was assessed by using only subjective criteria, objective criteria like clinical indices may be included. To conclude, the reliability and validity of the questionnaire could not be established in the present study. References 1. Bohlman L.N., Panzer A. L., Kindig D.A. Health literacy: a prescription to end confusion. Institute of Medicine, Accessed online , at: 2. US Department of Health and Human Services. Health Communication. In: Healthy People nd ed. Washington, DC: US Government Printing Office; Safeer R.S., keenan J. Health Literacy: The Gap Between Physicians and Patients. American Family Physician. 2005; 72(3): World Health Organization. Health promotion glossary. WHO, Geneva A report of a workgroup sponsored by the National Institute of Dental and Craniofacial Research, National Institute of Health, U.S. Public Health Services, Department of health and health services. The invisible barrier: literacy and its relationship with oral health. Journal of public health dentistry. 2005;65(3): Sabbahi D.A., Lawrence H.P. Development and evaluation of an oral health literacy instrument for adults. Comm. Dent Oral Epidemiol.2009;37: Lee J.Y., Rozier R.G., Lee S.Y.D., Bender D, Ruiz RE. Development of a Word Recognition Instrument to Test Health Literacy in Dentistry: The REALD-30 A Brief Communication. Journal of public health dentistry. 2007;67(2): Richman JA, Lee JY, Rozier RG, Gong DA, Pahel BT, Vann WF. Evaluation of a word recognition instrument to test health literacy in Dentistry: The REALD-99. Journal of public health dentistry. 2007; 67(2): Gong D.A., lee J.Y., Rozier R.G., Pahel B.T., Richman J.A., Vann W.F. Development and testing of Functional Health Literacy in Dentistry (TOFHLiD). Journal of public health dentistry. 2007;.67(2): Acharya S. Oral health related quality of life and its associated factors in an adult Indian Population. Oral Health Prev Dent 2008;6:

5 About the Authors 1. Dr. M. Aruna Devi Professor and Head 2.Dr. Sugandhi Soni Postgraduate student 3.Dr. Radha Reader 4.Dr. Sushi Kadanakuppe 5.Dr. Nagashree 6.Dr. Pallavi Corresponding Author: Dr. Sugandhi Soni Postgradute student V.S. Dental College and Hospital, K.R. Road, V.V. Puram, Bangalore id: Phone number: