Title: The impact of disseminating the whole-community project '10,000 Steps': a RE-AIM analysis

Transcription

1 Author's response to reviews Title: The impact of disseminating the whole-community project '10,000 Steps': a RE-AIM analysis Authors: Ragnar Van Acker (ragnar.vanacker@ugent.be) Ilse De Bourdeaudhuij (ilse.debourdeaudhuij@ugent.be) Katrien De Cocker (katrien.decocker@ugent.be) Lisa M Klesges (lmklsges@memphis.edu) Greet Cardon (greet.cardon@ugent.be) Version: 2 Date: 23 November 2010 Author's response to reviews: see over

2 Ghent University Department of Movement and Sports Sciences Watersportlaan Gent Belgium Ragnar.VanAcker@UGent.be Dr Annette E Maxwell Associate Editor, BMC Public Health Ghent, November 23 rd 2010 REVISION Manuscript MS: Paper title: The impact of disseminating the whole-community project 10,000 Steps : a RE-AIM analysis Dear Associate Editor, The authors would like to thank you and the reviewer for the interest in our work and the decision to reconsider acceptance for publication of the manuscript after revision. We carefully considered all comments and suggestions made by the reviewer. Please find below the complete list of changes, and our responses to each of the points made by the editor and reviewer. Sincerely yours, Ragnar Van Acker, corresponding author: Department of Movement and Sports Sciences Ghent University Watersportlaan Gent Belgium Phone: Fax: Ragnar.VanAcker@UGent.be 1

3 Comments editor 1. Document, within your manuscript, the name of the ethics committee which approved your study. The name of the ethics committee has been added to the Methods section of the revised paper: Page 9: Methods section: The study protocols were approved by the Ethical Committee of the Ghent University. 2. Provide copies of the questionnaires administered in your study as additional files to your submission. Both questionnaires that were administered in our study are provided as additional files; one for citizens (Additional file 1) and one for professional organizations (Additional file 2). As recommended by the Authors checklist these Additional files are mentioned in the text of the revised manuscript, and referred to in the following way: Page 9: Measures section: Questionnaires (see Additional file 1) assessed individual project awareness and included the following questions... The online organizational survey (see Additional file 2) was structured in four main parts assessing: Ensure that your revised manuscript conforms to the journal style ( ). It is important that your files are correctly formatted. We thank the editor for the link to the Authors' checklist and have formatted our files accordingly. Additional changes have been made to meet the journal style some examples: - Table 3 is now displayed in portrait format. - All references are cited in the text using consecutive numbers in square brackets. All brackets are highlighted in the text. - References in the revised paper contain all named authors, regardless of the number. The term 'et al.' is no longer used. Comments Reviewer: Major compulsory revisions 1. How different was the initially developed and tested project in Australia from the one pilot tested in Ghent? Were they identical with some adaptation in Ghent if so, what components were changes? (Authors used the term developed. ) On this same note, did the wide-scale dissemination result in any change to the pilot? These questions can get to the important notion of adaptation/fidelity. 2a. While authors provide references to the original Australian study and also to the pilot study in Ghent, a more detailed description of the actual intervention in this paper would be beneficial for the reader. Perhaps a table could be used to describe the different components and could even address differences in intervention elements across the three phases of testing (as described in my previous point) 2

4 As suggested by the reviewer in comments 1 and 2a, we have included a table (see Table 1) which describes the intervention components of the three different 10,000 Steps studies. These intervention components are ordered according to a socio-ecological model of health promotion (McLeroy, 1988), which facilitates the readers understanding of differences across the three phases of testing. In addition to Table 1, we have included a Development subsection in the methods section of the revised paper, which addresses the main adaptations across the three phases of testing: Page 5-6: Methods section: Development This study tests a third phase of 10,000 Steps' adapted from the original 10,000 Steps Rockhampton study from Australia [11,12] and the subsequent pilot study 10,000 Steps Ghent [13]. Based on these two local intervention studies, intervention guidelines for 10,000 Steps were formulated and disseminated in the present study in Flanders. These guidelines were addressed to potential adopters of 10,000 Steps and several intervention components were recommended. Table 1 provides an overview of the intervention components that were recommended to potentially adopting organizations in the present study in Flanders compared to the implemented intervention components of 10,000 Steps Rockhampton and 10,000 Steps Ghent Insert Table The intervention components across all three phases of testing represent a socio-ecological [18] approach to PA promotion in adults, with some small adaptations throughout the phases. Main adaptations included less focus on delivery via general practitioners and dog walking, and a stronger focus on promoting public places in both 10,000 Steps Ghent and the present study in Flanders. To emphasize the importance of the interpersonal dimension of PA promotion, wide ranging personal contact with citizens (e.g. personalized letter, mail, or phone) was included as an additional intervention guideline in the present study in Flanders. Furthermore, in 10,000 Steps Ghent and the present study in Flanders the guideline of 30 minutes moderate to vigorous PA on most, preferably all days, of the week [19] was added to the main project theme of 10,000 Steps a Day, Every Step Counts. A final adaptation in the present study in Flanders was the more general guideline of promoting PA in the entire population and all domains of active living (PA for transport, at work, for household and leisure time), while 10,000 Steps Ghent and 10,000 Steps Rockhampton focused on specific projects for primary care, workplaces, and older people (Table 1). This more general recommendation was believed to be less rigid and would facilitate more widespread dissemination and implementation. Table 1: see next page 3

5 Table 1 - Socio-ecological intervention components and dissemination strategies of 10,000 Steps studies Intrapersonal Interpersonal Organizational 10,000 Steps Rockhampton (2 year project) Sale (general practitioner (GP), health services) and loan (libraries and video shops) of pedometers Website of 10,000 Steps Rockhampton Promotion of PA by health professionals and print media 10,000 Steps Ghent (1 year pilot) Sale (local town shop, health services) and loan (sport service) of pedometers Website of 10,000 Steps Ghent Promotion of PA and distribution of folders through GP s, dieticians, physical therapists and schools; posters in public places 10,000 Steps in Flanders (present study)* Sale and loan of pedometers in every municipality (local public services) Website updated from 10,000 Steps Ghent Promotion of PA and distribution of folders and posters in public places Not specified Not specified Personalized contact with citizens (e.g. personalized letter, mail, or phone) Community events, specific projects for GPs, health services involvement and for workplaces Community events, specific projects for workplaces and for groups of older people Community events, projects for the entire population and all domains of active living (PA for transport, at work, for household and leisure time) Community Local mass media campaign Local media campaign Local mass media campaign in every municipality Policy Strategies for dissemination among potential adopter(s) 10,000 Steps a day Every Step Counts Environmental: street signs, distribution of maps, promotion of dog walking Partnerships between local government and key members of community organizations, some with high-level experience in PA promotion Sale and loan of pedometers Local: Recruitment of community partners by researchers (micro grants) to form a local PA task force, GP training 10,000 Steps a day Every Step Counts, 30 minutes MVPA guideline Environmental: street signs, walking circuits and billboards Partnerships between local city and provincial governments, health insurance companies, and the local health promotion service Sale and loan of pedometers Local: Recruitment of community partners by researchers to form a local steering committee 10,000 Steps a day Every Step Counts, 30 MVPA minutes guideline Environmental: street signs, billboards Partnerships between the adopting organization and minimum one (other) local government service or two professional organizations Sale and loan of pedometers Regional: website, mailing of the project manual and pilot study results, group meetings, displays at conferences, e- articles * The intervention components described for the present study in Flanders were recommended to potentially adopting organizations (guidelines), and therefore do not represent actual implementation. 4

6 2b. (continuation of reviewer s second comment) They can also help to clarify which components were part of the intervention and which are part of the D&I strategy. In order to clarify the distinction between the intervention components and the dissemination strategies of the present study in Flanders we included a separate description of the dissemination strategies in the bottom row of Table 1 (see previous page). Moreover, this description is not only provided for the present study in Flanders, but also for the two previous studies, 10,000 Steps Rockhampton (Australian study) and 10,000 Steps Ghent (pilot study for Europe). The revised paper also briefly addresses the main difference in dissemination strategies between the present study in Flanders and the two previous studies: Page 6: section Dissemination efforts for the present study in Flanders : Considering the limited funding for dissemination (one researcher part-time allocated to the project), the local engagement of potential adopters and the formation of local steering committees could not be established by researchers themselves as was the case in the local projects of 10,000 Steps Rockhampton and 10,000 Steps Ghent (Table 1). 3a. A more detailed description of dissemination strategies would be great under the dissemination efforts section. How did the study team disseminate/promote their project website?; As requested by the reviewer, additional information on the promotion of the website is now provided in the dissemination efforts section: - Page 7: section Dissemination efforts for the present study in Flanders : Two s containing the publication on the effectiveness of 10,000 Steps Ghent [12], the project implementation manual and links to the website s content were sent to municipal sports services, local health promotion services, and health insurance organizations - Page 7-8: section Dissemination efforts for the present study in Flanders : A total number of 26 group meetings provided a social platform for presentations with illustrative project materials that accentuated the positive scientific findings of 10,000 Steps Ghent, the compatibility of the current program with the organizations mission and local contexts, and its easy access through the website. The website and its online support materials for implementing 10,000 Steps were demonstrated. Present organizations were advised to take advantage of this project support (project implementation manual, downloads of intervention materials, contacts) - Page 8: section Dissemination efforts for the present study in Flanders : Finally, displays at regional conferences and articles for professional e-magazines also referring to the website were periodically provided between 2007 and the start of data collection. 3b. What did the project website contain?; The revised paper elaborates on website content for both potential adopters and citizens: Page 6-7: section Dissemination efforts for the present study in Flanders : A permanent media strategy consisted of a website (updated from 10,000 Steps Ghent ) that provided content for both potential adopters and interested citizens. Main web content for potential adopters in 5

7 Flanders included updated project implementation manuals for the implementation of 10,000 Steps in communities and workplaces, downloads of intervention materials (e.g. logo, flyers, posters, billboards), project and community contacts, and web space for announcing project activities and events. Main web content for citizens included information about the 10,000 Steps philosophy, PA and health, tips on how to increase daily activity, contacts, an online diary for recording personal step counts, and reports on planned or completed activities. 3c. How were the municipal sports services, local health promotion services, and health insurance organizations identified?; The answer to question 3c has been added to the revised paper: Page 7: section Dissemination efforts for the present study in Flanders : These organizations were identified by contacting their respective national or provincial agency and by consulting interested opinion leaders within the agency 3d. How did the research team analyze the peer networks (formal social network analysis?)?; The answer to question 3d has been added to the revised paper: Page 7: section Dissemination efforts for the present study in Flanders : For the interpersonal component, these opinion leaders subsequently helped analyze peer networks within the organizations by mapping dates, meeting locations and sponsoring organizations of upcoming supra-local group meetings. These group meetings were formal platforms where representatives of the organizations and their peers from other municipalities/provinces regularly joined to discuss professional matters of common interest. 3e. What was asked from these organizations? As mentioned before (3a), group meetings provided a social platform for presentations with illustrative project materials and discussions with the present organizations. The following was advised: Page 7-8: section Dissemination efforts for the present study in Flanders : Present organizations were advised to take advantage of this project support (project implementation manual, downloads of intervention materials, contacts), to form a local steering committee independently, and to implement 10,000 Steps according to the guidelines and intervention components described in the project implementation manual. 4. A more detailed discussion on how the limitations might influence the findings/conclusions from the study should be added. Especially the second limitation on social desirability (p. 20). This is a pertinent remark. We have extended the discussion on the study s limitations and have explained the limitation on social desirability more profoundly. Page 21-22: Discussion section: Limitations of the current results are noted. Generalizability of results may be limited because no more than half of the contacted citizens agreed to participate and no information was available about the characteristics of the non-participants. This could have led to a positive bias of the results on citizens 6

8 project awareness and PA levels. Sole use of self-reported data was another limitation. Due to social desirability this may also have led to a positive bias of the results on project implementation (organizational level) and PA levels (individual level). On the organizational level, self-reported data on project implementation could become more valid by adding objective measures [9], such as on-site observations by a team of researchers (which require more resources than available in this study). On the individual level, complementary objective measures of PA levels could include the use of pedometers or accelerometers. Nevertheless, the questionnaire to collect data on PA levels in this study (telephone-administered IPAQ) has proven acceptable validity against accelerometers in several countries [20]. Finally, the study s outcomes on maintenance are provisional because these are based on adopters intentions and not on actual program continuation. Follow-up research is already scheduled to study program continuation and implementation one year after the present study. Minor essential revisions 1a. There should be a better distinction between sections on Data collection and sample and Measures. E.g., Description of the IPAQ and the project awareness should be listed under the Measures section rather than under the Data collection and sample question. A sub-section then could discuss the RE-AIM dimensions. We thank the reviewer for this useful suggestion and have restructured the sections on Data collection and sample and Measures accordingly. Descriptions on both the questionnaires for citizens (IPAQ and questionnaire on respondents project awareness) and organizations (measuring project awareness, adoption, implementation and continuation) are now listed under the Measures section (page 9 highlighted text). Moreover, these descriptions are now followed by a sub-section entitled RE-AIM Evaluation (page 9 highlighted title text). 1b. It could also be interesting to see how the specific questions asked relate to this measures (which questions/data items were used to calculate Reach, Implementation, etc.). As advised by the reviewer, the specific questions related to the measures for these separate dimensions have been included in their respective paragraphs: - Page 10: Measures section: Reach - Reach was defined as the proportion of eligible citizens that reported being aware of 10,000 Steps (Have you heard of 10,000 Steps?). - Page 10: Measures section: Effectiveness - effectiveness was also measured on the organizational level by comparing the proportion of the studied sample of organizations that was aware of 10,000 Steps (Is your organization familiar with 10,000 Steps?). - Page 10: Measures section: 7

9 Adoption - Adoption was evaluated as the proportion and representativeness of organizations that had delivered a whole-community project based on 10,000 Steps (Did your organization adopt 10,000 Steps up till this day?). - Page 11-12: Measures section: Implementation - These components are: 1) the sale or loan of pedometers in public places (Did your organization sell or loan pedometers during the implementation of 10,000 Steps ), 2) use of the 10,000 Steps website (Did your organization use the supportive website for 10,000 Steps, 3) repeated dissemination of information using variants of flyers (Did your organization disseminate flyers of 10,000 Steps?) and 4) posters in public places (Did your organization disseminate posters of 10,000 Steps?), 5) wide-ranging personal contact with citizens (Did your organization contact citizens in a personalized manner (e.g. personalized letter, mail, or phone)?), 6) the organization of community events (Did your organization stage any community events to promote 10,000 Steps?), 7) repeated use of the media (Did your organization conduct a media campaign to promote 10,000 Steps?), 8) the repeated or permanent use of street signs or other strategically placed objects in the street scene (e.g. bill boards) to encourage PA (Did your organization put street signs, billboards, or other promotional materials of 10,000 Steps in the street scene?) and 9) the initiation of partnerships with local authorities and other associations (Did your organization have any partnerships with municipal services, associations or societies to implement 10,000 Steps?). - Page 12: Measures section: Maintenance The proportion of organizations intending to continue the delivery of 10,000 Steps (Does your organization have the intention to continue 10,000 Steps in the future?) evaluated the potential sustainability of the intervention. 2. Similarly to my previous note some information on the participating organizations should actually be described in the Results section (p.8 currently). Also, some characteristics of the individual respondents should also be summarized at the beginning of the Results section. Following the reviewer s recommendation, information on both participating organizations and individual respondents is now situated and bundled in the first paragraph of the Results section: Page 14: Results section: Those responding to the individual survey had a similar proportion of employed people [60.0% vs. 66.3%, respectively; χ²(1) = 0.87, p = 0.35], a higher mean age [Ms = 50.6 years vs years, respectively, t(754) = 4.69, p < 0.01], a lower proportion of men [40.9% vs. 49.0%, respectively; χ²(1) = 19.78, p < 0.01], and a higher proportion of persons with a higher education degree [39.7% vs. 25.0%, respectively; χ²(1) = 85.18, p < 0.01] compared to the general population of Flanders. Responding organizations included 25 municipal sports services (of 42 contacted and of 292 total), 18 local health promotion services (of 18 contacted and of 26 total), and 26 health insurance organizations (of 26 contacted and of 28 total). 8

10 3. This manuscript would also greatly benefit from some precision in language which could be easily addressed by a thorough revision of the text. To meet the reviewer s remark the paper was revised by a team of authors and a native American speaker. It was also attempted to shorten up the text in various sections. Some examples are provided for the reviewer: - Page 5: Development section: -> Revised sentence: This study tests a third phase of 10,000 Steps' adapted from the original 10,000 Steps Rockhampton study from Australia [11,12] and the subsequent pilot study 10,000 Steps Ghent [13]. -> Original sentence: This study represents a third phase of testing in the development of 10,000 Steps'. As indicated in the background section, the present study in Flanders stemmed from the original 10,000 Steps Rockhampton study in Australia [11,12] and the subsequent pilot study in a Flemish city, 10,000 Steps Ghent [13]. - Page 10: Measures section: -> Revised sentence: Measures of the RE-AIM dimensions were used to calculate summary metrics and are illustrated in Figure 1. -> Original sentence: This subsection reports on the specific measures for the RE-AIM dimensions. A flow chart illustrating these measures and the interconnected nature of the data is provided in Figure 1. - Page 14: Results section: -> Revised sentence: Those responding to the individual survey had a similar proportion of employed people, a higher mean age, a lower proportion of men, and a higher proportion of persons with a higher education degree compared to the general population of Flanders. -> Original sentence: On the individual level, the group of responding citizens had a similar proportion of employed people as in Flanders, a higher mean age, a lower proportion of men, and a higher proportion of persons with a higher education degree - Page 18: Discussion section -> Revised sentence: High external validity was found, with adoption rates in the present study found to be independent of organizational characteristics such as staff size, type of organization, working context, and age of staff. -> Original sentence: In terms of generalizability, it is encouraging to find that the adoption rates in the present study were independent of most studied organizational characteristics, namely staff size, type of organization, working context, and age of staff. 3A. Authors inconsistently refer to the project as a 10,000 Steps, 10,000 Steps, or the 10,000 Steps which makes it hard to decide whether they refer to one specific program or a group of programs? 9

11 We agree with the reviewer that the references to the project were inconsistent. Therefore, the revised paper uses just one formulation when referring to the project, which is 10,000 Steps (also highlighted in the revised paper). Some examples are again provided for the reviewer: - Page 4: Background section: -> Revised sentence: Therefore, this paper reports on the impact of the wide-scale dissemination of 10,000 Steps... -> Original sentence: The main purpose of the present paper is to report on the impact of the widescale project dissemination based on the 10,000 Steps project... - Page 10: Measures section: -> Revised sentence: Because an important focus of this dissemination study was to make professional organizations aware of 10,000 Steps while using limited resources... -> Original sentence: Because an important focus of this dissemination study was to make professional organizations aware of the 10,000 Steps project... - Page 20: Discussion section: -> Revised sentence: On the individual level, we found a moderate reach or awareness of 10,000 Steps... -> Original sentence: On the individual level, we found a moderate reach or awareness of 10,000 Steps projects... 3B. Dissemination efforts for the 10,000 Steps project were initiated by the end of 2007 would this be at the end of 2007? That is correct. We changed this in the revised paper: Page 6: section Dissemination efforts for the present study in Flanders : Dissemination efforts for the present study in Flanders (population: 6,160,600 inhabitants, surface: 13,521 km2, density: 456 inhabitants/km2) were initiated at the end of C. Basically this pilot project should this be This pilot project? In the revised paper we changed this sentence and refer to the 10,000 Steps Ghent project: Page 4: Background section: The 10,000 Steps Ghent project included multiple community-based strategies to promote PA in the adult population... 3D. When describing the 10,000 Steps Ghent project We suspect the reviewer refers to the narrative used in the following sentence of the Background section: Outcomes showed a significant increase in the average daily steps in the intervention community, while there was a slight decrease in the comparison participants step counts after one year. We replaced it by the following sentence in the revised paper: 10

12 Page 4: Background section: An increase of 8% in the number of people reaching the 10,000 steps standard was seen after a year, compared with no increase in the comparison community. 3E. The most mentioned reasons on page 17 should be the most commonly or the most often mentioned reasons This is adjusted in the revised paper: Page 18: Discussion section: Within the most often mentioned reasons for not initiating partnerships,... 3F. Last sentence of Discussion on page 21: If a similar wide-scale dissemination study would not include such a measure and a low adoption rate would be found, it would be difficult to know This sentence is reformulated in the revised paper (see underlined text below): Page 22: Discussion section: Fourth, an effectiveness dimension on the organizational level was added. This provided an answer to the question if efforts to make organizations aware of 10,000 Steps were effective. It also provided complementary information on adoption rates and to which extent these rates were mediated by the applied dissemination strategy or by the characteristics of the project itself. Discretionary Revisions: 1. It would be nice to clarify in the introduction section that Rogers Diffusion of Innovations framework was used to guide the D&I efforts and then the RE-AIM framework to evaluate the project. As suggested by the reviewer, we clarified the theoretical foundation for the dissemination efforts: Page 4-5: Background section: Rogers Diffusion of Innovations theory [6,7,17] was used to guide dissemination efforts and the RE- AIM framework [16] to evaluate and analyze the impact of 10,000 Steps. The RE-AIM framework has previously been applied to evaluate dissemination efforts for PA programs [5]. It assesses the reach, effectiveness, adoption, implementation, and maintenance of a project in order to estimate its public health impact. Moreover, this framework is compatible with a socio-ecological model and applied public health interventions [16], making it a particularly suitable framework for the evaluation of 10,000 Steps. 2. Because the D&I field is in need for agreed upon terminology (and because currently there is a lack of standardized language for the field), it would be nice if the authors included in the introduction their working definition for dissemination (or dissemination and translation as they referred to it at some places). 11

13 We agree with the reviewer that a working definition for dissemination should have been included as this is often forgotten in other studies on D&I. The revised paper has a definition included, based on Rogers work on Diffusion of Innovations: Page 4: Background section: Dissemination was considered as an active approach of spreading an evidence-based intervention (i.e. 10,000 Steps ) to the target audience via determined channels using planned strategies [15]. 12