Evidence to Policy: Identifying Strategies to Improve Knowledge Dissemination from Investigators

Evidence to Policy: Identifying Strategies to Improve Knowledge Dissemination from Investigators Maximilian Pany David Grande, MD, MPA & Zachary F. Meisel, MD, MPH, MSc Leonard Davis Institute of Health Economics

Presentation Outline Introduction Methods Preliminary findings Next steps Lessons learned

Presentation Outline Introduction Methods Preliminary findings Next steps Lessons learned

Ideal scenario: health policy influenced by research National efforts are underway:» NIH focus on improving translational research» PCORI How well does translation of evidence into policy actually work? How do policymakers use evidence to shape policy?

Gaps between producers (researchers) & end-users (policymakers) of research Incentives Social networks Time-frames Vocabularies Values

Policymakers Researchers Require research:» Timely» Relevant» Accessible» Presented by trusted individuals Research to:» Solve problems» Contribute to knowledge» Get promoted» Get paid

Knowledge Gap: Researchers Little evidence on how good researchers at translation of knowledge No research available on researchers actual knowledge of, or attitudes toward, translating evidence

How do researchers & policymakers interact? researchers policymakers

How do researchers & policymakers interact? peer-reviewed journals researchers policymakers

How do researchers & policymakers interact? peer-reviewed journals policy briefs researchers policymakers

How do researchers & policymakers interact? peer-reviewed journals policy briefs academic conferences researchers policymakers

How do researchers & policymakers interact? peer-reviewed journals legislative committees policy briefs academic conferences researchers policymakers

How do researchers & policymakers interact? peer-reviewed journals policy briefs legislative committees caucus academic conferences researchers policymakers

How do researchers & policymakers interact? peer-reviewed journals policy briefs legislative committees caucus lobbyists academic conferences researchers policymakers

How do researchers & policymakers interact? peer-reviewed journals policy briefs academic conferences legislative committees caucus lobbyists think tanks researchers policymakers

How do researchers & policymakers interact? peer-reviewed journals legislative committees policy briefs academic conferences news media caucus lobbyists think tanks researchers policymakers

How do researchers & policymakers interact? peer-reviewed journals legislative committees policy briefs academic conferences news media social media caucus lobbyists think tanks researchers policymakers

How do researchers & policymakers interact?? researchers policymakers

Project Aims 1. Identify barriers and facilitators to dissemination of evidence 2. Investigate role of social media in dissemination 3. Improve translation, dissemination, and impact of health-policy relevant scholarship to policymakers

Presentation Outline Introduction Methods Preliminary findings Next steps Lessons learned

Investigating Researchers: Recruitment Strategy Random sample from faculty attending Academy Health meeting» Professors» Researchers» Deans

Investigating Researchers: Recruitment Strategy

Investigating Researchers: Recruitment Strategy

Investigating Researchers: Recruitment Strategy

Investigating Researchers: Recruitment Strategy

Investigating Researchers: Recruitment Strategy

Investigating Researchers: Methods Measure attitudes and behaviors of health policy researchers towards dissemination channels» Preliminary qualitative interviews to refine hypotheses» Structured interviews at Academy Health» Phone interviews after Academy Health

Academy Health: SUMR researched the researchers

Academy Health: SUMR researched the researchers

Investigating Policymakers: Methods Convenience sample from policymakers attending National Conference of State Legislatures meeting» Legislators» Staff

Investigating Policymakers: Methods Measure attitudes and behaviors of legislators and their staff toward methods and channels of dissemination» Piloted survey with Wyoming legislators» Survey at NCSL meeting» Qualitative interview with legislators & staff

Presentation Outline Introduction Methods Preliminary findings Next steps Lessons learned

Preliminary findings: Researchers Hypothesis: researchers are accustomed to, value, and prioritize dissemination via the mainstream traditional media Variable Total AH Phone N 216 189 27 Female 58% 58% 59%

Survey: Experiment

Survey: Experiment Vignette: Dr. Smith is a faculty member at a large university. She recently completed a grant-funded study comparing the impact of several health policy options including taxes, food marketing regulations and educational interventions on childhood obesity. She published the study in a major peer-reviewed journal.

Survey: Experiment Vignette: Dr. Smith is a faculty member at a large university. She recently completed a grant-funded study comparing the impact of several health policy options including taxes, food marketing regulations and educational interventions on childhood obesity. She published the study in a major peer-reviewed journal. Group A: Traditional Dissemination

Survey: Experiment Vignette: Dr. Smith is a faculty member at a large university. She recently completed a grant-funded study comparing the impact of several health policy options including taxes, food marketing regulations and educational interventions on childhood obesity. She published the study in a major peer-reviewed journal. Group A: Traditional Dissemination Group B: Social Media Dissemination

Survey: Experiment Vignette: Dr. Smith is a faculty member at a large university. She recently completed a grant-funded study comparing the impact of several health policy options including taxes, food marketing regulations and educational interventions on childhood obesity. She published the study in a major peer-reviewed journal. Group A: Traditional Dissemination Group B: Social Media Dissemination Group C: Direct Outreach

Survey: Experiment Vignette: Dr. Smith is a faculty member at a large university. She recently completed a grant-funded study comparing the impact of several health policy options including taxes, food marketing regulations and educational interventions on childhood obesity. She published the study in a major peer-reviewed journal. Group A: Traditional Dissemination Her University communications office issued a press release leading to an interview and story in a major national newspaper and several other newspapers.

Survey: Experiment Vignette: Dr. Smith is a faculty member at a large university. She recently completed a grant-funded study comparing the impact of several health policy options including taxes, food marketing regulations and educational interventions on childhood obesity. She published the study in a major peer-reviewed journal. Group B: Social Media Dissemination She blogged about her findings on a major news website and then tweeted about them leading to widespread attention on other major websites and social media outlets.

Survey: Experiment Vignette: Dr. Smith is a faculty member at a large university. She recently completed a grant-funded study comparing the impact of several health policy options including taxes, food marketing regulations and educational interventions on childhood obesity. She published the study in a major peer-reviewed journal. Group C: Direct Outreach to Policymakers She contacted and met with the chair of the health committee in her state legislature. Her state had been considering policy options to reduce childhood obesity. After presenting her findings, she was invited to provide testimony to the full health committee.

Survey: Experiment Questions about vignette: 1. How effective are Dr. Smith s actions as a way to disseminate research findings to policymakers and other decision-makers? 2. How confident are you disseminating your own research in the way Dr. Smith did? 3. How much do your academic peers respect the way Dr. Smith disseminated her research? 4. How much does your academic promotion process value the way Dr. Smith disseminated her research?

Effectiveness (scale 1-10) N.S. N.S.

Self-efficacy (scale 1-10) ** *

Peer respect (scale 1-10) ** N.S.

Valued by promotion process (scale 1-10) ** *

Effectiveness of channels of communication (scale 1-10) Direct outreach Articles in news Policy briefs Editorials/commentaries Research articles Twitter Blogs Facebook n = 216

Use of communication methods in past year (Y/N) Direct outreach Interviewed by news media Article in news Contacted community group Policy brief Editorial Blogged Facebook Twitter n = 216

Qualitative coding in NVivo

Coding scheme: social media» Social media impact / efficacy (+/-)» Social media reach (+/-)» Traditional methods of dissemination (+/-)» Communication characteristics social media (+/-)» Social norms and scholarly reputation - social media (+/-)» Institutional resources and factors - social media (+/-)» How, when, what to post on social media» How to use social media for professional purposes other than posting» Self-efficacy social media (+/-)» Not my role social media

Coding scheme: challenges to affecting health policy & training» Challenge characteristics, priorities of research» Challenge cultural, communication gaps between researchers and policymakers» Characteristics, priorities of policymaking (+/-)» Type and content of training, resources» Format and venue of training, resources» Best practices» Great quotes

Quotes from AH interviews I get all these tweets and they seem like a foreign language sometime and I don't really get the message, and is it all about getting me to a link or what? I don't know. And maybe it's how to use it effectively. It's one thing of here's how to set up the account but how does one use it effectively and read it effectively? It feels like you need language skills. I still think of Twitter as my children using the account, and they are not my target audience.

Quotes from AH interviews The biggest challenge is just to make sure people understand the nature of your research - that sometimes it's not to answer a big question broadly, a big interesting question broadly, but to answer a small question very well that often the general public may think, 'Oh, I could have said that, I already knew the answer,' but there s a big difference between knowing the answer by gut feeling and having it established scientifically and rigorously. I guess it's the expectation between what the public or the consumer of research conceive of what research is, and what research actually is.

Quotes from AH interviews The vast amount of studies that are done and the tendency for people to expect that every study is gonna provide an answer to a specific question, as opposed to a recognition that each study is just one piece of evidence in a broad stream of intellectual progress.

Quotes from AH interviews Well I'm certainly very committed to the democratization of science. And you know that's one thing to make it accessible. Now it's not clear to me how successful it is because none of us really know how to talk to people. Health policymakers. Where they are. That's the biggest problem. Where do we find them?

Preliminary Discussion Direct outreach perceived to be most efficacious Traditional forms of dissemination outrank social media Think that peers respect many of these forms, including direct outreach, but peers don't respect social media Think that promotion process doesn't respect direct outreach

Presentation Outline Introduction Methods Preliminary findings Next steps Lessons learned

Next steps Complete data analysis (AH & NCSL) Social media analysis Write up results Disseminate findings traditional methods & social media

Presentation Outline Introduction Methods Preliminary findings Next steps Lessons learned

Lessons learned Effective subject recruitment requires many nudges & creative methods Dissemination can t stop at publication Research is much more fun when done collaboratively Health services researchers are a friendly bunch!

Thank you for a wonderful SUM(ME)R! Dave Grande & Zack Meisel Addie Goss & Austin Kilaru Joanne Levy Shanae Johnson & Safa Browne Hoag Levins & Megan Pellegrino SUMR cohort 2013

Thank you for a wonderful SUM(ME)R!

QUESTIONS?