Cooking Matters at the Store Evaluation: Executive Summary Introduction Share Our Strength is a national nonprofit with the goal of ending childhood hunger in America by connecting children with the nutritious food they need to lead healthy, active lives. Through its No Kid Hungry Campaign, Share Our Strength ensures that children in need are enrolled in effective federal nutrition programs; invests in community organizations fighting hunger; teaches families how to cook healthy, affordable meals; and builds public-private partnerships to end childhood hunger at the state and city levels. One of its key programs, Cooking Matters at the Store, was developed to provide practical skills to assist participants in making healthy food choices on a limited budget. Held at the grocery store, this program provides learning experiences in a real-world setting. Based on research showing that food shopping practices that include comparison shopping and nutrition label use are associated with measures of better dietary quality, 12 Cooking Matters at the Store identifies four key program behaviors that will assist program participants in their goal of healthy food purchases: (1) identify at least three economical ways of purchasing fruits and vegetables; (2) practice comparing unit prices; (3) practice comparing food labels; and (4) practice identifying whole grains. Since its beginning in 2010, the Cooking Matters at the Store program has steadily expanded to 41 states and is projected reach to more than 35,000 adults in 2013 alone. In conjunction with this quick expansion, the program is undergoing a critical phase of development to ensure maximum reach while ensuring that program outcomes are met. The goal of the Cooking Matters at the Store evaluation was to assess the implementation and impact of this interactive shopping tour for low-income families. Assessment of program implementation examined all facets affecting how the tour was carried out, from the recruiting of participants, to the facilitator s background, to the teaching style used, to the materials distributed and the store itself. The impact evaluation focused on changes in participants food selection purchases and their ability to stretch their budget, buying more nutritious foods at a cost equal to or less than before the tour. This Executive Summary highlights key findings and recommendations from the Cooking Matters at the Store evaluation, carried out by Altarum Institute under contract with Share Our Strength. Methodology The evaluation of the Cooking Matters at the Store program included two major components: (1) a process evaluation to understand how the program was implemented with a particular focus on facilitators and barriers to program success; and (2) an impact evaluation to measure whether the program had an influence on participant shopping behaviors. Process Evaluation To better understand how the Cooking Matters at the Store program was implemented and received by stakeholders, a process evaluation was conducted with select sites and stakeholders in three states (Arkansas, Colorado, and Maryland) more experienced in implementation. Data sources from these sites included qualitative interviews with key program stakeholders (tour facilitators, lead partner representatives, and retail store managers), focus groups with tour participants, and observations of the grocery tours. Impact Evaluation Participant Surveys In order to assess the impact of the intervention on participants attitudes and self-reported behaviors, Cooking Matters at the Store participants were surveyed at the time of the tour and again approximately six weeks later by mail. The 1 Hersey, J., Anliker, J., Miller, C., Mullis, R. M., Daugherty, S., Das, S., et al. (2001). Food shopping practices are associated with dietary quality in low-income households. Journal of Nutrition Education, 33, S16 S26. 2 Satia, J. A., Galanko, J. A., Neuhouser, M. L. (2005). Food nutrition label use is associated with demographic, behavioral, and psychosocial factors and dietary intake among African Americans in North Carolina. Journal of the American Dietetic Association, 105, 392 402.
surveys collected data on several relevant dimensions including participant demographic characteristics, shopping behaviors directly related to program specific goals, reactions to the tour itself, and the retention of and confidence to apply tour information. The survey population was recruited from participants of 24 regular tours conducted by facilitators during the study period. Receipt Study The impact evaluation also included the collection of participant grocery store receipts. Participants were recruited in advance of the store tour and asked to collect two weeks of pre-tour grocery receipts as well as six weeks of post-tour grocery receipts. In total, 63% of participants returned the mailing containing saved grocery receipts. In order to determine whether participants made any changes in purchasing behaviors as a result of the intervention, grocery receipts were coded according to MyPlate food groups, capturing fruits, vegetables, whole grains, dairy, and related products. Cost savings were also captured on each receipt and included in the analysis. Study Population As part of the process evaluation, six site visits were conducted in the three study states. These visits included in-person interviews with eight facilitators, five lead partners, and four retail store managers; seven grocery tour observations; and focus groups with a total of 46 tour participants. In-depth telephone interviews were also conducted with a diverse subset of nine facilitators from the three study states and California and Oregon (Table 1). Table 1: Number of Respondents and Completed Tour Observations during Process Site Visits Process Data Collection Method STATE Arkansas Colorado Maryland TOTAL Facilitator interviews 5 5 5 17* Lead partner interviews 2 2 1 5 Retail store manager interviews 2 1 1 4 Focus group participants 18 9 19 46 Grocery tour observations 2 2 3 7 *Total includes the two in-depth phone interviews completed with facilitators from California and Oregon. For the impact evaluation, participation data were collected from survey questionnaires and grocery shopping receipts. A total of 163 participants were recruited. In total, 169 participants completed the pre-intervention questionnaire administered to them by the tour facilitator after the tour and 114 participants completed the post-intervention questionnaire that was mailed to them five weeks after the tour. Additionally, 1,307 receipts were collected from 103 participants over an eight-week period of time (two weeks prior to the tour and six weeks after the tour) (Table 2). Table 2: Number of Study Participants, Questionnaires, and Receipts Collected Pre- and Post-Intervention Impact Data Collection Method Pre-intervention Post-intervention TOTAL Receipt study participants 163 103 103 Receipts collected 189 1118 1307 Completed questionnaires 169 114 114 2 P a g e
Findings: Process Evaluation Grocery Tour Observations The seven grocery tours observed were, on average, about six people in size and 66 minutes in length. Most tours were conducted during off-peak hours and were held at large, chain grocery stores with ample space (e.g. wide aisles) and a variety of food offerings. Overall, the produce, bread, canned- or dry-food, meat, and cereal sections were included in all or most tours. The dairy, frozen foods, and snacks/juice sections were included less often in about half or less tours. Only two of the tours completed the $10 Challenge activity during or after the tour, most often due to time constraints. In conducting the tours, most facilitators acknowledged challenges and brainstormed solutions and were respectful of participants decisions during tours; however, some were challenged in keeping participants engaged and in discussing all key talking points. Some of the key tactics facilitators identified in conducting tours included: grouping participants into small groups in order to maximize the tour discussion and minimize distractions, maintaining command of the tour and a good pace throughout the store, and including hands-on, interactive activities within each grocery section. Facilitator and Lead Partner Interviews In terms of facilitator background, most had been conducting tours for a year or less and had no previous experience and minimal practice in conducting grocery tours. The majority of facilitators received either online or in-person training to lead tours. While they were generally satisfied with the training, some recommended offering more opportunities to observe or practice leading a tour. Facilitators found the guide helpful in preparing for the tour but suggested that additional talking points be provided in some form that is easy to use during tours (e.g., ring of cards). Facilitators and partners reported using a variety of methods to recruit participants for tours (e.g., hanging flyers, recruiting in store or through other programs); however, many reported challenges with tour attendance which they believed to be related to transportation and time. Facilitators reported omitting some grocery sections or the $10 Challenge in the interest of time. When selecting stores, facilitators considered location, popularity with and proximity to participants, and size and experienced little resistance in working with local stores. Most partners reported that the mini-grants helped sustain their programs and some felt they needed more staff and/or funds to conduct more tours. Retail Store Manager Interviews Most store managers were supportive of the program and recognized that it s helpful to families in the community but also has benefits for their business. Managers felt that tours were not disruptive to their store and some even took the extra effort to make sure their store looked good for tours. In terms of planning for and participation in tours, most managers were not involved in recruitment of participants for tours and did not join or observe tours that were conducted in their stores. Participant Focus Groups A high level of participant satisfaction in the tours and key tour concepts was reported and many said they felt free to contribute and ask questions during tours. Many participants reported that they intended to make changes in their shopping as a result of the tour. In addition to reading labels and comparing unit prices more, many participants expressed a desire to switch to whole grains. Participants enjoyed receiving the tour incentives but indicated that they would have still participated without them. Many participants noted the lack of time to complete the $10 Challenge and suggested that this activity be better incorporated into tours. To improve participation in tours, participants suggested making tours smaller and more interactive, holding them at more stores and during more times throughout the week, and doing community outreach. 3 P a g e
Findings: Impact Evaluation Participant Surveys The majority of participants expressed high satisfaction and positive feedback about the tour. Data suggest the program is having an impact consistent with its goals, as all measures of key shopping behaviors increased after the tour. Participants expressed a high degree of confidence applying the skills they learned in the tour while shopping on their own. Participants appear to retain hands-on, skill-oriented aspects of the tour, such as reading the food label, comparing unit prices and identifying whole grains. Receipt Study Of the more than 1,300 receipts collected, almost half of receipts came from large supermarket chains such as Kroger and Safeway and nearly one quarter of the grocery receipts came from Walmart stores, alone. The remaining receipts came from small/medium grocery stores (20%), dollar stores (4%), warehouse club stores (3%), smaller convenience/corner stores, and other retailers. The grocery receipt data provided detailed information about participants purchase of fruits and vegetables, including quantity, variety and type (fresh, canned, frozen and dried). Findings from these data suggest the tour supported or otherwise encouraged participants to purchase fresh fruits and vegetables, with a sharp increase in the frequency fresh fruits and vegetables were purchased immediately following the store tour. There was little indication that the tour encouraged participants to purchase more canned, frozen or dried fruits and vegetables as a result of the tour, however, these changes may have been too small to detect among the limited number of cases. The grocery receipt data support the process evaluation findings, in that, participants appear to have put into practice the belief that fresh is best by prioritizing the purchase of fresh fruits and vegetables over canned, frozen or dried. The data do not suggest participants, on the whole, greatly altered the amount of money they spent on fruits and vegetables, comparing pre-tour receipts to post-tour receipts, nor did they increase the variety of fruits and vegetables purchased. However, this does not hold true when the data is broken out by participants household characteristics. The grocery receipt data suggest the grocery tour may have affected participants differently, depending on whether or not there were children in the household. Nearly half (43%) of tour participants reported having children in the household, and of these, they appeared to have spent more on fruits and vegetables following the store tour, as compared to participants without children in the household who spent about the same on fruits and slightly less on vegetables following the store tour. While there are not enough cases to establish statistical significance, these findings are suggestive of potential benefits for families with children in the household. The receipt data does not demonstrate any noticeable differences in participants purchase of whole grains following the store tour; however, there were limitations in accurately identifying whole grain products on the receipts so these results should be interpreted with caution. Recommendations and Post-Evaluation Program Modifications Recommendations for implementation of the Cooking Matters at the Store program are listed below. All recommendations were identified by program partners, facilitators, store representatives, tour participants, or Altarum staff through observations or other information gathered as part of this evaluation. At the conclusion of the evaluation, Share Our Strength considered the findings and recommendations to determine the appropriate modifications for the Cooking Matters at the Store Program. Encourage and expand opportunities for more hands-on training and shadowing for facilitators. In order to provide high-quality training to tour facilitators with a variety of backgrounds, Share Our Strength launched the Cooking Matters at the Store Learning Space in July of 2013. The Learning Space is a web-based learning and training portal through which all current and future tour leaders receive interactive and dynamic 4 P a g e
training on best practices to lead a tour. The new training includes eight short in-store mock tour videos which outline the key components of a tour for new tour leaders, as well as interactive self-assessments for tour leaders to practice what they learn. The learning space also includes a resource center, discussion board and forum for tour leaders to connect and share their experiences across the country. Study findings suggest that it is worth considering a future component that follows an entire store tour from beginning to end, either through video or virtually, so that tour leaders have the opportunity to observe how a perfect tour is run from start to finish. This build-out is planned for future iterations. Concurrently, new tour leader flip-cards were created to replace the larger, binder-style facilitator guide. This new format streamlines materials, eliminates bulk and focuses tour leaders on key messages to ensure greater consistency. Making tour talking points available on other tablets or smartphones, as recommended by facilitators could be a next-level project in the future. Work with corporate retailers to obtain preapproval for partner organizations or volunteers to conduct tours in stores. We continue to work with current and potential retail partners to explore the best ways to provide information on the Cooking Matters at the Store program to employees so they are aware and supportive of the program locally. Since the evaluation completed, we have also been collecting and sharing best practices from tour leaders on how to best engage and build positive relationships with stores. Adapt the model for implementing this program to encourage offering a series of tours and/or populationspecific tours. Currently, this product extension is on hold as we further refine and streamline the current Cooking Matters at the Store program.. Offer partners and volunteers better guidance on overcoming recruitment challenges and introduce more channels for partners to share best practices in tour recruitment. In the year since the evaluation completed, there has been a big focus on exploring and collecting best practices from the field on tour recruitment, which will be shared on the Learning Space and enhance through the buildout of interactive features and discussion boards. 5 P a g e