Username or Email Address
Lost your password?
International association for the measurement and evaluation of communication
Category: Innovation award for new measurement methodologies
Client/Entering Company: Public Health England
Campaign title: Change4Life Sugar Smart
Company Name: Change4Life
Change4Life is the British Government’s flagship brand focused on tackling childhood obesity. For the January 2016 Sugar Smart campaign, Public Health England designed an evaluation that tracked the impact of communications all the way through the AMEC valid metrics framework, proving its influence, not just on communications outputs and outtakes, but on robust outcomes, including decreases in sales of added-sugar products and on the amount of sugar children consumed, a tangible step forward in the fight against obesity-related illness.
Change4Life targets families with children aged 5-11, to reduce childhood obesity by influencing knowledge, attitudes and behaviours. A multifaceted programme, with several campaigns throughout the year, the initiative ran for six weeks during January/February 2016.
The SMART objective was to ensure that, as a direct result of the campaign, at least 400,000 families downloaded the new Sugar Smart mobile app by 28 February 2016, and used it to reduce the amount of sugar in their children’s diets.
The campaign was developed using our behaviour change model, which has been refined through the programme’s seven-year history. This model is alert -> motivate -> support -> sustain.
The role for the campaign was set out at each stage:
An evaluation approach was required that could measure the impact of the campaign throughout this model, aiming for statistically significant improvements against the following 2015 baseline:
While the data has consistently shown that Change4Life engages families at scale, establishing a link between communications and clinically significant behaviour changes has always been challenging. However, for a campaign looking to improve the lives of children, it was essential we understood whether it worked.
The 2016 Sugar Smart campaign provided an excellent opportunity to prove the link to clinically significant outcomes, including sales of featured added-sugar food categories, and children’s overall dietary intake.
This required an evaluation approach that measured the campaign’s impact throughout the behaviour change journey, specifically:
The evaluation needed to keep track of a complex range of communications outputs (ratings, coverage, information packs provided etc.), outtakes (changes in awareness, knowledge and attitude), immediate outcomes (downloads and usage of the app), intermediate outcomes (sales of high sugar product categories) and longer-term outcomes (the amount of sugar children consumed).
Finally, the evaluation needed to address challenges such as:
This was no easy task, and no single research method could provide the richness of data need to meet this requirement. Accordingly, Public Health England brought together an evaluation approach in concert with its communications agencies, its evaluation partner (Kantar Public), a retail partner (Tesco), and Newcastle University, to devise a holistic evaluation protocol. This included a basket of measurement approaches, moving beyond standard communications metrics, to include experimental approaches, academic studies and use of industry data.
This complex evaluation task was coordinated by Kantar Public, the independent market research agency to ensure objectivity in the collection, analysis and interpretation of the data, with oversight from PHE’s insight and planning team. We used multiple data sources to look at impacts in different ways so that we could assess the project’s success against its objectives.
The main elements of the evaluation involved:
Quantitative tracking study
Setting up a bespoke tracking study:
Analysing trends in purchasing of key product categories:
Controlled study using digital screens
Setting up a unique experiment using digital screens, to measure the impact of the presence of the campaign creative on sales of featured products:
Dietary intake study
Commissioning an dietary intake study with Newcastle University, to provide best-in-class analysis of changes to children’s diets:
In addition, these sources were supplemented with digital metrics (including app usage, scans of products, social media metrics and website visits), PR metrics and metrics from face to face events (numbers of participants and exit survey responses).
The evaluation approach met Public Health England’s needs: we can now confidently state that a communications campaign drove changes to parental purchasing habits, leading significant reductions in the purchase of featured high-sugar product categories, and in children’s intake of sugar. The evaluation demonstrated that these changes were over and above underlying trends in the market; moreover, it showed that reducing sugar from one source was not compensated for by increasing sugar from other sources.
The multiple sources and methods used (claimed data from parents, usage data from the app, sales data from retailers and dietary intake data from Intake24) all tell a consistent and compelling story: in response to the campaign, parents used the app to reduce the sugar content of their children’s diets.
Since all data sources concur, Public Health England has had the confidence to repeat and expand the programme to cover salt and saturated fat, which is currently running as the “Be Food Smart” campaign in 2017.
In line with the AMEC Valid Metrics Framework, we have reported our results in terms of awareness, knowledge, consideration, preference and action.
The tracking study established that 70% mothers with children aged 5-11 were aware of the campaign.
Similarly, the tracking study indicated that knowledge of the maximum amount of sugar children should consume increased from 5% before the campaign to 18% after.
It also showed a significant increase in the proportion of mothers who recognised that their children ate more sugar than was good for them (from 41% before the campaign, to 47% afterwards).
Consideration and Preference:
The tracking study indicated that:
Data from Tableau showed that the campaign drove immediate, measurable action:
The tracking study established that 75% of all mothers who were aware of the campaign said they had made changes as a result.
The test and control study showed that the campaign led to a 4% decrease in sales of sugary cereals, both during the campaign period and 8 weeks later, a 3% decrease in sales of sugary drinks and a 4% increase in the sales of diet drinks during and post campaign in the test stores compared to the control.
The purchase data analysis confirmed a greater decline in purchasing of our key sugary categories during the campaign compared to the overall trend over the year.
Finally, the dietary intake study enabled us to establish the impact of the campaign. However, the results of this are not yet publicly available.
Results were submitted to the Parliamentary Under Secretary of State for Public Health & Innovation and will be published in Public Health England’s 2017 – 2020 Marketing Strategy. They are also currently being written into a an academic paper for publishing so that the results can be shared widely to help others with campaign planning.
Click on the different category headings to find out more. You can also change some of your preferences. Note that blocking some types of cookies may impact your experience on our websites and the services we are able to offer.
These cookies are strictly necessary to provide you with services available through our website and to use some of its features.
We provide you with a list of stored cookies on your computer in our domain so you can check what we stored. Due to security reasons we are not able to show or modify cookies from other domains. You can check these in your browser security settings.
We also use different external services like Google Webfonts, Google Maps, and external Video providers. Since these providers may collect personal data like your IP address we allow you to block them here. Please be aware that this might heavily reduce the functionality and appearance of our site. Changes will take effect once you reload the page.
Google Webfont Settings:
Google Map Settings:
Vimeo and Youtube video embeds: