Category: Best measurement of a not-for-profit campaign

Client/Entering Company: Isentia

Campaign title: White Ribbon 2016

Company Name: White Ribbon Day

Summary

White Ribbon Day is held annually on 23 November and aims to end violence towards women. It relies on grants, funding and donations to run its events and awareness programmes. Isentia has a longstanding partnership with White Ribbon to help it demonstrate the effectiveness of the campaign each year to ensure resources are well directed for the next campaign. In 2016 it worked with Isentia to use the AMEC Integrated Evaluation Framework to plan its campaign and its subsequent measurement.

Objective/Brief

The core brief for the White Ribbon Trust is having an overview of the return of their communications strategy and activity during the campaign period (November). As a charitable trust, the small budget available increases the need to communicate via low (or no) cost channels such as traditional and social media.

Similarly, the topic of domestic violence is complex, so requires a more detailed view and sensitive analysis than lowcost tools can provide. In 2017, the Trust needed to know:

  • If earned media were still supportive of the campaign and engaged with the core messages of violence, consent and respectful relationships.
  • The impact of an improved social content strategy.
  • The effectiveness of its ambassadors and partner organisations in communicating key messages.
  • How to keep improving its campaign to reach new audiences in 2017.

 

Strategy

Isentia has been working with White Ribbon since 2009, when the purpose of the project was to show potential partners and sponsors the benefit of being associated with the campaign. While the campaign steadily improved its favourability and reach over time, the 2015 campaign was marred by controversy. Following on from two separate incidents in which headline ambassador and NZ Prime Minister John Key appeared to make light of the topic of rape, public outcry over the campaign’s continued use of Key as a representative snowballed, culminating in sustained criticism of the organisation on social media, and the presentation of a public petition with over 11,000 signatures demanding Key be dropped by the campaign. The effects of this reaction were borne out in Isentia’s postcampaign analysis for 2015 – which signalled a sharp decline in average favourability across both traditional media and public social platforms year-on-year, and overall one of the poorest campaign performances in recent years.

As a result of these findings, White Ribbon and Isentia worked collaboratively on setting new objectives for the 2016 campaign. Reviewing the previous year, a key weak point was identified in the fact that whilst previous frameworks had listed outputs and outcomes for social media audiences, these channels were largely used to amplify existing traditional media coverage, or to direct traffic back to owned White Ribbon assets such as the campaign website. Use of the new integrated framework approach in planning for the 2016 campaign challenged organisers to differentiate their approach, and to consider unique content and messaging for traditional and social media, and for paid, earned, shared and owned platforms, and in turn the evaluation of these efforts.

As the campaign has a very limited budget, continuing to examine traditional earned media as a way to start conversations and convey key messages at no cost remained important. The approach for the measurement of the 2016 campaign was to continue to qualitatively analyse print, broadcast and online news, as well as public social media posts about White Ribbon over a three-month period (October–December 2016). In order to more effectively look at how content was being discussed and shared on Facebook, this year Facebook Topic Data was introduced to the dataset. The data collected from Facebook focused on the wider topic of domestic violence during October– December, to determine how influential White Ribbon and its partnerships had been at generating discussion and engagement with a complicated, and traditionally quite avoided, topic.

 

Execution/Implementation

The campaign was analysed using two different datasets and methodologies:

  1. The CARMA qualitative methodology was used to analyse all mentions of White Ribbon in New Zealand print, broadcast and online news as well as all public social media posts that mentioned the campaign or an associated hashtag that were posted within New Zealand.
  2. Facebook Topic Data was used to analyse how Facebook users discussed the wider topic of domestic violence in order to see how White Ribbon drove coverage and engagement during the campaign.

Qualitative Analysis:
Content was gathered from two proprietary databases based on the keywords “White Ribbon” across traditional media and social media.

A total of 868 traditional media items and over 1,000 social media posts were collected and individually analysed using the CARMA qualitative methodology (see supporting materials, page 1). Content was analysed for the presence of spokespeople, detractors, tone, language and messages. Message analysis was central to the campaign due to the emotive and complicated topic of domestic violence, the analysis therefore tracked broader messages regarding consent and attitudes towards violence (see supporting materials, page 2). The campaign also utilised ambassadors, usually male celebrities or family members from high-profile domestic violence cases, to convey messages across all media types. Each ambassador was tracked and their ability to convey the key campaign messages analysed.

Facebook Topic Data:
Facebook Topic Data was collected through a partnership between Isentia and Datasift. Datasift is able to collect aggregated, anonymised data across from all of Facebook, regardless of privacy settings. While individual posts cannot be analysed, users’ demographic data, link and hashtag shares, and the key topics they have engaged with are able to be collected.

For White Ribbon, the Facebook Topic Data research was set up so that all mentions of domestic violence from New Zealand Facebook users were included in the dataset. A Facebook Topic Data analysis can be used as a guide on how Facebook users discuss a topic without directly interacting with a public group or brand. We compared the tone and concerns reflected of users in the Facebook Topic data to those of traditional media coverage, to further determine if the concerns expressed in traditional media coverage represented the wider views of the public or not, and to examine how Facebook users interacted with the topic of domestic violence as a whole.

The demographic-led analysis of conversations about domestic violence on Facebook evaluated both private user feed conversations and conversations taking place on public pages, which were aggregated and anonymised to protect and maintain individual user privacy. This totalled nearly 71,000 interactions. This analysis provided insights into the age and gender demographics of New Zealanders who discussed the topic, what content they engaged with and shared, and the impact that the White Ribbon campaign had on the wider topic in this sphere.

Effectiveness of Assignment

The research conducted demonstrated three key outcomes for the White Ribbon campaign:

  1. A recovery from 2015
    The lasting impact of the previous year was minimal, and volumes and favourability increased compared to the 2015 campaign (supporting materials, page 2). The only media type less engaged with the campaign was broadcast, but this was the most unfavourable during 2015 due to the Prime Minister making inappropriate comments, so while the volume was lower, it was more favourable.
  2. Changing the quality of social media content
    The most pertinent finding of the research was that the shift to social media content that was tailored to social platforms (as opposed to replicating material from traditional media) proved successful. In particular, White Ribbon’s the strategy of encouraging partner organisations (such as the NZ Police) to create content that both organisations could share was beneficial to amplifying the conversation about domestic violence (supporting materials, slide 4).
    The most interacted-with link on Facebook was the link to the White Ribbon toolkit and a link to White Ribbon’s page about the release of Hollie Smith’s single ‘Please’, where all proceeds of the song went directly to the campaign.
    The second most interacted-with post on Facebook was a TVNZ report on the NZ Police video that featured Sergeant Tania Kingi. White Ribbon partnered with the Police to produce this content, but ultimately it was a Police-driven piece that was then shared. Without a plan to structure the campaign into a paid, earned, owned and shared model, this opportunity may not have occurred.
  3. Understanding the audience
    The addition of demographic information from Facebook provided specific insight into who engaged with the topic of domestic violence, and how they behaved. For example:Women carried out 71% of all interaction with domestic violence-themed content; the most active segment of the female audience was those aged 35–44, and women aged 18–24 were the most likely to interact with video content. (supporting material, page 3) The ratio of women re-sharing content related to domestic violence outnumbered men 7:1. The primary audience for the campaign – men – interacted with traditional media coverage of the White Ribbon Ride online more than any other content. The content that men interacted with the most was traditional media coverage of the White Ribbon Ride (a motorcycle ride across the country).

Ultimately the measurement of the 2015 campaign encouraged a change in focus to gain a better understanding of how social media can be better utilised to create engagement and as a way to direct an audience to the back to the primary resource (the White Ribbon website). The 2016 evaluation demonstrates that this change in frame has resulted in more conversations and interactions with White Ribbon and its messages. It also helped to flesh out the knowledge of the current audience for White Ribbon and how to consider engaging with the desired target audience in 2017.

The ongoing evaluation of the campaign and the encouragement to plan content strategy based on strong data led to an increase in favourable coverage and social media interactions.

“Isentia are always pushing me to measure more and go out of their way to find additional data that will help us understand audiences and encourage engagement. We have learned from this research the benefit of our social partnerships and the need to invest further in our digital content for 2017. This information directly informs how we allocate resources and determine our strategy on a very limited budget for next year.” Rob McCann, White Ribbon Campaign Manager

Highlights

  • This research demonstrates the value of an ongoing evaluation and measurement plan and the direct impact of using this data to make strategic decisions.
  • White Ribbon utilised the Valid Metrics Framework to better understand the impact of Paid, Earned, Shared and Owned in their communications strategy.
  • The research showed the value of partnerships on social, with the paid Hollie Smith single and the video produced by NZ Police leading the interactions on Facebook.
  • The use of demographic data allowed for better understanding of how different audiences interact with content and to aid in planning of the 2017 campaign.

Isenta White Ribbon – supporting materials