The following are several key lessons that the global M&E team responsible for the programme learned after extensive consultation.
- Keep it simple
In order to allow the M&E framework to be adapted in various countries – both in size and situation – simplicity and flexibility are essential to the implementation of such a complex global programme. Initially, we developed an exhaustive list of KPIs, but we soon discovered that we needed the flexibility to tailor these to each country where budget constraints, office size and/or local context may or may not have made it possible to track all the KPIs. We continue encouraging countries to do all they can to track as many KPIs as possible – mainly those that matter the most to them – but are flexible enough to allow them to move forward at their own pace. The Handbook for the Implementation of KPIs, developed by DOC, has proved to be a valuable tool for explaining the main concepts and rationale, and to detail techniques to calculate each of the KPIs.
- Be serious about evidence-based communications: M&E should not be an afterthought
This is the first time that UNICEF has undertaken such a strategic approach to measurement and evaluation in its communications efforts. All strategies, including the global strategy and national strategies (based upon the global strategy, adapted to the local context) are informed by data, collected both in advance of the implementation phase and during it. This provided the opportunity to take the time to reflect on what contributed to the success – or the failure – of the campaigns and initiatives UNICEF implements. This was proven as DOC and many country offices organised evaluation sessions to identify areas for improvement and to better interpret achievements in order to replicate or bring to scale models of success.
- Data and impact analysis are helping us change the way we do business; the way we communicate
This new approach to measurement is contributing to changing the way we work; the way we communicate. We are now able to get a better idea of the impact and the reach of our communications initiatives, identifying gaps and progress. The use of frequent data monitoring has informed timely course corrections in communications plans and strategy and public advocacy efforts. This has also spearheaded efforts to be more responsive to the reality on the ground and has helped accelerate sustainable results for all children.
- Hire the right people, develop human capital and invest in capacity building
Investing in human capital is fundamental. Creating new positions, bringing in new skills and developing the capacity of the existing communications teams should be a priority to ensure the proper implementation of such an ambitious M&E plan. Candidates with relevant and up-to-date skills allow the organisation to remain abreast of the ongoing changes in the digital and media world.
DOC, in collaboration with the seven Regional Communications Officers, are regularly providing training and capacity building to field colleagues through webinars, bilateral calls and guidance tools.
- Work with the right partners – you cannot do it alone!
It is simply not possible to implement a truly global and multi-faceted framework like the one designed, without collaborating with external vendors and partners. DOC is working with several industry leaders that provide technical support, including Ketchum Global Research & Analytics. The service provided by Gorkana, the vendor contracted for the global monitoring and analysis, is complemented by Brandwatch (social media); Factiva (paid content); TV Eyes (TV and radio clips) and other tools. DOC encourages offices to use external media monitoring companies to assist in tracking KPIs, even in the most well-funded and staffed offices.
Having an external company provides both quantitative but also qualitative analysis to better inform communications activities. A local media monitoring company can also fill the gaps that a global provider may not be able to address (local languages/dialects, print media, as opposed to online editions, etc).
- Seek advice and do not reinvent the wheel
Others may have already found a solution to the problems you encounter. At the HQ level, Ketchum Global Research and Analytics (KGRA) provides strategic advice and helps coordinate the work of an advisory board, that includes representatives from other large global operations, including private sector companies, NGOs and academic institutions. The advisory board meets bi-annually and serves as a space to provide UNICEF with guidance and feedback on the approach and direction our measurement work is taking at a global level. The forum is also a place to share expertise and to discuss potential solutions and additional innovations to common challenges. UNICEF is greatly benefitting from the expertise and know-how of these senior communications professionals.
- Fail fast, fail cheap, try again, improve and scale up
Recent experience has shown that when support and guidance are shared, Country Offices can absorb them (as appropriate) and then generate greater results than initially envisaged. There have been remarkable innovations and agility from a wide cross-section of offices – and when DOC can act as a knowledge enabler, facilitating the exchange of specific Country Office experiences, new ideas and ways of working can be contagious and replicable. Each month, DOC produces a monthly compendium of examples of how the “voice, reach and engagement” pillars of the strategy are being put into action at country level, as well as facilitating case studies on the M&E framework implementation that can be useful learning tools for other offices tracking KPIs.
- Share data and insights in a timely manner. Do it right!
Sharing data and insights is important, but in order for it to have impact, it must be clear and shared in a timely manner. Data can be useful in decision-making – but it can also create confusion if not explained well. It is therefore important to be clear when communicating to audiences that may not understand the technicalities, or the assumptions behind the metrics.
Consistency and timeliness of data is essential. In 2015 DOC released two new internal communications products: a) a set of daily email alerts that keep colleagues informed on the main articles in top tier media that mention UNICEF and issues related to children and b) a monthly newsletter that includes the best examples of external communications and public advocacy campaigns from UNICEF teams around the globe. DOC continues producing the bi-weekly communications highlights that are also widely distributed internally.
We are also packaging information for senior managers in a straightforward and transparent way, allowing them to better understand UNICEF ́s presence in the global media landscape.
- Measurement is an art: do not try to apply the same model in all countries
Our aim is to have all country and National Committee offices implementing the M&E framework. This demands the creation of tailored solutions to respond to local needs and realities in each of the countries. We quickly realised it would be a mistake to try and use the same model everywhere. Each country has its own needs and local contexts and this includes the various human and financial capacities of each office.
DOC has since expanded the scope of its research to address the following questions:
- Who’s our audience?
To assist UNICEF’s digital production team in creating more relevant and engaging content for UNICEF’s global audience, the team has started to conduct literature reviews and evaluate the demographic profile of various social media platforms.
- What do these numbers mean?
To set meaningful benchmarks against which to evaluate the posts, UNICEF has been developing high-level global scorecards on a weekly and monthly basis. The new scorecards additionally start to evaluate emerging social media networks and platforms, beyond Facebook and Twitter, including Instagram, Pinterest, LinkedIn, etc.
- What learning can we share?
DOC has been working on knowledge sharing. Newsletters showcase the best initiatives as a means of providing concrete examples of success, so that other country offices can adopt and further tailor their own efforts without reinventing the wheel. This is in line with the organisational vision of efficiency and effectiveness.
Name of person entering: Arturo Romboli