AMEC’S INTEGRATED EVALUATION FRAMEWORK
Welcome to the resource centre for AMEC’s new Integrated Evaluation Framework. This special section of AMEC’s website has been put together to answer the most common challenges and pressures faced by communicators today as they look to prove their value in a rapidly evolving media landscape.
Many of the evaluation methods and techniques that the industry took for granted for so many years no longer suffice. As organisational silos are coming down, PR professionals are being asked to work across more communication disciplines and then measure their effectiveness and prove value in a meaningful manner. Work must encompass paid, earned, shared and owned media. To be effective at PR, we now need to plan and measure our communications in a truly integrated manner, across each of these dimensions, beyond counting activity, to demonstrating effect.
In today’s unforgiving era of accountability, it is vital to move beyond measuring just the content or ‘media outputs’ that largely sufficed for the last 20 years. Doing this only counts elements of our activity. Activity without meaningful outcomes is irrelevant. PR professionals can’t afford to run the risk of being seen as activity-based ‘busy fools’. Instead, we must advance beyond activity-focussed metrics and look to show how PR and comms have driven the objectives that matter to the organisation.
AMEC’s Integrated Evaluation Framework shows how to do this, helps PR professionals to plan appropriately, and supports them to tell a meaningful measurement story. It shows how to ‘operationalise’ the Barcelona Principles and turn the Principles into action, and to prove the value of PR in a meaningful and credible manner.
The interactive element of the Integrated Evaluation Framework guides you through the process from aligning objectives, establishing benchmarks, creating a plan, setting targets and then measuring the outputs, out-takes, outcomes and impact of your work. When used to full effect, the framework is both a PR planning tool , as well as an evaluation tool. Work through it in advance of a campaign with your team and use it to define what good looks like at each step of the process. Establish your benchmarks, set and record your KPIs and your targets, and then at a later stage revisit with results to make sure that you are measuring what matters.
As you work through the interactive version of the framework, it provides additional information and suggests potential approaches and metrics that might be appropriate for you to consider. Don’t miss the AMEC Taxonomy which is so helpful for this process. Of course, this cannot be an exhaustive list and the Framework can’t provide the numbers for you. You will still need to source relevant data to input into the Framework.
AMEC’s Integrated Evaluation Framework provides a consistent and credible approach that works for organisations of all sizes, any desired objective, and with any size of budget. Today, organisations from the world’s largest multinationals to the smallest non-profits are using the framework to plan and measure their communications effectiveness. Anyone can use it, it is free and non-proprietary, allowing any organisation, be it university, in-house department, PR agency or measurement company to benefit from its approach. It has been translated into 22 languages and is taught in academia on multiple PR and communications courses around the world.
The Integrated Evaluation Framework replaces both AMEC’s previous Valid Metrics and Social Media Measurement Frameworks. The industry now has one integrated approach to the measurement and evaluation challenges of today.
In addition to the framework itself, this site has a number of articles and additional resources to bring measurement to life. These include an evaluation ‘taxonomy’ which lists and explains for each step of the process the key steps required, the metrics and milestones and the methods that should be considered. This is one of AMEC’s greatest resources so don’t miss it – you will find it under the ‘Supporting Material’ dropdown menu. My thanks to Distinguished Professor Jim Macnamara of the University of Technology Sydney for creating the taxonomy and sharing it generously with the AMEC community, as well as his leadership of our Academic Advisory Group and liaising with other trade associations to ensure that we can all speak with a common consistent voice. Finally, don’t miss this short webinar with a practical guide on how to use the framework.
It has been a great privilege for me to lead the task force that worked so hard to create and develop the framework. From an original idea conceived between me and great friend of AMEC, Don Bartholomew, a world-class educational resource has been created. I have a sense of enormous pride in how widely it has been adopted across the world. More than 2,000 organisations of all sizes now base their approach to planning, measurement and evaluation of communications on the Integrated Evaluation Framework. Its development was the result of a lot of hard work by a lot of great people in the team and my sincere thanks are due to them all.
Richard Bagnall, Co-founder, CommsClarity Consulting
Richard Bagnall, Co-founder, CommsClarity Consulting
Board Director AMEC, Integrated Evaluation Framework team lead,
INTRODUCTION TO THE NEW INTERACTIVE TOOL
Since our inception, AMEC has been championing a better way of measuring communications with its education initiatives and measurement frameworks. These have helped many agencies, clients and organisations improve their measurement and evaluation. But we needed to make a new one fit for the modern complexities of PR and communications. And while we were about it, we wanted to make a new framework that was user friendly, intuitive and engaging.
So a working group spanning agency, in-house, market research and academia designed and built the new Interactive Integrated Evaluation Framework.
We wanted to make something that took users on a clear measurement journey from planning and setting SMART objectives, defining success, setting targets through to implementation and the measurement and evaluation itself. Importantly, we wanted to find a mechanism that would help credible and meaningful measurement pervade the industry.
THE NEW INTERACTIVE TOOL
We have built it in a tile format for a clear step-by-step process. Each tile is numbered so you know where you are in the process. The journey takes you from organisation objectives and communication objectives all the way to business impact.
When you click each tile, you get the space to fill in your work. To guide you, within each tile is an (i) icon, which when clicked provides a pop-up with additional information and an explanation of the section. Furthermore, inside the pop-up is a link to a measurement taxonomy – giving you even more information on what types of measures to include.
Below we have provided some basic information on how to use the framework and how it can be put at the heart of the planning, research and evaluation process.
Like all good measurement, it should start with clear organizational objectives. These can come in many different forms, whether they be awareness, advocacy, adoption or demand related. Following on from organizational objectives, is communication objectives. These should reflect and mirror the organizational objectives. Remember, the difference between an objective and a goal is that an objective has a measure of impact (e.g. 20% increase in brand awareness), compared to a goal that is an aspiration (e.g. increase brand awareness).
In outputs, this covers the core measures across PESO. So for example what was the reach of the paid advertising, how many visitors to the website, how many posts, tweets or retweets, how many people attended the event, and how many potential readers of the media coverage. This is quantitative and qualitative measures of outputs.
This section covers two important areas. Firstly, to define the target audiences of the campaign. Second, is the strategic plan and other inputs such as describing some of the situation analysis, resources required and budgets.
This section is outlining what activities were carried out, any testing or research, content production etc. Importantly, the tool recognizes the importance of paid, earned, shared and owned (PESO) and gives users the ability to tag accordingly.
In outtakes, this refers to the response and reactions of your target audiences to the activity. How attentive were they to the content, what was their recall, how well understood is the topic, did the audience engage with the content or did the audience subscribe to more information.
In outcomes, this measures the effect of the communications on the target audience. Have the target audience increased understanding, has it changed their attitude to the topic, has it increased trust and/or preference, has it had an impact on the intention to do something (e.g. trial, subscribe, register) or increased online advocacy.
This final section is where impact on the organizational objectives is evaluated. So here the tool is looking to cover reputation improvement, relationships improved or established, increase in sales or donations, change in policy, or improved social change. This is a clear demonstration of business outcome and link to organizational objectives.
Now what? Once completed click the SUBMIT buttons and you can convert your work into PDF for sharing or using in meetings or presentations. If you want to go back and EDIT, then click the red button at the top to make changes. Then click SUBMIT again. Save the PDF to your computer.