A wide range of models of PR and communication evaluation exist using a wide range of terms including inputs, outputs, outtakes, outflows, outgrowths, effects, results, and impact. An even wider range of metrics and methods for evaluation are proposed for each stage. The field is confusing for many practitioners.
This page presents a taxonomy of evaluation tailored to strategic public communication – a taxonomy being a mapping of a field to produce a categorisation of concepts and terms – in short, to show where things go and where they fit in relation to each other. This taxonomy identifies:
- The major stages of communication (such as inputs, outputs, etc.);
- The key steps involved in each stage (such as distribution of information, reception by audiences, etc.);
- Examples of metrics and milestones that can be generated or identified as part of evaluation at each stage; and
- The most commonly used methods for generating these metrics and milestones.
A taxonomy is not the same as a model, as a taxonomy attempts to list ALL the main concepts, terms, metrics, methods, etc. in a field, while a model is an illustration of a specific program or activity to be applied in practice. However, models should be based on the concepts and methods identified as legitimate in the field and apply them appropriately.
An important benefit of a taxonomy is that it puts concepts, metrics, methods, etc. in their right place – e.g., it avoids output metrics being confused with outcome metrics. The authors of the widely used PR text Effective Public Relations, Cutlip, Center and Broom have noted repeatedly in editions from 1985 to the late 2000s that “the common error in program evaluation is substituting measures from one level for those at another level” (1985, p. 295; 1994, p. 44; Broom, 2009, p. 358). Emeritus Professor of Public Relations Jim Grunig similarly says that many practitioners use “a metric gathered at one level of analysis to show an outcome at a higher level of analysis” (2008, p. 89). PR and strategic communication is not alone in this. The widely-used University of Wisconsin (UWEX) guide to program logic models for evaluation says, for example, “people often struggle with the difference between outputs and outcomes” (Taylor-Power & Henert, 2008, p. 19).
No taxonomy is ever complete, but the taxonomy presented here draws on a wide range of research studies to be as comprehensive as possible (see ‘Introduction to the AMEC Integrated Framework for Evaluation’ for details of the origin and basis of this taxonomy and the framework itself).
NOTES FOR USING THIS TAXONOMY
- The key steps, metrics and milestones, and methods are not exhaustive, and not all are required in every program. They are indicative of common and typical approaches to evaluation of public communication such as advertising, public relations, marketing communication, etc. Practitioners should choose relevant metrics and milestones and methods, ideally selecting at least one at each stage.
- The arrangement of inputs, activities, outputs, etc. should not be interpreted as a simple linear process. Feedback from each stage should be applied to adjust, fine-tune, and change strategy and tactics if necessary. Evaluation is an iterative process.
- Not all evaluation can show impact, particularly when evaluation is undertaken within a relatively short time period following communication. Impact often occurs several years ‘downstream’ of communication. Also, the objectives of some public communication is to create awareness (an outtake or short-term outcome) or build trust (an intermediate outcome)[xii]. However, as a general rule, evaluation should report well beyond outputs and outtakes. Evaluation should identify and report outcomes at a minimum and, when possible, impact.
- An important feature of this taxonomy is that impact includes organizational, stakeholder, and societal impact/outcomes. This aligns with program evaluation theory and program logic models (e.g., Kellogg Foundation, 1998/2004; Taylor-Power & Henert, 2008; Wholey, 1979; Wholey, Hatry, & Newcomer, 2010) and with Excellence Theory of PR, which calls for evaluation to be conducted at (a) program level; (2) functional level (e.g., department or unit); (3) organizational level; and (4) societal level (L. Grunig, J. Grunig & Dozier, 2002, pp. 91–92).