Data is the backbone of modern communications and PR strategies. Yet, the quality of that data remains a persistent challenge for industry professionals, especially as the reliance on AI and advanced analytics continues to grow. During AMEC Measurement Month 2024, a dedicated series of events focused on the best practices and emerging trends in communication measurement, I had the pleasure to host an expert panel with industry leaders to discuss these pressing issues. The webinar was a collaborative effort between AMEC’s Europe Chapter, Latin America Chapter, and Technology Hub, and featured a distinguished lineup of speakers:

• Frank Gregory, Social Intelligence Lead, Centre of Marketing Excellence, Nestlé USA
• Kyle Mason, Head of External Monitoring – Corporate Relations, Shell (AMEC Board Member)
• Rob Key, CEO, Converseon (Tech Hub, AMEC Board Member)
• Aldo Vietri, Research and Measurement Manager, Edelman DXI (AMEC Board Member)

The discussion explored the critical issues surrounding data quality, its impact on the PR and communications industry, and practical solutions for the future.

Why Data Quality Matters More Than Ever

Data quality has become a hot topic in the PR and communications industry, and for good reason. Data accuracy is not just a metric but a foundation of trust. For agencies like Edelman, the accuracy of data is crucial for building strong, credible client relationships. “If there is an error in the data, it can undermine even the most insightful analysis,” said Aldo Vietri, emphasising that this can have significant consequences on strategic decision-making.

Frank Gregory from Nestlé highlighted a different perspective, focusing on the internal challenges faced by brands. As the sole social intelligence lead supporting over 30 brands and 400 stakeholders in the U.S. market, Frank underscored the importance of democratising data tools: “I need the AI to be tight. The segmentation needs to be accurate.”

Without reliable data quality, it’s nearly impossible for him to manage the volume of insights required, especially when automated dashboards are used extensively across the organisation.

Kyle Mason from Shell added that the importance of data quality has intensified as AI and advanced analytics are more integrated into communication strategies. “The stakes are higher now,” Kyle noted, explaining that poor data quality can lead to flawed insights, eroding trust in data-driven decisions across the organisation.

The Hidden Truth About Data Quality

Rob Key from Converseon opened the webinar by underscoring the significance of finding “truth” within data in the AI era. He described data quality as one of the greatest challenges facing society and the PR industry in the coming decade. Rob’s presentation highlighted several alarming statistics:

• High operational costs: According to an IDC study, correcting for data quality issues consumes up to 40% of a data analyst’s time, significantly increasing operational costs.
• Low satisfaction with data quality: Only 8% of Chief Data Officers are satisfied with their organisation’s data quality, based on a recent study by MIT.
• Impact on AI initiatives: 46% of data leaders believe that poor data quality will hinder their ability to maximise the value of generative AI initiatives.

Poor data quality can have a cascading effect, leading to mistrust in the insights generated and ultimately resulting in flawed business decisions, warned Rob, a sentiment echoed by the rest of the panel.

Garbage In, Garbage Out: The Industry’s Commitment Problem

Rob posed a fundamental question to the industry: Do we have a duty and commitment to “get this right”? He emphasised the well-known adage, “Garbage in, garbage out,” highlighting that the quality of input data directly affects the reliability of AI-driven insights. In today’s data landscape, sentiment accuracy for social data averages only 65%, a figure that we all find deeply concerning.

Rob’s presentation revealed that several industry groups, including ESOMAR and MRII, have already recognised the severity of the problem and begun collaborating to address it. He advocated for a collective industry commitment to improving data quality, suggesting that companies need to prioritise accuracy and establish robust validation processes to ensure their data can be trusted.

Understanding the Scope of the Problem: Key Questions to Ask

There are four essential questions that organisations must ask to assess the quality of their data:

1. Is the data human and authentic? Rob pointed out that only 20-30% of category conversations on social media may be genuinely human, with the rest being noise or spam.
2. Are we translating the data accurately? He emphasised the importance of using effective metadata and analysis to ensure that sentiment scores and other classifications reflect the underlying data accurately.
3. Is the data meaningful and tethered to the real world? Data should be able to predict behaviour or outcomes. If it doesn’t, it risks becoming irrelevant to business decision-making.
4. Is the data trusted by decision-makers? Rob stressed that even if data quality is high, it must be presented in a way that inspires confidence among executives and stakeholders.

These foundational questions set the stage for a deeper exploration of the issues facing the industry, as discussed by the panel.

The Struggle with Sentiment Analysis

One of the central issues discussed was the challenge of sentiment analysis. Despite being a standard feature in most social listening and media monitoring tools, sentiment analysis is notoriously unreliable. The industry benchmark for sentiment accuracy is often as low as 65%, a figure compounded by upstream issues, such as irrelevant or misclassified data, which can further degrade the quality of insights derived.

The implications of this are far-reaching. For instance, Frank shared that at Nestlé, sentiment analysis plays a significant role in executive reporting. However, he often finds himself cautioning stakeholders about the limitations of sentiment metrics. “When there’s a small shift in sentiment, I have to remind executives not to overreact.”

He explained that minor changes in sentiment scores might not reflect genuine shifts in public opinion, especially given the current limitations of automated sentiment analysis tools.

Kyle added that a one-size-fits-all approach to sentiment analysis is inherently flawed.

“Generic models simply don’t work across all brands and sectors,” Mason argued.

He pointed out that the context, industry, and even the specific language used in discussions about a brand can vary widely, making it difficult for a standard sentiment model to capture nuances accurately.

Custom AI Models: The Way Forward?

Given the limitations of generic sentiment analysis models, the panel discussed the potential of custom AI models tailored to specific brands or industries. Rob argued that this approach could significantly enhance data accuracy.

“Custom models can be fine-tuned to deliver higher precision and more reliable insights.”

He pointed out that these models are built using domain-specific data, allowing them to understand industry jargon, context, and brand-specific language far better than off-the-shelf models.

Frank agreed, noting that while custom AI models require an upfront investment, the long-term benefits are substantial. “The cost of correcting poor data quality can be enormous,” he noted. He explained that at Nestlé, the time spent manually adjusting sentiment scores and verifying data can be significant, and custom models could help reduce this workload.

However, Kyle cautioned that while custom models are promising, they are not a panacea. “Developing a custom model requires sufficient data, expertise, and budget,” he pointed out, emphasising that not all brands may have the resources to build these models, especially smaller companies or those just beginning their data journey. Nevertheless, Kyle believes that as the technology matures and becomes more accessible, we are likely to see broader adoption.

Integrating Multiple Data Sources: A Complex Challenge

The conversation then shifted to the challenge of integrating multiple data sources. In today’s digital landscape, PR and communications professionals must analyse data from a variety of channels, including social media, mainstream news outlets, paid media, and proprietary research. Each of these sources comes with its own set of challenges, from data availability to varying levels of accuracy.

Frank shared that social platforms like TikTok and Reddit offer valuable insights but are difficult to analyse using traditional sentiment models. “Reddit, in particular, provides rich, long-form discussions, but the tools often default to a neutral sentiment because the content is so nuanced,” Frank said. He emphasised the need for sentence-level sentiment analysis, which could offer a more granular and accurate understanding of user sentiment.

Aldo added that changes in social media APIs have further complicated data integration efforts.

“Platforms are constantly adjusting their APIs, which can disrupt data collection,”

he noted, citing recent examples from LinkedIn and Facebook, where restrictions on data access have made it more difficult to gather comprehensive insights.

A Call for Industry-Wide Collaboration

A recurring theme throughout the discussion was the need for greater collaboration and industry-wide best practices to address the challenges of data quality. Rob suggested that the industry should come together to develop standardised metrics and validation processes.

“We need a unified approach to ensure that the data we use is not only accurate but also trusted by all stakeholders.”

Rob introduced the concept of “trusted AI,” which involves rigorous model validation, data observability, and continuous monitoring.

Kyle echoed this sentiment, noting that collaboration between different vendors and data providers could help improve data quality. “No single tool or provider can solve all the challenges we face,” he said. Kyle advocated for an open ecosystem where companies can integrate specialised models and data sources to create a more holistic analysis.

Frank added that education is also crucial.

“Internal stakeholders need to understand the limitations of the tools we use,”

he explained, highlighting the importance of training teams to interpret data correctly and not take automated insights at face value. This, Frank argued, is essential for building trust in the data and ensuring that it is used effectively in decision-making.

Looking Ahead: The Future of Data Quality in PR and Communications

The webinar concluded with a discussion on the future of data quality and the steps the industry must take to keep pace with evolving technology. The panellists agreed that while challenges remain, there is a clear path forward.

Rob was optimistic about the potential of new AI technologies, particularly custom language models, to revolutionise the industry. “We are at a turning point where AI can deliver unprecedented levels of accuracy,” Rob said. However, he stressed that this will only be possible if organisations commit to investing in the necessary tools and processes.

Frank and Kyle both highlighted the importance of continuous improvement. “Data quality is not a one-time fix,” Kyle pointed out. He emphasised that as the industry evolves, so too must our approaches to measuring and validating data.

Join the Conversation

This discussion is only the beginning. As part of AMEC Measurement Month, we invite all PR and communications professionals to explore more events and resources on the AMEC website and join us in advancing best practices in communication measurement.

By Raina Lazarova, AMEC Global Chair & Co-founder of Ruepoint