Best evaluation on a small budget (<£10,000 or <$10,000)

Summary
In collaboration with Dansk Magisterforening, Infomedia created a list of the top-50 most cited experts in Danish media. This initiative sparked concrete actions towards diversity and inclusion, with media outlets and organizations increasing their focus on diversity and female experts. Platforms like “diverseeksperter.dk” were launched, showcasing female experts from various fields. Several academic institutions embraced the movement by compiling female-only expert lists. The project put representation on the agenda, highlighting the importance of diversity in the media. The great success of the list prompts its repetition next year, where the initiatives from organizations and media can be further analyzed.

Objective/Brief
Dansk Magisterforening (DM) is an academic trade union for over 68,000 bachelor’s, master’s, and doctoral students and graduates in the humanities, natural sciences, and social sciences. Every quarter, DM publishes “Akademikerbladet,” a journalistic magazine with editorial freedom read by over 40,000 members.

The Danish Association of Masters and PhDs (Dansk Magisterforening) also plays a crucial role in representing the interests of independent researchers. In a time marked by rising misinformation, polarization, health crises, and conflicts such as the war in Ukraine, Akademikerbladet wants to create awareness and highlight the importance of independent experts who make themselves available to the media to guide and provide the public and decision-makers with their specialized knowledge.

With the objective of designing a methodologically robust list of the 50 most quoted independent experts in the media, Akademikerbladet contacted Infomedia, seeking assistance with data collection and guidance on the methodology employed. Little did we know that this rather simple assignment turned out to have a massive impact and outcome on the state of affairs on diversity in Denmark. With a low budget, it started out quite simple but turned out as a game-changer in the diversity debate.

Strategy
As expert interviews and statements are particularly prominent in editorial media, we decided to cut some costs by focusing the analysis on editorial media and only using social media as a research tool. Also, the assignment was established on editorial media alone to address journalists and media editors’ responsibilities.

To achieve the key objective of creating a list of the most-cited experts in Danish media, three questions must be addressed:
1) How do we define an expert?
2) What is an independent expert? and
3) How do we ensure that the expert is not just mentioned in the media but speaks in an expert capacity?

We collaborated with Dansk Magisterforening and conducted research to establish the expert criteria. Drawing from public and academic discussions on expertise, we prioritized common academic staff titles like Ph.D, professor and lecturer.

Afterwards, we incorporated additional expert titles such as scientist, doctor and archaeologist. These titles were found via repeated research tests in editorial and social media content until no further new titles were observed.

To ensure expert independence, we excluded individuals with ties to commercial activities. After finalizing the expert titles, we initiated the data-collection process.

Planning steps:
1. Scoping the assignment with the client, reviewing academic research material and desktop research on related assignments and gains. Pre-test of the potential material in social and editorial media. Outline of existing gender proportion within academics.
2. Identifying the framework to be used and the potential output of the framework, hereby aligning objectives and possible outcome/impact.

Execution/Implementation
Our approach involved several implementation steps:
Implementation

1. Identifying expert titles to be included through research and repeated query searches in social and editorial media.
Concluding the identification of expert titles, the data collection showed over 350,000 annual articles containing an expert title.
2. Identifying the first 250 individuals who frequently appear with expert titles by techniques of automated Named
Entity Recognition (NER) and manually removing those without an expert role (desk research).
3. The preliminary list was operationalized by creating elaborate search queries on the individual names to ensure differentiated spelling in editorial media was caught.
4. To ensure that the experts were quoted as experts, rather than merely being mentioned, we made use of a well-tested citation search query that considers the person’s name with an expert title appears near various speech verbs such as “informs,” “uncovers,” “says,” “states,” etc. This query was developed through many years of testing.
5. After conducting individual searches using citation queries in the database with “expert article” it was possible to compile a top-50 list of the most cited experts in Danish media.
6. Another round of desk research ensured that each person did not have primary relations to commercial activities.
7. Reporting: The compiled list of the top-50 most cited experts allows filtering by gender, title, and institution. In collaboration with DM and Infomedia, Professor Rikke Andreassen from the University of Roskilde reviewed the list and selected potential viewpoints for DM to use.
8. Presentation/Advisory: The list was presented, leading to discussions during and after its development. Infomedia offered valuable insights into the methodology and provided ongoing assistance to address questions from media outlets and stakeholders regarding the published list.
9. DM pushed articles and posts on the findings from the expert list in several earned and owned media outputs.

Effectiveness of Assignment
Our analysis of media experts brought significant value to Akademikerbladet, generating attention among union members and major Danish media outlets like TV2, DR, and Politiken across social and editorial media. One key finding was the underrepresentation of women, with only 3 out of the top 50 cited experts being female. This prompted media outlets, organizations, and companies to acknowledge and respond to the analysis with actions and comments.
Outcome
Akademikerbladet’s expert list has generated significant attention and earned them a nomination for the prestigious Anders Bording Prize in Denmark, as their decision to exclusively feature female experts led to a nomination for the Ideation Award, recognizing their exceptional journalistic project/idea. “For us, the expert list provides value as it allows us to acquire extensive knowledge about the individuals we represent. We have put representation on the agenda. We have never experienced such a level of attention based on something we have
written and published. Multiple media outlets have interviewed several of our journalists due to the results of the list.” – Akademikerbladet
Impact
The analysis has driven major media outlets and organizations to increase their focus on diversity. In response to feedback from society, influential Danish media outlets are collectively prioritizing the inclusion of female experts. To enhance diversity and showcase a wider range of sources, “Kvindekompagniet” (Women’s Company) and “Female Leadership”, along with the Journalist Union and journalists from leading Danish media outlets, have launched “diverseeksperter.dk”. This platform brings together over 1,400 female experts from diverse fields, aiming to strengthen source diversity. In academia, both Roskilde University and Aalborg University have compiled an expert list consisting solely of female researchers, while the University of Copenhagen has asked their male researchers to identify female alternatives to themselves.

Evaluation
The expert list began as an idea to create a list of the most cited experts for the magazine’s members but ended up receiving significant media attention and triggering concrete initiatives among the country’s largest organizations and universities. It has been such a success that Akademikerbladet intends to follow up on the analysis: “We plan to monitor all the initiatives undertaken by the various stakeholders and look forward to compiling the list again to assess the state of society,”.

Infomedia has responded to the increasing focus on diversity by making additional investments in AI solutions that can effectively monitor diversity parameters, including age, gender and ethnicity of contributors and presenters in media content.

Infomedia Supporting Material