In the dance of progress, humans and machines create a powerful blend of intuition and algorithms, crafting a masterpiece of collective intelligence.

Ever dreamed of having the incredible power to discover the hidden emotions within news articles with astonishing precision? Well, brace yourself, because with the right technology and training, you can unleash a machine learning marvel boasting a jaw-dropping 90% accuracy in sentiment analysis! Yes, you read it correctly – 90%! And the thrill doesn’t stop there; with additional training, we can push those boundaries even further. Welcome to the realm of supervised learning, where we’ve harnessed the power to achieve something great. Are you ready to dive into the exhilarating world of cutting-edge technology and redefine the way we decode the sentiments embedded in news? The adventure begins now!

In a world of information overload, where data is being generated at an unprecedented pace, the traditional way of analyzing news articles falls short of meeting customer demands. While humans offer better results due to their wider understanding and adaptability, the pace at which they work impacts cost and time. Opting for machine output over human makes sense when real-time analysis is crucial or when budget constraints come into play. As a media measurement company, we recognize the importance of innovation and improvisation in today’s fast-paced environment.

Driven by our innate curiosity and commitment to embracing new technologies, we delved into the realms of Artificial Intelligence (AI) and Machine Learning (ML) to optimize and automate our services. In this quest, we developed ML models for real-time sentiment analysis of news articles. Why sentiment analysis? It empowers decision-makers, PR specialists, and communication managers to make informed decisions.

We approached this problem by two methods – 1) Classical machine learning models, 2) Large Language Models (LLMs) like BERT.  Both these methods have their own pros and cons. Classical machine learning models are easy to understand and implement. They tend to have shallow architectures and are often preferred in scenarios with smaller datasets, interpretability requirements, or when computational resources are limited. Experts also recommend that one should start their model training journey with less complex models. Whereas LLMs are a type of artificial intelligence model that is trained on massive amounts of data. They understand the context better as they are built like that only. People prefer LLMs when they are working with natural language as here context matters a lot, and hence, their prediction is better. Having said that LLMs have interpretability issues and fine-tuning them for specific tasks can be challenging.

Knowing their pros and cons, we collected data, labelled them, and passed them through various classical machine learning models – Naïve Bayes, Logistic Regression, Support Vector Machines, Decision Trees and Random Forests. The basis of using so many models lie in ‘No Free Lunch Theorem’. It states that there is no one model that works best for every situation. So, trying different models on the same data with different parameters may help you in finding out the best model suitable for your task. We compared the accuracy of the different models and picked the best one. The best model gave an accuracy of 85%. It was really a promising start to a problem that is consuming a lot of our time and effort. From the beginning, our goal was to reduce the analysis time and this result showed us that we can achieve the desired results with right technology and training if we willing to invest time in technology.

However, we didn’t stop there as we were getting the feeling that the current models are unable to capture context, and hence the accuracy was also not improving despite numerous training sessions. So, we turned to LLMs. The advent of LLMs and their better understanding of context was not just technological breakthroughs, but it also changed the way we were approaching natural language problems. Earlier, we had to train the models from the beginning to solve the specific task but now with the availability of pre-trained LLMs, the need to train the model on large amounts of data and the requirement of significant computing resources diminish a bit.

We picked Bidirectional Encoder Representations from Transformers (BERT) for this problem as BERT considers both left and right context in all layers of the neural network during pre-training. This bidirectional context understanding allows BERT to capture more nuanced and complex relationships in language. We fine-tuned the BERT model using our data and got an accuracy of 90% – a 5% increase over the classical machine learning model. Now, we see the context being captured in the output which is what we wanted.

As we stand at the crossroads of human intuition and artificial intelligence, the journey into the realm of sentiment analysis has been transformative. With 90% accuracy achieved through the synergy of classical machine learning models and the prowess of LLMs, we’ve not only solved the problem to an extent but also unearthed the potential to navigate the fast-paced currents of our data-rich world.

Yet, as we celebrate technological triumph, we must acknowledge the challenges that accompany this AI-driven solution. The quest for high-quality, representative data persists, and the intricacies of algorithmic complexity demand careful consideration. AI, while a powerful ally, complements but does not replace the nuanced expertise of human intuition.

The future of AI in sentiment analysis holds promise and potential. Natural Language Processing (NLP) can decipher the subtle nuances of language, unlocking the depths of sentiment within news articles. However, as we stride forward, let us not forget that the marriage of human creativity and AI innovation is the true catalyst for success. Striking a harmonious balance between leveraging the capabilities of AI and preserving the essence of human expertise is not just a choice but a necessity.

In the ever-evolving dance between man and machine, sentiment analysis becomes a testament to the possibilities that emerge when we harness technology not as a replacement, but as a partner in our quest for deeper understanding. So, let us march forward, embracing the promise of innovation, the challenges of the unknown, and the uncharted territories where the human mind and machine intelligence converge.

by Vikrant Tripathi, Impact Research & Measurement
@vikranttripathi

Vikrant Tripathi is an experienced media research analyst with 16 years of practical experience. He’s skilled at analysing and explaining media trends. Currently he’s been exploring Artificial Intelligence (AI) and Machine Learning (ML) to improve how we analyse media. Vikrant is dedicated to streamlining scoring techniques and revolutionising media analysis in this ever-evolving industry.

Read more in the AMEC Innovation Hub Series here

Share the social post with colleagues here