Why your sentiment monitoring is misleading you

Most sentiment scores measure the mood around your company’s name. They do not measure what anyone thinks of you. This article explains how to measure that instead, and what to ask your provider.

The same six posts about a fictional company read two ways. Legacy monitoring scores the mood around the name at +17. The TSC approach sets aside two posts that are not about the company, reads the other four for who is for or against it and why, and finds opinion split.

Communications wants to know whether a campaign landed or a crisis moved opinion. Corporate affairs reports reputation to the board and guards the licence to operate. Public affairs watches how legislators, regulators and campaigners line up before a decision.

Underneath, all three need the same answer.

What do the people who matter think of the organisation, and is that changing?

Most teams turn to sentiment analysis for that answer. Sentiment analysis uses a model to label text as positive, negative or neutral. Monitoring tools run it over every post and article that names you, then roll the labels up into a headline score.

Reading opinion at scale is hard. Many providers make bold claims about it. This article explains what a provider actually means when they say:

  • We built our sentiment model on a method from the early 2000s, and have mostly been iterating since.
  • Our sentiment model still runs on the pre-LLM standard.
  • We score the mood around your name, race results and share prices included.
  • We score the tone near each name. Direction and attribution are likely still a challenge.
  • This could be highly promising. Or it could mean a chat window wrapped around a legacy platform.
  • Measured on a test set. Ask how it scores your posts.

How is sentiment measured today?

Every sentiment tool makes two choices. It chooses how to read text, and it chooses what to score. Together they decide what the number means. Terms are defined as they come up, and gathered under Key terms .

How models read text has evolved substantially over the years:

  1. 19661Word lists

    A lexicon gives each word a fixed score. "Great" is +1, "disaster" −1, and the post scores the total. Easy to audit. Blind to context, so "not great" still counts as positive.

  2. 20022Classic machine learning

    A Naive Bayes or similar classifier learns from labelled examples which words lean positive or negative. It weighs words better than a fixed list. It still ignores word order.

  3. 20183Transformer classifiers

    A transformer such as BERT reads each word in light of its neighbours. Fine-tuned on labelled examples, it gives one label per text. The labels it learned come from general text, so it reads tone, not position.

  4. 20224Large language models

    A large language model (LLM) takes a question in plain language: who is this about, and what position does the speaker take?

    LLMs bring a step change in context and attribution. They also take real capability to run at scale, on a rigorous data pipeline. This is the TSC standard. Find out more

There is also wide variation in what models score:

Granularity

  • The full text. One score for the whole post or article.
  • The mention or snippet. The sentence or passage around your name.

Target

  • None. The overall tone of the text.
  • Each named entity. Tone toward each name. This is entity-level sentiment.
  • A position, by speaker. Who is for or against an organisation or issue, from posts that are about it and state an opinion. This is targeted sentiment.

What that means in practice:

Most media monitoring platforms now score each named entity with a transformer classifier. That handles a post that praises one company and attacks another. It still scores tone, not position, and does not ask who said it. Social listening tools mostly score the whole post. None we found reports accuracy measured on a client's own posts.

For decades, the way to make sentiment more accurate was to have analysts label coverage by hand. A provider that still leads with its analysts is likely still running those older models. Modern AI changes the analyst's job from labelling to governing: setting the questions and checking the answers.

Several of the largest monitoring tools aggressively market AI sentiment while their own pages describe pre-2022 models. They are quick to start and tell you how warm the coverage sounded. A tailored capability asks more of your team, and tells you who thinks what, and why.

How the main tools score sentiment

ToolWhat it scoresHowStrengths and limits
Media monitoring
MeltwaterThe article, and each named entity5BERT-based classifier. Stance and aspect models through a partner6Broad coverage, with a score per name. Tone, not stance, without the partner models.
CisionThe article, and each detected entity7“AI models”, type not statedBuilt for PR workflow. The editable score is still the article. Stance is upcoming8.
Signal AIEach entity9Transformer-based ML classifierBuilt for volume across many sources. No published accuracy.
Social listening
BrandwatchThe postPre-trained model, fine-tuned to label each post10Wide social coverage. Candid about accuracy: 60 to 75%.
TalkwalkerEach resultDeep learning, likely a fine-tuned classifier11Broad and quick to set up. One accuracy claim, from 2017.
PulsarThe post, from −1 to +1Words with explicit tone, per its help centre12Strong on audiences and culture. Irony gets through.
Specialised AI labelling
TSCEach snippet, both ways: who holds the view and who it is aboutA specialised AI model, built into the Genie stakeholder graphGeneral sentiment is just the start. The real value comes from tailoring. See how we use driver-level models to evaluate corporate reputation
From each provider’s own pages, read 17 November 2026 and checked again on 20 November 2026. Numbered notes give the source and the wording.

Why is my sentiment score wrong?

Generic sentiment measures warmth: how positive the words near your name are. Targeted sentiment measures direction: whether a named speaker is for or against you, and on what. Generic sentiment breaks wherever the two differ. Below, thirteen posts that mention Halden Energy, a fictional company, are scored both ways.

Generic versus targeted sentiment

Mood around Halden

+15

6 positive, 3 neutral and 4 negative across 13 posts.

Opinion of Halden

0

2 positive, 2 neutral and 2 negative across the 6 posts that judge Halden. The other 7 are left out.

  1. 1#HaldenGP was electric tonight. What a finish from Ruiz.PositiveLeft out: about the race
  2. 2Halden's chief executive spoke at the climate summit today.NeutralLeft out: report of fact
  3. 3WIN a Halden fuel card!! Click the linkPositiveLeft out: giveaway
  4. 4Oh great, another Halden net zero pledge. Can’t wait.Anonymous accountPositiveNegative
  5. 5Halden’s new refinery brings 2,000 jobs. Nobody is asking about the water.Local campaigner. Neutral overall, split by driverPositiveNeutral
  6. 6Shame on Rival Energy for dropping its climate targets. Halden at least kept theirs.NGO analystNegativePositive
  7. 7Halden shares fell 4% after the quarterly update.NegativeLeft out: report of fact
  8. 8Loved the Halden-sponsored jazz night at the harbour. Best gig this year.PositiveLeft out: about the event
  9. 9Halden reports record quarterly profit.PositiveLeft out: report of fact
  10. 10Halden fixed the pipeline leak within a day and told residents straight away. Credit where due.ResidentNegativePositive
  11. 11Terrible weekend for the Halden team. The car is a disaster.NegativeLeft out: about the racing team
  12. 12Halden says it consulted the community. We were never asked.Residents' associationNeutralNegative
  13. 13Halden’s transition plan is credible on gas and thin on renewables. Jury’s out.Equity analystNeutralNeutral
Net sentiment is positive minus negative, as a share of what is scored. Posts chosen to show each failure mode. Not a sample.

Common failure modes account for the gap.

Proximity is scored as opinion

A post enters scope because it carries your name, your handle or a hashtag. That is proximity, and it says nothing about what the writer thinks of you. The tool then scores the tone of the whole post. Delight at a race or a concert next to your name reads as delight with you. Gloom about a losing car reads as gloom about you. .

One post holds several views

A post can praise you and attack a competitor in the same sentence. One score per post averages the two, or gives both companies the same verdict. .

Facts are scored as feelings

News carries words like loss, leak, closure and record. A neutral report of falling shares scores negative. A report of record profit scores positive. Neither is anyone's opinion. Loaded words mislead opinions too. A resident who credits Halden for fixing a leak scores negative on the word leak. .

Irony and understatement read straight

Every word that carries tone in "Oh great, another net zero pledge. Can't wait." is positive. The view is negative. Sarcasm, slang and posts that switch language defeat any method that scores words rather than meaning. Criticism made in neutral words scores neutral or positive. .

Noise is counted

Spam, bots, giveaways and syndicated copies of one article each add a data point. A competition to win a fuel card is enthusiastic. It tells you nothing. .

What does a misleading score cost?

A misleading score leads to wrong decisions, and nobody can see why.

Example

Halden sponsors a racing team. Over a race weekend, its sentiment among motorsport fans jumps.

But read the posts. Fans are thrilled by the race. Some mock the team in posts that read as praise. None of it is a view of Halden.

So the board hears that fans love the sponsorship. And they miss the insights that matter.

Wrong decisions

A sponsorship is renewed, a campaign is judged a success, an issue is judged contained. Each rests on a number that measured something else.

False comfort

Proximity material tends to run positive, because people post most about things they enjoy: the race, the match, the launch. A score built on it runs warm. A team reading it is reassured when it should be watching.

No explanation

A score built on posts about something else cannot say who holds a view, or why. When it swings, nobody can trace the swing to a group, an event or an argument.

No trust

Once a team has been caught out, someone checks the output by hand before anyone acts on it. The measure now adds work.

How do you measure stakeholder sentiment accurately?

Start with a precise definition of what you are measuring.

Definition

Stakeholder sentiment is holds , read from .

Surveys answer the same question for a sample, a few times a year. Reading what people say in public answers it continuously. It also reaches regulators, campaigners and legislators, whom panels rarely reach.

Build the measure from the definition, then check it.

Name the targets

The organisation, its peers, its executives, its assets and the issues tied to them. Score each one separately.

One post, two targets

@haldenwatch · Energy policy analyst, NGO
Shame on Rival Energy for dropping its climate targets. Halden at least kept theirs.
Rival Energy
Negative
Halden
Negative

Both companies inherit one label. Halden is marked down for praise.

Rival Energy
Negative
Halden
Positive
Speaker
NGO analyst

Name the stakeholders

Who said it matters as much as what was said. A regulator's criticism and an anonymous account's carry different weight. Group speakers before you count them.

Choose the sources the group uses

Regulators speak in consultations and speeches. Investors speak in analyst notes. Communities speak on local news and social media. Capture what each group reads and writes, then remove spam, bots, giveaways and duplicate copies before anything is scored.

Separate fact from opinion

A report of falling shares is logged as an event. Only a judgement counts toward sentiment.

Read stance as well as tone

Tone is the mood of the words. Stance is the position the speaker takes toward the target. They often disagree.

Tone versus stance

“Love how Halden keeps ignoring the community.”

Generic sentiment reads

Positive

Tone

Targeted sentiment detects

Opposed

Stance toward Halden

Score by driver

One direction score can still hide a split. Post 5 praises the jobs and questions the water, so it scores neutral overall. Scored by driver, it counts once for the economy and once against the environment. Built up across posts, the drivers form a scorecard. Many teams map it to a reputation framework they already report against.

Direction by driver, Halden

Environment1 positive, 3 negative
Community1 positive, 1 negative
Economy and strategy2 positive
The same six statements as above, scored by what each one is about. Overall, Halden nets to zero. By driver, it is credited on jobs and gas, and criticised on climate and water.

Measure accuracy

Check each classifier every month against a sample of posts graded by people, in each language. Report precision and recall for each one. A provider who cannot give you those numbers cannot tell you how often the score is wrong. Language models cost more to run than classifiers, and they can be wrong with confidence. This step is what keeps them honest.

The result is a smaller number that means something.

From mentions to opinion

Mentions100Everything that matched the organisation's names, handles and hashtags.
Without noise6238 removed: spam, bots, giveaways and duplicate copies of one article.
About the organisation or its issues3131 removed: posts about a race, a match, an event or a rival, where the organisation's name sits nearby.
Carrying an opinion about it1516 removed: reports of fact, such as results, appointments and share moves. What is left carries a judgement, and each one has a speaker.
MediaCampaigners and NGOsInvestorsCommunities

Everything that matched the organisation's names, handles and hashtags.

Illustrative index, mentions = 100. The last row is split by stakeholder group.

The aim is accuracy, whichever way the reading points. The rest of the conversation still matters for awareness and reach. It does not belong in a measure of what people think.

TSC builds stakeholder intelligence this way. All of its sentiment runs on modern AI. Each step above is a separate question the AI answers, about the client's own organisation, stakeholders and issues. Analysts govern it. They set the questions and check the answers against samples people have graded, before results reach a client. Any team can ask its provider for the same.

What should you ask your provider?

Ask for evidence on your own data. Each question below needs more than a slide to answer.

1Show us the precision and recall of our sentiment, on a sample of our own posts graded by a person, in each language we track. When was it last measured?

A good answerA figure for each language, the sample size, a date, and the graded sample on request.

A red flagAn accuracy figure from a public benchmark, or a single percentage with no sample behind it.

2What exactly is scored: the whole post, the passage around our name, or each statement about us?

A good answerEach statement, with the target named.

A red flagThe post. Or an answer about the dashboard rather than the method.

3When a post criticises a competitor and praises us, what score does each of us get? Show us a real example from our data.

A good answerTwo scores, one for each company, shown on a real post.

A red flagOne score for the post, or a promise to look into it.

4What share of the posts in our score carry an opinion about us, rather than a mention?

A good answerA share, and how it was measured.

A red flagNo figure, or all of them.

5Does a news report of fact, such as falling shares, move our sentiment score?

A good answerNo. Facts are logged as events and kept out of the score, unless the report itself takes a view.

A red flagYes, every article that names us is scored.

6Does a language model score our sentiment, or only summarise coverage after a machine-learning classifier has scored it?

A good answerA plain account of which model does which job.

A red flagWe use AI, with no word on where.

7Can you trace last month's biggest swing to the posts and the people behind it?

A good answerThe posts, the speakers grouped by stakeholder type, and the event that set it off.

A red flagA word cloud.

Take it with you

The cheat sheet

The questions, what a good answer sounds like, and the key terms, on one page.

Download the cheat sheet

Appendices

AKey terms

Word list (lexicon)
A dictionary that gives each word a fixed tone score. A text's score is the total of its words. Simple to audit, and blind to context.
Driver
What a statement is about, such as the environment, the community or the economy. Scoring by driver splits one mixed post into its parts and builds a scorecard over time.
Deep learning
Machine learning with neural networks of many layers. Includes transformers such as BERT.
Naive Bayes classifier
A classic machine learning method. It learns from labelled examples how likely each word is to appear in positive or negative text, then estimates the odds for a new text. It treats words independently, so word order is lost.
Transformer
A type of AI model, introduced in 2017, that reads each word in light of the words around it. Sentiment tools fine-tune one on labelled examples to give each text a single label.
BERT
A transformer model published by Google in 2018, and the base of many sentiment classifiers since. It learns language from general text, then is fine-tuned to label texts for one task.
Fine-tuning
Further training of a general model on labelled examples for one task. The examples decide what it learns to notice. A model fine-tuned on product reviews learns how reviewers express tone, not how a campaigner criticises a company.
Large language model
A very large transformer, trained on vast amounts of text, that follows instructions in plain language. It can be asked who a statement is about and what position the speaker takes, without being retrained for each task.
Stakeholder sentiment
The opinion a defined group holds about an issue or organisation, read from what its members say in public.
Generic sentiment
Sentiment scored on a whole post, or the passage around a name, with one label per text.
Entity-level sentiment
Tone scored separately toward each name in a text. Also called aspect-based sentiment when it scores a product's features. It does not ask whether the text is about that name, or who said it.
Targeted sentiment
The position a speaker takes toward a named organisation or issue, read only from posts that are about it and state an opinion, with the speaker recorded.
Stance
The position a speaker takes toward a target: support, opposition or neither. It can differ from the tone of the words.
Proximity
When a post is in scope only because it sits near a company's name, handle or hashtag, without saying anything about the company.
Precision and recall
Precision is the share of a classifier's labels that are right. Recall is the share of the true cases it finds. Both are measured against posts graded by a person.

BFrequently asked questions

How accurate is sentiment analysis?

It depends on what is scored and what it is checked against. Accuracy quoted on general benchmark data says little about posts on energy policy or a licence to operate. Ask for precision and recall on your own data, graded by a person, in each language.

What does transformer-based sentiment mean?

A transformer is a type of AI model that reads each word in light of the words around it. Transformer-based sentiment fine-tunes such a model, often BERT, to label a text positive, negative or neutral. It reads phrasing better than a word list. Pointed at each named entity, it scores tone toward that name. It still cannot tell whether the post is about the company at all.

What is targeted sentiment analysis?

Targeted sentiment reads the position a speaker takes toward a named organisation or issue. It keeps only posts that are about the organisation and state an opinion, and it records who said each one. Entity-level sentiment is a partial form. It scores tone toward each name, but not stance, the speaker, or whether the post is about that name.

What is the difference between sentiment and stance?

Tone is the mood of the words. Stance is the position the speaker takes toward a target. Most tools score tone and call it sentiment. Stakeholder sentiment, as defined here, is closer to stance. "Love how they keep ignoring us" is positive in tone and opposed in stance.

Can sentiment analysis detect sarcasm?

Word lists cannot. Trained classifiers catch some of it. Language models catch more when asked to read for meaning. All three should be checked on a graded sample.

Why is my sentiment score so positive?

Usually because it scores posts that mention you rather than posts about you. Fans, customers and attendees post about events and products they enjoy, and that tone is credited to the company named beside them.

CSources

  1. 1Stone, Dunphy, Smith and Ogilvie, The General Inquirer: A Computer Approach to Content Analysis, MIT Press, 1966. The first widely used dictionary of word tone. Jurafsky, sentiment lexicons, Stanford (web.stanford.edu)
  2. 2Pang, Lee and Vaithyanathan, “Thumbs up? Sentiment classification using machine learning techniques”, EMNLP 2002. ACL Anthology (aclanthology.org)
  3. 3Devlin, Chang, Lee and Toutanova, “BERT: Pre-training of deep bidirectional transformers for language understanding”, 2018. arXiv (arxiv.org)
  4. 4OpenAI released ChatGPT on 30 November 2022. History.com (history.com)
  5. 5Meltwater describes its entity sentiment as a BERT encoder with a classifier, and reports that it “reduced inference costs by 45.5% while also improving accuracy by 3.02%”. No absolute accuracy is given. April 2026. Meltwater engineering blog (underthehood.meltwater.com)
  6. 6Meltwater offers entity and aspect sentiment and stance detection through Converseon’s models. The only figure given is a “3x lift in AI visibility tracking accuracy” for stance. June 2026. Meltwater blog (meltwater.com)
  7. 7CisionOne rates the article on a five-point scale “which you can manually adjust”. Entity scores are view-only and appear only where entities are detected. Ratings come from “our AI models”, type not stated. CisionOne help centre (help.cision.one)
  8. 8“Stance is an upcoming feature in the new CisionOne platform.” Undated. Cision (cision.co.uk)
  9. 9Signal AI “uses transformer-based AI to determine sentiment at the entity level”. The page gives no accuracy figures for sentiment. Signal AI (signal-ai.com)
  10. 10Brandwatch: “Brandwatch customers can typically expect to see an average accuracy of around 60-75%”, and “two humans only agree on the sentiment of something around 80% of the time”. Sarcasm is “harder”. January 2022. Brandwatch blog (brandwatch.com)
  11. 11Talkwalker claimed “90% accuracy” in September 2017, without naming a test set. The same release says its neural networks “are already able to understand basic sarcasm and irony”. Press release, 2017 (prnewswire.com)
  12. 12Pulsar’s help centre describes looking for words that “carry an explicit positive or negative meaning”, notes that sarcasm, irony and slang can be missed, and returns neutral when a meaning “cannot be assigned with certainty”. February 2025. Pulsar help centre (intercom.help)

Download the PDF.

Leave your work email and organisation, and the file is yours.

We use these details only to reply to you. Privacy note. You can also write to info@tsc.ai.