Risk Score vs Confidence: How to Read a Scam Checker Result

A scam checker returns two numbers:

Is the message definitely a scam? Is 58% the chance the sender is guilty? Should you ignore the result because confidence is not high?

These numbers answer different questions.

In the anonymised IsThisSpam dataset, confirmed scam classifications clustered at much higher risk scores than unknown results. Unknown phone-only checks also had substantially lower confidence because the number alone did not provide enough evidence.

Understanding the distinction helps you avoid both panic and false reassurance.

What a risk score means

A risk score summarises how concerning the observed signals are.

Depending on the input, those signals may include:

A high risk score means the available evidence contains serious warning signs.

It should not be read as a legal finding or a precise probability that a person is a criminal.

What confidence means

Confidence describes how strongly the available evidence supports the classification.

Confidence can be affected by:

High confidence means the evidence points consistently in one direction.

Low confidence means the tool has important uncertainty, even if some signals are concerning.

The four useful combinations

High risk, high confidence

Several strong indicators agree.

Example: an impersonated account alert links to a lookalike login page and asks for credentials.

Action: Do not interact. Verify through the official service and report the message.

High risk, low confidence

The visible clues are serious, but the input is incomplete or conflicting.

Example: a short fragment demands urgent payment, but the sender and destination are missing.

Action: Treat the request cautiously, add context, and independently verify before doing anything irreversible.

Low risk, high confidence

The message appears consistent, expected and supported by reliable metadata.

Action: Ordinary caution still applies. High confidence is not a guarantee against a compromised account or a future change in the conversation.

Low risk, low confidence

Few warning signs are visible, but there is not enough information to say much.

A phone number with no message history is a common example.

Action: Do not translate "little evidence" into "safe". Add the full message or verify the sender separately.

Why unknown is not the middle of safe and scam

An unknown result is not necessarily a 50/50 verdict.

It often means the evidence is too thin to place the item reliably on the scale.

Consider a number-only check:

The correct output is uncertainty, not an invented risk claim.

Scores are not interchangeable across input types

A website, email address, full email, phone number and screenshot provide different evidence.

A risk score of 40 for a bare phone number may not mean the same thing as 40 for a complete email with links and headers. The result should be interpreted with the input type and limitations.

Good results explain:

How to improve confidence safely

You can often improve the assessment by adding:

Remove passwords, verification codes, account numbers and identity documents.

The guide to providing enough information for a scam check explains what to include.

Do not optimise for a green result

People sometimes remove suspicious details until a checker returns a safer verdict. That defeats the purpose.

Keep the evidence that explains the request:

Redact private values, not the scam structure.

A simple decision rule

When risk is high, pause regardless of confidence.

When confidence is low, gather more evidence regardless of risk.

When the action is irreversible, such as payment, credential sharing or software installation, verify through an independent channel even if the score looks reassuring.

A score is a decision aid. It is not permission to switch off judgement.

The best result explains its uncertainty

Trustworthy security tools should show their work.

They should distinguish between:

That transparency is more useful than a dramatic number without context.

Data note: This article is based on aggregate relationships between classifications, risk scores and confidence values in 13,547 public checks. Numeric examples in the article are illustrative. No source message, sender, score record or personal identifier has been reproduced.