Skip to content
  • clinical AI
  • assessment

No algorithm detects suicide risk: why, with the numbers

Gesell Team12 min read

“Artificial intelligence can now predict suicide with remarkable accuracy.” The headline, from a 2017 Quartz piece, was reposted on the news pages of the Association for Psychological Science, the society whose journal published the study: “92% accurate in predicting whether someone will attempt suicide within the next week.” Wherever you practise, the answer is the same, and a 2025 meta-analysis gives it. Published in PLOS Medicine and pooling 53 machine-learning studies, it found these algorithms too inaccurate to be useful for screening or for deciding whom to treat first, and found that they label as low risk more than half of the people who later present to hospital for self-harm or die by suicide. This is not a flaw the next version will fix. It is arithmetic: when an event is rare, even a very good classifier produces mostly false alarms.

A scope note: there is no risk-assessment protocol here. Any disclosure of risk, with or without an instrument behind it, calls for an immediate clinical assessment under your own protocol and training. If you are reading this for yourself or someone close to you: in the United States, call or text the 988 Suicide and Crisis Lifeline; in Mexico, the Línea de la Vida answers on 800 911 2000, free, 24 hours a day, 365 days a year (CONASAMA); elsewhere, whichever service actually answers where you are. If someone is in immediate danger, call the local emergency number (911 in both countries).

What the study behind “92% accurate” measured

The claim travelled well beyond English. In Spanish, Psyciencia carried the story of an algorithm that predicts next week’s attempts «con una impresionante precisión del 92%» (“with an impressive 92% precision”), and Xataka went further, promising to predict «quién va a cometer suicido [sic]» (“who is going to take their own life”). The underlying paper is by Walsh, Ribeiro and Franklin in Clinical Psychological Science, and its abstract takes the headline apart.

The sample: 5,167 adults who already had a self-injury claim code in their records. Expert review found that 3,250 had made a suicide attempt (the cases) and 1,917 had self-injured without suicidal intent, accidentally, or unverifiably (the controls). The outcome was an attempt, not a death. And 63% of the sample were cases.

The metrics: AUC 0.84, precision 0.79, recall 0.95. The 92% figure is not in the abstract, and we could not check the full text, which is paywalled. Precision is the positive predictive value: of every 100 people the model flagged, 79 had made an attempt. That sounds strong until you remember that 63 in every 100 in that sample had made one. Labelling everyone “at risk”, with no model at all, would have been right 63 times out of 100.

The 2025 meta-analysis by Spittal and colleagues names the mechanism: in case-control studies the design sets the prevalence, often half and half, so “the high positive predictive values reported in some studies may be an artefact of the case-control design.”

Four numbers that are not the same number

Metric The question it answers Does it change with prevalence?
AUC (area under the curve) Pick one case and one non-case at random: how often does the model score the case higher? No
Sensitivity Of those who did it, how many were flagged? No
Specificity Of those who did not, how many were left unflagged? No
Positive predictive value (PPV) Of those flagged, how many did it? Yes, entirely

Your clinical question — will this person attempt? — is the last row: the only one that depends on how common the event is, and the one headlines leave out. It is the same logic by which a single PHQ-9 cutoff yields very different predictive values at different prevalences, as the guide to whether a scale is validated in Spanish shows.

The pattern is not confined to journalism. In Large and colleagues’ 2018 meta-analysis of high-risk categories among psychiatric inpatients, the AUC was 0.83 and the PPV 0.43%. Belsher and colleagues (JAMA Psychiatry, 2019) reviewed 64 models covering more than 14 million participants: global classification accuracy of 0.80 or more in most, and a positive predictive value for suicide death of 0.01 or less in most. Their verdict: good overall classification, but an accuracy for predicting a future event that “is near 0”.

The arithmetic, worked with Mexico’s rate

Mexico makes a clean worked example. Its statistics office, INEGI, counted 9,020 suicides in 2025, a rate of 6.9 per 100,000 inhabitants (registered deaths statistics, report of 10 November 2026). Substitute your own country’s rate; the structure does not change. Now imagine an algorithm nobody has published, with 90% sensitivity and 90% specificity — more generous than Large’s pooled 53.1% and 84.2% — applied to one million people for a year:

  1. At that rate, 69 of the million would die by suicide.
  2. The model flags 62 of them and misses 7.
  3. Of the remaining 999,931, it wrongly flags 10%: 99,993 people.
  4. It flags 100,055 people in all, of whom 62 die by suicide. PPV: 0.06%, roughly one in 1,610.

No tuning rescues that sum while the event is this rare. Carter and colleagues (2017), meta-analysing clinical risk scales, put it in six words: “Prevalence imposes a ceiling on PPV.”

Spittal ran the same calculation across populations, using a positive likelihood ratio (LR+, sensitivity divided by one minus specificity) of 10, just above the 6.5 to 9.9 found in real algorithms:

Population (the meta-analysis’s examples) Annual prevalence PPV at LR+ of 10
Suicide, general population 0.01% 0.10%
Suicide after discharge from inpatient psychiatric care 0.5% 4.8%
Self-harm, general population 1.5% 13.2%
Suicide after discharge for self-harm 2.0% 16.9%
Self-harm after discharge from psychiatric care 6.5% 41.0%
Self-harm after discharge for self-harm 16% 65.6%

Place your own practice in that table honestly. The only row that reaches two right calls in three describes self-harm, not suicide, among people discharged after hospital treatment for self-harm. Even at the fourth row’s prevalence, 30 clients would yield 0.6 suicide deaths a year: no practice accumulates enough cases to know whether a score works for it.

When the model really is good

Simon and colleagues (American Journal of Psychiatry, 2018) trained a model on records from seven US health systems: 2,960,929 patients, more than 10 million specialty mental health visits, and 313 predictors, from prior attempts to routinely administered depression questionnaires. Its C-statistic, equivalent to the AUC, was 0.861 for suicide death after a mental health visit.

Of patients scoring in the top 5%, 5.4% attempted suicide and 0.26% died by suicide within 90 days. Put the other way round: 94.6% of those flagged had no recorded attempt in that window, and 99.74% did not die by suicide. And that top 5% accounted for 48% of suicide deaths: the other half fell outside it.

The authors conclude, fairly, that the model substantially outperforms existing tools. Being far better than what came before and saying nothing useful about one particular person are not in contradiction.

The half you do not see: false negatives

Public debate fixes on false alarms; the more serious clinical harm sits on the other side. In Spittal’s summary, machine-learning algorithms “incorrectly classify more than half the people who subsequently present to hospital for self-harm or die by suicide as low risk.”

A “low risk” label produced by a system has an effect an intuition lacks: it gets written down, filed, and looks objective. Instruments behave the same way, as the guide to documenting a suicide risk assessment records: in one follow-up study, 71.6% of suicides occurred among patients who had answered “not at all” to item 9 on their most recent PHQ-9. A negative rules nothing out; a negative dressed as a calculation rules out even less.

Why the next version will not fix it

Three structural reasons, all set out by Spittal:

  1. Prevalence is the ceiling. The only theoretical scenarios with a clinically useful PPV would be an LR+ of 50 or more with an annual prevalence of at least 6.5%, or an LR+ of 20 or more with at least 16%.
  2. Raising one metric lowers the other. Most ways of reaching a high LR+ require specificity of 97% or more; sensitivity then falls and most cases end up classed as low risk.
  3. The useful window is short, and short windows have lower prevalence. Prediction matters most from 24 or 48 hours to 30 days after discharge. In their example, self-harm after psychiatric discharge drops from 6.5% over a year to 2.1% over four weeks, and at an LR+ of 20 the PPV lands at 30%.

None of this depends on the technique. A language model reading your notes or a client’s messages is one more classifier, and its PPV obeys the same formula (how a consumer chatbot responds to someone in crisis is a different problem, covered in the guide to clients using chatbots between sessions). Nor does more data fix it: Franklin and colleagues (2017), pooling 365 studies, found that predictive ability had not improved across 50 years, and Spittal, with 35 million records, found modern algorithms “no better than traditional risk assessment scales”. Of Spittal’s 53 studies, 30 came from the United States, one from Spain, and none from Latin America — worth knowing before a model is sold to you as validated for your clients.

Two health systems, two answers

England’s answer is the most explicit. NICE guideline NG225 on self-harm, published 7 September 2022, says in section 1.6:

“1.6.1 Do not use risk assessment tools and scales to predict future suicide or repetition of self-harm.”

“1.6.3 Do not use global risk stratification into low, medium or high risk to predict future suicide or repetition of self-harm.”

Recommendations 1.6.2 and 1.6.4 extend this to deciding who is offered treatment or discharged, and 1.6.6 asks instead for a risk formulation in every psychosocial assessment. The committee’s reasoning is the line to keep: the potential harms of stratification, “including the implication that risk is static instead of dynamic”, outweigh its benefits as a communication tool or an adjunct to clinical assessment.

The United States shows the other use. In the Veterans Health Administration’s REACH VET programme, an algorithm identifies each site’s top 0.1% risk tier so that designated clinicians re-evaluate treatment and reach out. Its evaluation (McCarthy and colleagues, 2021), covering 173,313 people, associated the programme with more new safety plans and fewer mental health admissions and suicide attempts, with no difference in suicide mortality. The difference that matters for you is scale and direction: a national programme uses a model to add care to a group, while a clinician with 30 clients would need it to say something about one person — exactly what the arithmetic does not allow.

Questions for anyone selling “risk detection”

If a vendor, an insurer or a clinic you work with proposes a system that “detects” or “predicts” suicide risk, five questions separate the evidence from the brochure:

“What is the positive predictive value, and at what prevalence was it measured? An AUC or ‘accuracy’ on its own does not help me.”

“Was the study a cohort or a case-control design?”

“What share of the attempts or suicides fell into the low-risk group?”

“In which population and which language was it validated? Is there data from clients like mine?”

“What does the system do with a negative result — does it write it into the record?”

The last one matters most. An extra alert costs you time; false reassurance filed as data can cost far more, and it sits in a record under your signature.

What a tool can honestly do

Without pretending to predict, a system can make sure that what a client said is not lost. An answer other than “not at all” on PHQ-9 item 9, or a yes on the Columbia scale (C-SSRS), predicts nothing: it triggers a clinical assessment, and that assessment — your formulation, your reasoning, your plan — is what gets documented.

Gesell does not calculate a probability of suicide or give clients a predictive risk score: at these base rates such a number would be mostly noise, and its silence false reassurance. If a client endorses PHQ-9 item 9 or a C-SSRS item, it raises an alert for you to assess; on the C-SSRS the alert also carries a triage level, from low to emergency, derived by a fixed rule from which items were endorsed. That is a label of the kind NICE says not to use for prediction: it orders how urgently you review, and estimates no probability. On the Profesional plan, if the AI reads something in your signed note that suggests risk, it proposes it, with an indicative severity, for you to review before accepting. Neither alert says who will act, and the absence of an alert does not mean low risk. What happens next remains, as the map of what AI can and cannot do in psychotherapy puts it, human work.

References

Share this article

Ready to spend less time on documentation?

gesell.ai turns your session notes into professional clinical documentation — SOAP, DAP, BIRP, or GIRP — and generates treatment plans with SMART goals.

Try gesell.ai

Have a question about your practice?

How to document a case, which format suits you, or how Gesell fits your practice: write to us and we will answer.

Write to us

About the author

Gesell Team

Clinical and product content written by the gesell.ai team together with certified clinical psychologists.

← Back to the blog