The future of personalized TMS may depend on proving that sophisticated prediction tools offer something simpler methods cannot. A new study in the Journal of Affective Disorders found that more complex machine learning approaches did not significantly improve TMS remission prediction over a model using three clinical variables.
The finding challenges an assumption behind some medical AI development: adding information and computational complexity will necessarily produce more useful predictions. Here, the simpler benchmark proved difficult to beat.
The Clinical Question Behind The Algorithm
Predicting remission could help clinicians discuss the likelihood of minimal remaining depressive symptoms after treatment. However, estimating an outcome is different from determining whether a treatment is appropriate for an individual.
An earlier study in Translational Psychiatry illustrates the challenge. Its machine learning models distinguished TMS remitters from nonremitters, but the remission model was not well calibrated. Ranking patients by likelihood of remission and assigning accurate probabilities are separate tasks, both relevant to clinical usefulness.
Testing Whether Complexity Adds Value
Researchers retrospectively examined 263 adults with treatment-resistant depression who received repetitive transcranial magnetic stimulation. Their reference model used age, sex, and baseline Montgomery–Åsberg Depression Rating Scale, or MADRS, scores.
They compared this logistic regression model with tuned regression and SuperLearner ensembles using increasingly detailed information, including questionnaires, symptom dimensions, and individual items. Remission meant a follow-up MADRS score of 10 or lower; 71 participants, or 27%, met that threshold.
Evaluation used 10 repetitions of stratified 10-fold cross-validation. This repeatedly separates observations used for model development from those used for evaluation, providing an internal test of performance.
TMS Remission Prediction Shows No Clear Improvement
Across 12 configurations, average ROC-AUC values ranged from 0.685 to 0.711. The three-variable model achieved 0.707, while a MADRS-only benchmark achieved 0.708.
None of eight paired comparisons demonstrated a statistically significant advantage for richer or ensemble strategies after correction for multiple testing. Every confidence interval for the differences included zero.
ROC-AUC describes how well a model ranks people who experience an outcome above those who do not. A value near 0.71 does not mean that a system correctly predicts remission for 71% of patients. It also does not establish that its probability estimates are suitable for treatment decisions.
The result therefore concerns an unproven improvement in discrimination, rather than proof that the competing approaches are identical.
Why More Inputs May Not Produce Better Predictions
Logistic regression estimates how selected variables relate to the odds of an outcome. Ensembles combine predictions from multiple models, potentially capturing relationships that a simpler approach misses.
That flexibility creates an opportunity, not a guarantee. Additional measurements may overlap with information already available, contribute little independent signal, or introduce patterns that do not persist in new patients.
These are possible explanations for the findings, rather than biological mechanisms established by this study. The analysis does not show why a particular patient remits or how stimulation changes that person’s brain.
A Stronger Benchmark For Medical AI
The study’s distinctive contribution is its direct comparison against a predefined, accessible reference model. Beating chance answers whether a model detects predictive information. Beating a practical alternative answers whether its added complexity earns a place.
For developers, that distinction supports reporting incremental gains alongside headline performance. For clinics, it raises questions about whether extra questionnaires, software requirements, or implementation costs produce a meaningful benefit.
These are implications of the comparison, rather than evidence that the model is ready for deployment.
Independent Validation Remains The Next Step
Internal cross-validation cannot establish performance in other clinics or patient populations. The authors explicitly caution that their findings establish neither equivalence nor transportability.
The next test is whether predictions remain reliable in independent settings and improve decisions prospectively. For now, this study supports a measured standard for AI-assisted TMS: demonstrate useful gains over simple clinical information before making complexity part of care.
Citations
- Elkrief L, et al. Predicting Remission Following Repetitive Transcranial Magnetic Stimulation In Treatment-Resistant Depression: A Comparison Of Parsimonious And Ensemble Machine Learning Models. Journal of Affective Disorders. Published online October 8, 2026.
- Benster LL, et al. Predictive Modeling Of Response To Repetitive Transcranial Magnetic Stimulation In Treatment-Resistant Depression. Translational Psychiatry. 2025;15:160.
Explore more at https://www.interventionalpsychiatry.org/
