The STAR*D scandal is the gap between what the largest antidepressant effectiveness trial reported and what its own protocol supported. The 2006 summary paper reported a 67% cumulative remission rate after up to four treatment steps.¹ A 2023 reanalysis of the patient-level data, using the protocol’s blinded Hamilton scale and eligibility rules, found 35%.² The trial had no placebo group.

This piece argues that STAR*D did not fail because research is the wrong tool for understanding depression. It failed at outcome transparency. The summary report changed the outcome measure, let back in patients that an earlier paper had excluded, and put a theoretical number in the abstract as though it had happened. The full extent of the problem was documented only because the patient-level data had been archived with NIMH, where outside researchers could apply for access, and someone checked. Clinicians who take STAR*D as proof that intuition beats research learn the wrong lesson, because therapists overrate their own results in the same way. Evidence-based practice works when every claim, whether it comes from a trial or a therapist, has to answer to outcome data that someone else can see.

What STAR*D was and what it reported

STAR*D stands for Sequenced Treatment Alternatives to Relieve Depression. The National Institute of Mental Health (NIMH) paid about $35 million for it, and the 2023 reanalysis describes it as the largest and most expensive prospective antidepressant trial ever conducted.² Most drug trials enroll symptomatic volunteers, typically recruited through advertising, and screen out many people with other illnesses. STAR*D instead enrolled 4,041 adults who were already seeking care at 18 primary care and 23 psychiatric clinics, and it kept people with medical illnesses and most co-occurring psychiatric conditions. Its own investigators later showed why that mattered. Of 2,855 participants they analyzed, only 22.2% would have qualified for a typical phase III efficacy trial, and the 77.8% who would not have qualified had lower response and remission rates on the same drug.³

The design was a ladder. Everyone started on citalopram, an SSRI, for 12 weeks, with two more weeks for patients close to remission. Patients who did not remit could move to level 2, where they chose among the options they found acceptable and were randomized within them: switching to sertraline, bupropion, venlafaxine or cognitive therapy, or adding bupropion, buspirone or cognitive therapy to citalopram. Level 3 offered further switches and add-ons, and level 4 a final switch to tranylcypromine or a venlafaxine and mirtazapine combination. Care was “measurement-based”: clinicians used short symptom scales at each visit, including a self-report scale called the QIDS-SR, to guide dosing.²

The cognitive therapy arm deserves a note, because therapists rarely hear about it. Fewer than a third of level 2 participants agreed to randomization groups that allowed cognitive therapy to be compared with medication. Among those who did, outcomes were broadly similar. Adding a medication brought remission faster than adding cognitive therapy, and switching to cognitive therapy caused fewer side effects than switching to another antidepressant.⁴

In November 2006 the American Journal of Psychiatry published the summary report. Its abstract gave remission rates of 36.8%, 30.6%, 13.7% and 13.0% for the four steps and stated that “the overall cumulative remission rate was 67%.”¹ Only in the Results section did readers learn that this was a theoretical figure, built on the assumption that people who left the study would have remitted at the same rate as people who stayed.² That caveat did not travel. In 2009 Thomas Insel, then director of NIMH, and Philip Wang wrote that roughly 70% of participants were in remission at the end of 12 months, with up to four treatment steps. The figure became what the reanalysis calls accepted clinical wisdom.^2,5

It is worth asking why the number stuck. “Keep going and two out of three people get well” is a story with a clean ending, and depressed patients, families and prescribers all wanted that story. Numbers carry narratives, which is one reason psychotherapy has to take narrative seriously rather than pretend that data speak for themselves.

What the 2023 reanalysis found

H. Edmund Pigott had been arguing since at least 2010 that STAR*D’s effectiveness was probably lower than its authors reported.⁶ In March 2019 he and colleagues, including Irving Kirsch, announced in the BMJ that they would reanalyze the trial under RIAT, the Restoring Invisible and Abandoned Trials initiative. RIAT asks sponsors to publish or correct abandoned and misreported trials, and it encourages independent researchers to do so from the original data when the sponsors will not.⁷ The STAR*D investigators replied that the data were in the public domain and that they had no interest in reanalyzing them. NIMH’s data archive granted the team access in July 2019, and the results appeared in BMJ Open in July 2023.²

The paper identifies four departures from the protocol and from the investigators’ own earlier publications:

• The outcome measure was swapped. The protocol named the 17-item Hamilton Rating Scale for Depression (HRSD) as the primary outcome, administered by telephone by assessors who were blind to each patient’s stage in the study, with remission defined as a score under 8. It specifically excluded clinic-visit assessments from use as research outcomes. The summary report used the clinic-administered QIDS-SR instead, a scale developed by the study’s own principal investigators.
• Patients excluded earlier were let back in. The level 1 paper had excluded 931 patients from analysis because they lacked a baseline HRSD of 14 or more from the blinded assessors. They reappeared in the later reports. Ninety-nine of them had scored under 8 at baseline, which means they already met the remission threshold at study outset.
• Early dropouts disappeared. The investigators had said that patients with no exit HRSD would be counted as non-remitters. In the summary, 370 patients who left after their first visit were removed from the analysis instead.
• Patients already in remission were analyzed as new starters. 125 patients entered a next treatment level with an HRSD under 8, although the design paper had said that “patients who begin a level with HRSD <8 will be excluded from analyses.”

With the protocol measure and the eligibility rules applied, 3,110 of the 4,041 patients qualified for analysis. Remission rates fell from step to step: 25.5%, 21.3%, 13.2% and 10.4%. The cumulative remission rate after up to four treatments was 35.0%. Using the clinic scale to fill in patients who had no exit HRSD, the best case was 41.3%. The protocol had predicted that 20.7% of patients would drop out over the four levels. In the event, 53.7% did.²

The original investigators answered in the American Journal of Psychiatry with a letter titled “The STAR*D Data Remain Strong.”⁸ As of November 2026, Crossref and OpenAlex list no retraction or correction for the 2006 paper. Readers can weigh both sides, which is the real point. Because the patient-level data are archived, the argument now happens in public, over numbers both sides can inspect.

What the STAR*D scandal is not

It is not a placebo comparison. STAR*D had no placebo or waitlist group at any stage. Pigott and colleagues say directly that it is impossible to know how much of the improvement came from the drugs, how much from placebo effects and how much from the passage of time.² A 35% remission rate cannot be called “no better than placebo,” because nothing in the trial was measured against placebo.

It is not proof that antidepressants do not work. A 2018 network meta-analysis covered 522 trials and 116,477 participants. In double-blind trials of acute treatment, it found every one of the 21 drugs studied more effective than placebo, with modest odds ratios for response between 1.37 and 2.13 and evidence it rated moderate to very low in certainty.⁹ On the reanalysis’s numbers, STAR*D’s ladder of medication trials in real-world patients produced about half the remissions that were advertised, and many people dropped out. It does not settle whether a given drug helps a given person. If you take an antidepressant, talk with your prescriber before changing or stopping it.

It is not a fraud finding. Pigott’s team describes “scientific errors” that inflated the reported outcomes.

It is not mainly a story about drug-company money. STAR*D was publicly funded. The reanalysis points out that bias in the clinical literature is usually associated with industry-funded trials, and that here it turned up in a government-funded one.² Flexible outcome reporting is a habit of the whole field, which is why the fix has to be structural. The commercial side of the problem is real, and it is covered in our critique of evidence-based practice as science-flavored capitalism and in a closer look at the incentives behind evidence-based branding.

Psychotherapy research has the same problem

Therapists who read STAR*D as a drug-trial problem should look at their own literature. In 2008 Turner and colleagues compared 74 antidepressant trials registered with the FDA against what the journals published. The journals made it look as though 94% of trials were positive. The FDA’s own reviews put the figure at 51%, and publication inflated the overall effect size by 32%.¹⁰ In 2015 Driessen and colleagues, with Turner among the authors, ran the same check on psychotherapy. Of 55 NIH grants that began psychotherapy trials for major depression, 13 never produced a publication. When the unpublished data were added, the effect of psychotherapy against control conditions fell by a quarter, from g = 0.52 to 0.39. Their conclusion was that both kinds of treatment work, but not as well as the published literature suggests.¹¹

The replication problem is wider than treatment research. When the Open Science Collaboration repeated 100 published psychology studies with high-powered designs, 97% of the originals had reported significant results and 36% of the replications did. The average replication effect was half the size of the original.¹² A related argument, that the diagnostic categories of American psychiatry and psychology promised more scientific rigor than they delivered, is made in The Insecure Science.

Even well-known findings about therapy get revised. A 2015 meta-analysis by Johnsen and Friborg reported that the effects of CBT for depression had declined steadily across 70 studies published from 1977 to 2014.¹³ Two teams reanalyzed it. Ljótsson and colleagues found the decline concentrated in studies published before 1995, with no significant decline after. Cristea and colleagues restricted the analysis to randomized trials, found 30 more that qualified and concluded that the linear fall was most likely spurious.^14,15 Johnsen and Friborg replied that when all the studies are taken together the decline is still there.¹⁶ The honest summary is that the question is open. One detail in the original paper deserves more attention than the headline. In its subgroup analysis, experienced psychologists got larger effects (g = 1.55) than student therapists (g = 0.98).¹³ That is a comparison across studies, not an experiment, so it cannot show that experience caused the difference. It does suggest that the skill of the person delivering a protocol may be part of the treatment.

The label “evidence-based” can also hide how modest the evidence is. Jonathan Shedler argues that the term has become a code word for brief manualized therapy, usually CBT, and that research does not show these treatments to be superior to other therapies.¹⁷ You do not have to accept all of his argument to see the point in a well-studied area. In a JAMA review of randomized trials for military-related PTSD, cognitive processing therapy and prolonged exposure produced large within-group effects, and 49% to 70% of patients improved meaningfully. Yet about two-thirds still met criteria for PTSD after treatment.¹⁸ Those are good treatments. They are not finished ones.

What evidence-based practice was supposed to mean

The phrase has drifted a long way from its definition. Writing in the BMJ in 1996, Sackett and colleagues defined evidence-based medicine as “the conscientious, explicit, and judicious use of current best evidence in making decisions about the care of individual patients,” which meant integrating individual clinical expertise with the best available external clinical evidence from systematic research.¹⁹ The American Psychological Association adopted a parallel policy in 2005. Evidence-based practice in psychology is “the integration of the best available research with clinical expertise in the context of patient characteristics, culture, and preferences.”²⁰

That is a model with three legs, not one. Berg argues that the APA’s own policy statement defines and justifies clinical expertise and patient preference through research findings, so the model collapses into science alone, and that those two legs need to be defined on their own terms rather than as more research.²¹ Researchers who also practice are less narrow than the policy debates suggest. In a survey of psychotherapists who produce and review research, Safran and colleagues found that these researcher-clinicians rated research as less helpful to their clinical work than other sources, including their ongoing experience with clients, while still valuing it as one input among several.²² When payers and risk managers use the label instead of clinicians, the patient’s leg can be the first to go, a tension examined in our piece on insurance, liability and patient care.

The materials for a wider definition already exist. In 2018 the APA published reporting standards for qualitative and mixed-methods research, the first time qualitative research was included in APA Style. Studies of how clients experience therapy now have their own published standards alongside the quantitative ones.²³ The so-called common factors are not unmeasurable either. A meta-analysis of 295 studies covering more than 30,000 patients found a correlation of r = .278 between the strength of the therapeutic alliance and outcome, consistent across therapy types, measures and countries.²⁴ Transparency also means access. Larivière and colleagues found that by 2013 the five most prolific publishers accounted for more than half of all papers indexed in the Web of Science, and about 70% in the social sciences.²⁵ The STAR*D reanalysis, by contrast, was published open access, so any clinician can check its tables.

Clinical judgment needs outcome data too

The tempting conclusion from STAR*D is that experienced clinicians should trust what they see in the room over what journals say. The data on clinicians do not support that either. When Walfish and colleagues surveyed mental health professionals, 25% placed their own skills at the 90th percentile compared with their peers, and none rated themselves below average. As a group they overestimated how many of their clients improved and underestimated how many got worse.²⁶ In a study from Lambert’s group at Brigham Young University, clinicians rarely predicted which clients would not benefit from therapy. A brief questionnaire combined with decision rules about expected progress flagged all of the clients who had deteriorated by the end of treatment, and 85% of them by the third session.²⁷

Lilienfeld and colleagues catalogued 26 ways an ineffective therapy can look effective to the people involved in it. They group them into three kinds of error: seeing change that is not there, crediting therapy for change caused by events outside it, and crediting a specific method for change caused by nonspecific factors. Underneath sit four ordinary habits of mind: naive realism, confirmation bias, illusory causation and the illusion of control.²⁸ STAR*D’s summary report and an overconfident therapist make the same mistake. Each tells a favorable story without holding it to a measure chosen in advance.

The remedy has been tested. In routine outcome monitoring, the client fills out a short measure regularly and the therapist sees the trend, ideally with an alert when the client is off track. A 2018 meta-analysis of 24 studies found that this feedback reduced deterioration and nearly doubled rates of reliable improvement among clients predicted to do poorly.²⁹ A larger 2021 multilevel meta-analysis of 58 studies and 21,699 patients found a smaller overall benefit, d = 0.15, along with a modest reduction in dropout.³⁰ The effect is small. The principle is the one STAR*D’s critics are defending: decide what counts as getting better before you start, then report what happens, including the people who leave.

What this looks like in therapy

For a client, outcome transparency is concrete. In the first sessions, client and therapist agree on what “better” would mean, both in the client’s own words and on a brief standard questionnaire, so that there is something to check later besides anyone’s impression. The measure is repeated every few sessions and the client sees the numbers. When the numbers are flat or worse, the therapist says so and something changes: the focus, the method, the pace, sometimes the therapist. A run of sessions in which the therapist feels good about the work and the scores do not move is a warning sign, not a success.

Many people who come to therapy for depression have already climbed something like the STAR*D ladder: two, three or four medication trials, each started with hope. It helps them to hear the actual numbers. In STAR*D, remission became less likely with each step and dropout became more likely, so a person who has not responded to several medications is not an unusual failure.² That history is information about what has and has not helped. It is not a verdict on the person. Some of that suffering does not fit a diagnostic box at all, the territory of the third part of our Healing the Modern Soul series. Therapy can run alongside medication, and changes to medication belong in a conversation with the prescriber, with the therapist sharing what the scores show if the client wants that.

Evidence-based work also means saying how strong the evidence is for each method. Some approaches have large trial literatures behind them. Others, including several somatic and experiential methods, rest on much less, and the long argument over whether EMDR is evidence-based shows how contested these questions get. A client deserves to know where a method stands before agreeing to a plan, and to know that a method’s research base and its fit for them are separate questions. Three questions are worth asking any therapist. What will we measure? How will we know if this is not working? What happens then?

If you have been through several treatments for depression without the relief you were told to expect, our page on depression treatment in Hoover describes how we approach it. You are welcome to contact us with questions before deciding anything.

References

1. Rush, A. J., Trivedi, M. H., Wisniewski, S. R., Nierenberg, A. A., Stewart, J. W., Warden, D., Niederehe, G., Thase, M. E., Lavori, P. W., Lebowitz, B. D., McGrath, P. J., Rosenbaum, J. F., Sackeim, H. A., Kupfer, D. J., Luther, J., & Fava, M. (2006). Acute and longer-term outcomes in depressed outpatients requiring one or several treatment steps: A STAR*D report. American Journal of Psychiatry, 163(11), 1905–1917. https://doi.org/10.1176/ajp.2006.163.11.1905
2. Pigott, H. E., Kim, T., Xu, C., Kirsch, I., & Amsterdam, J. (2023). What are the treatment remission, response and extent of improvement rates after up to four trials of antidepressant therapies in real-world depressed patients? A reanalysis of the STAR*D study’s patient-level data with fidelity to the original research protocol. BMJ Open, 13(7), e063095. https://doi.org/10.1136/bmjopen-2022-063095
3. Wisniewski, S. R., Rush, A. J., Nierenberg, A. A., Gaynes, B. N., Warden, D., Luther, J. F., McGrath, P. J., Lavori, P. W., Thase, M. E., Fava, M., & Trivedi, M. H. (2009). Can phase III trial results of antidepressant medications be generalized to clinical practice? A STAR*D report. American Journal of Psychiatry, 166(5), 599–607. https://doi.org/10.1176/appi.ajp.2008.08071027
4. Thase, M. E., Friedman, E. S., Biggs, M. M., Wisniewski, S. R., Trivedi, M. H., Luther, J. F., Fava, M., Nierenberg, A. A., McGrath, P. J., Warden, D., Niederehe, G., Hollon, S. D., & Rush, A. J. (2007). Cognitive therapy versus medication in augmentation and switch strategies as second-step treatments: A STAR*D report. American Journal of Psychiatry, 164(5), 739–752. https://doi.org/10.1176/ajp.2007.164.5.739
5. Insel, T. R., & Wang, P. S. (2009). The STAR*D trial: Revealing the need for better treatments. Psychiatric Services, 60(11), 1466–1467. https://doi.org/10.1176/ps.2009.60.11.1466
6. Pigott, H. E., Leventhal, A. M., Alter, G. S., & Boren, J. J. (2010). Efficacy and effectiveness of antidepressants: Current status of research. Psychotherapy and Psychosomatics, 79(5), 267–279. https://doi.org/10.1159/000318293
7. Doshi, P., Dickersin, K., Healy, D., Vedula, S. S., & Jefferson, T. (2013). Restoring invisible and abandoned trials: A call for people to publish the findings. BMJ, 346, f2865. https://doi.org/10.1136/bmj.f2865
8. Rush, A. J., Trivedi, M., Fava, M., Thase, M., & Wisniewski, S. (2023). The STAR*D data remain strong: Reply to Pigott et al. American Journal of Psychiatry, 180(12), 919–920. https://doi.org/10.1176/appi.ajp.20230869
9. Cipriani, A., Furukawa, T. A., Salanti, G., Chaimani, A., Atkinson, L. Z., Ogawa, Y., Leucht, S., Ruhe, H. G., Turner, E. H., Higgins, J. P. T., Egger, M., Takeshima, N., Hayasaka, Y., Imai, H., Shinohara, K., Tajika, A., Ioannidis, J. P. A., & Geddes, J. R. (2018). Comparative efficacy and acceptability of 21 antidepressant drugs for the acute treatment of adults with major depressive disorder: A systematic review and network meta-analysis. The Lancet, 391(10128), 1357–1366. https://doi.org/10.1016/S0140-6736(17)32802-7
10. Turner, E. H., Matthews, A. M., Linardatos, E., Tell, R. A., & Rosenthal, R. (2008). Selective publication of antidepressant trials and its influence on apparent efficacy. New England Journal of Medicine, 358(3), 252–260. https://doi.org/10.1056/NEJMsa065779
11. Driessen, E., Hollon, S. D., Bockting, C. L. H., Cuijpers, P., & Turner, E. H. (2015). Does publication bias inflate the apparent efficacy of psychological treatment for major depressive disorder? A systematic review and meta-analysis of US National Institutes of Health-funded trials. PLOS ONE, 10(9), e0137864. https://doi.org/10.1371/journal.pone.0137864
12. Open Science Collaboration. (2015). Estimating the reproducibility of psychological science. Science, 349(6251), aac4716. https://doi.org/10.1126/science.aac4716
13. Johnsen, T. J., & Friborg, O. (2015). The effects of cognitive behavioral therapy as an anti-depressive treatment is falling: A meta-analysis. Psychological Bulletin, 141(4), 747–768. https://doi.org/10.1037/bul0000015
14. Ljótsson, B., Hedman, E., Mattsson, S., & Andersson, E. (2017). The effects of cognitive–behavioral therapy for depression are not falling: A re-analysis of Johnsen and Friborg (2015). Psychological Bulletin, 143(3), 321–325. https://doi.org/10.1037/bul0000055
15. Cristea, I. A., Stefan, S., Karyotaki, E., David, D., Hollon, S. D., & Cuijpers, P. (2017). The effects of cognitive behavioral therapy are not systematically falling: A revision of Johnsen and Friborg (2015). Psychological Bulletin, 143(3), 326–340. https://doi.org/10.1037/bul0000062
16. Friborg, O., & Johnsen, T. J. (2017). The effect of cognitive–behavioral therapy as an antidepressive treatment is falling: Reply to Ljótsson et al. (2017) and Cristea et al. (2017). Psychological Bulletin, 143(3), 341–345. https://doi.org/10.1037/bul0000090
17. Shedler, J. (2018). Where is the evidence for “evidence-based” therapy? Psychiatric Clinics of North America, 41(2), 319–329. https://doi.org/10.1016/j.psc.2018.02.001
18. Steenkamp, M. M., Litz, B. T., Hoge, C. W., & Marmar, C. R. (2015). Psychotherapy for military-related PTSD: A review of randomized clinical trials. JAMA, 314(5), 489–500. https://doi.org/10.1001/jama.2015.8370
19. Sackett, D. L., Rosenberg, W. M. C., Gray, J. A. M., Haynes, R. B., & Richardson, W. S. (1996). Evidence based medicine: What it is and what it isn’t. BMJ, 312(7023), 71–72. https://doi.org/10.1136/bmj.312.7023.71
20. American Psychological Association Presidential Task Force on Evidence-Based Practice. (2006). Evidence-based practice in psychology. American Psychologist, 61(4), 271–285. https://doi.org/10.1037/0003-066X.61.4.271
21. Berg, H. (2019). Evidence-based practice in psychology fails to be tripartite: A conceptual critique of the scientocentrism in evidence-based practice in psychology. Frontiers in Psychology, 10, 2253. https://doi.org/10.3389/fpsyg.2019.02253
22. Safran, J. D., Abreu, I., Ogilvie, J., & DeMaria, A. (2011). Does psychotherapy research influence the clinical practice of researcher–clinicians? Clinical Psychology: Science and Practice, 18(4), 357–371. https://doi.org/10.1111/j.1468-2850.2011.01267.x
23. Levitt, H. M., Bamberg, M., Creswell, J. W., Frost, D. M., Josselson, R., & Suárez-Orozco, C. (2018). Journal article reporting standards for qualitative primary, qualitative meta-analytic, and mixed methods research in psychology: The APA Publications and Communications Board task force report. American Psychologist, 73(1), 26–46. https://doi.org/10.1037/amp0000151
24. Flückiger, C., Del Re, A. C., Wampold, B. E., & Horvath, A. O. (2018). The alliance in adult psychotherapy: A meta-analytic synthesis. Psychotherapy, 55(4), 316–340. https://doi.org/10.1037/pst0000172
25. Larivière, V., Haustein, S., & Mongeon, P. (2015). The oligopoly of academic publishers in the digital era. PLOS ONE, 10(6), e0127502. https://doi.org/10.1371/journal.pone.0127502
26. Walfish, S., McAlister, B., O’Donnell, P., & Lambert, M. J. (2012). An investigation of self-assessment bias in mental health providers. Psychological Reports, 110(2), 639–644. https://doi.org/10.2466/02.07.17.PR0.110.2.639-644
27. Hannan, C., Lambert, M. J., Harmon, C., Nielsen, S. L., Smart, D. W., Shimokawa, K., & Sutton, S. W. (2005). A lab test and algorithms for identifying clients at risk for treatment failure. Journal of Clinical Psychology, 61(2), 155–163. https://doi.org/10.1002/jclp.20108
28. Lilienfeld, S. O., Ritschel, L. A., Lynn, S. J., Cautin, R. L., & Latzman, R. D. (2014). Why ineffective psychotherapies appear to work: A taxonomy of causes of spurious therapeutic effectiveness. Perspectives on Psychological Science, 9(4), 355–387. https://doi.org/10.1177/1745691614535216
29. Lambert, M. J., Whipple, J. L., & Kleinstäuber, M. (2018). Collecting and delivering progress feedback: A meta-analysis of routine outcome monitoring. Psychotherapy, 55(4), 520–537. https://doi.org/10.1037/pst0000167
30. de Jong, K., Conijn, J. M., Gallagher, R. A. V., Reshetnikova, A. S., Heij, M., & Lutz, M. C. (2021). Using progress feedback to improve outcomes and reduce drop-out, treatment duration, and deterioration: A multilevel meta-analysis. Clinical Psychology Review, 85, 102002. https://doi.org/10.1016/j.cpr.2021.102002