Table of Contents
Clinical Takeaway
Standardized diagnostic interviews show variable but generally acceptable test-retest reliability across common adult psychiatric disorders, meaning repeated administrations of the same structured interview do not always produce identical diagnoses. Clinicians should be aware that no single diagnostic tool is perfectly consistent, and results may shift depending on the disorder being assessed, the time between interviews, and the specific instrument used. This reinforces the importance of using diagnostic interviews as one component of a comprehensive clinical evaluation rather than relying on them as a definitive standalone measure.
#1 Test-Retest Reliability of Standardized Diagnostic Interviews for Common Adult Psychiatric Disorders: A Systematic Review and Meta-Analysis.
Citation: Xie Weiyi et al.. Test-Retest Reliability of Standardized Diagnostic Interviews for Common Adult Psychiatric Disorders: A Systematic Review and Meta-Analysis.. JAMA network open. 2026. PMID: 42207515.
Want to apply this research to your care?
CED Clinic translates emerging research into individualized clinical care. Dr. Caplan has treated 30,000+ patients.
Book a consultation →Design: 6 Journal: 4 N: 4 Recency: 3 Pop: 1 Human: 1 Risk: 0
Abstract: IMPORTANCE: Standardized diagnostic interviews (SDIs) are structured assessments based on established criteria to improve the consistency and reliability of diagnoses. The pooled test-retest reliability of SDIs for adult psychiatric disorders is unknown. OBJECTIVES: To estimate the test-retest reliability of SDIs used to classify common adult psychiatric disorders, examine variations in test-retest reliability between disorders, and assess prespecified factors associated with between-study heterogeneity. DATA SOURCES: MEDLINE, Embase, Emcare, PsycINFO, and Applied Social Sciences Index and Abstracts were searched without date or language limitations from inception until September 2025. References of eligible articles and relevant reviews were also screened. STUDY SELECTION: Primary studies that evaluated test-retest reliability of SDIs assessing adult psychiatric disorders were selected. Disorders were selected based on estimated prevalence in the general adult population, clinical relevance, and frequent appearance in SDIs. DATA EXTRACTION AND SYNTHESIS: Data were extracted and study quality was assessed based on the Consensus-based Standards for the Selection of Health Measurement Instruments checklist. Multilevel random-effects meta-analysis and meta-regression were performed. Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) guidelines were followed. MAIN OUTCOMES AND MEASURES: Test-retest reliability estimates (Cohen κ) of SDI-based adult psychiatric disorder diagnoses. Pooled estimates were calculated for 5 groups of mental disorders (anxiety, bipolar, depressive, personality, and nonaffective psychoses) and 8 groups of substance use disorders (SUDs; alcohol, cannabis, cocaine, hallucinogens, opioids, sedatives, stimulants, and tobacco). RESULTS: Fifty-seven studies were analyzed, 46 of which were included in the meta-analysis (535 κ estimates; N = 8146 participants [mean age range, 22.0-54.3 years]). The pooled estimate of SDI test-r
What This Study Teaches Us
This meta-analysis pooled test-retest reliability data across standardized diagnostic interviews for common psychiatric disorders and substance use disorders. It provides the first systematic estimate of how consistently these structured interviews classify the same person on repeated assessment.
Why This Matters Clinically
Clinicians rely on diagnostic interviews to establish psychiatric diagnoses that guide treatment selection and prognosis. Knowing the actual reliability of these tools (not just assuming they work) helps you interpret whether a diagnosis is stable or potentially an artifact of measurement error, especially when making major treatment decisions.
Study Snapshot
| Study Design | Systematic review and meta-analysis of test-retest reliability studies using multilevel random-effects models |
| Population | Adult subjects assessed with standardized diagnostic interviews for psychiatric and substance use disorders; N of included studies and total participants not specified in abstract |
| Intervention | Not applicable. This is an observational synthesis of reliability coefficients (Cohen kappa) from existing studies |
| Primary Outcome | Test-retest reliability (Cohen kappa) of standardized diagnostic interviews for five psychiatric disorder groups and eight substance use disorder groups |
| Key Result | Not specified in abstract. The abstract states objectives and methods but does not report the actual pooled kappa values or reliability findings |
Where This Paper Deserves Skepticism
The abstract provides no actual numerical results, making it impossible to evaluate the magnitude of reliability or identify which disorders showed poor agreement. The search period ends September 2025, which means this is a future-dated publication being reviewed prematurely or the abstract itself is incomplete. Without knowing how many studies were included, the diversity of interview types tested, or time intervals between test and retest administrations, we cannot assess whether heterogeneity was adequately explained.
Dr. Caplan’s Take
I appreciate that someone is finally systematizing what we know about the test-retest performance of our diagnostic tools. In cannabis medicine and all of psychiatry, we often cite structured interviews as the gold standard without actually knowing how reliably they perform across repeated administrations. That said, I’d want to see the actual data before drawing conclusions. The absence of results in this abstract is frustrating, and I’d need the full paper to comment on what this means for my diagnostic confidence in clinical practice.
Clinical Bottom Line
This systematic review is methodologically sound in approach but cannot be meaningfully evaluated without the actual reliability estimates and disorder-specific findings reported in the full paper.
Clinical Angles to Consider:
- Does this suggest a role in treatment-resistant depression or as monotherapy?
- How does mood-related ECS dysregulation contextualize this finding?
- What monitoring frequency is appropriate given this data?
|
Have thoughts on this? Share it: