Journal of Substance Abuse Treatment 2014;47(3):222-228. [doi: 10.1016/j.jsat.2014.05.008]
K. Michelle Peavy, PhD (Evergreen Treatment Services, PN Node), Joseph R. Guydish, PhD (University of California, San Francisco, WS Node), Jennifer Knapp Manuel, PhD (San Francisco VA, WS Node), Barbara K. Campbell, PhD (Oregon Health & Science University, WS Node), Nadra Lisha, PhD (University of California, San Francisco, WS Node), Thao Le (University of California, San Francisco, WS Node), Kevin L. Delucchi, PhD (University of California, San Francisco, WS Node), Sharon B. Garrett, MPH, MA (University of Washington, PN Node).
This ancillary investigation of data from National Drug Abuse Treatment Clinical Trials Network protocol CTN-0031 ("Stimulant Abuser Groups to Engage in 12-Step (STAGE 12)") investigated the correspondence among four groups of raters on adherence to STAGE-12, a manualized 12-step facilitation (TSF) group and individual treatment targeting stimulant abuse. The four rater groups included the study therapists, supervisors, study-related ("TSF expert") raters, and non-project related ("external") raters. Results indicated that external raters rated most critically Mean Adherence -- the mean of all the adherence items -- and global performance. External raters also demonstrated the highest degree of reliability with the designated expert. Therapists rated their own adherence lower, on average, than did supervisors and TSF expert raters, but therapist ratings also had the poorest reliability.
Conclusions: Findings highlight the challenges in developing practical, but effective methods of fidelity monitoring for evidence-based practice in clinical settings. While funding and licensing agencies increasingly call for use of evidence-based treatments, community-based organizations implementing them will seek the simplest, most reliable and cost-effective ways of monitoring their delivery. These results suggest that there may be a role for on-site therapists or supervisors rating adherence, and that raters unaffiliated with the treatment being tested may provide the most objective ratings. Future research should examine the impact of training therapists on self-rating to determine whether this group and achieve acceptable reliability and objectivity in ratings. (Article (Peer-Reviewed), PDF, English, 2014)
Keywords: Adoption of interventions | Behavior therapy | CTN platform/ancillary study | Evidence-based treatment | Fidelity of implementation | Group therapy | Stimulant abuse | Twelve-Step Facilitation (TSF) | Twelve-step programs | Journal of Substance Abuse Treatment (journal)
Document No: 1072, PMID: 25017448, PMCID: PMC4131987 (available 9/1/2015).
Submitted by CTN Dissemination Librarians, 6/17/2014.