Helping teens heal: Peabody researchers develop a new tool to improve teen mental health services

topics: Psychology and Human Development

Fall 2007, Melanie Moran and Kurt Brobeck, Ideas In Action

Imagine archers trying to improve their accuracy by practicing blindfolded, never seeing how close they are to hitting the target, never getting any information to help correct their aim.

Doctors and clinicians treating teens with mental health concerns have been in much the same position, providing services week after week with no objective and systematic feedback about the effects of their treatment. But a new tool developed by Peabody College researchers will remove that blindfold by providing ongoing feedback to service providers, with a goal of enabling mid-course treatment corrections. The tool is called the Peabody Treatment Progress Battery, or PTPB. The battery is available to qualified clinicians for free on the Web.

“There are laboratory studies that show treatments are very effective with youth who have mental health concerns, but when we look in the real treatment world we are hard pressed to identify services that are effective,” Leonard Bickman, director of the Center for Evaluation and Program Improvement and associate dean for research at Peabody, said. Bickman led the team that developed the PTPB. “The PTPB offers a revolutionary opportunity to improve mental health services. Mental health professionals need to know if they are succeeding during treatment and, if they are not, they need to know what to change. The PTPB gives them that information.”

The PTPB is intended primarily for use in schoolbased or home-based settings for youths aged 11-18 years. Often, these are children in foster care situations or whose mental health care has been court-ordered.

The battery includes 10 measures by which to assess how teens are responding to treatment. All were rigorously evaluated to ensure their clinical relevance. The measures include a symptoms and functioning severity scale, a life satisfaction scale, a hopefulness scale, an outcome expectations scale, a therapeutic alliance scale (which focuses on the quality of the relationship between the client and therapist), a counseling impact scale (as reported by the client), and a motivation for treatment scale.

“What we’re trying to do is use measures other than pathology to look at progress. So we examine the factors that are common to any treatment situation,” says Bickman. “Many of our measures are strength-based.” Hopefulness, for example, has been rarely measured with youths, though increasingly it is measured in adults. “Without hope,” Bickman notes, “it is less likely that there will be improvement.” Bickman points out that for some clients effective treatment may not mean improvement but preventing the youth from getting worse.

In addition to monitoring the teens’ responses to treatment, the PTPB also asks for information from their caregivers, including how they are managing stress, their own life satisfaction, treatment expectations, and perceptions of the child’s motivation and treatment progress. “If the caregiver isn’t invested,” said Bickman, “they’re less likely to keep appointments and more likely to drop out of treatment. Some of these treatments require cooperation from the family; treatment is less likely to be helpful if there is not a good relationship between the caregiver and the therapist. We think caregivers are an integral part of the team and need to be involved in the youth’s treatment.”

The clinician, too, completes measures for symptoms and severity and therapeutic alliance. It is this comparison of what the clinician thinks is effective with what the child and caregiver report that Bickman believes makes the battery an especially powerful tool. “If the therapist thinks treatment is going great and the teen thinks it’s going terribly, then the therapist is probably not going to be very helpful to that youth.”

“The therapist needs to collect systematic feedback concurrent with treatment,” Bickman says. “The PTPB will provide feedback that is hard to disregard and will also insert some accountability for producing results into the mental health system.”

Abram Rosenblatt, professor and researcher in the Department of Psychiatry at University of California, San Francisco, sees the PTPB as an important development. “Providers of children’s mental health services have long expressed their frustration to the research community at the lack of a comprehensive, feasible and scientifically developed set of measures for assessing process and outcome. Existing measures have been too time consuming, too limited in the domains and perspectives assessed, too costly, or insufficient with regard to scientific validity,” he said. “The PTPB is the first comprehensive set of measures that can be used routinely to assess process and outcomes of treatment services across multiple domains and perspectives, and it meets the highest level of scientific rigor. This is a landmark contribution to children’s mental health services and research.”

The PTPB takes approximately five to eight minutes weekly to complete per patient. It was designed to apply to most types of treatment and was written at a fourth-grade reading level, both in English and Spanish. Bickman has already received an offer to translate it to Portuguese. Development of the PTPB was partially funded by the National Institute of Mental Health.

“What we’ve done is license it with a registration,” Bickman says. So far, over 300 individuals or organizations have completed the registration. In the meantime, Bickman and his team are anxious to implement a computerized version that will automate much of the process of administering the measures, scoring them and providing reports to the clinician and clinical supervisor. They are working with a computer application service provider to develop and test the program.

They also are working with Providence Service Corporation, a large mental health provider for children, to develop and test the PTPB. Bickman says, “Thanks to Providence, we’re in 12 different states and almost 40 different locations.

Natasha S.Walsh, vice president of clinical services for Providence, speaks highly of the results. Says Walsh, “Our clinicians are getting information they are able to use to inform and enhance treatment. Our supervisors are getting information they can use to help the clinicians grow into better clinicians. Our agency is getting the information on clinical outcomes and processes needed to improve the quality of our services. Our counselors have said ‘it’s giving me information I never would have gotten.’”

PTPB in Action: Feedback and Flexibility

The digital version of the Peabody Treatment Progress Battery goes by the name Contextualized Feedback Intervention and Training (CFIT). While still under development, the program is being piloted by the Providence Service Corporation in 12 states.

Len Bickman, who led the team that created it, acknowledges that introducing it into an organization poses a variety of challenges. “This needs the strong support of the institution going into it, because CFIT adds significant new responsibilities for the organization, especially supervisors and therapists. We knew there was not much time for data collection so we focused on developing short and brief measures. But short and valid measures are difficult to develop. It was also important that we develop weekly feedback reports that were useful.”

Barry Beagles, a therapeutic foster care counselor in Springfield, Illinois, is an enthusiastic adopter.

According to Beagles, “The feedback reports give me insight into the kids I’m working with. I had a kid who was internalizing a lot of feelings that he was not talking about. The item alert on the symptoms and functioning scale came up several times. The child kept writing about his self esteem issues which led to his hope declining. When I got the feedback report, I dug deeper and talked with him one on one without the foster parents present. The child let loose and said he was thinking about hurting himself, having violent nightmares and hearing voices.

“We had the child screened, and he was admitted into the hospital for two weeks and placed on medications. Now, the child keeps a journal and talks with me openly. The foster parent has a different outlook too and doesn’t think the child is being manipulative. I’m very thankful for CFIT. It led me in a different direction.”

Beagles has years of experience as a high school and college teacher of psychology and sociology, and twelve years in the Illinois Department of Human Services, working with abused and neglected children.

“Evolutionary revolutionary” is how Bickman describes this interplay between therapist, child and data. “We don’t tell the therapist what to do. We don’t say, ‘Here’s our manual: do what it says.’ We give them information and they have to decide what actions to take. We give them some guidance about that, but there’s no set program, per se. However, we are testing a training program designed to enhance factors common to all treatments such as the therapeutic relationships and motivation to improve.”