"Multi-Center Study of Youth Outcomes" (Behrens, 2006)

[Ursus]

Some pertinent posts from the Lester Roloffs Rebekah Home for Girls Survivors-Cult-mindcon thread are re-posted below.

From this study's title page:

Report of Findings from a Multi-Center Study of Youth Outcomes in Private Residential Treatment

Ellen Behrens, Ph.D.
Kristin Satterfield, B.S.

Canyon Research & Consulting, Inc.
Salt Lake City, UT
(801) 205-2330
[email protected]

Presented at the 114th Annual Convention of the American Psychological Association at New Orleans, Louisiana, August, 2006
August 12, Saturday, 2:00-2:50 PM[/list]

-------------- • -------------- • --------------

Originally posted on 16 Jun 2010 15:22:

Sorry to burst your bubble Anne... Read it and weep:

The following Study was presented at  the Annual Convention of the
American Psychological Association at New Orleans, Louisiana, August, 2006 .

The study covered about 1,000 kids who graduated from programs and parents of the kids.

There was an independent Board which approved the research and audited the study for any conflicts of interest.

Link to Study


So now are you ready to accept that you have stated :  Programs help the majority of kids and that you embellished your own story here on fornits at times just to win an argument?

You know you cant win if you challenge me to provide the link.

...

Originally posted on 16 Jun 2010 15:31:

Sorry to burst your bubble Anne... Read it and weep:

The following Study was presented at  the Annual Convention of the
American Psychological Association at New Orleans, Louisiana, August, 2006 .

The study covered about 1,000 kids who graduated from programs and parents of the kids.

There was an independent Board which approved the research and audited the study for any conflicts of interest.

Link to Study


So now are you ready to accept that you have stated in the past that Programs help the majority of kids and that you embellished your own story here on fornits at times just to win an argument?

You know you cant win if you challenge me to provide the link.

...

Well, off the top of my head (I've only read to page 3 so far).....it says that it was conducted between 2003 - 2005 and was presented in August of 2006.  Does not a longitudinal study make, which is a major point that we've been trying to get through your head for a few years now.  

But nice try.  I'll get back to you after I've read it all the way through.      :seg:

Originally posted on 16 Jun 2010 15:43:

….Does not a longitudinal study make, which is a major point that we've been trying to get through your head for a few years now.

A few years ago we would hear:   “We want proof, show us that programs work”.  So various posters provided them with first hand accounts of kids who did well and were set back on a healthy path.

Then they said:  ”Well, first hand accounts don't mean squat, the kids could have been brainwashed, we want to see studies”.  So we provided them with studies which showed 60 - 80% of the kids succeed in programs and they said “Ha,Ha they are not accurate!!  You call that a study?  Ppsssfftt…They are merely internal studies.  They could have just made that stuff up!”

Then we provided external studies which showed programs to be better than 80% effective and they said:  “The studies are no good!!  One of the people on the studies use to work in the Troubled Teen industry!!!”  Ha,Ha  “The entire effort is negated.”

So we provided them with Independent studies which show programs to be 80% effective in getting kids placed back on a healthy path with third party oversight to insure there were no conflict of interests.   They said:  "Well…well…. Ooommm…..we want long term clinical trials!!!... yeah that’s it Clinical Trials which cover 5 years out.  We don’t believe your studies we want clinical trials!”  

Lol.
Sorry, anne, I dont really mean to rub it in, but in all fairness it is good to keep pushing for more and better controlled studies.  I think we can all agree that the more studies that are done the better everyone will feel that the information is accurate and feel more confident  about sending our kids to these places.



...

Originally posted on 16 Jun 2010 16:14:

….Does not a longitudinal study make, which is a major point that we've been trying to get through your head for a few years now.

A few years ago we would hear:   "We want proof, show us that programs work".  So various posters provided them with first hand accounts of kids who did well and were set back on a healthy path.

Then they said:  "Well, first hand accounts don't mean squat, the kids could have been brainwashed, we want to see studies".  So we provided them with studies which showed 60 - 80% of the kids succeed in programs and they said "Ha,Ha they are not accurate!!  You call that a study?  Ppsssfftt…They are merely internal studies.  They could have just made that stuff up!"

Then we provided external studies which showed programs to be better than 80% effective and they said:  "The studies are no good!!  One of the people on the studies use to work in the Troubled Teen industry!!!"  Ha,Ha  "The entire effort is negated."

So we provided them with Independent studies which show programs to be 80% effective in getting kids placed back on a healthy path with third party oversight to insure there were no conflict of interests.   They said:  "Well…well…. Ooommm…..we want long term clinical trials!!!... yeah that's it Clinical Trials which cover 5 years out.  We don't believe your studies we want clinical trials!"  

Lol.
Sorry, anne, I dont really mean to rub it in, but in all fairness it is good to keep pushing for more and better controlled studies.  I think we can all agree that the more studies that are done the better everyone will feel that the information is accurate and feel more confident  about sending our kids to these places.

Wait a minute...

This study was performed on kids from nine Aspen programs, and the primary author is Ellen Behrens. Conflict of interest perchance? Moreover, these programs are not inherently different from one another to any significant degree.

There were two time points: at admission, and at discharge. In other words, the kids are basically still under the influence of the program.

The study consisted of a survey filled out by both kids and their parents. Some examples of discharge questions are "How do you feel about your experience at this program?" and "Compared to when you began the program, how would you describe your problems?"

There was no control group. There was no independent professional assessment of the parent and adolescent self-reports (e.g., by a psych professional not affiliated with the program).

[Whooter]

Look I don’t expect anyone to accept the studies here on fornits.  I really do get that and I am not trying to convince anyone here because it falls on deaf ears.  I mean how could you accept that programs help over 80% of the kids that enter them.  It would mean the problems you experienced were resolved long ago or isolated and will never be acted upon.  You are not interested in the study, your only goal is to find a flaw in it or a negative finding and exploit that fact over all the others and believe it or not I do understand why.  I am not knocking you for that.

The fact is that these studies are widely accepted and have been approved by an oversight committee who looks for things like conflicts of interest, questions and group (subgroup) structure, nature of the research etc.  You can submit your protest to the oversight committee if you like, Ursus, but neither you, I or anyone else can refute the study based on a few google searches and personal standards.



...

[Ursus]

I mean how could you accept that programs help over 80% of the kids that enter them.

Yes, how can you possibly accept that percentage as "fact" ... based on exit surveys before the kid is even out of the program. With NO follow-up data. With NO independent oversight. And conducted by a researcher with prior program affiliations.

[Ursus]

The fact is that these studies are widely accepted and have been approved by an oversight committee who looks for things like conflicts of interest, questions and group (subgroup) structure, nature of the research etc. You can submit your protest to the oversight committee if you like, Ursus, but neither you, I or anyone else can refute the study based on a few google searches and personal standards.

I am sure these studies are widely accepted by the marketing firms who churn out the press releases for future investors, the glossy brochures to glaze the eyes of desperate parents.

And I fully understand who the oversight committee is. They do it for pay. For researchers who are in a hurry and who have the bucks. And, all things considered, they do have their place in this world.

But there's a reason that reputable research institutions will not accept Western Institutional Review Board as an oversight organization for later phase clinical trials.

The fact is, this study does not look for "success rates" where they are least likely to be, namely, a number of years down the line, when the PSTD sets in, oft times along with worsened attendant behaviors, and when one is no longer under the threat of being sent elsewhere for more remedial behavior modification should one's outlook be significantly less than optimal. But isn't such a later time point more appropriate given the nature of what these programs promise?

Tell me, was this paper ever published? Or was its debut — two and a half weeks shy of the one-year anniversary of Hurricane Katrina — the sole venue for public critique?

[Whooter]

I mean how could you accept that programs help over 80% of the kids that enter them.

Yes, how can you possibly accept that percentage as "fact" ... based on exit surveys before the kid is even out of the program. With NO follow-up data. With NO independent oversight. And conducted by a researcher with prior program affiliations.

If they did the study at admission and then 5 years out you wouldn’t accept it either.  You would argue that the kids got better on their own after they left the program.
The study shows the improvements made from admission until the child left.  Did the child improve?  Grade  point average increase/decrease…depression  etc.

Maybe the rate goes from 80% to 90% after 3 years out.. maybe it goes from 80% to 60% 3 years out.  That wasn't studied here.

There was an ethics review which looks at conflicts of interest in the event there was a problem with ties to the industry studied.

[DannyB II]

I mean how could you accept that programs help over 80% of the kids that enter them.

Yes, how can you possibly accept that percentage as "fact" ... based on exit surveys before the kid is even out of the program. With NO follow-up data. With NO independent oversight. And conducted by a researcher with prior program affiliations.

Ursus when was the last time you volunteered for a Hyde graduate study. As a matter of fact when was the last time any of us here have been a part of a study since we left our programs.
Why does it surprise you so that the researcher would have program affiliations, does this mean the man/woman cannot have integrity. Who else would be performing this survey, someone with no experience in the TTI at all, then you would negate that finding based on the researcher having no prior experience.
Just asking....

danny

[Whooter]

I am sure these studies are widely accepted by the marketing firms who churn out the press releases for future investors, the glossy brochures to glaze the eyes of desperate parents.

Of course they do.  The marketing firms and investors like to hear anything that is positive and would increase profits.  If the report showed a negative outcome we probably wouldn't be reading about it right now.

And I fully understand who the oversight committee is. They do it for pay. For researchers who are in a hurry and who have the bucks. And, all things considered, they do have their place in this world.

People getting paid seems to bother you Ursus.  With exception of those on welfare most people in the US gets paid.  Payment isn’t a sign of something going wrong or a mistake being made.  Its very common and pays for them to hire people, pay the rent, buy computers, conduct research, get the kids to the dentist etc.  If they hired some friend of the family to do it for free I think we would be more concerned.

But there's a reason that reputable research institutions will not accept Western Institutional Review Board as an oversight organization for later phase clinical trials.

You would have to tell that to the medical industry…. Temple University…Drexel University… UCLA medical.  (Found in 10 minute Google search) I don’t think they would be happy with you calling them non-reputable.

Drexel Medical clinical Studies

UCLA Medical

TempleU Clinical Studies

The fact is, this study does not look for "success rates" where they are least likely to be, namely, a number of years down the line, when the PSTD sets in, oft times along with worsened attendant behaviors, and when one is no longer under the threat of being sent elsewhere for more remedial behavior modification should one's outlook be significantly less than optimal. But isn't such a later time point more appropriate given the nature of what these programs promise?

I could choose any heart study at random and do the same thing you are.  Lets see the study doesn’t look at patients by nationality… did they consider genetic disorders.. how come they stopped at 15 years?  How many patients smoked weed?  See what I mean?  The questions are endless.  They didn’t take everything into account.  There is no way they could.  The boundary conditions are very well defined in the study, Ursus.

Tell me, was this paper ever published? Or was its debut — two and a half weeks shy of the one-year anniversary of Hurricane Katrina — the sole venue for public critique?

Not sure.

I think the study does a good job in showing how well the child does from beginning to end of the program (which was the design).  What you are looking for is how the child does from graduation until say 5 years out.  But how does this directly measure the effectiveness of the program?  What would they compare it to?  I think this would be a great study myself, but I don’t know how it would benefit the industry.  How could they take credit for the success 5 years out.. or the failures 5 years out.  There are so many other factors which effect the child outside the program which are unknown and  out of their control.



...

[Awake]

I agree there is far too much lacking to take this study seriously, but even if someone were to, I would point out that the results clearly negate the perspective of the subject and only can speak from the group examining him.  A question we should ask is whether or not therapy is ethical if it relies on lying to the subject to maintain the behavior?

We can’t deny the ‘troubled teens’ are not there by choice, and if you review the criteria used to determine the results you will then also see that the criterion for treatment are ambiguous as to who is the patient, the individual or the social body that has exerted control?  

For a teen held in a program it will be presented to them as a situation where there is enormous pressure to work toward individual growth and accomplishment, overcoming personal obstacles, spiritual growth and understanding of self.  Yet the real goal is to produce results are only concerned with changes in their (the TTI) standards for what represents a positive change in the individual as it suits the social environment.

I look at all the criteria with which they make their judgment and it seems that the analysis of behavior changes can only be cosmetic at best, due to the situation.  A subject, like a ‘troubled teen’, is indisputably not individually motivated to endure therapy, and therefore actions in therapy are related to avoidance behavior.  This whole study was presented based on perceived changes as to these focal issues:

Depressed, withdrawn, social problems, attention problems, rule breaking behavior, Compliance, relationship with parents, communication with parents…

First of all, the ‘subject’ or ‘troubled teen’ is, inherently, based on the TTI situation, not choosing to make these changes, so we cannot consider any of these results to include the true feelings of the teens themselves, but the outward expression achieved from the structure of the environment that is pressuring to produce behaviors that support that result.

And second, these ‘criteria’ take on different meanings depending on the perspective.  ‘depressed’ ‘withdrawn’ ‘relationship with parents’… and so on.  The suggested meaning here is that the statistical change occurs due to the denial of the ‘patient’, his feelings and ability to freely comment on them, and the resulting relief of tension on the parents and program.  These results necessitate lying to the patient as to the intent of the treatment.


.

[Ursus]

I am sure these studies are widely accepted by the marketing firms who churn out the press releases for future investors, the glossy brochures to glaze the eyes of desperate parents.

Of course they do.  The marketing firms and investors like to hear anything that is positive and would increase profits.  If the report showed a negative outcome we probably wouldn't be reading about it right now.

Does a glossy brochure make the results any more credible or seemingly postive?

And I fully understand who the oversight committee is. They do it for pay. For researchers who are in a hurry and who have the bucks. And, all things considered, they do have their place in this world.

People getting paid seems to bother you Ursus.  With exception of those on welfare most people in the US gets paid.  Payment isn’t a sign of something going wrong or a mistake being made.  Its very common and pays for them to hire people, pay the rent, buy computers, conduct research, get the kids to the dentist etc.  If they hired some friend of the family to do it for free I think we would be more concerned.

Nope. You sure do make a lot of presumptions 'bout folks, don't you? People getting paid does not bother me. It's when that potentially interferes with the veracity of the results that my fur gets a lil spiky. Does it interfere here? I honestly don't know. I'm bringing it up as a potential concern, that's all.

Personally, I'm not entirely sure that WIRB is in a position to assess whether this is a credible study or not. Based strictly on population percentages, it's unlikely that any of its personnel have been through a program, and hence would not be likely to spot the studies' blind spots. And I'm also not so sure that they are in a position to even be concerned with that issue in the first place. From my apparently less than savvy perspective, it would appear that WIRB's role is mostly focused on making sure that all human subjects have filled out all the proper consent forms, all researchers maintain regulatory compliance, etc. etc.

But there's a reason that reputable research institutions will not accept Western Institutional Review Board as an oversight organization for later phase clinical trials.

You would have to tell that to the medical industry…. Temple University…Drexel University… UCLA medical.  (Found in 10 minute Google search) I don’t think they would be happy with you calling them non-reputable.

Drexel Medical clinical Studies

UCLA Medical

TempleU Clinical Studies

Yup. Got me here. The first one *I* had found was Tufts, which will not allow WIRB review for anything other than Phase III clinical trials which are industry (for-profit) sponsored or non-federally funded foundation sponsored studies. I guess I was wondering about the Phase IV trials... I had researched this long ago, when you first started talking about this study (which is the exact same one you've been bringing up for a year it seems), and I had forgotten the reason for said limitations (hence not stated, and ha! on me).

The fact is, this study does not look for "success rates" where they are least likely to be, namely, a number of years down the line, when the PSTD sets in, oft times along with worsened attendant behaviors, and when one is no longer under the threat of being sent elsewhere for more remedial behavior modification should one's outlook be significantly less than optimal. But isn't such a later time point more appropriate given the nature of what these programs promise?

I could choose any heart study at random and do the same thing you are.  Lets see the study doesn’t look at patients by nationality… did they consider genetic disorders.. how come they stopped at 15 years?  How many patients smoked weed?  See what I mean?  The questions are endless.  They didn’t take everything into account.  There is no way they could.  The boundary conditions are very well defined in the study, Ursus.

Those parameters do not address the heart (!) of Behrens' study in the least bit, and are certainly not analogous to the concern I brought up re. lack of meaningful time points. You also forgot to bring up your chemotherapy analogies...

Tell me, was this paper ever published? Or was its debut — two and a half weeks shy of the one-year anniversary of Hurricane Katrina — the sole venue for public critique?

Not sure.

I think the study does a good job in showing how well the child does from beginning to end of the program (which was the design).  What you are looking for is how the child does from graduation until say 5 years out.  But how does this directly measure the effectiveness of the program?  What would they compare it to?  I think this would be a great study myself, but I don’t know how it would benefit the industry.  How could they take credit for the success 5 years out.. or the failures 5 years out.  There are so many other factors which effect the child outside the program which are unknown and  out of their control.

I agree that over time well past program it becomes more and more difficult to assess matters, but that *is* when the long term damage generally begins to show up. I imagine that that is when the long term benefits should show up as well, for those individuals who allegedly benefit from these programs. Since the first credo of behavior health professionals is "First, Do No Harm," you would think this would be a concern for Aspen ... given that they market themselves as "behavior health professionals."

[Whooter]

Nope. You sure do make a lot of presumptions 'bout folks, don't you? People getting paid does not bother me. It's when that potentially interferes with the veracity of the results that my fur gets a lil spiky. Does it interfere here? I honestly don't know. I'm bringing it up as a potential concern, that's all.

You brought up the money issue. Most people in the world would assume everyone gets paid for what they do.

Personally, I'm not entirely sure that WIRB is in a position to assess whether this is a credible study or not. Based strictly on population percentages, it's unlikely that any of its personnel have been through a program, and hence would not be likely to spot the studies' blind spots.

Ha,Ha,Ha (I suspect you are joking, but I will bite)  so if someone from WIRB had previously worked for a program you would feel more comfortable?  Lol.  You are concerned if an employee has worked for the industry and then you are concerned if none of the employees have a background in the industry.  So you try to discredit the study either way?

And I'm also not so sure that they are in a position to even be concerned with that issue in the first place. From my apparently less than savvy perspective, it would appear that WIRB's role is mostly focused on making sure that all human subjects have filled out all the proper consent forms, all researchers maintain regulatory compliance, etc. etc.

All these are good questions.  They have vast experience overseeing clinical studies for the medical industry.  I am sure they do a little more than review consent forms.  But I think we both know that.

Those parameters do not address the heart (!) of Behrens' study in the least bit, and are certainly not analogous to the concern I brought up re. lack of meaningful time points. You also forgot to bring up your chemotherapy analogies...

The time points are meaningful to the industry because they define the boundary conditions of the program.  Anything outside of that are subject to other effects not under the control of the program and therefore would not clearly credit the program.
The Chemotherapy analogy I don’t believe applies here.

I agree that over time well past program it becomes more and more difficult to assess matters, but that *is* when the long term damage generally begins to show up. I imagine that that is when the long term benefits should show up as well, for those individuals who allegedly benefit from these programs. Since the first credo of behavior health professionals is "First, Do No Harm," you would think this would be a concern for Aspen ... given that they market themselves as "behavior health professionals."

It’s a much cleaner and more exact study to show how the child has progressed under the programs care.  Once they loss control of the child then how would they assess which external force was responsible for the child going back to being depressed.  Was it an uncle who raped her or a boyfriend who left her.  What controlled group would they compare these kids to?  Could they compare the program kids to the average cross-section of society and compare suicide rates, high school dropout rates?  Would it be accurate to report a suicide as a programs fault if the child killed himself 6 months after graduation?  What if he had been molested by a teacher 3 months after graduation or started on a new medication 5 months after graduation?  Or his parents got divorced right after graduation?  How is this measured?
I think the  “First, Do no harm”  Applies to the time period that they have the child.  There is no reason to believe that these children are being harmed long term.  I have never seen any reports on this, so why would this be considered?  I suspect this was fabricated here on fornits to justify all the 80% of the kids who graduate and move on to college and thus lose track of their progress, assuming to themselves that at some point they will fail.



...



...

[Whooter]

This was one of the significant findings of the study:

What Parents And Their Teens Told Researchers: The majority of teens entering the program were experiencing emotional problems, including higher levels of:
•   Anxiety
•   Depression
 both parents and teens indicated the teens' emotional problems had been reduced to a normal range. Aspen's programs helped teens develop a better sense of self, providing the confidence to be less susceptible to peer pressure and other outside influences.

For anxiety and Depression the kids were brought well within the normal range by graduation time.





...

[Anne Bonney]

I mean how could you accept that programs help over 80% of the kids that enter them.

Yes, how can you possibly accept that percentage as "fact" ... based on exit surveys before the kid is even out of the program. With NO follow-up data. With NO independent oversight. And conducted by a researcher with prior program affiliations.

If they did the study at admission and then 5 years out you wouldn’t accept it either.  You would argue that the kids got better on their own after they left the program.

Why do you insist on telling others what they think and "would say"?  

The study shows the improvements made from admission until the child left.  Did the child improve?  Grade  point average increase/decrease…depression  etc.

Well, of course they "improve" (read, conform) while they're in a program....they have no choice.  It's what all that programming/confrontation/breaking down does to the psyche and the effects of that during the rest of their life that bothers me.

Maybe the rate goes from 80% to 90% after 3 years out.. maybe it goes from 80% to 60% 3 years out.  That wasn't studied here.

That's what we've been trying to tell you for years now.  Glad you can finally admit that there have been no longitudinal studies done.

[Ursus]

I agree that over time well past program it becomes more and more difficult to assess matters, but that *is* when the long term damage generally begins to show up. I imagine that that is when the long term benefits should show up as well, for those individuals who allegedly benefit from these programs. Since the first credo of behavior health professionals is "First, Do No Harm," you would think this would be a concern for Aspen ... given that they market themselves as "behavior health professionals."

It's a much cleaner and more exact study to show how the child has progressed under the programs care. Once they loss control of the child then how would they access which external force was responsible for the child going back to being depressed.

But what is actually being measured is simply the kid's and his or her parents' perception of change between start of program and the conclusion of program. No further. And no deeper.

And while that analysis may be "clean," from a certain perspective, it is also quite shallow. We are not dealing with an analysis on the efficacy of certain vacuum cleaners here (well, perhaps metaphorically speaking, we are!). We're dealing with complex psychological changes that occur in human beings under the duress of a controlled environment. That is, a therapeutic milieu which utilizes thought coercion, and sometimes also physical coercion, to effect a modification of behavior.

I think the "First, Do no harm" Applies to the time period that they have the child.

But that is not the expectation of parents who enroll their children in these programs, is it? After all, we are dealing with "behavioral health" here, and the effects of "therapy" are not expected to miraculously cease the minute the kid is no longer in program, eh?

There is no reason to believe that these children are being harmed long term. I have never seen any reports on this, so why would this be considered? I suspect this was fabricated here on fornits to justify all the 80% of the kids who graduate and move on to college and thus lose track of their progress, assuming to themselves that at some point they will fail.

I beg to differ. *I* feel harmed. Long term. Testimonies of other posters here on fornits attest to same or similar assessments.

The ACE study also attests to long term damage brought about by traumatic childhood events (occurring up to and including age 18 in this study) which have very real, far in the future, long term ramifications on the simple measurable parameters of life-expectancy and frequency of illness or propensity for addiction or other self-destructive behaviors.

[Dr. Acula]

:beat:

[Dr. Acula]

:beat: