"As recently as last year, AARC described the study as an “
independent outcomes validation study,” according to an AARC funding submission document sent to the Alberta Alcohol and Drug Abuse Commission, which the fifth estate obtained through the province’s freedom of information legislation.
We obtained a version of the 2003 study and showed it to three psychology professors who specialize in addiction—the University of Calgary’s David Hodgins, the University of Lethbridge’s Robert Williams and Bruce Alexander, professor emeritus at Simon Fraser University.
All three raised questions about the way the study was carried out. While Hodgins described the study as “not a bad program evaluation,” he, Williams and Alexander all listed flaws. Among them:
The success rate doesn’t include people who didn’t finish the program.
The grads were interviewed by people linked to AARC. This could bias what was reported, Alexander said. “Imagine calling up somebody who’s graduated from a program and saying: ‘Hey, are you taking drugs any more?’ And this person has already been put in the program against their will perhaps precisely because they took drugs. And what are they going to say? ‘Oh yes, I’m taking lots of drugs now,’” Alexander said.
As for whether the study is “independent,” Williams characterized it as “
semi-independent.” He said in an email, “It is always better to have a totally independent evaluation. However, it is not unusual for ‘in-house’ evaluations to occur.”
http://www.cbc.ca/fifth/2008-2009/power ... study.htmlThe fifth estate also asked the man who AARC says completed the study—Dr. Patton. He told the fifth estate his involvement was largely limited to supervising a graduate student who crunched the data—data gathered by people associated with AARC.
“
I did not conduct the study. They conducted the study. I oversaw the analysis,” he said.
Patton said that while the study was a good preliminary “internal evaluation” with positive results, the next step would be to review AARC’s success rate independently. He noted that the study was rejected for publication by two journals.
“It’s expensive of course to commission an external evaluation. But, that would be the next step. I do remember that the internal evaluation results were quite positive. But, the evaluation that was done did not independently examine the process.
The graduate student that I supervised did not independently talk to any of the young people or the parents. He simply analyzed the data that they sent him. And I was the supervisor of him which is how my name ends up on the report,” Patton said.http://www.cbc.ca/fifth/2008-2009/power ... study.htmlAlberta Hansard April 4, 2006
688
Mr. Speaker, in 2005 an evaluation of the AARC program was
conducted by a noted addiction program evaluation authority, Dr.
Michael Patton, PhD. He studied and interviewed 100 consecutive
cases and found that 85 per cent of the graduates were
still clean and
sober after five years.
http://www.assembly.ab.ca/Documents/isy ... /doc/#hit1Of the sample of 100 graduates, 85 reported being sober and
48 of the 100 were continuously sober since graduation.
http://www.aarc.ab.ca/alumni/AlumAdv_winter05.pdf