The survey results are absolutely worthless. The respondents are entirely self-selected, as we can see by the propensity of respondents to rate their experiences zeros or fives. Only the people who felt strongest on the issue responded, and the places the survey was advertised strongly affected the number of people who were aware of it.
In addition, research has been done on surveys and polls that compares the results when the survey uses self-selected respondents, versus when the respondents are a true random sample. The research has proven that you get a lot more people who agree with or are positive about the subject of the survey when you let respondents self-select than when you have a random sample.
In Lon's survey, this means that all the available research predicts that his self-selected sample has more people in it that like the Programs than a random sample would show.
You can't say anything statistically about anything if you don't have more than 35 data points. Even then, your conclusions are very, very weak unless your data shows a very strong relationship. If it's not a very strong relationship, then it's very likely that any "trend" you think you found is just random statistical noise, and the real trend could go entirely the other way.
With 52 "data points" and all of those self-selected and at the extremes, Lon's "survey" is a piece of crap.
He'd do far better to get a research grant and commission a professional survey firm to do a large, national poll by mail. Ask people to give the name of the place and years they attended purely for verification, and pay 'em $20 to return the survey (provided there really was a school there for those years). Encourage them to reply by having the survey sheet on a separate sheet of paper from the name and address needed to send them their check. National polling firms, if you tell them *not* to do a push poll but actually find out the real data, are very good at what they do.
When you see polls on TV that turn out to have horrible data, it's either because the methodology was crap (like Lon's survey) in the first place, or because somebody has commissioned a polling firm to do a "push poll" that pushes respondents towards the opinion the people buying the poll want them to have.
When you pay the professionals to do you an honest poll, you usually get pretty good data.
Thing is, in my opinion Lon's choices in how to put together this survey demonstrates that he doesn't want good data. He wants to get what he wants to hear.
Julie
