So I was taking a look at this NY Times/CBS Poll about whether the Edwards pick helped Kerry. Now without any comment on the poll numbers themselves, I really want to comment on this little link they throw in the bottom of the article which talks about How the Poll Was Conducted (Emphasis Added).

Residents of 21 political swing states were oversampled for analysis and then weighted to their proper share of the population.

Based on the 2000 presidential vote, residents of heavily Republican counties, heavily Democratic counties and politically competitive counties were weighted to their proper share of the population.

In theory, in 19 cases out of 20, overall results based on such samples will differ by no more than three percentage points in either direction from what would have been obtained by seeking out all American adults.

Huh? Can someone please tell me what that bold text really means in terms of sampling? It essentially is saying that "Red Counties" and "Blue Counties" should be spread evenly across the country, and not really count for anything extra. That is basically saying that there are an exact even number of "Red Counties" as there are "Blue Counties". Isn't that an inherently flawed assumption?

Why were swing states oversampled at all? What you end up doing is pre-ordaining the results to be close to the center. Swing states are by definition evenly divided. So by oversampling them, and then spreading those results across the country as a whole, you're saying that the general poplulation is evenly divided as well. While that may be true, you are not discovering that by the poll. You're letting your previous assumption about the results affect the method in which you conduct the poll, getting the result you expected. What a shock.

There is another possibly flawed assumption here as well. That "Red Counties" and "Blue Counties" in 2000 will continue to remain respectively Red and Blue. Is that valid? A lot has happened in 4 years, and I'm not really sure if that's a very good assumption to make. Worse yet is the fact that they word their methodology in a mathematically vague way. They simply say, "We adjusted for it properly" without saying exactly how they did it.

That's why I have a hard time with a lot of polls.