Historically, pollsters have relied on samples drawn from
lists of people with landlines. Many
pollsters are now relying on a combined landline and cell phone list.

My initial thoughts on this approach are presented through a
review of a PEW research study comparing landline and dual-frame election
polls.

The Growing Gap between Landline and Dual Frame Election
Polls:

**Summary of Study**: The PEW study looked at difference in poll outcomes for a poll based on a sample from a list of people with landlines and a sample based on a combined list of people with landlines and cell phones. PEW looks at the 2008 Presidential elections and the 2010 general Congressional results.

*Main findings of the PEW study*:

The 2008 election combined sample had the Obama-McCain
margin at 8.2 percentage points. The
landline only sample had this margin at 5.8 percentage points.

The 2010 poll results gave Republican candidates a 7.6
percentage point lead with the combined sample and a12.7 percentage point lead
with the landline sample.

The 2010 analysis considers political preferences of four
groups – landline only, duals from landline, duals from cell, and cell
only. Groups from most to least Democratic are cell
only, landline only, duals from cell, and duals from landline.

**Some Comments**:

**Comment One**: The actual Obama versus McCain election differential in 2008 was 7.3 percentage points. This outcome is less than the margin predicted by the combined sample and more than the outcome predicted by the landline-only sample.

**Comment Two**: In my view, there is no such thing as a combined sample. We have two samples one drawn from the landline list and the other drawn from the cell list. This is not a semantic point.

Simply basing estimates on the combined sample means that
the importance of cells versus landlines is entirely determined by the relative
size of the two samples.

Since landline respondents tend to be Republican a
landline-only poll is biased for Republicans.
Since cell respondents tend to be Democratic the cell-only poll is biased
for Democrats.

An unbiased poll would be a weighted average of the two
polls with weights equal to the proportion of landline and cell phone voters.

**Comment Three**: The use of a combined sample leads to an incorrect (too small) estimate of the standard error. The standard error of the results should be based on the sum of variance of the two samples not a single combined sample.

(I’ll work out a simple math example on this point next
week. Intuitively, the variance of the
combine sample is the sum of the variance of all random binary individuals
outcomes in the combined sample. The
variance of the separate landline and cell phone surveys is the sum of the two
variances. Sample size of both separate
samples are larger so the variance of the two separate samples is larger than
the variance of the combined sample.)

The use of the correct method of calculating the standard
error for combine samples is especially important for analysis based on a single
poll with likely voters only because the sample size for likely voters with
cell phone only is quite small.

**Comment Four:**The 2010 results involve an aggregate of many elections. Hence it is difficult to compare 2010 landline/cell versus landline only results to actual outcomes.

**Comment Five**: The finding of difference between political preferences of duals reached by landline and duals reached by cell phone is difficult to evaluate without more information on why some people are being reached by landline and others are being reached by cell phone.

**Comment Six**: Duals from landline are by far the largest of the four groups. The next largest group duals from cell is 28% its size. These polls were still dominated by landline owners and users.

**Final Comment**: A lot of pollsters got it wrong in 2012, partially because they didn’t fully account for the growth of cell phone use. Many of these pollsters chose to move towards the combined landline/cell phone technique. This fix is inadequate.

The number of people dropping their cell phones continues to
grow. I have some interesting data on
this point that I plan to share either tomorrow or the next day depending on
family obligations.

## No comments:

## Post a Comment