Hypothesis tests for the QB performance problem
This post on three different hypothesis tests – the standard test for differences in two populations means, the paired t-test, and the Wilcoxon signed difference test – pertain to my blog on the first-choice versus second-choice QB draft problem.
Question: The previous blog comparing career TD passes from a first-choice and second-choice QB draft pick suggests that while some second-choice picks had stellar careers the first-choice pick tended to out perform the second-choice pick.
What test should be used to determine whether there is a statistically significant difference in career TDs for first-choice and second-choice QBs?
Is this difference statistically significant.
Three statistical tests -- the standard z-score test for differences in means, the paired-difference test, and the Wilcoxon signed difference test --- could be used for this problem.
The standard test:
The most basic statistical test involves comparing the mean touchdowns for first-choice QBs to mean touchdowns for second-choice QBs.
Steps for standard test:
Take means of QB TDS for both first-choice and second-choice QBs
Calculate Standard Error of the difference in mean
Create a test statistic and calculate the p-value
The TTEST command can be used in Excel.
A paired difference test:
Some of the variability in QB career performance stems from the fact that some draft years have several great QBs and other draft years have few good QBs. The paired difference test procedure corrects for variability in QB performance across draft years.
Steps for paired difference test:
Calculate Di the difference in career TDs first-choice minus second choice for each draft year.
Take standard error of Di
Create a test statistics and calculate the p-value.
Again, the TTEST command can be used in Excel. Just specify the paired experiment option
The Wilcoxon signed difference test:
A lot of the variability in career touchdowns and in the difference between first-choice and second-choice QB performance can be attributed to a small number of super stars. The difference between Peyton Manning and Ryan Leaf and the difference between Phil Simms and Jack Thompson is huge. The Wilcoxon signed difference test does not assume the data is normally distributed.
Steps for Wilcoxon signed difference test:
Calculate Di the difference in career TDS choice 1 minus choice 2.
Take absolute value of Di
Exclude all Di=0 from sample
Rank all absolute value of Di.
Add all the ranks for all draft years when Di is positive. (Di is positive when the first-choice QB TD total is larger than second-choice TD total.) Denote this sum W.
The z-score used for the Wilcoxon signed rank test depends on W and its standard error.
Readers who want more info on the Wilcoxon test can go to the following site.
Excel does not have a convenient function for Wilcoxon so I had to build a table in a spreadsheet. For details on my calculation go to
Results for the Career QB Total Example:
Below are the p-values for the three tests -- the standard test on difference in means, the paired t-test, and the Wilcoxon signed difference test.
P-values for difference in first-choice and
second-choice QB touchdowns
Wilcoxon Signed Difference Test
The difference between career QB TDS between QB choice 1 and QB choice 2 is statistically significant for all three tests.
Somewhat surprisingly, the difference is most pronounced for the standard two-sample t-test.