Hypothesis tests for the QB performance problem
This post on three different hypothesis tests – the standard
test for differences in two populations means, the paired ttest, and the
Wilcoxon signed difference test – pertain to my blog on the firstchoice versus
secondchoice QB draft problem.
Question: The previous blog comparing career TD passes
from a firstchoice and secondchoice QB draft pick suggests that while some
secondchoice picks had stellar careers the firstchoice pick tended to out
perform the secondchoice pick.
What test should be used to determine whether there is a
statistically significant difference in career TDs for firstchoice and
secondchoice QBs?
Is this difference
statistically significant.
Methodological
Discussion:
Three statistical tests
 the standard zscore test for differences in means, the
paireddifference test, and the Wilcoxon signed difference test  could be
used for this problem.
The standard test:
The most basic statistical test involves comparing the mean
touchdowns for firstchoice QBs to mean touchdowns for secondchoice QBs.
Steps for standard
test:
Take means of QB TDS for both firstchoice and secondchoice
QBs
Calculate Standard Error of the difference in mean
Create a test statistic and calculate the pvalue
The TTEST command can be used in Excel.
A paired difference
test:
Some of the variability in QB career performance stems from
the fact that some draft years have several great QBs and other draft years
have few good QBs. The paired
difference test procedure corrects for variability in QB performance across
draft years.
Steps for paired
difference test:
Calculate D_{i} the difference in career TDs
firstchoice minus second choice for each draft year.
Take standard error of D_{i}
Create a test statistics and calculate the pvalue.
Again, the TTEST command can be used in Excel. Just specify the paired experiment option
The Wilcoxon signed
difference test:
A lot of the variability in career touchdowns and in the
difference between firstchoice and secondchoice QB performance can be attributed
to a small number of super stars. The
difference between Peyton Manning and Ryan Leaf and the difference between Phil
Simms and Jack Thompson is huge. The
Wilcoxon signed difference test does not assume the data is normally
distributed.
Steps for Wilcoxon
signed difference test:
Calculate D_{i} the difference in career TDS choice
1 minus choice 2.
Take absolute value of D_{i}
Exclude all D_{i}=0 from sample
Rank all absolute value of D_{i}.
Add all the ranks for all draft years when D_{i }is
positive. (D_{i }is positive
when the firstchoice QB TD total is larger than secondchoice TD total.) Denote this sum W.
The zscore used for the Wilcoxon signed rank test depends
on W and its standard error.
Readers who want more info on the Wilcoxon test can go to
the following site.
Excel does not have a convenient function for Wilcoxon so I
had to build a table in a spreadsheet.
For details on my calculation go to
Results for the Career QB Total Example:
Below are the pvalues for the three tests  the standard
test on difference in means, the paired ttest, and the Wilcoxon signed
difference test.
Pvalues for difference
in firstchoice and
secondchoice QB
touchdowns


pvalue


TwoSample ttest

0.019

Paired Ttest

0.033

Wilcoxon Signed
Difference Test

0.043

Onetailed pvalues.
The difference between career QB TDS between QB choice 1 and
QB choice 2 is statistically significant for all three tests.
Somewhat surprisingly, the difference is most pronounced for
the standard twosample ttest.
No comments:
Post a Comment