Analyzing Predictions from Two Regressions

The post analyzes predictions from two regression models of the relationship between probability of completing school in six years or less and school average SAT scores.  Models were estimate over separate samples of private and public universities inside the state of California.  All data was obtained from the College Scorecard.

Question:  Regression models where on-time graduation rates were explained by an average college wide SAT score were estimated for 31 public  universities and 20 private universities in the state of California.

The dependent variable in this model was log of the odds of the likelihood a person in the college graduates within 6 years of entering the college.

The explanatory variable in the model was the average of four SAT numbers -- the 25^th percentiles SAT verbal and math scores and the 75^th percentile SAT verbal and math scores. 

The results from two regression models are presented below.

On Time Graduation Results
Public Universities
	Coeff	t
SATAVG	0.0122	13.83
_cons	-6.008	-12.95
n	31
F	191.2
Private universities
	Coef.	t
SATAVG	0.0098	10.17
_cons	-4.714	-8.32
n	20
F	103.4

Discuss the implications of the regression results?

Calculate the predicted on-time graduation probability of a public and private college with SAT averages of 550, 600, and 650 consistent with this model.

Are the distribution of SATAVG values the same for public and private institutions?  Compare predicted on-time graduation rates for the median public SATAVG school and the median private SATAVG school.

What other factors might influence on-time graduation rates?

Discussion of Regression Estimates:

Note the constant term for both regression estimates is negative and significantly different from zero.   This is nonsensical because the estimated log odds can’t be negative.   However, note the constant term is the on-time graduation rate when  SATAVG  is 0 and this can never occur because the worse possible SAT score is 200.

The coefficient of  the SATAVG variable in both regressions is positive and significantly different from zero.     The  coefficient for public schools is over 24 % higher than the coefficient for private schools indicating SAT scores have a larger impact on on-time graduation at public schools than at private schools.

Method for obtaining predicted on-time graduation rates:

The  predicted value of the dependent variable, the log of odds, must be converted to the on-time probability.  

First step is to calculate the predicted log of odds which is

P=_cons+b*SATAVG

Second step is to set P equal to the log of odds.  Denote PROB as on-time graduation prob.

log(PROB/(1-PROB))=P

Third step is to solve the above equation for PROB.   Take EXP of both sides and rearrange to get.

PROB= EXP(P)/(1+EXP(P))

On-time Graduation Probability Calculations:

Relationship between on-time graduation and SAT Average for Public and Private Universities
Public Universities
_cons	-6.008	-6.008	-6.008
satavg	0.0122	0.0122	0.0122
SATAVG	550	600	650
P	0.727	1.340	1.952
EXP(P)	2.069	3.817	7.041
PROB	0.674	0.792	0.876
Private Universities
_cons	-4.714	-4.714	-4.714
satavg	0.0098	0.0098	0.0098
SATAVG	550	600	650
P	0.676	1.166	1.656
EXP(P)	1.966	3.209	5.238
PROB	0.663	0.762	0.840

Observations:

Over the SATAVG domain 550 to 650 Public universities have a higher on-time graduation rate for identical SATAVG levels.

Difference in SATAVG for Private vs

 Public Universities

SAT Average Distribution
	Public University	Private University
1%	425	467.5
5%	440	471.25
10%	452.5	482.5
25%	462.5	550.375
50%	507.5	580
75%	547.5	612.5
90%	620	671.25
95%	650	713.75
99%	677.5	737.5

Implications:

Private school SAT scores are consistently higher than public school SAT scores.  It may be more relevant to compare the on-time graduation rates at the same SATAVG percentile for both public and private school.

On-time Graduation Comparisons Public vs Private

at Median SATAVG

Below is a comparison of on-time graduation rates at the 50^th SATAVGs percentile.

Relationship between on-time graduation and SAT Average for Public and Private Universities
	Public Universities	Private Universities
_cons Ceoeff.	-6.008	-4.714
Satavg Coeff.	0.0122	0.0098
Median SATAVG Variable	507.5	580
P	0.207	0.970
EXP(P)	1.230	2.638
PROB	0.552	0.725

Implications:

The probability a person in the median SATAVG private university graduates within six years of starting education is 31 percent higher than a person starting college in the median SATAVG public school.

Caveats:

It would be wrong to assert that the observed differences are entirely related to SAT scores.   Family wealth and income allow people to  stay in school and these variables should also be considered.

The on-time graduation rate involves people who graduate within six years, the measure published on the College Score cared.   Actual on-time graduation from a four year program could arguably be set at four years.   It would be highly useful to estimate models with the four-year graduation rate as the dependent variable.