## Sunday, May 19, 2019

### Analyzing Predictions from Two Regressions

The post analyzes predictions from two regression models of the relationship between probability of completing school in six years or less and school average SAT scores.  Models were estimate over separate samples of private and public universities inside the state of California.  All data was obtained from the College Scorecard.

Question:  Regression models where on-time graduation rates were explained by an average college wide SAT score were estimated for 31 public  universities and 20 private universities in the state of California.

The dependent variable in this model was log of the odds of the likelihood a person in the college graduates within 6 years of entering the college.

The explanatory variable in the model was the average of four SAT numbers -- the 25th percentiles SAT verbal and math scores and the 75th percentile SAT verbal and math scores.

The results from two regression models are presented below.

 On Time Graduation Results Public Universities Coeff t SATAVG 0.0122 13.83 _cons -6.008 -12.95 n 31 F 191.2 Private universities Coef. t SATAVG 0.0098 10.17 _cons -4.714 -8.32 n 20 F 103.4

Discuss the implications of the regression results?

Calculate the predicted on-time graduation probability of a public and private college with SAT averages of 550, 600, and 650 consistent with this model.

Are the distribution of SATAVG values the same for public and private institutions?  Compare predicted on-time graduation rates for the median public SATAVG school and the median private SATAVG school.

What other factors might influence on-time graduation rates?

Discussion of Regression Estimates:

Note the constant term for both regression estimates is negative and significantly different from zero.   This is nonsensical because the estimated log odds can’t be negative.   However, note the constant term is the on-time graduation rate when  SATAVG  is 0 and this can never occur because the worse possible SAT score is 200.

The coefficient of  the SATAVG variable in both regressions is positive and significantly different from zero.     The  coefficient for public schools is over 24 % higher than the coefficient for private schools indicating SAT scores have a larger impact on on-time graduation at public schools than at private schools.

Method for obtaining predicted on-time graduation rates:

The  predicted value of the dependent variable, the log of odds, must be converted to the on-time probability.

First step is to calculate the predicted log of odds which is

P=_cons+b*SATAVG

Second step is to set P equal to the log of odds.  Denote PROB as on-time graduation prob.

log(PROB/(1-PROB))=P

Third step is to solve the above equation for PROB.   Take EXP of both sides and rearrange to get.

PROB= EXP(P)/(1+EXP(P))

 Relationship between on-time graduation and SAT Average for Public and Private Universities Public Universities _cons -6.008 -6.008 -6.008 satavg 0.0122 0.0122 0.0122 SATAVG 550 600 650 P 0.727 1.340 1.952 EXP(P) 2.069 3.817 7.041 PROB 0.674 0.792 0.876 Private Universities _cons -4.714 -4.714 -4.714 satavg 0.0098 0.0098 0.0098 SATAVG 550 600 650 P 0.676 1.166 1.656 EXP(P) 1.966 3.209 5.238 PROB 0.663 0.762 0.840

Observations:

Over the SATAVG domain 550 to 650 Public universities have a higher on-time graduation rate for identical SATAVG levels.

Difference in SATAVG for Private vs
Public Universities

 SAT Average Distribution Public University Private University 1% 425 467.5 5% 440 471.25 10% 452.5 482.5 25% 462.5 550.375 50% 507.5 580 75% 547.5 612.5 90% 620 671.25 95% 650 713.75 99% 677.5 737.5

Implications:

Private school SAT scores are consistently higher than public school SAT scores.  It may be more relevant to compare the on-time graduation rates at the same SATAVG percentile for both public and private school.

On-time Graduation Comparisons Public vs Private
at Median SATAVG

Below is a comparison of on-time graduation rates at the 50th SATAVGs percentile.

 Relationship between on-time graduation and SAT Average for Public and Private Universities Public Universities Private Universities _cons Ceoeff. -6.008 -4.714 Satavg Coeff. 0.0122 0.0098 Median SATAVG Variable 507.5 580 P 0.207 0.970 EXP(P) 1.230 2.638 PROB 0.552 0.725

Implications:

The probability a person in the median SATAVG private university graduates within six years of starting education is 31 percent higher than a person starting college in the median SATAVG public school.

Caveats:

It would be wrong to assert that the observed differences are entirely related to SAT scores.   Family wealth and income allow people to  stay in school and these variables should also be considered.

The on-time graduation rate involves people who graduate within six years, the measure published on the College Score cared.   Actual on-time graduation from a four year program could arguably be set at four years.   It would be highly useful to estimate models with the four-year graduation rate as the dependent variable.