Sample Size Matters

USA Today published a piece on NCAA penalties and the author of the study concludes that the NCAA is harsher on non-BCS schools and on historically black colleges. I have not read the study, and am looking for it on the internet, but it seems as if the NCAA has a legitimate response to the author. Specifically, the NCAA spokeswomen (Stacey Osburn) claims that the author's conclusions "are based upon an inadequate examination of the facts. Specifically, its 'research' is reliant on a very small sample size of a handful of institutions and a methodology that fails to tests the claims against standard statistical criteria". (The quote comes from the article linked above).

As we mention in Chapter 3 of the Wages of Wins, for samples, size matters. It seems that the NCAA is refuting the conclusions due to an inadequately small sample size, which is a legitimate statistical objection. Ms. Osburn also makes a valid objection that standard statistical criteria need to be applied. (i.e. is 2.58 statistically different from 2.74?)

If none of this is done, then we have a lot of smoke - but no fire. So what really needs to be done is for the researcher to demonstrate that there is a real effect going on, and given the researcher is a lawyer - they should know better.

Once I get a hold of the study, I will update this post - if warranted.