Another great AALL program! -- Law School Rankings

I really recommend buying the tape for G-2, The Ratings Game - US News and World Report, with Billie Jo Kauffman, Rebecca Trammel, Dean Theodore Seto of Loyola, Marymount, and Jack Crittenden, publisher of National Jurist. Prof. Seto’s excellent talk was the core of the program. He has written an in-depth analysis of the U.S. News rankings system. A tax professor, he brings an understanding of statistical systems and the effects of weighting that really inform a better understanding of what are the “pressure points” in the rankings.

See his article, Understanding the U.S. News Law School Rankings, which will appear soon at 60 SMU L. Rev. __ (2007) or at http://ssrn.com/abstract=937017 Among other interesting findings, we heard that the rankings are very sensitive to tiny changes in certain variables – this is completely counter to what appears to be an immoveable rank when we try to advance. Prof. Seto played with inserting small changes in the raw scores. He showed how a change in the bottom ranked school’s score reshuffles all the schools above, sometimes by as much as 5 ranks, without those schools changing anything in their score. This is because the ranking system uses a very rigid scaling process. After the school are all scored on the variables, the top school is arbitrarily given an adjusted score of 100, and the bottom-ranked school, an adjusted rank of 0. Then all the schools in between are given an adjusted score, which is rounded to the nearest whole number.

The most sensitive and easily affected (or even gamed) variable is the 9-month employment number. A 1% change in the bottom school’s score there can jump middle-level schools’ ranks by 5 slots, without any change at all in those schools’ scores. This leads to criticism of the validity of the rankings (from yet another angle). Read the article in full, and buy the tape because there is a lot of interesting commentary. The final recommendation is to cultivate many other ways to rank law schools. Speakers told us that there are 6 different ranking systems for MBA programs, with the result that none of them distort the schools’ practices the way law school policies are impacted by the US News rankings.

Rebecca Trammel prepared a useful bibliography which is available at both Stetson’s website and at American University ‘s site. I recommend visiting and picking up the bibliography, which I don’t think was in the handouts, in order for it to be as up-to-date as possible.