Advice for the Selection Committee: Please Keep the RPI

The Selection Committee has been inundated with appeals from sports writers and sports aficionados to supplant their use of the Rating Performance Index (or RPI) with what are considered more sophisticated metrics to assess teams for seeding and determine those that are worthy of an at-large bid. They recently launched their own index, termed the NCAA Evaluation Tool (or NET). After a few seasons, we will be able to access its effectiveness in evaluating and comparing team performance.

The Basketball Power Index (or BPI, developed by ESPN), the Sagarin rating (or SAG), and Ken Pomeroy’s rating (KP), all claim to more accurately capture team performance. They also use some proprietary information, limiting their transparency. Given the availability of such ratings, is there useful information to be gleaned from them that is not already contained in the RPI?.

We conducted a pairwise comparison of these ratings using the Spearman correlation coefficient, which provides a measure of the monotonic relationship (moving in the same direction) between any pair of ratings. This can be interpreted to capture whether there is new information in the metrics. A Spearman correlation coefficient of zero means that there is no monotonic relationship between a pair of metrics, while a value of one means that there is a perfect monotonic relationship between a pair of metrics.

The table below gives the Spearman correlation coefficients for all pairwise combinations for the RPI, the BPI, the SAG and the KP, using data from 2012 through 2016. For the pairs BPI and SAG, BPI and KP, and SAG and KP, all the Spearman correlation coefficients were between 0.99 and 1.00. This means that these three metrics are sufficiently related that using more than one provides little new information.

The Spearman correlation coefficient for the RPI with the BPI, the SAG and the KP are all between .96 and .97, suggesting that in spite of the criticism of the RPI that it focuses too heavily on schedule, it is highly correlated with the other three ratings.

What does this all mean? Given that the RPI is simple to understand, easy to compute, and contains non-proprietary information, the Selection Committee would be wise to keep using it as a rating tool of assessing which teams should be given at-large bids and how teams should be seeded. There are always outliers in agreement between the RPI and the other three ratings (while the BPI, SAG, and KP are typically in agreement). However, such outliers are not a reason to abandon the RPI, but rather, a reflection on the limitations of compressing complex performance factors into a single metric.

RPI BPI Sagarin
BPI 0.967 - -
Sagarin 0.968 0.994 -
Ken Pomeroy 0.965 0.991 0.994 ©  2019 - The Board of Trustees of the University of Illinois.
All Rights Reserved.

Web analytics by GoStats