NET versus RPI: Does it Matter?

The Selection Committee has been inundated with appeals from sports writers and sports aficionados to supplant their use of the Rating Percentage Index (or RPI) with what are considered more sophisticated metrics to assess teams for seeding and determine those that are worthy of an at-large bid. In the summer of 2018, the NCAA launched their own index, termed the NCAA Evaluation Tool (or NET). After a few seasons, we will be able to access its effectiveness in evaluating and comparing team performance.

The Basketball Power Index (or BPI, developed by ESPN), the Sagarin rating (or SAG), and Ken Pomeroy’s rating (KP), all claim to more accurately capture team performance. They also use some proprietary information, limiting their transparency. Given the availability of such ratings, is there useful information to be gleaned from them that is not already contained in the RPI?.

To uncover the benefits of and differences between the various metrics, we conducted a data-driven analysis by making a pairwise comparison of these metrics using the Spearman correlation coefficient, which provides a measure of the monotonic relationship (moving in the same direction) between any pair of metrics. This can be interpreted to capture whether there is new information in the metrics. A Spearman correlation coefficient of zero means that there is no monotonic relationship between a pair of metrics, while a value of one means that there is a perfect monotonic` relationship between a pair of metrics. In general, the closer the Spearman correlation coefficient is to one, the less new information is provided from having an additional metric.

To assess the relationship between these metrics for the 2018-2019 season, we computed their Spearman correlation coefficients. These values are updated weekly as we approach Selection Sunday. We expect that as the season progresses, and there are more data points incorporated into the RPI values, the RPI Spearman correlation coefficients with the other metrics will all converge to the historical values (see table below.) For the 2019 Spearman correlation coefficients, click here .

To provide a historical perspective, the table below gives the Spearman correlation coefficients for all pairwise combinations for the RPI, the BPI, the SAG and the KP, using data from 2012 through 2016. For the pairs BPI and SAG, BPI and KP, and SAG and KP, all the Spearman correlation coefficients were between 0.99 and 1.00. This means that these three metrics are sufficiently related that using more than one provides little new information.

The Spearman correlation coefficient for the RPI with the BPI, the SAG and the KP are all between .96 and .97, suggesting that in spite of the criticism of the RPI that it focuses too heavily on schedule, it is highly correlated with the other three ratings.

What does this all mean? Given that the RPI is simple to understand, easy to compute, and contains non-proprietary information, the Selection Committee would be wise to keep using it as a rating tool of assessing which teams should be given at-large bids and how teams should be seeded. There are always outliers in agreement between the RPI and the other three ratings (while the BPI, SAG, and KP are typically in agreement). However, such outliers are not a reason to abandon the RPI, but rather, a reflection on the limitations of compressing complex performance factors into a single metric.

Historical Analysis: 2012-2016

 

bracketodds.cs.illinois.edu ©  2019 - The Board of Trustees of the University of Illinois.
All Rights Reserved.