Statisticians have built a better way to make sense of competing sports rankings.
Who, really, is the best player in baseball?
Ask five sports media outlets and you may get five different answers. Rankings can vary dramatically from one source to another, change from year to year and leave out different players altogether, making it difficult to determine where there is genuine consensus and where opinions diverge.
Now, Rice University statisticians, in collaboration with Cornell University, have developed a new method designed to find clearer answers in messy rankings data. The research appears in the Journal of Quantitative Analysis of Sports.
The approach, called Bayesian Multivariate Rank Regression, or BMRR, combines rankings from multiple sources and across multiple time points while accounting for disagreements among the rankers, incomplete lists, and factors that might influence the rankings. And just as importantly, the method measures just how certain or uncertain the resulting consensus is.
“Rankings look simple on the surface, but statistically they’re actually very complicated,” says Rose Graves, a doctoral student studying statistics at Rice and the study’s corresponding author.
“Two experts may rank different numbers of items, disagree about the order or even change their opinions over time. We wanted to create a way to bring all of that information together while accounting for uncertainty.”
For their real-world test case, the researchers turned to one of the most closely scrutinized ranking systems in sports: annual lists of Major League Baseball’s best players. They analyzed preseason rankings published by ESPN, CBS, Bleacher Report, Yahoo Sports, and MLB from 2021 through 2024. Each outlet ranked its top 100 players, but the lists varied in both who was included and where players were ranked. To create a consistent pool for comparison, the researchers focused on 55 players who appeared in at least one outlet’s rankings in each of the four years.
Those differences illustrate exactly what makes ranking data difficult to analyze. Simply averaging a player’s position across several lists can mask meaningful information, especially when a player is omitted from one list altogether or when assessments shift substantially from year to year.
BMRR instead treats the rankings as evidence about an underlying value and uses a hierarchical Bayesian framework to combine that evidence. The model can incorporate incomplete rankings and ties, borrow information across years and estimate how closely each source aligns with the overall consensus. Unlike a single definitive list, the result also includes a measure of uncertainty, showing when the data strongly favor one player and when two players are effectively too close to separate.
That distinction matters because rankings can carry real economic consequences. In professional baseball, for example, perceptions of player value can influence trades, salary negotiations, arbitration and draft decisions.
“Rather than asking only who ranks first, this framework lets us ask how confident we are in that conclusion, why someone may be ranked highly, and how much agreement actually exists among the people doing the ranking,” says Marina Vannucci, a professor of statistics at Rice and one of the authors of the paper. “That kind of uncertainty quantification can be extremely important when rankings are being used to inform decisions.”
The model also allowed the researchers to investigate what characteristics were associated with higher rankings. They incorporated three player-level variables: age, base salary and wins above replacement, or WAR, a widely used statistic that estimates a player’s overall contribution to his team.
All three were associated with rankings. Higher WAR had the strongest positive effect, followed closely in magnitude by age with younger players tending to rank more favorably. Higher salaries were also associated with higher rankings. The findings suggest that media rankings reflect not just recent on-field performance but expectations about a player’s future potential and value.
The statisticians’ analysis also produced some recognizable results. For the 2023 season, the model’s four highest-ranked players were Shohei Ohtani, Aaron Judge, Mike Trout, and Mookie Betts. All four were selected as All-Star starters that season, and Ohtani went on to win the American League MVP Award.
Looking across multiple years allowed the researchers to capture something that a one-season ranking cannot: the difference between sustained excellence and a rapid rise or fall.
Ohtani, for example, showed unusually high variability across the four-year period. After injuries affected his earlier seasons, only one of the five outlets included him in its top 100 for 2021. By 2022, two ranked him No. 1, and by 2024 all five placed him in their top 10. The model captured both his eventual ranking near the top and the dramatic trajectory that got him there.
This method can also calculate the probability that one player should rank above another, potentially making it useful for head-to-head decisions. Among four infielders who entered free agency after the 2024 season, for example, the analysis showed strong consensus that Alex Bregman, Pete Alonso, and Paul Goldschmidt ranked above Willy Adames.
BMRR does not only evaluate the players being ranked—it can also reveal how individual rankers behave.
In the baseball analysis, MLB’s own rankings aligned most closely with the model’s overall consensus, followed closely by ESPN. Bleacher Report and Yahoo deviated more substantially from the aggregate rankings, and the authors note that Yahoo’s lists, for example, appeared more inclined to elevate younger, rising players over established veterans. That does not necessarily make one ranking “right” and another “wrong.” Instead, the model provides a quantitative way to see which sources tend to track the broader consensus and which bring a different perspective.
When the researchers tested BMRR on simulated data, it consistently performed well compared with existing rank-aggregation methods. Notably, its advantage was especially pronounced when rankings were incomplete, which is a common real-world scenario in which different people rank only their preferred subset of a much larger group.
While MLB provided a compelling proving ground, the researchers designed BMRR for a much broader problem.
Rankings appear almost everywhere: Search engines order websites, voters rank candidates, experts prioritize intelligence information, and clinicians may rank treatment preferences. Existing statistical approaches often struggle when different raters evaluate different items, when rankings are repeated across multiple criteria or points in time or when analysts want to incorporate additional information about what is being ranked.
“The biggest challenges when working with ranking data are remarkably similar across different fields and industries,” says Dan Kowal, a previous Rice statistics faculty member now at Cornell University and second author on the paper. “We believe that BMRR can be impactful in many of these areas beyond Major League Baseball.”
The authors encourage broader application of their method and have made the code for BMRR available on GitHub, so other statisticians can apply this method to their own data.
“Wherever multiple sources disagree about what belongs at the top, our framework offers a way to identify a consensus while measuring how confident that consensus should be,” Graves adds.
Source: Rice University
