The NCAA was good enough to host me and a couple dozen other members of the media last week at a mock selection exercise in Indianapolis. Armed with my newfound knowledge I will, some day soon, make good on my threat to a terrified general public to post a piece that will be much too long on the Ratings Percentage Index.
This state of affairs is ironic, given that at least part of the impetus behind the NCAA putting media types through such an exercise is to demonstrate to people “outside the room” that the RPI doesn’t really play such a large role in the selection process after all. So before I respond to that impetus with a few thousand words on what the NCAA clearly thinks is already the subject of way too many words, I wish to be clear on three points.
The mock selection exercise does what it’s intended to do: it triggers epiphanies.
Having been “in the room,” even though it was a fake room (well, the room was real — our bracket’s fake), I now know that the institutional memory and knowledge that NCAA staffers hold collectively is absolutely pivotal to the challenge of working under severe if not draconian time constraints to select and seed a 68-team field for an event that has a $10.8 billion television deal. To choose one mundane but nevertheless vital example of that memory and knowledge, those staffers have a really cool piece of bracketing software that makes voting a breeze for committee members. Those staffers regard themselves, quite rightly, as experts. Show me someone else on the planet that’s done what they’ve done this many times with the stakes this high. That being said, I do worry at times about the voice of those experts prevailing over those of the committee members, particularly newbies. When I’m committee chair (I’m preparing my campaign brochure as we speak), one of my biggest challenges will be to remind NCAA full-timers, collegially yet firmly, that just because they’re in the room doesn’t mean they’re on the committee.
Seen at the macro level, the NCAA does an outstanding job with selection.
Drawing a distinction between selection on the one hand, and bracketing and seeding on the other, I have already stated my admiration for how the NCAA goes about things:
I have no problem at all with how the Committee selects the field. Selection comes down to making a decision that no person, computer, or group of people should ever have to make: pulling up the drawbridge after Team No. 68 comes in, even though Team No. 69 will always look just as good. I think the only way to meet that challenge is precisely the way the committee does it: get a group of well-respected people in a room and have them ponder the problem at length. I won’t always agree with the decision that results, of course, but for my money that’s the only way to reach the decision. The process itself confers legitimacy.
You will always be able to make an outstanding factual case for at least one team that doesn’t get in. Literally, always. If Team A that got left out is from a major conference, the chances are very good you’ll be able to find a Team B that got in and say, correctly, “Team A went 2-0 against Team B!” If Team X that got in is from a mid-major conference and is not named “Long Beach State,” chances are excellent that you’ll be able to say, correctly, “Team X’s strength of schedule was nowhere close to Team B’s!” These statements will always be available when just 19.7 percent of Division I programs are waved through to the field of 68. I guess the statements should still be voiced, but we who hear those statements should understand going in that their availability is hard-wired into a situation that’s been presented to and not created by a given year’s committee. There are just 68 slots for 345 teams.
Seen at the micro level, the NCAA is right: The RPI plays a smaller role than generally believed.
Committee members do not sit around saying, “Let’s put Team B in the bracket because their RPI is so good.” But of course part of the reason that outside observers make such a big deal about the RPI has less to do with the relative weight that generic Metric X should have in the process, and more to do with the specific evaluative failings that the RPI does have. To this the NCAA responds, in effect, hey, all rating systems have failings. This is correct. The logical course of action, as suggested by no less an authority than Nate Silver to the NCAA at the mock exercise, would therefore be to use an index comprised of multiple rating systems. That would be logical, however, I don’t expect the NCAA to adopt that course of action. I will explain why in my much too long piece on the RPI.