Science v. Subjectivity

Selecting College Football Playoff Teams as a Case Study

BY MARC J. DEFANT

If you are not familiar with how college football determines the four teams that are picked to contend for the national championship, I refer you to the Selection Committee Protocol which is a guide on how the committee chooses the four playoff teams at the end of the regular season and after the league championship games. The first words of the protocol are telling: “Ranking football teams is an art, not a science.” The protocol specifically calls into question any rigorous mathematical approach: “Nuanced mathematical formulas ignore some teams who ‘deserve’ to be selected.” Deserve?

So what are the guidelines that instruct the 13- member college playoff panel? They are somewhat obvious and include “conference championship wins, strength of schedule, head-to-head competition, comparative outcomes of common opponents, and other relevant factors such as key injuries that may have affected a team’s performance during the season or likely will affect its postseason performance.” I hasten to point out that strength of schedule can only be determined by “nuanced mathematical” rigor. The guidelines fall into two categories: facts (e.g., conference champions) and opinions (e.g., whether a key injury will impact team performance). My argument is to eliminate opinions and choose the final 4 teams in the most rational and unbiased fashion—that is, use computer algorithms. Exceptions to the computer rankings could be made by the committee when facts like conference championships play an important role.

Informed opinions can be important and a group of football experts might have insights into the game that mere mortals might not. Many of the committee members are former coaches and athletic directors, but I am concerned that their opinions might be influenced by the teams and conferences they come from. (They probably don’t even know they are biased.)

A massive amount of scientific research shows we have difficulties being unbiased. Nobel laureate Daniel Kahneman has written an entire book on heuristic and cognitive biases, Thinking, Fast and Slow.1 A good example comes from witnesses of crimes or traffic accidents. Any good detective knows to take eyewitness testimonies before the witnesses have had a chance to discuss the event because studies show that witnesses who share information will tend to foster similar errors about the event. Research also shows that eyewitnesses are notoriously inaccurate.

Psychologist Elizabeth Loftus has written an immensely entertaining book about her research involving witness error and bias, The Myth of Repressed Memories.2 She writes: “In my experiments, conducted with thousands of subjects over decades, I’ve molded people’s memories, prompting them to recall nonexistent broken glass and tape recorders; to think of a clean-shaven man as having a mustache, of straight hair as curly, of stop signs as yield signs, of hammers as screwdrivers; and to place something as large and conspicuous as a barn in a bucolic scene that contained no buildings at all. I’ve been able to implant false memories in people’s minds, making them believe in characters who never existed and events that never happened.” A recent study based on statistical analyses has shown that the writers’ and coaches’ college football polls are significantly affected by such things as point spreads and television coverage.3