The Evidentiary Threshold for Instant Replay Reversal is Too High
by Michael Dorf
Late in last week's NCAA Division I men's basketball championship game between Duke and Wisconsin the ball went out of bounds as several players were grasping for it. In real time, it was difficult to tell who had touched it last but the referee judged that it was a Wisconsin player and thus awarded possession to Duke. Because the play occurred within the last two minutes of the game, the referees were able to review it via instant replay. They did so and were unable to tell who touched the ball last. Thus, the original call of Duke ball was left standing because the extraordinarily high standard of Rule 11, Sec. 1, Art. 1 was not satisfied. That rule requires "indisputable evidence that the call on the floor was incorrect" in order for instant replay to change the call.
As it happens, the referees were wrong. There was indisputable evidence that the ball had been last touched by Duke player Justise Winslow--as millions of viewers watching the replay on television saw--but the replay that the referees reviewed did not include the recording from the angle at which the ball's trajectory off of Winslow's finger was evident. That blown call was one of several questionable decisions for Duke by the referees down the stretch of a tight game--they also failed to notice when a Duke player stepped out of bounds while in possession of the basketball and awarded a Duke player a basket and a blocking foul call on a Wisconsin player, when it appeared that the Duke player had committed a charge.
Perhaps Duke would have won anyway even absent the help from the refs. And it's tempting to dismiss the complaints as sour grapes from supporters of a Wisconsin team that would have won if they had hit their free throws and executed better. But looking beyond this one high-stakes game, the blown instant replay on the out-of-bounds play may prompt calls for rules changes designed to enable the referees to use all available replays. That would be sensible.
Here, however, I want to suggest that the use of instant replay is flawed in a more fundamental way. The requirement of "indisputable evidence" strikes me as too stringent, for reasons I shall elaborate. Before doing so, however, I would note that this sort of extremely high threshold--which, on its face, appears to be more than the beyond-a-reasonable-doubt standard for a criminal conviction--can be found in other sports as well. In the NFL, for example, a referee needs "indisputable visual evidence" to overturn the call on the field.
Are these extraordinarily high evidentiary thresholds justifiable? To answer that question, we need to understand the benefits and costs of instant replay.
The chief benefit is obvious: The outcome of a game should turn on the performance of the players rather than the limitations of the referees; even well-trained referees may not be in the right position to make an accurate call and occasionally, even if they are, the limitations of human perception in real time sometimes lead to errors. With instant replay available to viewers at home (and via jumbotrons at the game), it seems downright unfair to permit such fortuities to decide a close contest.
The chief cost of instant replay is that it disrupts the flow of the action. A football team that was driving down the field in a two-minute drill can lose the advantage of tiring the defense. And even when the players themselves are unaffected, fans may become bored waiting for replay officials to rule. Already, American basketball and football take about three times as much time to play the game as the time that officially elapses. Frequent resort to instant replay can exacerbate the problem.
But--and this is my crucial point, indeed my only point--an extremely high evidentiary threshold for reversal is not well calibrated to balance the costs and benefits of instant replay.
Let's begin with football. In the NFL, each team is permitted to request an instant replay (via a coach "challenge") twice during the game, with a third challenge awarded if the first two are successful. A coach who makes an unsuccessful challenge loses a timeout. The numerical limits mean that there is no possibility that coach-initiated replays will interrupt a game more than six times, and they rarely interrupt that many times. In the last two minutes of each half in football, there is the opportunity for "booth review" initiated by an instant replay official rather than by the coaches. Because these reviews only occur in a limited period, they are unlikely to seriously disrupt the flow of the game. And when they do occur, they are likely to be in just those situations in which instant replay is most valuable: on close calls in the crucial moments of close games.
Instant replay in basketball is even more limited. There are no coach-initiated replays, although coaches sometimes attempt to work the refs by screaming for a replay. But as in football, so in basketball, the kinds of plays that can be reviewed are limited, with those limits being somewhat relaxed in the last two minutes of regulation and overtime.
The upshot of all of this is that basketball coaches have no opportunity to call for instant replays and football coaches have no incentive to call for instant replays when they have little chance of prevailing. So far as coaches are concerned, there is no need for a very high standard of proof to prevent instant replay from interrupting the flow of the game.
Accordingly, if the very high evidentiary threshold serves a purpose, it must be to discourage officials from too willingly resorting to instant replay. But how exactly is it supposed to do that? An official will only be tempted to go to instant replay when he has reason to think his call (or the call of a fellow official) may be wrong. Until he actually looks at the replay, he won't have a good sense of whether there is indisputable evidence or any evidence that the call was wrong. So the standard of proof doesn't really bear on whether the official will want to look at the replay.
Meanwhile, the need to find indisputable evidence has two harmful effects. One is obvious: There will be some substantial number of cases in which the referee makes a call at an important juncture of the game, the replay makes pretty clear that the call was wrong, but the evidence is not so overwhelming as to justify changing the call. In these circumstances, the referees will have looked carefully at the replay, typically in slow motion, and yet decided to affirm a call that is likely wrong.
Second, the need for indisputable evidence will sometimes prolong the time necessary for reviewing instant replay. Suppose the officials look at a replay and quickly come to the conclusion that the call on the field or court was wrong. But they are not 100% confident in that conclusion. They now must continue to review the replay from various angles to assure themselves that the evidence of the original call's wrongness is indisputable.
I suspect that the reasoning behind the extraordinarily high evidentiary threshold for instant replay reversal even combined with the other safeguards against instant replay unduly disrupting the flow of the game is a kind of belt-and-suspenders thinking. But because the high threshold itself has costs, that thinking is flawed. Sports that utilize instant replay with the other safeguards of the sort one sees in football and basketball would do better to allow reversal based on a somewhat weaker threshold, perhaps clear-and-convincing evidence or--heretical though it may sound--even a mere preponderance of the evidence.
Late in last week's NCAA Division I men's basketball championship game between Duke and Wisconsin the ball went out of bounds as several players were grasping for it. In real time, it was difficult to tell who had touched it last but the referee judged that it was a Wisconsin player and thus awarded possession to Duke. Because the play occurred within the last two minutes of the game, the referees were able to review it via instant replay. They did so and were unable to tell who touched the ball last. Thus, the original call of Duke ball was left standing because the extraordinarily high standard of Rule 11, Sec. 1, Art. 1 was not satisfied. That rule requires "indisputable evidence that the call on the floor was incorrect" in order for instant replay to change the call.
As it happens, the referees were wrong. There was indisputable evidence that the ball had been last touched by Duke player Justise Winslow--as millions of viewers watching the replay on television saw--but the replay that the referees reviewed did not include the recording from the angle at which the ball's trajectory off of Winslow's finger was evident. That blown call was one of several questionable decisions for Duke by the referees down the stretch of a tight game--they also failed to notice when a Duke player stepped out of bounds while in possession of the basketball and awarded a Duke player a basket and a blocking foul call on a Wisconsin player, when it appeared that the Duke player had committed a charge.
Perhaps Duke would have won anyway even absent the help from the refs. And it's tempting to dismiss the complaints as sour grapes from supporters of a Wisconsin team that would have won if they had hit their free throws and executed better. But looking beyond this one high-stakes game, the blown instant replay on the out-of-bounds play may prompt calls for rules changes designed to enable the referees to use all available replays. That would be sensible.
Here, however, I want to suggest that the use of instant replay is flawed in a more fundamental way. The requirement of "indisputable evidence" strikes me as too stringent, for reasons I shall elaborate. Before doing so, however, I would note that this sort of extremely high threshold--which, on its face, appears to be more than the beyond-a-reasonable-doubt standard for a criminal conviction--can be found in other sports as well. In the NFL, for example, a referee needs "indisputable visual evidence" to overturn the call on the field.
Are these extraordinarily high evidentiary thresholds justifiable? To answer that question, we need to understand the benefits and costs of instant replay.
The chief benefit is obvious: The outcome of a game should turn on the performance of the players rather than the limitations of the referees; even well-trained referees may not be in the right position to make an accurate call and occasionally, even if they are, the limitations of human perception in real time sometimes lead to errors. With instant replay available to viewers at home (and via jumbotrons at the game), it seems downright unfair to permit such fortuities to decide a close contest.
The chief cost of instant replay is that it disrupts the flow of the action. A football team that was driving down the field in a two-minute drill can lose the advantage of tiring the defense. And even when the players themselves are unaffected, fans may become bored waiting for replay officials to rule. Already, American basketball and football take about three times as much time to play the game as the time that officially elapses. Frequent resort to instant replay can exacerbate the problem.
But--and this is my crucial point, indeed my only point--an extremely high evidentiary threshold for reversal is not well calibrated to balance the costs and benefits of instant replay.
Let's begin with football. In the NFL, each team is permitted to request an instant replay (via a coach "challenge") twice during the game, with a third challenge awarded if the first two are successful. A coach who makes an unsuccessful challenge loses a timeout. The numerical limits mean that there is no possibility that coach-initiated replays will interrupt a game more than six times, and they rarely interrupt that many times. In the last two minutes of each half in football, there is the opportunity for "booth review" initiated by an instant replay official rather than by the coaches. Because these reviews only occur in a limited period, they are unlikely to seriously disrupt the flow of the game. And when they do occur, they are likely to be in just those situations in which instant replay is most valuable: on close calls in the crucial moments of close games.
Instant replay in basketball is even more limited. There are no coach-initiated replays, although coaches sometimes attempt to work the refs by screaming for a replay. But as in football, so in basketball, the kinds of plays that can be reviewed are limited, with those limits being somewhat relaxed in the last two minutes of regulation and overtime.
The upshot of all of this is that basketball coaches have no opportunity to call for instant replays and football coaches have no incentive to call for instant replays when they have little chance of prevailing. So far as coaches are concerned, there is no need for a very high standard of proof to prevent instant replay from interrupting the flow of the game.
Accordingly, if the very high evidentiary threshold serves a purpose, it must be to discourage officials from too willingly resorting to instant replay. But how exactly is it supposed to do that? An official will only be tempted to go to instant replay when he has reason to think his call (or the call of a fellow official) may be wrong. Until he actually looks at the replay, he won't have a good sense of whether there is indisputable evidence or any evidence that the call was wrong. So the standard of proof doesn't really bear on whether the official will want to look at the replay.
Meanwhile, the need to find indisputable evidence has two harmful effects. One is obvious: There will be some substantial number of cases in which the referee makes a call at an important juncture of the game, the replay makes pretty clear that the call was wrong, but the evidence is not so overwhelming as to justify changing the call. In these circumstances, the referees will have looked carefully at the replay, typically in slow motion, and yet decided to affirm a call that is likely wrong.
Second, the need for indisputable evidence will sometimes prolong the time necessary for reviewing instant replay. Suppose the officials look at a replay and quickly come to the conclusion that the call on the field or court was wrong. But they are not 100% confident in that conclusion. They now must continue to review the replay from various angles to assure themselves that the evidence of the original call's wrongness is indisputable.
I suspect that the reasoning behind the extraordinarily high evidentiary threshold for instant replay reversal even combined with the other safeguards against instant replay unduly disrupting the flow of the game is a kind of belt-and-suspenders thinking. But because the high threshold itself has costs, that thinking is flawed. Sports that utilize instant replay with the other safeguards of the sort one sees in football and basketball would do better to allow reversal based on a somewhat weaker threshold, perhaps clear-and-convincing evidence or--heretical though it may sound--even a mere preponderance of the evidence.