Anonymous Tips, Risks, and Type 1 Errors
by Sherry F. Colb
In my column on Verdict today, part 2 of a 2-part series of columns, I continue my analysis of the U.S. Supreme Court case of Navarette v. California. This is a case in which the Court will decide whether anonymous tips provide a sufficient basis for stopping an allegedly reckless driver, absent corroboration of criminality.
In this post, I want to discuss Chief Justice Roberts's view of this issue and the competing risks of Type 1 and Type 2 errors. The reason we know Chief Justice Roberts's view is that he dissented from the Court's decision four years ago to decline to grant review of a similar case, in Virginia v. Harris. Regarding the notion of ignoring an anonymous tip about a drunk driver, the Chief Justice, joined by Justice Scalia, said the following, in his dissent from the Court's denial of certiorari:
When our existing danger-detection systems fail to identify a true danger, we call that a Type 2 error. For example, we might have a test for breast cancer in which a doctor uses a needle to draw some cells from a breast lump and then analyzes those cells under a microscope. Assume that the test is imperfect and that in some small proportion of cases, the cells that are drawn are normal cells, even though much of the lump is in fact malignant. On those occasions, we would have a Type 2 error or a "miss," because our test failed to detect -- and thus failed to generate a potentially life-saving intervention in -- a cancerous process.
In such a case, Chief Justice Roberts might observe that it would be difficult for the family of the cancer patient to understand, if the patient ultimately dies of breast cancer, why doctors did not simply remove the lump. Why look for more evidence, once doctors have found a lump in a woman's breast? Isn't the removal of the lump substantially less harmful and destructive than a woman's death by breast cancer?
One might, of course, legitimately believe that the answer to this question is yes. However, there is another side to the story and it is one worth examining before concluding that Type 2 errors must be avoided at all costs.
There are Type 1 errors, also known as "false hits." Type 1 errors occur when our detection system identifies a threat when no threat in fact exists. Take, for example, a very sensitive MRI machine, which is used to detect cancer. It may be the case that the machine misses nothing -- that it has no Type 2 errors, in other words. If there is any cancer, it will show up on the MRI. On the other hand, as is often the case with systems that miss nothing, it often makes Type 1 errors (false hits), so that doctors see what may be cancer in various parts of the patient's body, but often, none of the "hits" turns out to actually be cancer.
For someone whose life is saved because a fast-growing cancer, otherwise undetectable, is found through an MRI and later removed, the worry about false hits might seem an indulgence. That patient knows that because of the highly sensitive and reactive test he underwent, he is alive when he would otherwise be dead.
But there are likely to be other patients, perhaps many, many other patients who would feel quite differently. Those are the patients who are told that they might have breast cancer, lung cancer, or kidney cancer, for example. Those patients are then subjected to painful biopsy procedures, each of which gives rise to a risk of infection and, in a small number of cases, a risk of death. With enough false hits, it would plainly be inappropriate to use the MRI machine as an across-the-board diagnostic tool for cancer, despite the fact that its use might, in individual cases, save lives. The morbidity and mortality generated by Type 1 errors can outweigh the benefits of avoiding Type 2 errors.
Could the same be said for anonymous tips? Perhaps. Imagine that whenever the police receive an anonymous tip indicating reckless or drunk driving, the police stop the allegedly offending vehicle, despite the absence of any corroboration (such as observed reckless driving). Such a policy could, if there were many false anonymous tips, lead to a large number of stops of innocent drivers on the highway .
So what?, Chief Justice Roberts might ask. Police would pull over the driver, do a "quick check," and then allow the driver to move on, in that event. This is not necessarily true, however. If police believe that a driver is drunk, based on an anonymous tip, then they might ask the driver to blow into a breathalyzer. The driver might refuse, complaining that he did not do anything wrong and does not trust breathalyzers. What should the police do then? If the suspect does not want to cooperate with them, they might feel compelled to arrest him. After all, if he is in fact drunk, he could hurt someone on the highway. An arrest, in turn, would entail additional invasions of privacy, perhaps including a strip search -- if he is brought to the general population of a jail, under Florence v. Board of Chosen Freeholders.
Traumatized by his experience, even if he is ultimately freed, this driver could develop a sense of distrust toward the police. And with enough people experiencing this undeserved humiliation and trauma, it is easy to see how the police might become less respected, less legitimate in the eyes of the public, and thus less effective at keeping the community safe. To the extent that police must rely on partnerships with communities to keep crime down, such a development could prove catastrophic.
Perhaps this sounds like hyperbole, and perhaps it is. Much depends on the number of false anonymous tips (and of true anonymous tips) that police receive. But imagine, by analogy, that patients began to distrust the medical profession, because of the many false hits that yielded painful, traumatic, and harmful investigative interventions. This imaginary scenario is, in fact, close to reality for many patients, who have suffered the trauma of harmful, excessively invasive, and counterproductive medical interventions. What this could mean is that patients feeling ill or experiencing disturbing symptoms might decide to avoid going to the doctor, for fear of tests, tests, and more tests that could prove more damaging than helpful. If this happened, then the highly sensitive MRI machine and other similar tests could have the paradoxical effect of multiplying the number of Type 2 errors (misses), by driving the patient population away from doctors' offices and from the possibility of an accurate diagnosis and treatment.
With uncorroborated anonymous tips, it is very difficult to know when and how much to trust them. Ignoring them could be extremely costly, and the costs might seem utterly unjustified and inexplicable to those who must pay those costs in the form of highway fatalities. But we must remember, and I hope the Justices remember as well, that there are costs involved in scrupulously stopping everyone accused by an anonymous informants. There is much room for abuse, and the consequences -- for law and order and for public safety -- could conceivably be just as disastrous as the scenario that Chief Justice Roberts envisions. We must therefore tread carefully in this area. The anonymous tip could prove, as promising as it might seem, to be a curse in disguise.
In my column on Verdict today, part 2 of a 2-part series of columns, I continue my analysis of the U.S. Supreme Court case of Navarette v. California. This is a case in which the Court will decide whether anonymous tips provide a sufficient basis for stopping an allegedly reckless driver, absent corroboration of criminality.
In this post, I want to discuss Chief Justice Roberts's view of this issue and the competing risks of Type 1 and Type 2 errors. The reason we know Chief Justice Roberts's view is that he dissented from the Court's decision four years ago to decline to grant review of a similar case, in Virginia v. Harris. Regarding the notion of ignoring an anonymous tip about a drunk driver, the Chief Justice, joined by Justice Scalia, said the following, in his dissent from the Court's denial of certiorari:
"The conflict is clear and the stakes are high. The effect of the rule below will be to grant drunk drivers 'one free swerve' before they can legally be pulled over by police. It will be difficult for an officer to explain to the family of a motorist killed by that swerve that the police had a tip that the driver of the other car was drunk, but that they were powerless to pull him over, even for a quick check."It is difficult not to feel the force of the Chief Justice's argument. An anonymous tip about drunk driving gives the police the opportunity to save someone's life. It calls the officers' attention to a potential menace on the highway, and it would understandably appear inexplicable to the family of a drunk-drivers' victim that police knew about that menace and did nothing to stop it and spare the life of a loved one.
When our existing danger-detection systems fail to identify a true danger, we call that a Type 2 error. For example, we might have a test for breast cancer in which a doctor uses a needle to draw some cells from a breast lump and then analyzes those cells under a microscope. Assume that the test is imperfect and that in some small proportion of cases, the cells that are drawn are normal cells, even though much of the lump is in fact malignant. On those occasions, we would have a Type 2 error or a "miss," because our test failed to detect -- and thus failed to generate a potentially life-saving intervention in -- a cancerous process.
In such a case, Chief Justice Roberts might observe that it would be difficult for the family of the cancer patient to understand, if the patient ultimately dies of breast cancer, why doctors did not simply remove the lump. Why look for more evidence, once doctors have found a lump in a woman's breast? Isn't the removal of the lump substantially less harmful and destructive than a woman's death by breast cancer?
One might, of course, legitimately believe that the answer to this question is yes. However, there is another side to the story and it is one worth examining before concluding that Type 2 errors must be avoided at all costs.
There are Type 1 errors, also known as "false hits." Type 1 errors occur when our detection system identifies a threat when no threat in fact exists. Take, for example, a very sensitive MRI machine, which is used to detect cancer. It may be the case that the machine misses nothing -- that it has no Type 2 errors, in other words. If there is any cancer, it will show up on the MRI. On the other hand, as is often the case with systems that miss nothing, it often makes Type 1 errors (false hits), so that doctors see what may be cancer in various parts of the patient's body, but often, none of the "hits" turns out to actually be cancer.
For someone whose life is saved because a fast-growing cancer, otherwise undetectable, is found through an MRI and later removed, the worry about false hits might seem an indulgence. That patient knows that because of the highly sensitive and reactive test he underwent, he is alive when he would otherwise be dead.
But there are likely to be other patients, perhaps many, many other patients who would feel quite differently. Those are the patients who are told that they might have breast cancer, lung cancer, or kidney cancer, for example. Those patients are then subjected to painful biopsy procedures, each of which gives rise to a risk of infection and, in a small number of cases, a risk of death. With enough false hits, it would plainly be inappropriate to use the MRI machine as an across-the-board diagnostic tool for cancer, despite the fact that its use might, in individual cases, save lives. The morbidity and mortality generated by Type 1 errors can outweigh the benefits of avoiding Type 2 errors.
Could the same be said for anonymous tips? Perhaps. Imagine that whenever the police receive an anonymous tip indicating reckless or drunk driving, the police stop the allegedly offending vehicle, despite the absence of any corroboration (such as observed reckless driving). Such a policy could, if there were many false anonymous tips, lead to a large number of stops of innocent drivers on the highway .
So what?, Chief Justice Roberts might ask. Police would pull over the driver, do a "quick check," and then allow the driver to move on, in that event. This is not necessarily true, however. If police believe that a driver is drunk, based on an anonymous tip, then they might ask the driver to blow into a breathalyzer. The driver might refuse, complaining that he did not do anything wrong and does not trust breathalyzers. What should the police do then? If the suspect does not want to cooperate with them, they might feel compelled to arrest him. After all, if he is in fact drunk, he could hurt someone on the highway. An arrest, in turn, would entail additional invasions of privacy, perhaps including a strip search -- if he is brought to the general population of a jail, under Florence v. Board of Chosen Freeholders.
Traumatized by his experience, even if he is ultimately freed, this driver could develop a sense of distrust toward the police. And with enough people experiencing this undeserved humiliation and trauma, it is easy to see how the police might become less respected, less legitimate in the eyes of the public, and thus less effective at keeping the community safe. To the extent that police must rely on partnerships with communities to keep crime down, such a development could prove catastrophic.
Perhaps this sounds like hyperbole, and perhaps it is. Much depends on the number of false anonymous tips (and of true anonymous tips) that police receive. But imagine, by analogy, that patients began to distrust the medical profession, because of the many false hits that yielded painful, traumatic, and harmful investigative interventions. This imaginary scenario is, in fact, close to reality for many patients, who have suffered the trauma of harmful, excessively invasive, and counterproductive medical interventions. What this could mean is that patients feeling ill or experiencing disturbing symptoms might decide to avoid going to the doctor, for fear of tests, tests, and more tests that could prove more damaging than helpful. If this happened, then the highly sensitive MRI machine and other similar tests could have the paradoxical effect of multiplying the number of Type 2 errors (misses), by driving the patient population away from doctors' offices and from the possibility of an accurate diagnosis and treatment.
With uncorroborated anonymous tips, it is very difficult to know when and how much to trust them. Ignoring them could be extremely costly, and the costs might seem utterly unjustified and inexplicable to those who must pay those costs in the form of highway fatalities. But we must remember, and I hope the Justices remember as well, that there are costs involved in scrupulously stopping everyone accused by an anonymous informants. There is much room for abuse, and the consequences -- for law and order and for public safety -- could conceivably be just as disastrous as the scenario that Chief Justice Roberts envisions. We must therefore tread carefully in this area. The anonymous tip could prove, as promising as it might seem, to be a curse in disguise.