Empirical Suppositions in Supreme Court Decisions: Some Stray Thoughts
-- Posted by Neil H. Buchanan
Professor Dorf's post yesterday, "Empirical Scholarship In and Around Constitutional Law," described an important new project (and an associated conference that is taking place today) that attempts to make empiricism a more important part of constitutional legal scholarship. The idea is that, among all of the subjects addressed by legal scholars, constitutional law seems to be the one that has been least affected by formal empirical studies.
As Professor Dorf describes, con law's isolation from serious empirical research can hardly be explained by any claim that empirical matters are simply irrelevant to constitutional analysis. Notwithstanding the intuition that constitutional matters are theoretical, not factual, in reality we frequently (one might even say always) find that there are important empirical presumptions underlying constitutional decisions, from the Supreme Court on down.
For example, the paper that Professor Dorf and his co-author, Professor Brandice Canes-Wrone, are writing (soon to be summarized here on Dorf on Law) looks at the concept of "chilling effects," in this case whether bans on certain types of abortions change people's behavior with respect to activities not directly affected by such bans. If we were to find that such activity is not changed, then constitutional concerns about chilling behavior would be merely theoretical; but if people do act differently, then legislatures and courts should certainly take those unintended effects into account.
Thinking about yesterday's post, I found myself putting together a growing list of constitutional decisions from the Supreme Court that clearly rely on empirical suppositions by the justices about the world, even though their own contact with the world is notoriously "special." The famous story of a Supreme Court justice asking a passing stranger if he could borrow the stranger's pocket copy of the Constitution comes to mind. Supreme Court justices (and other judges, and legislators) often presume that they know what "normal people" think and do, but they are often quite wrong.
Consider a number of examples, from different areas of constitutional law. First, the line of cases that determine when a person has been "detained" by the police is rife with assertions from the justices about how a person would reasonably act when confronted by an officer. The most preposterous empirical statement from the Court came in a case in which officers had boarded a bus and were moving down the aisle, asking passengers questions. The Court's decision turned on whether the officers had physically blocked the aisle in a way that would stop a passenger from getting up and walking off the bus, because supposedly a reasonable person would know that he is otherwise not, in fact, detained -- and would feel perfectly comfortable getting off the bus without answering questions. Although one might defend the majority's statement as simply saying what a person should know under the Constitution, in fact the claim was entirely an empirical one: We do not need to worry about liberty being violated, because people who do not want to answer questions voluntarily will not respond to being confronted by officers by altering their behavior as if they had actually been told not to leave. As I heard one con law scholar put it after reading that case: "What planet does the Supreme court live on?"
But that question is only relevant if people on different planets in fact behave differently, which is an empirical question. The second example, from the late 1980's (if I recall correctly), had the Supreme Court deciding that trash that had been put out on the curb to be picked up was fair game to be searched by police officers. The Court's rationale explicitly included the supposition that people would know that their trash is no longer their private property (because they have voluntarily disposed of it), and thus that the police could search it. Is that what real people actually believe? I recall an economist predicting at the time that there would be a surge in sales of shredders, specifically because people would be shocked to learn that their trash was not protected by privacy doctrines. No one, as far as I know, carried out an empirical study of that claim, but the point is that the Supreme Court's majority blithely asserted something as fact that struck most people as simply "not what real people would think."
A slightly different issue is raised by a third example. In a case in the 1980's, the Court allowed a school district to censor a high school student newspaper. (I apologize for not having case names and citations for these cases, but I assume that interested readers can track them down.) I believe it was Justice White who wrote the opinion in which the Court held that the school's administrators had a reasonable concern that the views expressed in the student newspaper could be attributed to the school district itself. At the time, my immediate thought was that we see disclaimers all the time: "The views expressed herein do not necessarily represent the views of ... ." Although the Court did not even go that far in its analysis, it strikes me in the current context that there is an interesting pair of empirical questions here: (1) Without a disclaimer, would people really believe that what students write in a school newspaper expresses the views of the principal and superintendent? and (2) If so, would a disclaimer change people's views? But the Court was unconcerned, because they were willing to make an unexamined and unsupported assumption about reality.
This phenomenon is present also in cases where I happen to agree with the outcome. The Court's now-great Windsor decision invaliding the Defense of Marriage Act includes a lulu of an empirical claim by Justice Kennedy about how people think. He claimed that the children of same-sex couples are psychologically harmed by knowing that the state does not treat their families with the same respect with which it treats the families of opposite-sex couples. Although I (and the judges who have relied on Kennedy's soaring language in Windsor to invalidate bans on same-sex marriage) find that claim to be completely believable, how did Kennedy know? Maybe these children are simply unaware of the difference, or they do not care. Maybe they even take special pride in being part of families that are not plain vanilla, and their life experiences could become worse as a result of being mainstreamed. I strongly doubt any of those alternative possibilities, but they could be true. If Kennedy's claim is really an empirical assertion about the way people in the real world react to legal rules and norms, where did he get the evidence? Under his analysis, do we at least not need to examine whether my "dignity" is diminished by other people's thoughts, even if I neither know nor care what they think?
A threshold question in all of these examples is, where does the Court get its evidence? One answer is that empirical evidence (beyond what is available from the trial record, if any) can be provided in amicus briefs. But William & Mary Law School Professor Allison Orr Larsen's recent law review article, which she discussed earlier this month on The Colbert Report, shows just how unreliable those sources can be. For example, Orr Larsen notes that, in the Supreme Court's decision on "partial-birth" abortion, Justice Kennedy relied on a claim in an amicus brief that was ultimately based on an assertion from an "expert" whose claim to expertise was laughable. Nevertheless, this allowed Justice Kennedy to claim as an "unexceptionable" fact that many women regret having had abortions and ultimately become depressed and lose self-esteem.
Clearly, Justice Kennedy there was engaged in back-filling, casting about to find some support for a supposition that seemed intuitively obvious to Kennedy. The particular claim in that instance, however, has been subject to actual empirical inquiry, and the expert consensus is that there is no systematic support for Kennedy's claim (and that there are clear psychological gains for many women who are not forced to carry pregnancies to term, to rear unwanted children, and so on). But the Supreme Court's controlling opinion in that case states as simple fact something that is at best contestable. Oh well.
So what about the possibility of harnessing actual expertise? As a comment on Professor Dorf's post yesterday noted, the trial in a federal court in California that resulted in a ruling that Prop 8 is unconstitutional (affirmed, sort of, in Hollingsworth v. Perry in 2013) involved extensive factual inquiry, including testimony from experts. That case was, indeed, an outstanding example of a judge trying to answer some essential factual questions. The problem is that the Supreme Court has already made a bit of a hash of the question of how courts should handle empirical testimony. The famous/infamous Daubert case essentially enshrined the "95% statistical confidence" standard into law, as a shorthand for courts to determine whether a statistical analysis is reliable.
What is especially unfortunate about that holding is that Daubert was decided just as econometricians were confronting the fact that "statistical significance" has been grossly misused in empirical studies. Led by the methodologist Deirdre McCloskey, empiricists came to realize that being "95% confident" was neither necessary nor sufficient to establish something as an empirically reliable proposition. So, just as the field was becoming aware of having gone off the tracks, the Supreme Court came along and blessed the unscientific norm that had unthinkingly emerged in empirical studies.
As the title to this post notes, these are "some stray thoughts" about empiricism in constitutional legal analysis. Obviously, I am in the early stages of thinking through these issues. It is clear, however, that there are plenty of areas where the courts (prominently including the Supreme Court) are making decisions regarding constitutional questions on the basis of factual suppositions that have been untested and often not even articulated. Empirical work is never definitive (especially not the "95% confidence means it's true" variety), but having some empirical work available is better than the alternative.
Professor Dorf's post yesterday, "Empirical Scholarship In and Around Constitutional Law," described an important new project (and an associated conference that is taking place today) that attempts to make empiricism a more important part of constitutional legal scholarship. The idea is that, among all of the subjects addressed by legal scholars, constitutional law seems to be the one that has been least affected by formal empirical studies.
As Professor Dorf describes, con law's isolation from serious empirical research can hardly be explained by any claim that empirical matters are simply irrelevant to constitutional analysis. Notwithstanding the intuition that constitutional matters are theoretical, not factual, in reality we frequently (one might even say always) find that there are important empirical presumptions underlying constitutional decisions, from the Supreme Court on down.
For example, the paper that Professor Dorf and his co-author, Professor Brandice Canes-Wrone, are writing (soon to be summarized here on Dorf on Law) looks at the concept of "chilling effects," in this case whether bans on certain types of abortions change people's behavior with respect to activities not directly affected by such bans. If we were to find that such activity is not changed, then constitutional concerns about chilling behavior would be merely theoretical; but if people do act differently, then legislatures and courts should certainly take those unintended effects into account.
Thinking about yesterday's post, I found myself putting together a growing list of constitutional decisions from the Supreme Court that clearly rely on empirical suppositions by the justices about the world, even though their own contact with the world is notoriously "special." The famous story of a Supreme Court justice asking a passing stranger if he could borrow the stranger's pocket copy of the Constitution comes to mind. Supreme Court justices (and other judges, and legislators) often presume that they know what "normal people" think and do, but they are often quite wrong.
Consider a number of examples, from different areas of constitutional law. First, the line of cases that determine when a person has been "detained" by the police is rife with assertions from the justices about how a person would reasonably act when confronted by an officer. The most preposterous empirical statement from the Court came in a case in which officers had boarded a bus and were moving down the aisle, asking passengers questions. The Court's decision turned on whether the officers had physically blocked the aisle in a way that would stop a passenger from getting up and walking off the bus, because supposedly a reasonable person would know that he is otherwise not, in fact, detained -- and would feel perfectly comfortable getting off the bus without answering questions. Although one might defend the majority's statement as simply saying what a person should know under the Constitution, in fact the claim was entirely an empirical one: We do not need to worry about liberty being violated, because people who do not want to answer questions voluntarily will not respond to being confronted by officers by altering their behavior as if they had actually been told not to leave. As I heard one con law scholar put it after reading that case: "What planet does the Supreme court live on?"
But that question is only relevant if people on different planets in fact behave differently, which is an empirical question. The second example, from the late 1980's (if I recall correctly), had the Supreme Court deciding that trash that had been put out on the curb to be picked up was fair game to be searched by police officers. The Court's rationale explicitly included the supposition that people would know that their trash is no longer their private property (because they have voluntarily disposed of it), and thus that the police could search it. Is that what real people actually believe? I recall an economist predicting at the time that there would be a surge in sales of shredders, specifically because people would be shocked to learn that their trash was not protected by privacy doctrines. No one, as far as I know, carried out an empirical study of that claim, but the point is that the Supreme Court's majority blithely asserted something as fact that struck most people as simply "not what real people would think."
A slightly different issue is raised by a third example. In a case in the 1980's, the Court allowed a school district to censor a high school student newspaper. (I apologize for not having case names and citations for these cases, but I assume that interested readers can track them down.) I believe it was Justice White who wrote the opinion in which the Court held that the school's administrators had a reasonable concern that the views expressed in the student newspaper could be attributed to the school district itself. At the time, my immediate thought was that we see disclaimers all the time: "The views expressed herein do not necessarily represent the views of ... ." Although the Court did not even go that far in its analysis, it strikes me in the current context that there is an interesting pair of empirical questions here: (1) Without a disclaimer, would people really believe that what students write in a school newspaper expresses the views of the principal and superintendent? and (2) If so, would a disclaimer change people's views? But the Court was unconcerned, because they were willing to make an unexamined and unsupported assumption about reality.
This phenomenon is present also in cases where I happen to agree with the outcome. The Court's now-great Windsor decision invaliding the Defense of Marriage Act includes a lulu of an empirical claim by Justice Kennedy about how people think. He claimed that the children of same-sex couples are psychologically harmed by knowing that the state does not treat their families with the same respect with which it treats the families of opposite-sex couples. Although I (and the judges who have relied on Kennedy's soaring language in Windsor to invalidate bans on same-sex marriage) find that claim to be completely believable, how did Kennedy know? Maybe these children are simply unaware of the difference, or they do not care. Maybe they even take special pride in being part of families that are not plain vanilla, and their life experiences could become worse as a result of being mainstreamed. I strongly doubt any of those alternative possibilities, but they could be true. If Kennedy's claim is really an empirical assertion about the way people in the real world react to legal rules and norms, where did he get the evidence? Under his analysis, do we at least not need to examine whether my "dignity" is diminished by other people's thoughts, even if I neither know nor care what they think?
A threshold question in all of these examples is, where does the Court get its evidence? One answer is that empirical evidence (beyond what is available from the trial record, if any) can be provided in amicus briefs. But William & Mary Law School Professor Allison Orr Larsen's recent law review article, which she discussed earlier this month on The Colbert Report, shows just how unreliable those sources can be. For example, Orr Larsen notes that, in the Supreme Court's decision on "partial-birth" abortion, Justice Kennedy relied on a claim in an amicus brief that was ultimately based on an assertion from an "expert" whose claim to expertise was laughable. Nevertheless, this allowed Justice Kennedy to claim as an "unexceptionable" fact that many women regret having had abortions and ultimately become depressed and lose self-esteem.
Clearly, Justice Kennedy there was engaged in back-filling, casting about to find some support for a supposition that seemed intuitively obvious to Kennedy. The particular claim in that instance, however, has been subject to actual empirical inquiry, and the expert consensus is that there is no systematic support for Kennedy's claim (and that there are clear psychological gains for many women who are not forced to carry pregnancies to term, to rear unwanted children, and so on). But the Supreme Court's controlling opinion in that case states as simple fact something that is at best contestable. Oh well.
So what about the possibility of harnessing actual expertise? As a comment on Professor Dorf's post yesterday noted, the trial in a federal court in California that resulted in a ruling that Prop 8 is unconstitutional (affirmed, sort of, in Hollingsworth v. Perry in 2013) involved extensive factual inquiry, including testimony from experts. That case was, indeed, an outstanding example of a judge trying to answer some essential factual questions. The problem is that the Supreme Court has already made a bit of a hash of the question of how courts should handle empirical testimony. The famous/infamous Daubert case essentially enshrined the "95% statistical confidence" standard into law, as a shorthand for courts to determine whether a statistical analysis is reliable.
What is especially unfortunate about that holding is that Daubert was decided just as econometricians were confronting the fact that "statistical significance" has been grossly misused in empirical studies. Led by the methodologist Deirdre McCloskey, empiricists came to realize that being "95% confident" was neither necessary nor sufficient to establish something as an empirically reliable proposition. So, just as the field was becoming aware of having gone off the tracks, the Supreme Court came along and blessed the unscientific norm that had unthinkingly emerged in empirical studies.
As the title to this post notes, these are "some stray thoughts" about empiricism in constitutional legal analysis. Obviously, I am in the early stages of thinking through these issues. It is clear, however, that there are plenty of areas where the courts (prominently including the Supreme Court) are making decisions regarding constitutional questions on the basis of factual suppositions that have been untested and often not even articulated. Empirical work is never definitive (especially not the "95% confidence means it's true" variety), but having some empirical work available is better than the alternative.