What to Do about SCOTUS Mistakes: A Not-So Modest Proposal
By Eric Segall
This week ProPublica issued a report with the headline "It's a Fact, Supreme Court Errors aren't Hard to Find." The group reviewed "dozens of cases" and said that it found a "number of false or wholly unsupported claims." Examples included erroneous voter registration rates in Chief Justice Robert's Shelby County decision striking down a key section of the Voting Rights Act; an unsupported conclusion by Justice Alito about the percentage of American companies that require background checks for its workers similar to the ones used for federal employees; and Justice Kagan's statements about the reliability of drug sniffing dogs in an important Fourth Amendment case. ProPublica claimed in the report to review 83 randomly selected opinions over a five year period and found assertions of "legislative facts" in 24 of those cases. The group alleged that seven of those opinions, more than 25%, contained false or unsupported statements of fact.
This report, which should trouble all Court watchers, commentators, and the American people, caught the attention of Fordham Law Professor John Pfaff, who wrote an op-ed on the problem in the New York Times. Pfaff observed that "policy is a major part of the Court's docket now" but neither the Justices nor their clerks have "any serious training" in empirical methods such as statistics. He argued that the Court has traditionally relied on amicus briefs "to provide it with that broad empirical background" but unfortunately "many amicus briefs include false or unsubstantiated empirical assertions, at least some of which make it into the Justices' opinions." Pfaff also noted that many, if not most Supreme Court cases, get to the Court after years of lower court litigation "providing interested groups plenty of time to gin favorable findings." So, in Pfaff's words, "what to do?"
Pfaff makes several suggestions. One is to "establish a group of technical advisers to the Court." These social scientists and statisticians could help the Justices analyze the empirical evidence relevant to their decisions. As Pfaff correctly observes, "there is no shortage of scholars with Ph.D's who would be eager to do that work for the court." He also suggests that perhaps the Justices could hire law clerks with better empirical training or even hire outside experts to help the Justices handle specific issues.
Pfaff recognizes that these proposals run the risk of the Justices hiring "partisan experts arguably making things worse." But he says there are ways around that such as requiring 6 out of 9 Justices to agree on the experts or having the clerk's office make the hiring decisions. He concludes that, "in the end, the question is a comparative one. It's not, is there a perfect solution ... but ... can the Court make its policy decisions better? The answer to the latter question is clear."
The ProPublica report and Pfaff's op-ed come on the heels of Chief Justice Robert's quip during the recent redistricting oral arguments that empirical evidence in that case amounted to "sociological gobbledygook." Of course, as many have reported since that argument, the Chief regularly uses such evidence to support his opinions, such as the erroneous voter registration rates he highlighted in Shelby County v. Holder.
There is an unwritten premise in both the important ProPublica report and Pfaff's fine op-ed that I would like to challenge. There is an assumption that the Justices' use of empirical evidence plays a significant role in their decision-making. That may be true in some of the Court's less controversial cases, but does anyone really believe that the Justices' views on same-sex marriage, gun control, abortion, campaign finance reform, affirmative action, or even redistricting are impacted by "facts" about the real world? I am not arguing that this kind of evidence is never result-generating but I think the occasions when it really matters are rare.
Chief Justice Roberts and the other conservatives would have struck down the key section of the Voting Rights Act regardless of the rate of voter registration in the South, and Justice Alito (along with the other eight Justices who joined his opinion) would have likely allowed background checks for employees working for government contractors (a change of policy after 9/11) no matter the percentage of private employers who also used such checks. These kinds of cases don't turn on facts but on values. As Dean Erwin Chemerinsky has written, "constitutional law is about defining and protecting society's most cherished values .... Ultimately the [Court's] decisions must be criticized or defended for the value choices the Court made. There is nothing else."
This is not to say of course that the Court should use erroneous data when explaining "policy" decisions made on the basis of values. It is to say that in most cases the Justices should leave the data behind and more transparently highlight the real value choices driving the decisions. What the Court does not need are more technical advisers to add post-hoc justifications for their decisions based on social science data and statistics which inevitably can be manipulated by experts and mathematicians in most hard cases.
My proposal is that in most of the Court's cases the Justices first clearly announce the values choices that are driving their decisions. For example, in Shelby County, Chief Justice Roberts could have cited his own 1982 legal memo that he wrote when he worked for the Justice Department announcing that parts of the VRA amounted to "the most intrusive interference imaginable by federal courts into state and local processes." Only after the clear articulation of such value preferences, and the tying of those judgments directly to the Court's legal holdings (with a "therefore" or a "thus"), should the Justices discuss the social science data or other facts which may support their holdings. My guess is if the Court committed itself to that kind of approach, the Justices would employ fewer contestable facts, make fewer errors, and write more transparent and honest decisions. Win, win and win.
This week ProPublica issued a report with the headline "It's a Fact, Supreme Court Errors aren't Hard to Find." The group reviewed "dozens of cases" and said that it found a "number of false or wholly unsupported claims." Examples included erroneous voter registration rates in Chief Justice Robert's Shelby County decision striking down a key section of the Voting Rights Act; an unsupported conclusion by Justice Alito about the percentage of American companies that require background checks for its workers similar to the ones used for federal employees; and Justice Kagan's statements about the reliability of drug sniffing dogs in an important Fourth Amendment case. ProPublica claimed in the report to review 83 randomly selected opinions over a five year period and found assertions of "legislative facts" in 24 of those cases. The group alleged that seven of those opinions, more than 25%, contained false or unsupported statements of fact.
This report, which should trouble all Court watchers, commentators, and the American people, caught the attention of Fordham Law Professor John Pfaff, who wrote an op-ed on the problem in the New York Times. Pfaff observed that "policy is a major part of the Court's docket now" but neither the Justices nor their clerks have "any serious training" in empirical methods such as statistics. He argued that the Court has traditionally relied on amicus briefs "to provide it with that broad empirical background" but unfortunately "many amicus briefs include false or unsubstantiated empirical assertions, at least some of which make it into the Justices' opinions." Pfaff also noted that many, if not most Supreme Court cases, get to the Court after years of lower court litigation "providing interested groups plenty of time to gin favorable findings." So, in Pfaff's words, "what to do?"
Pfaff makes several suggestions. One is to "establish a group of technical advisers to the Court." These social scientists and statisticians could help the Justices analyze the empirical evidence relevant to their decisions. As Pfaff correctly observes, "there is no shortage of scholars with Ph.D's who would be eager to do that work for the court." He also suggests that perhaps the Justices could hire law clerks with better empirical training or even hire outside experts to help the Justices handle specific issues.
Pfaff recognizes that these proposals run the risk of the Justices hiring "partisan experts arguably making things worse." But he says there are ways around that such as requiring 6 out of 9 Justices to agree on the experts or having the clerk's office make the hiring decisions. He concludes that, "in the end, the question is a comparative one. It's not, is there a perfect solution ... but ... can the Court make its policy decisions better? The answer to the latter question is clear."
The ProPublica report and Pfaff's op-ed come on the heels of Chief Justice Robert's quip during the recent redistricting oral arguments that empirical evidence in that case amounted to "sociological gobbledygook." Of course, as many have reported since that argument, the Chief regularly uses such evidence to support his opinions, such as the erroneous voter registration rates he highlighted in Shelby County v. Holder.
There is an unwritten premise in both the important ProPublica report and Pfaff's fine op-ed that I would like to challenge. There is an assumption that the Justices' use of empirical evidence plays a significant role in their decision-making. That may be true in some of the Court's less controversial cases, but does anyone really believe that the Justices' views on same-sex marriage, gun control, abortion, campaign finance reform, affirmative action, or even redistricting are impacted by "facts" about the real world? I am not arguing that this kind of evidence is never result-generating but I think the occasions when it really matters are rare.
Chief Justice Roberts and the other conservatives would have struck down the key section of the Voting Rights Act regardless of the rate of voter registration in the South, and Justice Alito (along with the other eight Justices who joined his opinion) would have likely allowed background checks for employees working for government contractors (a change of policy after 9/11) no matter the percentage of private employers who also used such checks. These kinds of cases don't turn on facts but on values. As Dean Erwin Chemerinsky has written, "constitutional law is about defining and protecting society's most cherished values .... Ultimately the [Court's] decisions must be criticized or defended for the value choices the Court made. There is nothing else."
This is not to say of course that the Court should use erroneous data when explaining "policy" decisions made on the basis of values. It is to say that in most cases the Justices should leave the data behind and more transparently highlight the real value choices driving the decisions. What the Court does not need are more technical advisers to add post-hoc justifications for their decisions based on social science data and statistics which inevitably can be manipulated by experts and mathematicians in most hard cases.
My proposal is that in most of the Court's cases the Justices first clearly announce the values choices that are driving their decisions. For example, in Shelby County, Chief Justice Roberts could have cited his own 1982 legal memo that he wrote when he worked for the Justice Department announcing that parts of the VRA amounted to "the most intrusive interference imaginable by federal courts into state and local processes." Only after the clear articulation of such value preferences, and the tying of those judgments directly to the Court's legal holdings (with a "therefore" or a "thus"), should the Justices discuss the social science data or other facts which may support their holdings. My guess is if the Court committed itself to that kind of approach, the Justices would employ fewer contestable facts, make fewer errors, and write more transparent and honest decisions. Win, win and win.