What Happens When A Good Research Agenda Goes Stale (or Worse)?
-- Posted by Neil H. Buchanan
Last year, I was asked to review the job market papers for several entry-level professorial candidates in tax law, each of whom had written a paper in the "behavioral public finance" genre. For each review, I had to stop myself from writing something like this: "Kill me now. If I read another paper that jumps on the 'behavioral' bandwagon for no reason other than to follow the latest fad, I might just become a born-again believer in rational actor theory."
Of course, I did not write anything like that, because it would be unfair to the young scholars, who innocently followed their advisors' sensible suggestions about how to position themselves in the job market. In each case, what I actually wrote was that the young scholars had produced promising work (which was true). It is not their fault that they had been advised to write in an area that has been over-grazed and is no longer producing much (if any) intellectual nourishment. If we interview any tax candidates this year, I expect to go through the same process again.
In my Verdict column yesterday, I channeled the frustration that I have felt for the past few years into this question: Has behavioral law and economics jumped the shark? (As in the column, I will use the shorthand BLE for behavioral law and economics.) It is not just the entry-level scholars, and it is not just in tax, that we see too much of what was once truly a promising and interesting development in interdisciplinary scholarship. Maybe I am just being grouchy, but I do worry that too much time and talent is being poured into a scholarly genre that is just not making much progress.
Again, I do understand why BLE became trendy. I remember the first time I came across the concept of "tax salience," for example. Surely it matters how people perceive taxes in determining whether to pay (and how unhappy to feel about it). Value-added taxes abroad are included in the price, whereas sales taxes in the U.S. are added onto the posted price at the register. Tollbooths that require cash are both a hassle and confront people with the cost of driving, whereas EZPass booths are faster and do not tell you until later (if you even bother to look) how much you have been charged. Those differences, one would think, simply have to matter.
And I still think that they do. What I am less convinced of is that scholars will ever be able to figure out how to investigate those differences in a way that is interesting and useful, either for policy applications, or even simply for understanding people better than we do now. In my column, I describe how easy it is to use what sound like BLE "insights" to argue both sides of the same argument. I use as a particular example the penalty-versus-tax debate after the Supreme Court's ACA decision, which was quickly infected by BLE-style arguments that supposedly showed that calling the individual mandate a "tax" would cause people to buy more health insurance. The problem is that other BLE arguments (at least as plausible as the first set) cut exactly the opposite way.
Like so many questions, this could be resolved in principle by a good empirical test, but BLE has not produced consistent empirical results that can be generalized and applied. That allows everyone to shout about behavioral stuff, without fear of being confronted with actual evidence. This leaves the debate in a very bad place, where everyone who wants to play the BLE card can say, "Well, I have a decent-sounding theory, and you can't prove that it's wrong." That is fertile territory for charlatans and hacks. And for every careful scholar who writes about these questions, there are too many others who grab whatever they can to push an ideological agenda.
Speaking of careful scholars, after I posted my column yesterday, I happened to come across a link to a paper (posted last Fall) by Ed McCaffery, a deservedly renowned tax professor at USC Law. McCaffery, writing for a handbook of behavioral law and economics (!), ends his abstract by saying that "[t]he path from laboratory to real-world policy prescription must still be laid out, although the task is important ..." The path has not even been laid out! After thousands of papers (not just in law, but in economics, psychology, sociology, and so on), and tens of thousands of SSRN downloads, we do not yet even know what the path is from BLE to useful policy. We are still arguing over blueprints, without even the hint of breaking actual ground to start building the road. (And when the road is built, presumably, we will need to argue about whether to charge a toll, and whether to accept EZPass ...)
I doubt that McCaffery and I agree on every aspect of the problem. If I had to guess, I would say that he is probably still more hopeful than I am. Yet I do find it interesting that we independently reached the point where we both reluctantly concluded that there has been distressingly little progress on something that once looked so promising.
Interestingly, scholarly doubt seems to have become the order of the day lately. My Dorf on Law posts and Verdict columns frequently cover the same territory that Paul Krugman patrols, because we are responding simultaneously to releases of data, or to ridiculous political arguments, and so on. Recently, however, Krugman has also been looking at the scholars in his profession, and wondering what went wrong. His angst has mostly been driven, as I suggested last year, by Krugman's failure to recognize how politicized his field has always been -- notwithstanding the whole positive/normative dodge that economists hide behind, to pretend that they are scientists.
In a post on his blog yesterday, Krugman made a different argument. He asked what had happened to Theory in economics. I capitalize the "t" because, in economics, theory is a very particular -- and highly exalted -- thing, with the "smartest guys" in the top economics departments (and yes, it is almost always guys who are deemed smart in such inbred places) producing the mathematical modeling that soon constitutes "what we know." Some of those guys then take leaves from their departments, move to Washington to advise Presidents, and are either savvy enough to know that what counts as cutting-edge theory in economics is generally useless (and thus they instead rely on old-fashioned stuff that actually works -- like, say, Keynesianism) or they make a hash of it all -- again, often because of ideological predispositions.
Krugman's question yesterday echoed mine (and McCaffery's, surely among many others): Why are scholars not turning out useful theories any more? Discussing two sub-fields in economics -- international trade theory, and business cycle macroeconomic theory -- Krugman concludes, humorously, that "theory lost its luster in trade because it could prove anything; it lost its luster in macro because it proved things that weren’t so."
Notably, his story regarding the theory of international trade directly parallels my take on BLE: "After a while, the new approaches came to seem too liberating; by the early 90s the joke was that a smart graduate student could devise a model to justify any policy." Indeed, I remember in the late 1990's, interviewing a job candidate in an economics department who specialized in trade, and asking him: "So, if I wanted to prove that free trade was bad, what would be the best new arguments?" Without missing a beat, he had three ready answers, even though he also had three answers for the opposite conclusion. And none of the theories was "wrong," by any of the standards that economists use to evaluate theory.
What to do? This seems to be yet another case where we see that scholarly progress is hard to assess, and where scholarly regress is possible. I continue to think that far too much emphasis is put on theory in all of the social sciences (and in law), and that empirical inquiry is currently much more promising. Empirical inquiry has to have some theoretical basis, but it need not be the supposedly high Theory of economics. Moreover, empirical inquiry is not limited to -- or even best undertaken by using -- the latest econometric techniques. The better approach, I think, continues to be to bring a variety of empirical methods (and data sources) to bear, and to see whether something resembling a consistent truth emerges.
This, some readers will recognize, is a suggestion that I cribbed from the classic, "Let's Take the 'Con' Out of Econometrics," by Ed Leamer (only available, amazingly, for a $10 download). Now in print for 30 years, Leamer's point remains apt: There are no definitive empirical tests, and there is no definitive theory. Learning about the world is difficult, iterative work, requiring variation and creativity. As it pertains to my complaint this week, the point is that too much work is being done in too narrow an area, and it is not panning out. Time to redeploy our resources.
Last year, I was asked to review the job market papers for several entry-level professorial candidates in tax law, each of whom had written a paper in the "behavioral public finance" genre. For each review, I had to stop myself from writing something like this: "Kill me now. If I read another paper that jumps on the 'behavioral' bandwagon for no reason other than to follow the latest fad, I might just become a born-again believer in rational actor theory."
Of course, I did not write anything like that, because it would be unfair to the young scholars, who innocently followed their advisors' sensible suggestions about how to position themselves in the job market. In each case, what I actually wrote was that the young scholars had produced promising work (which was true). It is not their fault that they had been advised to write in an area that has been over-grazed and is no longer producing much (if any) intellectual nourishment. If we interview any tax candidates this year, I expect to go through the same process again.
In my Verdict column yesterday, I channeled the frustration that I have felt for the past few years into this question: Has behavioral law and economics jumped the shark? (As in the column, I will use the shorthand BLE for behavioral law and economics.) It is not just the entry-level scholars, and it is not just in tax, that we see too much of what was once truly a promising and interesting development in interdisciplinary scholarship. Maybe I am just being grouchy, but I do worry that too much time and talent is being poured into a scholarly genre that is just not making much progress.
Again, I do understand why BLE became trendy. I remember the first time I came across the concept of "tax salience," for example. Surely it matters how people perceive taxes in determining whether to pay (and how unhappy to feel about it). Value-added taxes abroad are included in the price, whereas sales taxes in the U.S. are added onto the posted price at the register. Tollbooths that require cash are both a hassle and confront people with the cost of driving, whereas EZPass booths are faster and do not tell you until later (if you even bother to look) how much you have been charged. Those differences, one would think, simply have to matter.
And I still think that they do. What I am less convinced of is that scholars will ever be able to figure out how to investigate those differences in a way that is interesting and useful, either for policy applications, or even simply for understanding people better than we do now. In my column, I describe how easy it is to use what sound like BLE "insights" to argue both sides of the same argument. I use as a particular example the penalty-versus-tax debate after the Supreme Court's ACA decision, which was quickly infected by BLE-style arguments that supposedly showed that calling the individual mandate a "tax" would cause people to buy more health insurance. The problem is that other BLE arguments (at least as plausible as the first set) cut exactly the opposite way.
Like so many questions, this could be resolved in principle by a good empirical test, but BLE has not produced consistent empirical results that can be generalized and applied. That allows everyone to shout about behavioral stuff, without fear of being confronted with actual evidence. This leaves the debate in a very bad place, where everyone who wants to play the BLE card can say, "Well, I have a decent-sounding theory, and you can't prove that it's wrong." That is fertile territory for charlatans and hacks. And for every careful scholar who writes about these questions, there are too many others who grab whatever they can to push an ideological agenda.
Speaking of careful scholars, after I posted my column yesterday, I happened to come across a link to a paper (posted last Fall) by Ed McCaffery, a deservedly renowned tax professor at USC Law. McCaffery, writing for a handbook of behavioral law and economics (!), ends his abstract by saying that "[t]he path from laboratory to real-world policy prescription must still be laid out, although the task is important ..." The path has not even been laid out! After thousands of papers (not just in law, but in economics, psychology, sociology, and so on), and tens of thousands of SSRN downloads, we do not yet even know what the path is from BLE to useful policy. We are still arguing over blueprints, without even the hint of breaking actual ground to start building the road. (And when the road is built, presumably, we will need to argue about whether to charge a toll, and whether to accept EZPass ...)
I doubt that McCaffery and I agree on every aspect of the problem. If I had to guess, I would say that he is probably still more hopeful than I am. Yet I do find it interesting that we independently reached the point where we both reluctantly concluded that there has been distressingly little progress on something that once looked so promising.
Interestingly, scholarly doubt seems to have become the order of the day lately. My Dorf on Law posts and Verdict columns frequently cover the same territory that Paul Krugman patrols, because we are responding simultaneously to releases of data, or to ridiculous political arguments, and so on. Recently, however, Krugman has also been looking at the scholars in his profession, and wondering what went wrong. His angst has mostly been driven, as I suggested last year, by Krugman's failure to recognize how politicized his field has always been -- notwithstanding the whole positive/normative dodge that economists hide behind, to pretend that they are scientists.
In a post on his blog yesterday, Krugman made a different argument. He asked what had happened to Theory in economics. I capitalize the "t" because, in economics, theory is a very particular -- and highly exalted -- thing, with the "smartest guys" in the top economics departments (and yes, it is almost always guys who are deemed smart in such inbred places) producing the mathematical modeling that soon constitutes "what we know." Some of those guys then take leaves from their departments, move to Washington to advise Presidents, and are either savvy enough to know that what counts as cutting-edge theory in economics is generally useless (and thus they instead rely on old-fashioned stuff that actually works -- like, say, Keynesianism) or they make a hash of it all -- again, often because of ideological predispositions.
Krugman's question yesterday echoed mine (and McCaffery's, surely among many others): Why are scholars not turning out useful theories any more? Discussing two sub-fields in economics -- international trade theory, and business cycle macroeconomic theory -- Krugman concludes, humorously, that "theory lost its luster in trade because it could prove anything; it lost its luster in macro because it proved things that weren’t so."
Notably, his story regarding the theory of international trade directly parallels my take on BLE: "After a while, the new approaches came to seem too liberating; by the early 90s the joke was that a smart graduate student could devise a model to justify any policy." Indeed, I remember in the late 1990's, interviewing a job candidate in an economics department who specialized in trade, and asking him: "So, if I wanted to prove that free trade was bad, what would be the best new arguments?" Without missing a beat, he had three ready answers, even though he also had three answers for the opposite conclusion. And none of the theories was "wrong," by any of the standards that economists use to evaluate theory.
What to do? This seems to be yet another case where we see that scholarly progress is hard to assess, and where scholarly regress is possible. I continue to think that far too much emphasis is put on theory in all of the social sciences (and in law), and that empirical inquiry is currently much more promising. Empirical inquiry has to have some theoretical basis, but it need not be the supposedly high Theory of economics. Moreover, empirical inquiry is not limited to -- or even best undertaken by using -- the latest econometric techniques. The better approach, I think, continues to be to bring a variety of empirical methods (and data sources) to bear, and to see whether something resembling a consistent truth emerges.
This, some readers will recognize, is a suggestion that I cribbed from the classic, "Let's Take the 'Con' Out of Econometrics," by Ed Leamer (only available, amazingly, for a $10 download). Now in print for 30 years, Leamer's point remains apt: There are no definitive empirical tests, and there is no definitive theory. Learning about the world is difficult, iterative work, requiring variation and creativity. As it pertains to my complaint this week, the point is that too much work is being done in too narrow an area, and it is not panning out. Time to redeploy our resources.