Downloads and Inflation
Earlier this year, I posted some thoughts on the question of using downloads from the SSRN repository to evaluate the "scholarly impact" (or some other notion of importance) of faculty and law schools. My biggest concern remains that the use of these download data in rankings gives people an incentive to do the opposite of what SSRN was designed to do in the first place: encourage people to look at each others' work and to engage with it. Specifically, since we know that a decision to download a paper will pump up the author's rankings, that knowledge can cause us to pass over a paper either for straightforward strategic reasons ("I don't want to pump up his numbers!") or for more benign reasons ("I'm not sure what this paper says; but the very act of finding out will pump up the numbers for a paper that I might end up wishing I had not downloaded.")
I'll add here that there is an additional difference between download counts and citation counts. While citation counts are also problematic, they cannot really be stopped. That is, there is nothing to stop any motivated party from going out and counting citations. All you have to have is the time and technology, and you can count the citations of any given paper in any set of publications you like. SSRN download counts, by contrast, are not (so far as I know) inherently public data. If the management of SSRN decided that the information that the download data provide is less valuable than the distortions that they cause, they could simply choose not to provide this information anymore. If I'm right that they have this option, I wish that they would exercise it. In my mind, the cost/benefit analysis clearly disfavors publishing these numbers -- especially because they undermine what I take to be SSRN's core purpose.
That said, I also wanted to add a criticism that was suggested to me by my new colleague at GW, Sarah Lawsky. (For people who are destined by their surnames for careers in the law, perhaps only Sylvia Law has a better name than Lawsky.) Sarah suggests, without taking a position on whether it is actually a useful exercise to count downloads, that downloads need to be adjusted for the equivalent of inflation. That is, the download counts are typically offered as "all-time downloads" as well as "recent downloads." (A recent example is here.) What could all-time downloads possibly tell us? In its early years, SSRN naturally had very few papers, and most scholars had not yet heard of it and were not downloading papers as part of their research work. An author could have written a paper that had only 30 downloads in an environment where there were only 3000 downloads in a week, whereas today a relatively minor paper could have two to three times as many total downloads. It's like saying that someone earning $200,000 per year today is richer than someone who earned $100,000 per year in 1967.
What is the right "price deflator," to use the economics term? As Sarah points out, this depends on the question we're asking. I'm not sure that there is a useful question that is answered by all-time downloads (or current downloads, for that matter), even if we could engineer a nice price deflator. I do know, though, that those who do think that downloads are a good measure of something would make a more compelling argument if they could identify what they think they're measuring and then devise an appropriate price deflator.
-- Posted by Neil H. Buchanan
I'll add here that there is an additional difference between download counts and citation counts. While citation counts are also problematic, they cannot really be stopped. That is, there is nothing to stop any motivated party from going out and counting citations. All you have to have is the time and technology, and you can count the citations of any given paper in any set of publications you like. SSRN download counts, by contrast, are not (so far as I know) inherently public data. If the management of SSRN decided that the information that the download data provide is less valuable than the distortions that they cause, they could simply choose not to provide this information anymore. If I'm right that they have this option, I wish that they would exercise it. In my mind, the cost/benefit analysis clearly disfavors publishing these numbers -- especially because they undermine what I take to be SSRN's core purpose.
That said, I also wanted to add a criticism that was suggested to me by my new colleague at GW, Sarah Lawsky. (For people who are destined by their surnames for careers in the law, perhaps only Sylvia Law has a better name than Lawsky.) Sarah suggests, without taking a position on whether it is actually a useful exercise to count downloads, that downloads need to be adjusted for the equivalent of inflation. That is, the download counts are typically offered as "all-time downloads" as well as "recent downloads." (A recent example is here.) What could all-time downloads possibly tell us? In its early years, SSRN naturally had very few papers, and most scholars had not yet heard of it and were not downloading papers as part of their research work. An author could have written a paper that had only 30 downloads in an environment where there were only 3000 downloads in a week, whereas today a relatively minor paper could have two to three times as many total downloads. It's like saying that someone earning $200,000 per year today is richer than someone who earned $100,000 per year in 1967.
What is the right "price deflator," to use the economics term? As Sarah points out, this depends on the question we're asking. I'm not sure that there is a useful question that is answered by all-time downloads (or current downloads, for that matter), even if we could engineer a nice price deflator. I do know, though, that those who do think that downloads are a good measure of something would make a more compelling argument if they could identify what they think they're measuring and then devise an appropriate price deflator.
-- Posted by Neil H. Buchanan