Part 6: Is There Too Much Legal Scholarship: The Research Impact Statement

 

 Is There a Way Out of Here? The Research Impact Statement

The short answer is that without of a complete restructuring of legal education along the lines discussed above, I doubt it.  This is especially true since law schools and law professors seem resistant to anything that smacks of cost benefit analysis and the concept of opportunity costs. These ideas are likely to be threatening because following their logic may mean upsetting the status quo.  Whether or not these concepts are employed, the reality is that though decisions about funding scholarship are made, the method of making those decisions seems untethered to any systematic analysis or specific goals. Perhaps all we can hope for is baby steps toward a more rational system.

Even baby steps need an objective and my choice would be to fund legal scholarship that has an actual, articulable, and rational  relationship to benefits of clients and consumers, broadly defined. Put in terms discussed above, there should be a justification based on attention to the demand side of the “market.”   Its impact can be  indirect but an article that does not start with an introduction that says, in effect,  “clients and others who rely on legal education or a system of laws will benefit by this effort because” is not worth the investment.  

So what steps could be taken? The first is to eliminate the assumption that all law professors are entitled to funding for research. The burden should be shifted to the professors themselves to make a case for funding. Thus, general rules that everyone receives a summer grant or has a lower teaching load to recognize their research obligations should be reversed. To use a common phrase --   research support should be a privilege not a right that automatically follows from being employed by a law school.

Each professor desiring to do legal research should submit a  “research impact statement” which notes the benefits, the research methodology, hypotheses to be tested, time schedule. The decision makers should view the process as a competitive one. This is not only competition with other professors but with other uses of the desired funding. The most obvious possibility is additional teaching obligations.

              Clearly this leads to the issue of how the decision to fund is made. Ideally it would be a committee composed of a law school dean or associate dean, a high ranking student, a member of the bar, and the faculty.  Proposals would be made to this group and viewed in the context of other possible uses of the professor’s time or other resources used to support his or her efforts. Returning to the market analogy noted above,  professors should be required to identify the demand (or need) even though it is not one actually expressed in the market.

In closing I concede that a limitation on this analysis is that  I have not addressed the problem of law reviews themselves. After all, law reviews are the most direct demanders of legal scholarship and the proliferation of reviews has just intensified demand. In theory, they should be viewed as and act as middle men. Their demand should be what economists call derived demand – it should be based on the demand by consumers and users of scholarship. There is little sign that this is the case.

In fact, as demanders of legal scholarship, law reviews exhibit a bias for institutional authority as opposed to reflecting the needs of readers and society more generally. Greater selectivity based on the actual needs of readers as opposed to what well credentialed authors choose to supply would be a more than a baby step. This does not, however, solve the problem of too great an investment in legal scholarship. In fact, the existing level of scholarship would likely redistributed from one review to another. On the other hand, knowledge that reviews are attentive to actual need could redirect resources for scholarship, much of which is just self-indulgent, into paths with greater value to readers.  

Part 5, Is There Too Much Legal Scholarship: What Accounts For Too Much of the Wrong Type of Scholarship

 

Part 5  Is There Too Much Legal Scholarship:  What Accounts for Too Much of the Wrong Type of Scholarship?

              A perfect academic storm has created this trouble state. Each facet is a necessary part. They are 1) the self-interest of administrators and professors, 2) the goals of law schools and law professors, 3) the narrow socioeconomic background of law professors and 4) the need to find some indicia of success. First, law professors are as self-interested as anyone else.  Many are ambitious and have the goal of being recognized, praised, and rising through the ranks of law schools. In terms of legal scholarship that means being published as much as possible in the highest ranked law review or by the most prestigious academic presses. It also requires picking the “right” kind of topic which nearly always means something theoretical or empirical. This is not only the way to “get ahead” for the professor,  it advances the reputation of the law school as well.

              Law school reputation is important not just to professors but perhaps more importantly to law school deans who are also self-interested and may define success as moving up through administrative ranks. This impact will be exacerbated by the inclusion of citation counts in U.S. News rankings.  The ranking of a law school is likely to be important to both University officials and law school students and graduates. Possibly the easiest to understand in this group are students and graduates. Employment opportunities and the value of a law degree in terms of earning power are likely to increase as a school moves up in the rankings. Finally, the students who select law review articles are likely to be interested in the ranking of the review which also influences article selection.

              It is interesting to focus for a moment on the role of institutions in encouraging (demanding) writing regardless of the topic or usefulness. This is actually rent-seeking behavior by institutions.  Rent is a term economists use to denote the payment for a resources above the least necessary to draw the resource into a specific type of production. In the case of monopolies. it can be profit. In fact, one of the costs of monopoly is the resources spent by parties vying to become the monopolist. This is rent seeking behavior. The defining characteristic of rent seeking is that welfare, profits, or some other benefit may change hands but the total does not increase. In legal education the competition is not for profits but for rankings. One school’s increase in the rankings means another school’s decline. In short, this competition between schools of which legal scholarship is a critical feature does not increase of overall welfare. Instead, the objective is recognition and any actual benefits are a fortunate coincidence.

There are two more elements of this perfect storm. The first is the socioeconomic background of those who become law professors. The second is a need for a method of comparison or ranking.  People who become law professors come disproportionately from a higher socioeconomic class than most others. They attended elite schools, clerked for judges and sometimes engaged in high level practice where clients are likely to be drawn from the wealthiest segments of society.   Unintentionally they are sheltered.  In simple terms, the universe of topics they are equipped to write about is a narrow one. This is further narrowed by lack of training in empirical analysis.  In one sense they are not well educated.  Articles on constitutional law or obscure contracts issues are far more in their wheel-house than articles on issues of poverty, class, prison conditions, and barriers to the poor.

The  final factor is that any meritocracy or pseudo meritocracy needs a way to determine merit.  The standard has to be something that those ranking can assess and which is viewed as difficult.  I am sure there is disagreement with this but I think it is safe to assume that law professors view theoretical work as more difficult or challenging than work with practical applications or even empirical work. Perhaps this is because of the unfortunate view of law school as comparable to graduate school and the perceived pressure to write at the lofty levels of those in conventional graduate programs.  In fact, a  more rational placement of what is now taught in law schools would be in the form of a jurisprudence department within a college of arts and sciences and a completely separate program dedicated to training those who want to practice law.  

Part 4: Is There Too Much Legal Scholarship? Justifications for Legal Scholarship

 Part 4: Is There Too Much Legal Scholarship?

 Justifications for Legal Scholarship

There are many arguments in response to the position that there is too much legal scholarship.  As noted above, one of those justifications is based on the public good character of legal scholarship. That argument falls short for the reasons discussed above. Other justifications are addressed here.   I think it is not controversial to say that any justification must, at its core, be an argument that the investment in legal scholarship creates more benefits (however measured or defined) than costs. None of the arguments addressed here seem to come close to that standard.

1. Faith Based

One argument  is what I would call “faith based.” It goes something like this. “Law review articles are important as a way of communicating with other professors and people outside the profession.”  (Robin West) A recent version of this plea is, “If we forgo this work, our worlds, not only the legal world, but our social and cultural and political worlds, will be the worse for it.” (Robin West)   Both arguments are hard to refute since both are non falsifiable.  They are also ironic.  The norm in legal scholarship is to cite authority for every declarative statement . These assertions, however, evidently need no support because they are just “truths.” No rational – a term I understand is disfavored by the “normative scholars” -- system of resource allocation would permit an expenditure as large as that devoted to legal scholarship without some requirement that the proponents produce proof of results.

2. Cherry Picking

Many people, when they hear an objection to the level of investment in legal scholarship, reel off the titles of five or six classic articles. I suppose the logic is that if  some articles are worth  $30,000 or more then all articles are worth $30,000. Of course, those who make this argument, and who should know better, fall into the fallacy of composition. There is no reason to suppose that because some of the articles are successful then they all must be. This is probably best known by all those who do laboratory research or prospect for minerals or energy sources. One successful experiment is unrelated to the success of others and one productive oil well does not mean all others are equally productive.

3. The Posner Argument

Citing a handful of articles for the proposition that the current level of investment in legal scholarship can make sense if there is a connectedness between those articles and total expenditures. Returning to the laboratory idea, the argument could be that each article is itself an experiment. We know ahead of time that they cannot all produce useful outcomes but we must do all of them in order to discover the important ones. This is essentially an argument made by Richard Posner who likens law review articles to salmon swimming upstream. Since we do not know ahead of time which ones are going to achieve their goals, we must let all of them try in order to ensure the strong ones do make it.

This is an appealing argument for salmon but not for law review articles. First, salmon swimming upstream do not ask to be subsidized by others.  The cost of their effort is entirely internalized by each one attempting the trip. This is hardly the case of any law professor. Second, there are few, if any, opportunity costs of allowing salmon to give it a try.  In the case of law review articles, something that constitutes a better use of the funding may be neglected.  Plus, the law review salmon are hardly trying to make it up the same stream. We know that, no matter how powerful their swimming is, professors at lower ranked schools with non elite credentials are unlikely to be cited in a meaningful way.  In fact, there are likely to be other indicia of articles that are unlikely to enhance social welfare. Unlike salmon, we can examine the project before it sets out on its journey and, in light of opportunity costs, we can determine that the expected value of some is not worth the sacrifice. Yes, it is true that this amounts to playing the probabilities and risking what might be called false negatives – published works that are not beneficial. Given the huge opportunity costs, not to weigh this possibility against the demonstrable instances of false positives seems irresponsible and quite different from allowing all salmon, who have nothing to lose, to attempt to make it upstream.

4. Academic Freedom

The argument can be made that limiting the support for legal scholarship interferes with academic freedom. Funding one project or even limiting funding limits a law professor’s “right” to express him or herself.  This confuses freedom of expression with funding. Law professors, especially those with tenure, generally have freedom to express themselves without fear of reprisal. This does not equal a right to have that speech subsidized. If that were the case, then it would be an attack on academic freedom not to excuse professors from meeting class, attending faculty meetings, and all other activities that limit the time available for “expression.”

This particular concern should not be completely discounted. Suppose there is limited funding available for summer grants. Grants are made on a competitive basis by a committee whose job it is to prioritize the proposed projects. They are prioritized based on “importance to society.” Many areas in which law professors write are controversial and political in nature. Thus, in the hypothetical, suppose the following proposals receive support: Up With Whales, Affirmative Action for All; Higher Wages for Women; Protecting our Children from Abuse. The project denied support is “Reviving Ojus.” (Ojus is a tiny town north of Miami.) Surely, the professor proposing the last project will feel he was discriminated against based on his topic.  In a sense, the argument could be made that there was a reprisal by virtue of unequal funding. This may or may not raise issues of academic freedom, but the notion that everything a law professor desires to write at a cost of $30,000 per article will be permitted hardly seems like a necessary cure. Just as schools have no obligation to offer courses for which few enroll, universities are similarly unlikely to be required to fund projects with little or no promise in terms of readership or usefulness. Even if the academic freedom argument has some purchase, a response superior to unlimited writing may be to redefine the contours of academic freedom.  

5. The Law Review Experience

              One explanation that has a cart before the horse or tail wagging the dog character is that legal scholarship exists so that students will have the valuable experience of operating a journal. They surely do get research and writing experience. Aside from ignoring opportunity costs, the argument falls short for other reasons. Precisely why is this experience reserved for students who have made high grades or survived a writing competition? In effect, it appears the experience is reserved for the student least likely to benefit from it. In fact, a sincere effort to improve research and writing skills would hardly involve producing a legal publication that entails many activities that are menial and hardly enriching.

Part 3: Is There Too Much Legal Scholarship: The Costs and Qualities of Legal Research

 Part 3: Is There Too Much Legal Scholarship: The Costs and Qualities of Legal Research

              Before examining arguments in support of legal research at current levels, it is important to be specific about what is being defended both in terms of costs and quality.

1. The Cost of Legal Scholarship

Legal research is expensive. One, probably conservative, estimate is that each article produced costs $30,000.  This figure is based on a series of assumptions. Readers are invited to alter these assumptions and determine their own average cost figure. Thirty thousand dollars is likely to be low because this estimate does not include law review publication costs, submission fees, secretarial expenses, or any allocation of fixed costs. In addition, it is based on the assumption that the 8000 law review articles published each year are the result of the efforts of law professors. If some of the 8000 are published by researchers who are not supported by law schools then the average cost for law professor produced articles increases. On the other hand, it could be lower. The 8000 articles does not account for books, casebooks, and refereed journals.

Costs may be better understood in terms of opportunity costs, real or hypothetical. For example, the average Habitat for Humanity house costs $85,000. In effect, three law review articles are more expensive to produce than a home for a low income family. Obviously there are commensurability issues with this idea – both in theory and in practice.   First, there is no way, as far as I know, to actually compare the value of housing with the value of law review articles. Second, even if we stopped funding legal scholarship today, housing may not be a realistic opportunity cost since funds, at least in the short run, would not be automatically shifted to increased housing for the poor. 

There is, perhaps, a comparison that we can relate to a little better.  The average law student graduates with about $140,000 in debt or, in scholarship currency, about 4.5 articles.   Based on that number, the investment in legal scholarship is the equivalent to 1700 students per year graduating without debt. So, is it worth it for a student be saddled a modest mortgage-sized debt so 4.5 more articles can be written?  That depends on the articles lost but as suggested below, the loss would not be noticeable.

Student debt is not disconnected from other issues. The need to pay student loans means that students are less likely to be able to afford the salaries of public interest work and low paying jobs that may serve the less affluent. Perhaps it is a leap, but the high cost of legal education, part of which is attributable to the scholarship requirement, may actually have an impact on those who need but cannot afford legal services. To some this may be strained reasoning but clearly it is no more strained than a topic discussed below that runs along the lines  of “we just know the investment in legal scholarship is worth it” or “we know it is worth it because Kingsley’s article on retribution had a major impact on punishment theories.”

2. The Character of Legal Research

The high cost of legal research would make sense if there were clear evidence that the level of investment was offset by the importance of what is written. The key notion is “importance,” which, for this analysis, is independent of the care or quality of effort that went into the production of the scholarship. Although this is impossible to determine with any level of certainty, there are general and empirically verified characteristics of legal scholarship that cut against its relevance, at least at current levels. Without conceding that citations are a measure of usefulness, it is not unreasonable to surmise that citations are at least a rough measure of exposure. What we know is that citations are correlated with the ranking of the law review in which the scholarship appears, the ranking of the school at which the author teaches, and the ranking of the school from which the author graduated. Unless one equates these indicators of institutional authority with the usefulness of a work and then equates usefulness with frequency of citation, there is little evidence that the current investment in legal research is warranted or that citation counts should have any bearing on law school rankings. In fact, it suggests that even the most important and useful articles may not be found and relied upon because they are buried in very low ranked reviews or secondary journals.

Even if one were willing to take these leaps of faith, there is another possibly more disconcerting problem: It does not appear that authors citing other authors are really responding to or elaborating on the ideas of the original author. In fact, the most frequent type of citation is one that cites a prior author for a factual assertion. This may be driven by the requirements of law review editors but still there are remarkably few citations in which one author considers, debates, or builds on the ideas of another.

Part 2: Is There Too Much Legal “Scholarship”?: A Quasi Economic Perspective The Author and the Audience: Supply, Demand, and Public Goods

 

 Part 2: Is There Too Much Legal “Scholarship”?: A Quasi Economic Perspective 

The Author and the Audience: Supply, Demand, and Public Goods

              One methodology for determining whether too much or too little is produced is to rely on supply and demand. For a number of reasons, a traditional economic approach is not appropriate for legal scholarship.  Still, there is a central aspect of supply and demand that is directly applicable. In a market in which supply and demand are operative, there is an interdependence. Suppliers are responsive to demand. In the context of legal scholarship, this leads to the question of to what legal scholars are responding.

              For the most part, legal scholars write (supply) because they want to be noticed, they have tenure requirements,  they have a sense of obligation to continue being productive, or they love the process and have a desire to express themselves. Legal scholarship may be a reaction to all of these factors at various times. What is remarkable is that none of them are driven by demand in the sense of the benefit to others. The notion of interdependence is lacking.   If fact, the “demand” for legal scholarship is based on the needs of suppliers – like General Motors demanding more Chevrolets.

 This  does not necessarily mean that there is a surplus of legal scholarship. It does indicate that supply is disconnected from any meaningful measure of the value or usefulness of what is produced. In fact, legal scholarship largely exists to satisfy the needs of the writers, regardless of the external  impact. In this sense law professors are both the audience and the actors similar to a company of actors that only perform plays for others in the company.

              There is an economically-based argument that may seem to respond to this analysis. It is based on the idea that legal scholarship is a public good.  Consequently, it will be undersupplied unless subsidized. Public goods are ones that can be used by others without payment to the producer.  In addition, they can be used without interfering with the use of others.   The argument for subsidization is based on the idea that producers of public goods are unable to internalize the benefits of their efforts and, even though there are benefits, suppliers will not  have incentives to produce. There is, however, an implicit interdependency. Public goods are produced because of a demand that does not manifest itself in the market.

When it comes to legal scholarship, arguments based on a public good rationale are weak and fall victim to faulty logic.  There is nothing about the classification of something as a public good that also means production must or should be subsidized. To understand this, consider that public goods include everything from a kindergartener’s  sketch to a poorly written screenplay and a truly evocative oil painting. The critical question before subsidization occurs is whether there is a latent demand for the good.  One has to reach outside the technical public good classification to justify the subsidization of legal scholarship at current levels. If the public good rationale were enough, we would, in fact, build highways for a handful of drivers.

Reliance on the public good rationale may be inapt for another reason.  The fact that  producers cannot internalize all the gains from their efforts does not mean a good will not be produced if enough of the benefits can be internalized to make the effort worthwhile.  The question, then, is whether some or even a great deal of research would take place even if not subsidized.  For example, in a billion dollar antitrust case or securities case, a law firm is not going to stand idly by in hopes a law professor will do the necessary research. In fact, the most useful legal research likely involves issues in which benefits cannot be monetized and the market is unlikely to produce the necessary research.

Is There Too Much Legal “Scholarship”?: A Quasi Economic Perspective

Is There Too Much Legal “Scholarship”?: A Quasi Economic Perspective 

PART 1 Introduction

Discussions about the value of legal scholarship lead to a cascading series of questions. These questions become more critical as the all important U.S. News and World Report law school ranking officials begin exploring ways to include scholarly impact in their calculations. First, what exactly is legal “scholarship” as opposed to other forms of writing by law professors?  Second, does the supply of legal scholarship exceed the demand?  Framing the question  this way will be unsettling to some but not if one thinks in terms of whether there is more “scholarship” than could possibly be useful. Third, what are the costs and qualities of legal scholarship? Fourth, what are the justifications for current levels of legal scholarship or writing generally?  Put differently, how do those who favor the current level of legal scholarship defend it against critics? Fifth, what accounts for the production of scholarship that, for the most part, does not increase overall welfare? Finally, is there a way out of that predicament? These questions are addressed below.  

A. Legal Scholarship and Legal Writing

              The unfortunate truth is that there is a shortage of legal scholarship but an overabundance of legal writing. Why is that? Law professors, as a general matter, are not trained to do scholarship and with a few exceptions, do not do it. Scholarship is characterized by research designed to discover answers no matter how inconvenient. Law professors tend to have a position and then write to promote it. This is sometimes characterized by the oxymoron “normative scholarship.” In short, they write to advocate positions. The critical distinction is this: The advocate knows the answer before starting to conduct research and write. The scholar does not know the answer and goes where the research takes him or her.

              This may seem like a slim distinction, but it is not. Writers of advocacy can be compared to expert witnesses who testify on one side or another of a case. Like expert witnesses, writers of “normative scholarship” offer opinions. Without getting into question of the ethics of expert witnesses, to some extent any expert witness bias is offset by cross-examination. Moreover, an opposing opinion is likely to be offered. In the case of law professors this is less likely to be true. The problem is compounded by the fact that advocacy, opinions, or normative scholarship by law professors is a form of subsidized speech and those who may hold opposing views are typically not beneficiaries of similar aid. In sum, in the case of legal scholarship there is less likely  to be the type of accountability that leads to rigor.

              This is not to say that all writing should be scholarship at the expense of all advocacy. In fact, it may be that true scholarship will lead to policy proposals that seem obvious. There is also a way for law schools and law reviews to overcome the anti-intellectualism of normative scholarship. Similar to the long gone Fairness Doctrine, law reviews that publish advocacy and law schools that sponsor advocacy speakers should go to whatever lengths are necessary to allow opposing viewpoints to be aired. The combination of the two viewpoints results in something comparable to what accomplished scholars should do. Instead, as currently conceived, it is difficult for two opposing normative views to be heard.  

              Another facet of the scholar/advocate distinction is that when people think of scholarship or advocacy they naturally think in terms of “producing” it. In other words, is there a written product? Scholarship does not require this outcome. Some of the most scholarly people do not write or, if they do, it is infrequent. Their scholarship may only be reflected in the quality of their teaching. Advocacy, unfortunately, may also be reflected in teaching but it can be likened to a form of exploiting the students. Those who confuse their own advocacy with scholarship seem likely to bring their own personal beliefs into the classroom because they confuse opinion with “truth.” Whether by their case selection, responses to questions, or observations by students, the advocacy mindset can mean that critical thinking and learning are discouraged.

              Before leaving the scholarship issue, one gap in the efforts of law professors is worth noting.  As far as I know the Journal of Legal Education is the only one devoted, at least in part, to teaching technique. Perhaps law professors read other journals or reviews that may carry an article from time to time about teaching. This has always struck me as a bit strange. Why is it that if you do well in law school you become qualified to teach? There are no courses on how to present material or on the reliability and validity of various testing techniques, yet law professors are quick to defend whatever they do with little or no support.  The issues range from the length of classes to different types of testing.  A number of factors affect teaching evaluations. What we do not know is whether high teaching evaluations are correlated with actual learning.   Perhaps these are not matters that law professors know how to address but that does not keep them from addressing virtually anything else and, to me, it is a sad commentary on the idea that scholarship should complement teaching when so little scholarship is devoted to teaching.

PART 2 The Author and the Audience: Supply, Demand, and Public Goods (coming soon).