Meta-scholarship: Law Profs Studying Legal Scholarship

I had not realized what a buzzing area of scholarship there is right now on empirical studies of legal scholarship. Besides loving the inherent navel-gazing quality of the field, I am intrigued by the thinking and results these guys are coming up with.

I have been thinking about what effects digital publishing is having, but have really looked at the full-text access to primary law, law reviews and treatises. I have thought about whether judges and professor will game the search systems by trying to include more searchable terms and titles. After all, how can you have much influence if your decision or article can't be located by searchers? Already, articles in journals not available electronically are nearly lost.

I also have thought about the effect of these full-text databases with superior, sophisticated search options on the researchers. That is, do students, paralegals and young associates search or even think about legal questions differently than before CALR was so ubiquitous? Of course they do! We all do. Barbara Bintliff pointed out in From Creativeity to Computerese: Thinking Like a Lawyer in the Computer Age, 88 Law Lib J 338 (Summer, 1996), that a researcher using online sources to find a case usually creates a query out of the material facts of the case at hand. In searching the index to the Digest to find a case, on the other hand, a researcher has to do some preliminary legal analysis to at least decide the general topics to search. I think the changes go much deeper, resulting in seriously changed and weakened legal analysis. More and more, researchers seem to be relying on the cases they find online for the analysis rather than doing independent thinking.

The image above is a recursive photo, courtesy of Nick Gall's radioblog at http://radio.weblogs.com/0126951/2004/09/20.html A recursive photo, of a photo of a photo, like repeating mirrors, is a pretty good illustration of meta-scholarship.

To see what the profs are thinking about, read on...

Here are some interesting sites:

Yale Pocket Part
The Future of Legal Scholarship link

Online Legal Scholarship: The Medium and the Message by Jack M. Balkin link
Balking begins his essay with an account of how his blog, Balkinization, educated the media on a spurious "compromise" bill purporting to settle the disagreement between the Bush Whitehouse, National Security Agency and Senator Arlen Spector over intrusive domestic surveillance. The blog entries analyzed the bill in more depth than the reporters could do themselves, and revealed serious weakness in the bill. The media picked up on the bloggers' analysis and changed the tone of reports from praising the bill to damning it.

This story encapsulates many of the most important effects online media will have on legal scholarship. It shows how digital technologies affect the style, subject matter, tempo, intermediaries, and audience for legal scholarship, and how online media will shape the diffusion of legal expertise and the incentives for producing legal knowledge.

In my “official” (i.e., non-blogging) legal scholarship, I’ve argued that Internet speech has two key characteristics. It routes around traditional media gatekeepers and it gloms onto existing cultural sources, appropriating them for its own purposes rather than displacing them. A similar analysis applies to the production and diffusion of legal expertise.

Both online media like the Social Science Research Network (SSRN) and blogging route around the traditional gatekeepers of legal scholarship: law journals.

Balkin notes these features of digital publishing and how they will affect legal scholarship:
1. Speed of publication increases. (Notes 8 and 9, below flow from this)
2. Author retains more or total control over style, editing.
3. Reach a broader audience.
4. Use a broader range of resources. The same electronic publishing that creates the other features also make scholarship from other fields instantly, freely available to legal scholars.
5. Interactive publication, like blogs which allow comments. This creates both a sense of loyalty in the consumer and a feeling of responsibility in the producer.
6. Shorter, more informal pieces.
7. Fewer citations, but those used are hyper-links so the reader can "see for yourself."
8. Quicker response to new issues and court decisions.
9. Bloggers are rewarded for fast reaction, and pressured for speed, which can reduce the quality of analysis and writing.
10. Quick response means the easy conclusions are reached first and fast, and comments following may move the thinking along faster than it might go in a law journal call and response. You have a community working on the analysis. This might be good for some issues and deleterious to analysis of others.
11. The rewards and pressure for speed make legal scholarship on blogs more like journalism, also the changed relationship between reader and public is more like journalism than traditional legal scholarship.
12. The relationships between author and reading public changes. It is a more intimate connection, and can feed back and forth. For instance, journalists read certain blogs for expert legal commentary, but they are also influenced by the bloggers. Some blogs become excellent platforms for commentary because of their relationship with the media.
13. Blogging and other digital publishing also affects the subject matter:

Routing around [traditional legal publishing outlets] shapes the subject matter and the message of legal scholarship in multiple ways. Online media—and particularly blogging—drive legal writing toward issues that are timely and particularly important to practitioners, policy analysts, journalists, and politicians. Blogging allows law professors to comment on successive drafts of pending legislation both in Congress and in state governments—something that traditional legal scholarship can almost never do. It allows focused commentaries on recent state and lower federal court decisions that most law professors would not want to spend an entire law review article addressing, and that most student-edited law reviews—which tend to focus on constitutional and other “hot” topics—would not be interested in publishing.

Blogging’s quick response time could discourage commentary that is not timely, that takes the longer view of a problem, or concerns an issue or a field of law unrelated to current controversies. Legal bloggers who write about topics with no connection to current events won’t get large audiences; they will have to content themselves with niche audiences of specialists. Fortunately, online media are perfect for niche publications. They reduce the cost of publication and data storage, and they lower the costs of speakers and audiences finding each other. Hence online media have two interesting and opposing effects: they drive legal commentary toward timely issues for bloggers who seek a mass audience, and they create a friendly venue for specialized legal commentary aimed at smaller audiences who can now easily form communities of interest. Both SSRN and niche blogs are partial antidotes to the skewed focus of student-edited journals, which provide too little coverage of business law and statutory questions that would be most useful to lawyers and judges. In fact, there are now niche legal blogs on almost every conceivable subject, from corporate law to disability law.
(snip)
Blogging pushes law professors to analyze contemporary legal issues instead of high theory; equally important, it drives them toward doctrinal analysis that lawyers and judges can use. I have felt these effects myself. Before 2003, when I started my blog, most of my “official” scholarship was nonprescriptive and interdisciplinary. But most of my blogging has been normative, offering lawyerly arguments about the merits of legislation, executive actions, and judicial decisions. That result may be overdetermined, as my scholarly interests have changed in the past decade. But if online media have helped turn a deconstructionist into a doctrinalist, imagine what they will do to everyone else.

But what about quality? Won’t online media generate a veritable Tower of Babel of half-baked legal arguments that nobody wants to listen to? Online media don’t just route around traditional gatekeepers. They also glom onto them—they depend on them rather than displace them. This is the second of the two effects of online media I mentioned earlier. The old gatekeepers don’t go away entirely, and new ones arise that partially supplement and partially compete with them.

The Long Tail of Legal Scholarship by Paul L. Caron link
The title refers to the article and blog expansion on the idea of The Long Tail in markets, by Wired editor Chris Anderson. The basic idea is that pre-digital age, publishers and other producers of information, books, movies, etc., looked for the big hit. They figured 80% of their sales came from 20% of their produced books, etc., and were always focused on trying to identify a potential hit. The development of cheaper production available so easily to a broader consuming public changed all that by recognizing that larger profits could actually be realized by producing many niche products than by producing one or two big hits. Caron applies the long tail to legal scholarship, examining Tom Smith's project The Web of the Law, noted at Right Coast blog, below. Caron would look instead at hits for an article rather than the citation use of articles in Smith's study. He includes an interesting graph based on data from SSRN, ranking law schools and professors by the number of hits and downloads. Caron concludes

Consistent with the long tail thesis, 97% of authors have had at least one download in the past year and 100% have had at least one download at some time. (This long tail is not present in citations data; 40% of articles have never been cited.) The shift in legal scholarship toward the tail is clear from a comparison of new downloads to total downloads

Caron hopes the group empirical study of legal scholarship by Arewa, Morriss, Henderson and Dau-Schmidt will improve understandings of the consumption of legal scholarship and in turn, influence the development of legal scholarship.


Empirical Legal Studies
Applying Empirical Methods to Legal Scholarship, a report on a group project by Andrew Morriss, Funmi Arewa, Bill Henderson, and Ken Dau-Schmidt link. Arewa also reports briefly on the project at Conglomeratehere

Conglomerate
(note Arewa's post at Conglomerate noted just above)

Navel Gazing (Part XXVII) by Dave Hoffman link
Hoffman reacts to the citation study by Tom Smith noted at Right Coast. He wonders if the citation study wouldn't have different results in a more comprehensive database of law reviews, and wishes Smith would use his citation study to produce a more sophisticated measure of law review rankings.

The Right Coast
A voice, crying in the wilderness, and then just crying by Tom Smith here
This is a fascinating study where Smith analyzes how many of the law review articles in Lexis' Shepards' database (?) are cited and how often:

... the top .5% of law review articles gets 18% of all citations (snip); the top 5.2% gets about 50% of all citations; and the top 17% of articles gets 79% of all citations. And about 40% of articles never get cited at all.

TCS Daily (Tech Central Station aka Technology Commerce Society)
Little Things by Glenn Harlan Reynolds
Reynolds is thinking that electronic access and searching of law reviews is going to flatten the hierarchy of law reviews and of individual professors' reputation based on which school they work at:

Nowadays, very few people do research this way [searching the Index to Legal Periodicals]. Instead, the first stop is the computerized Lexis and Westlaw databases, where articles are searched by keyword, and where results appear ranked (depending on which you use) either in reverse-chronological order or in alphabetical order by journal. And since you can view the section of the article where the search terms appear just by clicking your mouse, there's less incentive to look only at the most prestigious venues.

I believe that this has led me to cite more articles in comparatively obscure and un-prestigious journals than I used to, and colleagues I've spoken with say the same thing. So although articles in the Harvard Law Review, the Yale Law Journal, and the Columbia Law Review still command more prestige than articles in, say, the Southern California Interdisciplinary Law Review, they are no longer more likely to be noticed.

That's true even for casual browsing. Until about ten years ago, most law libraries kept faculty members aware of new scholarship by photocopying the contents pages of new law reviews as they arrived in the mail, then distributing them weekly in a thick wad. In skimming through them, many faculty members would tend to look only at the contents pages for the most prestigious law reviews, because there were often fifty or a hundred pages in a bundle. Now, to save on copying costs, they're distributed by email and indexed by subject - meaning that I'm just as likely to be aware of an article in my field that appears in the most obscure as of one in the most eminent journal.